Helping Girls Get Back on Track

AN IMPLEMENTATION STUDY OF THE PACE CENTER FOR GIRLS

 Louisa Treskon

Megan Millenky

 Lily Freedman

April 2017

Helping Girls Get Back on Track

An Implementation Study of the PACE Center for Girls

Louisa Treskon

Megan Millenky

Lily Freedman 

 

April 2017
This project has been funded by the Edna McConnell Clark Foundation, the Corporation for 

National and Community Service, the Healy Foundation, and the Jessie Ball duPont Fund. The 

contents of this publication do not necessarily reflect the views of these organizations, nor does 

mention of trade names, commercial projects, or organizations imply endorsement of same by 

these organizations.

This material is based upon work supported by the Social Innovation Fund, a program of the 

Corporation for National and Community Service (CNCS). The Edna McConnell Clark Foundation’s Social Innovation Fund includes support from CNCS and 15 private co-investors: The Edna 

McConnell Clark Foundation, The Annie E. Casey Foundation, The Duke Endowment, The 

William and Flora Hewlett Foundation, The JPB Foundation, George Kaiser Family Foundation, 

The Kresge Foundation, Open Society Foundations, The Penzance Foundation, The Samberg 

Family Foundation, The Charles and Lynn Schusterman Family Foundation, The Starr Foundation, 

Tipping Point Community, The Wallace Foundation, and the Weingart Foundation.

Dissemination of MDRC publications is supported by the following funders that help finance 

MDRC’s public policy outreach and expanding efforts to communicate the results and implications of our work to policymakers, practitioners, and others: The Annie E. Casey Foundation, 

Charles and Lynn Schusterman Family Foundation, The Edna McConnell Clark Foundation, Ford 

Foundation, The George Gund Foundation, Daniel and Corinne Goldman, The Harry and Jeanette 

Weinberg Foundation, Inc., The JPB Foundation, The Joyce Foundation, The Kresge Foundation, 

Laura and John Arnold Foundation, Sandler Foundation, and The Starr Foundation.

 

In addition, earnings from the MDRC Endowment help sustain our dissemination efforts. Contributors to the MDRC Endowment include Alcoa Foundation, The Ambrose Monell Foundation, 

Anheuser-Busch Foundation, Bristol-Myers Squibb Foundation, Charles Stewart Mott Foundation, Ford Foundation, The George Gund Foundation, The Grable Foundation, The Lizabeth and 

Frank Newman Charitable Foundation, The New York Times Company Foundation, Jan Nicholson, Paul H. O’Neill Charitable Foundation, John S. Reed, Sandler Foundation, and The Stupski 

Family Fund, as well as other individual contributors.

The findings and conclusions in this report do not necessarily represent the official positions or 

policies of the funders. 

For information about MDRC and copies of our publications, see our website: www.mdrc.org. 

Copyright © 2017 by MDRC®. All rights reserved.
iii

Overview 

PACE Center for Girls is a unique program that provides academic and social services to girls ages 

11 to 18. Girls eligible for PACE exhibit multiple health, safety, and delinquency risk factors, such 

as poor academic performance, exposure to abuse or violence, truancy, risky sexual behavior, and 

substance abuse. PACE seeks to help them onto a better path and reduce the likelihood of negative 

outcomes, such as involvement with the juvenile justice system. 

PACE operates 19 nonresidential, year-round program centers across the state of Florida. Girls 

attend PACE daily during normal school hours and receive academic and extensive social services 

in a gender-responsive environment — that is, one tailored to the needs of girls. Most girls plan to 

attend PACE for approximately one year; during this time, they receive academic instruction and 

advising, a life skills curriculum, assessment and care planning, individual and group counseling,

and service learning and work readiness opportunities. Parental engagement and transition and 

follow-up services are also key components of the PACE program. When girls leave PACE, they 

often return to other schools in their communities to complete their secondary education.

This report presents implementation research findings from MDRC’s ongoing evaluation of 14 

PACE centers. A final report presenting the impacts of the program is planned for release in 2018.

Key Findings

•

•

•

•

The PACE program model, defined through both general program principles and a detailed 

manual, was implemented consistently across multiple locations. Girls at PACE received most 

services at the intended levels. Services varied somewhat across locations because of differences 

in staff and local resources and in program areas where the model gave less specific guidance.

PACE incorporated gender-responsive programming into all services through a focus on safety 

and relationships, an emphasis on recognizing and building on girls’ individual strengths, and an 

awareness of the effects of trauma.

Girls who attend PACE tend to be low-income, and they often struggle with school and have a 

range of other health, safety, and delinquency risk factors.

PACE differed from the traditional school environment by offering smaller classes, access to 

frequent individual academic advising and counseling, life skills programming, and connections 

to other services in the community, such as transportation or health care.

v 

Contents

Overview iii

List of Exhibits vii

Preface ix

Acknowledgments xi

Executive Summary ES-1

Chapter

1 Introduction 1

PACE Center for Girls 3 

The PACE Evaluation 7 

Overview of Chapters 12 

2 PACE Implementation 13

Implementation of the PACE Model 13 

Implementation Fidelity and Variation Across PACE Centers 17 

3 PACE Girls 21

Recruitment 21 

The Program Application Process 22 

Why Girls Come to PACE 25 

Background Characteristics of the Study Sample 26 

Program Group Participation 31 

4 Gender-Responsive Programming and PACE’s Program 

Environment 33

Gender-Responsive Programming 33 

Program Environment 34 

Fostering a Gender-Responsive Program Culture 41 

Summary 47 

5 Academic Services and Life Skills 49

Introduction to the PACE Academic Services Model 50 

Classes at PACE 52 

Assessment and Advising 58 

Life Skills Training 59 

Girls’ and Parents’ Reflections on Academic Services 62 

Academic Progress 64 

Services Received 65 
vi

6 Social Services 67

Social Services Structure and Staffing 67 

Assessment and Care Planning 68 

Individual Counseling 70 

Group Counseling 75 

Parental Engagement 75 

Transitions 77 

Follow-Up 79 

Impacts on Services Received 81 

7 Conclusion 85

Appendix

A Supplementary Tables on Sample Member Characteristics and Risk Factors 89 

B CLASS-Secondary Observations 95 

C Data Collection and Analysis 99 

References 105 
vii

List of Exhibits

Table

ES.1 Gender-Responsive Programming Principles and PACE Program Components ES-6 

ES.2 Selected Characteristics of Sample Members at Baseline ES-8 

ES.3 Impacts on Service Receipt ES-10

1.1 Characteristics of PACE Centers in the Study 9 

3.1 Characteristics of Sample Members at Baseline 27

3.2 Risk Factors of Sample Members at Baseline 29

3.3 Program Participation (Program Group Only) 32

4.1 Gender-Responsive Programming Principles and PACE Program Components 35

5.1 Academic Characteristics of PACE Centers 51

5.2 Academic Staffing Characteristics at PACE Centers 52

5.3 Participation and Satisfaction in Academic Services 60

5.4 One-Year Impacts on Academic Services Received 65 

6.1 Characteristics of PACE Counselors 69

6.2 Social Service Staffing Characteristics at PACE Centers 69

6.3 Counseling and Parental Involvement at PACE Centers 72

6.4 Transition Services at PACE Centers 78

6.5 One-Year Impacts on Social Services Received 82

A.1 Characteristics of Sample Members at Baseline, by Random Assignment 

Group 91

A.2 Risk Factors of Sample Members at Baseline, by Random Assignment Group 93

Figure

1.1 PACE Center for Girls Logic Model 6 

1.2 Population Map of Florida with PACE Centers 8 

2.1 Distribution of Program Fidelity Ratings by Domain 18

3.1 Eligibility and Enrollment Process 23
viii

Figure

4.1 Distribution of Organizational Social Context Scores by Center 46

5.1 PACE CLASS-S Scores Across Centers, by Dimension 56

Box

1.1 The Edna McConnell Clark Foundation (EMCF) Social Innovation Fund 4 

2.1 Excerpt from PACE Policy and Procedures Manual 15

4.1 PACE Values and Guiding Principles 42

4.2 Organizational Social Context Subscales 44

5.1 Classroom Assessment Scoring System-Secondary (CLASS-S) Domains 55

5.2 Spirited Girls! Curriculum: Content by Developmental Domain 61

5.3 One Girl’s Perspective on Classes at PACE 63

6.1 Example of Care Plan Goals 71

6.2 Two Perspectives on Counseling 74
ix

Preface

When young people drop out of school or become involved in the juvenile justice system, the 

consequences can extend far into adulthood. Thus, there is a compelling policy need to understand how to support young people who exhibit warning signs of academic failure and delinquent behavior. Such behavior is often a symptom of other challenges in the lives of girls and 

boys — and girls face their own distinct challenges. Girls in the juvenile justice system are more 

likely than boys to have experienced sexual abuse and maltreatment as children, and their 

responses to trauma differ from those of boys. Yet a juvenile justice system designed for boys is 

too often ill equipped to address those issues and may only worsen girls’ problems. Genderresponsive programs, such as the one described in this report, were developed in recognition of 

this need. 

MDRC’s evaluation of the PACE Center for Girls offers a valuable opportunity to understand how the gender-responsive approach translates into actual program operations. PACE 

takes a preventive approach, aiming to help troubled girls ages 11 to 18 stay in school and avoid 

involvement, or deeper involvement, with the juvenile justice system. At locations across the 

state of Florida, PACE provides academic and social services during regular school hours in a 

safe, supportive environment tailored to girls’ needs, with an emphasis on relationships, relevant 

life skills, and the cultivation of girls’ strengths. A low staff-to-participant ratio allows for 

individual attention and an awareness of each girl’s history of trauma. And, recognizing that the 

girls’ problems are often intertwined with family and peer relationships, either as cause or 

result, PACE staff members also strive to engage this larger community in the girls’ care. 

This report describes the implementation of PACE at the 14 centers that are participating in the evaluation. The research found that PACE successfully implemented its unique model 

as planned in multiple locations. Besides detailing the program’s dissemination of its genderresponsive culture and services, these findings provide useful information to social service 

providers who seek to replicate their own programs. In addition, the study has found that, after 

12 months, girls in PACE were more likely than girls in a control group to have received 

academic advising and mental health counseling and to have been enrolled in school. The final 

report, due in 2018, will provide experimental evidence of the impact of this gender-responsive 

program on girls’ well-being, along with a cost-effectiveness analysis. 

Gender-responsive programming has sparked federal interest and funding for research 

and development. This study will provide foundational knowledge about the effectiveness of 

this approach in helping girls stay in school and out of the court system. 

Gordon L. Berlin 

President, MDRC

xi

Acknowledgments

The PACE evaluation is made possible through the support of many individuals and organizations. It is funded by the Edna McConnell Clark Foundation’s Social Innovation Fund, the 

Corporation for National and Community Service, the Jessie Ball duPont Fund, and the Healy 

Foundation. At the Edna McConnell Clark Foundation, we are grateful to the leadership of 

Teresa Power and Gabriel Rhoads.

We are grateful to the staff members of the PACE centers that are participating in the 

evaluation. They work tirelessly to provide services to girls in their communities and played a 

critical role in helping to recruit and enroll participants into the evaluation. Staff members 

generously assisted with arranging implementation research visits and making time in their busy 

schedules to participate in interviews.

We also relied on the assistance of staff members at the PACE headquarters, who spoke 

with us about the history and structure of PACE and provided us with essential data about 

programs. In particular, we are grateful to Mary Marx, Shana Brodnax, Lymari Benitez, Thresa 

Giles, Yessica Cancel, Janie Smalley, James Kindelsperger, and Debbie Moroney. We also 

thank Vicki Burke, the founder of PACE, for meeting with us to share information about 

PACE’s early years. This report also benefited from the contributions of staff members at other 

organizations that either partnered with PACE centers or provided alternative services to young 

people in the communities where PACE operates. 

This research would not have been possible without the work of many individuals at 

MDRC. Rob Ivry, Dan Bloom, Carolyn Hill, Farhana Hossain, Jean Grossman, and Christopher

Boland provided valuable feedback on report drafts. Galina Farberova and her team managed 

the random assignment system. Hannah Wagner coordinated the production of the report. 

Danielle Craig assisted with fact-checking. Jennie Kaufman edited the report, and Carolyn 

Thomas and Ann Kottner prepared it for publication. We also thank the other staff members at 

MDRC not already mentioned who contributed greatly to the research effort, including Caroline 

Mage, Ada Tso, Emily Terwelp, Melanie Skemer, Sara Muller-Ravett, Julianna Alson, Hannah 

Siegelberg, Brit Henderson, Nicole Alexander, and Janae Bonsu.

We are also grateful to Charlotte Bright from the University of Maryland School of Social Work, who provided her expertise on gender-responsive services throughout the project.

Joan Flocks, Marina Prado-Steiman, and Emily Calvin from the University of Florida assisted 

with site visits. In addition, Jessica Walker-Beaumont worked closely with MDRC and PACE 

to provide technical assistance and strategic planning on recruitment and other program efforts.

Finally, we are deeply appreciative of the girls and their parents and guardians who 

agreed to participate in the research. Without them, this research would not have been possible.

The Authors

ES-1 

Executive Summary

In Florida, girls who are falling behind in school or exhibiting troubling behavior have access to 

a unique program that offers them a chance to get back on track. PACE Center for Girls 

employs what is known as a gender-responsive approach to provide both academic and extensive social services, including classes with a low student-to-teacher ratio, regular counseling 

sessions, and a life skills curriculum designed for girls. PACE, which has centers in 19 counties 

across the state, began more than 30 years ago as a program to meet the needs of girls involved 

with the juvenile justice system. The program serves middle school and high school-age girls 

who have multiple risk factors.

These risk factors, which include individual, peer, family, school, and community characteristics, increase the likelihood that a girl will struggle in school and engage in delinquent 

behavior. Delinquency and involvement in the juvenile justice system, in turn, result in considerable personal and societal costs. Juvenile charges or detention may damage a young person’s 

relationships with friends and family, negatively affect mental health, and interrupt the academic progress and work experience that should accumulate during adolescence.1 And from a 

societal perspective, court and detainment costs are high. Therefore, effective prevention or 

early intervention programs that can help young people avoid involvement in the juvenile 

system and succeed in school offer a significant return on investment.2

 

Research has shown that adverse childhood experiences affect boys and girls differently. Girls have a greater incidence of depression than boys and respond differently to trauma; for 

example, girls are more likely to engage in self-medicating behaviors.

3 Their pathways into the 

justice system are also different. Girls are more often detained for nonserious offenses, such as 

truancy or violating probation, and more often enter the juvenile justice system with a history of 

physical, sexual, or emotional abuse, extreme family conflict, and neglect.

4 Gender-responsive 

 1

Anna Aizer and Joseph J. Doyle Jr., “Juvenile Incarceration, Human Capital, and Future Crime: Evidence 

from Randomly-Assigned Judges,” Quarterly Journal of Economics 130, no. 2 (2015): 759-803. 2

Steve Aos, Roxanne Lieb, Jim Mayfield, Marna Miller, and Annie Pennucci, Benefits and Costs of Prevention and Early Intervention Programs for Youth (Olympia: Washington State Institute for Public Policy, 

2004). 

3

Margaret A. Zahn, Robert Agnew, Diana Fishbein, Shari Miller, Donna-Marie Winn, Gayle Dakoff, 

Candace Kruttschnitt, Peggy Giordano, Denise C. Gottfredson, Allison A. Payne, Barry C. Feld, and Meda 

Chesney-Lind, Causes and Correlates of Girls’ Delinquency (Washington, DC: Office of Juvenile Justice and 

Delinquency Prevention, U.S. Department of Justice, 2010); Emily J. Salisbury and Patricia Van Voorhis, 

“Gendered Pathways: A Quantitative Investigation of Women Probationers’ Paths to Incarceration,” Criminal 

Justice and Behavior 36, no. 6 (2009): 541-566. 4

Charlotte Lyn Bright and Melissa Jonson-Reid, “Young Adult Outcomes of Juvenile Court–Involved 

Girls,” Journal of Social Service Research 36, no. 2 (2010): 94-106; Charlotte Lyn Bright and Melissa Jonson-

(continued)
ES-2 

approaches were developed as a response to the recognition that the current juvenile justice 

system is not well positioned to meet the particular needs of girls, as most services are based on 

the needs of boys.5 The term “gender-responsive” thus describes treatment approaches for 

serving women and girls. Principles of gender-responsive programs include an understanding of 

the effects of trauma, a focus on relationships, and life skills and health education that is tailored 

to the lives of girls and women. 

Rigorous research on gender-responsive programming is limited, however. There has 

been national interest in understanding gender-responsive programs in the context of improving

the juvenile system more broadly,

6 but the current literature is more robust in its description of 

concepts and principles than in its evaluation of program performance.7 Until recently, it was

largely unknown how gender-responsive services are implemented, how similar they are to one 

another, or how effective they are.

8 The evaluation of PACE Center for Girls — perhaps the 

largest and most well-established program of its kind — provides an opportunity to answer 

foundational questions about the implementation and effectiveness of a gender-responsive 

program. The research aims to help practitioners and policymakers better understand, and 

possibly replicate, services for at-risk girls. The evaluation is being conducted by MDRC and is 

funded mainly through the Edna McConnell Clark Foundation’s Social Innovation Fund (SIF), 

a program of the Corporation for National Community Service (CNCS), with additional funding 

provided by the Jessie Ball duPont Fund and the Healy Foundation.

About PACE Center for Girls

PACE Center for Girls currently operates 19 nonresidential, year-round program sites across the 

state of Florida. Girls eligible for PACE are between the ages of 11 and 17 and exhibit such risk 

 

Reid, “Onset of Juvenile Court Involvement: Exploring Gender-Specific Associations with Maltreatment and 

Poverty,” Children and Youth Services Review 30, no. 8 (2008): 914-927; Zahn et al., Causes and Correlates 

of Girls’ Delinquency. 5

Margaret A. Zahn, Stephanie R. Hawkins, Janet Chiancone, and Ariel Whitworth, The Girls Study Group 

— Charting the Way to Delinquency Prevention for Girls (Washington, DC: Office of Juvenile Justice and 

Delinquency Prevention, U.S. Department of Justice, 2008). 6

Juvenile Justice and Delinquency Prevention Act, 42 U.S.C. 5633 § 242 (1992). 7

Dana Jones Hubbard and Betsy Matthews, “Reconciling the Differences Between the ‘GenderResponsive’ and ‘What Works’ Literatures to Improve Services for Girls,” Crime & Delinquency 54, no. 2 

(2008): 225-258. 8

Meda Chesney-Lind, Merry Morash, and Tia Stevens, “Girls’ Troubles, Girls’ Delinquency, and Gender 

Responsive Programming: A Review,” Australian & New Zealand Journal of Criminology 4, no. 1 (2008): 

162-189; Patricia K. Kerig and Sheryl R. Schindler, “Engendering the Evidence Base: A Critical Review of the 

Conceptual and Empirical Foundations of Gender-Responsive Interventions for Girls’ Delinquency,” Laws 2, 3 

(2013): 244-282.
ES-3 

factors as exposure to abuse or violence, poor academic performance, truancy, risky sexual 

behavior, substance abuse, and other stressors that may contribute to trauma and negative 

outcomes. PACE aims to get them back on track by providing services in a gender-responsive 

environment that addresses these risk factors and develops their strengths.

Girls in this voluntary program, who live primarily at home, attend PACE daily during 

normal school hours and receive academic and extensive social services. These services include 

academic instruction and advising, a life skills curriculum, assessment and care planning, 

individual and group counseling, parental engagement, volunteer service and work readiness 

opportunities, and transition and follow-up services. Girls typically plan to attend PACE for 

approximately one year and often return to other schools in their communities to complete their 

education.9 A low staff-to-girl ratio allows for individual attention and opportunities to build 

relationships, contributing to the girls’ sense of safety and belonging while they are in attendance. PACE centers strive to create inclusive environments in which a variety of support 

services “wrap around” each girl, and they rely on a strengths-based approach — emphasizing a 

girl’s assets rather than deficits — and an understanding of trauma and its effects when dealing 

with girls’ risky or challenging behaviors.

The PACE Evaluation

In response to the growing need to better understand and evaluate the services available to girls

at risk of school failure, delinquency, substance abuse, or other poor physical and mental health 

outcomes, this study aims to provide evidence on the execution and effectiveness of the PACE 

program. The evaluation has three main components: an impact study, an implementation study, 

and a cost-effectiveness analysis. Fourteen PACE centers participated in the evaluation during 

the two-year study enrollment period, from August 2013 through October 2015.

The impact evaluation employs a random assignment design. With this design, girls 

who were deemed eligible for PACE enrolled in the study and were assigned at random either 

to a program group, whose members are offered PACE services, or to a control group, whose 

members are referred to other services in the community. Between August 2013 and October 

2015, 1,134 girls were enrolled in the study (679 in the program group and 455 in the control 

group). The results of the impact study, which will be published in a future report, will provide 

information on the degree to which PACE prevented negative outcomes and created positive 

opportunities for girls.

 9

In some cases, girls seek options other than returning to the public school they attended previously or 

another school in the district; for example, earning a high school equivalency diploma and gaining employment. In rare cases, PACE centers provide a high school diploma through the local school district.
ES-4 

This report focuses on how PACE implements its gender-responsive services at each of 

the centers in the study. As noted above, few gender-responsive programs have been evaluated, 

and information on how this type of program actually operates is limited. This research also will 

inform policymakers and practitioners interested in understanding how the PACE program 

model is replicated across locations.

The implementation study focused on answering three main questions:

•

•

•

How is PACE implemented at each center? This included a close examination of how gender-responsive programming is provided.

Whom does PACE serve? This involved understanding how girls were selected to participate in the program and how girls in the study compared with 

the general population of girls in Florida.

How does PACE differ from other services available in the community for 

at-risk girls? 

The research team reviewed the intended implementation of services according to the 

PACE program model and compared these with the actual implementation of services. Additional analysis examined whether implementation of the model or services varied across centers. 

These analyses draw from a rich set of qualitative and quantitative data. Quantitative data 

presented in this report include the baseline characteristics of the research sample, program 

participation data, a survey of PACE staff members, a validated classroom observation scoring 

system, and a follow-up survey to the study sample of girls (both the program and control 

groups) 12 months after study enrollment.10 Qualitative data include staff interviews, observations of program activities, in-person individual interviews and focus groups with current PACE 

participants, and follow-up phone interviews with program group participants.

Key Findings

This section summarizes findings from the implementation study. The full report provides 

additional findings, including details on how services were delivered, for each component of 

PACE’s service model.

• The PACE program model was implemented consistently across multiple locations. Some variation in services across locations occurred be-

 10Fielding of this survey was ongoing at the time the current report was written. Therefore, the survey responses presented here are from girls enrolled in the study between August 2013 and March 2015, about twothirds of the full study sample.
ES-5 

cause of differences in staffing and access to resources and in program 

areas where the model provides less guidance.

PACE takes a hybrid approach to defining its model. In addition to a written set of values and guiding principles that describe how staff members should approach their work, PACE 

provides detailed guidance on program activities in a lengthy manual. PACE headquarters 

supports implementation through staff training, data systems, and a quality assurance process. 

The central office plays a key role, monitoring fidelity and providing guidance or support as 

needed. The research team found that individual PACE centers were implementing the model as 

intended. Staff members described in interviews how PACE’s values and guiding principles 

provided the foundation for how they did their work, and both management and direct service 

staff members reported using the manual regularly to guide program implementation. Data from 

PACE’s management information system show that girls at PACE are receiving most services 

at the prescribed frequency and intensity. 

Activities that were not specified in the manual had more variation in implementation. 

In these areas — for example, the approach counselors should use with girls — variation 

seemed to be driven largely by the experience and approach of individual staff members. 

Availability of resources was another factor. Though every center receives a basic level of 

funding through PACE’s contracts with the Florida Department of Juvenile Justice and local 

school districts,

11 each center drew on community resources to support or augment core services. Centers with more resources were able to offer additional services, such as health care 

and therapy on site. Centers with fewer resources had fewer staff members to dedicate to certain 

activities (for example, transition services, volunteer services, and work readiness).

• PACE incorporated gender-responsive programming into all services 

through a distinct program culture and through specific program components such as assessments, life skills classes, and parental engagement.

PACE’s model incorporates many of the key tenets of gender-responsive programming 

that are cited by practitioners and researchers in the field. The implementation research found 

that PACE infused gender-responsive programming into many aspects of service delivery 

through a distinct program culture, focusing on safety and relationships, an understanding of 

trauma, and an emphasis on building girls’ individual strengths. Key aspects of genderresponsive programming were also incorporated into assessments, life skills, and parental 

engagement. Table ES.1 provides an overview of common elements of gender-responsive 

programs and how they are put in practice at PACE.

 11PACE, Seek Excellence: 2014 Annual Report (Jacksonville, FL: PACE Center for Girls, 2014).
ES-6 

Table ES.1

Gender-Responsive Programming Principles and 

PACE Program Components

Category

Principle of 

Gender-Responsive 

Programming PACE Program Component

Program 

environment

Safety PACE provides secure facilities, behavior management, and a 

program culture that is intended to be safe from bullying and 

trauma triggers.

Focus on high-quality 

relationships

Staff members focus on building positive and supportive 

relationships with the girls. Care is informed by the other key 

relationships in a girl’s life, including family relationships. 

Strengths-based 

approach

Staff members are trained to recognize a girl’s assets and orient 

care toward building strengths rather than focusing on deficits. 

Trauma-informed 

approach

Staff members are trained to recognize the symptoms of trauma 

and to understand how trauma can affect a girl’s behavior. Staff 

members use knowledge of a girl’s trauma history to inform care. 

Assessment Holistic approach to 

treatment

PACE implements a comprehensive assessment process to 

understand a girl’s risk factors and protective factors across five 

domains: family, school, behavior, victimization, and health. 

Life skills Education about 

women’s health

The Spirited Girls! life skills curriculum educates girls about 

healthy relationships and general and reproductive health. Staff 

members work with girls to address specific women’s health 

needs.

Educational and 

vocational opportunities

Academic services provide girls with an opportunity to catch up to 

grade level by providing individual support in small classes. 

Career exploration is provided in Spirited Girls! classes or through 

separate career classes. Staff members provide individual support 

on career planning. 

Connections to the 

community

Volunteer service provides girls with the opportunity to connect 

with the community in a positive way.

Parental 

engagement

Emphasis on family Staff members engage a girl’s family in her care through regular 

updates on her progress and by seeking to address needs within 

the family when possible. Staff members use an awareness of 

each girl’s family dynamics to inform her care.

SOURCES: Developed from Patricia K. Kerig and Sheryl R. Schindler, “Engendering the Evidence Base: A 

Critical Review of the Conceptual and Empirical Foundations of Gender-Responsive Interventions for Girls’ 

Delinquency,” Laws 2, 3 (2013): 244-282; and interviews with PACE staff members. 
ES-7 

• PACE serves girls who tend to be low-income, to be struggling with 

school, and to have other risk factors, such as prior abuse or involvement with the juvenile justice system.

PACE implements a thorough assessment and screening process to assess whether a 

girl meets eligibility requirements, to understand her history and risk factors, and to determine 

whether she would be a “good fit” for the program. PACE serves girls who are struggling 

academically and who exhibit a range of health, safety, and delinquency risk factors. As shown 

in Table ES.2, at the time of study enrollment, about half the study sample had been held back 

at least once, and a large portion had low school attendance. Many girls had experienced abuse 

or neglect or reported having thoughts about harming or killing themselves. A significant 

portion of the sample were sexually active. Nearly 30 percent of participants had been previously arrested, and a majority of the sample had a family member with a criminal history. Participants also came predominantly from low-income families and often from single-parent households.

• PACE differed from what girls experienced in other school settings in 

several distinct areas. Girls assigned to the program group and invited 

to attend PACE were more likely than girls in the control group to have 

been enrolled in school and to have received academic advising, counseling, and other services in the 12-month period since study enrollment.

In a review of other programs available in the communities served by PACE, none offered a similar combination of academic and social services in a gender-responsive setting. 

Some programs offered both academic and social services but without the gender-responsive 

approach. PACE’s academic services differed from those offered at public schools, which many 

control group members attended, in terms of class size and access to regular academic advising. 

Classes were capped at 14 students, and the low student-teacher ratio allowed teachers to 

provide girls with more individual support. PACE also provided more academic advising than 

was typically provided by public schools. In the 12 months since random assignment, as shown 

in Table ES.3, girls in the program group were more likely to have been enrolled in school and 

to have received academic advising than girls in the control group. On the quality of classroom 

instruction, as measured by one common assessment tool, PACE scored similarly to public 

schools.

Social services at PACE also distinguished it from the traditional school environment. 

Responses from the follow-up survey indicate that girls in the program group received more 

social service support than girls in the control group during the 12 months following study 

enrollment. For example, girls in the program group were 19 percentage points more likely than 
ES-8 

Table ES.2

Selected Characteristics of Sample Members at Baseline

Full

Characteristic (%) Sample

Demographic

Age

11-12 8.5

13-14 32.5

15-16 49.5

17 or older 9.5

Race/ethnicity

Black, non-Hispanic 45.1

Hispanica 16.0

White, non-Hispanic 38.1

Other 0.8

People participant lives with 

Two parents 34.8

Single parent 51.8

Relative 10.6

b Other 2.8

c Family income

$28,050 or lower 41.2

$28,051-$44,900 35.5

Above $44,900 23.3

Academic

School level at time of referral to PACE

d 6th grade 8.8

7th-8th grade 37.2

9th-10th grade 45.3

11th-12th grade 8.7

Recently expelled or suspendede 39.6

Has more than 15 absences in past 3 months 41.7

Held back at least once 51.8

Has a learning disability 29.6

(continued)
ES-9 

Table ES.2 (continued)

Characteristic (%)

Full

Sample

Delinquency

Ever arrestedf 27.7

Ever been on probation 12.6

g Has family member with criminal history 64.1

Health and safety

Ever sexually active 44.1

Abused/neglectedh 38.1

Ever had thoughts about harming/killing herself 39.3

Sample size 1,134

SOURCE: MDRC calculations based on data from the PACE management 

information system.

NOTES: Certain characteristics listed here were captured in two different ways 

during the random assignment period, as noted below. a

Sample members are coded as Hispanic if they answered "yes" to Hispanic 

ethnicity. 

b"Other" includes nonrelative or foster care. cThe U.S. Department of Housing and Urban Development's guidelines were 

used in the data collection process to determine which income range the 

participant's family fell into. Since these figures could vary by county and 

household size, the ranges presented here correspond to statewide income limits 

for low income and very low income for a four-person household in Florida in 

fiscal year 2014.

dThis category includes sample members who were in fifth grade at the time of 

referral. e

For approximately half of the sample, this was defined as being currently 

expelled or suspended. For the other half of the sample, this referred to one or 

more expulsions or suspensions in the most recent school term. 

f

In the juvenile justice system, people are not technically "arrested"; the 

terminology used is either "incurred a charge" or "referred." gFor approximately half of the sample, this measure referred to a criminal 

record (including imprisonment, probation, parole, and house arrest) for a parent, 

guardian, or sibling of the sample member. For the other half of the sample, 

"family" included other members of the household as well. 

hFor approximately half of the sample, this measure referred only to 

documented instances of abuse or neglect. For the other half of the sample, the 

measure also included suspected incidents of abuse. 
ES-10

Table ES.3

Impacts on Service Receipt

Program Control Difference

Outcome (%) Group Group (Impact) P-Value

Academic Service Receipt

Ever enrolled in a school or educational program 99.1 92.9 6.2 *** 0.000

Received academic advising 81.1 67.6 13.5 *** 0.000

Frequency of academic advising sessions

More than once per month 38.0 33.8 4.2 0.293

Once per month 19.6 11.5 8.2 *** 0.009

1-3 times per year 23.4 22.4 1.0 0.770

Never 19.0 32.4 -13.4 *** 0.000

Social Service Receipt

Received help finding services in the community 38.5 23.3 15.2 *** 0.000

Received mental health counseling or therapy 64.2 45.5 18.7 *** 0.000

Frequency of counseling or therapy sessions

Once per week or more 39.3 23.9 15.4 *** 0.000

1-3 times per month 19.9 13.4 6.5 ** 0.043

Less than once per month 4.6 8.0 -3.5 * 0.081

Never 36.2 54.6 -18.5 *** 0.000

Received help related to sexuality, sex, 

or sexual and reproductive health 72.0 57.8 14.3 *** 0.000

Received help related to social and emotional skills 80.3 63.0 17.3 *** 0.000

Sample size (total = 668) 407 261

SOURCE: MDRC calculations based on girls' responses to the PACE evaluation 12-month follow-up 

survey.

NOTES: The sample size reported here is based on responses to the follow-up survey among girls 

randomly assigned between August 2013 and March 2015. Due to missing values, the number of girls 

included varies by outcome.

Results in this table are regression-adjusted, controlling for pre-random assignment characteristics. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent.
ES-11

girls in the control group to have received mental health counseling or therapy. They were also 

more likely to have received help connecting to other services in the community, such as 

transportation or housing, than the control group. And PACE offers more life skills programming focused on the needs and perspectives of girls than is typically available in public schools. 

For example, program group girls were more likely to report that they had received help related 

to sex and reproductive health than the control group (72 percent compared with 58 percent).

PACE also offers follow-up services to girls after they leave the program, though the intensity 

of services offered varied among the centers.

Summary

For girls at risk of an array of negative outcomes, this report highlights the ways that PACE is 

different from the other options available to girls in the communities where PACE operates. 

Few programs offer the same combination of services. The impact on reported service receipt 

bears this out. 

PACE is also an example of a program that provides similar services and a consistent 

culture in multiple locations in diverse communities. PACE’s approach — specifying its model 

through both principles and a manual, supporting staff members through training, and using 

data to monitor implementation and fidelity — offers lessons for the field more broadly. 

Variation tended to occur in areas where the program model was not specified — for example, 

the approaches counselors took with girls. Finding the balance between specification and 

flexibility is an ongoing tension in the replication of human service programs.

The implementation study of PACE also offers an opportunity to understand how a 

gender-responsive program actually operates, an area where current research is lacking. The 

report describes how PACE creates a gender-responsive culture as a framework for providing 

its services. The culture serves as the foundation for its gender-responsive programming and is 

infused into all aspects of program delivery. Building from this culture, PACE offers a combination of services that is hypothesized to meet the specific needs of at-risk girls.

This implementation report is one in a series of publications from the PACE evaluation 

that will add to the evidence base regarding gender-responsive programming and its effectiveness. In early 2016, a research brief provided an introduction to the study. Another brief, 

released as a companion to the current report, delves further into the history and literature 

around gender-responsive programming, using PACE implementation as a case study. And in 

2018, a final report will present the results of the impact study and a cost-effectiveness analysis, 

which will evaluate the costs of PACE in the context of its outcomes for girls.

1 

Chapter 1 

Introduction

In Florida, middle and high school-age girls who are falling behind academically or exhibiting 

behavioral problems may have an opportunity to get back on track through a unique program: 

PACE Center for Girls. PACE centers, located in 19 counties across the state, employ a “gender-responsive” approach to provide both academic and extensive social services. Girls attend 

daily and year-round. PACE offers classes with a low student-to-teacher ratio, regular counseling sessions, and a life skills curriculum designed for girls, among other services. PACE began 

more than 30 years ago as a program to meet the needs of girls involved with the juvenile 

justice system. The program focuses on serving girls who exhibit multiple risk factors for 

delinquency. 

These risk factors, which include individual, peer, family, school, and community characteristics, increase the likelihood that a girl will struggle in school and engage in delinquent 

behavior. Delinquency and involvement in the juvenile justice system, in turn, result in considerable personal and societal costs. Juvenile justice involvement may damage a young person’s 

relationships with friends and family, negatively affect mental health, and interrupt the academic progress and work experience that should accumulate during adolescence.1 And from a 

societal perspective, court and detainment costs are high. Therefore, effective prevention or 

early intervention programs that help young people avoid involvement in the juvenile system 

can offer a significant return on investment.2 Policymakers seek to identify and evaluate 

promising approaches.

Girls at risk of entering the juvenile court system, or those who are involved in the system already, have profiles that differ from those of their male counterparts: Girls are more often 

taken in for nonserious offenses, such as truancy or violating probation, and they are more likely 

to enter the system with a history of physical, sexual, or emotional abuse, extreme family 

conflict, or neglect.

3 Girls, more broadly, also have a greater incidence of depression than boys.4

Girls and boys tend to respond differently to trauma, and there is a stronger association between 

traumatic stress and mental health problems among girls.5 As most services are based on the 

needs of boys, the current juvenile justice system is not well positioned to meet the particular 

 1

Aizer and Doyle (2015).

2

Aos et al. (2004).

3

Bright and Jonson-Reid (2008, 2010). 4

Kleinfeld (2009).

5

Zahn et al. (2010).
2 

needs of girls. According to a report by the Georgetown Law Center on Poverty, Inequality and 

Public Policy, “the juvenile justice system only exacerbates [the girls’] problems by failing to 

provide girls with services at the time when they need them most.”6

Gender-responsive prevention programs offer a promising way to address girls’ unique 

needs.7 Specifically, “gender-responsive” describes treatment approaches for serving only girls 

and women, based on the understanding that the default approach is designed for boys and 

men.8 While good gender-responsive services share the basics of any good services — such as a 

well-trained staff and solid treatment approaches9

 — they are distinctive in bringing an awareness of girls’ particular development and gender-specific issues into the program. (Chapter 4 

provides more detailed information on gender-responsive programming.) Federal and local 

policymakers have lent their support for this type of program. A 1992 amendment to the 

Juvenile Justice and Delinquency Prevention Act provided funding for research and development of gender-responsive services.10 In 2004, the U.S. Department of Justice’s Office of 

Juvenile Justice and Delinquency Prevention (OJJDP) created a Girls Study Group to further 

the research base around programming for girls.11 More recently, OJJDP partnered with a 

national organization to create the National Girls Initiative, which provides training, technical 

assistance, and other resources to programs serving this population.12 And in 2015, OJJDP 

released a statement affirming its commitment to provide funding for research about girls in the 

juvenile justice system.13 Thus, at the federal level, gender-responsive services are considered 

an important part of the service array.

Even with this growing interest, rigorous research on gender-responsive programs remains scant. The current literature is robust in its description of concepts and principles, but 

until recently, it was largely unknown how gender-responsive services are implemented, how 

similar programs are to one another, or how effective they are.

14 Researchers have characterized 

the empirical literature in this area as “limited and inadequate”15 and “in its infancy.”16 

 6

Watson and Edelman (2013), p. ii. 7

Office of Juvenile Justice and Delinquency Prevention (2016a). 8

Covington and Bloom (2006); Zahn, Hawkins, Chiancone, and Whitworth (2008). 9

Maniglia (1998).

10Office of Juvenile Justice and Delinquency Prevention (2015, 2016a). 11Office of Juvenile Justice and Delinquency Prevention (2016b). 12National Crittenton Foundation (2016). The National Girls Initiative was formerly known as the National Girls Institute. 13Office of Juvenile Justice and Delinquency Prevention (2016a, 2016b). 14Chesney-Lind, Morash, and Stevens (2008); Kerig and Schindler (2013); Hubbard and Matthews (2008). 15Chesney-Lind, Morash, and Stevens (2008), p. 183. 16Bright and Jonson-Reid (2010), p. 103. 
3 

The evaluation of PACE Center for Girls — perhaps the largest and most wellestablished gender-responsive program of its kind — provides an opportunity to assess this 

approach. The evaluation is answering foundational questions about the implementation and 

effectiveness of a gender-responsive program, helping practitioners and policymakers better 

understand these services for at-risk girls. The evaluation is being conducted by MDRC and is 

funded mainly through the Edna McConnell Clark Foundation’s Social Innovation Fund (SIF), 

a program of the Corporation for National Community Service (CNCS), with additional funding 

provided by the Jessie Ball duPont Fund and the Healy Foundation. (See Box 1.1 for further 

information about the SIF.)

This report specifically provides information on the PACE program model and how 

PACE implements that model. The key findings from this report include the following: 

●

●

●

The PACE program model was consistently implemented across multiple locations, with a core set of similar services. Services differed somewhat across 

locations in program areas where the PACE model does not provide detailed 

guidance or when resources were limited.

PACE incorporated gender-responsive programming into its services by creating a safe, relationship-focused environment and by using tools and approaches that fit within the gender-responsive principles.

PACE’s environment and combination of services differed from what girls 

experienced in a more traditional school setting. Girls invited to attend PACE 

were more likely than a control group to have enrolled in school, received 

academic advising, participated in counseling, and received other services in 

the 12-month period since study enrollment.

PACE Center for Girls

PACE Center for Girls currently operates 19 nonresidential program locations across the state 

of Florida. Applicants to this voluntary program are 11 to 17 years old and are typically struggling academically and may have behavioral problems. PACE aims to help by providing

services in a gender-responsive environment that develops their strengths and addresses their 

risk factors, such as exposure to abuse or violence, poor academic performance, truancy, risky 

sexual behavior, substance abuse, and other family or social stressors.

PACE locations are referred to as “centers” — to differentiate them from the school setting that 

girls were usually coming from — and offer services year-round. Girls travel daily to the 

program during normal school hours and receive academic services and extensive social 
4 

Box 1.1

The Edna McConnell Clark Foundation (EMCF)

Social Innovation Fund

The Social Innovation Fund (SIF) — an initiative enacted under the Edward M. Kennedy 

Serve America Act — directs millions of dollars in public-private funds to expand effective 

solutions in three issue areas: economic opportunity, healthy futures, and youth development 

and school support. This work seeks to create a catalog of proven approaches that can be 

replicated in communities across the country. The SIF generates a 3:1 private-public match, 

sets a high standard for evidence, empowers communities to identify solutions to social 

problems, and creates an incentive for grant-making organizations to target funding more 

effectively to promising programs. Administered by the federal Corporation for National and 

Community Service (CNCS), the SIF is part of the government’s broader agenda to redefine 

how evidence, innovation, service, and public-private cooperation can be used to tackle urgent 

social challenges.

The Edna McConnell Clark Foundation, in collaboration with MDRC and The Bridgespan 

Group, is leading a SIF project that aims to expand the pool of organizations with proven 

programs that can help low-income young people make the transition to productive adulthood. 

The project focuses particularly on young people who are at greatest risk of failing or dropping 

out of school or of not finding work; who are involved or likely to become involved in the 

foster care or juvenile justice system; or who are engaging in risky behavior, such as criminal 

activity or teenage pregnancy.

EMCF, with its partners MDRC and Bridgespan, selected an initial group of nine programs 

and a second group of three programs to receive SIF grants: BELL (Building Educated Leaders for Life), the Center for Employment Opportunities, Children’s Aid Society-Carrera 

Adolescent Pregnancy Prevention Program, Children’s Home Society of North Carolina, 

Communities In Schools, Gateway to College Network, PACE Center for Girls, Reading 

Partners, The SEED Foundation, WINGS for Kids, Youth Guidance, and Children’s Institute, 

Inc. These organizations were selected through a competitive selection process based on prior 

evidence of impacts on economically disadvantaged young people, a track record of serving 

young people in communities of need, strong leadership and a potential for growth, and the 

financial and operational capabilities necessary to expand to a large scale.

The EMCF Social Innovation Fund initiative is called the “True North Fund” and includes 

support from CNCS and 15 private coinvestors: The Edna McConnell Clark Foundation, The 

Annie E. Casey Foundation, The Duke Endowment, The William and Flora Hewlett Foundation, The JPB Foundation, George Kaiser Family Foundation, The Kresge Foundation, Open 

Society Foundations, Penzance Foundation, The Samberg Family Foundation, The Charles 

and Lynn Schusterman Family Foundation, The Starr Foundation, Tipping Point Community, 

The Wallace Foundation, and Weingart Foundation.
5 

services. Girls typically plan to attend PACE for about one year and often return to schools in

their communities to complete their education.17 A low staff-to-girl ratio allows for individual 

attention and opportunities to build relationships, contributing to the girls’ sense of safety and 

belonging while they are in attendance. PACE centers strive to create inclusive environments in

which a variety of support services “wrap around” each girl, with attention to each girl’s 

individual strengths. Figure 1.1 shows program inputs, the services provided, and the intended 

outcomes. 

The gender-responsive PACE program model includes:

●

●

●

●

●

●

●

Academic instruction. Girls receive daily middle school- or high schoollevel instruction in a small class setting. Individual academic plans guide 

progress, which is monitored through regular advising sessions.

Life skills curriculum. Girls regularly attend a Spirited Girls! class that covers six domains believed to be essential for girls’ healthy development: physical, emotional, intellectual, relational, sexual, and spiritual.

Individual assessment and care planning. Assessments of each girl’s needs 

are used to create tailored plans for the girl’s time at PACE. Staff members 

meet regularly to share information and review progress. Staff members refer 

girls to services outside PACE as needed. 

Individual and group counseling. Girls attend frequent individual sessions 

with counselors and regular psychoeducational group sessions. 

Parental engagement. Program staff members maintain parental engagement through initial home visits, monthly progress reports, office sessions, 

and phone contact. 

Volunteer service and work readiness. The centers provide volunteer service opportunities, career exploration, and work readiness training.

Transition and follow-up services. Additional support is available for girls 

as they make the transition out of PACE and back to their home school or 

another appropriate placement. Staff members also check in with girls at regular intervals after they leave the program to provide services or referrals, if 

needed.

 17In some cases, girls seek options other than returning to their previous public school or another school in 

the district; for example, earning a high school equivalency diploma and gaining employment. In rare cases, 

PACE centers can provide a high school diploma through the local school district.
6
7 

Each center is led by an executive director, with additional leadership provided by a 

program director and other managers.18 A social service manager oversees a counseling staff, 

and an academic manager oversees the academic and life skills teachers. The statewide PACE 

headquarters provides supportive services to all the centers, assisting with fundraising, finance, 

human resources, legal matters, training, technical assistance, and information technology. The 

headquarters management team also advocates for resources and public policy at the state level 

and coordinates regular meetings of the center staff to allow for information sharing statewide.

PACE receives more than two-thirds of its funding through two sources: the state’s educational system and the Florida Department of Juvenile Justice (DJJ).19 For academic services, 

the centers contract directly with the local school districts to receive per pupil funding that 

supports a full academic school day.20 The DJJ funding supports the social services provided to 

each girl and her family, as well as facility expenses; this funding comes with specific eligibility 

criteria and limits a girl’s length of stay at PACE to 15 months.21 Additional contributions come 

from federal and state grants as well as local grants from public, corporate, and private sources.

The PACE Evaluation

In response to the need to understand and evaluate the services available to girls at risk of an 

array of negative outcomes, this study aims to provide evidence on implementation and effectiveness of the well-established PACE program. The evaluation has three main components: an 

implementation study, an impact study, and a cost-effectiveness analysis. In early 2016, a 

research brief provided an introduction to the study.22 A second brief, focused on genderresponsive programming, serves as a companion to this implementation report. And in 2018, a 

final report will present the results of the impact study and evaluate the costs of PACE in the 

context of its outcomes for girls.

Fourteen PACE centers actively participated in the evaluation.23 Typically, PACE centers serve a county-wide area. Figure 1.2 shows the location of the PACE centers by population 

 18Not all PACE centers are staffed with a program director. In some cases, a program director’s responsibilities are divided between the executive director and other managers. 19PACE (2014).

20Specifics of these contracts vary by center. Contracts may provide for academic materials, part-time 

staff, or transportation, among other items. 21This 15-month maximum length of stay went into effect during the study period. PACE can appeal to 

DJJ in individual cases to ask for an extension beyond this 15-month period. 22Millenky and Mage (2016).

23As shown in Figure 1.2, at the time of this report’s publication, 19 PACE centers operated in Florida. 

Two centers opened after the evaluation began and thus were not eligible to be included; two other centers that 

(continued)
8 

 

density. Lower-density counties tended to have higher poverty rates.24 Table 1.1 presents 

selected characteristics of the participating centers. Most centers had the capacity to serve about 

were initially slated to participate in the study were excluded because of low enrollment during the study 

period; and one rural center was not included because of ethical concerns about a lack of other services in the 

community that would be available to a control group. 24Florida Legislative Office of Economic and Demographic Research (2011). 
9 

 

(continued)

Table 1.1

Characteristics of PACE Centers in the Study

Characteristic Mean Min Max

Executive director tenurea,b (years) 7.5 0.2 22.6

Staff tenureb (years) 4.0 1.9 8.1

Years in community 17.9 7.0 28.0

Number of program slots 54.3 41.0 81.0

Number of staff members 18.4 13.0 33.0

Staff-to-slot ratio 1:3 1:4 1:2.33

Cost per slotc

 ($) 34,486 27,975 39,596

Sample size 14

SOURCES: MDRC calculations based on responses to the PACE evaluation staff survey, 

MDRC site visit data, and program information provided by PACE.

NOTES: a

This measure is based on data from the PACE evaluation staff survey (N =13).

bTenure refers to years at current center.

c

This is an average of program costs for fiscal years 2014 and 2015.

50 girls at a time, though the number of program slots ranged from about 40 to 80.25 Staffing 

varied accordingly, based on the center size. Nearly all the participating centers had operated for 

at least 15 years.

For the impact analysis, the evaluation employed a random assignment design. Girls 

who were deemed eligible for PACE enrolled in the study and were assigned at random either 

to a program group, whose members were offered PACE services, or to a control group, whose 

members received referrals to other services in the community. The program’s existing application and screening processes were used to determine eligibility, and 1,134 girls were enrolled in 

the study between August 2013 and October 2015 (679 in the program group and 455 in the 

control group).26 (See Appendix C for further information about the random assignment

25This represents the average number of program slots during the two-year study enrollment period, from 

summer 2013 through summer 2015. 26All girls who applied and were eligible for PACE during the study period were included in the study, 

except girls who had previously attended PACE, siblings of current or recent PACE participants, and girls in 
10

procedures.) The impact analysis will use a 12-month follow-up survey and administrative 

records to examine outcomes for girls in each research group. Follow-up data collection 

activities are the same for the program and control groups. Key outcomes are academic progress, academic engagement, juvenile justice involvement, healthy relationships, and risky 

behavior (such as high-risk sexual activity and substance abuse). Findings from the impact 

study will indicate whether PACE prevented negative outcomes and created positive opportunities for girls in and out of school.27

Focusing mainly on implementation, the current report provides detailed information 

about the services that PACE provides, how those services are delivered, and whether services 

vary across PACE centers. As noted earlier, research on implementation and effectiveness of 

gender-responsive programs is quite limited. The current study addresses this gap and may also 

inform policymakers and practitioners who are interested in understanding how a program 

model is implemented across a number of locations. The implementation analysis draws on rich 

data, such as multiday visits to each participating center, surveys, and program participation 

records. 

●

●

●

Site visits: Members of the research team visited each of the participating 

centers to interview program staff members at all levels, conduct interviews 

and focus groups with girls, speak to local board members and stakeholders, 

and observe classrooms and staff meetings.28 These visits occurred during the 

study enrollment period, between May 2014 and March 2015.

Staff survey: A web-based survey was administered to all permanent program staff in the participating programs between October 2014 and March 

2015. This survey gathered information about staff roles and backgrounds 

and about PACE’s organizational culture.29

12-month follow-up survey of girls: A lengthy follow-up survey was administered by phone or in person to study sample members approximately 

 

state custody through the Florida Department of Children and Families. In addition, MDRC granted each 

center a small number of “hardship passes” to serve girls with specific circumstances who PACE believed 

could not be served well elsewhere in the community. These girls were not a part of the study. 27As a condition of funding, the SIF requires a “large, well-designed and well-implemented randomized 

controlled, multisite trial” to provide a “strong level of evidence” (Corporation for National and Community 

Service 2014, p. 3). The impact study follows these guidelines. 28Classroom observations used the CLASS-Secondary tool, which is described in more detail in Chapter 5. 29Interns and temporary staff members did not complete the survey. The response rate for the survey was 

high (91 percent). Information on organizational culture was collected using the 105-item Organizational 

Social Context instrument (Glisson, Green, and Williams 2012). Management-level staff members were not 

asked this set of questions.
11

one year after they applied to PACE and enrolled in the study.

30 The information collected in this follow-up survey will be used mainly in the impact 

analysis, but responses from the program group girls about their experiences 

with the PACE program are included in the current report. On average, girls 

in the program group responded to this survey approximately four months after leaving PACE. In addition, the survey provides information, presented in 

Chapters 5 and 6, on services received over the one-year period by both the 

program and control groups.

●

●

●

Program participation data: PACE collects extensive data in a centrally 

managed management information system (MIS) at its state headquarters.31

The data are the most complete source of information on the level and types 

of services girls receive while at PACE. They include baseline demographic 

and risk factor information (collected during the program application process) and extensive program participation details. As discussed in this report, 

program staff members regularly enter information about girls’ attendance 

and receipt of services throughout the girls’ time at PACE. See Appendix C 

for further information.

Interviews with PACE management and stakeholders: Members of the 

research team interviewed the founder of PACE and key members of the 

PACE leadership team. These interviews occurred in January 2015 and focused on understanding the history of PACE, the responsibilities of PACE’s 

central state office, PACE’s funding structure, and PACE’s advocacy role 

within Florida and at the national level.

Follow-up interviews with girls and parents: Girls who completed the 12-

month follow-up survey and their parents or guardians were asked to participate in an additional interview. Among those who agreed, girls and parents 

were contacted between July 2015 and March 2016. A total of 52 girls and 

 30Fielding of this survey was ongoing at the time the current report was written. Therefore, the survey responses presented throughout the report are from girls enrolled in the study between August 2013 and March 

2015, about two-thirds of the full study sample. While all sample members were approached to complete the 

survey, not all girls could be reached or agreed to participate. Specifically, the response rate was 71 percent 

overall; 407 girls in the program group (73 percent) and 261 in the control group (69 percent) responded to the 

survey. 31PACE uses Social Solutions ETO (Efforts to Outcomes) software for its MIS.
12

40 parents were interviewed by telephone in an in-depth, semistructured interview about their experiences with PACE and other schooling or services.32

Overview of Chapters

The remainder of this report presents the findings from the implementation study. Chapter 2 

provides background information and findings on the overall implementation of the PACE 

model. Chapter 3 presents information on girls participating in the study and the intake application process at PACE. Chapter 4 provides information on gender-responsive programming and 

the PACE environment. Chapters 5 and 6 discuss PACE’s academic and social services, 

respectively, including a comparison of the services received by the program and control 

groups. Chapter 7 summarizes the key findings and discusses the ongoing work at PACE and 

for the evaluation.

 32Girls were selected to represent the different center locations and both program and control groups, as 

much as possible. The pool of parents was smaller, so all parents who agreed to participate were contacted.
13

Chapter 2 

PACE Implementation

This chapter describes how PACE implements its program model and how it ensures that 

centers provide services as designed. The chapter first describes how PACE codifies its model 

and how staff members are trained and supported in service delivery, then presents evaluation 

findings on fidelity to the model. Subsequent chapters build on this context and go into greater 

detail about PACE implementation and the variation of services across centers. This chapter 

presents the following key findings on implementation: 

●

●

●

PACE defines its model through a combination of general principles and a 

highly specific manual. Through ongoing staff training and quality assurance 

initiatives, PACE headquarters supports individual centers in their implementation of the model to ensure consistency. As a result of these efforts, PACE 

centers are similar in the types and quantity of services they provide. 

PACE centers had high fidelity to the aspects of the model specified by 

PACE headquarters. The few instances where fidelity was lower occurred at 

centers that had lower staff capacity or had struggled with recent staff turnover.

Variation in implementation mostly occurred in service areas where the program model was not specific and the staff was given flexibility in how to implement the program. Differences in services provided by centers were also 

associated with access to local resources.

Implementation of the PACE Model

In order to assess implementation of a program, researchers must understand both the program 

model and how its developers disseminate that model to the staff. Program developers can take 

a variety of approaches to dissemination. One approach is described as “principle-based,” in 

which programs define a set of principles, or broad guidelines, for the activities that the staff 

should carry out. These principles are based on the program’s theory of change. Although the 

activities are defined, principle-based models provide staff members with flexibility and 

discretion in how they implement individual components. At the other end of the spectrum are 

“manualized” program models. In these models, each program activity is defined in detail, so

staff members generally have less flexibility in how program components are implemented.
14

PACE takes a hybrid approach to defining its model, combining principle-based and 

manualized approaches. On the principle-based side, PACE has a written set of values and 

guiding principles that describe how staff members should approach their work. For example, 

the “Focus on Strengths” principle states: “We look to identify strengths in our girls, their 

families, our staff and supporters. Using these strengths as our foundation, we build strong, 

confident, productive community partnerships.” During interviews, staff members described the 

values and guiding principles as providing the framework for how services are delivered, how 

decisions are made, and how staff members interact with each other and with families. How 

PACE puts its principles into operation is discussed in greater depth in Chapter 4. 

Specific guidance on program activities is provided in the PACE policy and procedures 

manual, a document of nearly 400 pages that covers policies from staff training to required 

staff-to-girl ratios to field trip logistics. Box 2.1 provides an example of the content of the 

manual. Staff members described the manual as central to staff training and decision-making. 

New hires read the manual upon starting work at PACE. Management and direct service staff 

used the manual regularly to answer questions that came up in day-to-day work, such as how to 

respond to chronic absenteeism. Managers also said that the policy manual gave them credibility when giving guidance to other staff members because they could point to a decision as 

following policy rather than being subjective. Important policies and changes passed down from 

headquarters were discussed at staff meetings.

Although the manual offers specifics about what activities should occur within a PACE 

center and how often, the manual is less focused on the content of services or how services 

should be provided. For example, though the manual specifies that girls should meet every other 

week with their counselors to review their care plans, there is no policy in the manual outlining 

the approach that counselors should take in their meetings with girls. One senior manager at 

PACE described the limitations of the PACE manual in this way: “You can follow the letter of 

the policy and be accomplishing nothing. Or you can follow the letter of the policy and be 

making great progress. There’s lots of variability there.”

PACE’s staff recruitment and training practices also support implementation of the program at the centers. Job descriptions are created at headquarters, so the required qualifications 

and job description for a counselor or a teacher are the same across all centers. Staff training is 

mostly delivered by each center’s own staff, with PACE headquarters providing guidance and 

developing some specific training modules. Staff members who work directly with the girls on 

a daily basis (such as teachers and counselors) must complete 80 hours of training when they 

first join PACE, plus 40 hours of annual training in subsequent years. New staff members 
15

Box 2.1

Excerpt from PACE Policy and Procedures Manual

Policy # 3.11 Policy Title: Staff to Girl Ratio

Revised Date: 07/01/2013

Original Date: 09/01/1992

Policy

PACE will generally adhere to a 1:12 staff to girl ratio.

Purpose

• To provide care management services and academic instruction.

• To provide individualized attention to each girl.

Procedure

A. Upon enrollment to the program, a girl will be assigned a Teacher Advisor, advisee group 

and a social services support staff member.

B. The [staff to girl] ratio of academic classes may be 1:14 in order to meet the following 

needs:

1. To enable the program to effectively meet girls’ academic needs by offering classes to 

girls based on need and interest.

2. To meet academic special needs by placing a small grouping of girls in a class, thereby 

increasing the number in other classes.

3. To enable classes to be staffed in the case of teacher absence and the unavailability of a 

substitute.

C. Social service groups will generally adhere to the 1:12 ratio. Some groups may exceed this 

ratio to meet the needs of the girls.

D. The number of girls assigned to a specific Social Service Support Staff will be based on 

the individual Center’s organizational chart.

E. The 1:12 ratio will be maintained during all program activities not mentioned above. The 

ratio should be reduced when participating in activities outside of the Center.

F. A group list can be viewed and/or printed from the ETO system to ensure appropriate staff 

to girl ratio.

G. Any exception to the policy must have prior approval from the Chief Program Officer or 

designee.

__________________________

SOURCE: PACE policy and procedures manual.
16

receive training on the PACE model, gender-responsive programming, and trauma-informed 

care.1 Each center develops an annual “master training plan” to ensure that training requirements are met; many of the training requirements are passed down through the Florida Department of Juvenile Justice (DJJ) or school district contracts. Although the content of the required 

training is similar, centers can tailor training sessions to suit their own needs; for example, one 

center provided staff training on human trafficking when the center saw an increase in the 

number of girls affected by it.

PACE’s data systems play a key role in supporting implementation. The policy and 

procedures manual stipulates the frequency and intensity of all key services. These services are 

tracked for each girl in PACE’s Efforts-to-Outcomes (ETO) database. Counselors and teachers 

have the primary responsibility for entering data into the database. Managers regularly audit 

these entries to ensure that staff members are including the proper information in their entries 

and that girls are receiving services at the prescribed frequency. The focus on data was apparent 

in staff interviews and the staff survey. Counselors reported that they spent almost a quarter of 

their time on paperwork and data entry responsibilities; on average, teachers reported spending 

13 percent of their time on those tasks.

PACE headquarters further supports implementation through organization-wide training and model development activities, including peer learning, and takes the lead role in quality 

improvement efforts around identified needs. Examples include creating working groups that 

draw from both headquarters and center staff to update the Spirited Girls! curriculum or refine 

eligibility guidelines.

Finally, PACE headquarters plays an important role in monitoring implementation. At 

the time of MDRC’s implementation visits, PACE was using an extensive pen-and-paper

monitoring instrument developed from the policy and procedures manual that focused on 

ensuring that centers were in compliance with PACE and DJJ policies.2 The instrument was 

administered by headquarters staff members and DJJ staff members during a two-day annual 

visit to each center. Activities during the visits included the review of documents, review of a 

random selection of entries into PACE’s management information system, and interviews with 

girls. If the monitoring visit revealed areas where the center was not in compliance with the 

model, headquarters and the center would together create a plan of action to address the issue.

 1

Chapter 4 provides details on PACE’s gender-responsive programming and approach to trauma-informed 

care. In trauma-informed care, staff members are trained to recognize the symptoms of trauma and to understand its effects on behavior. 2

Headquarters staff members were in the process of changing this monitoring method at the time of 

MDRC’s interviews.
17

Implementation Fidelity and Variation Across PACE Centers

The implementation research sought to understand how closely individual centers implemented 

the PACE model and the degree to which there were differences in implementation across the 

centers. These lines of inquiry were important for two main reasons. First, because the research 

sample is pooled across centers for the impact analysis, it is important to understand whether

girls in the study received similar services regardless of which center they attended, in order to

interpret the results. Second, policymakers, practitioners, and researchers are interested in 

successfully replicating program models in new contexts. PACE’s approach to implementation 

may offer lessons for ensuring that multiple providers of a program adhere to an intended 

intervention.

The MDRC evaluation team developed its own tool to assess the fidelity of PACE centers to the core program model. The tool was based on PACE's manual, with an additional 

section on gender-responsive programming. MDRC researchers used data gathered during site 

visits and reviewed documents provided by the centers to rate each center on fidelity to 48 

items. The items were divided into six domains:

●

●

●

●

●

●

Staffing, including training and communication among staff (5 items)

Program culture, including program environment, incentive systems, and facilities (8 items)

Social services, including intake, assessment, and counseling (15 items)

Transition services, including follow-up services (5 items)

Academics, including classes and academic advising (9 items)

Gender-responsive programming (6 items)

For each item at each center, site visitors recorded a rating of “fidelity,” “partial fidelity,” “no fidelity,” or “unable to observe.” Site visitors were given guidance on what benchmarks 

needed to be met for each item to achieve a rating of “fidelity” or “partial fidelity.” Each 

completed tool was reviewed by the lead implementation researcher to ensure consistency 

across ratings.

Figure 2.1 shows that fidelity in PACE centers was strong overall. Across all centers 

and domains (a total of 672 ratings), the rating “fidelity” was given 77 percent of the time. 

Across all centers and domains, about 3 percent of items were rated “no fidelity.” The figure 

shows that when broken out by domain, “partial fidelity” or “no fidelity” ratings were more 

likely to occur in academic services or transition services. These two areas, discussed in 

Chapters 5 and 6 of this report, were areas where staffing was less stable.
18

Figure 2.1

Distribution of Program Fidelity Ratings by Domain

0

10

20

30

40

50

60

70

80

90

Percentage of ratings

Domain

No fidelity

Partial fidelity

Fidelity

Rating

SOURCE: Calculations based on MDRC site visit data.

NOTES: Fourteen centers were rated on 48 items organized across six domains. For each item, centers earned a 

rating of either "fidelity," "partial fidelity," or "no fidelity." The percentages shown for each domain are across 

all centers and items. For example, for academic services, of the 126 ratings given in that domain (9 items × 14 

centers), 70 percent were "fidelity."

The fidelity analysis provides one perspective on how similarly PACE centers are implementing the model, but because the evaluation tool was primarily based on the PACE 

manual, it is limited in terms of assessing the content or quality of services received by the girls 

in different centers. Counseling approaches, as noted earlier, are not specified in the manual. 

Interviews with the staff at each center provided further information about service content and 
19

quality that were not reflected in the fidelity instrument. Overall, the research found there was 

variation in how individual centers provided services, and variation within centers in how 

individual staff members delivered services.

In the areas where staff members had discretion in how they implemented the model, 

variation in services was related to the experience and approach of those individuals. As 

discussed in Chapter 5, analysis of data from a validated classroom observation tool found that 

teacher quality varied across teachers, rather than across centers. Similarly, as described in 

Chapter 6, interviews with counselors indicated that the approach a counselor took with a girl 

depended largely on the girl’s individual needs and the counselor’s background and training.

At the center level, differences in implementation were primarily associated with the 

availability of resources. Each center drew on local resources to support its programs and would 

use these resources to support core services or augment the services it was able to provide. The 

fidelity analysis found that lower fidelity to the program model was most common in the areas 

of transition services and, within academic services, volunteer service and work readiness. 

Centers with lower fidelity in these areas tended to have fewer staff resources to dedicate to 

these activities, or had recently experienced staff turnover in these roles. A few centers were 

able to draw on local resources to provide services beyond PACE’s core model, such as on-site 

access to health care and mental health therapy.

This chapter provides a framework in which to understand the findings presented in 

Chapters 4 to 6, which examine how PACE centers implemented each core service component. 

Each chapter discusses how services were intended to be provided according to the PACE 

model and PACE’s implementation plan and what services actually were provided, drawing on 

observations, interviews, and data from PACE’s management information system. Areas of 

variation in implementation among the centers or staff members are also discussed.

21

Chapter 3 

PACE Girls

This chapter describes the ways in which PACE recruits girls and determines their eligibility, 

and draws on interviews to offer the perspectives of girls and their parents on why they came to 

PACE. The chapter then presents background characteristics of the entire study sample (both 

the program and control groups) and provides an overview of program group participation in 

PACE. Key findings are as follows: 

●

●

●

●

Developing referral relationships is central to PACE’s recruitment efforts; 

schools are the primary referral source.

PACE uses a multistep process to assess applicants for suitability for the program. PACE staff members administer a formal assessment and then review

the girl’s academic and social service needs, as well as her motivation to attend the program.

Girls at PACE have a range of risk factors that qualify them for PACE’s services, including behavior problems, academic underachievement, truancy,

and a record of suspension or expulsion from school. 

Girls in the program group enrolled in PACE at a very high rate, and their 

average length of stay at the program was approximately eight months.

Recruitment

PACE engages in recruitment efforts in order to maintain an active stream of applicants to the 

program. As PACE’s main funding from the Florida Department of Juvenile Justice (DJJ) and 

the school districts is dependent on the number of girls served, consistently receiving referrals to 

keep program slots filled is central to a PACE center’s financial health. Furthermore, because 

girls may enter and exit the program at any point during the year, recruitment efforts continue 

year-round. Each PACE center had an outreach counselor to manage recruitment during the 

evaluation period.1

 1

The outreach counselor position was specifically created to support the additional recruitment and program intake responsibilities brought on by the study. Before the evaluation, these tasks were largely spread 

among social service staff members. Once study intake activities ended, each center decided whether to 

continue using an outreach counselor or to use a different staffing structure for recruitment efforts.
22

A major part of recruitment for PACE involves building relationships with referral 

sources in the community — conveying what PACE does and what kinds of girls it serves, so 

that partners can identify potential applicants and direct them to the program. PACE’s main 

referral relationships are with schools; half the outreach counselors interviewed noted that 

school guidance counselors or social workers were primary referral sources. Juvenile probation 

officers (JPOs) are another source; while PACE is primarily a prevention program, girls who 

have been or are currently involved with the juvenile justice system can also be referred to 

PACE. JPOs are the source of fewer referrals than schools, however, for at least two reasons 

reported by outreach counselors: Turnover among JPOs results in a constant need to teach new 

officers about PACE, and many girls on probation may not be appropriate for PACE. 

Besides developing these main referral relationships, outreach counselors post flyers or 

leaflets in the community to recruit new girls and often receive new applicants by word of 

mouth from girls who previously attended PACE.

The Program Application Process

After receiving a referral or being contacted by a girl’s parent or guardian, the PACE staff 

implements a thorough process to determine whether an applicant is eligible for PACE. Figure

3.1 illustrates these steps. This application process is largely specified in the policy and procedures manual. All applicants must first meet a set of basic requirements, including age, sex,2 and 

having risk factors across a set of domains (family, school, behavior, victimization, and health).3

During the application process, the girl and her family receive an orientation to PACE, 

which includes an overview of the program and a tour of the facilities. Staff members conduct a 

formal assessment with the girl, called the Initial Needs Assessment (INA).4 The assessment 

consists of questions structured around the domains given above and helps the staff understand 

the girl’s history and risk factors in greater detail. During the evaluation, an additional step in 

 2

Girls who identify as boys are eligible for the program; however, some PACE staff members questioned 

whether they were appropriate for PACE. Staff members also noted that sometimes issues related to a girl’s 

gender/sex identity arose after she enrolled. PACE is reviewing its policies on gender/sex identification to 

address these questions and concerns. 3

DJJ (a PACE funder) requires that girls exhibit risk factors in at least three domains in order to enroll. 4

After about half the study sample had been randomly assigned, DJJ required that PACE use a different 

assessment called the Prevention Assessment Tool (PAT). There are minor differences between the PAT and 

the INA. Staff members were trained to use motivational interviewing to collect data for the PAT.
23
24

the application process provided an overview of the study and asked the girl and her parent or 

guardian to consent to participate in it.

5

 

Last, a group of designated staff at the center meets to discuss whether the girl is a good 

fit for PACE and make a final determination of eligibility. The manual offers broad guidance on 

determining eligibility, stating that PACE will make “every attempt” to provide services, but 

that PACE may not be the best fit for all girls referred. Staff members discussed with researchers three main considerations in determining a girl’s fit for the program:

●

●

●

Academic needs. Staff members consider whether PACE has the ability to 

serve a girl’s academic needs. Almost all outreach counselors noted that they 

could not serve girls with Individualized Educational Programs that exceed 

the capacity of the center, such as girls with special needs who require a separate classroom. Centers also considered how close the girl was to graduation; since most PACE centers cannot issue high school diplomas, girls nearing graduation may not be a good fit for the program. 

Social service needs. Staff members focus on a girl’s behavioral history to 

determine whether they can meet her social service needs. The manual does 

not provide specific guidance on what behavior issues would make a girl a 

poor fit for PACE; it appeared that centers took an individual approach to 

each girl. In reviewing a girl’s behavioral history, staff members try to understand the cause of behavior issues and determine whether they would be a 

problem at PACE, particularly if there is the possibility of harm to other girls 

at the center. Two counselors also mentioned that PACE was not equipped to 

serve girls with severe mental health problems. Specifically, if the girl falls 

under the Florida Mental Health Act,6 then by state law, PACE must refer 

her to an institutional facility. There did not seem to be any other firm rules 

on what mental health issues would mean a girl could not be served by 

PACE.

Motivation. The manual states that staff members should consider whether 

the girl’s choice to enroll in PACE is voluntary. Nearly all outreach counselors said that they considered the girl’s motivation in determining eligibility. 

 5

Nearly all girls who applied and met both PACE and study eligibility criteria consented to participate in 

the study. In a few cases where PACE believed girls could not be served adequately elsewhere, the study 

participation requirement was waived. 6

The Florida Mental Health Act, Fla. Stat. § 394.459 (2008), commonly known as the Baker Act, gives 

people with severe mental illnesses the right to emergency mental health services and temporary detention 

(University of Florida Health 2017).
25 

Counselors suggested that if girls are not motivated, it would be difficult for 

them to succeed at PACE, but they indicated that they had a low threshold 

for what qualified as motivation. Many described motivation simply as a 

girl’s willingness to attend the program or acknowledgment that she needed 

change in her life, as opposed to showing overt enthusiasm for the program. 

In fact, many PACE girls reported that they were not originally interested in 

coming to PACE. Several counselors noted that they worked with resistant 

girls to persuade them of the value of the program; one explained that often 

such resistance emerged when a girl was told by her parent to go to PACE, 

rather than deciding for herself. Overall, while staff members seemed to view 

motivation as an ideal quality in an applicant, low motivation did not appear 

to be a reason to turn someone away.

As noted, the manual offered limited guidance on determining eligibility. In December 

2014, after the majority of implementation research visits took place, PACE headquarters 

circulated more specific guidance about eligibility determination and factors to be considered 

during intake. The overall message was one of inclusivity, stressing that turning a girl away 

should be a “rare and serious occurrence.”

During the study period, as reflected in Figure 3.1, girls were randomly assigned after 

staff members followed all steps in the eligibility determination process described above and 

deemed them eligible for PACE. In most cases during the study period, girls assigned to the 

program group were invited to enroll at the center immediately; if all spaces were filled at the 

center, program group girls were placed on a waiting list and invited to enroll as soon as a spot 

opened up. Girls assigned to the control group were provided a referral list of other options in 

the community. The list was often tailored or customized based on the staff’s determination of 

what other programs would be the best fit for the girl based on their assessment of her during 

the intake process.

Why Girls Come to PACE

Interviews and focus groups with PACE girls provided insight into the reasons they came to the 

program. Most girls reported that they had attended a public school before coming to PACE and 

had been having some sort of trouble at school. The most prevalent issues included behavior 

problems (such as disrespecting teachers, skipping classes, or not following the rules at home), 

failing classes, truancy, expulsion, and suspension. Girls also cited mental health issues, drug 

and alcohol use or abuse, negative peer influence, and bullying as factors in their arrival at

PACE.
26

In one interview, a girl discussed her experience with drug use and negative peer influence before PACE and how it affected her progress in school. She said she was spending time 

with the wrong group in high school; her best friend was doing drugs and pressured her to 

smoke marijuana, and the more she smoked, the more her grades dropped. She also struggled 

with grief after two of her friends died.

Many girls said that they did not want to attend PACE initially; some girls did not want 

to go to an all-girls school, and others reported that their mothers or other family members made 

them go to PACE. At the same time, many girls had seen potential benefits to attending PACE. 

Girls had looked forward to the opportunity to catch up academically, to the access to counselors, and to a caring staff and community. Some of these sentiments are illustrated in excerpts 

from individual interviews with the girls. In one interview, a girl described what she found most 

appealing when she first visited PACE — a place where she felt welcomed: “It is actually going 

to feel like a school. The other ones [programs] didn’t really feel like a school. It just felt like, 

this is where bad people come when they get in trouble. Here it’s not all about bad people. 

Some people just come to get their school work done.” One girl said that she wanted to come to 

PACE because of the program’s close-knit community: “When I first came here I was seeing 

how everyone was so close and how it was like a big family and it made me think of my own 

family, so I just like, fell in love with it.” Another girl was especially drawn to the idea of 

having her own counselor, since at her previous school it was very difficult to meet with the 

counselor. She explained, “It was hard having nobody to talk to.”

Background Characteristics of the Study Sample

Tables 3.1 and 3.2 present demographic and risk characteristics of the study sample at the time 

the girls applied to PACE.7 This sample closely resembled the types of girls PACE serves more 

generally.8 Additionally, the risk factors presented are consistent with girls’ self-reports to the 

implementation research team about the issues that brought them to PACE.

As shown in Table 3.1, the majority of girls in the study (82 percent) were 13 to 16 

years old. Most came from low-income families (77 percent). About half the sample lived in 

single-parent households, and just over one-third came from two-parent households. Forty-two 

percent of sample members came from families that had had some type of involvement with the 

Department of Children and Families, which handles cases of abuse and neglect.

 7

See Appendix Tables A.1 and A.2 for characteristics by research group. 8

In fiscal year 2015, PACE served a total of 2,130 girls. As context for some of the figures cited in the rest 

of this section, among all girls served in fiscal year 2015, 74 percent came from low-income families, 74 

percent had failed a class recently, 29 percent exhibited runaway behavior, and 28 percent had been arrested in 

the past. 
27

Table 3.1

Characteristics of Sample Members at Baseline

Characteristic (%)

Full

Sample

Age 

11-12

13-14

15-16

17 or older

Race/ethnicity

Black, non-Hispanic

Hispanica

White, non-Hispanic

Other

School level at time of referral to PACE

b 6th grade

7th-8th grade

9th-10th grade

11th-12th grade

English is second language

Qualifies for special education or ESEc

People participant lives with

Two parents

Single parent

Relative

d Other

e Family income

$28,050 or lower 

$28,051-$44,900

Above $44,900

Family has had case with the 

f Florida Department of Children and Families

8.5

32.5

49.5

9.5

45.1

16.0

38.1

0.8

8.8

37.2

45.3

8.7

2.1

11.2

34.8

51.8

10.6

2.8

41.2

35.5

23.3

42.2

Sample size 1,134

(continued)
28

 

Table 3.1 (continued)

SOURCE: MDRC calculations based on data from the PACE management 

information system.

NOTES: aSample members are coded as Hispanic if they answered "yes" to 

Hispanic ethnicity. 

bThis category includes sample members who were in fifth grade at the time of 

referral. c

PACE uses the Florida Department of Education definition of Exceptional 

Student Education (ESE), referring to programs for students with disabilities and 

gifted programs. 

d"Other" includes nonrelative or foster care.

eThe U.S. Department of Housing and Urban Development's guidelines were 

used in the data collection process to determine which income range the 

participant's family fell into. Since these figures could vary by county and 

household size, the ranges presented here correspond to statewide income limits 

for low income and very low income for a four-person household in Florida in 

fiscal year 2014.

f

This measure is a reflection of what sample members self-reported; it is not 

according to their parents or guardians.

These data indicate that sample members struggled in school before coming to PACE. 

As shown in Table 3.2, at the time of random assignment, three-quarters of girls in the study 

had failed a class recently, and just over half had been held back at least once. Significant 

portions of the study sample also had low school attendance (42 percent) or had been recently 

suspended or expelled (40 percent).9 These rates of academic risk are considerably higher than

the rates among all students in the counties where PACE centers are located; in the 2013-2014 

school year, only 5 percent of students were held back, 10 percent had low school attendance, 

and 15 percent were expelled or suspended.10

Girls in the study sample face a number of health and safety risk factors. As shown in 

Table 3.2, a significant portion of the sample, 44 percent, had been or were currently sexually 

active. This figure is similar to the percentage of high school girls in Florida in 2013 who had 

ever had sexual intercourse (40 percent), even though the PACE study sample consists of both 

9

Low school attendance is defined as more than 15 absences in the past three months. 10These data were not available by grade. They were calculated from Florida Department of Education

data as the number of students absent 21 or more days over total enrollment for the counties where participating PACE centers are located.
29

Table 3.2 

Risk Factors of Sample Members at Baseline

Characteristic (%)

Full

Sample

School engagement

Recently expelled or suspendeda

Currently enrolled in school

Skipped school at least 3 times in past 2 months

Had more than 15 absences in past 3 months

Held back at least once

Failed 1 or more classes in past 6 months

Has a learning disability

Attention deficit disorder

Dyslexia

Other learning disability

Delinquency

Ever been arrestedb

Ever stolen from family, home, or neighbors

Ever been on probation

Currently on probation

c Has family member with criminal history

Has friends with delinquent record or who engage in

delinquent behavior

Health and safety

d Currently using tobacco

Currently using drugs or alcohole

Ever sexually active

Currently pregnant

39.6

73.1

34.7

41.7

51.8

76.6

29.6

19.4

1.5

8.8

27.7

16.7

12.6

10.1

64.1

49.8

9.9

14.8

44.1

1.4

(continued)
30

 

Full

Characteristic (%) Sample

Ever run away from home 27.6

Ever had thoughts about harming/killing herself 39.3

Abused/neglectedf 38.1

Neglected 8.9

Physically abused 15.9

Sexually abused 15.1

Emotionally abused 21.7

Sample size 1,134

Table 3.2 (continued)

SOURCE: MDRC calculations based on data from the PACE management 

information system.

NOTES: Certain characteristics listed here were captured in two different ways 

during the random assignment period, as noted below. aFor approximately half of the sample, this was defined as being currently 

expelled or suspended. For the other half of the sample, this referred to one or 

more expulsions or suspensions in the most recent school term. 

bIn the juvenile justice system, people are not technically "arrested"; the 

terminology used is either "incurred a charge" or "referred." cFor approximately half of the sample, this measure referred to a criminal 

record (including imprisonment, probation, parole, and house arrest) for a 

parent, guardian, or sibling of the sample member. For the other half of the 

sample, "family" included other members of the household as well. 

dFor approximately half of the sample, this was defined as having used 

tobacco three or more times in the past 30 days. For the other half of the sample, 

this was defined as currently using tobacco. eFor approximately half of the sample, this was defined as having used drugs 

or alcohol three or more times in past 30 days. For the other half of the sample 

this was defined as current drug and or alcohol use.

f

For approximately half of the sample, this measure referred only to 

documented instances of abuse or neglect. For the other half of the sample, the 

measure also included suspected incidents of abuse. 

middle and high school girls.11 Approximately 28 percent of sample members had ever run 

away from home. In line with some of the issues girls reported during interviews, 38 percent of 

11Florida-wide percentages refer to female-only Florida responses to the Youth Risk Behavior Surveillance Survey in 2013 (Centers for Disease Control and Prevention 2016). 
31

girls in the study had experienced at least one type of abuse (physical, sexual, or emotional) or 

neglect, and 39 percent reported having thoughts about harming or killing themselves. The high 

incidence of these experiences presents a need for what is known as trauma-informed care, as 

discussed in Chapters 4 and 6.

About 28 percent of sample members reported having been arrested before coming to 

PACE. Unsurprisingly, given PACE’s target population, this percentage is much higher than the 

official arrest rate for the broader population. In 2014, only 1 percent of the Florida female 

population ages 10 to 17 had ever been arrested. Ten percent of the study sample were on 

probation at the time of study enrollment, while less than 1 percent of the Florida female youth 

population had been on probation in fiscal year 2013-2014.12 In terms of other delinquency risk 

factors faced, a majority of the sample had a family member with a criminal history (64 percent).

Program Group Participation

Most girls assigned to the program group enrolled at PACE (90 percent) and did so fairly 

quickly — within about two weeks. Table 3.3 provides measures of participation at PACE 

among the program group.13 According to PACE’s program data, among those who enrolled at 

PACE, about three-quarters stayed for longer than 90 days.14 On average, girls were enrolled for 

just under eight months, but some girls in the study sample stayed much longer, up to 33 

months.

In the 12-month follow-up survey, the percentage of girls who reported having attended 

PACE matched the program’s administrative records.15 As Table 3.3 shows, among the few 

who did not attend, the most commonly reported reason was the decision to enroll in another 

school or program instead (36 percent). Almost one-quarter reported not coming to PACE 

because they lacked transportation; this was also an issue regularly noted by center staff 

members in interviews with the implementation team. Nonattendees also reported that they did 

not enroll in PACE because they decided they did not like the program (23 percent) based on 

their introduction to it during the application process. This disinclination could involve not 

wanting to go to an all-girls school or other components of PACE that are unlike the traditional 

public school.

 12Data derived from Florida Department of Juvenile Justice (2016). 13These figures are based on data from the PACE management information system and a partial sample 

from the 12-month follow-up survey. The final report will update these figures for the full sample. 14This measure captures only a girl’s first enrollment at PACE. 15Only 31 program group girls among the survey sample respondents thus far had not attended PACE for 

at least one day.
32

Table 3.3

Program Participation (Program Group Only)

Service Program Group

Enrolled in PACE day program (%)

Among girls who enrolled in PACE

Time from random assignment to enrollment (days)

Average daily attendance (%)

Length of staya (months)

Length of staya (%)

Less than 30 days

Between 30 and 90 days

90 days or more

Among girls who did not enroll in PACE b

Reasons for not coming to PACE (%)

Enrolled in another school or program

Did not have transportation

Did not like the program

Moved

Family member became ill

Parent or guardian did not like the program

c Other

90.3

19.4

72.0

7.9

7.3

18.3

74.4

35.5

22.6

22.6

12.9

9.7

9.7

25.8

Sample size 679

SOURCES: MDRC calculations based on data from the PACE 

management information system and the 12-month follow-up survey.

NOTES: aThis measure represents only a participant's initial stay at 

PACE.

bCalculations are based on responses to the 12-month follow-up survey 

and include girls randomly assigned through March 2015 (N = 31). cThe percentage in the "other" category appears high, but it represents 

only eight girls. 
33

Chapter 4 

Gender-Responsive Programming

and PACE’s Program Environment

This chapter provides a description of the key components of PACE’s intended program 

environment and approach to gender-responsive programming and explains how staff members

put these concepts into action.1 The chapter presents the following main findings:

●

●

●

PACE’s program environment is its foundation for providing genderresponsive programming. The key components of this environment are a focus on safety, relationships between staff members and girls, recognizing and 

promoting girls’ individual strengths, and responding to the needs of girls 

who have experienced trauma. 

Ensuring that programs foster a common culture across multiple locations is 

a challenging task. PACE has successfully done so, with staff members describing a similar program environment regardless of their role or location. 

Interviews with girls and parents corroborate this: They experienced a safe 

and welcoming program environment similar to what the staff intended. 

Findings from the Organizational Social Context tool, a measure of organizational culture and climate, were consistent with staff descriptions from interviews. Results indicate a culture and climate that gave priority to meeting the 

needs of the client; high morale among the staff; and indications that staff 

members felt a limited ability to change organizational processes, likely reflecting PACE’s formalized approach to model implementation.

Gender-Responsive Programming

Gender-responsive programming grew out of the recognition that girls involved in the juvenile 

and criminal justice systems have particular needs. Girls’ pathways into the justice system are 

typically different from those of boys. Girls in the juvenile justice system are more likely than 

boys to have experienced sexual violence, extreme family conflict, and child maltreatment.2

 1

In this chapter, program environment refers to the internal environment of the PACE centers, specifically 

the environment that girls, families, and staff members experience while at the centers. This chapter does not 

cover the external program environment in which the center operates. 2

Bright and Jonson-Reid (2008, 2010).
34

Because of these different experiences, girls and boys need different services, but most prevention programs have been designed for boys.3 Gender-responsive programs were developed to 

provide an approach that is tailored for girls. However, the field is still developing, and evidence 

on the effectiveness of gender-responsive programming is sparse.4

A key research question for this evaluation was to understand how PACE puts the theory of gender-responsive programming into action. There is no single definition of genderresponsive programming, but Table 4.1 presents commonly cited principles and shows how the 

components of PACE’s model correspond to these principles. The components fall into four 

main areas:

●

●

●

●

Program environment. Providing girls with a program environment that is 

safe and that integrates relational, strengths-based, and trauma-informed approaches in all aspects of program delivery (detailed in this chapter).

Assessment. Employing a comprehensive assessment process to understand 

a girl’s history, strengths, and risk factors for delinquency (detailed in Chapter 3).

Life skills. Providing girls with a gender-specific life skills curriculum that 

focuses on issues particular to a women’s health, supports career readiness, 

and offers connections to community (detailed in Chapter 5).

Parental engagement. Engaging the family in a girl’s care at PACE (detailed in Chapter 6).

Program Environment

Program environment is often a focus in youth programs because it is viewed by many researchers and practitioners as a key aspect of service delivery and as connected with youth 

outcomes. Aspects of program environment include how staff members interact with participants, rules governing behavior, the program’s physical space (decorations and security features), and how participants are involved in leadership within the program. Program environment is also seen as central to the implementation of gender-responsive programming.5 For 

these reasons, MDRC’s implementation research examined PACE’s program environment in 

depth. 

 3

Zahn, Hawkins, Chiancone, and Whitworth (2008). 4

Hubbard and Matthews (2008). 5

Covington and Bloom (2006).
35

Table 4.1 

Gender-Responsive Programming Principles and 

PACE Program Components

Category

Principle of 

Gender-Responsive 

Programming PACE Program Component

Program 

environment

Safety PACE provides secure facilities, behavior management, and a 

program culture that is intended to be safe from bullying and 

trauma triggers.

Focus on high-quality 

relationships

Staff members focus on building positive and supportive 

relationships with the girls. Care is informed by the other key 

relationships in a girl’s life, including family relationships. 

Strengths-based 

approach

Staff members are trained to recognize a girl’s assets and orient 

care toward building strengths rather than focusing on deficits. 

Trauma-informed 

approach

Staff members are trained to recognize the symptoms of trauma 

and to understand how trauma can affect a girl’s behavior. Staff 

members use knowledge of a girl’s trauma history to inform care. 

Assessment Holistic approach to 

treatment

PACE implements a comprehensive assessment process to 

understand a girl’s risk factors and protective factors across five 

domains: family, school, behavior, victimization, and health. 

Life skills Education about 

women’s health

The Spirited Girls! life skills curriculum educates girls about 

healthy relationships and general and reproductive health. Staff 

members work with girls to address specific women’s health 

needs.

Educational and 

vocational opportunities

Academic services provide girls with an opportunity to catch up to 

grade level by providing individual support in small classes. 

Career exploration is provided in Spirited Girls! classes or through 

separate career classes. Staff members provide individual support 

on career planning. 

Connections to the 

community

Volunteer service provides girls with the opportunity to connect 

with the community in a positive way.

Parental 

engagement

Emphasis on family Staff members engage a girl’s family in her care through regular 

updates on her progress and by seeking to address needs within 

the family when possible. Staff members use an awareness of 

each girl’s family dynamics to inform her care.

SOURCES: Developed from Kerig and Schindler (2013) and interviews with PACE staff members.
36

PACE staff members at all levels and across all participating centers were asked to describe the environment they intended to create for girls. Staff members were also asked how 

they implemented a gender-responsive approach in their day-to-day work. In all, 112 staff

members across 14 centers were asked questions about the environment and gender-responsive 

programming. Responses to these questions focused on the program environment principles 

listed in Table 4.1: safety, high-quality relationships, a strengths-based approach, and a traumainformed approach. These components are also more generally cited as important to effective 

programs for young people.

6 Staff responses related to each of these topics are summarized 

below. The following sections also describe the perspectives of girls and parents based on

interviews and the 12-month survey of girls. 

Safety

In line with the principles of both youth and gender-responsive programming, staff

members first and foremost emphasized that the environment should be safe from both physical 

and emotional threats. One senior manager at PACE described the importance of safety at 

PACE and its impact on girls: “I’ve never seen anything like the level of ... emotional and 

physical safety that’s present in a center. And you can see it in the girls in the way they act. And 

the way they show up as girls, as kids, instead of with all those extra sort of layers.”

PACE employed several strategies to support physical safety. To control who enters 

and leaves the facilities, all PACE centers are secured with locked external doors that require 

visitors to be buzzed in and registered at a front desk. Girls were required to store their personal 

belongings in a designated area upon entering to prevent them from bringing in unsafe or 

distracting items. Rooms that were not in use were locked to prevent girls from accessing areas 

that were not supervised by staff. The main physical threat mentioned was fighting between 

girls enrolled in the program. To prevent this, the staff practiced “sight and sound” supervision, 

which meant ensuring that all girls were within hearing or sight of a staff member at all times. 

Staff members also emphasized the importance of knowing the girls and their current situations 

as a way to maintain safety within the center. Staff members would pay closer attention to girls 

they knew were in crisis. Daily communication, such as morning meetings, were used to update 

staff members on emerging issues with or between girls in the program.

PACE’s discipline policy promotes safety. Many staff members said the response to a 

behavior issue was tailored to each girl and her circumstances, but staff members would 

consider the safety of all girls in the center when deciding on consequences. Some centers had 

zero tolerance policies around fighting, and girls who fought would be expelled (“transitioned”). 

Nonphysical infractions, such as bullying or being insubordinate to a teacher, could result in a 

 6

Catalano et al. (2004); Ko et al. (2008). 
37

suspension, either in school or out of school (known as “days of reflection”). The PACE policy 

and procedures manual provides guidance on discipline policies, outlining the approaches that 

centers should take and prohibiting discipline techniques such as “use of isolation.” Some 

centers were required to adopt the local school district’s student code of conduct and zerotolerance policies.

Preventing bullying was a particular focus, and centers provided antibullying education 

through Spirited Girls! (life skills) classes or special events, such as schoolwide assemblies 

dedicated to celebrating diversity and promoting a “sisterhood” culture within PACE. Centers 

structured responsibility for discipline in different ways. At about half the centers, management 

staff members rather than counselors disciplined girls so as to preserve the therapeutic relationship between counselors and girls. At the other half, counselors did have a role in discipline, and 

managers believed it was beneficial to have a counselor help a girl process the issues that 

caused the behavior.

Most girls and parents described PACE as a safe place. Parents were also overwhelmingly positive about the environment they encountered at PACE; most described it as a safe

environment and said incidents of bullying were dealt with appropriately. In describing the safe 

aspects of the environment in interviews, girls said that staff members monitored the center 

closely, and girls who were a threat to the safety of the school would be expelled. One girl said: 

“If the staff finds out there’s going to be a fight they are on top of it and end it right then and 

there.” Girls and parents liked that the buildings were secure and that PACE limited access by 

outsiders. Most girls did not think bullying was a problem, noting that staff members were on 

the lookout for bullying and would address it immediately. Not all girls felt safe, however: 15 

percent of girls in the follow-up survey said they felt concerned about safety while at PACE.

Girls did say that being in an all-girls environment could lead to “drama,” which was 

described as distinct from bullying, referring instead to minor issues between girls, such as two 

girls having the same romantic interest. Though girls in interviews could be cavalier about the 

“drama,” the follow-up survey of girls suggests that it could become a more significant issue for 

some; of girls who had left PACE, nearly a third of them reported that not liking or getting 

along with other girls in the program was a contributing factor.

High-Quality Relationships

After emphasizing that the centers should have a foundation of safety, staff members 

most often said that the environment should provide nurturing relationships for the girls. 

Common terms used by staff members included “loving,” “family-like,” “positive,” and 

“supportive.” One counselor said: “It’s a loving environment. When girls walk in the door, they 

know that PACE staff care for them.” Staff members described their relationships with girls as 

central to implementing a gender-responsive approach, and their answers aligned with gender-
38

responsive theory about the relational needs of girls. This theory holds that relationships are 

central to girls’ lives and to their self-esteem.7 In the words of one female staff member: “Girls 

are relational, everything that happens in our lives is based on a relationship, whether it’s good, 

bad, or indifferent.” Staff members reported that most girls at PACE have had harmful relationships in their lives, and staff members attempted to counteract that by building positive, affirming relationships.

Staff members described a number of strategies to create nurturing relationships, most 

of which focused on the staff’s role and interaction with the girls. They emphasized the importance of knowing each of the girls and their backgrounds, both to help the girls feel welcomed and to try to avoid actions or activities that might be a trigger, or reminder, of a girl’s 

prior trauma. Staff members described being friendly with all the girls at the center, not just 

those who were in their classes or on their caseloads. Research team members observed this 

strategy in practice during site visits; staff members would greet all the girls who walked by 

them, and it appeared that many girls felt comfortable dropping in on the executive director or 

other members of the management staff.

Noting that girls might have had limited opportunities to see what a healthy relationship 

looks like, staff members reported modeling such relationships both with the girls and with each 

other: For example, they would interact with girls in the way that they wanted girls to interact 

with them, being positive and upbeat in their exchanges. Staff members would also encourage 

other positive relationships in girls’ lives. Within the center, these efforts could involve peer 

mediation between girls who were not getting along, to address the source of conflict and help 

girls learn how to prevent further problems; in addition, counselors would work with girls on 

ways to promote healthy relationships with their families and romantic partners. 

Girls described positive relationships with the staff members at PACE. Most girls interviewed said the staff members were the best thing about PACE. They described them as 

“family-like” and “loving,” similar to how the staff described the intended environment. In the 

words of one girl, there was “unsolicited, genuine care between the staff and the students.” 

Nearly all girls interviewed said there was at least one PACE staff member with whom they 

could discuss personal issues; this was most commonly a girl’s counselor but also included 

others at the center. Positive feelings toward PACE staff members were not universal, however: 

In the follow-up survey, 18 percent of girls in the program group said they felt misunderstood 

by the PACE staff.

Though most girls interviewed had positive feelings about the program culture, some 

expressed dislike or discomfort with the way staff members related to them. These girls tended 

 7

Gilligan (1982).
39

to say they felt “babied” by the staff, giving the example of teachers or counselors talking down 

to them or treating them as if they were younger than they were. One girl felt uncomfortable at 

PACE because, in such a small school, she could not “hide” in the way she could at her larger 

district school.

Nearly all parents interviewed said that they felt comfortable at their daughter’s PACE 

center, many because the staff made them feel welcomed. Parents said they could get to know 

the staff at the center because of its small size compared with public school. Describing the 

environment at PACE, one parent said: “I had a smile on my face when I first went into that 

school.”

A Strengths-Based Approach

Staff members also talked about the importance of building girls’ self-confidence 

through a strengths-based approach. The strengths-based approach, which originated in the 

social work field, focuses on identifying and building on a client’s strengths and assets to help 

her achieve her goals. Staff members noted that girls may not get much encouragement from 

their families and may not be used to hearing positive things about themselves. This emphasis 

on strengths was implemented formally through PACE’s rewards system and informally 

through the day-to-day interactions between the staff and the girls.

The PACE manual requires each center to implement a rewards system to recognize 

and encourage positive behaviors in the girls. The specifics of the rewards program varied by 

center, but in general girls could advance to reward “levels” by completing specified milestones, 

such as maintaining a certain attendance rate. Rewards were either tangible (such as clothes or 

make-up) or privileges (a lunch date with a staff member or allowances in the dress code). 

Some centers also had a points system that was separate from the level system and offered girls 

ways to earn rewards by the day or week. The points systems offered girls more immediate 

recognition for positive behavior, while the level system focused on longer-term goals. Although each center offered a rewards system, interviews with staff members and girls revealed 

that girls’ involvement varied. Staff members strived to create a system that would appeal to 

diverse interests, but not every girl wanted to participate. In addition to the system, staff members would be on the lookout for positive behavior and used “gotchas” (where a staff member 

would provide immediate and spontaneous recognition) to reward a girl for doing something 

positive.

Counselors also took a strengths-based approach in their one-on-one sessions with girls. 

In these sessions, counselors helped girls recognize their strengths and referred back to those 

strengths when helping girls develop strategies to address the challenges in their lives. Counselors also tried to help parents see their daughters’ strengths by noting them in parent-counselor 

meetings along with areas for improvement. Some staff members focused on “female empow-
40

erment” themes when talking about their strengths-based approach, such as helping girls 

understand that their job opportunities in life were not limited by their gender.

In follow-up interviews about PACE’s impact on their lives, girls most commonly said 

that they learned to recognize positive things about themselves while at PACE: for example,

helping others with school work, being a good listener, and being a good mother. More broadly, 

many girls said they had higher self-esteem after attending PACE. Reflecting on her time there, 

one girl said she learned that “I was worth helping, worth giving a second chance, and worth 

realizing my potential.”

A Trauma-Informed Approach

A trauma-informed approach describes an organization’s response to the needs of those 

who have experienced trauma. A trauma-informed approach refers to a component of an 

organization’s culture; trauma-specific interventions refer to clinical approaches used to 

promote a client’s recovery from trauma. Commonly cited components of a trauma-informed 

approach include a focus on safety, transparency in organizational operations to build trust, a 

focus on collaboration and relationships, a strengths-based approach, and recognition of 

cultural, historical, and gender issues.8 PACE’s implementation of several of these key components were described in earlier sections of this report. The current section describes how PACE 

supports the implementation of a trauma-informed approach through training and ongoing 

professional development.

All PACE staff members who interact with girls go through trauma training when they 

first start at PACE and annually thereafter. Staff members are trained to understand the impact 

of trauma on a person’s life, to recognize the symptoms of trauma, and to interact with girls in a 

way that avoids inadvertently contributing to their trauma and supports their healing. Staff 

interviews indicate that PACE has been successful in this training. Staff members described 

efforts to ensure that PACE was a safe place emotionally and that its environment would not 

trigger the girls’ prior traumas. For example, staff members would ask permission before 

touching a girl or warn girls if the content of a movie or book being read in class dealt with a 

difficult subject.

Another aspect of PACE’s trauma-informed approach involves understanding the impact of trauma on a girl’s behavior. For example, staff members said, if a girl was being 

disruptive in class, they would try to recognize how her experience might be affecting her as 

opposed to taking a punitive approach. One staff member put it this way: “If [the girls] had a 

trauma that caused them to have high anxiety about social situations, then that’s going to come 

 8

Substance Abuse and Mental Health Service Administration (2014).
41

up in everything that they do every day. You have to keep that in mind that she has had this 

horrible experience.”

Fostering a Gender-Responsive Program Culture

Fostering a similar program culture across multiple program locations is challenging, as each 

center operates in its particular external environment and staff members are not shared across 

centers. PACE promotes its program culture primarily through a set of values and guiding 

principles, shown in Box 4.1. These values and guiding principles were woven into many 

aspects of program operations. They were prominently displayed at the centers, and staff

members referred to them when they talked to girls and sometimes when they talked to each 

other. New staff members received training on the values and guiding principles, as well as 

training on staff role modeling, gender-responsive programming, strengths-based behavior 

management, and trauma-informed care.

The research found that each PACE center reflected a program culture that was consistent with the overall PACE values and guiding principles. Staff responses to the interview 

questions about program environment showed strong consistency in descriptions of the ideal 

PACE environment and the strategies to achieve it across roles and centers. Moreover, most 

staff members at different levels in the centers — executive directors, middle management, and 

counselors — gave similar answers to questions about how they incorporate a genderresponsive approach into their work.

Teachers were the only group that gave less consistent responses. Although many 

teachers provided answers that demonstrated use of a gender-responsive approach, about 40 

percent of teachers interviewed either had difficulty describing how they used a genderresponsive approach or said that they do not use such an approach in their work. A few teachers 

admitted to not fully subscribing to the gender-responsive approach. Others would offer 

descriptions of how they incorporated such programming into their classrooms, but the examples they provided did not in fact reflect gender-responsive principles. For example, in describing the primary way that they were gender responsive, one teacher said that girls should be 

taught to be “civil” while another said she focused on the decorations in the classroom. The 

lower level of knowledge among the teachers about gender-responsive approaches was associated with shorter tenures at PACE. The majority of teachers who had difficulty describing their 

use of a gender-responsive approach had been at PACE for less than a year; half of them had 

been at PACE for less than six months.

Broadly, however, the PACE centers in this study shared a common program culture, 

even though they were operating in different local contexts. The uniformity of staff responses 
42

Box 4.1

PACE Values and Guiding Principles

Honor the Female Spirit. We value and promote the female perspective by respecting its 

distinct needs, creating safe gender-responsive environments, and celebrating the female 

experience.

Focus on Strengths. We look to identify strengths in our girls, their families, our staff and 

supporters. Using these strengths as our foundation, we build strong, confident, productive 

community participants.

Act with Integrity and Positive Intent. We believe that all actions and decisions must be 

guided by the highest ethical principles, respecting the uniqueness of all involved and honoring 

the differences.

Embrace Growth and Change. We believe that everyone is capable of remarkable growth, 

and only by encouraging change can individuals, organizations and society reach their full 

potential.

Value the Wisdom of Time. We understand that patience can be as powerful as immediate 

action, and each has its place. We value the discernment required for their effective use.

Exhibit Courage. We think courage is essential in making a difference, enabling us to speak 

for those who cannot, take risks to do what is right, deliver just and fair consequence and be 

accountable for our actions.

Seek Excellence. We strive for excellence in all we accomplish by holding true to our mission 

while consistently meeting high standards of performance, reflecting critically upon our 

accomplishments, seeking innovative solutions, and believing all things are possible.

Create Partnerships. We believe in developing effective partnerships and long-term relationships, by listening to our staff, our girls, their families and our communities, incorporating their 

input and involving them in our decision-making.

Invest in the Future. We place our faith in the long-term growth and development of our 

girls, staff, agency and communities, believing it is the best strategy for creating results that 

have lasting impact.

__________________________

SOURCE: PACE policy and procedures manual. 

(with the exception of some teachers, noted above) is related to the training staff members 

receive and the structured approach to model implementation. The consistency of staff descriptions of the program culture may also be a reflection of the consistency of messages from PACE 

leadership: The chief executive officer and executive directors used a common language to 
43

describe the intended environment and gender-responsive programming, and the language used 

by direct service staff members mirrored that of their managers. The finding that teachers with 

shorter tenures had a difficult time articulating PACE gender-responsive approaches indicates 

that staff turnover, and the need to train new staff members on the program environment, can 

cause barriers to implementing the program environment as intended.

Measuring Program Environment

Measuring the environment of a program is often a difficult and subjective task — programs sometimes refer to their program environment as the “secret sauce” that no one knows 

how exactly to measure or replicate. This evaluation used the Organizational Social Context 

(OSC) tool as a measure of PACE’s program environment.9 Social context, or an organization’s 

culture and climate, is thought to be a factor in human service program implementation, 

associated with staff adoption of organizational practices, fidelity, staff turnover, and relationships between the staff and clients.10 This study uses the OSC as one measure of the social 

context at PACE centers. As described in this section, the OSC results are consistent with the 

research team’s findings from interviews with PACE staff members. 

The OSC is designed to measure organizational culture, organizational climate, and 

worker morale, discussed further below. These constructs are assessed through a 105-item 

survey that asks staff members to rate on a five-point scale how often each statement applied to 

their center experiences. The scale ranges from “never” to “always.” Culture and climate are 

each made up of subscales, described in Box 4.2. Morale, also shown in Box 4.2, measures 

individual job satisfaction and commitment to the center.11 OSC scores are calculated for each 

organizational unit as t-scores, or standardized scores, where the mean is 50 and the standard 

deviation is 10.12 The mean and standard deviation were established from a national sample of 

100 mental health organizations across the nation; the respondents in the national sample were 

all mental health clinicians providing services to children and adults.13

 9

The OSC was administered to all direct service staff members. 10Glisson et al. (2007); Hemmelgarn and Glisson (1998); Glisson and Green (2006). 11Scores for culture and climate subscales are calculated at the organizational level, which in this case was 

each PACE center. Morale scores are first calculated at the individual level and then averaged for each center. 12Standardized scores ensure that the scores are normally distributed, with half the scores falling below the 

mean and half falling above the mean. As an example, a center that had a score of 60 on proficiency would fall 

above the scores of more than 84 percent of the agencies in the sample. 13The sample was derived from a subset of participating clinics in the National Survey of Child and Adolescent Well-Being. Across the participating clinics, 1,154 clinicians took the OSC. See Glisson, Green, and 

Williams (2012). 
44

Box 4.2

Organizational Social Context Subscales

Culture

Proficiency measures competency and responsiveness. Competency is the degree of emphasis 

an organization places on job knowledge and expectations of excellence from employees. 

Responsiveness measures the extent to which the staff is expected to place the well-being of 

clients first and meet their unique needs. One example of a survey statement that measures 

proficiency is “Members of my center are expected to be responsive to the needs of each 

client.”

Rigidity measures centralization (the extent to which power is diffused within the organization) 

and formalization (the extent to which procedures guide work-related interactions). In rigid 

environments, the staff has less ability to exercise discretion and limited power to influence 

key management decisions. One example of a survey statement that measures rigidity is “The 

same steps must be followed in processing every piece of work.” 

Resistance measures apathy toward change and suppression of change. Apathy is the extent to 

which the staff is resigned to the status quo versus interested in actively seeking new ways of 

providing service. Suppression looks at how attempts at change are handled within the organization. One example of a survey statement that measures resistance is “Members of my center 

are expected to not make waves.” 

Climate

Engagement looks at the degree to which the staff feels connected and involved with clients, as 

well as the degree to which staff members feel that they are effective in their roles. A survey 

item that measures engagement is “I have accomplished many worthwhile things in this job.” 

Functionality looks at cooperation among staff members, clarity of staff roles, and opportunity 

for growth and advancement. Two survey items that measure functionality are “My job 

responsibilities are clearly defined” and “PACE provides numerous opportunities to advance if 

you work for it.”

Stress measures whether staff members feel overloaded, overwhelmed, and emotionally 

exhausted by their jobs. Survey items include “I feel like I am at the end of my rope” and “The 

amount of work I have to do keeps me from doing a good job.”

Morale

Morale measures staff satisfaction with and commitment to their jobs. Survey items include “I 

really care about the fate of this organization.” 

__________________________

SOURCE: Paraphrased from a report provided to MDRC by the Center for Behavioral Health Research. 
45

The scores for PACE centers are shown in Figure 4.1. Each dot represents the t-score 

for a center, and the national mean of 50 is shown as a horizontal line. As shown, nearly every 

center had a t-score above 50 on each subscale. These scores indicate that PACE centers 

exhibited a different OSC profile from that of the national comparison sample, which likely 

reflects, at least in part, the different contexts that PACE programs operate in compared with 

mental health organizations (for example, academic services rather than health care) and the 

different types of roles that staff members at PACE play compared with mental health clinicians. While t-scores tended to be high, the culture and climate subscales show a range of 

scores. Morale scores, by contrast, are clustered. The next paragraphs discuss possible interpretations of these scores and limitations of their use.

Organizational climate is defined by the OSC as “the psychological impact of the work 

environment on the individuals who work there.”14 Compared with a national sample of mental 

health organizations, most PACE centers scored above the mean on the OSC climate subscales: 

engagement, functionality, and stress. Engaged cultures reflect organizations where staff

members feel personally invested in their jobs, which is how many staff members described 

their work at PACE. Nearly all PACE centers scored high on the OSC’s measure of functionality, which indicates that PACE staff members generally feel that there is clarity around individual staff roles and opportunities for advancement within the organization. Most PACE centers 

scored near or above the mean on stress, indicating that although they were engaged in their 

work and there was clarity around roles, they still found their jobs to be emotionally exhausting 

or overwhelming to some degree. This is not surprising given the needs of the population PACE 

works with.

Organizational culture is defined by the OSC as “how things are done” and the expectations of the organization for its staff.15 Compared with the national sample of mental health 

organizations, most centers scored above the mean on the culture subscales: proficiency, 

rigidity, and resistance. According to OSC documentation, a high proficiency score is associated with organizations where employee excellence and competence are valued and staff members are expected to put the needs of clients first.16 As described in the previous section, a strong 

“client first” culture emerged from the interviews with staff members, where staff decisionmaking is driven by what is in the best interest of the girls. PACE’s scores on the rigidity and 

resistance subscales indicate that staff members feel they have limited discretion over job tasks 

and limited input into organizational practice. PACE’s manual and implementation support, 

 14Center for Behavioral Health Research (2016). 15Center for Behavioral Health Research (2016). 16This information is from a report prepared for MDRC by the Center for Behavioral Health Research.
46

Figure 4.1

Distribution of Organizational Social Context Scores by Center

20

50

80

T-score

OSC subscales

Climate Culture Morale

SOURCE: PACE evaluation staff survey using the Organizational Social Context (OSC) tool (Center for 

Behavioral Health Research 2016). 

NOTE: OSC scores are calculated as t-scores, or standardized scores, where the mean is 50 and the standard 

deviation is 10. Each dot represents the t-score for a center, and the national mean is shown as a horizontal line. 

Standardized scores were calculated by the OSC developer, the Center for Behavioral Health Research at the 

University of Tennessee at Knoxville.
47

including extensive training and quality assurance audits described in Chapter 2, may be 

reflected in these scores. Policy and procedures are promulgated by PACE headquarters in 

Jacksonville, so most staff members who work with girls would have limited interaction with 

the PACE policymakers. Additionally, many components of the PACE model are highly 

specified in the manual, such as how frequently data should be entered into the management 

information system. The specificity of many aspects of staff responsibilities seems to be 

reflected in the high OSC rigidity subscale score.

OSC morale scores reflect “employees’ satisfaction with their jobs and their commitment to the organization.”17 PACE centers’ scores clustered above the mean, indicating that 

morale at PACE is higher than the average from the national sample of mental health organizations. This finding aligns with the personal investment in PACE that staff members described 

during interviews and that researchers observed during site visits.

Summary

As the chapter shows, PACE’s intended program environment appears to be central to providing gender-responsive programming. Analysis of staff interview data found that the program 

environment provided across the study centers was similar, with staff members focusing on 

safety, building positive relationships with the girls, recognizing girls’ strengths, and being 

aware of the girls’ prior exposure to trauma. The program environment provides the internal 

context for PACE’s academic and social services, which are detailed in Chapters 5 and 6. 

 17Center for Behavioral Health Research (2016).

49

Chapter 5 

Academic Services and Life Skills

PACE’s academic services include academic advising, assessment, and middle school and high 

school classes. PACE also provides a gender-responsive life skills class called Spirited Girls!, as 

well as opportunities for volunteer service and career exploration. This chapter details how 

these services are provided at PACE centers and presents parents’ and girls’ perspectives of 

these services. The chapter concludes with data on the credits that girls earned while at PACE 

and findings on academic service receipt from the 12-month follow-up survey. Key findings are 

as follows: 

●

●

●

●

Although the content of academic classes was similar to what girls would 

learn in regular public school, academics at PACE differed in student-teacher 

ratio, availability of academic advising, and a focus on individual, self-paced 

work. Classes were also provided in the context of a gender-responsive program environment, as described in Chapter 4.

Evidence was mixed as to whether the quality of instruction was similar to or 

better than what the girls received in public schools. PACE classrooms were 

rated similarly to public school classrooms on the CLASS-Secondary, a validated tool for assessing the quality of classroom instruction. Although small 

class sizes offered an opportunity for teachers to work closely with girls, 

classroom observations found that many teachers offered limited academic 

support during independent work. Still, girls described the support they received from teachers as being superior to what they received in public 

school.

Spirited Girls!, PACE’s life skills curriculum, was offered at every center 

and was central to PACE’s gender-responsive programming. Spirited Girls!

was the main vehicle for delivering career readiness activities and played an 

important role in providing opportunities for girls to volunteer in the community.

On average, girls at PACE accrued credits at a rate that was on track with 

what is needed in Florida for on-time high school graduation. The program 

group was more likely to have been enrolled in school and to have received 

academic advising than the control group during the 12-month follow-up 

period.
50

● Staff members closely followed the PACE model for frequency of academic 

advising, assessment, and volunteer service. In areas where the model was 

less specific, such as the instructional approaches teachers should use and the 

types of career development activities centers should offer, there was more 

variation in the services girls received.

Introduction to the PACE Academic Services Model

The primary goal of academic services at PACE is to provide girls with the opportunity to earn 

credits and progress toward middle and high school completion. The PACE policy and procedures manual details what academic services each center should provide, including procedures 

for assessment, advising, course offerings, and instructional hours. The academic curriculum at 

PACE is driven by the Next Generation Sunshine State Standards, Florida’s modified version of 

the Common Core. All schools in Florida follow these standards; thus the content of the 

curriculum at PACE mirrors that of all public schools in Florida.

Although the curriculum and academic services are similar across centers, how centers 

provide those services varies. These differences are primarily driven by the contracts that 

individual PACE centers have with their local school districts. These contracts specify how 

much the centers are reimbursed per girl served, what support the district provides to the center, 

and whether PACE has the authority to grant high school diplomas. Table 5.1 provides an 

overview of the key differences in academic services across the centers.

The local school district also determines how girls will accrue credits while at PACE, 

which is one of the most significant differences between the centers in how academic services 

are structured. There are two models for earning credits while at PACE: programs based on seat

time and, more commonly, competency-based programs. In competency-based education 

programs, girls earn credits by demonstrating mastery of the subject matter, typically through 

completing a packet that includes worksheets, quizzes, and tests for the subject following the 

Sunshine State Standards. In seat-time programs, girls earn credits much as they would in a 

traditional school: They must spend a certain amount of time in class and pass the class requirements. Five centers had seat-time programs, and one center used seat time for middle 

school girls only. PACE managers described pros and cons of each approach. Competencybased education programs, in use at nine centers, were viewed as advantageous to girls for

catching up on credits: Girls who were motivated could finish classes faster than girls in seattime programs. On the other hand, seat-time programs were seen as providing girls with a 

structure more similar to regular school and thus better preparing girls for their eventual 

transition out of PACE.
51

Table 5.1

Academic Characteristics of PACE Centers

Characteristic Centers

a System for accruing credits

Competency

Seat time

8

5

Combination of seat time and competency 1

Able to grant high school diploma 6

Offers GED preparation 4

b Serves ESE students 14

Consultation-only services 4

Offers credit for Spirited Girls! classes 13

Academic staff turnover is a problem 11

Sample size 14

SOURCE: MDRC calculations based MDRC site visit data.

NOTES: GED = General Educational Development certificate. a

Competency programs award credits based on girls' mastery of 

the subject matter. Seat-time programs award credits based on girls' 

time in class and other class requirements.

bPACE uses the Florida Department of Education definition of 

Exceptional Student Education (ESE), referring to programs for 

students with disabilities and gifted programs. Centers that provide 

consultation-only services are restricted to serving students who 

have low-level or periodic needs.

Staffing of academic services was similar across centers. Teachers at PACE are certified and often have experience in the public school system before they come to PACE. 

Paraprofessionals support teachers and assist with testing at some of the larger centers, but for 

the most part teachers are alone in their classrooms. An academic services manager or program 

director oversees the teachers and academic program.

The average tenure at PACE for academic staff is shown in Table 5.2. Academic teachers (excluding those teaching Spirited Girls!) had an average tenure of less than three years. 
52

Table 5.2

Academic Staffing Characteristics at PACE Centers

Characteristic Mean a Sample Size

Class size 11.0 81

Academic advising caseload 17.1 53

Tenure of academic service manager (years) 3.6 14

Teaching staff tenure at PACE (years)

Spirited Girls! Teachers

Other than Spirited Girls! Teachers

4.6

2.9

9

66

Years of teaching experience 8.9 75

SOURCE: MDRC calculations based on responses to the PACE 

evaluation staff survey.

NOTE: aSample consists of program staff members who completed 

the survey; sample size varies according to staff position.

Interviews with management staff members at PACE revealed that staff turnover was greatest 

among teachers; 11 of 14 centers said that teacher turnover was an issue (Table 5.1). Staff 

members identified several reasons why they struggled to hire and retain teachers; competition 

with the school district was the primary reason. As a year-round program, PACE centers could 

not offer the same amount of time off to teachers as traditional schools. In addition, management staff members reported that girls at PACE could be more difficult to work with than the 

general student body in public schools, which also contributed to staff turnover.

Classes at PACE

When a girl enters PACE, her school transcripts and test scores are reviewed to determine her 

academic level and what credits she needs, and her academic schedule is created. Since girls 

enter the program at different points and need different credits, each girl at PACE will have a 

different schedule of classes. As a result, girls in the same classroom are often working on 

different subjects. For example, half of a social studies class may be working on world history 

while the other half is focusing on U.S. history. Due to rolling enrollment and the flexibility for 

girls to do self-paced work in competency-based programs, girls may be at different points in 

the curriculum even when they are working on the same subject in the same classroom. This 

means that the teachers must be able to support different levels and curricula within the same 
53

class period. This type of “mixed” class was less common in seat-time programs, because seattime classes needed a critical mass of girls to be working on the same subject at the same time

— a scheduling challenge in itself, academic staff members reported. 

Teachers described a number of complications related to the mixed nature of the classes. They noted that teaching multiple subjects at once requires familiarity with different 

curricula and each girl’s point of progress in the class. Mixed classes also limited the amount of 

full-group instruction that a teacher could provide during a class period. One tactic that teachers 

employed at many centers was to start the class with a “warm-up” lesson that involved the full 

class and was targeted to a subject that the whole class could benefit from before breaking up 

the class into smaller groups or independent work. Ultimately, the mixed nature of the classes 

meant that a large percentage of class time focused on independent work, with girls working on 

packets while the teachers provided individual support.

PACE classes were much smaller than classes in the public schools that many girls attended before PACE. PACE classes are generally capped at 12 students,

1 while Florida class 

sizes for high school approach 20 students per class.2 Teachers reported that classes usually had 

fewer than 12 students, which they unanimously said was a manageable size. In classroom 

observations during research visits, the average class size was just under seven. The smaller 

class sizes were due to student absences or girls being pulled from class for counseling or other 

activities.

The PACE manual offers minimal guidance on the instructional methods teachers 

should use in their classes, simply stating that instruction should be individually tailored to the 

girl and teachers should accommodate different learning styles. In interviews, teachers said they 

made the primary decisions about the instructional approaches they used in class, and they 

reported using a wide variety of approaches. Classroom observation supported these reports. 

The research team observed 80 teachers across 14 centers during site visits, typically for the 

length of a full class period (50 minutes).3 Researchers recorded the types of instructional 

methods teachers employed during the observations using four categories: teacher-led, group 

work, independent work, and other. During the observations, most teachers used two or more 

methods during the class period. Nearly two-thirds of classes included teacher-led instruction. 

About one-third included group work and about one-fifth included some other type of instruction, such as watching a video or playing a game. The overwhelming majority of classes 

 1

PACE policy and procedures manual. 2

Florida Department of Education (2014b). 3

For each center, the research team attempted to observe each full-time teacher (excluding substitutes) for 

one full class period. Four to seven teachers were observed at each center. Three observations were subsequently dropped from the analysis, because the usual classroom teacher was not present for the observation.
54

included individual work (84 percent), and 18 percent of class periods observed consisted of 

only independent work.

Findings from the CLASS-S Observations

During classroom observations, the team assessed interactions between teachers and 

students in the areas of emotional support, classroom organization, and instructional support 

using the Classroom Assessment Scoring System-Secondary (CLASS-S), a validated tool. Box

5.1 describes the CLASS-S domains of emotional support, classroom organization, and instructional support; Appendix B provides more details about the CLASS-S observations. During a 

50-minute session, researchers conducted two cycles of CLASS-S observations, with each cycle 

lasting approximately 25 minutes (15 minutes for observation and 10 minutes for coding). The 

researchers observed each teacher only once, rather than making repeat visits over a period of 

time, as is the procedure when CLASS-S is used for teacher performance evaluations; this limits

what conclusions can be drawn about the quality of individual teachers, but the average scores 

across PACE provide an indication of the quality of PACE teachers overall. The distribution of

scores by classroom is shown in Figure 5.1.4

Most PACE classrooms received midrange to high scores in the emotional support domain, which reinforces the finding in Chapter 4 that PACE centers strive to create a positive 

environment where staff members are sensitive to the needs of the girls. Most classrooms 

received scores in the mid- to high range in the positive climate and teacher sensitivity dimensions within this domain. However, PACE teachers scored lower on regard for adolescent 

perspectives, which reflects two findings: Girls were infrequently given leadership roles in the 

classroom, and teachers did not focus on connecting the content of the lessons to the girls’ lives. 

In an example of a class with high scores on regard for adolescent perspectives, the teacher 

involved the students in creating the rules for classroom behavior, offering them leadership 

opportunities and the chance to have input on the program. Classrooms with low scores tended 

to focus on independent work.

Nearly all PACE classrooms scored in the mid- to high range on all classroom organization dimensions, indicating that classrooms were orderly and productive, and behavior was 

generally not a problem. The high classroom organization scores are notable given that many 

girls have a history of behavioral issues when they come to PACE. Very few teachers received 

 4

For each dimension, the lowest possible score is 1 and the highest possible score is 7. Low, midrange, and 

high scores were assigned to each calculated score using the ranges described by the CLASS-S developers: 

Scores of 3 or below were classified as low and scores of 5 or higher were classified as high.
55

Box 5.1

Classroom Assessment Scoring System-Secondary (CLASS-S) Domains

Emotional support focuses on how the teacher supports students’ social and emotional 

functioning in the classroom. This includes classroom climate, the relationship between the 

teacher and students, the responsiveness of teachers to student needs, and how the teacher 

creates opportunities for student autonomy and leadership. This domain comprises the following dimensions: 

•

•

•

Positive climate

Teacher sensitivity

Regard for adolescent perspectives

Classroom organization reflects the organization and management of students’ behavior, 

time, and attention in the classroom. This domain comprises the following dimensions: 

•

•

•

Behavior management

Productivity

Negative climate

Instructional support assesses the quality of instruction provided and the strategies that 

teachers use to develop understanding of content among students. Attention is paid to the 

format of the lesson, teaching methods, and feedback provided to students. This domain 

comprises the following dimensions: 

•

•

•

•

•

Instructional learning formats

Content understanding

Analysis and inquiry

Quality of feedback

Instructional dialogue 

In addition, one dimension stands alone: 

• Student engagement

__________________________

SOURCE: Pianta, Hamre, and Mintz (2012). 
56

Figure 5.1

PACE CLASS-S Scores Across Classrooms, by Dimension

0

10

20

30

40

50

60

70

80

Positive climate Teacher sensitivity Regard for adolescent

perspectives

Number of clsasrooms 

Emotional support

Low

Mid

High

0

10

20

30

40

50

60

70

80

Instructional

learning formats

Content

understanding

Analysis and

inquiry

Quality of feedback Instructional

dialogue

Number of classrooms

Instructional support

0

10

20

30

40

50

60

70

80 Student engagementb

0

10

20

30

40

50

60

70

80

Behavior

management

Productivity Negative climate

Number of classrooms

Classroom organization

a

SOURCE: MDRC calculations based on observational assessments using the Classroom Assessment Scoring 

System (CLASS-S; Pianta, Hamre, and Mintz 2012).

NOTES: For each dimension, the lowest possible score is 1 and the highest possible score is 7. Each 

calculated score was assigned to a category: Scores of 3 or less were classified low; scores greater than 3 but 

less than 5 were classified midrange; scores 5 or more were classified high. aNegative climate scores are displayed here in reverse: Higher scores mean a less negative climate. 

bStudent engagement is considered a separate dimension outside the three domains.
57

low scores for behavior management or productivity; no teachers received low scores for 

negative climate.5 However, though disruptive behavior was rare, off-task behavior by girls was 

commonly observed by the researchers and infrequently redirected by the teachers. Teachers 

noted during interviews that they would let girls who were having a bad day put their heads 

down or spend time outside class as a way of being sensitive to their needs, which may explain 

some of the allowances for off-task behavior.

Most PACE classrooms scored in the mid- to low range in the instructional support 

domain, which is a common finding in other studies that have used the CLASS-S tool. The 

instructional support domain captures how teachers format lessons to engage students, the depth 

at which content is explored to support student learning and the development of higher-order 

thinking skills, and the quality of the feedback that students are provided on their work. PACE 

classrooms tended to score in the lower range in part due to the focus on independent work. The 

range of scores on the quality of feedback, with many classrooms scoring low and a few 

classrooms scoring high, reflect that teachers varied widely in their approach to supporting 

students during independent work. In classes with high quality of feedback scores, teachers 

consistently circulated among girls to provide support. They were aware of girls’ needs; 

sometimes, questions from a girl would prompt them to provide unplanned group instruction if 

it seemed like it would benefit others. In classes that scored low on quality of feedback, teachers 

gave responses that did little to advance girls’ understanding of the material, such as pointing 

them to a section of the textbook. There were also instances where teachers merely provided 

answers to the questions that the girls were struggling with, rather than helping the girls figure 

out the answer.

Most PACE classrooms scored in the midrange for student engagement, reflecting that 

girls’ participation in classroom activities was mixed. A low score indicates that the majority of 

the class is distracted or disengaged, while a high score indicates that most students are actively 

participating in the classroom discussions and activities. Although some observed classrooms 

fell at each end of the range, most were in the midrange. Some girls would be on task, while 

others were off task.

Analyses of the CLASS-S scores showed that they varied primarily across teachers 

within centers, with little variation in scores across centers. Further analyses (not shown) 

examined CLASS-S score differences by classroom or teacher characteristics, such as class 

size, instructional method, and teacher tenure, and found several statistically significant differences. Classes that used only independent work had lower scores on all dimensions within 

emotional support, as well as lower scores in instructional learning formats, content understand-

 5

Negative climate is reverse coded; a low score on negative climate would indicate the presence of a negative climate.
58

ing, and instructional dialogue. Longer teacher tenures were associated with higher scores in the 

classroom organization domain. Spirited Girls! classes scored higher on teacher sensitivity, 

regard for adolescent perspectives, and instructional dialogue, discussed further below.

Several published studies have used the CLASS-S tool to evaluate the quality of instruction in middle and high schools in the United States.6 Each of these studies used a different 

methodology to select and observe classrooms. However, when comparing the CLASS-S scores 

across the studies, PACE classrooms performed similarly. Like PACE, teachers in the other 

studies tended to have midrange scores on emotional support dimensions, midrange to high 

scores on Classroom Organization dimensions, and midrange to low scores on instructional 

support dimensions. As in PACE, the lowest scores were observed in the analysis and inquiry 

dimension.

PACE scores on the CLASS-S should be considered in the context of serving girls who 

were struggling in traditional schools, indicating that maintaining a productive learning environment may be more challenging, though PACE’s low staff-to-student ratio may help offset

those challenges. Additionally, the reality of girls entering PACE at different levels and with 

different class needs means that PACE is not able to form classes in which every student is 

working on the same lesson at the same time, as typically seen in traditional schools, and 

instead must rely more on independent work. The instructional support scores reflect a very 

different style of classroom organization and teaching at PACE from what is common in 

traditional schools.

Assessment and Advising

The amount of academic advising girls receive at PACE sets the program apart from the 

traditional school environment. Florida public schools are not required to provide school 

guidance counselors. One study from 2012 found that Florida schools’ average ratio was one

counselor for every 480 students.7 At PACE the ratio was closer to one academic adviser per 12 

students. PACE requires that girls receive advising on their academic progress every other week 

from their assigned academic adviser. This adviser is different from the girl’s assigned social 

services counselor (discussed in the next chapter) and is usually a teacher, though two centers 

had a separate guidance counselor on staff who served as the academic adviser. Advising 

meetings cover academic progress (grades, progress toward earning credits, test results).

 6

Donaldson, LeChasseur, and Mayer (2016). CLASS results from MyTeachingPartner-Secondary Study 

and Measure of Effective Teaching Study appear in the CLASS-Secondary manual (Pianta, Hamre, and Mintz

2012).

7

Todd (2014).
59

PACE also requires each girl to have an Individual Academic Plan (IAP), which includes goals in reading, math, writing, and career. The goals are often focused on skill gaps. 

PACE uses the STAR assessment to evaluate a girl’s reading and math needs, which then 

informs the IAP.8 Teachers said the STAR assessment was useful because it gives specific 

information on a girl’s knowledge and needs and can help a teacher understand whether a girl’s 

prior grades reflect her actual knowledge and ability. The STAR assessment may be administered periodically to check on progress, and it is administered before girls make the transition 

out of PACE.

According to the management information system data (shown in Table 5.3), the frequency of academic advising and assessment met PACE’s specifications. Among girls who 

enrolled in PACE, 93 percent took the STAR assessment and 96 percent had an IAP in place. 

Girls received academic advising on almost a biweekly basis, and advising sessions lasted, on 

average, about 13 minutes. Most girls who did not have an IAP or STAR assessment on file had 

short tenures (less than three months) at PACE.

Life Skills Training

As described in Chapter 4, key components of gender-responsive programming include providing education about women’s health issues, promoting career opportunities for successful 

transitions to adulthood, and providing connections to the community. This section discusses 

PACE’s approach to providing gender-responsive programming through life skills training.

Spirited Girls!

Spirited Girls! (SG), PACE’s life skills class, is a central component of its genderresponsive programming. Box 5.2 provides an overview of the six developmental domains it 

covers. The study found that SG was taught for the most part by teachers, supervised by the 

academic manager, and included in a girl’s schedule alongside her other classes. The schedule 

for SG varied at each center, but at minimum classes would meet weekly. At some centers, SG 

classes met daily. The PACE model requires that girls take two semesters of the class, and at all 

but one of the centers girls could receive credit for the class.

Until recently, PACE had used an SG curriculum developed in-house in the 1990s. 

Recognizing that the curriculum had become outdated, PACE recently began piloting the 

 8

The STAR assessment is widely used in K-12 education throughout Florida and the United States. See 

Renaissance Learning Inc. (2016).
60

Table 5.3

Participation and Satisfaction in Academic Services

Measure Program Group

Individual academic plan developed (%)

Received academic advising (%)

Number of sessions

Number of sessions per month enrolled

Time spent per session (minutes)

STAR assessment administereda(%)

b Credits earned by length of stay

3 months or less

More than 3 months to 6 months

More than 6 months to 10 months

More than 10 months to 15 months

More than 15 months

Participated in service learning projects (%)

Number of projects participated in

Number of projects participated in per quarter enrolled

Time spent on service projects (minutes)

c Among survey respondents (%)

Felt academic classes met needs

Felt that PACE staff helped with job or career planning

Felt Spirited Girls! taught her something new

96.1

93.5

17.0

1.8

13.1

92.8

0.4

1.7

4.0

7.3

10.1

73.6

6.8

1.9

88.5

88.8

82.1

78.9

Sample size 613

SOURCES: MDRC calculations based on data from the PACE management 

information system and responses to the 12-month follow-up survey.

NOTES: Calculations include all program group members who enrolled at PACE, 

except as noted. a

The STAR assessment (Renaissance Learning 2016) is widely used in K-12 

education.

bThis measure includes only girls who were no longer enrolled as of October 2016

and refers to a girl's cumulative length of stay (N = 556). c

Percentages are based on responses to the 12-month follow-up survey among 

program group members randomly assigned through March 2015 (N = 376).
61

evidence-based and gender-responsive Girls Circle curriculum, developed by the One Circle 

Foundation.9 Researchers visited two centers that were using the Girls Circle curriculum. The 

remaining centers were using SG, and teachers explained that they would tailor the original SG 

curriculum to current topics, drawing on resources from outside the curriculum.10 For example, 

the SG curriculum does not cover the topic of online bullying, so the teachers would bring in 

outside resources or news stories to cover that topic.

SG classrooms were different from other academic classes. Unlike in many of the academic classes, all girls in SG receive the same lesson at the same time.11 This gave the teachers 

more opportunities to lead full-group lessons or engage the girls in group work. Indeed, SG 

classes had the highest frequency of teacher-led instruction and group work of all subjects 

observed by researchers. SG classes also showed statistically significant differences from other 

academic classes on the CLASS-S scores, scoring higher in teacher sensitivity, regard for 

adolescent perspectives, and instructional dialogue. These differences reflect the contentoriented discussions about real-life issues that were more prevalent in SG classrooms than in

regular classrooms.

 9

One Circle Foundation (2012). The Girls Circle curriculum was first developed in 1994. 10At the time of publication of this report, all PACE centers were implementing Girls Circle as part of 

Spirited Girls! class. 11Since enrollment at PACE is rolling, not all girls in the SG classes would have been exposed to the same 

parts of the curriculum.

Box 5.2

Spirited Girls! Curriculum: Content by Developmental Domain

Physical. Healthy eating, drugs and alcohol, physical fitness.

Sexual. Menstruation, sexual activity, sexual transmitted infections, contraception.

Intellectual. Basic computer and test-taking skills, understanding grade point average and 

credit requirements, career exploration, job readiness.

Emotional. Self-esteem, anger and stress management, coping skills, recognizing feelings.

Relational. Healthy relationships, friendships, romantic relationships, family relationships.

Spiritual. Topics are secular and aim to help girls think about their place in the world and their 

own approach to spirituality.

__________________________

SOURCE: PACE Spirited Girls! curriculum.
62

Career Exploration and Volunteer Service

The PACE model includes a career component, which focuses on educating girls about 

possible career options and strengthening academic and soft skills to support work readiness. 

Upon entering PACE, girls take an assessment to identify career interests, and these become 

part of their IAP. The PACE model also requires each girl to participate in a volunteer service 

project each semester. SG was the primary vehicle to expose girls to career options and work 

readiness activities; activities in SG included résumé writing, mock interviewing, and filling out 

job applications. Three centers had a separate career class apart from SG.

Career development activities outside SG varied among the centers; though a few centers provided robust career activities outside SG, such as internships or job shadowing, they did 

not seem to involve many girls. The ad hoc nature of such career development activities could 

be attributed to the lack of dedicated staffing and reliance on external partnerships to deliver the 

services. Also, since most girls return to middle or high school (or attend another alternative 

school) after PACE, careers may be a lower priority for centers than getting girls ready to go 

back to regular school.

Girls’ participation in volunteer service projects is seen by PACE as promoting selfesteem, building work readiness skills, and contributing to the community. Most centers 

described providing volunteer service activities on a monthly basis. These were also typically 

delivered through the SG class or through another class or group. Activities included on-site 

activities, such as letter writing or putting together care packages for the homeless, and off-site 

activities, such as volunteering at nursing homes. Staff members said the off-site activities were

more engaging for the girls but more challenging to implement for a variety of logistical and 

behavioral reasons. Some centers used participation in off-site community service projects as an 

incentive, so not all girls were eligible to participate in all projects. As shown in Table 5.3, 

about three-quarters of girls participated in projects, with a frequency of two projects per quarter 

enrolled. This level of participation in community service corresponded to PACE requirements.

Girls’ and Parents’ Reflections on Academic Services

Overall, girls and parents had positive experiences with academic services at PACE. As shown 

in Table 5.3, 89 percent of program group girls surveyed felt the academic classes met their 

needs. Eighty-two percent of girls felt that staff members were helpful with job or career 

planning, and most girls felt that they learned something new from SG. Box 5.3 presents one 

girl’s story about the benefits of PACE’s academic services and SG.

In interviews, both while they were still in the program and after they had left, girls had 

mostly positive things to say about academic services. Most commonly, girls praised their 
63

Box 5.3

One Girl’s Perspective on Classes at PACE

Brittany: Appreciated academic assistance and Spirited Girls!

A guidance counselor at her previous high school referred Brittany* to PACE because she had 

fallen behind in credits. She was working toward her high school diploma and planned to 

graduate from PACE.

Brittany emphasized the benefits of the engaged teachers and small class sizes she has experienced at PACE. “The classes are small . . . they don’t have a lot of students, so it’s better 

because they are able to focus on the students more, instead of public school where they have a 

whole lot of kids so they don’t have enough time to help you.” Brittany appreciated the way 

teachers interacted with the students: “The teachers are very helpful, and they do like to help 

the students, and they like their job.”

Brittany also said she was more focused on her academic classes at PACE and therefore able 

to complete more work. She compared this with public school, where she was “busy listening 

to [her] friends,” which distracted her from her school work. The one drawback of PACE was 

that the classes offered were “basic,” meaning that PACE does not offer electives like foreign 

language or physical education as regular public schools do.

Brittany also said that as a pregnant young woman, she benefited from the knowledge she 

gained in Spirited Girls! She enjoyed learning about her body, about health, and about how to 

care for herself and her baby.

*Name has been changed. 

teachers and found them to be helpful. Some attributed teachers’ helpfulness to the small class 

size. One girl explained: “Because there weren’t a lot of students, you could have one-on-one 

time with the teacher and they can explain stuff to you more.” The small class also meant that 

there were fewer distractions. Some girls remarked that they felt the teachers were more 

invested in them, in line with the relational focus of the program environment described in 

Chapter 4. One girl put it this way: “I guess I felt like they were more than just a teacher; they 

actually cared about you.”

A minority of girls interviewed did not like the academic program. For some, the selfpaced nature of the classes did not work for their learning style; they found the independent 

work boring and would have liked more teacher-led instruction. One girl said, “I just don’t like 

how I have to do the work on my own. When I do my work, I don’t really read it; I just skim 

and skim. I don’t learn nothing.”
64

Girls were also generally positive about the SG class. Among girls interviewed, four 

times as many girls said they liked SG as said they disliked it. Girls who liked SG mentioned 

enjoying the conversations in class, both the variety of topics covered and the relevance of the

topics. One girl described it this way: “You learn about yourself as a girl. Your body. Some of 

the activities we do make you look at yourself differently, and your relationships differently. 

You get more in tune with yourself.” The minority of girls who did not like SG expressed 

discomfort with the topics discussed in SG and with group discussions about those topics.

Girls interviewed described receiving a variety of career services from PACE. Services

ranged from general career readiness activities, such as mock interviewing and career exploration, to more in-depth college and career planning services around specific career tracks. Girls 

also offered positive reflections on their participation in community service activities; they 

appreciated the opportunity to help others and thought the activities were fun.

Parents and guardians interviewed had mostly favorable things to say about PACE’s 

academic services. In fact, most parents said that it was the academics that drew them to PACE 

in the first place. Like the girls, parents liked the small class sizes, which they viewed as 

providing more individual attention for the girls. Parents also liked the self-paced nature of the 

academic program and felt that it was a good opportunity for girls to catch up on credits.

Academic Progress

Catching up on credits was the primary academic goal for most girls at PACE. As described in 

Chapter 3, more than half the girls entering PACE had been held back in school at least once, 

and three-quarters had recently failed a class. PACE offers the opportunity for girls to earn those 

credits, and in centers with competency programs, staff members said that girls who were 

motivated could get back to grade level faster than they could in regular school.

Table 5.3 shows the number of credits earned by girls who enrolled in PACE.12 As expected, the longer a girl was enrolled in PACE, the more credits she received. In Florida, 

students must earn 24 credits in high school to earn a standard diploma.13 A yearlong course, 

such as Algebra I, would earn a student one credit. The PACE credit data suggest that girls on 

average earned credits at a rate similar to what they would in high school; students who were in 

PACE for more than six months but less than a year (the equivalent of one or two semesters) 

earned four credits. Students progressing normally in regular high school earn about six credits 

in two semesters. It is not possible to make direct comparisons between regular school semes-

 12This measure includes only girls who were no longer enrolled as of October 2016 and refers to a girl’s 

cumulative length of stay; 57 girls in the study were still enrolled as of October 2016. 13Florida Department of Education (2014a).
65

ters and credits earned at PACE by length of stay. However, considering that most girls in 

PACE were not accruing credits at the rate needed to graduate on time before they came to 

PACE, these data suggest that PACE helped girls catch up on their credits. The final report will 

include an analysis of school records data to estimate PACE’s impacts on school progress for 

study participants.

Services Received

Table 5.4 presents preliminary impacts on academic services received. These measures come 

from the 12-month follow-up survey and represent only a partial sample, because the 12-month 

Table 5.4

One-Year Impacts on Academic Services Received

Outcome (%)

Program Control Difference

Group Group (Impact) P-Value

Ever enrolled in a school or educational program in past year

Received academic advising in past year

a Primary source of academic advising

Parent, guardian, or other relative

Someone from school 

Someone from PACE

Other

Frequency of academic advising sessions

More than once per month 

Once per month 

1-3 times per year

Never

99.1

81.1

8.6

8.5

82.1

0.8

38.0

19.6

23.4

19.0

92.9

67.6

44.1

44.9

2.2

8.9

33.8

11.5

22.4

32.4

6.2 ***

13.5 ***

-35.5

-36.4

79.9

-8.1

4.2

8.2 ***

1.0

-13.4 ***

0.000

0.000

0.293

0.009

0.770

0.000

Sample sizeb (total = 668) 407 261

SOURCE: MDRC calculations based on responses to the PACE evaluation 12-month follow-up survey.

NOTES: Results in this table are regression-adjusted, controlling for pre-random assignment characteristics. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. aResponses include only sample members who reported receiving academic advising. Therefore, the 

measures are nonexperimental and statistical significance was not calculated.

bDue to missing values, the number of girls included varies by outcome. The sample size reported 

here is based on responses to the 12-month follow-up survey among girls randomly assigned through 

March 2015. 
66

follow-up period was not complete for the full sample at the time this report was written. 

Overall, findings are promising. Nearly all the girls in the study sample enrolled in a school or 

educational program during the follow-up period. The program group was statistically more 

likely to have been enrolled in school than the control group (99.1 percent versus 92.9 percent). 

The majority of girls in the control group enrolled in a traditional middle or high school, but 

many girls enrolled in other types of schools, including charter, alternative, and virtual schools, 

reflecting the wide array of academic options available in Florida.

Girls in the program group were 13.5 percentage points more likely to have received 

advising. They received this service more frequently, and more often from a professional — 

either someone at PACE or another school. The control group was much more likely than the 

program group to report receiving academic advising from a family member. This finding 

reinforces the indications discussed above that PACE provides more academic advising than 

traditional schools, where many in the control group were served.
67

Chapter 6 

Social Services

Through social services, PACE strives to address the nonacademic needs that may hinder a 

girl’s success in the program and beyond. Social services include case management and 

counseling, parental engagement, connections to supportive services, support for transitions, 

and follow-up contact. This chapter describes PACE’s social service provision, the experiences 

of girls and parents with those services, and the difference in social service receipt between 

program and control groups as reported in the 12-month follow-up survey. The key findings on 

social services are as follows: 

●

●

●

●

PACE’s comprehensive social services set the program apart from most public schools. Girls in the program group were more likely than girls in the control group to have received counseling and case management during the 12-

month follow-up period.

PACE centers closely followed the prescribed PACE model for the frequency of assessment, counseling, and parental engagement while girls were in 

the program. PACE does not specify the approach counselors should take 

with girls during their meetings, so the content of these sessions varied considerably.

Once girls left PACE, follow-up services were limited. Transition and follow-up services tended to be understaffed, which helps explain why implementation of these services was more uneven.

Girls and parents reflected mostly positively on the social services they received while at PACE, describing the counselors as helpful and supportive. 

Girls were less satisfied with follow-up services.

Social Services Structure and Staffing

The goal of social services is to provide each girl with the support she needs to be successful in 

academics and build the foundation for success in the future. As one executive director explained: “Social services give girls stability as they go about academics. They are able to focus 

on themselves for the first time at PACE.” Social services are where the differences between 

PACE and regular public school are most apparent — PACE offers far more support outside the 

classroom. As a point of comparison, one study found that the average ratio of social workers to 
68

students in Florida was greater than 1 to 2,500.1 As shown in Table 6.1, the ratio at PACE was 1 

to 17.

Typically, a social service manager oversaw a team of three to five counselors in a few 

types of positions. Outreach counselors focused on recruitment and intake activities and had 

limited interaction with girls once they were enrolled in the program.2 Day counselors had 

primary responsibility for girls enrolled in the program. Transition counselors typically became 

involved with a girl’s care as she neared the end of her time at PACE and were also responsible 

for follow-up services. Table 6.1 presents the characteristics of social service staff members. 

Counselors were at both the bachelor’s and master’s levels. Counselors had an average caseload 

size of about 17 girls. In the staff survey, about 10 percent of day counselors said that their 

caseload was not manageable; 30 percent of survey respondents indicated that their caseload 

was very or extremely manageable. Since counselors are not clinical staff, they are not required 

to have a license, and few reported that they were licensed. As Table 6.2 shows, a small number 

of centers had additional social services capacity, including staff therapists and nurses. Most 

centers also had interns on staff, and interns might carry a caseload at some centers. Unlike the 

situation with teachers, no managers reported that counselor turnover was an issue.

Assessment and Care Planning

PACE evaluates the needs of each girl enrolled through a comprehensive assessment process, 

which is documented in PACE’s management information system (MIS). The process starts 

with the Initial Needs Assessment (INA), described in Chapter 3, which helps determine 

whether the girl is a good fit for the program. The On-going Needs Assessment (ONA) is 

administered within the first 30 days of a girl’s enrollment in the program and provides a more 

detailed assessment of the girl’s history and her current needs.

The ONA is organized into six domains: relational, intellectual, physical, sexual, emotional, and spiritual. The ONA is primarily administered through interviews between the girl

and her assigned counselor, but the counselor can also involve other sources in completing the 

ONA, including parents, outside case managers or counselors, and a review of the girl’s files

from previous schools or other providers. The ONA probes into sensitive areas, such as a girl’s 

sexual and abuse history. Counselors described tailoring the administration of the ONA to each 

girl: With some girls the ONA could be administered in one sitting; with others the counselors 

 1

Student Support Services Project (2012). 

2

Most centers added this position during the study period. Some centers eliminated this position after study 

enrollment concluded.
69

Table 6.1

Characteristics of PACE Counselors

Characteristic Value Sample Size

Average tenure at PACE (years) 3.7 66

a Average years of experience 7.3 66

Percentage with master's degree 62.1 66

Percentage with bachelor's degree 37.9 66

b Percentage with license 9.4 53

Average counselor caseloadc 17.1 53

SOURCE: MDRC calculations based on responses to the PACE evaluation 

staff survey.

NOTES: Values are based on responses to the PACE evaluation staff survey 

and include counselors, transition counselors, and outreach/intake counselors. aMeasure includes experience during practicum and internships.

bA license is not required for counseling positions at PACE. The sample 

size is smaller than for questions above because this question was added to 

the survey after staff members at three centers had completed it. cSample includes only social service staff members who carry a caseload 

of girls for counseling.

Table 6.2

Social Service Staffing Characteristics at PACE Centers

Characteristic Centers

Full-time transition counselor on staff 5

Full-time therapist on staff 

a On-site nurse

. 4

2

Interns on counseling staff

Interns carry a caseload

13

9

Counseling staff turnover is a problem 0

Sample size 14

SOURCE: MDRC calculations based on data collected during site 

visits.

NOTE: aMeasure includes nurses provided by an outside partner.
70

would break it up over several meetings and build trust and rapport with the girl before asking 

the most sensitive questions. PACE requires that the physical domain of the ONA be completed 

on a girl’s first day in order to determine whether a girl has immediate medical, dental, or 

mental health needs.

Counselors used the information gathered in the ONA to create each girl’s care plan. 

Care plans are organized along the same domains as the ONA and focus on the specific goals 

that a girl will work on while at PACE. As shown in Box 6.1, care plans included up to six 

goals, aimed at both reducing risk factors and building strengths. Counselors used these goals to 

guide their sessions with each girl and to update parents on her progress during monthly 

meetings. Care plans were not directly used by noncounseling staff, but the counselors used the 

care plan and goals to inform discussions with other staff members about a girl’s care.

Each girl’s progress is required to be reviewed at least monthly during a meeting of the 

care team, which includes the girl’s assigned counselor, academic adviser, and at least one 

manager. Typically, PACE centers held care review meetings every week or every other week. 

The researchers observed that these meetings were attended by most staff members — counselors, teachers, and managers. During these meetings, counselors discussed a girl’s progress 

toward her care plan goals and shared updates on her issues or needs, and teachers shared 

information on her progress in class. Staff members discussed any steps that needed to be taken 

to help the girl reach her goals, such as addressing problems with attendance or behavior. For 

girls who were progressing well, the team would discuss ways to reward or acknowledge her 

performance and keep her on track. Care review was the formal way PACE staff members 

shared information about girls. Outside these meetings, staff members shared emergent issues 

about the girls through morning staff meetings, emails, and ad hoc meetings.

The MIS data indicate that centers were closely following PACE requirements for administering assessments and creating and monitoring care plans. As shown in Table 6.3, 96 

percent of girls had their ONA administered within 30 days of enrollment, and 98 percent had 

their care plan developed within 30 days. Girls who did not have an ONA or a care plan tended 

to be those who left the program shortly after enrollment.

Individual Counseling

Based on the PACE policy and procedures manual, a girl should meet with her assigned 

counselor every two weeks to review her progress. According to MIS data (see Table 6.3), 

meetings between counselors and girls happened more frequently than the recommended twice 

per month. The average time per session was 39 minutes. Counselors reported that rather than 
71

Box 6.1

Example of Care Plan Goals

Decrease substance abuse Increase anger management skills

Decrease self-harm Increase positive self-esteem

Decrease social isolation Increase healthy relationships

__________________________

SOURCE: PACE’s management information system.

following a set schedule for the meetings, they would pull the girls from class at a time that was 

convenient for the counselor and the girl. For example, counselors would avoid pulling girls 

from classes in which they were behind on their work.

Although the primary purpose of these biweekly meetings is to review the girl’s progress toward the care plan goals, the research team found that counselors used a broad range of 

approaches in the meetings. Most counselors described the care plan goals as a jumping-off 

point for discussions with the girls. Interviews with counselors revealed that there was a fine 

line between discussing goals with a girl and delving into sensitive topics that might require a 

more therapeutic approach. A quarter of counselors interviewed said they did not use clinical 

approaches in their work with girls, with some of those counselors stating that they were not 

qualified to provide clinical therapy. The majority of counselors, however, described bringing 

aspects of clinical practice into their session with the girls; the most common approaches 

mentioned were cognitive-behavioral therapy, solutions-focused therapy, and art therapy. 

Interview data suggest that the nature of topics girls discussed with their counselors often led 

counselors into playing a more therapeutic role. For example, a girl might have the goal of 

better handling her anger, which might stem from abuse by a family member. Helping the girl 

find a strategy for responding in upsetting circumstances could require addressing the underlying trauma or depression.

Counselors used a variety of strategies in their work with the girls; most counselors said

they tailored the approach to a girl’s individual needs and personality. For example, a counselor 

might use hands-on activities like worksheets or art projects with girls who were having trouble 

opening up to the counselor. A review of counselor case notes entered into PACE’s MIS system 

provides examples of the range of content these sessions could have. One type of session 

focused more on helping a girl support her academic progress or address behavior issues within 

the program. As an example of this type of session from the case notes: “[Counselor] reviewed 
72

Table 6.3

Counseling and Parental Involvement at PACE Centers

Service Program Group

Ongoing needs assessment (ONA) ever administereda

 (%) 94.0

ONA administered within 30 days of enrollment

b ONA updated every 6 months

96.2

73.7

Physical domain assessment completed at enrollment (%) 95.9

Care plan ever developed (%) 96.9

If yes, initial care plan developed within 30 days 97.8

Girl reviewed at care review staff meeting (%) 99.5

Initial care review staff meeting held within 1 week (%) 88.5

Number of times reviewed at a care review staff meeting 12.5

Girl received individual counselingc

 (%) 97.4

Number of sessions 21.9

Number of sessions per month enrolled 2.5

Time per session (minutes) 38.7

Girl attended psychoeducational or extracurricular groupd (%) 85.2

Number of group sessions 18.4

Number of group sessions per month enrolled 1.8

Time per group session (minutes) 59.5

Parent/guardian ever contactede (%) 97.2

Number of parent/guardian contacts attempted over course of enrollment 18.7

Number of parent/guardian contacts attempted per month enrolled 2.5

Sample size 613

SOURCE: MDRC calculations based on data from the PACE management information system.

NOTES: Calculations include all program group members who enrolled in PACE. aMeasures are based on ONAs administered only during a girl's first enrollment. 

bMeasure includes only girls who were enrolled for at least six months.

cIf a girl received more than one counseling session in a given day, only the most recently 

entered session of that day was accounted for in these calculations.

dIf a girl attended more than one group session on a given day, only the most recently entered 

group session of that day was accounted for in these calculations. eIf there was more than one parent/guardian contact on a given day, only the most recently 

entered instance of contact was accounted for in these calculations. Initial home visits could also 

be captured in these measures.
73

girl’s progress and addressed some concerns about girl’s academic performance because she is 

not showing as much progress as she could. She blames peers in the class for distracting her by 

their behavior. We discussed some options available to her in the classroom to separate herself 

and focus on her own academic success.” Another type of session included a focus on relational 

issues in the girl’s life, for example: “Girl reports feeling very anxious about a friend’s family 

situation. Girl reports that she is worried about this friend and cannot stop worrying about her. 

Discussed with girl the importance of maintaining personal awareness of how others’ issues 

impact her and the need to ‘fix’ other people’s issues.” Often, sessions would cover more than 

one topic.

Girls’ Perspectives on Counseling

Most girls interviewed had positive things to say about the counseling they received at 

PACE. They described talking with their counselor about their goals and the issues they faced at 

school and at home, and most girls said that their sessions with their counselor were helpful. 

Girls reported that the counselors were both problem-solvers who helped them with strategies to 

address the challenges in their lives and sources of emotional support, providing them with 

encouragement and reassurance. One girl said: “I feel like sometimes I’m going to break down. 

She believes in me. She tells me I can do it.” A minority of girls interviewed (less than 20 

percent) said that they did not open up to their counselors, either because they did not feel that 

they connected with the counselor or because they did not open up to anyone. Box 6.2 provides 

two different perspectives on counseling.

Case Management and Ancillary Services

Counselors also functioned as case managers, working with girls to identify unmet 

health or other needs and connect them with appropriate services, either through PACE or 

through partners. All PACE centers had informal partnerships with outside providers to connect 

girls to ancillary services. The most common referrals to outside services were for health 

services, such as reproductive or dental care, or for mental health services, when a girl needed 

services beyond what the center could provide. Counselors described several challenges with 

connecting girls with services outside PACE. One challenge was the availability of services in 

the community. Services available to the girls, such as free counseling, often had long waiting 

lists or were perceived to be of low quality. Another challenge with outside referrals involved

the parents — referrals often required the parent to give approval or to schedule the appointment 

and arrange transportation. Lack of parental follow-through could mean that the girls were not 

able to receive the services they were referred to.
74

Box 6.2

Two Perspectives on Counseling

A few centers were able to provide health and mental health services on site. Four centers had therapists on staff to work with girls who had clinical or more intensive needs. Two 

centers had a nurse on site, provided through a contract with an outside provider. These centers 

Kendyll: Counseling was helpful

“It was okay to be me and to keep being who I am and not change for nobody.”

This was among the key lessons that Kendyll* learned over the eight months she spent at a 

PACE center in central Florida.

Kendyll began at PACE when she was 17. The center proved to be a safe, secure, and supportive environment for her. She was able to make close connections to staff members as well as to 

some other girls at the center.

In particular, Kendyll identified counseling as the most helpful component of PACE. She was 

struggling with depression and anxiety when she arrived, and though she continues to experience those issues, she said that they were “not as bad” now, an improvement which she 

attributes to the work she did with her counselor. She and her counselor talked about how to 

handle herself when she became upset. Her counselor also gave Kendyll small projects. For 

one of them, Kendyll wrote down her good qualities and kept the list in her binder so that she 

could easily remind herself of them whenever she got upset. Kendyll appreciated that her 

counselor and other PACE staff members took her depression seriously and made dealing with 

it a priority. She found it easy to talk to PACE staff members about her goals and problems 

because they would try to help her, rather than judge her.

Roxane: PACE was not the right fit

Although Roxane* spent about eight months at a PACE center in central Florida, she ultimately did not find the program to be a good fit for her and does not believe that her experience at 

PACE benefited her.

Roxane, who was 15 when she started at PACE, struggled most with how much she was 

expected to share her feelings there. She said that her counselor was nice and sometimes 

helpful, but she did not have the right type of attitude to respond to counseling because she did 

not want to talk about her feelings. Roxane was uncomfortable with discussing issues in her 

life or her goals with PACE staff members, though she said that she did talk with them about 

problems she had with friends and her grandfather, with whom she lives. Similarly, Roxane 

strongly disliked Spirited Girls! because the class also asked her to talk about her feelings.

*Names have been changed.
75

reported that having these additional services on site not only lowered the barriers to services

but also improved access to providers with experience serving the PACE population.

Counselors also provided support to girls who needed it outside their biweekly sessions. 

Girls who were having emergent issues, such as a crisis at home, could request a same-day

meeting with a counselor — either with their regular counselor, depending on availability, or a 

counselor-on-call. Counselors might also be called by a teacher to meet with a girl who was

having behavior problems in the classroom. Counselors reported meeting daily, if needed, with 

girls who were in crisis.

Group Counseling

PACE centers provide psychoeducational or therapeutic groups to girls who need them. The 

PACE manual stipulates that participation in therapeutic groups is voluntary. Centers determined which groups to offer based on the needs of the girls enrolled. Therapeutic group topics 

included anger management, grief and loss, and sexual abuse. Some centers offered nontherapeutic groups that were more like extracurricular clubs offered in regular high school, such as 

gardening or book clubs. A few centers did not have any groups operating at the time of the 

implementation visits. Staffing for groups varied quite a bit — from counselors to interns to 

outside providers.

Staff members described groups that generally met weekly for about an hour for 8 to 10 

weeks. Some centers had a specific day and time in their weekly schedule when girls would 

attend groups, with some girls attending therapeutic groups and other girls attending nontherapeutic groups. Centers that had groups built into their weekly schedule required all girls to be 

enrolled in a group, though girls could choose a group that interested them. As shown in Table 

6.3, 85 percent of girls participated in groups, attending an average of 18 sessions. The data do 

not differentiate between therapeutic groups and nontherapeutic groups; thus the high participation rate is likely to be related to the broad range of groups offered and the requirement at some 

centers that all girls participate. Interviews with girls supported the rates calculated using the 

MIS; 80 percent of girls reported participating in a group. Many girls said these groups were

required, and their description of the groups emphasized extracurricular-type activities more 

than therapeutic ones. A few girls reported attending therapeutic groups at the recommendation 

of their counselors. 

Parental Engagement

Parental engagement is a central component of the PACE model and gender-responsive 

programming more broadly. Parental engagement is built into the program model in several 

ways. First, counselors are required to conduct a home visit within 30 days of a girl starting the 
76

program. The purpose of this visit is to assess the home environment, such as sleeping arrangements and the availability of food. Counselors then meet monthly, ideally face to face, with a 

parent to provide an update on the girl’s progress. Counselors must also make a home visit if a 

girl has three unexcused absences, and staff members check in with parents on an ad hoc basis if 

there are issues to be discussed, including positive updates on a girl’s progress.

According to the MIS data (Table 6.3), counselors attempted to contact parents about 

two and a half times per month on average. Due to limitations in how the data on contacts were 

documented in PACE’s MIS system, it is not possible to determine how many of these contacts 

were successful. Staff surveys and interviews with counselors indicate that, in fact, many of the 

attempted contacts were not successful. The majority of staff survey respondents (84 percent) 

said that no more than half the parents were actively engaged in supporting their daughter’s 

goals. During site visits, counselors also described difficulties reaching and engaging parents, 

noting that parents’ phone numbers would change or they would not respond to phone calls. In 

addition, counselors said, it was difficult to schedule meetings with parents around their work 

schedules. Counselors reported that some parents were part of the daughter’s struggles, but the 

parents did not want to see or acknowledge their role.

Some staff members described engagement strategies to improve parental involvement, 

such as helping meet the parents’ needs by connecting them to resources for food stamps or 

counseling. Another tactic was reaching out to parents when things were going well, not just 

when there was an issue, which was an element of PACE’s strengths-based approach. These 

positive reports could be a new experience for many parents accustomed to hearing from school 

only when their daughter was in trouble. Counselors described being flexible with meeting 

locations, such as going to a parent’s workplace to meet if the parent’s work schedule made it 

difficult to come to the center. Some centers tried to involve parents in special events or as 

volunteers at the center to encourage their connection to PACE. The PACE manual states that 

staff members should make “every attempt” to meet with parents face to face, but counselors 

provided varying descriptions of how far they would go to involve parents. About half described aggressive efforts to involve parents, such as going to their houses and workplaces to 

track them down.

In interviews with the research team, parents described positive interactions with the 

staff at PACE.3 Most parents said that the counselors did a good job of keeping them up to date 

on their girls’ progress and appreciated that the counselors would work around their schedules 

for meetings. Parents said that PACE initiated most of the interaction with the parents, and 

 3

Parents who participated in the interviews cannot be considered a representative group of parents. Parents 

interviewed were those who answered the phone when the interviewer called to survey the girl and expressed 

interest in being interviewed by the researchers.
77

some were surprised when PACE would call them to check on a girl’s attendance or share 

something about her progress. The minority of parents who had less positive things to say about 

their interactions with PACE found the home visits intrusive or felt that the counselor should 

have shared more information, such as telling the parent what was discussed in counseling 

sessions. To illustrate some of the tensions counselors may experience in working with parents, 

one parent interviewed told of being upset with the counselor because the counselor reported 

her to the police for abusing her daughter.

Transitions

PACE is not intended to be a girl’s final step on an academic or career pathway. Most commonly, the girl returns to her zoned school or enrolls in an alternative or charter school in the school 

district. There are two main limits on a girl’s stay at PACE. First, Florida Department of 

Juvenile Justice funding caps a girl’s time at PACE at 15 months, granting an extension in

limited circumstances. Second, only six centers in the study were authorized by the local school 

district to grant high school diplomas; in the remaining centers girls had to enroll in a different 

school in order to graduate. Many girls made the transition in less than 15 months, depending on 

their academic progress and progress on care plan goals. Unplanned transitions — in which a 

girl drops out of the program or is asked to leave — were also common; two-thirds of transition

staff members at centers said that unplanned transitions happened at least monthly. Table 6.4 

shows girls’ survey responses to why they left PACE without completing the program. Most 

commonly, girls gave reasons that indicated that the program was not a good fit for them, such 

as not liking the program, not getting along with other girls there, or being asked to leave by 

PACE. Some girls selected multiple reasons for leaving the program.

PACE’s motto on transitions is “transition begins at intake”; planning for transition is a 

key part of the model. Girls are given a planned transition date when they start at PACE, and the 

eventual transition is supposed to be part of the care planning process and monthly meetings 

with parents. Interview data indicated that most planning for transition fell within 90 days of the 

transition date. Counselors had the primary role in planning for transitions for the girls on their 

caseload.

Counselors described two main ways they supported the transition process. Most commonly, they described working closely with the girl and her family to determine the best school 

placement, including helping the family with the enrollment paperwork. Counselors also 

worked to prepare the girl emotionally for leaving PACE. Some centers, often those with 

dedicated transition staffing, did this through a formal transitions group attended by girls 

nearing that point. Other counselors did this one on one by talking with the girl about her 

feelings about transition during regular biweekly meetings and working with her on coping 

strategies.
78

Table 6.4

Transition Services at PACE Centers

Service (%) Program Group

Transition planning before exit

Transition outcome checklist completed

Exit plan completed

Transition care plan developeda

Among survey respondents

Girl felt transition planning was helpful after leaving PACEb

c Reason for leaving PACE before completing the program

Did not like the program

Did not like or get along with girls

Was asked to leave or transition

Did not like or get along with staff

Moved

Did not have transportation

Parent or guardian did not like the program

Expecting child or had child care problems

Had health problems or an injury

Incarcerated

Family member became ill

d Other

93.9

95.7

58.1

59.2

27.8

31.7

22.2

10.6

12.8

11.1

9.4

6.7

6.1

4.4

1.1

29.4

Sample size 578

SOURCES: MDRC calculations based on data from the PACE management information 

system and the PACE evaluation 12-month follow-up survey.

NOTES: Calculations include all program group members who have made the transition 

out of PACE, except as noted. aCalculations include only girls who were enrolled for at least 30 days and are based 

on a girl's first transition period (N = 523). PACE is not required to create a transition 

care plan or provide transition services to girls who were enrolled for fewer than 30 

days.

bThis measure is based on responses to the 12-month follow-up survey among 

respondents randomly assigned through March 2015 who attended PACE (N = 218). cThese measures are based on responses to the 12-month follow-up survey among 

respondents randomly assigned through March 2015 who left PACE but did not 

graduate or complete the program (N = 180). Respondents could select more than one 

reason.

dExamples of "other" responses include feeling the program was not helping, 

wanting to return to a prior school or go to a different school, having trouble with 

attendance, and finding academics at PACE inadequately rigorous. 
79

In the final weeks before transition, the counselor is to complete the transition outcome 

checklist, which summarizes the girl’s progress while at PACE, and create an exit plan. The exit 

plan has information about the girl’s next placement and is used to set academic and personal 

goals for her during the transition period. As shown in the top panel of Table 6.4, there was high 

compliance in completing the required paperwork before transition, with nearly all girls having 

complete transition paperwork. During interviews, counselors explained that in the case of 

unplanned transitions where there is not much warning, the counselor can complete the paperwork without the involvement of the family. The high compliance rates reflect such flexibility.

Girls interviewed described varying transition experiences, which seemed to be associated with their experience in the program and the terms of their transition. Girls who stuck with 

the program and had to leave because they reached their goals or the 15-month limit described a 

coordinated transition process similar to that outlined in the manual. Girls who had unplanned 

transitions, such as having to leave because of behavior issues, said their transition process was 

not smooth and reported problems transferring credits accrued while at PACE.

For the most part, parents interviewed had positive things to say about the transition 

process. They said that the staff provided them with information on the transition well in 

advance and helped with the logistics of the new placement. The parents who did not feel 

supported during the transition process were mostly parents of girls who had unplanned transitions, some of whom had been removed from the program by the parents. These parents 

described limited communication or support from the PACE staff.

Follow-Up

For girls who attended the program for at least 90 days, PACE requires 12 months of follow-up 

services after the transition.

4 The manual specifies that a designated PACE staff person makes 

contact monthly for the first three months and then quarterly. The purpose of these contacts is 

twofold: to document the girl’s situation (in school and in her living situation) and to connect 

girls with services if needed. As one counselor explained, “Just because they are leaving us 

doesn’t necessarily mean that we’re leaving them. They always have a safe place to go to.”

The staffing structure for follow-up services varied by center. For 9 of 14 centers, follow-up was a part-time role, either staffed by a part-time person or part of the duties of a fulltime staff member, such as a day counselor. At five centers, the transition counselor was a fulltime role, and the counselor would typically become involved in a girl’s care during her last few 

 4

At the beginning of the evaluation period, PACE required three years of follow-up services, but this was 

reduced to 12 months during the study enrollment period. Since MDRC’s visits to study implementation 

occurred after PACE had changed the policy, this report focuses on the 12-month follow-up period.
80

months at PACE to enable the girl to get to know the transition counselor before transition. 

Transition staff members who responded to the staff survey reported an average caseload of 35 

girls in follow-up; 29 percent said their caseload was not manageable. The academic staff 

played a minimal role in follow-up. Follow-up contacts between academic staff members and 

girls mainly occurred when girls reached out to teachers with whom they had been close.

Staff members responsible for transitions described a few methods for keeping in contact with the girls, including phone calls, social media (such as Facebook), and in-person visits 

to the girl’s school. It was often difficult to stay in touch; respondents to the staff survey 

reported that they were able to reach fewer than half the girls on their caseload during followup. The level of effort varied. Five centers reported minimal efforts to contact girls, limited to 

calling the numbers on file, leaving voice mails, and taking girls off the call list after the 

required three attempts. The remaining centers described more intensive efforts that included 

going to girls’ homes or trying to use their social networks to get in touch with them.

Many staff members indicated that the girls drove the level of follow-up: If a girl wanted help from PACE she would reach out to the program. Girls who did not want support from 

the program essentially opted out of services by not responding to contact attempts. In the 

words of one girl: “‘They call me but I ignore it.” A few staff members indicated that their 

personal relationships with the girls affected follow-up efforts: “Sometimes we try really hard to 

find them, sometimes if we can’t reach them we just let them go.”

The PACE model requires that each girl in the follow-up period have a transition care 

plan in place, which the transition staff will review with the girls during the required contacts; 

similar to the first care plan, it contains academic, behavioral, and vocational goals. At most 

centers, however, the transition care plan did not seem to be a focus of follow-up services. Most 

transition staff members reported using it either not at all or in a limited way. The MIS data 

(Table 6.4) support this finding: Only 58 percent of girls had had a transition care plan. Centers 

varied in their completion rates. On the high end, five centers had transition care plans for at 

least 75 percent of girls, while four centers had rates of 25 percent compliance or below.5 Most, 

but not all, of the centers with a full-time transition staff had higher compliance rates. A few 

centers without a full-time transition counselor also fell in the high range. This indicates that 

centers could have high compliance in this area with or without a full-time transition staff 

member. 

Because MIS data are not available on services provided to girls during the transition 

period, it is not possible to say what proportion of girls received services and the intensity of 

 5

Transition care plan compliance is calculated among girls in transition who had a length of stay longer 

than 30 days.
81

those services. Staff members described mainly connecting girls to resources in the community, 

though PACE centers offered a limited amount of services directly. The most common services 

provided to girls in follow-up were help connecting to academic services, help with job searches 

(such as providing interview clothes or résumé writing), and counseling (either directly or 

through referrals to outside providers).

Follow-up services were an area where implementation was more uneven across the 

centers, with lower rates of fidelity in quantity of services provided compared with other 

services. Although a few centers described robust services, most did the minimum — calling 

girls at the required intervals and connecting girls who sought help with supportive services. 

This finding is not surprising, given that PACE does not receive funding for follow-up services 

from the Florida Department of Juvenile Justice or the school district, so funds to support this 

work must be raised from other sources. As a result of the lack of funding, follow-up services 

lacked a dedicated staff person at most centers. Uneven implementation may also be reflected in 

the girls’ reported satisfaction with transition services — compared with the overwhelmingly 

positive responses on other measures, only 59 percent of girls surveyed reported that the 

transition plan was helpful after leaving PACE (Table 6.4). Girls interviewed also described 

mixed experiences with follow-up services. Of the girls who had made a transition, about half

reported being in regular contact with PACE and receiving support. The remaining girls said 

they were not in contact with PACE; girls indicated that they program did not contact them or 

they were not interested in keeping in touch with the program.

Impacts on Services Received

Table 6.5 shows that girls in the program group were more likely to receive support for nonacademic needs than girls in the control group, according to the 12-month follow-up survey of 

girls. The program group reported much higher rates of receiving mental health counseling or 

therapy (64 percent compared with 46 percent) and having more frequent sessions. More girls in 

the program group also reported receiving help connecting to additional services (such as 

transportation, housing, or food) than in the control group (39 percent compared with 23 

percent). These impacts demonstrate that PACE offers girls greater access to counseling and 

case management than they would receive in a traditional school setting.

Table 6.5 also shows differences in the receipt of services that are closely connected to 

the PACE model, such as attention to women’s reproductive issues and healthy relationships. 

The program group was much more likely to receive help around reproductive health (including 

access to contraception or testing for sexually transmitted infections). The program group was 

also more likely to have received help with social and emotional skills, such as dealing with 

peer pressure and developing self-confidence.
82

(continued)

Table 6.5

One-Year Impacts on Social Services Received

Program Control Difference

Outcome (%) Group Group (Impact) P-Value

Received help finding services in the community 38.5 23.3 15.2 *** 0.000

a Primary source of help finding services in the community

Public, alternative, or virtual school 8.1 36.2 -28.1

PACE 56.5 6.9 49.7

Department of Children and Families 7.3 6.6 0.7

JPO or person from Department of Juvenile Justice 

or the court system 8.2 19.2 -11.0

Health or mental health center, clinic, or private practice 7.9 13.3 -5.4

Other 12.1 17.8 -5.7

Received mental health counseling or therapy 64.2 45.5 18.7 *** 0.000

a Primary counseling or therapy provider

Health or mental health center, clinic, or private practice 17.9 47.3 -29.5

Public, alternative, or virtual school 1.9 12.7 -10.8

PACE 65.1 2.6 62.5

Department of Children and Families 2.9 5.0 -2.1

Department of Juvenile Justice, parole office, 

or the court system 4.6 5.8 -1.2

Community organization 2.3 7.4 -5.1

Other 5.4 19.2 -13.8

Frequency of counseling or therapy sessions

Once per week or more 39.3 23.9 15.4 *** 0.000

1-3 times per month 19.9 13.4 6.5 ** 0.043

Less than once per month 4.6 8.0 -3.5 * 0.081

Never 36.2 54.6 -18.5 *** 0.000

Received help related to sexuality, sex, 

or sexual and reproductive health 72.0 57.8 14.3 *** 0.000

Received help related to social and emotional skills 80.3 63.0 17.3 *** 0.000

Sample sizeb (total = 668) 407 261
83

Table 6.5 (continued)

SOURCE: MDRC calculations based on responses to the PACE evaluation 12-month follow-up survey.

NOTES: JPO = juvenile probation officer. 

Results in this table are regression-adjusted, controlling for pre-random assignment characteristics. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. aResponses include only sample members who reported receiving the services. Therefore, the measures 

are nonexperimental and statistical significance was not calculated.

bDue to missing values, the number of girls included varies by outcome. The sample size reported here is 

based on responses to the 12-month follow-up survey among girls randomly assigned through March 2015. 

85

Chapter 7 

Conclusion

This report, along with the companion brief on the principles and evidence behind genderresponsive services that uses PACE as a case study, adds to the body of knowledge about how 

gender-responsive programs operate in actuality. Specifically, PACE creates a culture that is 

sensitive to the needs of girls as a framework for providing its services: PACE centers provide a 

safe, relationship-focused environment, and program staff members emphasize fostering girls’ 

individual strengths while understanding how the effects of trauma may influence the girls’ 

responses and behavior. This culture is infused into all aspects of program delivery, as PACE

offers a combination of services that is hypothesized to meet the specific needs of girls at risk 

for poor life outcomes. Program components such as comprehensive assessments, life skills

classes, and parental engagement — along with the program environment — align with principles of gender-responsive programming.

The report also highlights the ways that PACE is different from the other services available to girls in the communities where PACE operates. Few programs offer the same combination of services, and those that do provide a similar array of services to both girls and boys. In 

most cases, girls in the control group attended a more traditional public school in their community. Girls attending PACE were more likely to participate in counseling sessions and receive 

academic advising — and with greater frequency — than girls in other academic settings. 

PACE’s staff-to-girl ratio provides for more individual attention and support in both academic 

and social services.

This evaluation also examines replication of the program — often a difficult task, and 

one of great interest to policymakers, practitioners, and researchers. PACE was able to implement its model consistently in a diversity of locations — not only providing the same services

but establishing a similar program culture in centers across the state of Florida. As described in 

this report, PACE successfully specifies its model through principles shared with the centers

and a lengthy policy and procedures manual, bolstered by comprehensive staff training and the 

use of program data to track implementation. The central office plays a key role, monitoring 

fidelity and providing guidance or support as needed. This combination of methods offers 

lessons for the field.

PACE centers did vary in some ways. Variation tended to occur in areas where the program model did not offer specific guidance: for example, the approaches used by counselors in 

their sessions with girls. This is another important lesson, underscoring the trade-offs programs 

face when thinking about how — and how thoroughly — to define their model and monitor 

fidelity. Programs must decide how much discretion staff members have in providing services: 
86

Research on adaptation and fidelity suggests that in some cases adaptation is beneficial, as it 

allows staff members to tailor an intervention to the needs of the population served.1 Finding the 

balance between specification and flexibility is an ongoing discussion for programs such as

PACE.

PACE continues to assess the services provided and its implementation practices. Over 

the course of the evaluation period, PACE implemented a new management structure at its 

central office and revised policies on program eligibility, among other changes. On the subject 

of eligibility, the management team at the state office collaborated with leaders at the centers to 

draw on their hands-on knowledge of current practices. A revamping of the life skills curriculum is under way as well. Working groups have been formed to delve into findings shared from 

the implementation research, such as the content of counseling sessions and the instructional 

methods used in the classrooms. In addition, PACE continues to evaluate whom it should serve

and how the larger societal conversation about gender identity plays out at the centers.

The organization also must continue to examine its place in the ever-changing academic 

landscape in Florida. Some communities offer a wide array of academic options, and PACE 

seeks to differentiate itself and the services it offers. As a whole, the state offers an increasing 

number of charter, alternative, and virtual (online) school options. In the 2013-2014 school year, 

more than a quarter of kindergarten through twelfth-grade public school students chose a school 

other than the one they were assigned.2 Florida offers more virtual school options than any other 

state; online learning is available in all its school districts.3 In addition, Florida has experienced 

an increasing enrollment in charter schools. From the 2010-2011 school year to 2015-2016, 

over 130 charter schools were created, serving an additional 90,000 students from prekindergarten to twelfth grade.

4 Finally, there has been a notable increase in single-gender education 

options within the state, including in the communities served by PACE.5

Attitudes about approaches to delinquency continue to evolve as well, both within Florida and nationwide. In schools, there has been a move away from “zero tolerance” discipline 

policies that resulted in expulsions and a move toward handling behavior problems within the 

school setting.6 This probably means fewer referrals for girls with behavioral issues to programs 

like PACE. Or, perhaps, girls might now be referred further down the line, when their behavioral problems have grown to be too much for a school to handle.

 1

Dusenbury, Brannigan, Falco, and Hansen (2003).

2

Florida Department of Education (2015).

3

Florida Department of Education (2017). 4

Florida Department of Education (2016). 5

Thompson (2015).

6

Alvarez (2013).
87

Similarly, practices in the juvenile justice system are shifting. Recently, there has been a 

nationwide movement to confine fewer delinquent youth.

7 In Florida, 38,267 young people

(ages 10 to 17) were arrested in fiscal year 2015-2016, down 34 percent from 57,597 in fiscal 

year 2011-2012. Females are making up a smaller percentage of juvenile arrests, falling from 30

percent to 28 percent in the same period.8

The implementation findings herein add to the evidence base on gender-responsive 

programming and its effectiveness. The companion brief delves further into the history and 

literature of the gender-responsive approach, using PACE implementation as a case study.9 In 

2018, a final report will present the results of the impact study, further reflections from girls on 

their experiences, and a cost-effectiveness analysis. The impact findings, which will provide a 

rigorous assessment of PACE’s effectiveness as a gender-responsive program for girls, will 

look at such outcomes as academic engagement and progress, involvement in the juvenile 

justice system, healthy relationships, and risky behavior (such as high-risk sexual activity and 

substance abuse). The cost-effectiveness analysis will evaluate the costs of PACE in the context 

of its outcomes for girls. Together these reports will provide needed evidence on a genderresponsive approach to helping girls.

 7

Data from Kids Count Data Center (2013): In 2006, 289 per 100,000 young people resided in a juvenile 

detention center, correctional facility, or residential facility. By 2013, the rate had dropped to 173 per 100,000

young people. 8

Florida Department of Juvenile Justice (2016). 9

Treskon and Bright (2017).

Appendix A 

Supplementary Tables on Sample Member 

Characteristics and Risk Factors

91

Appendix Table A.1

Characteristics of Sample Members at Baseline, by Random Assignment Group

Program Control

Characteristic Group Group

Age (years) 14.6 14.8

Age (%)

11-12 8.8 7.9

13-14 33.7 30.8

15-16 47.9 51.9

17 or older 9.6 9.5

Race/ethnicity (%)

Black, non-Hispanic 44.6 45.9

Hispanica 15.8 16.3

White, non-Hispanic 38.9 36.9

Other 0.7 0.9

School level at referral to PACE (%)

b 6th grade 9.8 7.2

7th-8th grade 35.3 40.1

9th-10th grade 45.9 44.4

11th-12th grade 9.0 8.2

English is second language (%) 2.1 2.1

Qualifies for special education or ESEc

 (%) 11.0 11.7

People participant lives with (%)

Two parents 34.9 34.7

Single parent 52.9 50.2

Relative 9.8 11.7

d Other 2.4 3.3

e Family income (%)

$28,050 or lower 43.0 38.5

$28,051-$44,900 35.2 35.8

Above $44,900 21.8 25.7

Family has had case with the 

Florida Department of Children and Families (%) 43.0 41.0

Sample size 679 455

(continued)
92

Appendix Table A.1 (continued)

SOURCE: MDRC calculations based on data from the PACE management information system.

NOTES: No statistically significant differences were found between the program and control groups on 

any characteristics. aSample members are coded as Hispanic if they answered "yes" to Hispanic ethnicity. 

bThis category includes sample members who were in fifth grade at the time of referral.

c

PACE uses the Florida Department of Education definition of Exceptional Student Education (ESE), 

referring to programs for students with disabilities and gifted programs. 

d"Other" includes nonrelative or foster care.

eThe U.S. Department of Housing and Urban Development's guidelines were used in the data 

collection process to determine which income range the participant's family fell into. Since these figures 

could vary by county and household size, the ranges presented here correspond to the statewide income 

limits for low income and very low income for a four-person household in Florida in fiscal year 2014.
93

(continued)

Appendix Table A.2

Risk Factors of Sample Members at Baseline, 

by Random Assignment Group

Characteristic (%)

Program

Group

Control

Group

School engagement

Recently expelled or suspendeda 39.5 39.8

Currently enrolled in school 72.6 73.8

Skipped school at least 3 times in past 2 months 33.0 37.5

Had more than 15 absences in past 3 months 42.9 39.7

Held back at least once 51.2 52.8

Failed 1 or more classes in past 6 months 76.3 77.2

Has a learning disability

Attention deficit disorder

30.5

20.9

28.1

17.2

Dyslexia

Other learning disability

1.3

8.7

1.7

9.1

Delinquency

Ever arrestedb 28.3 26.9

Ever arrested for domestic violence 4.8 7.8 *

Ever charged with burglary 4.4 4.0

Ever charged with a drug crime 4.2 2.4

Ever stolen from family, home, or neighbors 16.9 16.4

Ever been on probation 12.3 13.0

Currently on probation 10.0 10.3

c Has family member with criminal history 63.8 64.5

Has friends with delinquent record or who engage in

delinquent behavior 48.9 51.2
94

Program Control

Characteristic (%) Group Group

Appendix Table A.2 (continued)

Health and safety

d Currently using tobacco 10.6 8.8

Currently using drugs or alcohole 15.0 14.5

Ever sexually active 44.1 44.1

Currently pregnant 1.8 0.7 *

Ever run away from home 27.9 27.3

Ever had thoughts about harming/killing herself 38.6 40.5

Abused/neglectedf 38.4 37.6

Neglected 9.4 8.3

Physically abused

Sexually abused

14.9

15.0

17.3

15.3

Emotionally abused 21.3 22.2

Sample size 679 455

SOURCE: MDRC calculations based on data from the PACE management information system.

NOTES: Certain characteristics listed here were captured in two different ways during the 

random assignment period, as noted below. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 

percent. aFor approximately half of the sample, this was defined as being currently expelled or 

suspended. For the other half of the sample, this referred to one or more expulsions or 

suspensions in most recent school term. 

bIn the juvenile justice system, people are not technically "arrested"; the terminology used is 

either "incurred a charge" or "referred." cFor approximately half of the sample, this measure referred to a criminal record (including 

imprisonment, probation, parole, and house arrest) for a parent, guardian, or sibling of the 

sample member. For the other half of the sample, "family" included other members of the 

household as well. 

dFor approximately half of the sample, this was defined as having used tobacco three or more 

times in past 30 days, and for the other half of the sample this was defined as currently using 

tobacco. eFor approximately half of the sample, this was defined as having used drugs or alcohol three 

or more times in past 30 days, and for the other half of the sample this was defined as current 

drug and or alcohol use.

f

For approximately half of the sample, this measure referred only to documented instances of 

abuse or neglect. For the other half of the sample, the measure also included suspected incidents 

of abuse. 
Appendix B 

CLASS-Secondary Observations

97

As part of the implementation data collection on PACE’s academic services, the research team 

used the Classroom Assessment Scoring System-Secondary (CLASS-S) observation tool. The 

CLASS-S is a validated measure for assessing interactions between teachers and students in a 

secondary classroom setting.1

The CLASS-Secondary System

Observers using the CLASS-S score the classroom on 12 dimensions. These dimensions are 

organized under three domains, except for student engagement, which stands alone.

Emotional support Classroom 

organization

Instructional support

• Positive climate

• Teacher sensitivity 

• Regard for 

adolescent 

perspectives

• Behavioral 

management

• Productivity 

• Negative climate

• Instructional learning 

formats 

• Content understanding 

• Analysis and inquiry 

• Quality of feedback

• Instructional dialogue 

• Student 

engagement

Observers give a score ranging from 1 to 7, with 1 being the lowest and 7 the highest. 

Individual scores must be whole numbers. CLASS-S coders generally observe for a cycle, take 

time to note the scores, and then repeat the process for another observation cycle.

Training in CLASS-Secondary

A trainer from Teachstone, the company that oversees the CLASS-S, led a two-day training for 

PACE evaluation team members in September 2014. The training materials included a manual, 

scoring sheets, and an extensive library of videos. Following the training, team members 

completed and passed a reliability test to receive formal certification. Team members who 

visited centers more than six months after training was complete were required to take and pass 

a calibration test (provided through Teachstone) before the visits.

Observations at PACE Centers

During implementation research visits to the 14 participating centers, all teachers (except 

substitutes) who were present on the day or days of the visit were observed in their academic 

classrooms, including the life skills instructor. The researchers conducted a total of 80 observa-

 1

University of Virginia, Curry School of Education (2015); Piante, Hamre, and Mintz (2012). 
98

tions between November 2014 and March 2015.2 No interviews with teachers were scheduled 

before observations of their classes to avoid causing bias on the part of the observer. 

The goal at each center was to observe each teacher for one class period. This allowed 

for two cycles of CLASS-S observations, with each cycle involving one 10- to 15-minute 

period of observation, followed by a period of scoring. Observers referred to their CLASS-S 

manuals during the scoring process for guidance. Observations were not conducted if the class 

spent the entire period in testing or watching a video. No specific minimum class size was 

required for the observation to take place, as long as the number of students present was not 

atypical. In a handful of cases, classrooms with very few students (one to three) were observed.

Observers occasionally encountered challenges to scoring classrooms on site, such as 

teachers who engaged them in the lesson or disruptions from students entering and leaving the 

classroom during the observation cycle. Questions and concerns were discussed regularly during 

team meetings and sometimes in consultation with Teachstone, which provided useful guidance.

Inter-Rater Reliability

To determine whether classroom visitors were scoring in a consistent manner, about 25 percent 

of classes observed were “double-scored,” meaning that they were observed and scored by two 

site visitors during the same period. Inter-rater reliability (IRR) was established by comparing 

the scores of the two observers. For CLASS-S, the two scores on each dimension had to be the 

same or within one point of each other to be considered in agreement. Overall, the IRR was 88 

percent, comfortably above the standard threshold of 80 percent. In comparisons on specific

dimensions, IRR scores ranged from 79 percent to 97 percent. 

Calculating CLASS-S Scores

CLASS-S scores were calculated for each dimension by averaging the scores from the two 

observation cycles in a specific classroom. For classrooms that were double-coded, the classroom score was calculated by taking the average of the two observers’ average scores. To create 

the overall score for a dimension at each center, the scores for all classrooms on that dimension

were averaged. The negative climate score was reverse coded so that higher scores mean less 

negative climate. Scores of 1 to 3 are classified as low, scores above 3 but below 5 are classified 

as midrange, and scores of 5 to 7 are classified as high.

 2

Three observations were subsequently dropped from the analysis because the usual classroom teacher 

was not present for the observation. In two cases, a substitute teacher was present; in the other, the academic 

services manager had stepped in. 
Appendix C

Data Collection and Analysis

101

Random Assignment Procedures

Each of the 14 PACE centers used similar procedures to conduct random assignment from 

August 2013 through October 2015. Specifically, each center used its existing intake and 

eligibility procedures to determine whether an applicant was a good fit for PACE. During intake 

meetings, trained PACE staff members used MDRC-provided tools, including a short video, to 

discuss the evaluation and gather consent from the girl and her parent or guardian.

Once a girl was determined eligible for PACE and her study paperwork was complete, 

PACE staff members entered a few pieces of her information into MDRC’s web-based random 

assignment system. The system was set up to assign 60 percent of girls to the program group 

and 40 percent to the control group. This random assignment ratio was consistently used across 

sites. Random assignment was usually conducted the same day a girl was deemed eligible for 

PACE, and she was notified of the random assignment result within 24 hours.

PACE’s Management Information System

MDRC processed two types of data from the PACE management information system (MIS): 

data collected at intake on all sample members’ demographics and risk factors, and program 

participation data collected once a sample member enrolled at PACE.

Every two months from August 2013 through December 2015, MDRC pulled the intake data directly from the PACE MIS. Though each data pull was cumulative, any demographic or risk factor information that had been updated from a prior pull was not used. Information 

on program enrollment and transition dates, also included in the intake data, was updated with 

the processing of each pull. In order to capture the most recently updated information on 

program entry and exit for this report, MDRC made a few additional pulls of these data in 2016, 

with the last file processed in October 2016.

Because retrieving program participation data from the MIS was more complex, the 

PACE administrator performed this task. From August 2014 to November 2016, cumulative 

program participation data records were periodically pulled for study participants only. MDRC 

worked extensively with the PACE administrator to understand which elements from the 

Efforts-to-Outcomes (ETO) software corresponded to the program participation measures of 

interest. Each data pull produced about 20 separate extracts, with each extract providing data on 

a specific type of program activity. MDRC then processed each data pull, which included 

performing checks for duplicates and invalid data and restructuring the data as needed to 

produce final measures of program participation across several domains. If any study participants were missing from an extract, MDRC notified the PACE administrator, who was often 
102

able to recover the missing data. Most of the program participation data recorded in ETO is 

documentation of an activity that occurred. In most cases, if a girl did not have any data entered 

for a specific activity, MDRC assumed no participation.

Twelve-Month Follow-Up Survey of Girls

A follow-up survey was administered to sample members by phone or in person approximately 

one year after random assignment. Fielding of the survey began in October 2014 and ended in 

December 2016. The survey covered a number of domains, including program satisfaction, 

academic progress and employment, outlook and positive youth development, social support 

and mental health, risky behavior, and control group service receipt. Survey responses presented 

throughout this report are from participants enrolled in the study between August 2013 and 

March 2015, about two-thirds of the full study sample. While all sample members were approached to complete the survey, not all girls could be reached or agreed to participate. Specifically, the response rate for this period was 71 percent overall; 407 girls in the program group 

(73 percent) and 261 in the control group (69 percent) responded to the survey.

Staff Survey

The staff survey was fielded to 299 PACE staff members at 16 centers using a list of email 

addresses obtained from PACE headquarters. The list included all staff members employed at 

the centers, including part-time administrative support staff. The survey was piloted in Fall 2014 

to the staff at three centers, and a revised version of the survey was fielded to the remaining 13 

centers in early 2015. Two centers receiving the survey ultimately did not participate in the 

implementation study because of an inadequate number of girls in their research samples. The 

survey results for these two centers are not included in the results presented in this report. The 

staff survey had a response rate of 91 percent.

Missing Data in Quantitative Data Processing

Program staff members enter the baseline data as part of their existing intake procedures. This 

information is required for a program eligibility determination before random assignment. 

Values were imputed for any missing baseline data used in the service receipt impact analysis.

Sample members with missing data on any of the survey responses or outcomes were 

removed from the impact analysis for that particular measure. In the case of the 105-item 

Organizational Social Context measure administered in the staff survey, cases were eliminated 

if they had 11 or more items missing, were inconsistent in responses, or showed irregularity of 

response patterns. In all, eight cases were eliminated.
103

Qualitative Data Analysis

Qualitative data were primarily gathered from interviews with staff members, girls, and parents 

using structured interview and observation protocols. A team of 12 researchers participated in 

the site visits. After the visits, the researchers recorded the information gathered in structured 

write-up templates designed to ensure that similar data were collected across staff roles and 

centers. A team of two implementation leads reviewed each write-up for clarity and consistency.

All qualitative data were uploaded and coded in Dedoose, a mixed-methods analysis 

software. Data were coded by a small team of four researchers using a prespecified set of codes. 

All coded data used in this report were checked for proper code application by the lead implementation researcher. Qualitative data were also attached to both a staff- or individual-level 

descriptor set and a center-level descriptor set in Dedoose. Descriptor sets are categorical or 

numeric variables that can be used to create subgroups of centers or staff members within 

Dedoose to aid in the analysis of the qualitative data. The descriptor sets were used to analyze 

the coded data across staff roles and centers to assess variation.

105

References

Aizer, Anna, and Joseph J. Doyle Jr. 2015. “Juvenile Incarceration, Human Capital, and Future 

Crime: Evidence from Randomly-Assigned Judges.” Quarterly Journal of Economics 130, 

2: 759-803. 

Alvarez, Lizette. 2013. “Seeing the Toll, Schools Revise Zero Tolerance.” New York Times

(December 2), p. A1.

Aos, Steve, Roxanne Lieb, Jim Mayfield, Marna Miller, and Annie Pennucci. 2004. Benefits 

and Costs of Prevention and Early Intervention Programs for Youth. Olympia: Washington 

State Institute for Public Policy.

Bright, Charlotte Lyn, and Melissa Jonson-Reid. 2008. “Onset of Juvenile Court Involvement: 

Exploring Gender-Specific Associations with Maltreatment and Poverty.” Children and 

Youth Services Review 30, 8: 914-927. 

Bright, Charlotte Lyn, and Melissa Jonson-Reid. 2010. “Young Adult Outcomes of Juvenile 

Court–Involved Girls.” Journal of Social Service Research 36, 2: 94-106. 

Catalano, Richard F., M. Lisa Berglund, Jean A. M. Ryan, Heather S. Lonczak, and J. David 

Hawkins. 2004. “Positive Youth Development in the United States: Research Findings on 

Evaluations of Positive Youth Development Programs.” The ANNALS of the American 

Academy of Political and Social Science 591, 1: 98-124. 

Center for Behavioral Health Research. 2016. “OSC Measure and OSC Profiles.” The 

University of Tennessee, Knoxville. Accessed December 20. Website: 

http://cbhr.utk.edu/osc/osc-measure-and-osc-profiles/. 

Centers for Disease Control and Prevention. 2016. “Youth Online: High School YRBS, Florida 

2013 Results.” Accessed December 20. Website: https://nccd.cdc.gov/youthonline/

App/Results.aspx?LID=FL. 

Chesney-Lind, Meda, Merry Morash, and Tia Stevens. 2008. “Girls’ Troubles, Girls’ 

Delinquency, and Gender Responsive Programming: A Review.” Australian & New 

Zealand Journal of Criminology 4, 1: 162-189. 

Corporation for National and Community Service. 2014. Social Innovation Fund Evaluation 

Plan Guidance: A Step-by-Step Guide to Designing a Rigorous Evaluation. Washington, 

DC: Corporation for National and Community Service. 

Covington, Stephanie S., and Barbara E. Bloom. 2006. “Gender-Responsive Treatment and 

Services in Correctional Settings.” Women and Therapy 29, 3/4: 9-33. 

Donaldson, Morgan L., Kimberly LeChasseur, and Anysia Mayer. 2016. “Tracking 

Instructional Quality Across Secondary Mathematics and English Language Arts Classes.” 

Journal of Educational Change. doi:10.1007/s10833-015-9269-x
106

Dusenbury, Linda, Rosalind Brannigan, Mathea Falco, and William B. Hansen. 2003. “A 

Review of Research on Fidelity of Implementation: Implications for Drug Abuse 

Prevention in School Settings.” Health Education Research: Theory & Practice 18, 2: 237-

256. 

Florida Department of Education. 2014a. Students Entering Grade Nine in the 2014-2015 

School Year: What Students and Parents Need to Know. Website: http://www.fldoe.org. 

Florida Department of Education. 2014b. Traditional Schools: 2014-15 Class Size Averages 

and FTE/Classes Over Cap. Website: www.fldoe.org. 

Florida Department of Education. 2015. School Choice Options: Florida Continues to Lead the 

Nation. Tallahassee: Florida Department of Education.

Florida Department of Education. 2016. Florida’s Charter Schools. Fact sheet. Website: 

www.floridaschoolchoice.org.

Florida Department of Education. 2017. “Virtual Education.” Website: www.fldoe.org/schools/ 

school-choice/virtual-edu. 

Florida Department of Juvenile Justice. 2016. “Delinquency Profile FY 2014-2015.” Accessed 

December 31. Website: www.djj.state.fl.us/research/delinquency-data/delinquencyprofile/delinquency-profile-dashboard. 

Florida Legislative Office of Economic and Demographic Research. 2011. “Florida 

Population Density by County: 2010.” Website: http://edr.state.fl.us/content/populationdemographics/2010-census/maps/Census_2010_density.pdf. 

Gilligan, Carol. 1982. In a Different Voice: Psychological Theory and Women's Development. 

Cambridge, MA: Harvard University Press.

Glisson, Charles, and Philip Green. 2006. “The Effects of Organizational Culture and Climate 

on the Access to Mental Health Care in Child Welfare and Juvenile Justice Systems.” 

Administration and Policy in Mental Health Services Research 33, 4: 433-448. 

Glisson, Charles, Philip Green, and Nathaniel J. Williams. 2012. “Assessing the Organizational 

Social Context (OSC) of Child Welfare Systems: Implications for Research and Practice.” 

Child Abuse & Neglect 36, 9: 621-632. 

Glisson, Charles, John Landsverk, Sonja Schoenwald, Kelly Kelleher, Kimberly Eaton 

Hoagwood, Stephen Mayberg, and Philip Green. 2007. “Assessing the Organizational 

Social Context (OSC) of Mental Health Services: Implications for Research and Practice.” 

Administration and Policy in Mental Health Services Research 35: 98-113. 

Hemmelgarn, Anthony, and Charles Glisson. 1998. “The Effects of Organizational Climate and 

Interorganizational Coordination on the Quality and Outcomes of Children’s Service 

Systems.” Child Abuse & Neglect 22, 5: 401-421. 
107

Hubbard, Dana Jones, and Betsy Matthews. 2008. “Reconciling the Differences Between the 

‘Gender-Responsive’ and ‘What Works’ Literatures to Improve Services for Girls.” Crime 

& Delinquency 54, 2: 225-258. 

Kerig, Patricia K., and Sheryl R. Schindler. 2013. “Engendering the Evidence Base: A Critical 

Review of the Conceptual and Empirical Foundations of Gender-Responsive Interventions 

for Girls’ Delinquency.” Laws 2, 3: 244-282. 

Kids Count Data Center. 2013. “Children Ages 6 to 17 Who Repeated One or More Grades 

Since Starting Kindergarten: 2011-2012.” Annie E. Casey Foundation. Website: 

http://datacenter.kidscount.org/data#USA/1/0/char/0.

Kleinfeld, Judith. 2009. “The State of American Boyhood.” Gender Issues 26: 113-129. 

Ko, Susan J., Julian D. Ford, Nancy Kassam-Adams, Steven J. Berkowitz, Charles Wilson, and 

Marleen Wong. 2008. “Creating Trauma-Informed Systems: Child Welfare, Education, 

First Responders, Health Care, Juvenile Justice.” Professional Psychology: Research and 

Practice 39, 4: 396-404. 

Maniglia, Rebecca. 1998. “Juvenile Female Offenders: A Status of the States Report.” 

Washington, DC: Office of Juvenile Justice and Delinquency Prevention, U.S. Department 

of Justice. Website: www.ojjdp.gov/pubs/gender/. 

Millenky, Megan, and Caroline Mage. 2016. Preventing Juvenile Justice Involvement for Young

Women: An Introduction to an Evaluation of the PACE Center for Girls. New York: 

MDRC.

National Crittenton Foundation. 2016. “National Girls Initiative.” Website: 

www.nationalcrittenton.org/what-we-do/national-girls-initiative/. 

Office of Juvenile Justice and Delinquency Prevention. 2015. “Legislation/JJDP Act.” U.S. 

Department of Justice, Office of Justice Programs. Website: www.ojjdp.gov/about/ 

legislation.html.

Office of Juvenile Justice and Delinquency Prevention. 2016a. “Girls and the Juvenile Justice 

System.” U.S. Department of Justice, Office of Justice Programs. Accessed December 20.

Website: www.ojjdp.gov/policyguidance/girls-juvenile-justice-system/. 

Office of Juvenile Justice and Delinquency Prevention. 2016b. “Girls at Risk.” U.S. Department 

of Justice, Office of Justice Programs. Accessed December 31. Website: 

www.ojjdp.gov/programs/girlsatrisk.html.

One Circle Foundation. 2012. “Girls Circle.” Website: https://onecirclefoundation.org/GC.aspx. 

PACE. 2014. Seek Excellence: 2014 Annual Report. Jacksonville, FL: PACE Center for Girls.

PACE. 2016. PACE Center for Girls. “A Statewide Network with a Local Touch.” Accessed 

December 28. Website: http://www.pacecenter.org/locations/all-locations. 
108

Pianta, Robert C., Bridget K. Hamre, and Susan Mintz. 2012. Classroom Assessment Scoring 

System, Secondary Manual. Charlottesville, VA: Teachstone Training, LLC.

Renaissance Learning Inc. 2016. “Renaissance Star 360.” Website: www.renaissance.com/

products/assessment/star-360/. 

Student Support Services Project. 2012. “Social Worker : Student Ratio, Based on Florida 

Department of Education Membership by District, Fall 2012.” Florida Department of 

Education and the University of South Florida. Website: http://sss.usf.edu/aboutus/

professions/socialwork/SWratios13.pdf. 

Substance Abuse and Mental Health Service Administration. 2014. SAMHSA’s Concept of 

Trauma and Guidance for a Trauma-Informed Approach. HHS Publication No. (SMA) 14-

4884. Rockville, MD: Substance Abuse and Mental Health Services Administration.

Thompson, Rhema. 2015. “Image of Single-Gender Schools Is Changing, but Debate 

over Merits of Programs Continues.” Jacksonville.com (August 17). Website: 

http://jacksonville.com. 

Todd, Guila. 2014. “United States Student-to-Counselor Ratios for Elementary and Secondary 

Schools — 2011-2012 Data Year.” American Counseling Association, Government 

Affairs Office. Website: www.counseling.org/docs/default-source/public-policy-faqs-anddocuments/2013-counselor-to-student-ratio-chart.pdf?sfvrsn=2. 

Treskon, Louisa, and Charlotte Lyn Bright. 2017. Bringing Gender-Responsive Principles into 

Practice: Evidence from the Evaluation of the PACE Center for Girls. New York: MDRC. 

University of Florida Health. 2017. “Baker Act.” Website: https://ufhealth.org/baker-act. 

University of Virginia, Curry School of Education. 2015. “Classroom Assessment Scoring 

SystemTM.” Website: http://curry.virginia.edu/research/centers/castl/class.

Watson, Liz, and Peter Edelman. 2013. Improving the Juvenile Justice System for Girls: 

Lessons from the States. Washington, DC: Georgetown Center on Poverty, Inequality and 

Public Policy.

Zahn, Margaret A., Robert Agnew, Diana Fishbein, Shari Miller, Donna-Marie Winn, Gayle 

Dakoff, Candace Kruttschnitt, Peggy Giordano, Denise C. Gottfredson, Allison A. Payne, 

Barry C. Feld, and Meda Chesney-Lind. 2010. Causes and Correlates of Girls’ 

Delinquency. Washington, DC: Office of Juvenile Justice and Delinquency Prevention, 

U.S. Department of Justice. 

Zahn, Margaret A., Stephanie R. Hawkins, Janet Chiancone, and Ariel Whitworth. 2008. The 

Girls Study Group — Charting the Way to Delinquency Prevention for Girls. Washington, 

DC: Office of Juvenile Justice and Delinquency Prevention, U.S. Department of Justice.
About MDRC

MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 

to learning what works to improve the well-being of low-income people. Through its research 

and the active communication of its findings, MDRC seeks to enhance the effectiveness of social and education policies and programs.

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 

for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 

Its projects are a mix of demonstrations (field tests of promising new program approaches) and 

evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 

combination of research and organizational experience to their work, providing expertise on the 

latest in qualitative and quantitative methods and on program design, development, implementation, and management. MDRC seeks to learn not just whether a program is effective but also 

how and why the program’s effects occur. In addition, it tries to place each project’s findings in 

the broader context of related research — in order to build knowledge about what works across 

the social and education policy fields. MDRC’s findings, lessons, and best practices are proactively shared with a broad audience in the policy and practitioner community as well as with the 

general public and the media.

Over the years, MDRC has brought its unique approach to an ever-growing range of policy areas and target populations. Once known primarily for evaluations of state welfare-to-work programs, today MDRC is also studying public school reforms, employment programs for exoffenders and people with disabilities, and programs to help low-income students succeed in 

college. MDRC’s projects are organized into five areas:

•

•

•

•

•

Promoting Family Well-Being and Children’s Development

Improving Public Education

Raising Academic Achievement and Persistence in College

Supporting Low-Wage Workers and Communities

Overcoming Barriers to Employment

Working in almost every state, all of the nation’s largest cities, and Canada and the United 

Kingdom, MDRC conducts its projects in partnership with national, state, and local governments, public school systems, community organizations, and numerous private philanthropies.