Improving Service Delivery for

Children Affected by Trauma 

An Implementation Study of Children’s Institute, Inc.

Michelle S. Manno

Louisa Treskon 

 

 

August 2016
This material is based upon work supported by the Social Innovation Fund (SIF), a program of the 

Corporation for National and Community Service (CNCS). The Edna McConnell Clark Foundation 

and the SIF include support from CNCS and 15 private co-investors: The Edna McConnell Clark 

Foundation, The Annie E. Casey Foundation, The Duke Endowment, The William and Flora 

Hewlett Foundation, The JPB Foundation, George Kaiser Family Foundation, The Kresge Foundation, Open Society Foundations, The Penzance Foundation, The Samberg Family Foundation, The 

Charles and Lynn Schusterman Family Foundation, The Starr Foundation, Tipping Point Community, The Wallace Foundation, and the Weingart Foundation.

Dissemination of MDRC publications is supported by the following funders that help finance 

MDRC’s public policy outreach and expanding efforts to communicate the results and implications of our work to policymakers, practitioners, and others: The Annie E. Casey Foundation, 

Charles and Lynn Schusterman Family Foundation, The Edna McConnell Clark Foundation, Ford 

Foundation, The George Gund Foundation, Daniel and Corinne Goldman, The Harry and Jeanette 

Weinberg Foundation, Inc., The JBP Foundation, The Joyce Foundation, The Kresge Foundation, 

Laura and John Arnold Foundation, Sandler Foundation, and The Starr Foundation.

 

In addition, earnings from the MDRC Endowment help sustain our dissemination efforts. Contributors to the MDRC Endowment include Alcoa Foundation, The Ambrose Monell Foundation, 

Anheuser-Busch Foundation, Bristol-Myers Squibb Foundation, Charles Stewart Mott Foundation, Ford Foundation, The George Gund Foundation, The Grable Foundation, The Lizabeth and 

Frank Newman Charitable Foundation, The New York Times Company Foundation, Jan Nicholson, Paul H. O’Neill Charitable Foundation, John S. Reed, Sandler Foundation, and The Stupski 

Family Fund, as well as other individual contributors.

The findings and conclusions in this report do not necessarily represent the official positions or 

policies of the funders. 

For information about MDRC and copies of our publications, see our website: www.mdrc.org. 

Copyright © 2016 by MDRC®. All rights reserved.
iii

Overview 

The Children’s Institute, Inc. (CII), is a multiservice organization in Los Angeles, California, that 

combines clinical mental health and other supportive services to meet the needs of children and their 

families who have been affected by trauma, such as physical or sexual abuse, domestic violence, or 

violence in the community. Through its Integrated Service Model, CII provides holistic and coordinated support to children and families by potentially engaging them in multiple services: clinical 

services to address children’s mental health needs, programs for parents and guardians to help them 

better support their children, and youth activities to develop protective factors. The comprehensive

nature of this model sets it apart from the often fragmented and uncoordinated child welfare system. 

A central aspect of CII’s model is using evidence-based practices — highly specified treatment 

models that research has shown to be effective in treating a targeted population — in its clinical 

services.

The CII evaluation had two main components: an implementation study of CII’s service model and a 

study of CII’s delivery of evidence-based practices, including an in-depth fidelity study of its 

Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) services. 

Key Findings

•

•

•

•

CII is achieving its goal of engaging clients in multiple services to holistically meet their needs.

A majority of clients receiving clinical services from CII also participated in another service at 

the organization.

Analysis of management information system data indicates that nearly a third of the children 

engaged in clinical services received an evidence-based practice. While little is known about 

national norms for the use of evidence-based practices, the study’s findings suggest that CII is a 

leader in providing them.

Analysis also indicates that the dosage levels of Functional Family Therapy and TF-CBT — 

two prominent evidence-based practices at CII — were both in line with model expectations.

The in-depth fidelity study of TF-CBT indicated that CII’s implementation of the treatment 

model was aligned with that of other community-based organizations in similar fidelity studies. The average client had at least a 50 percent chance of receiving half of the model’s core 

components.

A Technical Resource for this report presents the complete set of findings from the in-depth fidelity 

study of CII’s delivery of TF-CBT and is available on the MDRC website. 

CII is also involved in MDRC’s Building Bridges and Bonds study of fatherhood programs, funded 

by the U.S. Department of Health and Human Services.

v 

Contents

Overview iii

List of Exhibits vii

Preface ix

Acknowledgments xi

Executive Summary ES-1

Chapter

1 Introduction 1

Child Trauma and Treatment 4 

Overview of this Report 8 

Data Collection and Data Sources for the Implementation Study 9 

Data Collection and Data Sources for the Fidelity Study 10 

Roadmap to the Report 11 

2 Overview of CII 13

History and Background of CII 13 

The Communities CII Serves 13 

Overview of CII’s Services and Operating Philosophy 17 

CII’s Funding Structure 22 

History and Development of Evidence-Based Practices at CII 25 

Conclusion 26 

3 Pathways to CII 27 

CII’s Target Population 27 

Intake and Assessment 29 

Conclusion 34 

4 Implementation of CII Services 35 

Staffing Structure 37 

Clinical Mental Health Services 37 

Family Support 45 

Youth Development 46 

The Integrated Service Model 50 

Conclusion 54 

5 A Closer Look at Two Evidence-Based Practices 57

Functional Family Therapy (FFT) 59 

Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) 66 

TF-CBT Fidelity Study Findings 71 

Conclusion 75 
vi

6 Discussion 77 

Integrated Service Model 77 

Evidence-Based Practices 78 

Changes at CII 78 

Lessons Learned and Next Steps 79 

Appendix

A Data Analysis Limitations 81 

References 89 
vii

List of Exhibits

Table

2.1 Key Indicators of Health by Service Planning Area 15

3.1 Demographic Characteristics of CII Clients 28

3.2 Demographic Characteristics of CII Clients (Ages 9-24 Year Olds) 29

4.1 Client Participation in Any CII Services 36

4.2 Clinical Services Staff Tenure and Training in Evidence-Based Practices 42

4.3 Demographic Characteristics of CII Clients Engaged in Clinical Services 43

4.4 Duration and Intensity of Client Participation in CII Clinical Services 44

4.5 Client Demographic Characteristics and Participation in Family Support 

Services 47

4.6 Client Demographic Characteristics and Participation in Youth Development 

Services 51

4.7 Client Participation in CII Services 53

5.1 Demographic Characteristics of CII Clients Engaged in Evidence-Based 

Practices 58

5.2 Client Participation in Functional Family Therapy 65

5.3 Client Participation in Trauma-Focused Cognitive Behavioral Therapy 72

A.1 Characteristics of Clinician Survey Respondents 88

Figure

2.1 Map of Los Angeles County Service Planning Areas Served by CII 14

2.2 CII’s Integrated Service Model Conceptual Framework 20

3.1 Basic Client Flow Through CII Services 31

4.1 Sample SPA Staffing Organizational Chart 38

Box

1.1 The EMCF SIF Initiative 2 
viii

2.1 Programs of the Los Angeles County Departments of Mental Health and 

Children and Family Services that CII Provides 21

2.2 California Policy Context 24

4.1 Evidence-Based Practices Used by CII for the Study’s Target Age Group 39

4.2 Evidence-Informed Practices Used by CII for the Study’s Target Age Group 40

4.3 Examples of Youth Development Activities Offered by CII 49

5.1 Phases of FFT 61

5.2 Maya’s Story 62

5.3 Sofia’s Story 68

5.4 TF-CBT Components 69
ix

Preface

There is overwhelming evidence that traumatic experiences in childhood — such as physical or 

sexual assault, gang violence, domestic violence, or sudden loss of a loved one — can lead to 

poor outcomes in adulthood. While the child welfare field is extensive and works to improve 

the life prospects of trauma-affected children and families, the available services are nevertheless often fragmented and uncoordinated. Research has identified evidence-based practices that 

improve outcomes for these children and families, and there has been a push at the federal level 

in recent years to increase the use of such practices in children’s mental health care. However, 

many of the current services available lack evidence of their effectiveness.

In this context, the Los Angeles-based Children’s Institute, Inc. (CII), operates its wide 

range of programs and services, including clinical mental health services, early child care and 

Head Start programs, programs for parents and guardians, and youth development activities. 

Integrating and coordinating these services to address the holistic needs of children and families 

is a critical component of CII’s service model, as is the use of evidence-based practices in 

mental health treatment when appropriate.

This report describes how in implementing its Integrated Service Model CII sought to 

overcome the barriers associated with the fragmented and uncoordinated child welfare system 

through an approach that attempts to identify clients’ full range of needs and ensure they receive 

all the support required to address those needs. It offers lessons in how multiservice organizations such as CII can structure services to meet the holistic needs of clients. Integrating services 

as CII has done, however, is not without its challenges. Tailoring services to the varied needs of 

each client requires navigating the complex funding system on which multiservice organizations 

rely, and which includes public agencies, private foundations, and health insurance providers.

The report also adds to the understanding of the challenges of implementing evidencebased practices in community-based settings, where the highly specified protocols of these

practices meet the realities of providing services in high-needs and under-resourced communities. The in-depth fidelity study of CII’s delivery of Trauma-Focused Cognitive Behavioral 

Therapy services highlighted some of the difficulties therapists encounter when delivering a 

structured treatment to high-needs clients. The study found that therapists at CII did not provide 

all of the model’s required treatment components, which is consistent with findings from other, 

similar studies of community-based providers. This finding suggests the need for robust and 

low-cost tools to help providers deliver evidence-based treatments with fidelity. 

Gordon L. Berlin 

President, MDRC

xi

Acknowledgments

This report is based upon work supported by the Social Innovation Fund (SIF), a major White 

House initiative and program of the Corporation for National and Community Service (CNCS). 

The SIF combines public and private resources with the goal of increasing the impact of 

innovative, community-based solutions that have compelling evidence of improving the lives of 

people in low-income communities throughout the United States.

The Edna McConnell Clark Foundation and the SIF include support from CNCS and 15 

private co-investors: The Edna McConnell Clark Foundation, The Annie E. Casey Foundation, 

The Duke Endowment, The William and Flora Hewlett Foundation, The JPB Foundation, 

George Kaiser Family Foundation, The Kresge Foundation, Open Society Foundations, The 

Penzance Foundation, The Samberg Family Foundation, The Charles and Lynn Schusterman 

Family Foundation, The Starr Foundation, Tipping Point Community, The Wallace Foundation, 

and the Weingart Foundation. The Wallace Foundation provided additional support separate 

from its involvement with the SIF. This report would not have been possible without the 

support of these funders.

The assistance and support of many staff at Children’s Institute, Inc. (CII), were critical 

to the success of this study throughout all phases of the project. We especially want to thank 

current CII leadership, in particular Mary Emmons, Nina Revoyr, Jacqueline Atkins, Marion 

Dave, and Manuel Rivera. We also thank Steve Ambrose, Todd Sosna, and Cynthia ThompsonRandle, all of whom have since moved on from CII but who were critical to the success of this 

evaluation. Thanks also goes to Bruce Baker for his insight. Our data analysis would not have 

been possible without the assistance of Joshua Shaw, Jade Wong, Bill Monro, and Patrick Foy. 

A special thanks goes to all the CII staff we interviewed during our site visits. We would also 

like to thank CII’s funders and partners who participated in interviews, including representatives 

from Los Angeles County Departments of Mental Health and Children and Family Services.

The fidelity study of Trauma-Focused Cognitive Behavioral Therapy benefited from the 

efforts of many individuals. Patricia Chamberlain provided advice during the planning process. 

Our colleagues at the Medical University of South Carolina conducted the fidelity study: 

Rochelle Hanson, Jason Chapman, Sonja Schoenwald, Michael de Arellano, Carrie Jackson. 

CII’s clinical supervisors obtained informed consent from the clients. Many thanks go to the 

therapists who agreed to participate in the fidelity study. We also thank Matthew Dutcher who 

entered all the data from the Brief Practice Checklists for the fidelity study. We would be remiss 

if we also did not extend our deepest thanks to the clients and their families who agreed to 

participate in the fidelity study as well.
xii

The research team received useful feedback and encouragement throughout the project 

and comments on drafts of this report from Gabriel Rhoads, Jehan Velji, and Teresa Power at 

The Edna McConnell Clark Foundation.

At MDRC, John Martinez, Dan Bloom, David Butler, Rekha Balu, Kate Gualtieri, 

Robert Ivry, and Michael Johns provided thoughtful comments on several drafts of this report. 

We are grateful to Sally Dai for all the work she did to analyze the data. Melina Davis, Janae 

Bonsu, and Mifta Chowdhury assisted in qualitative data collection, and Mifta Chowdhury 

coordinated the production of the report. Christopher Boland edited the report, and Carolyn 

Thomas prepared it for publication.

The Authors
ES-1 

Executive Summary 

Child abuse and neglect are significant problems in the United States, touching millions of lives 

each year. The U.S. Department of Health and Human Services reported that in fiscal year 

2012, 3.2 million children nationwide were the subject of a report to child protective services. 

The majority of youth in the child welfare system exhibit behavioral or social issues that are 

severe enough to warrant mental health treatment, a rate up to five times greater than mental 

health needs among their peers in the community who are not involved in the child welfare 

system.1

Recent trends in the children’s mental health care field have indicated that these behavioral and emotional issues can be largely attributed to trauma experienced earlier in life, which 

leads to “toxic stress responses” that can have a wide variety of adverse psychological and 

physiological consequences, some of which continue into adulthood.

2 Trauma can result from

many experiences and events, including physical or sexual assault, gang violence, domestic 

violence, serious accidents, sudden or violent loss of a loved one, and natural disasters.3 Children do not have to be the direct victims of violence to be affected by it; researchers have shown 

that exposure to community violence, such as hearing gun fire, has traumatic effects on children.4 Child welfare organizations throughout the country combat trauma in all its forms 

through a combination of prevention programs, direct services to affected families, and advocacy. While the lifelong impact of childhood mental illness and trauma is well documented, many 

children and youth do not receive the mental health treatment they need. Even when they do 

receive treatment, services may be inadequate or ineffective, and are often fragmented and 

uncoordinated.

 1

This estimate is based on studies across child welfare systems in several states, as well as recent results of 

the National Survey for Child and Adolescent Well-Being. John A. Landsverk, Barbara J. Burns, Leyla F. 

Stambaugh, and Jennifer A. Rolls Reutz, Mental Health Care for Children and Adolescents in Foster Care: 

Review of Research Literature (Seattle, WA: Casey Family Programs, 2006). 2

Wendy K. Silverman, Claudio D. Ortiz, Chockalingham Viswesvaran, Barbara J. Burns, David J. Kolko, 

Frank W. Putnam, and Lisa Amaya-Jackson, “Evidence-Based Psychosocial Treatments for Children and 

Adolescents Exposed to Traumatic Events,” Journal of Clinical Child and Adolescent Psychology 37, 1 

(2008): 156-183; Vincent J. Felitti, Robert F. Anda, Dale Nordenberg, David F. Williamson, Alison M. Spitz, 

Valerie Edwards, Mary P. Koss, and James S. Marks, “Relationship of Childhood Abuse and Household 

Dysfunction to Many of the Leading Causes of Death in Adults: The Adverse Childhood Experiences (ACE) 

Study,” American Journal of Preventive Medicine 14, 4 (1998): 245-258. 3

Child Welfare Committee, National Child Traumatic Stress Network, Child Welfare Trauma Training 

Tool Kit: Comprehensive Guide (2nd ed.) (Los Angeles and Durham, NC: National Center for Child Traumatic 

Stress, 2008). 4

Joy D. Osofsky (ed.), Children in a Violent Society (New York: Guilford Press, 1998).
ES-2 

This report presents findings from a study of the Children’s Institute, Inc. (CII). A multiservice organization in Los Angeles, CII combines a broad range of clinical and nonclinical 

services to meet the needs of children and families who have been affected by trauma. Each 

year, CII serves more than 20,000 children and family members. CII’s range of activities, which 

it calls its Integrated Service Model, serve the “whole child, entire family.” Through its service

model, CII provides a broad range of supports that the child and family may need to overcome a 

history of abuse or trauma, including clinical services to address mental health needs, programs 

for parents and guardians to help them better support their children, and nonclinical youth 

development activities to help children and youth acquire protective factors.

5 CII also operates 

child care and Head Start programs for young children. CII’s treatment approach is trauma 

informed, and its services are designed to directly address the impact of trauma on children’s 

lives. An important aspect of CII’s Integrated Service Model is its focus on using evidence-based 

practices in clinical services. Evidence-based practices are highly specified treatment models that 

research has shown to be effective in treating specific symptoms in target populations.

The report’s findings are based on work supported by the Social Innovation Fund (SIF), 

a program of the Corporation for National and Community Service.6 The SIF combines public 

and private resources to increase the impact of innovative, community-based solutions with 

compelling evidence of improving the lives of people in low-income communities. As part of 

the Edna McConnell Clark Foundation SIF project, which focuses on children and youth ages 9 

to 24 years, CII sought to expand its youth services, including the youth development activities 

offered in the Central and South Los Angeles neighborhoods.

Child Trauma and Treatment

While the lifelong impact of childhood mental illness and trauma is well documented, many 

children and youth do not receive the mental health treatment they need. Research has shown 

that many young people in need of such care either do not receive services, or, when they do, 

receive services that are inadequate or ineffective and often unsupported by evidence.

7 This 

report uses the term “usual care” to describe mental health care that is not based on evidence.8

 5

Protective factors are characteristics of individuals, families, or communities that mitigate risks to health 

and well-being. Examples include positive social connections, parenting skills and knowledge of child 

development, and effective communication practices. 6

CII is one of 12 evidence-based programs selected in 2011 to be part of the SIF program. EMCF matched 

$30 million from the SIF program with $30 million from its own endowment. The True North Fund, developed 

by EMCF in 2011, helped the 12 SIF grantees secure the $60 million they were required by statute to raise to 

match this funding. 7

Barbara J. Burns, E. Jane Costello, Adrian Angold, Dan Tweed, Dalene Stangl, Elizabeth M. Z. Farmer, 

and Al Erkanli, “Children’s Mental Health Service Use across Service Sectors,” Health Affairs 14, 3 (1995): 

147-159; Sheryl H. Kataoka, Lily Zhang, and Kenneth B. Wells, “Unmet Need for Mental Health Care among 

(continued)
ES-3 

At the federal level, there has been a significant push to incorporate evidence-based 

practices into mental health care for children and youth. Unlike many other types of mental 

health care, the treatment models for evidence-based practices are well defined. Each model or 

treatment practice typically specifies the target population for which the treatment has been 

shown to be effective, as well as the treatment’s outcomes, content, dosage, and duration. While 

the current trend in the child welfare field is to increase the use of evidence-based practices, not 

every client may be appropriate to receive one of these treatments since they target highly 

specified symptoms or age groups. 

Providing empirically supported treatment in community-based settings presents a host 

of challenges. Community-based settings are often quite different from the university research 

settings where the practices are usually first developed and tested. Community-based providers

typically serve more diverse and higher-risk populations and have larger caseloads. Whether or 

not an evidence-based treatment offered in a community-based setting is effective depends on 

how the provider implements it. In order to transfer efficacy from research to practice, providers 

must implement the treatment with fidelity to the model that was originally tested, which can be

particularly challenging in community-based settings. Fidelity encompasses a number of areas: 

staff training practices, targeting the appropriate population, administering the correct dosage 

and frequency of treatment, and adherence to the prescribed model.

Another challenge to effectively treating children and youth with mental health needs is 

that services are often fragmented and uncoordinated. Available services are often spread across 

different agencies, and funding streams support only specific types of care or treatment. This

fragmentation limits the ability of providers to meet the full range of needs of children and

families, which may include clinical mental health care, services to help the parents or guardians better support their children, and child and youth development activities to help the children 

and youth acquire protective factors and succeed in school.

Overview of CII

CII operates in three of Los Angeles County’s eight Service Planning Areas: Downtown Los 

Angeles, Watts-South Central, and Torrance-Long Beach. Each of these areas is “high need,” 

which means that a significant proportion of adult residents are low income, have not completed 

high school, have poor physical and mental health, and have experienced or reported abuse or

community violence. In these areas, CII strives to implement a neighborhood approach, where-

 

US Children: Variation by Ethnicity and Insurance Status,” American Journal of Psychiatry 159, 9 (2002): 

1548-1555.

8

The term “usual care” refers to treatments that are not empirically supported. Usual care is a term commonly used in the medical field to refer to the treatment received by patients in the control group of a randomized controlled trial.
ES-4 

by CII builds relationships with residents and institutions, such as schools, churches, and other 

child welfare agencies.

CII’s services are roughly divided into four programmatic categories: clinical mental 

health services, family support, child and youth development, and early childhood care and 

education.

•

•

•

•

Clinical mental health services include diagnosis and assessments of mental 

health needs, individual and group therapy, and family therapy. Licensed 

therapists or psychologists typically deliver these services, which may include evidence-based or evidence-informed practices.9

Family support includes programs offered to parents or guardians. These 

programs address parent education, child development, and family economic 

success and stability through case management, parenting classes, support 

groups for fathers and grandparents, financial literacy workshops, and jobreadiness supports.

Youth development includes nonclinical activities, such as programs for 

young people of different ages that address life skills, social skills, literacy 

and education, creative arts, and health and wellness.

Early childhood care and education services are for infants and children 

from birth to 5 years of age. They include Head Start and child care programs. Though early childhood programming encompasses more than onefifth of CII’s overall budget, these services were not the focus of this SIF initiative, which targeted youth ages 9 to 24 years.

As an operating philosophy, CII coordinates the services it provides to meet the holistic 

needs of children and their families. This approach stands in contrast to the fragmented services 

that often characterize the child welfare system. CII conceptualized this philosophy around three 

components: recovery, resiliency, and readiness.

•

•

Recovery from adverse childhood experiences involves reducing the effects 

of trauma and high-risk behaviors. Recovery is the primary focus of CII’s

clinical services.

Resiliency is the capacity of young people and their families to persevere 

and prevent the effects of trauma, and it is developed by enhancing protective 

 9

Evidence-informed practices are treatments that share characteristics with evidence-based practices but 

fall short of the required threshold of evidence.
ES-5 

factors and reducing risks. It is the primary focus of early childhood, family 

support, and youth development programming.

• Readiness for success in school, work, and life involves positive and healthy 

personal behaviors and social relationships, engagement with education or 

occupational training, and the ability to connect to supports or resources. 

CII’s combined services support readiness. 

Through its Integrated Service Model, CII knits these components together to address 

the complex needs of the families it serves. Depending on their needs, clients may receive 

multiple types of services throughout their involvement with CII. The Integrated Service Model 

aims not to simply offer multiple services but to eliminate operating silos among its various 

services and create a system that accurately identifies clients’ full range of needs and ensures

they receive all the support required to address those needs. This report in large part assesses 

CII’s implementation of the Integrated Service Model.

The CII Evaluation

Building evidence is a core component of the SIF, and each SIF grantee is required to undergo 

an evaluation of its service model. The evaluation of CII consisted of two main components: an 

implementation study of CII’s service model and a study of CII’s delivery of evidence-based 

practices, including an in-depth fidelity study of its delivery of Trauma-Focused Cognitive 

Behavioral Therapy (TF-CBT). Both components focused on understanding how the agency 

implements services to address issues of childhood trauma. Overall, this evaluation provided 

CII staff with an independent review of its services, delivery model, and data system. Furthermore, it serves as a case study for other program operators or policymakers in how to structure 

services to meet the holistic needs of clients and how to overcome the barriers to effectively 

serving children associated with a fragmented child welfare system.

The study attempted to answer three main questions:

•

•

•

How do CII’s services and delivery model meet the needs of the diverse 

population it serves, particularly services provided to children and youth 9 to 

24 years old?

How does CII’s Integrated Service Model combine and coordinate its clinical 

and nonclinical services to address the holistic needs of children and families?

How does CII integrate evidence-based practices, particularly Functional 

Family Therapy (FFT) and TF-CBT, into its array of clinical services?
ES-6 

To answer these questions, the MDRC research team analyzed a mix of quantitative and 

qualitative data. The quantitative data included clients’ demographic data and service participation records for services received during 2012 and 2013; the research team collected these data 

from CII’s management information system. The MDRC research team also gathered qualitative data about program operations through interviews with CII staff and representatives from 

some of its partners, primarily during three site visits in 2013. The team also asked CII clinicians and their supervisors to complete a web-based survey in 2013. In partnership with MDRC, 

a team from the Medical University of South Carolina conducted a fidelity study of CII’s 

delivery of TF-CBT services using an observational method. 

Implementation of CII Services

Analyses of these data point to the following findings:

• CII’s Integrated Service Model is innovative and highly ambitious.

CII’s Integrated Service Model seeks to overcome the shortcomings of the child welfare 

system, but CII staff still had to work within that fragmented system to fund its services. CII 

staff confronted numerous external funding constraints to providing services in the holistic way 

envisioned by the Integrated Service Model. CII funds its programs and services through a 

combination of service fees and contracts. However, each of these funding streams comes with 

a host of requirements and stipulations. As a result, the set of services that a client may need 

does not always fit neatly into one of the available funding streams. To tailor services to each 

client’s needs, CII staff must therefore find flexible funding streams or creatively combine 

contractual or other funding streams. 

• Preliminary analysis shows that CII’s implementation of the Integrated 

Service Model appears to be strong with respect to clients receiving clinical services.

CII strives to provide multiple services to clients to address their many needs and highrisk factors. The overwhelming majority of clients receiving CII’s clinical services also participated in another service at CII, and nearly half of clients in clinical services participated in all 

three types of services. It was not possible to fully assess CII’s progress in implementing the 

Integrated Service Model because the research team did not have access to data about clients’ 

risk factors, which are integral to determining whether clients had unmet needs.
ES-7 

• CII is a leader in adopting and implementing evidence-based practices; 

nearly a third of clients receiving clinical services engaged in an evidence-based practice.10

CII also provides a number of evidence-informed therapies; more than 20 percent of 

clients in the analysis received one of these treatments.11 CII is viewed as a leader in implementing and providing evidence-based practices. As an early adopter, CII first began incorporating 

evidence-based practices into its clinical model in 1999. Little is known about national norms 

for usage of evidence-based practices; according to one estimate of youth receiving care 

through California’s county mental health plans, only 2 percent of youth received an evidencebased practice.12 Importantly, evidence-based practices are neither appropriate for every client 

nor are they available for every age group. While the proportion of CII’s clients receiving 

evidence-based practices exceeds the California estimate many times over, it is difficult to know 

whether this concentration of clients receiving these practices was appropriate without knowing 

more about each client’s circumstances. This study, however, was not designed for such an 

analysis, and the research team did not have the detailed data about each client to conduct one. 

• The dosage levels of both FFT and TF-CBT aligned with model expectations.

Analysis of data from CII’s management information system indicated that, on average, 

clients receiving FFT attended 15 sessions over the course of five months. Similarly, clients 

receiving TF-CBT on average attended 19 sessions over five months. The dosage of both 

therapies fell within the bounds set by their respective treatment models, although both fell on 

the higher end of the models’ acceptable ranges.

13 Data were not available on factors that may 

have contributed to the relatively high number of sessions; however, CII staff indicated in 

interviews that the complex nature of the clients’ history of trauma and tumultuous lives often

resulted in frequent family crises during treatment.

 10Evidence-based practices include the following treatments and programs: Cognitive Behavioral Intervention for Trauma in Schools, Child Parent Psychotherapy, FFT, Incredible Years, Managing and Adapting 

Practice, Multidimensional Treatment Foster Care, Parent-Child Interactive Therapy, Reflective Parenting 

Program, TF-CBT, and Trauma Systems Therapy-Substance Abuse. 11Evidence-informed practices include the following treatments and programs: Domestic Violence Treatment Groups, Project Fatherhood, Wraparound services, Youth with Sexual Behavior Problems, and social 

skills and parent support groups. These practices are informed by some evidence but not as much as evidencebased practices have accumulated. 12Technical Assistance Collaborative and Human Services Research Institute, California Mental Health 

and Substance Use System Needs Assessment Appendices (California Department of Health Care Services, 

2012). Website: www.dhcs.ca.gov/provgovpart/Documents/Data%20Appendices%203%201%2012.pdf. 13The acceptable range for FFT is 8 to 12 sessions; the acceptable range for TF-CBT is 12 to 20 sessions.
ES-8 

• CII’s fidelity to the TF-CBT model was in line with previous fidelity 

studies of TF-CBT in community-based settings. Clients on average 

were more than 50 percent likely to receive half of the treatment’s core 

components.

A rigorous examination of the adherence of therapists at CII to the TF-CBT model using an observational method found that the average client had at least a 50 percent chance of 

receiving half of the core components of the model.14 The finding that clients did not receive all 

components of TF-CBT is consistent with other research on the implementation of TF-CBT in 

community-based settings. Clients were most likely to receive the cognitive coping, relaxation, 

affective expression and modulation, psychoeducation, and trauma narrative components of TFCBT. The study found therapists delivered the parent component at low rates, which is also 

consistent with prior research.15 The study found that fidelity varied at the client level rather 

than the therapist level, indicating that clients seen by the same therapist could have had varying 

experiences with TF-CBT.

The fidelity study also found that a therapist self-report tool, the Brief Practice Checklist, led to similar conclusions about the usage of TF-CBT components as did an observational 

method. This finding indicates that the Brief Practice Checklist may be a promising low-cost 

tool to monitor fidelity. Observational methods of monitoring fidelity, such as the one used in 

this study, are time and resource intensive and not practical on a large scale for many community-based organizations. Therapists and supervisors could use the Brief Practice Checklist to 

monitor whether or not therapists are delivering the TF-CBT components, and supervisors 

could use the information in the checklist to advise therapists on cases and on how to eliminate 

any roadblocks to providing the treatment as intended. Organizations could also use data from 

the checklists to compare differing outcomes among cases and identify and assess any patterns. 

However, there are some limitations to using the checklist on its own to evaluate fidelity. 

Therapists in this study had the tendency to over-report their use of components, relative to the 

observational data. Additionally, observational methods can measure the extent to which 

therapists implement each component, whereas the Brief Practice Checklist can only measure 

whether or not therapists implement the components. However, organizations could use the 

Brief Practice Checklist in combination with others tools, such as periodic direct or audiorecorded observations, to monitor fidelity more comprehensively. The use of the Brief Practice 

Checklist as a fidelity tool merits further study. 

 14A Technical Resource for this report presents the full study and is available on the MDRC website at 

www.mdrc.org. 15TF-CBT requires that the therapist meet separately with the child and the parent or guardian with similar 

frequency, and meet conjointly with both at particular points during treatment.
ES-9 

Conclusion

As policymakers, practitioners, and researchers in the child welfare field work to improve 

services available through the child welfare system, CII and its experience developing and 

implementing its Integrated Service Model as well as delivering evidence-based practices offer 

important lessons. These lessons could be useful not only to similar multiservice organizations 

but to all those in the child welfare field looking for the best ways to serve children through an 

often fragmented child welfare system. Those interested in evidence-based practices may find 

the findings from the fidelity study of TF-CBT useful. These findings suggest that one area for 

further research could be investigating how to cost-effectively combine self-reporting tools and 

observational methods to support fidelity.

1 

Chapter 1 

Introduction

Child abuse and neglect are significant problems in the United States, touching millions of lives 

each year. Traumatic events in childhood can have effects throughout an individual’s lifespan, 

diminishing the chances for a happy, productive life. Child welfare organizations throughout the 

country combat these issues through a combination of prevention programs, direct services to 

affected families, and advocacy.

This report describes how the Children’s Institute, Inc. (CII), a multiservice organization in Los Angeles, combines mental health care and other supportive services to address the 

issue of childhood trauma. Each year, CII serves more than 20,000 children and their families,1

providing them an array of services through its Integrated Service Model. Developed to serve 

the “whole child, entire family,” the model aims to identify, coordinate, and deliver a broad 

range of supports that children and their families may need to overcome a history of abuse or 

trauma. These supports include early childhood care and Head Start programs, clinical mental 

health services, programs for parents and guardians to help them better support their children, 

and youth development activities to help children and youth develop protective factors.2 CII’s 

treatment model is trauma informed and designed to directly address the impact of trauma on 

children’s lives. A central aspect of the model is its focus on using evidence-based practices in

clinical services. Evidence-based practices are highly specified treatment models that research 

has shown to be effective in treating a target population.

In 2010, CII received a Social Innovation Fund (SIF) grant through the Edna 

McConnell Clark Foundation (EMCF) to expand its services, including the youth development 

programs in the Central and South Los Angeles neighborhoods. The initiative focuses on youth 

who are ages 9 to 24 years. (See Box 1.1 for more information on the EMCF SIF initiative.) 

Building evidence is a core component of the SIF, and each SIF grantee is required to 

undergo an evaluation of its service model. The evaluation of CII centered on two main elements: an implementation study of CII’s service model and an in-depth fidelity study of CII’s

delivery of Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) services. 

 1

Children’s Institute, Inc. (2014). 2

Protective factors are characteristics of individuals, families, or communities that mitigate risks to health 

and well-being. Examples include positive social connections, parenting skills and knowledge of child 

development, and effective communication practices.
2 

Box 1.1

The EMCF SIF Initiative

The Social Innovation Fund (SIF), an initiative enacted under the Edward M. Kennedy Serve 

America Act, directs millions of dollars in public-private funds to expand effective solutions 

across three issue areas: economic opportunity, healthy futures, and youth development and 

school support. This work seeks to create a catalog of proven approaches that can be replicated 

in communities across the country. The SIF assembles a three-to-one private-public funding 

match, sets a high standard for evidence, empowers communities to identify and drive solutions to address social problems, and creates an incentive for grant-making organizations to 

target funding to promising programs more effectively. Administered by the Corporation for 

National and Community Service (CNCS), the SIF is part of the federal government’s broader 

agenda to redefine how evidence, innovation, service, and public-private cooperation can be 

used to tackle urgent social challenges.

The Edna McConnell Clark Foundation (EMCF), in collaboration with MDRC and The 

Bridgespan Group, is leading a SIF initiative that aims to enlarge the pool of organizations 

with programs proven to help low-income young people make the transition to productive 

adulthood. The initiative particularly focuses on young people who are at greatest risk of 

failing or dropping out of school or not finding work, who are involved or likely to become 

involved in the foster care or juvenile justice system, who engage in risky behavior such as 

criminal activity, or who experience teenage pregnancy. 

EMCF and its partners selected a first cohort of nine organizations and a second cohort of three 

to receive SIF grants: BELL (Building Educated Leaders for Life), Center for Employment 

Opportunities, Children’s Aid Society-Carrera Adolescent Pregnancy Prevention Program, 

Children’s Home Society of North Carolina, Communities in Schools, Gateway to College 

Network, PACE Center for Girls, Reading Partners, The SEED Foundation, WINGS for Kids, 

Youth Guidance, and Children’s Institute, Inc. These organizations were selected through a 

competitive selection process based on prior evidence of impacts on economically disadvantaged young people, a track record of serving young people in communities of need, strong 

leadership and a potential for growth, and the financial and operational capabilities necessary 

to expand to a large scale.

The EMCF SIF grant, called the True North Fund, includes support from CNCS and 15 private 

co-investors: EMCF, The Annie E. Casey Foundation, The Duke Endowment, The William 

and Flora Hewlett Foundation, The JPB Foundation, George Kaiser Family Foundation, The 

Kresge Foundation, Open Society Foundations, The Penzance Foundation, The Samberg 

Family Foundation, The Charles and Lynn Schusterman Family Foundation, The Starr Foundation, Tipping Point Community, The Wallace Foundation, and the Weingart Foundation.
3 

The implementation study sought to answer three principal questions:

●

●

●

How does CII’s program model and services meet the needs of the diverse 

population it serves, particularly those of children and youth ages 9 to 24 

years?

How does CII’s Integrated Service Model combine clinical services and nonclinical services to address the holistic needs of children and families?

How does CII integrate evidence-based practices, particularly Functional 

Family Therapy (FFT) and TF-CBT, into its various clinical services?

The fidelity study assessed CII’s delivery of TF-CBT and the value of a therapist selfreport tool in monitoring fidelity.

The key findings of the study were:3

●

●

●

●

CII’s Integrated Service Model is innovative and highly ambitious, as it 

seeks to overcome the shortcomings of how child welfare organizations typically operate. CII confronts numerous funding constraints that it must work 

around to provide services in the holistic way envisioned by the model.

Preliminary analysis shows that the Integrated Service Model is well implemented in CII’s clinical services. The overwhelming majority of clients receiving clinical services also participated in another, nonclinical service at 

CII. Nearly half of clients in clinical services also participated in parent or 

guardian programs and youth development activities. 

CII is a leader in adopting and delivering evidence-based practices, and nearly a third of clients engaged in clinical services received an evidence-based 

treatment. However, it is difficult to interpret the saturation of clients receiving evidence-based treatments without more information about each client’s 

circumstances and suitability for such treatments. 

Analysis of data from CII’s management information system records indicated that fidelity to FFT and TF-CBT dosage aligned with the respective models’ prescribed dosages. 

 3

Data from CII’s management information system were analyzed for clients who enrolled between January 1, 2012, and June 30, 2013, and received services January 1, 2012, through December 31, 2013. Interviews 

with CII staff took place over three site visits in 2013.
4 

● The fidelity study found that on average clients were more than 50 percent 

likely to receive half of the core components of TF-CBT, which is in line with

findings from previous fidelity studies of TF-CBT delivered in communitybased settings.

Child Trauma and Treatment

CII’s mission is to help children overcome the adverse effects of trauma on their lives. This 

section provides some background on the issue of childhood trauma, and describes how the 

child welfare system attempts to address the issue and how national and local policies shape the

services provided.

The Effects of Childhood Trauma

The country’s child welfare system confronts serious and widespread threats to the safety of children. The U.S. Department of Health and Human Services reported that in federal 

fiscal year 2012, 3.2 million children were the subject of a report to child protective services. Of 

these reports, 678,810 were substantiated. As the most populous state, it is not surprising that 

California had the largest share of child abuse victims, 76,026, or 11 percent of the nation’s total 

substantiated victims.4

The impact of trauma on children is well documented. Between half and three-fourths 

of youth in the child welfare system exhibit behavioral or social issues serious enough to 

warrant mental health treatment, a rate up to five times greater than that among youth who are 

not involved in the child welfare system.5 Recent trends in the children’s mental health care 

field have indicated that these behavioral and emotional issues can be largely attributed to

trauma experienced earlier in life.6 Trauma can result from many experiences and events, 

including physical or sexual assault, gang violence, domestic violence, serious accidents, 

sudden or violent loss of a loved one, and natural disasters.7 Children do not have to be the 

direct victim of violence to be affected by it; research has shown that exposure to community 

violence has traumatic effects on children.8 In a national survey of adolescents ages 12 to 17

years, nearly 40 percent reported witnessing violence, and close to 25 percent reported being

 4

U.S. Department of Health and Human Services, Administration for Children and Families, Administration on Children, Youth and Families, Children’s Bureau (2013). 5

This estimate is based on studies across child welfare systems in several states, as well as recent results of 

the National Survey for Child and Adolescent Well-Being. See Landsverk, Burns, Stambaugh, and Rolls Reutz 

(2006). 6

Silverman et al. (2008); Felitti et al. (1998). 7

Child Welfare Committee, National Child Traumatic Stress Network (2008). 8

Osofsky (1998).
5 

physically assaulted.9 Children and youth in the child welfare system are particularly at risk and 

have often experienced multiple traumatic events over time, with the event that precipitated a 

report to child protective services generally being the last in a long series.

Studies have demonstrated that exposure to traumatic events has many adverse shortand long-term effects on children and youth. Psychologically, children and youth referred to 

child welfare experience high rates of post-traumatic stress disorder, with estimates ranging 

from 11 to 60 percent.10 These youth also exhibit high rates of depression, anxiety, and attention 

deficit hyperactivity disorder.11 These psychological and emotional responses often manifest as 

antisocial or otherwise maladaptive behaviors, which may in turn create more issues for them in 

school or in their foster families.

12

Experiencing traumatic events as a child can also have negative effects on physical 

health.13 The continual “toxic stress response” caused by traumatic events, particularly early in 

life, can have detrimental effects on brain development and may cause learning difficulties 

(which, in turn, can lead to negative psychological effects such as low self-esteem).14 Substance 

abuse is another major health problem affecting this population; youth and adults who experienced childhood trauma are more likely to develop substance use disorders.

15 Adults who

experienced multiple traumatic experiences in childhood are at significantly higher risk of 

developing substance use disorders, depression, and obesity; smoking tobacco; and contracting

sexually transmitted diseases.16

Children living in poverty are particularly at risk. They are more likely to experience 

trauma and to experience multiple traumas over the course of their childhood than their more 

affluent peers. They are also less likely to have access to resources that address the negative 

effects of trauma. Their parents, who are likely experiencing stress from living in poverty as 

well as trauma of their own, are less likely to be effective in helping their children with their 

trauma-related challenges.17

CII’s Integrated Service Model is designed to address both the short- and long-term effects of trauma on children and their families. As discussed in more detail in Chapter 5, CII 

 9

Kilpatrick and Saunders (1999).

10Kolko et al. (2010); Dubner and Motta (1999); Éthier, Lemelin, and Lacharité (2004); Marsenich (2002). 11Ackerman et al. (1998); Hildyard and Wolfe (2002); Marsenich (2002); Weisz and Gray (2008). 12Richardson, Henry, Black-Pond, and Sloane (2008); Child Welfare Committee, NCTSN (2008). 13Felitti et al. (1998).

14Center on the Developing Child at Harvard University (2010).

15Felitti et al. (1998); Keller, Salazar, and Courtney (2010); Landsverk, Burns, Stambaugh, and Rolls 

Reutz (2006). 16Felitti et al. (1998).

17Collins et al. (2010).
6 

provides services directed at helping children recover from trauma (recovery), build protective 

factors such as coping strategies to prevent future trauma (resiliency), and prepare for productive lives (readiness).

Evidence-Based Treatment Practices

While the lifelong impact of childhood mental illness and trauma is well documented, 

many children and youth do not receive the mental health treatment they need. Research has 

shown that many young people in need of mental health care either do not receive services, or, 

when they do, receive services that are inadequate or ineffective.18 Another challenge to 

effective treatment is that children often receive fragmented and uncoordinated services because 

providers are spread across agencies and funding streams support only specific types of care or 

treatment.

The mental health treatment that trauma-affected children receive is often not empirically supported. This report uses the term “usual care” to describe mental health care that is not 

evidence based.

19 Therapists have diverse training backgrounds and therapeutic approaches, and

thus what usual care typically entails can vary widely; there is also little research on usual care, 

which makes it challenging to categorize and assess the various types of treatment it typically 

encompasses. Therapists often mix and match approaches to tailor treatment to the individual. 

One example of usual care is psychodynamic therapy, which focuses on helping clients uncover 

the unconscious content of their psyche that underlies their issues. Another example is play 

therapy, which uses play to help children process difficult life experiences. In contrast, an 

evidence-based treatment is prescribed; while therapists may have slightly different clinical 

approaches, they administer the same treatment to any client who is prescribed it. 

At the federal level, there is growing momentum to increase the use of evidence-based 

practices in children’s mental health care. Evidence-based practices are those that research has 

shown to be effective. An evidence-based treatment model is well specified, defining the target 

population for which it has been proven to be effective, the intended outcomes, and its content, 

dosage, and duration. However, as such, a given evidence-based practice may not be appropriate for every client since the target population that benefits from it may be a subgroup of clients, 

for example, a particular age group or a group with specific symptoms.

While evidence-based practices are gaining ground in the child welfare field, there is 

still a lack of consensus about what criteria should be used to determine whether a treatment is 

 18Burns et al. (1995); Kataoka, Zhang, and Wells (2002). 19The term “usual care” in this report refers to treatments that are not empirically supported. Usual care is 

a term commonly used in the medical field to refer to the treatment received by patients in the control group of 

a randomized controlled trial.
7 

evidence based. For the purposes of this report, the research team used the criteria that 

Chambless and Hollon delineate in their 1998 study.20 Among other criteria, Chambless and 

Hollon set the threshold of evidence for such a practice at two randomized control trials, 

conducted by independent researchers.21 In this report, the term “evidence-informed practice” 

refers to a practice that shares characteristics with evidence-based practices but falls short of this 

threshold of evidence.

Fidelity is a major challenge to implementing evidence-based practices. Whether or not 

an evidence-based treatment is effective largely depends on how well service providers implement it. In order to “transfer” efficacy from research to practice, providers must implement the 

treatment with fidelity to the model that was originally tested and proven to be effective. 

Evidence-based practices typically include an implementation plan, or a set of instructions.22

This plan may include guidelines for recruiting and training staff, monitoring the implementation, and evaluating clients. Fidelity comprises several elements. One way to assess fidelity is to 

compare the implementation plan for a specific treatment with a provider’s actual implementation. For example, researchers may compare the planned number of hours of staff training with 

the actual numbers of training that staff received. Another way to assess fidelity is to examine 

treatment fidelity, or the extent to which the treatment received by the client matches the 

intended treatment. For example, if the treatment called for a client to receive a curriculum 

consisting of five units, an evaluation of treatment fidelity would assess the extent to which the 

client received those five units as planned.

Providing evidence-based practices with fidelity presents several challenges. The practices require that providers adhere to the prescribed treatment methods, which may be a difficult 

adjustment for a provider accustomed to a different approach. Community-based settings, such 

as those where CII operates, pose particular challenges since they are often quite different from 

the university research settings where the practices are usually first developed and tested. 

Community-based providers typically serve more diverse and higher-risk populations and

manage larger caseloads.

Regarded by many as an early adopter, CII has embraced evidence-based and evidenceinformed practices and has been integrating them into its clinical services since 1999. For this 

reason, this report focuses in part on CII’s implementation of evidence-based practices in its 

clinical services. However, since trauma-affected children and families often need other 

 20Chambless and Hollon (1998). 21Chambless and Hollon (1998). Other entities use more relaxed criteria. Blueprints for Healthy Youth 

Development’s criteria for model programs requires two randomized controlled trials or one randomized 

controlled trial and one quasi-experimental design. See Blueprints for Healthy Youth Development (2015). 22Weiss, Bloom, and Brock (2013).
8 

services, such as case management and coaching for parents or guardians, the report also 

assesses CII’s additional supports.

CII takes a holistic approach to health care, combining mental health treatment with 

wraparound supports for families to meet the needs of the “whole child, entire family.” Called 

the Integrated Service Model, the approach includes clinical mental health services, programs 

for parents and guardians to help them better support their children, and youth activities to help 

children and youth develop protective factors.

The Role of National and Local Policies

The national and local policy context plays a significant role in how providers serve 

children and families in the child welfare system. Policies at the federal, state, and local levels 

can act as both facilitators and barriers to the successful implementation of interventions by 

organizations directly serving children.

At the federal level, government agencies and nongovernmental entities help shape policy related to the treatment and services children and families in the child welfare system 

receive and how those services are delivered and funded. National initiatives are responsible for 

the development of many of the evidence-based practices discussed in this report.

The policy context in California, including laws and policies at the state and county 

levels, also has a major impact on how mental health services are funded and delivered. State 

policy constitutes much of the structure for how counties provide child welfare and mental 

health services. Counties also enact their own specific requirements and programs, and 

providers such as CII must navigate this complex system. Chapter 2 discusses these issues in 

more detail. 

Overview of This Report

Building evidence is a core component of the SIF, and each SIF grantee is required to undergo 

an evaluation of its service model. Early discussions of the evaluation design for CII centered 

on conducting a randomized controlled trial. However, a randomized controlled trial proved to 

be untenable because it would have meant withholding treatment from children. Moreover, the 

research team determined that it was not necessary to conduct an experimental evaluation since

a significant portion of CII’s clients receive evidence-based treatments that research has already 

shown to be effective. Accordingly, the premise of the evaluation design is that if CII is implementing the evidence-based treatments with fidelity, then it can be reasonably assumed that 

these treatments are achieving the intended results. The evaluation design thus called for indepth fidelity studies of CII’s evidence-based practices.
9 

The evaluation design initially called for fidelity studies of CII’s three evidence-based 

practices, which according to available data nearly 50 percent of CII’s clients 9 to 24 years old 

were receiving. However, during the early stages of the evaluation, the research team decided to 

drop the fidelity study of the Cognitive Behavioral Intervention for Trauma in Schools because 

of small sample sizes and scaled back the study of FFT. Chapter 5 describes the remaining 

fidelity study of Trauma-Focused Cognitive Behavioral Therapy (TF-CBT). The in-depth 

fidelity study of TF-CBT assessed (1) the degree to which CII implemented TF-CBT with 

fidelity to the treatment model (treatment fidelity), (2) how fidelity might vary by client or 

therapist characteristics and (3) the utility of the TF-CBT Brief Practice Checklist in monitoring 

therapist fidelity.

23 The MDRC research team managed the data collection, and a team from the 

Medical University of South Carolina oversaw the data analysis. 

The evaluation design also included a broader implementation study of CII’s service

model to provide context for the fidelity studies. Chapter 4 presents the findings from the 

implementation study. The implementation study set out to (1) describe how CII’s services

model and services meet the needs of the diverse population it serves, with a focus on services 

provided to youth 9 to 24 years old (2) describe how CII’s Integrated Service Model combines 

clinical and nonclinical services to address the holistic needs of children and families, and

(3) evaluate how CII implements evidence-based practices in its clinical services.

Data Collection and Data Sources for the Implementation Study

The MDRC research team collected various forms of data for the implementation study covering the years 2012 through 2014.24 The team analyzed the data in 2014 and 2015.

Data sources for the implementation study include:

● Three site visits to CII in 2013, during which the research team conducted 

semi-structured interviews with 83 CII staff, including executive staff, clinicians, and supervisors.25 MDRC researchers also conducted semi-structured 

interviews with staff from eight other agencies in Los Angeles, including the 

Los Angeles County Department of Mental Health and providers serving 

similar populations.

 23The TF-CBT Brief Practice Checklist is a self-monitoring tool for therapists to assess their fidelity to the 

treatment model. 24The research team secured institutional review board (IRB) approval from MDRC and CII to conduct 

interviews with CII staff. All staff who participated in interviews did so only after consenting. 25Visits occurred in February, May, and October of 2013.
10

●

●

●

A web-based survey of CII clinicians and their supervisors conducted in 

2013. The research team fielded the survey to 69 therapists and psychologists 

and 27 supervisors. The response rate was 39 percent.

Documents provided by CII, including assessments and outcome measures 

and promotional materials.

Records from CII’s management information systems, which included separate databases for clinical records, family support and youth development activities, outcome data, and staff records. The research team analyzed the data 

for services delivered in the calendar years 2012 and 2013.26

Appendix A describes the data sources for the implementation study, and some of their 

limitations, in more detail. 

Data Collection and Data Sources for the Fidelity Study

Research subjects in the fidelity study included new TF-CBT clients and their assigned therapists.27 Enrollment for the fidelity study began in November 2013 and ended in August 2014. A 

total of 126 TF-CBT clients and 34 therapists enrolled in the study.

Data sources for the fidelity study include:

●

●

●

●

Demographic data on 126 clients enrolled in the study

Demographic, education, and caseload data on 34 therapists enrolled in the 

study

108 TF-CBT Brief Practice Checklists

1,009 audio recordings of client therapy sessions

 26In 2014, CII replaced the management information system from which the research team collected data 

for this report. CII expects the system, which incorporates new checks and balances to improve data quality, to 

alleviate many challenges. 27The research team obtained IRB approval to conduct the research from MDRC, CII, and the Medical 

University of South Carolina. The team secured additional approval from the Los Angeles County Juvenile 

Court and Department of Mental Health Human Subjects Review Committee.
11

Roadmap to the Report

Chapter 2 provides an overview of CII’s organizational structure, including its history in the 

communities it serves, operating philosophy, and funding mechanisms. It explains recent SIF 

involvement and business planning that resulted in strategic changes.

Chapter 3 describes the target population that CII serves, with an emphasis on 9- to 24-

year-olds, and the pathways through which clients enter CII services. It also explains the client 

referral, assessment, and enrollment processes. 

Chapter 4 describes the implementation of CII services, including the staffing structure. 

It also explains the Integrated Service Model in detail. 

Chapter 5 examines CII’s implementation of evidence-based practices, including TFCBT and FFT. The chapter assesses CII’s implementation of these evidence-based practices, 

and describes some barriers and facilitators to implementation. The chapter summarizes the 

findings from the TF-CBT fidelity study. (A Technical Resource for this report presents the full 

study and is available on the MDRC website.) 

Chapter 6 summarizes the report’s conclusions and identifies areas for further research.

13

Chapter 2 

Overview of CII

The Children’s Institute, Inc. (CII), is a large organization with an operating budget of $50 

million, serving more than 20,000 children and family members each year at three multiuse 

campuses and dozens of centers and child care facilities throughout Los Angeles. This chapter 

begins with a summary of CII’s long and varied history, followed by a brief description of the 

communities it currently serves. The chapter then gives an overview of CII’s organizational 

structure, including the main services it provides, its operating philosophy and Integrated 

Service Model, and funding sources. It concludes with a description of how CII incorporates 

evidence-based practices into its clinical services. 

History and Background of CII

Founded in Los Angeles in 1906 and originally named Big Sister League, CII began as a 

home for unwed pregnant women. The organization shifted the focus of services to child care 

in the 1960s, as more of the pregnant single women it served chose to keep their children 

rather than put them up for adoption. In the 1970s, CII started providing services to at-risk 

children in response to the growing need for child abuse prevention services. The organization changed its name to the Children’s Institute International in 1980 and continued to 

expand its services for at-risk children, including mental health services, in subsequent 

decades to become the large multiservice organization it is today. In 2006, it changed its 

name, again, to Children’s Institute, Inc.

The Communities CII Serves

CII operates in three of Los Angeles County’s eight Service Planning Areas (SPAs): SPA 4 

(Metropolitan Los Angeles), SPA 6 (South), and SPA 8 (South Bay). Figure 2.1 shows the 

SPAs that CII serves. In each SPA, CII operates multiple office locations. In these communities, 

CII strives to implement a neighborhood approach, whereby CII builds on connections with 

residents and other institutions in the community, such as schools, churches, and other child 

welfare agencies. While all the communities CII serves are considered high need, there are 

nonetheless regional variations among them with respect to socioeconomic demographics. (See 

Table 2.1.) 

CII has the largest and most significant presence in SPA 4, where its flagship Otis 

Booth campus opened in 2011. The Otis Booth campus serves as CII’s headquarters; in addition
14

to extensive client services, the location houses many of CII’s managerial functions, including 

its human resources, training, billing, and research and evaluation departments. SPA 4 is the 

most densely populated of the three areas where CII operates and includes the neighborhoods of 

downtown Los Angeles, Rampart, Echo Park, and Silverlake. Its poverty rate is 25 percent, 

higher than average in Los Angeles County. CII also operates the Mid-Wilshire campus in SPA 

4, which is located near Koreatown. As part of CII’s Social Innovation Fund (SIF) grant, CII

expanded its youth development programs in this service area.

SPA 6 includes the neighborhoods of Compton, Crenshaw, and Watts in South Central 

Los Angeles, and has the poorest socioeconomic indicators of the three SPAs where CII 

operates. The poverty rate is 31 percent, the highest in Los Angeles County. As part of CII’s

Figure 2.1 

Map of Los Angeles County Service Planning Areas Served by CII

Service Planning Areas 

1 – Antelope Valley

2 – San Fernando

3 – San Gabriel

4 – Metro

5 – West 

6 – South

7 – East

8 – South Bay 

Shaded regions served by CII
(continued)

Table 2.1

Key Indicators of Health by Service Planning Area

15

Los Angeles

Characteristic National County SPA 4 SPA 6 SPA 8

Age (%)

0-5 years 7.9 8.0 7.2 10.6 8.1

6-17 years 16.2 16.8 13.2 21.0 16.9

18-39 years 29.8 32.8 39.1 34.6 31.5

40-64 years 33.2 31.6 30.2 26.4 32.7

65 years or older 13.0 10.7 10.2 7.4 10.9

Race (%)

Latino 16.3 48.1 52.2 67.7 39.5

White 64.8 28.9 24.9 2.0 29.7

African American 12.8 8.5 4.7 28.5 14.8

Aisan/Pacific Islander 5.2 14.3 18.0 1.6 15.8

American Indian/Alaskan Native 0.8 0.2 0.2 0.1 0.2

Education (%)

Less than high school 14.3 23.2 27.6 38.8 18.9

Completed high school 28.5 22.3 22.9 24.2 25.6

Some college, trade school, or associate's degree 31.2 27.9 21.8 25.5 28.4

College or post-graduate degree 26.0 26.6 27.6 11.5 27.0

Poverty (%)

Household income less than 100% Federal Poverty Level 15.9 18.0 25.0 31.1 17.2
Los Angeles

Characteristic National County SPA 4 SPA 6 SPA 8

Parental support (%)

Children ages 0-5 years whose parents say they can easily find someone

to talk to when they need advice about raising their children N/A 87.1 80.0 83.9 89.9

Child care (%)

Children ages 0-5 years for whom parents report difficulty finding child

care (excludes 23.4% who reported they do not need child care) N/A 26.9 30.4 36.7 19.2

Physical and mental health (%)

Adults reporting their health to be fair or poor 16.1 20.7 24.5 30.5 17.6

Children ages 3-17 years who tried to access mental or behavioral health 

care in the past year N/A 7.8 8.1 5.8 8.3

Adults who tried to access mental health care in the past year N/A 7.5 8.4 6.6 6.5

Adult mental health (%)

Ever diagnosed with depression 17.5 12.2 13.4 10.8 10.7

Currently diagnosed with depression N/A 8.3 9.3 8.0 7.7

At risk for major depression N/A 10.4 11.6 13.3 9.3

Ever diagnosed with anxiety N/A 11.3 12.0 10.1 10.2

Currently diagnosed with anxiety N/A 6.4 7.4 6.9 5.5

Insurance (%)

Children ages 0-17 years who are uninsured 7.0 5.0 6.6 8.6 2.9

Adults ages 18-64 years who are uninsured 21.3 28.5 35.5 38.2 26.7

Regular source of care (%)

Children ages 0-17 years with no regular source of health care 3.3 4.8 5.2 7.3 4.5

Adults ages 18-64 years with no regular source of health care N/A 23.4 25.4 29.4 21.0

Table 2.1 (continued)

SOURCE: Los Angeles County Department of Public Health (2013).

16
17

SIF grant, CII set out to intensify its efforts and expand and solidify its presence in the Watts 

neighborhood. During the study period, CII staff in SPA 6 operated out of several offices spread 

across the neighborhood and they lacked space for many youth development activities. However, CII has been planning to build a campus similar to the Otis Booth facility in Watts. As a 

relative newcomer to SPA 6, CII has faced challenges in building its presence and earning trust 

in the community. 

SPA 8, located on the southern tip of Los Angeles County, encompasses a much larger 

geographical area than either SPA 4 or 6 and includes the communities of Torrance and Long 

Beach. In 1992, CII expanded its services to SPA 8, where it continues to operate out of 

multiple locations. SPA 8 boasts better socioeconomic indicators than SPAs 4 and 6. Its overall 

poverty rate is just 17 percent. CII’s offices in SPA 8 are more geographically spread out than in 

the other SPAs, and there are no plans to establish a main campus.

SPAs 4, 5, and 8 are some of the more service-rich areas in Los Angeles County, particularly SPA 4. A number of other organizations in these areas have contracts with the Los 

Angeles County Departments of Mental Health and Child and Family Services to serve children 

and families. Similar to CII, these other organizations provide mental health services to children 

and youth. Apart from that, their services vary considerably; they offer different prevention or 

supportive services, mental health services for adults, or services for non-English-speaking

clients, or they operate at a different scale. Yet while these and other organizations may provide 

similar services throughout Los Angeles County, CII has been described by individuals outside 

the organization as offering a rich set of core mental health services, evidence-based practices, 

and cultural adaptations to better serve clients from Spanish-speaking and several other foreignlanguage communities.

Overview of CII’s Services and Operating Philosophy

CII’s services are roughly divided into four programmatic categories: clinical mental health 

services, family support, youth development, and early childhood care and education.

● Clinical mental health services include diagnostic treatment and needs assessment, individual and group therapy, and family therapy. Licensed therapists or psychologists typically deliver these services. They may include evidence-based and evidence-informed practices. As discussed later in this 

chapter, most clinical services are provided through contracts with the Los 

Angeles County Departments of Mental Health and Children and Family 

Services.
18

●

●

●

Family support includes programs offered to parents or guardians. These 

programs address parent education and child development, and family economic success and stability through case management, home visitations, parenting classes, support groups for fathers and grandparents, financial literacy 

workshops, and job-readiness supports.

Youth development includes nonclinical programs for young people of different ages that address life skills, social skills, literacy and education, creative arts, and health and wellness.

Early childhood care and education services are for infants and children 

from birth to 5 years of age. They include Head Start and child care programs. Though early childhood programming encompasses more than onefifth of CII’s overall budget, these services were not the focus of this SIF initiative, which targeted youth ages 9 to 24 years. 

As an operating philosophy, CII coordinates the services it provides to meet the holistic 

needs of children and their families. This approach stands in contrast to the fragmented services 

that often characterize the child welfare system. CII conceptualized this philosophy around three 

components that work together to support a child’s well-being: recovery, resiliency, and 

readiness.

●

●

●

Recovery from adverse childhood experiences involves reducing the effects 

of trauma and high-risk behaviors. Recovery is the primary focus of CII’s

clinical services. Programs and services that support parents and guardians in 

helping their children also support recovery.

Resiliency is the capacity of children and families to persevere and prevent 

the effects of trauma and is developed by enhancing their protective factors

and reducing risks.1 It is the primary focus of early childhood, family support, and youth development programming.

Readiness for success in school, work, and life involves positive and healthy 

personal behaviors and social relationships, engagement in education or occupational training, and the ability to connect to needed supports or resources. CII’s combined services support readiness.

 1

Protective factors are characteristics of individuals, families, or communities that mitigate risks to health 

and well-being. Examples include positive social connections, parenting skills and knowledge of child 

development, and effective communication practices.
19

Figure 2.2 depicts the conceptual framework of these services and their intended shortand long-term outcomes. Underlying mediating factors affect the successful delivery of these 

services, including the effectiveness of and providers’ fidelity to the evidence-based and 

evidence-informed practices, the trauma lens that the service model applies to all its components, the funding context, and the clients’ characteristics.

Structure of Clinical Mental Health Services

Clinical mental health services fall into two main programmatic category, which are associated with specific funding streams. Each category includes a variety of therapeutic approaches, which may or may not be evidence based:

●

i

●

i

Community Mental Health Services are typically for clients with lessntensive needs. Clients see a therapist at a CII office, typically weekly, and 

may receive an evidence-based or evidence-informed practice. The treatment 

may be a standalone service, though some families may receive some case 

management services from a care coordinator at CII.

Intensive In-Home Services tend to be for higher-need clients who likely

have an open case with the Los Angeles County Department of Children and 

Family Services. The Los Angeles County Department of Mental Health or 

Department Children and Family Services supports these services, which

Box 2.1 describes in more detail. Several different service programs belong 

n this category, including Full Service Partnership, Field Capable Clinical 

Services, and Wraparound services.

2 Though the specific components of 

each program vary, children and their families generally receive therapy, intensive case management, and support from a team of staff who are available 

24 hours a day, 7 days a week.

Structure of Family Support

Parents or guardians of children participating in any mental health service program may 

receive CII’s supportive services or enroll in a CII program or class not directly connected to the 

children’s clinical services. For example, parents or guardians of children enrolled in Full 

Service Partnership, a clinical service, receive supportive services through assigned “parent 

partners,” who coach them on how to better support their children, such as on how to advocate 

for their children at school or better deal with behavioral issues. Examples of programs in which

 2

The Los Angeles County Departments of Mental Health and Children and Family Services also have 

contracts with other organizations in the county that provide these same services.
Figure 2.2

CII’s Integrated Service Model Conceptual Framework

Referral

Assessment, 

screening, 

and 

diagnosis

Program

entry

Children’s Institute Services

Clinical Mental Health 

(for young people) 

• Individual, group, and 

family therapy

• Intensive in-home 

treatment or not

Family Support (for 

parents or guardians)

• Parenting classes

• Support groups

• Financial literacy 

workshops

• Case management

Youth development 

• “fun” activities

• Life/social skills

• Literacy and education

• Creative arts

• Health and wellness

Trauma lens

Funding context

Client characteristics (for example, demographics, family dynamics, school environment, community)

Long-term

outcomes

Youth are 

emotionally healthy 

and exhibit prosocial behavior

Youth achieve 

educational and 

occupational 

potential

Youth live in safe 

and stable homes

Youth have 

enhanced quality of 

life

Short-term

outcomes

Decreased risk 

factors

Decreased posttraumatic stress, 

depression, anxiety

Decreased behavior 

problems

Decreased feelings 

of shame and 

mistrust

Decreased sexualized 

behaviors

Decreased behavior 

problems in siblings

Increased capacity to 

support children

Evidence-based and evidence-informed practices

Early Childhood Care 

and Education

• Head Start

• Child care 

20
21

Box 2.1

Programs of the Los Angeles County Departments of Mental Health

and Children and Family Services that CII Provides

Wraparound is an intensive services program designed for high-risk youth in foster care 

placement, at risk of placement, or in urgent need of mental health services. The Los Angeles 

County Department of Children and Family Services or juvenile justice services refer youth to 

the program. The Wraparound team consists of a therapist, a child and family specialist, and a 

parent partner, and meets regularly with the family in home. The team is also on call 24 hours 

a day, seven days week, working collaboratively to ensure that the child lives in a safe and 

permanent home environment. CII generally provides Wraparound services for 18 months, but 

sometimes for up to two years. 

Full Service Partnerships are designed for underserved children from birth to age 15 years. 

The Los Angeles County Department of Mental Health refers these children, many of whom 

are homeless, have been hospitalized, or are at risk for hospitalization. The Full Service 

Partnership team includes a therapist and a parent partner, who provide crisis intervention 

services and family counseling 24 hours a day, seven days week. Full Service Partnerships are 

designed to do “whatever it takes” to help families secure their well-being, safety, and stability. 

Full Service Partnerships typically last from 9 to 12 months but families can receive services 

for up to two years.

Family Preservation provides support for families after an allegation of child abuse or neglect 

that is determined to be unsubstantiated. The service focuses on helping families remain 

together, live safely in the same home, or work toward family reunification. 

Field Capable Clinical Services are intensive services for families who are unable to access 

services in a traditional mental health care setting. Services may be provided in the home, 

schools, or community centers. They are appropriate for clients who are lower risk than those 

receiving Wraparound or Full Service Partnership services, but who may have experienced 

trauma and require intensive support. Field Capable Clinical Services can be used to transition 

clients completing a Full Service Partnership by allowing them to continue working toward 

their recovery goals.

Intensive Field Capable Clinical Services are services offered through a program launched 

in 2013, to accelerate access to Medi-Cal services for children in need of intensive mental 

health services.* Children from birth to age 15 years receive immediate access to Intensive 

Care Coordination as well as Intensive Home Based Services, two interventions that address 

serious mental health conditions. Intensive Field Capable Clinical Services are provided within 

24 hours of referral and are available to clients 24 hours a day, seven days a week.

__________________________

*Los Angeles County Department of Mental Health (2015).
22

parents or guardians may participate regardless of whether or not their children are receiving

clinical services include parenting programs such as Project Fatherhood and financial literacy 

classes.

Structure of Youth Development Services

Children and youth can access youth development services whether or not they are receiving clinical services. Examples of these activities include theater, drawing, or dance classes 

and soccer or basketball. As Chapters 3 and 4 describe in more detail, youth development 

services may precede a client’s enrollment in clinical services or be a client’s “step-down” 

support, which a client receives after completing clinical services. Eligibility requirements for 

youth development services are less restrictive than they are for clinical services, and these 

activities particularly target residents of the neighborhoods near CII locations.

Combining Services Through the Integrated Service Model

Through its Integrated Service Model, CII attempts to coordinate clinical, family support, and youth development services to best address the complex needs of the children and 

families it serves. Clients may receive multiple types of services, depending on their needs. The 

Integrated Service Model aims not to simply offer multiple types of services but to eliminate

operating silos among CII’s various services and to create a system that accurately identifies

clients’ full range of needs and ensures they receive all the support required to address those

needs. Chapter 4 describes the implementation of the Integrated Service Model in depth.

CII’s Funding Structure

How CII funds its services has a major impact on how it implements those services as well as 

the Integrated Service Model. Policymakers at the federal, state, and county levels make 

decisions about what services to provide and how to pay for them, and these decisions trickle 

down to direct service providers such as CII. While the Integrated Service Model aims to 

coordinate the fragmented services typical of the child welfare system, CII must work within 

that system to fund its services, weaving together a complex set of contracts, grants, and other 

sources of revenue.

Staff at CII and partner agencies described how the funding structure, including stateand county-level policies, has a huge impact on what services they are able to provide. For 

example, California’s Mental Health Services Act allocates funds to county mental health 
23

departments for Prevention and Early Intervention services,

3 of which Los Angeles County 

dedicates 20 percent of these funds to evidence-based practices. CII is one recipient of these 

funds. Box 2.2 summarizes important laws and policies in California that affect child welfare 

services. 

The majority of CII’s revenue (84 percent) comes from contractual allocations and service fees associated with providing services. These revenue streams include allocations through 

contracts with the Los Angeles County Departments of Mental Health and Child and Family 

Services and reimbursements for services from clients’ health insurers, most often Medi-Cal, 

California’s health coverage for low-income children and adults.4 The remaining 16 percent 

comes mostly from fundraising efforts.

CII has numerous contracts to provide services with several agencies, each of which has

its own requirements. CII’s single largest source of revenue comes from its contract with the 

Los Angeles County Department of Mental Health, which allocates funding through 18 different streams or “buckets.” Each of these buckets has requirements about eligible services and the 

target population that receives those services. In addition, each bucket may have further restrictions on how funds can be used. Spending all the allocations from contracts with multiple 

buckets and complex requirements can be a challenging calculus, and CII is always at risk of 

not fully expending these allocations. For example, CII may encounter a situation in which it 

would like to offer a client a particular evidence-based treatment but it does not have enough 

funding in that bucket. Conversely, CII may not be able to use all the funding available in a 

bucket because it does not have enough staff to deliver the full amount of a specific service or 

treatment that the available funding covers. 

Funding constraints can also affect which referrals CII can accept, since CII may receive referrals for clients who need services for which it does not have enough funding to 

provide. In these cases, CII will refer the clients to another organization. On the other hand, CII 

can find itself at risk of not spending all available funding earmarked for specific services if it 

lacks clients who meet the stipulated requirements, or if clients drop out of treatment toward the 

end of the contract period.

Ultimately, funding constraints present huge obstacles to CII in achieving the outcomes 

envisioned by the Integrated Service Model, which requires nonclinical, or “add-on,” supports 

such as youth development activities and programs for parents and guardians. Many of these

 3

Prevention and Early Intervention services are targeted at persons who are at the early stages of mental 

illness, before official diagnosis of mental illness or in the very early stages. Services are generally low

intensity. 4

The majority of CII’s clients are covered through Medi-Cal.
24

add-on services are not billable to their clinical funding streams, which is CII’s main source of 

revenue, making grants and other private support critical to the implementation of the Integrated 

Service Model. The SIF grant has been essential to the Integrated Service Model, as it was the 

primary support for CII’s expansion of its youth development activities.

Box 2.2

California Policy Context

The Mental Health Services Act (MHSA),

* passed in 2004, was designed to expand county 

mental health services across the state of California. The act requires 20 percent of funds to be 

used for Prevention and Early Intervention services, many of which assist children. It also led 

to the implementation of new services, including Full Service Partnerships. In Los Angeles 

County, a portion of MHSA funding is used for evidence-based practices.

Medi-Cal† is the California Medicaid welfare program and provides support to people with 

limited ability to pay for health coverage, including low-income adults, families with children, 

children in foster care, and former foster youth. Medi-Cal beneficiaries are members of their 

county’s Mental Health Plan, through which they can access county-administered mental 

health services.

California Work Opportunity and Responsibility for Kids (CalWORKs),

‡ California’s 

welfare program, provides assistance to families with children who have been deprived of 

parental support. CalWORKs participants have access to Medi-Cal (Medicaid) and other 

programs, through which they may receive mental health services for themselves or their 

children. In 2013, CalWORKs provided assistance to 50 percent of children living in poverty 

in California.

Child Abuse Prevention, Intervention and Treatment (CAPIT)§ was a law passed in 1982 

aimed at preventing and treating child abuse and neglect. It provides a funding stream for 

programs that provide support services such as child care, mental health services, parental 

education and support, counseling, and screening. Priority for CAPIT funding is given to 

private nonprofit agencies that serve children at risk of abuse or neglect. Counties are responsible for monitoring projects and reporting annually to the state’s Office of Child Abuse 

Prevention.

Realignment,

|| a state law enacted in 1991, shifted responsibility for running mental health 

programs from the state to the county level. It was intended to provide a stable funding source 

and client-centered approach to mental health services. In 2011, a second realignment allocated a portion of a state sales tax to each county’s substance abuse and children’s mental health 

programs, making counties responsible for funding and administering their mental health 

programs.

(continued)
25

History and Development of Evidence-Based Practices at CII

CII is widely considered to be a leader in implementing and delivering evidence-based practices. As an early adopter, CII first began incorporating evidence-based practices into its clinical 

model in 1999, focusing on just one model. Before that, according to one CII staff member, 

“people were just doing general therapy in their offices.” Staff members reported that the major

shift to evidence-based practices started in 2004 when CII joined the National Child Traumatic 

Stress Network and became involved in its efforts to develop and disseminate evidence-based 

practices for children recovering from trauma.

Internal and external forces drove CII’s shift to incorporate evidence-based practices into its clinical model. Internally, senior managers had bought into and championed the approach. 

Some staff members saw evidence-based practices as a way to provide better, more targeted 

services to clients. As one staff member put it, “[e]videnced-based treatments allowed us to do 

more focused treatment and to ask the right questions early on.”

Externally, the Los Angeles County Department of Mental Health had been pushing for 

shorter-term, evidence-based treatment models for quite some time, and this pressure intensified 

in 2009, when the Mental Services Act began allocating additional funding for Prevention and 

Early Intervention. “Now, it’s financially essential. You have to be able to do that [evidencebased practices] or you’re going to miss out on a tremendous amount of funding,” explained a 

manager at CII. However, staff members also reported that, as an early adapter, CII made the 

transition to evidence-based practices more easily than did other organizations in Los Angeles.

Box 2.2 (continued)

Katie A. v. Bonta,

# a lawsuit settled by the state of California in 2011, dictates how services 

are provided to children in foster care or at risk of entering foster care. The settlement calls for 

California Department of Health Care Services to implement statewide standards and monitor

mental health services. As a result of the lawsuit, California made available three types of 

mental health services: Intensive Home-Based Services, Intensive Care Coordination, and 

Therapeutic Foster Care. An additional program, Intensive Field Capable Clinical Service, was 

launched to facilitate access to these services.

__________________________

*

Arnquist and Harbage (2013).

†

California Department of Health Care Services (2016).

‡

California Department of Social Services (2014).

§

California Department of Social Services (2013).

||Arnquist and Harbage (2013).

#

California Department of Health Care Services (2016).
26

Even though CII had begun integrating evidence-based practices relatively early and 

senior managers and some therapists championed them, other therapists resisted their widespread integration. Those who were involved in the early efforts to use evidence-based practices 

described how some therapists preferred the freedom of general therapy or viewed the push for 

evidence-based practices as a criticism of how they were doing things. Others reportedly 

perceived the use of evidence-based practices as a “cookbook” approach and incompatible with 

the personalized relationships they were trying to build with their clients. Staff members also 

recalled some therapists having difficulty implementing the practices, either because they did 

not have the skills or because they found it difficult to focus on a new treatment model while 

dealing with clients’ crisis situations.

CII managers employed a number of strategies to encourage therapists to adopt evidence-based practices. They offered therapists incentives early on, such as promotions to those

who became proficient in evidence-based practices. They also gave therapists a choice of 

practices from which they could select the one that best suited their individual style and approach to mental health care. 

Implementing and delivering evidence-based practices still has its challenges. As one 

manager at CII asked, “[h]ow do you get 100 different therapists to do it right and not drift?” 

Additionally, the Los Angeles County Department of Mental Health periodically revises its list 

of approved evidence-based practices, forcing CII to adjust existing treatments it offers, or add 

new ones. Changing or adopting new evidence-based practices, however, requires an investment of resources, since each treatment model has specific training and implementation protocols. Funding for evidence-based practices is also limited. While CII receives funding dedicated 

to evidence-based practices, much of its funding is not. Additionally, evidence-based practices 

are highly specified and may not be appropriate for every client.

Conclusion

This chapter presented an overview of CII’s organizational structure, including the services it 

provides and how it funds those services. It has shown that each of CII’s structural components

are inter-related — the Integrated Service Model dictates how CII organizes staff and services, 

funding affects which services CII provides, and the services CII provides influence what roles 

staff members play. The next chapter describes the population CII serves and how clients enter 

services.
27

Chapter 3 

Pathways to CII

This chapter presents an overview of the target population that Children Institute, Inc. (CII), 

serves, with an emphasis on 9- to 24-year-olds, and how clients enter services. It describes the 

client referral and assessment process as well as the clients that CII most recently served. CII 

provides services to demographically diverse clients of all ages, at every stage of life, and with a 

wide range of health care needs.

CII’s Target Population

Each year, CII serves more than 20,000 children and family members. Many of these clients are 

served through one of CII’s one-day events, such as career fairs, tax preparation services, and 

sporting events, or through child care or Head Start programs.1 In 2013, CII provided over 

6,500 people with ongoing clinical, family support, or youth development services.

CII serves clients of all ages. From early childhood programs to Grandma’s House, a 

support group for grandparents raising grandchildren, CII helps children and their families at 

every stage of life. As shown in Table 3.1, more than a third of clients were under 9 years of 

age during the evaluation’s 2012 and 2013 follow-up period, reflecting CII’s expansive early 

childhood programs. Clients ages 9 to 24 years, the population that the Edna McConnell 

Clark Foundation Social Innovation Fund targets, also made up more than a third of CII’s

clients in the same period, with the majority between 9 and 13 years of age. At 15 years old, 

clients age out of several of CII’s clinical programs, including the Wraparound services and 

Full Service Partnership programs, which explains why the majority of clients between 9 and

24 year of age are 13 and under. A little over one-fourth of clients in the 2012 and 2013 

follow-up period were 25 years old or older. The demographic information presented in Table 

3.1 does not tell the whole story, however, since CII’s service receipt data did not always 

reflect parents or guardians who received concurrent services. For example, parents or 

guardians receiving Trauma-Focused Cognitive Behavioral Therapy may have participated in 

conjoint sessions with a therapist but the research team could not capture their data as distinct 

and separate clients. Since many parents or guardians receive some services in conjunction 

with their children’s services, the number of people over 25 years of age that CII serves was

likely understated.

 1

This analysis does not include clients who received services only though one-day events.
28

2012 2013

Characteristic All CII SPA 4 SPA 6 All CII SPA 4 SPA 6

Average age (years) 15.6 18.3 11.6 17.9 20.7 12.1

Age for all clientsa

 (%)

Under 9 years 37.3 31.7 47.7 33.6 28.3 48.7

9-17 years 33.7 31.4 36.4 28.6 26.1 33.3

18-24 years 4.4 4.9 2.4 5.5 6.4 3.0

25 years or older 23.0 31.2 11.3 30.0 38.6 12.3

Gender (%)

Male 51.1 52.1 52.9 52.2 52.4 55.7

Race/Ethnicity (%)

Hispanic/Latino

White

74.1

16.9

85.8

7.5

61.1

36.1

74.4

15.9

83.3

7.7

63.6

33.6

Black 4.3 1.7 1.7 3.8 2.2 1.0

Asian 1.1 1.2 0.0 1.7 2.5 0.2

Other 3.6 3.9 1.1 4.2 4.4 1.5

Sample size 5,018 2,348 958 6,637 4,017 1,112

Table 3.1

Demographic Characteristics of CII Clients

SOURCE: CII management information system.

NOTE: aAge category percentages may not sum to 100 percent because of missing data.

Table 3.1 shows that client demographics differ across regions. CII serves a relatively 

younger population in Service Planning Area (SPA) 6, with nearly half of the clients under 9 

years of age, mainly because it provides more Head Start services in the area. In SPA 4, CII 

offers more services to older clients, such as fatherhood and other parenting programs. CII 

serves a primarily Hispanic or Latino population. Nearly three-quarters of CII clients across 

locations are Hispanic or Latino, though the percentage varies by region. Most of the clients in 

SPA 4 are Hispanic or Latino, but that proportion drops to a little less than two-thirds in SPA 6. 

Among the target population of children and youth 9 to 24 years old, the average age was

between 13 and 14 years old (Table 3.2). The racial and ethnic make-up of this age group 

resembled that of the overall CII population.
29

Table 3.2

Demographic Characteristics of CII Clients (Ages 9-24 Years)

2012 2013

Characteristic All CII SPA 4 SPA 6 All CII SPA 4 SPA 6

Average age (years) 13.7 13.7 13.5 13.5 13.7 13.0

Race/Ethnicity (%)

Hispanic/Latino 75.7 90.8 58.6 76.1 87.0 58.6

Black 16.8 5.1 38.6 16.5 6.5 37.7

White 4.0 1.1 1.7 3.2 1.9 0.8

Asian 1.0 1.3 0.0 1.7 2.4 0.5

Other 2.6 1.7 1.1 2.5 2.2 2.4

Sample size 1,911 854 372 2,261 1,305 403

SOURCE: CII management information system.

Although not shown in the tables, all of CII’s clients are low income; many are enrolled 

in Medi-Cal, California’s state-operated insurance program for low-income families. CII 

receives some limited funds to support indigent clients who lack insurance and do not qualify 

for Medi-Cal (sometimes because they are not legal residents) or who have lost it temporarily.

CII clients have a wide range of needs. Those engaged in only community service activities may have few therapeutic needs. Clients receiving clinical services, on the other hand, may 

have needs ranging from intensive and requiring services 24 hours a day, seven days week, to

less intensive and requiring only weekly sessions with a therapist. Data related to clients’ risk 

factors, such as substance use, trauma history, or household characteristics, were not available. 

However, since the Los Angeles County Department of Children and Family Services referred 

many of the clients receiving CII’s clinical services, it is likely that a substantial proportion of 

them had a history of abuse and come from families with high needs and significant risk factors.

Though CII’s services address a broad range of client needs, there are some needs that 

CII is not equipped to address. For instance, CII does not serve children with mental health 

issues who also have developmental delays or autism. Other organizations that specifically 

focus on development delays serve these children. 

Intake and Assessment

As a large organization offering a wide range of services to a diverse population with varying 

needs, CII’s system for receiving and assessing referrals is complex. This section describes the 
30

various points of entry into CII services, including new referrals to CII and referrals of current 

clients to other services within CII. Figure 3.1 depicts this process.

CII’s intake staff are the first point of contact for referrals. In each SPA, CII employs

intake staff who handle all referrals in the geographic area. Referrals may come from families 

directly, case managers at other organizations or agencies, or even from within CII. Referrals 

may be submitted by phone, e-mail, or fax. As Chapter 2 describes, CII’s funding structure 

requires that CII knit together a diverse set of funding streams, with separate eligibility criteria, 

to meet the individual needs of each client. The intake and assessment process is thus a critical 

component of the Integrated Service Model.

External Referrals

The Los Angeles County Departments of Mental Health and Children and Family Services have contracts with CII for referrals. Other common sources of clients include service 

providers in the community (other nonprofits, schools, and so on) or referrals from families 

directly. Intake staff said that referrals from the Department of Children and Family Services

and from families directly were the most common. Referrals for clinical services may be

requests for general individual therapy or for a specialized treatment or program. Referrals for a 

specific clinical program are subject to the eligibility requirements of that program, which the 

funder often dictates. For example, if a family submits a referral to participate in the Full 

Service Partnership program and does not already have an open Full Service Partnership case 

with the Department of Mental Health, CII intake staff would have to first refer the family to the 

department for approval. As a less intensive program with fewer funding constraints, CII’s 

Community Mental Health services have more flexible requirements for referrals. Unlike 

intensive services, referrals for Community Mental Health services require only CII’s approval 

and not that of the relevant state agency. Though intake staff are ultimately responsible for 

tracking all referrals, specific programs may receive some referrals first. For example, the 

Wraparound services or Full Service Partnership team in a particular SPA may receive a referral

directly from a caseworker at the Department of Children and Family Services. In this case, the 

program team informs the intake staff, who in turn log and track the referral.

CII staff members reported schools to be an important source of referrals. One such 

source is CII’s Cognitive Behavioral Intervention for Trauma in Schools (CBITS) program, 

which CII operates only in schools with which it has a relationship. A form of group therapy, 

CBITS only works when there is a critical mass of participants. As such, CII relies on schools to 

identify program participants and obtain parent or guardian consent. A successful referral 

network for CBITS involves training staff at the school on how to identify children who are 

appropriate for the treatment. In some schools, staff may screen whole classes of students to 

identify those for whom CBITS is appropriate.
31

Referral 

screened by 

intake staff

Case 

disposition 

team reviews 

case and 

makes initial 

assignment

 

Therapist 

meets with 

client to 

conduct 

assessment

Diagnosis 

and 

treatment 

plan 

developed

Client begins 

treatment or 

is reassigned 

to more 

appropriate 

service

Figure 3.1 

Basic Client Flow Through CII Services

According to CII staff members, referrals to youth development and family support services come from a wider array of sources than do clinical service referrals and involve more 

residents of the neighborhoods immediately surrounding CII locations. Sources of these 

referrals include local schools, neighborhood outreach such as informational handouts, and 

word of mouth. In SPA 4 where CII operates its flagship facility, staff reported that area 

residents often walked in off the street to inquire about services. “They see our wonderful 

building,” explained one staff member. “They see people playing basketball outside. They come 

off the street, and they come and find our service because we’re the most low barrier. We’re 

easy access. We’re in their neighborhood.” Intake staff may also refer individuals or families 

who initially requested clinical services to youth development services if CII cannot address 

their clinical needs immediately or at all. If they request services that CII cannot provide, intake 

staff will refer them to another organization that can. Similarly, if there is no funding available 

or no funding stream for the type of service request received, intake staff will refer the individual or family to another organization. 

Internal Referrals

In line with the Integrated Service Model, staff members seek to coordinate CII’s services to respond to clients’ full-range of needs. In doing so, they may refer clients already

receiving services to additional services within the agency. An individual therapist or member 

of the program team may refer a client receiving clinical services to youth development or

family support services. In this case, the intake staff conduct an internal referral, whereby they

contact the client’s family and connect the client to the suggested service. The most common

internal referrals were those for clients receiving clinical services to youth development

services. 
32

Internal referrals for clients engaged in youth development activities to clinical services

were less common. Staff explained that because the population participating in youth development services is not typically high need, these types of referrals were less frequent. While a 

contractor often provides youth development services, a CII staff member typically facilitates 

the activities and may identify participants who would benefit from clinical services. In this 

case, the staff member contacts the intake staff, who in turn reach out to the client’s family.

Case Disposition and Assessment

CII intake staff in each service area reported that though they may receive dozens of referrals each week, they do not process them in the order in which they receive them. Instead, 

they prioritize them based on need. They first assign high-risk cases to a therapist, such as those 

in which clients express suicidal ideations or are cutting themselves. Intake staff also consider 

funding sources when assessing referrals. One intake team described referrals from the Los 

Angeles County Departments of Mental Health or Children and Family Services as “gold card” 

referrals and flagged them for immediate assistance.

Case disposition teams meet weekly to discuss where they should assign new referrals. 

The team assigns referrals for specific programs, such as Wraparound services and the Full 

Service Partnership, to those programs. For referrals to Community Mental Health services, the

team will assign clients to specific therapists based on the information in the referral. Once 

assigned, the therapist contacts the client and conducts an assessment of the client’s needs. 

Clients may meet their therapists and undergo the assessment at the CII office or in their 

schools or homes. The assessment typically includes the Child/Adolescent Initial Assessment, a 

nine-page tool used by the Los Angeles County Department of Mental Health. This tool gathers 

the client’s medical and psychiatric history and information on the client’s development and 

environmental stressors, academic history, involvement with the juvenile justice system, and 

family structure.2 Therapists use it to make an initial diagnosis and develop the client’s treatment plan.3 Therapists also use the Global Assessment of Functioning scale to rate the severity 

of the client’s mental illness. Clients must receive a score of 50 or below (on a scale of 0 to100) 

to be eligible for services covered by Medi-Cal.

Clients may undergo additional assessments depending on what program they enter or

treatment they receive and its respective funding source. For example, Wraparound services

team uses an extensive screening tool that breaks down a client’s needs by domains, such as 

 2

Los Angeles County Department of Mental Health (2013). 3

During site visits, the research team learned that CII was planning on implementing a new assessment 

tool called the Universal Screening Tool. The report does not discuss it because CII had not fully implemented

it during the evaluation period.
33

family, educational, and emotional needs. If this initial assessment determines that a client is 

appropriate for an evidence-based practice, the client may undergo additional assessments 

associated with that treatment. Functional Family Therapy (FFT), for example, has its own 

assessments that clients must undergo during the first phase of treatment. For clients receiving 

Community Mental Health services, a care coordinator will typically join the first assessment 

meeting with the therapist to learn if there is a need for nontherapeutic supports, such as housing

assistance. Some of the assessments serve as a baseline measure of the clients’ symptoms and 

are administered again near the end of treatment to assess the client’s progress. Such assessments include the Youth Outcome Questionnaire and the Post-Traumatic Stress Disorder 

Reaction Index, both of which have components for the child and parent or guardian. 

After the assessment process, the client begins treatment. Depending on the treatment 

plan, the client may continue treatment with the same therapist who administered the assessment, or may be reassigned. For example, if the client discloses a traumatic experience during 

the assessment process that had not been disclosed on the referral, the client may be reassigned to a therapist with an expertise in that particular area. The client’s family may also 

decide after the initial assessment that it wants to change the treatment plan. For example, 

after starting FFT, a family may decide that a family approach is not appropriate and request a 

different type of service.

CII often has waitlists for some of its clinical services, particularly Community Mental 

Health services, which receive more referrals for individual therapy from families directly. Staff 

noted that the more established CII offices in SPA 4 tend to have waitlists because people in the 

community know them and seek them out for services. SPA 6 staff reported that they often have 

a waitlist for the Full Service Partnership program because CII is among only a few organizations in the area who serve clients from birth to age 15 years. 

One strategy that CII staff use to shorten or eliminate waitlists is to refer waitlisted clients to other types of services. For example, staff members may refer clients waitlisted for 

clinical services to family support or youth development services. Such referrals may not 

always be appropriate, however, since clients with severe mental health needs may not have the

social skills to engage in these activities.

Changes to the Intake and Assessment Process

Over the course of the research team’s three site visits in 2013, CII was in the process 

of changing its client intake and assessment procedures to better align them with the Integrated Service Model. During the first two site visits, staff identified a number of challenges with 

these procedures. One challenge, described earlier, was that decisions made by the case 

disposition team were not always appropriate for the clients. Another challenge was that 

internal referrals sometimes resulted in a client falling through the cracks, as staff “passed the 
34

baton” and did not follow up to ensure the client received services. A third challenge was that 

intake staff lacked the qualifications to handle cases involving an immediate need, such as a 

client who requires hospitalization.

To address some of these challenges, CII developed and began implementing the Family Engagement Team model during the evaluation period. However, the research team could not 

fully assess its implementation in this evaluation since it was fully operational only in SPA 4 

and only during the final site visit. The Family Engagement Team model introduced important 

changes to the intake and assessment process. First, CII hired a full-time supervisor to oversee 

the intake staff and who had the qualifications to refer clients with immediate needs such as 

those requiring hospitalization. Second, the model established regional case disposition meetings. Previously, individual programs, such as Community Mental Health services, held their 

own case disposition meetings, which often stymied coordination. Regional case disposition 

meetings expanded the number of programs represented at these meetings to better integrate the 

delivery of services. A perceived benefit of the Family Engagement Team model is that it helps

shorten waitlists. With all available programs in a region on the table, CII staff could refer more 

clients on waitlists to other programs and services. At the time of the third site visit, staff 

reported that the introduction of the Family Engagement Team had reduced the waitlist by up to 

60 percent.

Conclusion

CII serves clients at every stage of life and with a wide range of needs. CII tailors its services to 

the specific needs of each client and client’s family, determining the most appropriate and 

effective treatments and coordinating them with other services and activities. CII’s complex 

funding structure makes the client intake and assessment process a critical component of its 

Integrated Service Model. The next chapter describes in detail CII’s services and the strategies it 

uses to meet clients’ needs.
35

Chapter 4 

Implementation of CII Services

The previous chapters describe the context in which the Children’s Institute, Inc. (CII), operates, 

its organizational characteristics, and the client intake and assessment process. In this chapter, 

the focus turns to the implementation of CII’s multiple service categories: clinical mental health 

services, particularly evidence-based practices; family support services; and youth development

services.

1 Vignettes of CII client experiences compiled from the discussions with their therapists, information from their case files, and analysis of data from CII’s management information 

systems are included throughout the chapter to present a clearer picture of CII’s services.2

 

As described earlier, CII serves a high-need population, which would otherwise typically receive fragmented services through the child welfare system. CII offers its clients a wide 

range of services — both clinical mental health services and nonclinical activities — to address 

their many needs. As presented in Table 4.1, over 40 percent of the more than 6,500 clients 

served during the study period were engaged in clinical mental health services;3 of these clients, 

about one-third were engaged in an evidence-based treatment and over 20 percent were engaged 

in an evidence-informed practice.4 Furthermore, over 50 percent of all clients received family

support services while over 60 percent participated in youth development activities. Each of 

these services is described in more detail below.

While CII offers clients a variety of services and activities, its operating philosophy is to 

coordinate these services to meet the holistic needs of the child and family. The philosophy

 1

A fourth service category, which encompasses a substantial portion of CII’s resources, is early childhood 

care and education; however, it is not a focus of this report. 2

In 2014, CII replaced the management information system from which the data for this report was pulled. 

The limitations of the old system prevented certain types of analyses, which this report describes. There were 

also some data quality issues, which limited the types of analyses that the research team could perform. CII 

expects the new system, which incorporates new checks and balances to improve data quality, to alleviate

many of these challenges. 3

Analysis is limited to any client who enrolled from January 1, 2012, through June 30, 2013, and received 

services from January 1, 2012, through December 31, 2013. 4

These categories are not mutually exclusive. Evidence-based practices include the following treatments 

and programs: Cognitive Behavioral Intervention for Trauma in Schools, Child-Parent Psychotherapy, 

Functional Family Therapy, Incredible Years, Managing and Adapting Practice, Multidimensional Treatment 

Foster Care, Parent-Child Interactive Therapy, Reflective Parenting Program, Trauma-Focused Cognitive 

Behavioral Therapy, and Trauma Systems Therapy-Substance Abuse. Evidence-informed practices include the 

following treatments and programs: Domestic Violence Treatment Groups, Project Fatherhood, Wraparound 

services, Youth with Sexual Behavior Problems, and social skills and parent support groups. Some level of 

evidence informs these practices but not as much as evidence-based practices have accumulated.
36

Table 4.1

Client Participation in Any CII Services

Characteristic All CII

Participation in any clinical servicesa (%)

b Ever engaged in evidence-based practice 32.8 c Ever engaged in evidence-informed practice 23.7

44.1

Participated in any family support services (%)

Participated in any youth development services (%)

54.8

61.1

Sample size 6,566

SOURCE: CII management information system.

NOTES: Analysis is limited to clients who enrolled from January 1, 2012, 

through June 30, 2013, and received services from January 1, 2012, through 

December 31, 2013.

This analysis excludes one-day youth development events. 

These characteristics are not mutually exclusive.

a

These measures represent the percentage of clients among all those who 

belong to this subset of CII clients.

bEvidence-based practices include the following treatments and programs: 

Cognitive Behavioral Intervention for Trauma in Schools, Child-Parent 

Psychotherapy, Functional Family Therapy, Incredible Years, Managing and 

Adapting Practice, Multidimensional Treatment Foster Care, Parent-Child 

Interactive Therapy, Reflective Parenting Program, Trauma-Focused Cognitive 

Behavioral Therapy, and Trauma Systems Therapy for Substance Abuse. c

Evidence-informed practices include the following treatments and programs: 

Domestic Violence Treatment Groups, Project Fatherhood, SOAR, Wraparound 

services, and Youth with Sexual Behavior Problems. 

consists of three core principles that inform all of CII’s operations: recovery, resiliency, and 

readiness. 

This chapter describes CII’s staffing structure and the three categories of service it provides. It then explains, how through the Integrated Service Model, CII coordinates these

services to create a system of care for each client and client’s family. 
37

Staffing Structure

CII operations in each Service Planning Area (SPA) have a similar staffing structure. Figure 4.1

presents an organizational staffing chart.

5 A regional vice president oversees all services and 

activities in an SPA. To prevent isolation and encourage information sharing among SPA

operations, CII convenes regular cross-area management meetings of the vice presidents. 

Regional directors report to the vice president. Each SPA operation has multiple regional 

directors, who are responsible for different aspects of service delivery. In each SPA, two 

regional directors oversee clinical services (one responsible for community mental health

services and the other for intensive in-home services) and one regional director oversees

community services. The regional directors meet regularly to update one another on developments in the different service areas. Therapists, care coordinators, case managers, parent 

partners, and other staff members report to the regional director overseeing the services they 

provide. 

Leaders of each SPA operation report to the CII executive team, which is responsible 

for all CII activities. The team includes the president and chief executive officer, executive vice 

president, and the senior vice president of programs who provides leadership to all of CII’s 

direct service programs. Other executive staff include those overseeing development activities, 

human resources, and facilities management.

Clinical Mental Health Services

Licensed therapists (or supervised clinical interns) typically provide CII’s clinical mental health 

services in individual, family, or group settings.6 In individual therapy, therapists meet one-onone with the client; in family therapy, therapists typically meet with all immediate family 

members together; and in group therapy, one or more therapists facilitates a small group of 

clients. Clinical services range in intensity in any of these settings, and may be delivered as a 

standalone treatment or as part of a multiservice program, such as Wraparound services (an 

intensive, in-home program), depending on eligibility and funding.

All of CII’s clinical services address the effects of traumatic experiences in the lives of 

children and youth. CII, however, does not take a one-size-fits-all approach to treatment; rather, 

intake and clinical staff assess clients individually to determine the best treatment option for 

their needs. CII also allows for flexibility in treatment in order to accommodate unforeseen

 5

Early childhood care and education has a different staffing structure and is not presented here. 6

CII is an American Psychological Association-accredited internship site and hosts interns pursing advanced degrees in clinical psychology at the University of Southern California School of Social Work. 
38

 

Figure 4.1 

Sample SPA Staffing Organizational Chart 

Regional Vice 

President

Regional Director for 

Community Mental 

Health

Community Mental 

Health therapists

Community Mental 

Health Care 

coordinators

Regional Director for 

Intensive In-Home

Wraparound staff

Full Service 

Partnership staff

Field Capable Clinical 

Services staff

Regional Director for 

Community Services

Community Services 

staff, contractors, 

and volunteers

circumstances. Varying degrees of evidence support CII’s therapies and treatment practices, 

with the strongest research backing its evidence-based practices. Box 4.1 describes several 

particularly prevalent evidence-based practices that the study’s target age group receives.

7 CII 

also uses a number of “evidence-informed” practices that are informed by some evidence but 

not as much as evidence-based practices have accumulated. Box 4.2 presents examples of these 

practices. Depending on a client’s situation, CII may use usual care practices to treat clinical 

mental health issues since not every issue requires an evidence-based or evidence-informed 

practice.

7

Los Angeles County contracts with other organizations to provide the same set of services throughout the 

county. Individuals can only seek treatment from one provider at a time.
39

Box 4.1

Evidence-Based Practices Used by CII

for the Study’s Target Age Group*

Trauma-Focused Cognitive Behavioral Therapy (TF-CBT)† is a conjoint child and parent 

psychotherapy approach for children and adolescents ages 3 to 18 years experiencing significant emotional and behavioral difficulties related to traumatic life events.

Functional Family Therapy (FFT)‡ is a family intervention designed for at-risk youth ages 

11 to 18 years experiencing family conflict and behavioral problems. FFT treats the entire 

family as the client and is a short-term treatment, lasting 12 to 14 sessions.

Incredible Years (IY)§ is a group parenting program designed to strengthen parent-child 

interactions by teaching parents to provide positive discipline, increase confidence, and 

become involved in their child’s school experiences and social development. 

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)§ is a school-based 

group intervention for youth ages 10 to 14 years who have been exposed to community and 

domestic violence. Over 10 sessions, students decrease symptoms of post-traumatic stress 

disorder and depression, while building resilience and increasing peer support.

Multidimensional Treatment Foster Care (MTFC)|| is a comprehensive intervention for 

foster children and adolescents ages 7 to 17 years with behavioral problems. Children are 

placed with foster parents who have been trained as part of the treatment team. MTFC seeks to 

help children develop positive relationships and behaviors.

Reflective Parenting Program (RPP)# is a 12-week workshop series providing parenting 

skills and instruction to groups of 6 to 10 parents with infants or adolescents. Parents acquire 

skills and strategies to support their children’s ability to develop and maintain positive relationships in the future.

Trauma Systems Therapy: Substance Abuse (TST-SA)§ is an application of the Trauma 

Systems Therapy model with a focus on treating adolescent traumatic stress and substance 

abuse for children or adolescents with ongoing traumatic stressors. The model identifies 

interventions that will improve self-regulation of emotional responses.

(continued)
40

Box 4.1 (continued)

Managing and Adapting Practice (MAP)§ is a system developed to coordinate and supplement the use of evidence-based practices to address children’s mental health needs. MAP 

provides support to therapists in treatment decisions, such as selection and delivery of evidence-based practices. The model measures clients’ progress and assists in adjusting the 

service plan as needed.

__________________________

*

CII uses other evidence-based practices for younger age groups. These practices include Parent-Child 

Interaction Therapy for children 2 to 7 years old and Child Parent Psychotherapy for children up to 6 years 

old.

†

National Child Traumatic Stress Network (2007).

‡

Functional Family Therapy, LLC (2015).

§

Children’s Institute, Inc. (2015).

||MTFC is now known as Treatment Foster Care Oregon.

#Center for Reflective Communities (2016).

Box 4.2

Evidence-Informed Practices Used by CII

for the Study’s Target Age Group*

SOAR, a 10-week social skill development group developed at CII. During weekly sessions, 

children ages 9 to 12 years, in gender-specific groups of 10 to 12, explore themes including 

individuality, cultural diversity, healthy relationships, teamwork, and leadership. The groups 

are co-facilitated by two adults who guide children through role play and reflection exercises. 

Parents of participating children have access to a concurrent cycle of weekly support meetings.

Project Fatherhood, a program providing support to fathers of at-risk children who are often 

in the child welfare system. Services offered through the program include therapy, jobreadiness training, and parenting education. Project Fatherhood aims to increase father involvement in children’s development and strengthen the father-child relationship.

Youth with Sexual Behavior Problems, a group for children ages 10 to 14 years who have 

exhibited inappropriate sexual behavior with other children, some of whom have been victims 

of sexual abuse themselves. Over 25 weeks, children learn about the consequences of their 

actions and how to exercise self-control. Parents participate in parallel groups that provide 

guidance in effective strategies for communicating with children about sexuality and safety.

Domestic Violence Treatment Groups, a 30-week group for adult survivors of domestic 

violence with parallel groups for their children. The groups teach coping and emotional 

regulation skills while helping parents make decisions to protect themselves and their children.

__________________________

*

Children’s Institute, Inc. (2015).
41

Table 4.2 presents information about tenure of CII’s clinical services staff and their 

training in evidence-based practices. Only staff members with specific qualifications were 

eligible for training in evidence-based practices, namely those holding a relevant master’s 

degree.

8 The majority of the therapists and psychologists included in this analysis were trained 

in an evidence-based practice, with the percentage increasing between 2012 and 2013. Over 40 

percent of all clinical staff were trained in Trauma-Focused Cognitive Behavioral Therapy (TFCBT), with a much smaller percentage trained in Functional Family Therapy (FFT).

9 The 

average tenure of clinical staff at CII was nearly four years in 2012 (and a bit less in 2013).

The majority of clients engaged in any of CII’s clinical services are female and Hispanic/Latino (Table 4.3). Nearly 40 percent of them are 9 to 24 years old — the study’s target age 

group.

10 Table 4.4 presents analysis of the duration and intensity of clients’ participation in 

clinical services (individual or group therapy); this analysis includes clients receiving usual care, 

evidence-based, or evidence-informed practices. Data show that younger clients 9 to 17 years 

old typically participated in individual therapy for nearly seven months, while those 18 to 24 

years old participated for about half of that time. The younger clients also tended to have more 

contact with their therapists than did the older ones. These differences are not unexpected since 

treatment intensity and duration varies based on the specific therapy, and different therapies 

target different age groups.

Clients participating in group therapy had slightly different experiences depending on 

their age and SPA. Clients 9 to 17 years old on average participated in group therapy for about 

the same duration and number of sessions as did clients 18 to 24 years old, though there was 

some variation by SPA. While it appears that older clients spent on average less time in each

group therapy session, this difference is actually the result of how the management information 

system recorded the data, which calculated this time using an equation that factored the number 

of clients in a group as well as the number of therapists facilitating it. Groups for older clients 

tended to be larger, and as a result the time spent in sessions was divided across more participants. The average service times for group therapy shown in Table 4.4 thus do not accurately

reflect the face-to-face time each client spent with a group therapist. 

 8

As the notes in Table 4.2 explain, this analysis was limited to therapists or psychologists during the analysis period and to those for whom training dates were available. 9

Two-thirds of staff trained in any evidence-based practice were trained in TF-CBT, while 14 percent were 

trained in FFT. 10The research team did not expect the large number of clients clustered in the age category of 25 years or 

older. CII does offer a program for adults through California Work Opportunity and Responsibility to Kids 

(CalWORKs), the state’s welfare-to-work program, which accounted for many of these clients. Data entry 

error may also have contributed to this high number. 
Characteristic All CII SPA 4 SPA 6 All CII SPA 4 SPA 6

Staff trained (%)

a Evidence-based practices 64.8 62.8 63.3 66.9 62.9 67.4

Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) 42.8 42.3 40.8 46.4 41.4 45.7

Functional Family Therapy (FFT) 8.8 6.4 14.3 7.2 4.3 13.0

Average time at CII (months) 44.8 51.9 41.7 40.7 47.0 35.9

For staff trained in TF-CBT 45.0 48.2 45.4 40.7 48.0 32.7

For staff trained in FFT 44.1 43.8 34.9 47.8 42.7 38.2

Sample size 159 78 49 166 70 46

Table 4.2

Clinical Services Staff Tenure and Training in Evidence-Based Practices

2012 2013

SOURCE: CII management information system.

NOTES: This analysis includes only clinical staff who were identified as therapists or psychologists during the study 

period; there are many other staff at CII who support clients in other ways but they are not represented here. Only staff for 

whom training dates were available were counted in this analysis, which likely underreports the percentage of staff who 

are trained.

These characteristics are not mutually exclusive.

a This analysis includes staff trained in the following evidence-based practices: Cognitive Behavioral Intervention for 

Trauma in Schools, Child Parent Psychotherapy, Functional Family Therapy, Incredible Years, Managing and Adapting 

Practice, Parent Child Interactive Therapy, Reflective Parenting Program, Trauma-Focused Cognitive Behavioral 

Therapy, and Seeking Safety. 

42
43

Clinical Mental Health Services Infrastructure

CII’s clinical mental health services require significant infrastructure dedicated to 

training therapists and supporting their work. All therapists are licensed clinicians (or supervised clinical interns) and have earned at least their masters in marriage and family therapy or

a related field; some are licensed psychologists or hold advanced degrees in clinical psychology. Newly hired clinical staff must successfully complete extensive training in CII policies 

and procedures and many receive training in specific evidence-based practices. As noted 

earlier, nearly two-thirds of CII staff in 2012 had been trained in an evidence-based practice 

during their tenure. Over 40 percent of staff received training in TF-CBT and less than 10 

percent in FFT.

In addition to formal trainings, CII dedicates core resources to varying levels of support 

for therapists. Therapists implementing and delivering evidence-based practices typically report 

Table 4.3

Demographic Characteristics of CII Clients Engaged in Clinical Services

Characteristic All CII SPA 4 SPA 6

Agea (%)

Under 9 years 39.5 35.3 48.2

9-17 years 33.9 36.5 34.7

18-24 years 4.5 5.1 1.9

25 years or older 17.5 20.6 9.9

Gender (%)

Male 44.4 44.7 47.6

Race/Ethnicity (%)

Hispanic/Latino 71.0 86.9 62.6

Black 19.0 6.5 33.9

White 6.2 2.5 2.5

Asian 0.9 1.1 0.0

Other 2.9 2.9 1.0

Sample size 2,778 1,005 644

SOURCE: CII management information system.

NOTES: Analysis is limited to clients who enrolled from January 1, 2012, through June 30, 

2013, and received services from January 1, 2012, through December 31, 2013. aAge category percentages may not sum to 100 percent because of missing data.
44

Table 4.4

Duration and Intensity of Client Participation in CII Clinical Services

Individual Therapy Group Therapy

Characteristic All CII SPA 4 SPA 6 All CII SPA 4 SPA 6

Number of successive months of participation

Age

9-17 years 6.8 6.8 6.8 3.9 3.8 4.2

18-24 years 3.2 2.6 5.4 4.0 5.3 2.0

Average number of contacts per participant

Age

9-17 years 21.3 20.0 22.7 10.9 12.1 9.3

18-24 years 9.3 5.2 18.5 9.9 12.3 4.0

Average service time per contacta,b,c (minutes)

Age

9-17 years 99.6 92.4 102.2 35.9 30.4 64.0

18-24 years 77.6 82.1 83.7 28.8 27.2 135.0

Average service time per participantc

 (hours)

Age

9-17 years 35.4 30.8 38.7 6.5 6.1 9.9

18-24 years 12.1 7.1 25.9 4.8 5.6 9.0

Sample size 1,054 423 249 297 127 16

SOURCE: CII management information system.

NOTES: Analysis is limited to clients who enrolled from January 1, 2012, through June 30, 2013, and 

received services from January 1, 2012, through December 31, 2013.

Youth engaged in family-based therapy were counted as participating in individual therapy. An 

example of group therapy is Cognitive Behavioral Intervention for Trauma in Schools. aAverage service time per contact (minutes) was calculated using a different unit of analysis from the 

unit of analysis used to calculate other measures in the table. For this measure, data are presented as an 

average per contact, while other measures are presented as averages per participant.

bService time per contact for group therapy was calculated by dividing group session time by the 

number of participants in the group, for billing purposes. As a result, reported service times and 

calculated averages are lower than the actual length of the group session. cFor sessions that require a therapist to travel to a client's school or home, service time includes travel 

time; session time includes documentation time for all clients.
45

to multiple supervisors and attend multiple supervision meetings each week. Therapists meet 

with their clinical supervisors monthly. (Interns meet with theirs weekly until they are 

licensed.) During these meetings, they review their caseloads and discuss any challenges they 

may be experiencing, such as difficulties working with a particular family or problems 

properly documenting services in CII’s management information system. The clinical supervisor is also responsible for evaluating therapists’ and interns’ performance, recommending

them for promotions, and dealing with any discipline issues. Therapists also attend supervision meetings specific to evidence-based practices they use. For example, therapists providing

TF-CBT attend either a weekly or bi-weekly supervision meeting depending on their level of 

expertise in the therapy. 

Despite this support system, therapists in different programs reported some shared challenges to delivering clinical services. Los Angeles County Departments of Mental Health and

Children and Family Services require that therapists complete a significant amount of documentation for each client. Nearly 80 percent of the staff who responded to the research team’s 

survey noted that it was somewhat to extremely challenging to complete documentation in a 

timely manner. In interviews with the research team, staff described using multiple evidencebased practices as particularly challenging, especially if the practices differed theoretically. 

They explained that each evidence-based practice requires additional documentation and its 

own supervision meetings, making the delivery of more than one burdensome. Regardless of 

the practices used, staff across programs described how treating young people with complex 

and multiple traumas and who often lived in perpetual crisis made it difficult to adhere strictly 

to a treatment plan.

Family Support

Family support services are those offered to parents and guardians to improve their capacity 

to support their children. CII offers many of these services to members of the larger community regardless of whether or not they participate in CII’s clinical services. Designed to help 

families achieve economic stability, develop positive parenting practices, and connect with 

other families, these services are frequently offered in English and Spanish and include 

support groups for grandparents raising grandchildren, fatherhood classes and other parenting 

classes, financial literacy workshops, and income tax preparation. Clinicians reported in the 

survey that they referred parents or guardians to support groups and special family events 

among other activities.

CII offers some family support services in conjunction with clinical services. For example, families participating in Wraparound services or the Full Service Partnership program
46

receive support services through their assigned parent partners, who as part of these programs 

assist individual parents or guardians with a range of issues.11 Clients receiving clinical services 

may also access case management services. Some evidence-based practices such as TF-CBT

require clients and their parents or guardians to attend conjoint or collateral therapy sessions 

together. And while CII offers FFT as a clinical service, it is generally classified as a family 

support service as well. 

A variety of staff deliver CII’s family support services. Therapists provide family-based 

therapy such as FFT and work directly with their clients’ parents or guardians to help them

support their children’s recovery. Care coordinators, case managers, and parent partners also 

provide family support services. They often work together as part of a care team in Wraparound 

services or the Full Service Partnership program and help parents or guardians advocate for 

their children at school, secure basic needs, or address their children’s behavioral issues.

The research team encountered several challenges when analyzing the available data related to CII’s family support services. As mentioned earlier, CII offers some family support 

services as part of clinical services, which made it difficult to distinguish them in the management information system data. Similarly, records of family support services were associated at 

times with the client and at other times with the parent or guardian in an unsystematic way, 

again making it difficult to determine the precise service dosages. For example, Table 4.5 

indicates that one-third of clients engaged in family support services were over 25 years old; 

however, most of these clients were parents or guardians participating in CII’s Project Fatherhood or another parent education group. 

Youth Development

Youth development services are those offered to children and youth to build resiliency, enhance 

protective factors, and reduce risks and perhaps the need for mental health care in the future. As 

one CII staff member said, “[i]f you do not also instill in kids the ability to develop, build selfesteem, and imagine a better future for them, their success will be limited.” Similar to family 

support services, these activities target youth and families living in neighborhoods immediately 

surrounding CII locations. CII’s youth development activities focus on life and social skills, 

literacy and education, creative arts, and health and wellness and help young people develop 

self-esteem and social skills.

 11Parent partners provide emotional support to parents and serve as links to community resources, among 

other things. One parent partner described working with a parent to address a child’s poor hygiene; another 

described working with a parent to get a child to do household chores.
47

As part of a Social Innovation Fund (SIF) grant it received, CII sought to expand its 

youth development services during the study period. This expansion of services was particularly 

evident in SPA 4.12 While CII in other SPAs offered some youth development activities, those 

provided in SPA 4 were far more comprehensive.13 In addition to expanding services, youth 

development staff also modified their approach to service delivery, in part by starting all group

activities at the same time and running them for the same duration.14 CII offers most activities to

specific age groups, from elementary schoolers through teenagers, typically at 10-week intervals. These activities include sports such as soccer and basketball and classes in theater, dance, 

 12Most of the expansion of services in SPA 6 occurred after the study period and thus is not reflected in 

this report. 13CII began developing a flagship facility in SPA 6 during the study period but the data do not account for 

this expansion. 14CII also attempted to implement pre- and post-tests to measure changes in outcomes for community 

services, but encountered some challenges.

Characteristic All CII SPA 4 SPA 6

Participates in any family support

Age (%)

Under 9 years 30.7 20.0 45.6

9-17 years 25.6 20.9 32.7

18-24 years 6.5 8.9 2.8

25 years or older 33.7 48.8 13.9

Gender (%)

Male 57.2 62.1 55.1

Race/Ethnicity (%)

Hispanic/Latino 69.8 78.1 62.3

White 17.8 9.2 34.3

Black 5.3 2.7 2.2

Asian 1.1 1.1 0.0

Other 6.0 8.8 1.2

Sample size 3,468 1,776 669

(continued)

Table 4.5

Client Demographic Characteristics and Participation in Family Support Services
48

cooking, and art. Box 4.3 presents more examples. CII also holds a variety of one-day events for 

young people and their families, such as field trips, movie nights, holiday celebrations, and

college or career fairs; this analysis does not include these events.

Clients may only participate in youth development services or participate in them as 

well as receive clinical services. For some clients receiving clinical services, participation in 

youth development activities is not appropriate since they may first need to stabilize their

Characteristic All CII SPA 4 SPA 6

Table 4.5 (continued)

Number of successive months of participation

Age

9-17 years 6.9 6.7 6.7

18-24 years 4.1 3.8 6.5

Average number of contacts per participant

Age

9-17 years 20.7 18.8 22.3

18-24 years 8.1 7.0 12.9

Average service time per contacta,b (minutes)

Age

9-17 years 67.4 52.2 78.5

18-24 years 16.9 12.9 15.6

Average service time per participantb

 (hours)

Age

9-17 years 23.2 16.3 29.2

18-24 years 2.3 1.5 3.3

Sample size 1,153 537 250

SOURCE: CII management information system.

NOTES: Analysis is limited to clients who enrolled from January 1, 2012, through June 30, 2013, and 

received services from January 1, 2012, through December 31, 2013. aAverage service time per contact (minutes) was calculated using a different unit of analysis from 

the unit of analysis used to calculate other measures in the table. For this measure, data are presented as 

an average per contact, while other measures are presented as averages per participant.

bFor sessions that require a therapist to travel to a client's school or home, service time includes 

travel time; service time includes documentation time for all clients.
49

situations through therapy. If appropriate, a therapist might refer a client to a particular youth 

development activity in conjunction with their treatment, for example, to improve social skills 

or overcome anxiety. In other cases, clinicians use youth development activities as a “step 

down,” or transition, for clients completing therapy. In both cases, therapists or case managers 

use youth development activities to enhance the client’s treatment on site and without external 

referrals. “The goal for mental health should never be a lifelong commitment,” one CII therapist 

explained. “Being able to provide services that are not mental health related, but that you can 

still practice social skills, stress management, coping skills, and all that, it’s something that they 

can generalize through their whole life.” Another CII staff member said that “[t]hey need their 

therapy but they need things to build resilience and hope and those kinds of things so that they 

can see beyond what’s happened to them and be kids, have some fun.” Occasionally, youth 

development staff identify participants with mental health needs and arrange clinical services 

for them.

CII’s wide-ranging youth development activities require a great deal of resources. A 

combination of CII staff, contractors, and volunteers with specific expertise facilitate the various

activities. The community service department, which organizes youth development activities, 

employs six to eight full time staff members. An additional 25 full-time staff at CII spend a 

portion of their time working on youth development activities. CII also engages approximately 

1,000 volunteers each year, of which about 100 are active at one time. Clinical staff reported 

Box 4.3

Examples of Youth Development Activities Offered by CII*

Life and Social Skills: employment preparation courses, a youth internship program, and 

SOAR, a group program improving communication and relationship skills.

Literacy and Education: homework help, a children’s library, and a computer lab. Education 

programs include Superstart, a program reinforcing basic literacy and math skills, and reading 

programs such as Stories Abroad and Camp Read-A-Lot.

Creative Arts: painting, drawing, and ceramics classes. Digital media arts programs teach 

film-making and visual design, while performing arts offerings include theater, music, and 

various dance classes.

Health and Wellness: workshops in nutrition and healthy eating and cooking classes. Services 

promoting physical activity include exercise classes and sports programs.

__________________________

*

Children’s Institute, Inc. (2015).
50

some difficulty in finding time to participate in the delivery of youth development activities. 

Many recognized the value of working with their clients in different ways and sometimes 

facilitated activity groups. However, they also said that clinical staff felt pressure to spend their 

time doing “billable” clinical work, which limited how much they could participate in youth 

development activities.

According to CII records, participants of all ages received youth development services 

during the study period. Table 4.6 shows that participants in SPA 4 attended more classes and 

spent more hours engaged in activities than did participants in SPA 6. This difference was 

expected since CII in SPA 4 had expanded its service offerings by the time of the study period.

Across all CII operations, nearly 50 percent of participants were under 9 years old and about 40 

percent were 9 to 24 years old. Participants ages 9 to 17 years, on average, attended nearly 11 

classes for over an hour, totaling 20 hours of participation. The activities targeting 18- to 24-

year-olds included employment supports, a domestic violence program affiliated with CalWORKs, and parenting classes. The differences in the types of activities may explain the 

varying participation levels among the age groups.

The Integrated Service Model

CII coordinates clinical, family support, and youth development services through its Integrated 

Service Model. Designed to ensure children and families receive multiple services tailored to 

their specified needs, it requires staff to weave together a complex set of contractual funding, 

grants, and other sources of revenue. CII developed the Integrated Service Model to eliminate 

the historic silos separating clinical services from the agency’s other activities and to create a 

more fluid communication system to better identify clients’ needs and connect them to all the

services required to address those needs.

Staff reported that implementing the Integrated Service Model involved changes in how 

CII organized staff and how staff related to one another and communicated new hires and other 

personnel issues, and how CII funded its services and made them available to clients and their 

families. Staff in different SPAs reported that these changes led to greater communication and

closer collaboration among programs and departments. The changes also introduced structures 

to convene staff from across programs and departments in order to share important information 

about their work, which in turn prepared staff to better serve clients and their families. CII also 

modified its hiring practices, adding the expectation that new staff members be flexible with 

respect to the locations where or programs in which they work.

CII strives to provide multiple, coordinated services to clients to address their many 

needs and high-risk factors. It is difficult to assess the progress that CII made in implementing
51

Table 4.6

Client Demographic Characteristics and Participation in

Youth Development Services

Characteristic All CII SPA 4 SPA 6

Participates in any youth development services

Agea

 (%)

Under 9 years 48.9 43.9 62.9

9-17 years 34.6 35.8 32.1

18-24 years 2.2 2.3 0.7

25 years or older 13.0 17.7 4.0

Gender (%)

Male 45.8 41.9 56.1

Race/Ethnicity (%)

Hispanic/Latino 79.9 91.1 70.0

White 12.3 3.3 28.5

Black 3.5 1.0 0.6

Asian 2.6 3.8 0.3

Other 1.5 0.9 0.6

Sample size 4,013 2,324 808

Average number of classes attended per participant

Age

9-17 years 10.7 12.9 6.6

18-24 years 12.3 14.7 5.8

Average service time per contactb (hours)

Age

9-17 years 1.5 1.3 1.7

18-24 years 0.6 0.6 1.4

Average service time per participant (hours)

Age

9-17 years 20.0 25.0 11.6

18-24 years 22.7 34.5 8.3

Sample size 1,477 886 265

(continued)
52

the Integrated Service Model because the research team did not have access to the data on client 

risk factors, which are necessary to determine whether clients receiving only one service would 

have benefited from other services. The team instead analyzed data on clients receiving multiple 

services, since the extent to which clients participated in multiple services serves as a proxy for 

measuring CII’s progress in connecting clients to multiple, coordinated services. As shown in 

Table 4.7, about 60 percent of clients received only one type of service during the analysis 

period, most of whom participated in a youth development activity. Only 9 percent of clients 

received only clinical services during the study period. Of the nearly 40 percent of clients 

receiving multiple services, most received clinical services and at least one other type of service. 

Of these clients, over 40 percent received clinical and family support services during the study 

period, which is unsurprising since many of CII’s clinical services integrate family support 

services. For example, parents or guardians in the Full Service Partnership program automatically receive services from a parent partner. Due to limitations of CII’s information management system at the time of the study, it was not possible to determine what percentage of clients 

received family support services as part of their clinical treatment, or as part of a separate 

program requiring a separate referral. More telling of CII’s progress in engaging clients in 

multiple services, however, is that more than 50 percent of clients receiving clinical services 

were also receiving family support and youth development services.15

 15Table 4.7 does not distinguish participation by SPA. If it did, it would be expected that SPA 4 would 

have higher participation levels because of the significant emphasis on the Integrated Service Model in that 

location. CII staff did not emphasize it as much in SPA 6 and even less so in SPA 8.

Table 4.6 (continued)

SOURCE: CII management information system.

NOTES: Analysis is limited to clients who enrolled from January 1, 2012, through 

June 30, 2013, and received services from January 1, 2012, through December 31, 

2013.

This analysis excludes one-day youth development events. a

Age category percentages may not sum to 100 percent because of missing data.

bAverage service time per contact (hours) was calculated using a different unit of 

analysis from the unit of analysis used to calculate other measures in the table. For 

this measure, data are presented as an average per contact, while other measures are 

presented as averages per participant.
53

Characteristic All CII

Participates in one servicea

 (%)

Clinical services only 8.9

Family support services only 27.9

Youth development services only 63.2

Participates in multiple servicesa,b (%) 38.7

Clinical and family support services only 42.6

60.8

Though it appears as if CII successfully implemented the Integrated Service Model, it 

was not entirely possible to assess the effectiveness of the integrated services. Hypothesizing

that participation in multiple services would improve the clients’ outcomes, the research team 

analyzed pre- and post-Youth Outcome Questionnaire scores for the limited number of clients 

who received TF-CBT or FFT as well as family support services, youth development services, 

or both.

16 Results of this analysis did not support the hypotheses; the research team found no 

 16The team chose TF-CBT and FFT because they were the services for which data were most likely to

have been collected or available.

Clinical and youth development services only 3.5

Family support and youth development services only 1.1

Clinical, family support, and youth development services 53.9

Sample size 6,566

Table 4.7

Client Participation in CII Services

SOURCE: CII management information system.

NOTES: Analysis is limited to clients who enrolled from January 1, 2012, 

through June 30, 2013, and received services from January 1, 2012, through 

December 31, 2013.

This analysis excludes one-day youth development events. a

These measures represent the percentage of clients among all those who 

belong to this subset of CII clients.

bThis characteristic includes only clients who participated in clinical services 

and at least one other type of service. It does not include clients who 

participated in multiple services but not clinical services; it excluded 29 clients 

who only participated in family support and youth development services only.
54

correlation. However, data on outcomes were not available for all clients, making it difficult to 

fully interpret this finding.17 Appendix A describes the limitations of this analysis in more detail.

Staff described several benefits of the Integrated Service Model, particularly the value 

of combining clinical services with youth development activities. All clinicians who responded

to the staff survey thought that receiving youth development services in conjunction with 

clinical services improved client outcomes to some degree compared with receiving only

clinical treatment; nearly 50 percent thought that it improved client outcomes a “great deal.” 

One CII therapist described the changes in a child who attended weekly therapy sessions but 

would not talk. The therapist referred the client to the performing arts program, where the client 

began writing poetry, developed an interest in media, experimented with filmmaking, and wrote 

part of a play. The therapist went to one of the client’s performances and it thrilled her. She 

described the ability of these enrichment activities to complement therapy as “magical.” 

Another therapist described similar positive effects on some clients who enrolled in CII’s art 

class. “A few of the kids who are in it have really opened up because I think it’s [art class] 

accessed a part of them that’s really allowed them to express themselves,” the therapist explained. “And so I think that’s helped them within their regular [therapy] sessions to really just 

start verbalizing their feelings more and just be more comfortable with who they are.”

A more comprehensive assessment of the impact of the Integrated Service Model on 

clinical services would require more data than are currently available and a different study

design. For instance, such an assessment would need more years of data than analyzed in this 

study to capture the full duration of the clinical and youth development services clients may 

receive. The assessment might also include pre- and post-tests for clients participating in youth 

development activities. CII is currently experimenting with these tests, which make them likely 

components of any such assessment in the future. 

Conclusion

CII provided clients with a variety of services during the study period. The largest proportion of 

clients participated in youth development activities; a smaller proportion received clinical 

services. Therapists and other CII staff have on the whole bought into the Integrated Service 

Model, which CII has implemented as evidenced in part by the analysis of clients receiving 

multiple services. It is notable that many clients in clinical services also receive other services at 

 17The research team also analyzed the relationship between the number of treatment sessions and significant improvements in outcome test scores; results similarly did not support the hypothesis that more treatment 

leads to more improvement.
55

CII. Delivering services according to the Integrated Service Model is not without its challenges, 

however. Perhaps the greatest challenge is providing multiple, coordinate services in the 

current, complex funding environment. And while the SIF grant allowed CII to expand its 

programming, as the grant comes to an end, CII must find new ways to sustain its activities.

57

Chapter 5 

A Closer Look at Two Evidence-Based Practices

Children’s Institute, Inc. (CII), has pioneered the use of evidence-based practices.1 In 2012, 65 

percent of CII clinicians were trained in more than a dozen different evidence-based practices 

(Table 4.2). Each evidence-based practice has its own requirements and regulations for adoption 

and implementation. The developers of many evidence-based practices make materials readily 

available to the general public at little or no cost and do not require specific trainings or monitoring.

2 Other developers, such as those behind Functional Family Therapy (FFT), require

certification and continual monitoring to implement and use the treatment. CII also uses a 

number of “evidence-informed” practices that, while not rigorously tested, are based on varying 

levels of evidence;

3 almost one-quarter of clients receiving CII’s clinical services engage in an 

evidence-informed practice. 

About one-third of CII’s clients receiving clinical mental health services engaged in an 

evidence-based practice (Table 4.1).4 And nearly half of clients receiving clinical services 

engaged in evidence-based or evidence-informed practices.5 As Table 5.1 shows, 57 percent of 

the clients engaged in an evidence-based practice were under 9 years of age. One reason for this 

high number is that more evidence-based practices are available to this age group. Another 41 

percent of clients engaged in an evidence-based practice were 9 to 24 years old; this group

represents just over one-third of all 9- to 24-year-olds receiving clinical services at CII (not 

shown). The percentage of clients receiving evidence-based treatment at CII is higher than

 1

Evidence-based practices include the following treatments and programs: Cognitive Behavioral Intervention for Trauma in Schools, Child Parent Psychotherapy, Functional Family Therapy (FFT), Incredible Years, 

Managing and Adapting Practice, Multidimensional Treatment Foster Care, Parent-Child Interactive Therapy, 

Reflective Parenting Program, Trauma-Focused Cognitive Behavioral Therapy (TF-CBT), and Trauma 

Systems Therapy-Substance Abuse. 2

For example, Cognitive Behavioral Intervention for Trauma in Schools makes all necessary materials 

available for free on its website. 3

Evidence-informed practices include the following treatments and programs: Domestic Violence Treatment Groups, Project Fatherhood, Wraparound services, Youth with Sexual Behavior Problems programs, and 

social skills and parent support groups. Some evidence informs these practices but not as much as evidencebased practices have accumulated. 4

The remaining young people received other therapies not classified as evidence based. However, without 

knowing more about client risk and medical necessity, it is not possible to say whether this saturation is 

appropriate or if it falls short. 5

After accounting for the overlap of those clients engaged in both evidence-based and evidence-informed 

practices during the study period, 47 percent of clinical clients participated in either an evidence-based or 

evidence-informed practice (not shown). 
58

 Table 5.1

Demographic Characteristics of CII Clients Engaged in 

Evidence-Based Practicesa

Characteristic All CII SPA 4 SPA 6

b Engaged in any Evidence-Based Practice (%)

Age

Under 9 years 56.9 57.7 62.7

9-17 years 40.1 38.6 35.9

18-24 years 0.6 0.0 0.0

25 years or older 1.1 1.0 0.5

Gender (%)

Male 51.4 54.3 50.7

Race/Ethnicity (%)

Hispanic/Latino 78.1 94.7 69.6

White 13.8 2.1 28.1

Black

Asian

Other

5.5

0.4

2.2

1.3

0.3

1.6

1.4

0.0

0.9

Sample size 938 371 215

SOURCE: CII management information system.

NOTES: Analysis is limited to clients who enrolled from January 1, 2012,

through June 30, 2013, and received services from January 1, 2012, through 

December 31, 2013. a

Evidence-based practices include the following treatments and programs: 

Cognitive Behavioral Intervention for Trauma in Schools, Child Parent 

Psychotherapy, Functional Family Therapy, Incredible Years, Managing and 

Adapting Practice, Multidimensional Treatment Foster Care, Parent-Child 

Interactive Therapy, Reflective Parenting Program, Trauma-Focused 

Cognitive Behavioral Therapy, and Trauma Systems Therapy for Substance 

Abuse.

bAge category percentages may not sum to 100 percent because of missing 

data.
59

average nationwide. According to one estimate, only 2 percent of youth receiving mental health 

services through California’s county-level departments of mental health received an evidencebased treatment.6 This estimate used a narrow definition of evidence-based practice and did not 

include Trauma-Focused Cognitive Behavioral Therapy (TF-CBT), the most popular evidencebased practice at CII. Still, even if this estimate is low, CII engages a notably high proportion of 

clients in evidence-based practices.

Therapists reported some common challenges to implementing evidence-based practices in general. Implementing any new practice involves a learning curve and requires additional 

training and supervision, which can be time consuming. CII provides training to clinicians and 

support staff for each evidence-based practice they deliver, which may be offered on site or at 

another agency or organization. Another challenge is the cost of training. Training staff in FFT, 

for example, is particularly expensive, and managers take its cost into account when deciding 

what staff members they train in the practice. As one executive staff member explained, “[n]ow 

we know it takes a certain kind of person that has that commitment to FFT ... Before we spend 

the money training them... we want a commitment that they’re the right person and that they 

will stay and that this is what they’re passionate about.”

This chapter describes two of CII’s most used evidence-based practices in more detail: 

FFT and TF-CBT. It also explains the requirements for implementing each evidence-based 

practice and how CII staff delivered them during the study period.7 The chapter presents the

dosage of each treatment that CII clients received, since dosage is an important indicator of 

whether or not the practice was delivered with fidelity to the model. Lastly, the chapter summarizes findings from a fidelity study of CII’s TF-CBT services.

8

 

Functional Family Therapy (FFT)

FFT is a family-based treatment for youth ages 11 to 18 years with a range of behavioral

problems, such as conduct disorder or substance abuse. The youth have often come in contact 

with the juvenile justice, mental health, or child welfare systems. Grounded in the theory that a 

child’s behavioral problems are a symptom of family dysfunction, FFT seeks to create functional family relationships by improving communication and relationships among family members, 

increasing a sense of mutual support, and decreasing intense negativity and dysfunctional 

patterns of behavior.9 FFT uses both behavioral and cognitive behavioral interventions to 

 6

Technical Assistance Collaborative and Human Services Research Institute (2012). 7

The analysis presented here is limited to clients who enrolled from January 1, 2012, through June 30, 

2013, and received services from January 1, 2012, through December 31, 2013. 8

A Technical Resource for this report presents the full study and is available on the MDRC website. 9

Henggeler and Schoenwald (2011).
60

develop the strengths of each family member, which in turn gives the family a foundation on 

which to build functional relationships. Randomized controlled trials and quasi-experimental 

studies have demonstrated FFT’s effectiveness in reducing antisocial behavior, recidivism 

among juvenile offenders, and criminal offenses committed by siblings.10

The therapist delivers FFT in five phases, which Box 5.1 describes in detail. The therapist also assesses the youth and family on an ongoing a basis, beginning with the Youth Outcome Questionnaire that the youth and parent or guardian complete at intake.

11 During the first 

phases of treatment, the therapist concentrates on engaging the client and family members and 

motivating them to change their behaviors. Next, the therapist works with individual family 

members as well as with the family as a group to change behaviors. Finally, the family focuses 

on sustaining the changes it has made and preventing relapses.12 At the end of FFT, the therapist 

administers the Youth Outcome Questionnaire again (and potentially other assessments) to 

assess treatment progress and determine if the youth or family should be referred to another CII 

program or activity. 

FFT is a relatively short-term treatment, on average ranging from 8 to 12 one-hour sessions for mild situations to as many as 30 sessions for more difficult situations.13 Sessions 

typically take place on a weekly basis over a three- to four-month period. Therapists will 

discharge families from treatment if they cannot make contact for one month. FFT has been 

conducted in clinical, outpatient, and home-based settings. Therapists at CII reported that they 

hold sessions at school, home, or the CII office, depending on what is most convenient for the 

family. Because the whole family participates in FFT, therapists usually hold sessions in the 

evening. “We meet the family where they are at,” a slogan therapists at CII often repeated. 

Typically, therapists conduct FFT as part of a team-based program such as Wraparound services 

or the Full Service Partnership program. During sessions, therapists engage the whole family 

and often use interactive, hands-on activities such as board and card games and improvisation. 

Box 5.2 presents one client’s experience with FFT.

Adopting FFT

Functional Family Therapy LLC (FFT LLC) developed FFT and is responsible for disseminating it and ensuring model fidelity. It requires that all providers throughout the world

 10Waldron and Turner (2008); Parsons and Alexander (1973).

11The child and parent or guardian complete different versions of the questionnaire. For a description of 

the questionnaires, see www.oqmeasures.com. 12Sexton and Alexander (2000).

13The Functional Family Therapy, LLC (FFT LLC) website states that the average treatment consists of 12 

to 14 sessions spread over three to five months, which varies slightly from what CII staff reported. 
61

complete a rigorous training and certification process and regularly monitors providers for 

fidelity assurance. For instance, FFT LLC operates the Clinical Service System, a computerbased tracking and assessment system in which providers must log information about their 

cases and progress in FFT.

14

 14At CII, therapists must also log information about their cases and progress in FFT in CII’s management 

information system. 

Box 5.1

Phases of FFT*

• Engagement: The goal of this phase is to build trust in the family-therapist relationship, 

so that family members are invested in the treatment process. The therapist demonstrates a 

commitment to be supportive, responsive, and respectful of the family by being available 

to address the family’s needs and by maintaining contact with all family members.

• Motivation: In this phase, the therapist works to increase a sense of hope and decrease 

hostility within the family by suggesting alternatives to maladaptive family dynamics. The 

therapist maintains a strengths-based approach to improving relationships, exploring consequences of positive and negative interactions and orienting family members toward a 

positive outlook.

• Relational Assessment: In this phase, the therapist identifies interaction patterns, focusing on each dyad within the family to understand the motivations or relational “functions” 

for each individual’s behaviors. Through observation and questioning, the therapist develops a relational perspective to inform later phases of treatment.

• Behavior Change: The therapist focuses on skill building to address behavior that 

prompted the referral to FFT. The therapist teaches communication skills using tasks, 

technical aids, and modeling strategies, while maintaining a culturally appropriate, context-sensitive, and tailored approach to address the family’s needs.

• Generalization: The goal of this phase is to build on progress made in the previous phase 

by applying newly developed skills to multiple contexts. Family members learn to address 

future challenges and plan for relapse prevention. This phase focuses on incorporating 

community systems (including education and justice systems) into the treatment process 

and helping the family establish links in the community.

__________________________

*

Functional Family Therapy, LLC (2015).
62

Box 5.2

Maya’s Story

The Los Angeles County Department of Children and Family Services referred Maya to CII 

when she was 7 years old, shortly after her biological father reunited with her and removed 

from her mother’s care. Maya and her father had difficulties adjusting to the change and she 

began exhibiting behavioral problems.

Staff at CII recommended Maya for Incredible Years, an evidence-based group intervention 

that aims to strengthen parent-child relationships. After completing the program, Maya 

showed a notable improvement in her behavior. Maya continued to receive services at CII for 

another two years, meeting weekly with a therapist to continue to work on her treatment goals, 

which included fewer outbursts toward her father.

At age 9, two months after reaching her treatment goals and being discharged by her therapist, 

Maya returned to CII. Prompted by an incident at school that suggested suicidal intent, the Los 

Angeles County Department of Children and Family Services referred Maya for additional 

mental health services. Maya was referred to Functional Family Therapy (FFT), with treatment goals centered on reducing Maya’s outbursts of anger, which were often directed toward 

her father. Maya had not reached the minimum age required by the FFT model, but because of 

her maturity level, prior participation in CII’s services, and removal from her mother’s care, 

CII staff considered the model an appropriate treatment for her. In making the determination, 

CII staff sought guidance from FFT LLC.

Maya’s FFT therapist visited her and her father at their home, and began treatment by engaging and motivating them with discussions of trauma and focusing on emotional development 

and psychoeducation. The therapist explained that Maya and her father had different ways of 

thinking, which sometimes led to conflicts between the two. Through games and discussion, 

she helped them begin to identify barriers in their communication.

After about two months, the therapist and family proceeded to the behavior change phase. The 

therapist recommended various skills and strategies for Maya and her father with which to 

work together to improve their interactions, including paying each other more compliments, 

creating a chore chart, and organizing family dinners and movie nights. After another two 

months, they were ready to generalize the skills learned, helping Maya learn to apply them in 

different settings, and plan appropriate responses to her triggers of aggressive behavior.

By the end of her treatment, Maya stated that she felt close to her father and enjoyed discussing her feelings with him. On her FFT discharge form, Maya indicated that before FFT her 

relationship with her family had been “very bad;” afterward, however, she considered it 

“mostly good.” Using the same questionnaire, her father also noted an improvement, noting 

that they had progressed from “so-so” to “very good.” On their forms, they both also indicated 

that they were hopeful or confident that their family dynamic would improve.
63

Organizations wishing to implement FFT must complete a thorough certification process. Once an organization applies to and is approved by FFT LLC to implement FFT, the 

organization enters a contract with FFT LLC and prepares for FFT training, which includes

selecting or hiring therapists and purchasing necessary materials and assessments. The first of 

three phases lasts 12 to 18 months and begins with an orientation to FFT and its data collection 

requirements. Therapists must log data about their FFT clients, sessions, and progress into the 

Clinical Service System; the organization and FFT LLC review this information. After orientation, therapists receive clinical training in FFT from certified trainers. Once they successfully 

complete training, therapists begin using the practice to treat clients under the supervision of an

FFT consultant, with whom the therapists have weekly teleconferences to review their cases and 

monitor their fidelity to the model. In California, organizations implementing FFT enter a 

contract with the California Institute for Behavioral Health Solutions (formerly California 

Institute of Mental Health), which provides ongoing supervision, training, and support to all 

FFT providers. 

In the second phase, which lasts one year, the organization takes steps to deliver FFT 

independently. The organization’s FFT supervisor attends training and subsequently takes over 

the weekly supervision of therapists from the FFT consultant or contractor. The supervisor 

instead meets with the consultant or contractor every other week. The supervisor also reviews

the information that therapists log into the Clinical Service System each week and gives them a 

fidelity score every quarter. The California Institute for Behavioral Health Solutions continues 

to provide ongoing supervision to providers in the state, but the organization assumes more 

responsibility for ensuring model fidelity. 

In the third and final phase, FFT LLC gives the organization more independence, and

FFT consultants or contractors only meet with supervisors once a month. The organization can

also renew its contract with FFT LLC for a one-year term in this phase. As part of the contract, 

FFT LLC provides continuing education and training.

FFT Infrastructure at CII

CII manages two teams of FFT therapists.15 The therapists work within programs, such 

as Wraparound services and the Full Service Partnership program, or provide standalone 

therapy. They serve families referred to them from schools, the Los Angeles County Department of Children and Family Services, or other entities. Ideally, therapists provide only FFT 

and no other therapies given the model’s intense demands. Therapists must attend weekly two-

 15At one time, CII managed three teams.
64

hour team supervision meetings, during which two therapists each week present cases that the 

team evaluates together.16 Therapists also receive individual supervision as needed. 

Executive staff at CII and FFT LLC and others overseeing FFT tend to agree that it 

takes a certain kind of person to implement and deliver FFT successfully. For example, one 

executive staff member at CII stated that the organization had learned to choose therapists who 

could work nights and occasional weekends since the therapy requires all family members to 

participate, including those who work regular business hours. Experience also taught CII that 

therapists must be 100 percent devoted to FFT to optimize the model. In other words, they 

cannot use other evidence-based practices or offer general therapy. “I think that out of all the 

EBPs [evidence-based practices] that we struggled the most with FFT,” explained an executive 

staff member. “I think the treatment is very intense. The kids are involved often in the legal 

system and they’re very high needs. Because you need to work with the family, the therapists 

that were doing FFT, all of their appointments were at night.”

Implementation of FFT

Analysis of FFT data from the CII management information system indicates CII implemented the model with fidelity with respect to treatment dosage.17 As Table 5.2 shows, 

families on average completed about 15 sessions over the course of five months.18 More than 

half of all clients attended 13 or more sessions. This number exceeded the expected 8 to 12 

sessions and may be explained by therapists extending treatment to address complex trauma, 

multiple behavioral issues, or family crises. About one-quarter of clients receiving FFT attended

five or fewer sessions; it can be assumed that these clients did not complete the treatment. A 

client may have stopped treatment for a number of reasons. For instance, the therapist may have

 16The FFT therapists at CII are currently participating in the Building Outcomes with Observation-Based 

Supervision: An FFT Effectiveness Trial (BOOST), a study of FFT supervision practices within the California 

Institute for Behavioral Health Solutions system. They were randomly assigned to two groups: a “supervision 

as usual” group (the control group) and a BOOST group which receives additional supervision from FFT LLC. 

See clinicaltrials.gov/show/NCT01614015. 17Fidelity consists of a number of dimensions, and researchers focus on different aspects of fidelity. These 

aspects include: targeting the specific population, providing the treatment with the appropriate strength (such as 

dosage and frequency of sessions), and adhering to the specified treatment model (therapist adherence). Due to 

the limited data, this study only assessed the fidelity of FFT implementation with respect to specific aspects of 

treatment strength. 18In comparison, the California Institute for Behavioral Health Solutions collects program and outcome 

data on FFT clients for agencies in California. In the most recent available data, the average number of sessions 

was 14. These data represent clients who participated in FFT through the California state-sponsored FFT 

Community Development Teams through the end of January 2009. Each dashboard report reflects only those 

sites that submitted data at the specified data submission date. See California Institute for Behavioral Health 

Solutions (2009). 
65

determined FFT was inappropriate for a family after a few sessions or because the child was 

placed in foster care or the family situation had changed in some other significant way. The 

data, however, did not indicate the reasons why clients left treatment. The average total FFT 

session time at CII aligned with the model’s expected 8 to 12 hours, after the research team 

subtracted the travel time (often to the family’s home) and the time taken to complete the 

required documentation for the funder and FFT LLC that therapists included in the session time 

they recorded. There were some differences in dosage between SPA 4 and SPA 6, although it is 

unclear what drove these differences. While the dosage that therapists at CII delivered appears 

in line with the recommended dosage, this study could not assess fidelity to FFT within these

Characteristics All CII SPA 4 SPA 6

Participated in any session (%)

Attending 1-2 sessions 16.5 15.0 23.3

Attending 3-5 sessions 8.2 10.0 16.7

Attending 6-12 sessions 17.7 15.0 16.7

Attending 13-20 sessions 28.2 50.0 23.3

Attending 21 or more sessions 29.4 10.0 20.0

Average number of sessions 15.3 13.4 11.7

Average total session timeb

 (hours) 33.1 26.6 26.0

Duration of sessions (months) 4.9 4.4 4.0

Sample size 85 20 30

Table 5.2

Client Participation in Functional Family Therapya

SOURCE: CII management information system.

NOTES: Analysis is limited to clients who enrolled from January 1, 2012, through 

June 30, 2013, and received services from January 1, 2012, through December 31, 

2013. aTo minimize the influence of data entry errors, participation in Functional Family 

Therapy was determined by using only those records associated with a trained 

Functional Family Therapy clinician. The data were further limited to only those 

records with procedure codes that should be attributed to the therapy.

bTotal session time includes time spent by the therapist conducting the session, as 

well as traveling and completing documentation required by the Los Angeles County 

Department of Mental Health, CII, and the model's developers. As a result, reported 

session times and calculated averages are higher than expected of typical Functional 

Family Therapy sessions.
66

treatment sessions. One therapist explained that ensuring fidelity with respect to dosage was 

easier when providing FFT as part of Wraparound services or the Full Service Partnership 

program because the therapist could rely on case managers and parent partners to address 

nonclinical needs. When delivering it as a stand-alone service, a family with nonclinical needs 

might sidetrack the therapist from treatment progress and thereby extend the length of the 

treatment (or dosage). 

Trauma-Focused Cognitive Behavioral Therapy (TF-CBT)

TF-CBT incorporates principles from cognitive, behavioral, and family therapy. Unlike FFT, 

which requires the participation of the whole family, TF-CBT primarily focuses on the child 

and only involves the parents or guardian at specific points in therapy. Cognitive therapy aims 

to change behavior by addressing an individual’s thoughts and perceptions, especially those that 

create distorted views; behavioral therapy seeks to modify habitual responses as a means to 

identify triggers; and family therapy examines patterns of interactions among family members 

to alleviate problems.19 With respect to family therapy, TF-CBT requires that the parent or 

guardian who did not cause the trauma attend separate sessions to learn how to support the 

child, improve parenting practices, and reduce emotional distress about the child’s trauma.

20 In 

TF-CBT, the therapist also meets with the child and the parent or guardian jointly to help them 

gradually confront and process and the trauma using cognitive behavioral techniques, such as 

teaching them how to disconnect negative emotions from trauma-related thoughts. The therapist 

guides the child in preparing the “trauma narrative,” who then shares it with the parent or 

guardian verbally, in writing, or artistically. In TF-CBT, trauma-affected clients gradually 

confront and process traumatic experiences, which in turn diminishes the prevalence of anxiety 

and avoidant responses, while the parents or guardians learn to therapeutically support their 

children.21

Originally developed to treat sexually abused children,22 TF-CBT has proven to be effective in treating children ages 3 to 18 years with significant behavioral problems or emotional

difficulties, such as post-traumatic stress disorder (PTSD), depression, anxiety, and feelings of 

shame and self-blame associated with exposure to traumatic events.

23 Examples of traumatic 

 19Barker (2003).

20Cohen, Mannarino, Murray, and Igelman (2006).

21Deblinger, Lippmann, and Steer (1996); Deblinger and Heflin (1996). 22Cohen, Deblinger, and Mannarino (2004).

23A number of studies have demonstrated the effectiveness of TF-CBT in helping children overcome 

symptoms following a traumatic event. See Deblinger, Lippman and Steer (1996); Cohen and Mannarino 

(1996a,b, 1998a,b); Deblinger, Stauffer, and Steer (2001); Cohen, Deblinger, Mannarino, and Steer (2004).
67

events include physical or sexual abuse, community violence, domestic violence, or accidents.24

TF-CBT has also proven to be effective in reducing the symptoms of depression and anxiety in

parents or guardians of trauma-affected children, as well as in improving their parenting 

practices.25 Box 5.3 presents one CII client’s experience before and after TF-CBT. 

To determine if TF-CBT is an appropriate treatment for a client, therapists at CII screen 

the client using the University of California Los Angeles (UCLA) PTSD-Reaction Index. Los 

Angeles County Department of Mental Health — the primary funder of TF-CBT services — 

also requires that clients and their parents or guardians complete the Youth Outcome Questionnaire before starting treatment. TF-CBT’s consists of the following central components: 

psychoeducation and parenting skills, relaxation, affective modulation skills, cognitive coping 

and processing, trauma narration and processing, in vivo desensitization, conjoint child-parent 

session, and enhancing safety skills (referred to as PRACTICE).26 Box 5.4 describes each 

component in detail. “The nice thing about TF-CBT is that it provides a framework,” explained 

one therapist. “You have the luxury of having an assessment period to where you can identify 

where the client sees the greatest need and provide psychoeducation. It helps them understand 

their situation from a more removed perspective and then gradually deal with their personal 

trauma. It gradually becomes more intimate and challenging.”

To deliver TF-CBT with fidelity, the therapist must use each of the PRACTICE components, and in the correct order, during the treatment within a “reasonable” time period.27 The

model does allow the therapists to skip a few components in particular situations. For example,

the therapist does not need to deliver the in vivo exposure component if the child is not experiencing trauma-related triggers. The therapist may also omit the trauma narrative component if 

an appropriate parent or guardian is unavailable. Therapists have flexibility to tailor sessions to 

fit the specific needs of the client. Therapists can use different techniques such as play or art 

therapy. Furthermore, the therapist may need to modify TF-CBT to accommodate a client’s

cultural background. For example, a therapist described a sexual abuse case, in which the client 

expressed cognitive distortion associated with shame that indicated the client’s religious

 24According to the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR), a traumatic event involves (1) actual or threatened death or serious injury, or a threat to 

one’s physical integrity, or witnessing an event that involves death, injury, or a threat to physical integrity of 

another person; or learning about unexpected or violent death, serious harm, or threat of death or injury 

experienced by a family member or other close associate; and (2) the person’s response to that event must 

involve intense fear, helplessness, or horror. See American Psychiatric Association (2000). 25Cohen, Mannarino, Murray, and Igelman (2006); Cohen and Mannarino (1997); Deblinger, Steer, and 

Lippmann (1999). 26Cohen, Mannarino, Murray, and Igelman (2006).

27The TF-CBT developers do not explicitly set parameters for a “reasonable” time period. 
68

Box 5.3

Sofia’s Story

In March 2013, 10-year-old Sofia visited CII for treatment. Her mother was concerned because Sofia cried frequently and was very sensitive and shy. She was also frequently irritable 

and had difficulty getting along with her two younger siblings.

Sofia’s mother believed that her daughter’s problems began four years earlier, soon after her 

separation from Sofia’s father. Then, two years after the separation, Sofia’s baby sister had an 

accident while in her care, resulting in a head injury. The accident prompted an investigation 

by the Los Angeles County Department of Children and Family Services and ultimately 

resulted in the removal of Sofia and her two sisters from their mother’s care. Sofia’s experience of the accident and its consequences traumatized her and led her to develop strong 

feelings of guilt and regret.

Following the accident, Sofia lived with her father for eight months. After her mother regained 

custody, Sofia began experiencing anxiety, especially when separated from her mother. Sofia 

felt guilty about the accident and its consequences.

As part of CII’s initial assessment process, Sofia met with a therapist who helped her complete 

an assessment of her past exposure to trauma. Sofia’s mother completed a parent version of the 

assessment, as well as another assessment designed to identify any other behavioral problems.

After Sofia’s initial assessment, she began in-home TF-CBT to reduce the frequency of the 

post-traumatic stress disorder symptoms that led her mother to seek help, including irritability 

and sensitivity. Sofia’s therapist, Marissa, began treatment with sessions devoted to the 

psychoeducation component of TF-CBT, teaching Sofia about accidents. She explained to 

Sofia the difference between regret and fault and spoke with her about how children may feel 

when involved in accidents. During these sessions, Sofia also learned and practiced stress 

management techniques such as controlled breathing.

With Marissa’s guidance, Sofia gradually progressed to the affect identification component of 

TF-CBT. Sofia spoke about her sister’s accident, and identified the resulting emotions she 

experienced. During their seventh session, Marissa introduced the trauma narrative component, working with Sofia to develop an account of the accident and helping her feel comfortable discussing it. By the tenth session, Sofia was able to practice sharing her narrative, with 

Marissa playing the role of her mother.

Throughout the course of treatment, Sofia’s mother participated in separate sessions with 

Marissa, where she learned about Sofia’s progress as well as strategies to support her daughter 

and reinforce her growth. She also participated in a joint session, in which Sofia shared the 

trauma narrative with her. Coached by Marissa, Sofia’s mother was able to positively reinforce 

and praise her daughter for sharing her narrative. She noted that Sofia seemed calmer and 

more confident, and Sofia agreed that she felt more relaxed overall.
69

upbringing, namely “I’m not a virgin anymore and am going to hell.” The therapist adjusted

TF-CBT by connecting the client with a priest, who explained that virginity is not something 

one person can take from another through abuse.

Therapists provide TF-CBT in the client’s home, at school, in the CII office, or wherever else the client is comfortable. Therapists do not typically deliver the treatment as part of a 

Box 5.4

TF-CBT Components*

Psychoeducation and Parenting Skills: Educating child and parent about typical reactions to 

trauma. Normalizing symptoms and reducing self-blame, using games, videos, and discussion. 

Parents receive coaching in effective behavior management strategies and communication.

Relaxation: Teaching children relaxation techniques including focused breathing, mindfulness, 

progressive muscle relaxation, and visual imagery.

Affective modulation skills: Helping children to identify and understand their emotions. 

Discussing feelings associated with their trauma and developing strategies to regulate emotions.

Cognitive coping and processing: Revisiting the role of thoughts in driving emotions, and 

helping children identify harmful or inaccurate thoughts. Teaching children that thoughts can 

be changed and helping to generate more accurate and positive thoughts.

Trauma narration and processing: Providing gradual exposure to trauma-related memories, by 

recounting the experience. Helping to identify inaccurate trauma-related thoughts and replacing them with more accurate thoughts about the traumatic event, one’s self, and others.

In vivo desensitization: Reducing avoidance that disrupts daily functioning by teaching 

children to distinguish trauma reminders from actual danger. Gradually exposing children to 

triggers and helping children practice regulating their emotions and reactions.

Conjoint child-parent session: Providing an opportunity for supportive communication between child and parent or guardian. Children share their trauma narrative with their parents or 

guardians and parents or guardians offer praise and encouragement. Children prepare for the 

conjoint session sharing the full trauma narrative with the therapist over several prior sessions.

Enhancing safety skills: Creating a written safety plan that identifies risks or triggers and 

provides prepared responses to help children feel safer after completion of treatment. Teaching 

skills for use in dangerous situations that may arise in the future.

__________________________

*

Cohen, Mannarino, Murray, and Igelman (2006).
70

team-based program such as Wraparound services or the Full Service Partnership program. 

Therapists conduct 60- to 90-minute sessions once a week for 12 to 20 weeks, depending on the 

complexity of the trauma. 28 According to therapists at CII, the relatively short, finite duration of 

the treatment — unlike traditional open-ended therapy, for instance — is attractive to families 

and can promote their engagement. At the end of treatment, therapists again administer the 

UCLA PTSD-Reaction Index and Youth Outcome Questionnaire to gauge clinical improvement and determine if a client should continue treatment — either TF-CBT or another practice. 

Adopting TF-CBT

Implementing and delivering TF-CBT involve fewer and less stringent requirements 

than FFT, and providers have sole responsibility for overseeing and supervising the implementation and delivery. The developers of TF-CBT do not regularly monitor the intervention; 

however, at some providers (including CII), therapists must engage in some TF-CBT supervision. For instance, therapists and their supervisors use the TF-CBT developers’ Brief Practice 

Checklist to track the progress of each client through the TF-CBT components and to support 

therapist adherence to the model. Although the developers offer a certification in TF-CBT, they 

do not require it to deliver the model.29 The Los Angeles County Department of Mental Health, 

however, does set some requirements for the TF-CBT providers it funds, including CII, such as 

submitting baseline and outcome data to the California Institute of Behavioral Health Solutions, 

which tracks model fidelity and providers’ performance.

TF-CBT Infrastructure at CII

CII requires therapists providing TF-CBT to participate in group supervision. CII operates two groups: one for therapists new to TF-CBT, which focuses on problem solving and case 

presentation, another for more experienced therapists, which meets every other week and 

focuses on aspects of cultural modification.

According to therapists interviewed by the research team, TF-CBT works best in a stable home since clients are usually confronting the most difficult events in their lives and need a 

supportive environment. However, given that CII serves low-income urban communities, 

therapists cannot always assume clients live in stable homes. Relatedly, TF-CBT theoretically 

focuses on the trauma-affected client, not the client’s living situation or community. At the end 

of treatment, clients are often still living in the same homes or communities associated with 

their trauma. 

 28Child Sexual Abuse Task Force and Research and Practice Core, National Child Traumatic Stress Network (2004). 29See Trauma-Focused Cognitive Behavioral National Therapist Certification Program website: tfcbt.org. 
71

Implementation of TF-CBT

Table 5.3 presents the analysis of TF-CBT treatment dosage during the study period. 

Findings indicate that the dosage is more or less aligned with model expectations. Clients 

typically participated in TF-CBT for 21 weeks, attending 19 sessions over that time, which 

averages close to one per week. About one-quarter of clients attended the expected 12 to 20 

sessions in that period, whereas 30 percent attended 10 or fewer sessions. These clients may 

have attended a small number of sessions because clients ended their treatment at CII or 

therapists referred them to a different treatment model. In comparison, the most recent data 

from the California Institute for Behavioral Health Solutions, which collects program and 

outcome data on TF-CBT clients in California, indicate that clients on average attended 19.5 

sessions over 26.1 weeks.

30 The average total TF-CBT session time at CII aligned with the 

model’s expected 12 to 30 hours, after the research team subtracted the travel time (often to the 

family’s home) and the time taken to complete the required documentation that therapists 

included in the session time they recorded. There are some differences in dosage between SPA 

4 and SPA 6, although it is unclear what drove these differences.

TF-CBT Fidelity Study Findings

The fidelity study of CII’s TF-CBT services used an observational method. (A Technical Resource for this report presents the full study and is available on the MDRC website.) CII 

offered new clients assigned to TF-CBT between November 2013 and August 2014 the opportunity to participate in the study; 126 clients enrolled in the study. CII distributed study participants across 31 therapists, who would audio record each treatment sessions, including clientonly sessions, parent- or guardian-only sessions, and conjoint sessions. During the period 

between November 2013 and February 2015, therapists collected more than 1,009 recordings, 

half of which a research team from the Medical University of South Carolina (MUSC) coded. 

The MUSC team selected recordings evenly across several factors: number of therapists, 

number of clients per therapist, and number of sessions per client. The team coded four clients 

per therapist and weighted the sampled recordings to ensure adequate representation of coded 

sessions across each TF-CBT component. 

The MUSC team used a version of the Therapy Process Observational Coding System 

for Child Psychotherapy designed for TF-CBT (TF-CBT TPOCS-S) to assess therapists’ 

 30These numbers are based on data submitted to California Institute for Behavioral Health Solutions in 

September of 2010 and represent clients who received TF-CBT through the Institute’s TF-CBT Community 

Development Teams through the end of July 2010. Each dashboard report reflects only those sites that 

submitted data at the specified data submission date. See California Institute for Behavioral Health Solutions

(2010).
72

adherence to the model through all of its 12 components.

31 The team then rated their use of and 

fidelity to these components on a seven-point scale. Fidelity was assessed across a course of 

treatment, reflecting that TF-CBT components are intended to be delivered in phases and

that it is not expected that every session will include all of the components.

 31The TF-CBT TPOCS-S is an observational coding system that the National Institutes of Mental Health 

developed as part of a multiyear randomized controlled trial studying the effects of clinical supervision 

strategies on how a large sample of community-based therapists in Washington State implemented TF-CBT. 

The tool is not yet publicly available.

Characteristic All CII SPA 4 SPA 6

Participated in any session (%)

Attending 1-5 sessions 19.8 16.9 11.6

9.9 9.7 10.5

11.9 6.5 23.3

16.2 16.9 22.1

14.9 16.1 15.1

27.4 33.9 17.4

19.3 21.3 18.0

25.5 23.6 30.5

21.4 22.7 22.1

Attending 6-10 sessions

Attending 11-15 sessions

Attending 16-20 sessions

Attending 21-25 sessions

Attending more than 26 sessions

Average number of sessions

Average total session timeb

 (hours)

Duration of sessions (weeks)

Sample size 303 124 86

Table 5.3

Client Participation in Trauma-Focused Cognitive Behavioral Therapya

SOURCE: CII management information system.

NOTES: Analysis is limited to clients who enrolled from January 1, 2012, through June 30, 

2013, and received services from January 1, 2012, through December 31, 2013. aTo minimize the influence of data entry errors, participation in Trauma-Focused 

Cognitive Behavioral Therapy was determined by using only those records associated with 

a trained Trauma-Focused Cognitive Behavioral Therapy clinician. The data were further 

limited to only those records with procedure codes that should be attributed to the therapy.

bTotal session time includes time spent by the therapist conducting the session, as well as 

traveling and completing documentation required by the Los Angeles County Department 

of Mental Health, CII, and the model's developers. As a result, reported session times and 

calculated averages are higher than expected of typical Trauma-Focused Cognitive 

Behavioral Therapy sessions.
73

In addition to measuring therapists’ adherence to the TF-CBT model, the study also 

evaluated the psychometric properties of the TF-CBT Brief Practice Checklist, the selfreporting fidelity monitoring tool therapists at CII use. Because observational methods such as 

the TF-CBT TPOCS-S are resource intensive, the research team wanted to assess the value and 

effectiveness of this less costly fidelity monitoring tool. Therapists complete the Brief Practice 

Checklist after each session, noting which components of TF-CBT they implemented in each 

session. TF-CBT supervisors review the checklists during group supervision meetings. The 

fidelity study found that, through the course of treatment, CII clients were more than 50 percent 

likely to receive half of the components. This finding indicates that CII therapists did not deliver 

the model’s full prescribed course of treatment and is in line with previous research. Clients 

were most likely to receive the cognitive coping, relaxation, affective expression and modulation, psychoeducation, and trauma narrative components. Notably, previous studies of community-based TF-CBT providers found that clients were less likely to receive the trauma narrative 

component.32 In contrast, nearly all CII clients received this component. Therapists at CII 

implemented the in vivo exposure and parenting components the least frequently. The fact that 

therapists only conduct in vivo exposure when clients are experiencing ongoing trauma-related 

triggers in their everyday environment — a subset of CII clients — may be one reason why 

therapists did not often implement that component. And the low number of parent- or guardianonly and conjoint sessions in the study sample may explain the infrequent use of the parenting 

skills component. The TF-CBT model prescribes a relatively equal number of sessions for 

clients and parents or guardians. However, in the study sample, 63 percent of sessions were 

client only, 35 percent were parent or guardian only, and 2 percent were conjoint. The low 

number of parent- or guardian-only sessions was consistent with findings from previous studies 

of community-based TF-CBT providers. Since the study included only observational and selfreported data on parent or guardian involvement, it is not possible to determine the reasons for 

this low level of parent or guardian involvement. Therapist may have chosen not to involve the 

parents or guardians at the levels prescribed by the model, or they may have encountered 

challenges in engaging parents or guardians in treatment similar to those therapists in other 

studies faced.

33

These findings are consistent with what the research team learned from interviews 

with TF-CBT therapists in the implementation study. Therapists gave a number of reasons 

why TF-CBT can be difficult to implement and why they may not implement it with complete 

fidelity. Clients in TF-CBT tend to have a history of multiple traumas and are often experiencing ongoing crises, and therefore therapists might have to conduct a crisis intervention in 

the midst of TF-CBT treatment. Such an intervention may prolong TF-CBT, or sometimes 

 32Allen and Johnson (2012).

33Cohen, Mannarino, Kliethermes, and Murray (2012).
74

compel a therapist to temporarily suspend the therapy until the client is stabilized or not as 

high risk. Therapists at CII also described challenges in properly balancing what happens in 

clients’ lives from week to week with making progress in treatment. The trauma narrative 

component, in particular, causes distress and clients often cancel sessions leading up to it. 

These interruptions can derail treatment, and reengaging clients can be difficult. Consistent 

with these accounts, the MUSC team found that nearly all of the sessions they coded included

crisis or case management.

The fidelity study aimed in part to assess how model fidelity varied by therapists’ characteristics. Using TF-CBT TPOCS-S, the MUSC team found that fidelity varied at the client 

level, rather than at the therapist level. In other words, clients treated by the same therapist 

experienced TF-CBT differently. There were no data available to assess the client-level factors

or characteristics that might explain this variability or the extent to which the modifications 

therapists made to accommodate individual clients were consistent with model fidelity.

The MUSC team also found that using the Brief Practice Checklist or the observational 

method to assess model fidelity over the course of treatment led to similar findings. However, 

consistent with similar studies, the team found that therapists were more likely to over-report 

adherence to the TF-CBT model on the Brief Practice Checklist. In addition, whereas the TFCBT TPOCS-S attributed variations in fidelity more to clients’ characteristics than those of

therapists, the study found that the Brief Practice Checklist attributed a higher percentage of the 

variation in fidelity to the therapists’ characteristics. This finding suggests that individual 

therapists respond differently to questions on the checklist.

The study nonetheless indicates that the Brief Practice Checklist may be a promising 

low-cost tool to monitor fidelity. Observational methods of monitoring fidelity, such as the one 

used in this study, are time and resource intensive and not practical on a large scale for many 

community-based organizations. Therapists and supervisors could use the Brief Practice 

Checklist to monitor the delivery of TF-CBT’s components, and supervisors could use the 

information on the checklist to advise therapists on how to address roadblocks to providing the 

treatment as intended. Additionally, organizations could use data from the checklists to compare 

differing outcomes among clients and identify and assess any patterns. However, there are some 

limitations to using the checklist alone to evaluate fidelity. Therapists in the study tended to 

over-report their use of the therapy’s components relative to findings from the TF-CBT 

TPOCS-S. Additionally, whereas observational methods such as the TF-CBT TPOCS-S can 

measure the extent to which therapists implemented each component, the Brief Practice Checklist can only determine whether or not therapists used each component. However, organizations 

could use the Brief Practice Checklist in combination with others tools, such as periodic direct 

or audio-recorded observations, to monitor fidelity more accurately.
75

The study included a fidelity assessment of TF-CBT delivered in Spanish, though small 

sample sizes limited the analysis to single therapy sessions rather than the course of treatment. 

The assessment used the TF-CBT TPOCS-S and found that therapists were more likely to 

deliver parenting skills in sessions conducted in Spanish, perhaps because a greater proportion 

of the Spanish-language sessions in the sample were parent- or guardian-only or conjoint 

sessions relative to the English-language sessions. Therapists in Spanish-language sessions were 

less likely to implement the trauma processing component than those in the English-language

sessions.

Conclusion

CII has been a pioneer in adopting and delivering evidence-based practices. However, it is 

difficult to interpret the saturation of clients engaged in evidence-based practices without 

knowing more about each client’s circumstances, since these practices are not appropriate for all 

clients in every circumstance. 

A sizable proportion of clients receiving CII’s clinical services engage in an evidencebased practice. Analysis of data from CII’s management information system indicate that the 

dosage of FFT aligns with model expectations. The dosage of TF-CBT also appears to align 

with model expectations. However, the TF-CBT TPOCS-S found that therapists, on average,

did not deliver all of the prescribed TF-CBT components. This finding corroborates those from 

previous fidelity studies of community-based TF-CBT providers, underscoring the need to 

improve supports for therapists to help them provide treatment with fidelity. The TF-CBT 

fidelity study also found the Brief Practice Checklist to be a promising low-cost tool for 

monitoring fidelity, though it alone may not be enough to ensure fidelity.

77

Chapter 6 

Discussion

There is overwhelming evidence that traumatic experiences in childhood can lead to poor 

outcomes in adulthood. While an extensive child welfare field — encompassing policymakers, 

researchers, and practitioners — focuses on improving the life prospects of these children and 

families, the services available to them are often fragmented and uncoordinated. Service 

providers are spread across different agencies, and funding streams often support only specific 

types of care or treatment. Many of the available services also lack evidence of their effectiveness, though there has been a push in recent years to increase the use of evidence-based practices in children’s mental health care.

This report describes in depth how Children’s Institute, Inc. (CII), tackles these issues. 

Through its Integrated Service Model, CII aims to provide holistic and coordinated support to 

children and families through a combination of clinical and nonclinical services. CII also 

prioritizes delivering evidence-based practices to clients when appropriate. CII serves as an 

important case study that may be useful to other multiservice organizations addressing these 

issues. This chapter summarizes the report’s findings and identifies areas for further research.

Integrated Service Model

Through its Integrated Service Model, CII attempts to knit services together to address the 

complex needs of the children and families it serves. Clients receive multiple types of services, 

depending on their needs, throughout their involvement with CII. The Integrated Service Model 

intends not simply to provide clients with multiple types of services, but to eliminate operating 

silos within CII and to create a system that accurately identifies clients’ full range of needs and 

ensures they receive all the support required to address those needs.

Based on the interviews and available data, MDRC researchers found that CII is 

achieving its goal of engaging clients in multiple types of services to meet their needs. An 

overwhelming majority of clients receiving clinical services were engaged in more than one 

type of service. Staff expressed support for the model and described how it benefited their 

clients. To implement the model, CII had to overcome the challenges of coordinating care 

across a fragmented system. Flexible funding was essential to providing clients with nonclinical services. Further research is needed to determine whether CII appropriately targets its 

services — namely, whether all clients who need multiple types of services receive them. 

Additional research is also needed to assess the “value-added” of the different types of services 

CII provides, for instance whether its model improves short- or long-term outcomes for clients. 
78

At the time of the study, CII was not collecting the necessary data in a way that would have 

allowed the research team to conduct these assessments.

Evidence-Based Practices

Though momentum has been building across the country to increase the use of evidencebased practices, adopting and delivering them in community-based settings comes with a host 

of challenges. Whether or not an evidence-based practice is effective depends on how the 

community-based provider implements it. In order to transfer efficacy from research to 

practice, providers must implement the practice with fidelity to the treatment model, which 

can be particularly challenging for community-based organizations such as CII.

This report describes how CII incorporates evidence-based practices into its clinical 

services. Nearly half of clients receiving clinical services at CII engaged in an evidence-based 

or evidence-informed practice, which according to the limited data available is a much higher 

saturation rate than that of other mental health providers. This report identifies the challenges 

that CII encountered implementing evidence-based practices, including securing the additional 

resources required to train and supervise staff and delivering a specified treatment model to

high-risk clients. Fidelity consists of several components, including adherence to the treatment 

model, dosage, and the quality of implementation. Due to the study design, MDRC researchers 

could only assess particular aspects of fidelity for each of the two evidence-based practices they 

examined. For Functional Family Therapy, they studied only fidelity to dosage, finding that 

clients at CII seemed to have received the model’s expected dosage of treatment. For TraumaFocused Cognitive Behavioral Therapy (TF-CBT), the research team evaluated fidelity to the 

treatment model and dosage, finding that clients received close to the model’s expected dosage 

of treatment. However, in terms of fidelity to the treatment model, the team found that therapists 

on average did not deliver all of TF-CBT’s required treatment components. Clients receiving 

the average dosage of TF-CBT at CII had at least a 50 percent chance of receiving half of the 

required components. This finding is consistent with findings from similar TF-CBT fidelity 

studies. Notably, TF-CBT therapists at CII delivered the trauma narrative component at a rate 

higher than the average rate suggested by previous studies. These challenges underscore the 

need for effective tools to support fidelity. This study found that the therapist self-report tool, 

the Brief Practice Checklist, may be an effective, low-cost tool for monitoring fidelity, albeit 

with some limitations.

Changes at CII

This evaluation is based on data gathered over a period of approximately two years (2012 and 

2013), which was a period of tremendous change at CII. Conducting an assessment of the 
79

Integrated Service Model while it was under development was challenging and limited the 

scope of the findings. During the study period, CII was in the process of scaling up its Integrated Service Model, working to improve its data systems and how it measures outcomes, streamlining its client enrollment and assessment process, and restructuring management. Additionally, during the study period, CII increased its operating budget as well as the number of clients it 

served and somewhat modified specific programs and services. In light of these many changes, 

the findings from this study should be interpreted with some caution. In retrospect, a study 

focused on CII’s development and scaling up of its Integrated Service Model may have been 

more appropriate than an implementation study.

Lessons Learned and Next Steps

Despite the many challenges, the study did produce some lessons. As policymakers, practitioners, and researchers in the child welfare field work to improve services available through the 

child welfare system, CII and its experience developing and implementing its Integrated Service 

model as well as delivering evidence-based practices offer important lessons. How CII overcame systemic and other challenges and creatively wove together public and private funding 

streams to integrate its services could be informative not only to similar multiservice organizations but to all those in the child welfare field looking for the best ways to serve children 

through an often fragmented child welfare system.

Those interested in evidence-based practices may find the findings from the fidelity 

study of Trauma-Focused Cognitive Behavioral Therapy useful. These findings suggest that

while self-monitoring tools may be effective and low cost, alone they are not enough to ensure 

fidelity. One area for further research could be investigating how to cost-effectively combine 

self-reporting tools and observational methods to support fidelity.

Appendix A 

Data Analysis Limitations

83

This Appendix describes various challenges MDRC researchers encountered during the data

analysis for this report. MDRC researchers began laying the groundwork for analysis in 2012,

and over the years a number of issues arose which complicated the data analysis. 

The management information system that Children Institute, Inc. (CII), was using at the 

time of the study was designed not for research but for billing and reporting purposes. As a 

result, how CII categorized its services in the system did not align well with the field-relevant 

research questions and thus the service categories designed by the research team. The team did 

not fully understand the issue until after it had processed the first data files. The challenges 

MDRC encountered in the analysis fall into four overarching categories:

• Integrated Service Model

• Evidence-based practices

• Outcome data

• Staffing data

Challenges Related to the Analysis of the Integrated

Service Model

CII’s Integrated Service Model is designed to address the holistic needs of the client and client’s 

family. As such, CII provides a variety of services to meet those needs, customized to each

client and family. To analyze the implementation of the Integrated Service Model, the research 

team and CII staff organized CII’s services into three categories: clinical services, family 

support, and youth development. (See Chapters 2 and 4.) However, the management information system CII used to document its services and fulfill its contractual requirements was

created primarily for reporting and billing purposes, and therefore the data were not already 

organized into those three categories. In order to fit the data fit into these categories, CII staff

had to create a crosswalk between CII’s billing codes and the three service categories.

As would be expected in any analysis of data gathered for one purpose and used for another, there were complications.

• Multiple data systems. During the study period, CII maintained multiple data systems. One system (TIER) captured the clinical service records and 

some family support and youth development activities, while a separate system (MINERVA) captured information about participation in some youth 
84

development activities.

1 There was not a universal client identifier that cut 

across the multiple data systems and clients could have multiple identifiers 

within the same system. Joining the clinical services data to the community 

services data was thus challenging and resulted in a couple of issues.

o CII staff had to create a new identifier to join systems using a combination of data points including birth date. There are likely instances where

one individual still has multiple identifiers in the data set used in the 

analysis because of cases in which staff could not match records in one 

system to records in the other system (for example, if a client’s birth date 

was missing or entered incorrectly in one system). This issue could have 

led to the over-reporting of the total number of clients or the underreporting of the number of clients receiving multiple services.

o The data systems did not connect the records of children to those of their 

parents or guardians. For example, a child may have received clinical 

services (documented in TIER) and his guardian may have received financial counseling (documented in MINERVA), but the research team 

could not tell from the data if the records were related or separate. In 

some cases, family support services were associated with the child’s record and in others they were associated with the parent’s or guardian’s 

record. This issue may have resulted in a misrepresentation of the clients’ 

age distribution, particularly in Table 4.5.

• Differences in the types of data collected in each system. TIER and 

MINERVA gathered different information, making it difficult to analyze the 

merged data. For example, MINERVA included data primarily on activities 

for which only attendance was captured; in other words, the number of sessions attended was captured, but not the length of each session. TIER, in contrast, included more detailed information about service time. The differences

between the data in the two systems ultimately limited the analysis since it 

was not possible to analyze the client flow through all services without service dates for the activities recorded in MINERVA.

• Service categories. Classifying CII services into clinical services, family 

support, and youth development categories was complex. While the research 

team described TIER as the “clinical services” system, it included records 

 1

In spring 2014, CII adopted a new management information system that replaced the systems from which 

the research team pulled the 2012 and 2013 data.
85

from some youth development and family support services. Since procedure 

codes determined whether a service was classified as family support or clinical service, it was impossible to separate family support services that were 

packaged with a clinical service from those that were not. Thus while the 

service categories made understanding the analysis easier, it also limited the 

analysis in other ways.

• Calculation of service time. TIER captured the allowable billing time for 

each service, not the actual amount of time a clinician or other staff person 

spent with a client. Allowable billing time could have included travel time to 

a client’s home or school and the time required to properly document a session or contact with the client. While this approach was necessary for billing 

purposes, it made calculating the face-to-face service time difficult.

Calculating clients’ actual service time for group therapy from the available 

data was doubly complicated. TIER populated this field for each participant 

by dividing the group session time (including travel and documentation) by 

the number of participants in the group. As a result, the reported service 

times and calculated averages for individual clients were lower than the actual length of the group session.

MINVERVA, on the other hand, did not include service time but only attendance records. The research team assigned session time to each activity 

based on information from CII staff about the length of each activity. For 

example, according to CII staff, acting and film production classes were 

assigned one hour and soccer programs were assigned two hours. As a result, the service time data in TIER and MINVERVA were not directly 

comparable.

In addition to these challenges, the research team also decided to exclude some data 

from the analysis.

• Excluding one-day events. Youth development activities are wide ranging, from back-to-school nights, field trips, and holiday celebrations to tax 

preparation days, organized sports teams, and art classes. Youth development activities could be reoccurring activities or one-time events. The 

MDRC research team chose to exclude one-day events from the analysis. 

While one-day events may be central to CII’s involvement in the community, they were less central to the theory of the Integrated Service Model.
86

• Limiting analysis to clients who enrolled during a particular period.

The Integrated Service Model calls for tailoring services to the needs of the 

individual and family and therefore CII sometimes offered services concurrently, and other times consecutively; for example, it may take a client a 

year or more to transition from clinical to youth development services. To 

account for cases such as these, MDRC researchers combined 2012 and 

2013 data and analyzed service receipt across the two years. The research 

team limited the analysis to those clients who enrolled in CII services from 

January 1, 2012, through June 30, 2013; the team analyzed their records for 

the full two years.

Challenges Related to Analysis of Evidence-Based Practices

Data about the use of evidence-based practices is a subset of the overall set of data received 

from CII. Analysis of this data had its own difficulties. Only certain staff members were trained 

to administer evidence-based practices. For billing purposes, all records in TIER associated with 

a client receiving Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) or Functional 

Family Therapy (FFT) were coded for that therapy, including records for related services that 

are not considered therapy. For example, a FFT client could have received medication management, which would have been coded as FFT though in fact it was not part of the therapy.

To account for these situations, the MDRC research team limited its analysis only to

those records with procedure codes that should have been attributed to the therapy. Further, to 

minimize the influence of data entry errors made by staff, the analysis included only records 

associated with staff trained in the specific evidence-based practice. This approach eliminated 

data entry errors that would have resulted in over counting.

Challenges Related to Analysis of Outcome Data

One goal of the study was to assess the outcomes of certain treatments. Data limitations, 

however, prevented the research team from conducting such analyses. Results of pre- and postassessments were stored separate from the clinical services data. MDRC researchers received 

outcome data for clients engaged in five evidence-based practices in 2012 and 2013: TF-CBT, 

FFT, Cognitive Behavioral Intervention for Trauma in Schools, Incredible Years, and Managing and Adapting Practice. Only a fraction of clients served in those years had any outcome 

records and many with pre-test scores were missing post-test scores. As a result, the research 

team did not have the required number of pre- and post-test score pairs needed for reliable 

analysis. For example, among clients starting TF-CBT in 2012 and 2013, fewer than one-third 
87

had parent or guardian pre- and post-test pairs and only about one-fifth had pre- and post-test

scores for the Youth Outcome Questionnaire.

Challenges Related to Analysis of Staffing Data

Data about staff training were also saved separate from the other data. Since CII stored training 

information in different ways depending on the program or practice, some staff had incomplete 

data. The MDRC research team made the decision to only use staff training records that 

included a date when staff completed the training, since that indicated the training had been 

documented in the system. This decision is most applicable to Table 4.2. The analysis of staff 

training data was therefore limited to therapists and psychologists — those staff eligible to be 

trained in an evidence-based practice.

Clinician Survey

In spring 2013, the MDRC research team fielded a web-based survey to all current CII therapists and psychologists (69) and their supervisors (27). Of the 96 staff asked to complete the 

survey, 39 percent submitted responses.

The survey, which included a combination of open- and closed-ended questions, addressed the following topics:

• Staff background. This series of questions asked about tenure at CII, as well 

as education and work background. It also included questions about their supervisory roles and movement across different positions within CII.

• General Work. These questions asked about staff workload and opinions 

about documentation, assessments, outcomes, and referrals to youth development services.

• Evidence-Based Treatments. This series of questions asked specifically 

about delivering evidence-based treatments or practices and the benefits and 

challenges of delivery.

• Group Supervision for Evidence-Based Treatments. These questions 

were specifically about the operation of the group supervision component associated with evidence-based practices and were intended to gauge the utility 

of these meetings.

• Individual Supervision and Support. These questions asked about individual supervision and other supports available to staff.
88

• CII Structure. These questions asked about overarching CII philosophies 

and the Integrated Service Model.

The survey was designed to be anonymous, and thus the amount of information collected about 

staff was limited, making it difficult to determine if the respondents were representative of all 

CII therapists or psychologists and their supervisors. Table A.1 presents some descriptive 

information about the respondents.

Table A.1

Characteristics of Clinician Survey Respondents

Therapists/

Characteristic Psychologists Supervisors

Average tenure at CII (years) 4 16

Held other positions at CII (%) 29 75

Ever interned at CII (%) 24 13

Licensed or registered Master's level social worker,

marriage and family therapist, or psychologist 33 75

Sample size 21 16

SOURCE: MDRC calculations from the CII Clinician Survey.
89

References

Ackerman, Peggy T., Joseph E. O. Newton, W. Brian McPherson, Jerry G. Jones, and Roscoe 

A. Dykman. 1998. “Prevalence of Post Traumatic Stress Disorder and Other Psychiatric 

Diagnoses in Three Groups of Abused Children (Sexual, Physical, and Both).” Child Abuse 

and Neglect 22, 8: 759-774.

Allen, Brian, and Jennifer C. Johnson. 2012. “Utilization and Implementation of TraumaFocused Cognitive-Behavioral Therapy for the Treatment of Maltreated Children.” Child 

Maltreatment 17, 1: 80-85.

Arnquist, Sarah and Peter Harbage. 2013. A Complex Case: Public Mental Health Delivery and 

Financing in California. Oakland, CA: California HealthCare Foundation.

Barker, Robert L. 2003. The Social Work Dictionary, 5th ed. Washington, DC: NASW Press.

Blueprints for Healthy Youth Development. 2015. Website: www.blueprintsprograms.com.

Burns, Barbara J., E. Jane Costello, Adrian Angold, Dan Tweed, Dalene Stangl, Elizabeth M. Z. 

Farmer, and Al Erkanli. 1995. “Children’s Mental Health Service Use across Service 

Sectors.” Health Affairs 14, 3: 147-159.

California Department of Health Care Services. 2016. “Katie A. Settlement Agreement 

Implementation.” Website: www.dhcs.ca.gov/Pages/KatieAImplementation.aspx.

California Department of Health Care Services. 2016. “What is Medi-Cal?” Website: 

www.dhcs.ca.gov/Services/medi-cal/pages/whatismedi-cal.aspx.

California Department of Social Services. 2013. Child Abuse Prevention, Intervention and 

Treatment Program (CAPIT). Sacramento, CA: Office of Child Abuse Prevention.

California Department of Social Services. 2014. CalWorks Overview. Sacramento, CA:

Assembly Budget Subcommittee #1.

Center for Reflective Communities. 2016. “Reflective Parenting Program.” Website: 

reflectivecommunities.org/programs/reflective-parenting-groups-rpp.

Center on the Developing Child at Harvard University. 2010. The Foundations of Lifelong 

Health Are Built in Early Childhood. Cambridge, MA: National Scientific Council on the

Developing Child. National Forum on Early Childhood Policy and Programs.

Chambless, Dianne L., and Steven D. Hollon. 1998. “Defining Empirically Supported 

Therapies.” Journal of Consulting and Clinical Psychology 66, 1: 7-18.

Child Sexual Abuse Task Force and Research and Practice Core, National Child Traumatic 

Stress Network. 2004. How to Implement Trauma-Focused Cognitive Behavioral Therapy.

Los Angeles and Durham, NC: National Center for Child Traumatic Stress.
90

Child Welfare Committee, National Child Traumatic Stress Network. 2008. Child Welfare 

Trauma Training Tool Kit: Comprehensive Guide (2nd ed.). Los Angeles and Durham, 

NC: National Center for Child Traumatic Stress.

Children’s Institute, Inc. “Programs and Services.” 2015. Website: 

www.childrensinstitute.org/ourwork/programsandservices.

Cohen, Judith A., Esther Deblinger, and Anthony Mannarino. 2004. “Trauma-Focused 

Cognitive-Behavioral Therapy for Sexually Abused Children.” Psychiatric Times 21, 10: 

109-121.

Cohen, Judith A., Esther Deblinger, Anthony P. Mannarino, and Robert A. Steer. 2004. “A 

Multisite, Randomized Controlled Trial for Children With Sexual Abuse-Related PTSD 

Symptoms.” Journal of the American Academy of Child and Adolescent Psychiatry 43, 4: 

393-402.

Cohen, Judith A., and Anthony P. Mannarino. 1996a. “Factors That Mediate Treatment 

Outcome of Sexually Abused Preschool Children.” Journal of the American Academy of 

Child and Adolescent Psychiatry 35, 10: 1402-1410.

Cohen, Judith A., and Anthony P. Mannarino. 1996b. “A Treatment Outcome Study for 

Sexually Abused Preschool Children: Initial Findings.” Journal of the American Academy 

of Child and Adolescent Psychiatry 35, 1: 42-50.

Cohen, Judith A., and Anthony P. Mannarino. 1997. “A Treatment Study for Sexually Abused 

Preschool Children: Outcome During a One-Year Follow-Up.” Journal of the American 

Academy of Child and Adolescent Psychiatry 36, 9: 1228-1235.

Cohen, Judith A., and Anthony P. Mannarino. 1998a. “Factors That Mediate Treatment 

Outcome of Sexually Abused Preschool Children: Six-and 12-Month Follow-Up.” Journal 

of the American Academy of Child and Adolescent Psychiatry 37, 1: 44-51.

Cohen, Judith A., and Anthony P. Mannarino. 1998b. “Interventions for Sexually Abused 

Children: Initial Treatment Outcome Findings.” Child Maltreatment 3, 1: 17-26.

Cohen, Judith A., Anthony P. Mannarino, Matthew Kliethermes, and Laura A. Murray. 2012. 

“Trauma-Focused CBT for Youth with Complex Trauma.” Child Abuse and Neglect 36, 6: 

528-541.

Cohen, Judith A., Anthony P. Mannarino, Laura K. Murray, and Robyn Igelman. 2006. 

“Psychosocial Interventions for Maltreated and Violence‐Exposed Children.” Journal of 

Social Issues 62, 4: 737-766.

Collins, Kathryn, Kay Connors, Sara Davis, April Donohue, Sarah Gardner, Erica Goldblatt, 

Anna Hayward, Laurel Kiser, Fred Strieder, and Elizabeth Thompson. 2010. 

Understanding the Impact of Trauma and Urban Poverty on Family Systems: Risks, 

Resilience, and Interventions. Baltimore, MD: Family Informed Trauma Treatment Center.
91

Deblinger, Esther, and Anne Hope Heflin. 1996. Treating Sexually Abused Children and Their 

Nonoffending Parents: A Cognitive Behavioral Approach. Thousand Oaks, CA: Sage 

Publications, Inc.

Deblinger, Esther, Julie Lippmann, and Robert Steer. 1996. “Sexually Abused Children 

Suffering Posttraumatic Stress Symptoms: Initial Treatment Outcome Findings.” Child 

Maltreatment 1, 4: 310-321.

Deblinger, Esther, Lori B. Stauffer, and Robert A. Steer. 2001. “Comparative Efficacies of 

Supportive and Cognitive Behavioral Group Therapies for Young Children Who Have 

Been Sexually Abused and Their Nonoffending Mothers.” Child Maltreatment 6, 4: 332-

343.

Deblinger, Esther, Robert A. Steer, and Julie Lippmann. 1999. “Two-Year Follow-Up Study of 

Cognitive Behavioral Therapy for Sexually Abused Children Suffering Post-traumatic 

Stress Symptoms.” Child Abuse and Neglect 23, 12: 1371-1378.

Dubner, Allison E., and Robert W. Motta. 1999. “Sexually and Physically Abused Foster Care 

Children and Posttraumatic Stress Disorder.” Journal of Consulting and Clinical 

Psychology 67, 3: 367-373.

Éthier, Louise S., Jean-Pascal Lemelin, and Carl Lacharité. 2004. “A Longitudinal Study of the 

Effects of Chronic Maltreatment on Children’s Behavioral and Emotional Problems.”

Child Abuse and Neglect 28, 12: 1265-1278.

Felitti, Vincent J., Robert F. Anda, Dale Nordenberg, David F. Williamson, Alison M. Spitz, 

Valerie Edwards, Mary P. Koss, and James S. Marks. 1998. “Relationship of Childhood 

Abuse and Household Dysfunction to Many of the Leading Causes of Death in Adults: The 

Adverse Childhood Experiences (ACE) Study.” American Journal of Preventive Medicine

14, 4: 245-258.

Functional Family Therapy, LLC. 2015. “About FFT Training: Clinical Model.” Website: 

fftllc.com/about-fft-training/clinical-model.html.

Henggeler, Scott W., and Sonja K. Schoenwald. 2011. “Evidence-Based Interventions for 

Juvenile Offenders and Juvenile Justice Policies that Support Them.” Society for Research 

in Child Development, Social Policy Report 25, 1: 1-20.

Hildyard, Kathryn L., and David A. Wolfe. 2002. “Child Neglect: Developmental Issues and 

Outcomes.” Child Abuse and Neglect 26, 6: 679-695.

Kataoka, Sheryl H., Lily Zhang, and Kenneth B. Wells. 2002. “Unmet Need for Mental Health 

Care among US Children: Variation by Ethnicity and Insurance Status.” American Journal 

of Psychiatry 159, 9: 1548-1555.

Keller, Thomas E., Amy M. Salazar, and Mark E. Courtney. 2010. “Prevalence and Timing of 

Diagnosable Mental Health, Alcohol, and Substance Use Problems among Older 

Adolescents in the Child Welfare System.” Children and Youth Services Review 32, 4: 

626-634.
92

Kilpatrick, Dean G., and Benjamin E. Saunders. 1999. Prevalence and Consequences of Child 

Victimization: Results from the National Survey of Adolescents, Final Report (Revised).

Charleston, SC: National Crime Victims Research and Treatment Center, Department of 

Psychiatry and Behavioral Sciences, Medical University of South Carolina.

Kolko, David J., Michael S. Hurlburt, Jinjin Zhang, Richard P. Barth, Laurel K. Leslie, and 

Barbara J. Burns. 2010. “Posttraumatic Stress Symptoms in Children and Adolescents 

Referred for Child Welfare Investigation: A National Sample of In-Home and Out-ofHome Care.” Child Maltreatment 15, 1: 48-63.

Landsverk, John A., Barbara J. Burns, Leyla F. Stambaugh, and Jennifer A. Rolls Reutz. 2006. 

“Mental Health Care for Children and Adolescents in Foster Care: Review of Research 

Literature.” Seattle, WA: Casey Family Programs.

Los Angeles County Department of Mental Health. 2013. “Child/Adolescent Initial 

Assessment.” Website: www.file.lacounty.gov/dmh/cms1_046530.pdf.

Los Angeles County Department of Mental Health. 2015. Mental Health Services Act: Three 

Year Program and Expenditure Plan. Fiscal Years 2014-15 through 2016-17. Los 

Angeles: Los Angeles County Board of Supervisors. Website: 

file.lacounty.gov/dmh/cms1_211892.pdf.

Los Angeles County Department of Public Health. 2013. Key Indicators of Health by Service 

Planning Area. Website: publichealth.lacounty.gov/ha/docs/kir_2013_finals.pdf.

Marsenich, Lynne. 2002. Evidence-Based Practices in Mental Health Services for Foster Youth. 

Sacramento, CA: California Institute for Mental Health.

National Child Traumatic Stress Network. 2007. Trauma-Focused Cognitive Behavioral 

Therapy (TF-CBT). Website: www.nctsn.org/nctsn_assets/ pdfs/promising_practices/TFCBT_fact_sheet_3-20-07.pdf.

Osofsky, Joy D. (ed.). 1998. Children in a Violent Society. New York: Guilford Press.

Parsons, Bruce V., and James F. Alexander. 1973. “Short-Term Family Intervention: A Therapy 

Outcome Study.” Journal of Consulting and Clinical Psychology 41, 2: 195-201.

Richardson, Margaret, Jim Henry, Connie Black-Pond, and Mark Sloane. 2008. “Multiple 

Types of Maltreatment: Behavioral and Developmental Impact on Children in the Child 

Welfare System.” Journal of Child and Adolescent Trauma 1, 4: 317-330.

Sexton, T. L., and J. F. Alexander. 2000. “Functional Family Therapy” (Bulletin). Washington, 

DC: Department of Justice, Office of Justice Programs, Office of Juvenile Justice and 

Delinquency Prevention.

Silverman, Wendy K., Claudio D. Ortiz, Chockalingham Viswesvaran, Barbara J. Burns, David 

J. Kolko, Frank W. Putnam, and Lisa Amaya-Jackson. 2008. “Evidence-Based 

Psychosocial Treatments for Children and Adolescents Exposed to Traumatic Events.” 

Journal of Clinical Child & Adolescent Psychology 37, 1: 156-183.
93

Technical Assistance Collaborative and Human Services Research Institute. 2012. California 

Mental Health and Substance Use System Needs Assessment Appendices. Submitted to the 

California Department of Health Care Services. Website: 

www.dhcs.ca.gov/provgovpart/Documents/Data%20Appendices%203%201%2012.pdf.

Trauma-Focused Cognitive Behavioral Therapy National Therapist Certification Program. 

2015. “Trauma-Focused Cognitive Behavioral Therapy (TF-CBT).” Website: tfcbt.org.

U.S. Department of Health and Human Services, Administration for Children and Families, 

Administration on Children, Youth and Families, Children’s Bureau. 2013. Child 

Maltreatment 2012. Washington, DC: Children’s Bureau.

Waldron, Holly B., and Charles W. Turner. 2008. “Evidence-Based Psychosocial Treatments 

for Adolescent Substance Abuse.” Journal of Clinical Child and Adolescent Psychology

37, 1: 238-261.

Weiss, Michael J., Howard S. Bloom, and Thomas Brock. 2013. A Conceptual Framework for 

Studying the Sources of Variation in Program Effects. New York: MDRC.

Weisz, John R., and Jane S. Gray. 2008. “Evidence‐Based Psychotherapy for Children and 

Adolescents: Data from the Present and a Model for the Future.” Child and Adolescent 

Mental Health 13, 2: 54-65.
About MDRC

MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 

to learning what works to improve the well-being of low-income people. Through its research 

and the active communication of its findings, MDRC seeks to enhance the effectiveness of social and education policies and programs.

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 

for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 

Its projects are a mix of demonstrations (field tests of promising new program approaches) and 

evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 

combination of research and organizational experience to their work, providing expertise on the 

latest in qualitative and quantitative methods and on program design, development, implementation, and management. MDRC seeks to learn not just whether a program is effective but also 

how and why the program’s effects occur. In addition, it tries to place each project’s findings in 

the broader context of related research — in order to build knowledge about what works across 

the social and education policy fields. MDRC’s findings, lessons, and best practices are proactively shared with a broad audience in the policy and practitioner community as well as with the 

general public and the media.

Over the years, MDRC has brought its unique approach to an ever-growing range of policy areas and target populations. Once known primarily for evaluations of state welfare-to-work programs, today MDRC is also studying public school reforms, employment programs for exoffenders and people with disabilities, and programs to help low-income students succeed in 

college. MDRC’s projects are organized into five areas:

•

•

•

•

•

Promoting Family Well-Being and Children’s Development

Improving Public Education

Raising Academic Achievement and Persistence in College

Supporting Low-Wage Workers and Communities

Overcoming Barriers to Employment

Working in almost every state, all of the nation’s largest cities, and Canada and the United 

Kingdom, MDRC conducts its projects in partnership with national, state, and local governments, public school systems, community organizations, and numerous private philanthropies.