Running Head: ADMINISTRATIVE SUPPORTS AND WRAPAROUND FIDELITY

Influence of Administrative and System Supports on Treatment Fidelity:

An Example from the Wraparound Process for Children and Families

Eric J. Bruns

University of Washington School of Medicine

Jesse C. Suter

Kristen M. Leverentz-Brady

University of Vermont

Kimberly E. Hoagwood

Columbia University and New York State Office of Mental Health

Corresponding author:

Eric J. Bruns, Ph.D.

University of Washington School of Medicine

Division of Public Behavioral Health and Justice Policy

146 N. Canal Street, Suite 100

Seattle, WA 98107

206-685-2477 (ph)

206-685-3430 (fax)

<ebruns@u.washington.edu>

Abstract

# Objective

Recognition of the "science to service" gap in health and mental health
service delivery has pointed to the need to better understand how
characteristics of provider organizations and service systems influence
service delivery. The current study aimed to further illuminate such
associations by examining the relationship between a set of
theoretically derived administrative and system variables and adherence
to standards of high-quality wraparound implementation for children and
families.

*Methods*

A validated measure of fidelity to the wraparound care planning and
management process was administered in 8 communities implementing
wraparound nationally (total N=295). Concurrently, program
administrators in each community completed a standardized interview
about the presence or absence of a set of administrative and system
supports hypothesized to influence the quality of implementation.
Regression analysis was used to assess the relationship between supports
and fidelity.

*Results*

Communities demonstrated great variation in degree and type of
administrative and system supports for implementation. Significant
variation was also found in fidelity scores across communities.
Regression analyses found a significant association between the number
of supports at the program and system level and wraparound fidelity
scores provided by caregiver and youth respondents.

*Conclusions*

The identified program and system issues should be attended to when
establishing and administering intensive, individualized care management
procedures for children with complex needs and their families. The
results also provide further support for this hypothesized relationship
in service delivery overall.

Influence of Administrative and System Supports on Treatment Fidelity:

An Example from the Wraparound Process for Children and Families

Emphasizing use of effective mental health treatments has become a
national priority (1, 2). However, there is a growing recognition that
translating research-based interventions to real world settings is a
formidable task, and "science to service gap" has become nearly as
prominent a catchphrase as "evidence-based practice" (3, 4, 5).

Understanding the dynamics of organizations and services systems has
been highlighted as one method for bridging the science to service gap.
As a result, researchers are beginning to consider such multilevel
influences, including the role of clinician, program, and system
variables in disseminating specific treatment approaches, as well as the
influence of organizational and system processes generally on
implementation and outcomes (6, 7, 8).

The importance of understanding organizational and system structures
within the children's services arena has been underscored through
several studies. A study by Glisson and Hemmelgarn (9) demonstrated
significant associations between organizational climate and culture
variables with outcomes achieved for children and adolescents. In
addition, research on the transportability of multisystemic therapy
(MST) has demonstrated a complex set of relationships between treatment,
provider, and administrative and system variables (10, 11).

Walker and colleagues (12) conducted a series of studies proposing a
matrix of necessary program and system conditions to properly implement
the wraparound care management process. However, there has not yet been
an attempt to empirically test the relationship between such supports
and actual wraparound implementation. The current study tested this
proposed relationship, in order to advance the research base for
wraparound as well as services implementation overall.

*The Wraparound Process*

Wraparound is not a specific treatment but rather an increasingly
well-operationalized series of engagement, planning, and implementation
activities. Through this defined set of activities, providers or
facilitators work in collaboration with families to create an
individualized plan of services and supports that are tailored to the
specific needs of the child and family, and then implement and adjust
that plan over time (13). More thorough descriptions of the wraparound
process have been described elsewhere including descriptions of the
phases and activities of the wraparound process (14, 15) and details on
implementation (13, 16).

The wraparound process is included as a "promising" intervention in the
Surgeon General's reports on both mental health and youth violence (1,
17). In addition, the label "wraparound" has proliferated, with over
200,000 youth with complex needs reported to be served via the
wraparound process in the U.S. annually (18). Recently, use of the
wraparound process was mandated for communities applying for funds
through the Substance Abuse and Mental Health Services Administration's
(SAMHSA) children's services program for those families who need access
to an individualized care planning and management process (19).

Despite a growing number of encouraging studies of effectiveness (see
15, for a review) and a theory and practice base that appears
well-grounded empirically (13, 20), the wraparound process model has not
been evaluated using the same types of rigorous methodology that has
been used with other integrated community-based treatments for youth.
Drawing conclusions from the small number of experimental (e.g., 21, 22)
and quasi-experimental studies (23, 24) is difficult due to the lack of
fidelity control and measurement in these studies.

To address these concerns, fidelity tools for wraparound, based on the
principles of the process, have been developed, revised, and validated
and are now in widespread use (25, 26). Several studies using these
measures have demonstrated a relationship between wraparound fidelity
and child and family outcomes (e.g., 27, 28). In parallel process, an
initiative to better define wraparound, based on input from developers
of well-operationalized models and administrators of effective programs
nationally, has clarified the specific procedures to be undertaken
within the process (13). Results of this model definition initiative
have informed manual development, training and coaching protocols,
development of state and local regulations, and development of improved
fidelity measures that are now being used in several controlled trials
of the wraparound process.

*The Current Study*

In sum, dissemination process research represents a priority for
services researchers, while widespread use of the wraparound process in
systems of care for children and families nationally makes investigation
of factors that may promote or impede its implementation a high
priority. The objectives of this exploratory study were to (a) examine
variability among sites on a set of administrative and system variables
proposed to be important to wraparound process implementation, and (b)
to assess the degree of association with adherence to the wraparound
principles.

Method

##### *Measures*

*Wraparound adherence.* Wraparound model adherence was assessed using
caregiver, youth (11 years and older), and resource facilitator (or care
managers) responses on the Wraparound Fidelity Index, version 2. The WFI
is structured interview that assesses adherence to 11 core principles of
wraparound (e.g., Family Voice and Choice, Individualized, Natural
Supports, Team-Driven). Four items serve as indicators for each element,
with responses ranging from 0 (*low fidelity*) to 2 (*high fidelity*).
The youth form of the WFI includes only 8 of the 11 elements (youth
often do not have enough information to be accurate reporters on several
elements). Thus, the resource facilitator and caregiver forms each
contain 44 items while the youth form includes 32 items. For this study,
total scores for each of the respondents were calculated ranging from 0
(*low fidelity*) to 8 (*high fidelity*).

Validation studies of the WFI have found adequate test-retest
reliability (two-week test-retest found to range from .68 to .78 for the
three forms) and internal consistency (*alphas* ranging from .78 to .90)
for overall respondent scores. Because of the small number of items,
alphas for several individual elements on each respondent form have been
found to be unacceptable, below .60. As a result, total respondent
scores combining all WFI items are primarily used in research studies,
with use of element scores and individual items reserved for use in
training and quality assurance activities (25).

Pilot tests of the WFI have found significant association with an
external expert's ratings of implementation (25), and a confirmatory
factor analysis has found good support for the underlying structure of
the measure (29), providing evidence of construct validity (see 25, for
more information on psychometrics and reliability and validity studies).
The WFI has also been found to distinguish wraparound programs from
non-wraparound comparison conditions in evaluation studies (28, 30).
Criterion-related validity is further established through several
studies finding associations between WFI scores and concurrently
assessed as well as future outcomes (27,31).

*Program and system supports.* The WFI-Program Administrator form
(WFI-PA) was designed to assess the presence of administrative and
system supports needed to implement a high-quality wraparound process.
The WFI-PA is administered in structured format to one or more key
informants from a program or system, such as a program's lead
administrator or clinical supervisor. Items on the WFI-PA were created
based on theory and research on organizational processes and program
implementation (e.g., 6, 7) and system and organizational conditions
conducive to implementing wraparound (e.g., 12, 15).

WFI-PA items (see Table 1) were designed to reflect ten domains or
indicators of favorable conditions for high-quality service delivery:
(1) program duration, (2) staff turnover, (3) resource facilitator
caseload, (4) interagency coordination and funding, (5) family
centeredness, (6) community-based services, (7) natural supports, (8)
strengths-based, (9) individualized services, and (10) outcomes
orientation in service delivery. For some of these domains, a single
item is used to achieve a score (e.g., "program duration" is simply the
reported number of years the program had delivered services via the
Wraparound process). Scores for other domains are calculated from
figures given for two items (e.g., "resource facilitator turnover" is
calculated by dividing the number of resource facilitators who had been
hired in the past 12 months by the total number employed at the site).
Finally, for several domains, scores are calculated by combining
responses to two or more items that are assessed via a Likert-format (0
= *No*, 1 = *Somewhat*, and 2 = *Yes*). For example, the score for
outcomes orientation is calculated by combining responses to several
items: (1) whether the site collects and uses data for use in quality
assurance; (2) whether caregiver and youth satisfaction data is
collected regularly; and (3) whether teams review outcome information at
each team meeting.

After examining frequency distributions for individual WFI-PA items, the
authors achieved consensus on ratings for each site for each of the 10
WFI-PA domains described above. Ratings assigned were either 1
(indicating a relatively *high* level of program/system support) or 0
(indicating a relatively *low* level of program/system support). For the
purposes of the current study, scores for the 10 domains were also
summed to create a total WFI-PA score for each site that could range
from 0 to 10. (The WFI-PA and description of criteria for ratings across
domains are available from the lead author.)

*Procedure*

The WFI-2 was released to potential collaborator sites in 2001. Sites
that wished to use the measure completed a formal request that provided
information on the programs, population served, evaluation and quality
assurance methodologies being employed, and procedures to be used to
administer the WFI. Approved sites then engaged families to participate;
trained interviewers based on the research team's guidelines and WFI
user's manual (32); administered interviews to caregivers, youth, and
resource facilitators (face-to-face or by phone); entered data into a
database provided by the research team; and forwarded data, with
identifying information deleted, to the research team. In turn, the
research team provided collaborators with summary fidelity reports. For
the current study, researchers approached eight collaborating sites
nationally that had collected WFI data for over 20 families. Each site
agreed to participate. The WFI-PA was administered by phone to the
individual considered most knowledgeable about the site's organizational
and system context, typically the program director.

*Participants*

*Participating sites.* Eight programs administering a wraparound process
for families with children experiencing serious emotional disturbance
(SED) participated in the current study. The agencies included two from
California, and one each from Arizona, Indiana, Missouri, Nebraska,
North Carolina, and Vermont. Participating sites had been in operation
from between 2.5 to 7 years (*M* = 3.9, *SD* = 1.45) and had served
between 41 and 497 families during their existence (*M* = 223, *SD* =
164). Two sites reported serving children and youths ranging from 0-18
years old, while six reported serving youths 6-18. Two sites were
primarily aimed at youths in state custody, while the other sites served
families with youths referred from several agencies. Three programs
explicitly served youth "eligible for/at risk of placement in high-level
residential care," while one program was specifically tailored to serve
youths stepping down from group homes or residential treatment
facilities.

*Participating youth and families.* The sample consisted of 295 families
(15-55 families per site) with a child or youth experiencing SED
(defined as resulting in a diagnosable disorder that causes significant
impairment in home, school or community functioning and involvement by
one or more public child-serving agencies). Information about wraparound
fidelity was provided by 221 caregivers, 158 youths, and 193 resource
facilitators who completed the WFI. At the time of the WFI
administration, families had been receiving wraparound for a mean of
13.39 months (*SD* = 9.04). Sixty-three percent of the identified
children in the sample were male, with ages ranging from 1 to 19 years
(*M* = 13.07, *SD* = 3.89). Fifty-four percent of the youth were
identified as Caucasian, 18% African-American, 4% Hispanic, 1% Native
American, and 1% Asian/Pacific Islander, and 3% other race (race was not
identified for the remaining 19% of the sample). Forty-four percent of
youths were in custody of at least one biological parent, 39% were in
state custody, 5% were in the care of a relative, and custody was not
reported for 11% of the sample.

Results

*Administrative and System Supports for Wraparound*

Table 1 shows the characteristics of study sites across the
organizational and system variables assessed by the WFI-PA. Consistent
with the wide range of number of youth served, the number of wraparound
care managers varied widely, from 8-35, as did the caseload of care
managers, from 6-15 across the eight sites (*M* = 9.38, *SD* = 2.72).
Care manager turnover ranged from .13 to .50 across sites, with a mean
turnover rate of .34 over the previous 12 months.

Only six of the eight sites reported having a community collaborative
team overseeing the project. Interestingly, though sites tended to serve
youth across all four major agencies, only 2 of 8 (25%) sites reported
blending of funds from multiple agencies.

Data from WFI-PA interviews provided additional information about how
well system characteristics reinforced wraparound principles in the
study sites. As shown in Table 1, among those sites with community
collaborative teams, the percent of team members described as consumers,
family members, and family advocates ranged from 0-55 percent, with an
average of 16.2% (*SD*=22.1). Four of the six sites with teams reported
having no non-agency community representation (e.g., clergy, business
owners, politicians), while the other two sites reported 5% and 10%
representation from this constituency.

Several other characteristics of the system context for study sites are
worth noting. All sites served a majority of youth in community
residential settings (range = 65%-100%; *M* = 81%, *SD* = 16.5%). The
majority, though not all, of sites reported specific mandates for
inclusion of natural supports on wraparound teams. In addition, the
majority of respondents reported program mandates around conducting team
planning activities using family strengths as a foundation, and using
outcomes and satisfaction data to aid individual team processes as well
as program improvement. All sites reported mandated recording of goals
and action steps during team planning process, and the majority reported
having flexible funding policies and procedures in place that encourage
individualization of treatment.

*Adherence to Wraparound*

Table 2 displays mean WFI scores for all sites in the study, as well as
site-level means. Across sites, mean scores were found to be 6.59 (*SD*
= 0.27) for the resource facilitator form, 6.22 (*SD* = 0.38) for the
caregiver form, and 6.19 (*SD* = 0.46) for the youth form. Analysis via
one-way ANOVA demonstrated significant differences across study sites'
total WFI scores for resource facilitators, *F*(6, 186) = 2.78
(*p*=.013, partial ^2^=.08), caregivers *F*(7, 213) = 2.52 (*p*=.016,
partial ^2^=.08), and youth *F*(6, 151) = 2.98 (*p*=.009, partial
^2^=.11).

Based on results of a benchmarking exercise using normative data from a
more recent version of the WFI (33), study sites' WFI scores are typical
of sites nationally using the WFI, and could be characterized as ranging
from "borderline" to "above average" fidelity. Though all sites in the
current study exceeded minimum thresholds for meeting criteria for
wraparound (70% for the WFI-2), none scored above the threshold for
"high fidelity" (90% for the WFI-2).

*Association between Administrative and System Variables and Wraparound
Adherence*

Regression analyses were used to determine the relationship between
number of administrative and system supports and adherence scores.
Results found a significant relationship between the number of WFI-PA
domains in place for a site and mean site-level caregiver fidelity
\[Standardized *Beta =* .71; *F*(1,6) = 6.025, *p* = .025\]. The
unstandardized *Beta* coefficient for this relationship was found to be
.125 (SE = .51), meaning that each additional support condition in place
in a site improved mean WFI-caregiver scores for a site by an average of
.125.

A marginally significant relationship was found for youth fidelity
ratings \[Standardized *Beta =* .58; *F*(1,6) = 2.478, *p* = .088\]. The
unstandardized *Beta* coefficient for this relationship was found to be
.117 (SE = .74). Finally, the association between number of WFI-PA
supports and resource facilitator WFI ratings was found to be positive
and moderate in strength (Standardized *Beta =* .46); however, this
relationship was not significant \[*F*(1,6) = 1.313, *p* = .152\].

Figure 1 shows mean WFI scores for each of the eight study sites, by the
number of administrative and system supports for which each site met
criteria. As shown, sites met criteria for between 2 and 8 of the 10
WFI-PA domains. Regression lines provided in Figure 1 illustrate that
there is an observable trend whereby sites with greater levels of
administrative and system supports demonstrate higher mean WFI scores.
As shown in the Figure and discussed above, this relationship is
strongest for caregiver reports of wraparound adherence, followed by
youth reports and resource facilitator reports.

Discussion

The objectives of this study were to assess the types of administrative
and system supports that are typically present to support the wraparound
process and to examine the association of such supports with adherence
to wraparound principles at the youth and family level. Our findings
suggest that provider agencies show wide variation in the types of
supports in place in the host organizations and broader systems of care
in which they are embedded. The results of our study provide preliminary
evidence of the association between such administrative and system
supports and how fully a program is able to conform to standards for
high-quality wraparound.

Results of our WFI-PA interviews demonstrated that even within a sample
of communities likely to be well-informed about the philosophy and
activities of wraparound (after all, they all requested to use the WFI
to monitor their own service processes), many did not have support
conditions in place posed as necessary to implement a high-quality
wraparound process (12, 34). While some communities featured supports
specific to implementing wraparound, such as community collaborative
teams, availability of flexible funds, and processes for ensuring
presence of natural supports or non-professionals on teams, others did
not. Programs also varied on more generic conditions likely to influence
practice. For example, in the communities studied, caseloads of resource
facilitators ranged from 6:1 to 15:1, and staff turnover ranged from 13%
to 50%.

Finding such variation in administrative and system conditions in a
sample of wraparound programs is not surprising. Such results have been
observed in both wraparound (12) and systems of care (35). Such
variation is also characteristic of the "science to service" gap whereby
implementation of promising models is hindered by conditions in the
"real world" that are less than ideal.

Results also revealed that sites significantly differed on adherence
scores across the three respondents. Further analyses showed that the
number of administrative and system supports were associated with
fidelity scores provided by caregivers and youths. Resource
facilitators' reports of fidelity were not associated with support
conditions, possibly because of lesser variance in their scores (which
has also been found in previous research using the WFI). It also may be
that these informants' reports are less reliable due to bias.

Determining which supports are most important to achieving higher
fidelity is an important research question that was difficult to discern
from the current study. Exploratory analyses suggest that having
wraparound-specific supports in place such as interagency collaboration,
flexible funding, and mandates for engaging natural supports may be
particularly strong predictors of wraparound adherence (36). Future
research should include a greater number of sites as well as assessment
of outcomes, allowing for more definitive conclusions to be drawn about
the causal chain from administrative and system conditions to
implementation to outcomes.

Other limitations of this study should be noted. First, researchers
lacked full control over data collection using the WFI in study sites.
It is possible that observed program and system characteristics may also
have been related to data collection methods used in individual sites,
and thus explain some of the shared variance. Second, it is important to
note that the WFI-PA was created to better understand the
characteristics of communities using the WFI and had not previously been
subjected to formal validation. Criteria for each of the 10 domains
assessed were set *a priori* of analyses; however, they were nonetheless
somewhat arbitrary and based on frequency distributions from the study
sample.

*Implications for Practice*

With these limitations noted, this study provides preliminary support of
the theory formally proposed by Walker, Koroloff, & Schutte (12) -- and
informally by trainers, providers, and advocates for years -- that
presence or absence of "necessary support conditions" will likely
influence the quality of service planning and implementation for
children and their families. This is an important finding given growing
evidence that greater fidelity will yield more positive youth and family
outcomes (16, 27). These results thus provide an additional link in the
causal chain that leads from program and system structures to service
quality to improved outcomes.

The conditions assessed span a number of different categories, such as
reinforcing components of the practice model, ensuring support to staff,
building collaboration and partnerships, and sustaining accountability
systems. These categories also span both the organizational and systems
levels. Given the support for the theory that the cumulative number of
facilitative conditions is associated with greater fidelity, results
suggest that communities that wish to implement a high-quality
wraparound process for individuals with complex needs and their families
should attend to a full range of administrative and system issues,
including characteristics of both the organization that hosts the
wraparound program as well as the system within which the program and
teams are embedded.

The results of the current study may also be generalized as important
for other treatment approaches. Indeed, the results of the current study
may be viewed as support for the importance of Rosenheck's (6) four
strategies for promoting effective practice. These include construction
of leadership coalitions, linking service delivery to concrete and
legitimated goals and values, monitoring fidelity and outcomes, and
developing self-sustaining communities of practice. Aspects of all these
principles of effective human service organizations were assessed to
some extent in this study.

A final implication of the current study is that measuring relevant
indicators of a program's organizational and system context may assist
high-quality practice. Such measures have supported prominent national
studies on service system initiatives in both the adult (37) and
children's (38) mental health service fields. Such measures are also a
component of systematic quality assurance approaches adopted by
developers of effective interventions (10). To support high-quality
wraparound implementation, applying measures such as the WFI-PA or other
assessments of organizational and system supports during planning and
evaluation may be useful.

Conclusion

In addition to continuing to conduct effectiveness studies on promising
treatment approaches, the mental health service delivery field needs to
better understand how characteristics of organizations and service
systems influence service delivery (6). This exploratory study found
support for the proposed relationship between administrative and system
conditions and wraparound implementation, contributing additional
evidence to this emerging line of research. The study also points to
areas in which administrators, researchers, and family members alike may
collaborate to assess the "fit" between community and program structures
and high-quality practice.

While some researchers have argued that these research studies should
ultimately guide modifications to treatments to fit the realities of
"real world" practice, such information may also be used to advocate for
changes in the policy and funding environments within which treatments
and interventions are embedded. Ultimately, advocating for and achieving
policies that support high-quality practice may better influence our
ability to achieve positive change for individuals in need than merely
tinkering with promising treatment approaches.

References

1.  US Public Health Service: Mental Health: A Report of the Surgeon
    General. Rockville, MD, US Department of Health and Human Services
    Administration, Center for Mental Health Services, National
    Institutes of Health, National Institute of Mental Health, 1999

2.  US Department of Health and Human Services: New Freedom Initiative
    on Mental Health, 2003. Downloaded January 22, 2004 from
    [www.mentalhealthcommission.gov/reports/FinalReport](http://www.mentalhealthcommission.gov/reports/FinalReport).

3.  National Institute of Mental Health: Bridging Science and Service.
    Washington, DC, Author, 1998

4.  Burns BJ, Hoagwood K, Mrazek PJ: Effective treatment for mental
    health disorders in children and adolescents. Clinical Child &
    Family Psychology Review 2:199-254, 1999

5.  Hoagwood K, Burns B, Kiser L, et al: Evidence-based practice in
    child and adolescent mental health services. Psychiatric Services
    52:1179-1189, 2001

6.  Rosenheck R: Organizational process: a missing link between research
    and practice. Psychiatric Services 52:1607-1611, 2001

7.  Schoenwald SK, Hoagwood K: Effectiveness, transportability and
    dissemination of interventions: what matters when? Psychiatric
    Services 52:1190-1197, 2001

8.  Schoenwald SK, Sheidow, AJ, Letourneau EJ, et al: Transportability
    of multisystemic therapy: evidence for multilevel influences. Mental
    Health Services Research 5: 223-239

9.  Glisson C, Hemmelgarn AL: The effects of organizational climate and
    interorganizational coordination on the quality and outcomes of
    children's service systems. Child Abuse and Neglect 22:401-421, 1998

10. Henggeler SW, Melton GB, Brondino MJ, et al: Multisystemic therapy
    with violent and chronic juvenile offenders and their families: the
    role of treatment fidelity in successful dissemination. Journal of
    Consulting and Clinical Psychology 65:821-833, 1997

11. Huey SJ, Henggeler SW, Brondino MJ, et al: Mechanisms of change in
    multisystemic therapy: reducing delinquent behavior through
    therapist adherence and improved family and peer functioning.
    Journal of Consulting and Clinical Psychology 68:451-467, 2000

12. Walker JS, Koroloff N, Schutte K: Implementing high-quality
    collaborative individualized service/support planning: necessary
    conditions. Portland, OR, Research and Training Center on Family
    Support and Children's Mental Health, 2003

13. Walker JS, Bruns EJ: The wraparound process: individualized care
    planning and management for children and families, in Community
    Mental Health Reader: Current Perspectives. Edited by Rosenberg S,
    Rosenberg J. Routledge, in press

14. Bruns EJ, Walker JS, VanDenBerg JD, et al: Phases and Activities of
    the Wraparound Process. Portland, OR, National Wraparound
    Initiative, Research and Training Center on Family Support and
    Children's Mental Health, 2004

15. Burchard JD, Bruns EJ, Burchard SN: The wraparound process, in
    Community Treatment for Youth: Evidence-based Treatment for Severe
    Emotional and Behavioral Disorders. Edited by Burns B, Hoagwood K.
    New York, Oxford University Press, 2002

16. Bruns EJ, Rast J, Walker JS, et al: Applying evaluation data and the
    wraparound process to achieve systems of change for children and
    families. American Journal of Community Psychology, in press

17. US Public Health Service: Youth Violence: A Report of the Surgeon
    General. Rockville, MD, US Department of Health and Human Services
    Administration, Center for Mental Health Services, National
    Institutes of Health, National Institute of Mental Health, 2001

18. Faw L: The state wraparound survey, in Systems of Care: Promising
    Practices in Children's Mental Health, 1998 series, Vol IV:
    Promising Practices in wraparound for children with severe emotional
    disorders and their families. Edited by Burns BJ, Goldman SK.
    Washington, DC, Center for Effective Collaboration and Practice,
    American Institutes for Research, 27-32, 1999

19. Substance Abuse and Mental Health Services Administration: Request
    for the Cooperative Agreements for the Comprehensive Community
    Mental Health Services for Children and Their Families. Retrieved on
    May 4, 2005 from
    <http://www.samhsa.gov/grants/2005/nofa/sm05010rfa_cmhi.aspx>, 2005

20. Bruns EJ: The evidence base and wraparound. The Wraparound Solutions
    Newsletter 4, 2004

> 21\. Evans ME, Armstrong MI, Kupperinger AD: Family-centered intensive
> case management: a step toward understanding individualized care.
> Journal of Child and Family Studies, 5, 55-65, 1996
>
> 22\. Clarke HB, Prange M, Lee B, et al.: An individualized wraparound
> process for children in foster care with emotional/ behavioral
> disturbances: Follow-up findings and implications from a controlled
> study. In M.Epstein, K. Kutash, & A. Duchnowski (Eds.), Outcomes for
> children and youth with emotional and behavioral disorders and their
> families, 513-542. Austin, TX: PRO-ED, 1998
>
> 23\. Bickman L, Smith CM, Lambert EW, Andrade,AR: Evaluation of a
> congressionally mandated wraparound demonstration. Journal of Child &
> Family Studies, 12, 135-156, 2003
>
> 24\. Hyde KL, Burchard JD, Woodworth K: Wrapping services in an urban
> setting. Journal of Child and Family Studies, 5, 67-82, 1996

25. Bruns EJ, Burchard JD, Suter JC, et al: Assessing fidelity to a
    community-based treatment for youth: the Wraparound Fidelity Index.
    Journal of Emotional and Behavioral Disorders 12:69-79, 2004

26. Epstein MH, Jayanthi M, McKelvey J, et al: Reliability of the
    wraparound observation form: an instrument to measure the wraparound
    process. Journal of Child and Family Studies 7:161-170, 1998

27. Bruns EJ, Suter JC, Force, MD, Burchard JD: Adherence to wraparound
    principles and association with outcomes. Journal of Child and
    Family Studies, in press.

28. Rast J, Peterson CR, Earnest L, et al: Service process as a
    determinant of treatment effect -- the importance of fidelity, in
    The 16^th^ Annual Research Conference Proceedings, A System of Care
    for Children's Mental Health: Expanding the Research Base. Edited by
    Newman C, Liberton CJ, Kutash K, et al. Tampa, FL, University of
    South Florida, The Louis de la Parte Florida Mental Health
    Institute, Research and Training Center for Children's Mental
    Health, 2004

29. Suter JC: Recipes for wraparound: Results of a confirmatory factor
    analysis using the Wraparound Fidelity Index. Paper presented at the
    18^th^ Annual Research Conference: A System of Care for Children's
    Mental Health, Tampa, FL, 2005

30. Ferguson C: California's title IV-E child welfare waiver
    demonstration project evaluation: an analysis of wraparound in
    Alameda County, in The 17^th^ Annual Research Conference
    Proceedings, A System of Care for Children's Mental Health:
    Expanding the Research Base. Edited by Newman C, Liberton CJ, Kutash
    K, et al. Tampa, FL, University of South Florida, The Louis de la
    Parte Florida Mental Health Institute, Research and Training Center
    for Children's Mental Health, 2004

```{=html}
<!-- -->
```
31. Hagen M, Noble K, Schick C, et al: The relationship between fidelity
    to wraparound and positive behavior outcomes, in The 16^th^ Annual
    Research Conference Proceedings, A System of Care for Children's
    Mental Health: Expanding the Research Base. Edited by Newman C,
    Liberton CJ, Kutash K, et al. Tampa, FL, University of South
    Florida, The Louis de la Parte Florida Mental Health Institute,
    Research and Training Center for Children's Mental Health, 2003

32. Suter JC, Burchard JD, Bruns EJ, et al: User's Manual to the
    Wraparound Fidelity Index 3.0. Burlington, VT: University of Vermont
    Department of Psychology, 2003.

33. Bruns EJ: Is it wraparound yet? Bootstrapping fidelity standards
    using the Wraparound Fidelity Index. Paper presented at the 18^th^
    Annual Research Conference: Systems of Care: Building the Research
    Base. Tampa, FL, 2005.

34. Burns BJ, Goldman K (eds): Systems of Care: Promising Practices in
    Children's Mental Health, 1998 Series, Vol IV: Promising Practices
    in Wraparound for Children with Severe Emotional Disorders and Their
    Families. Washington, DC, Center for Effective Collaboration and
    Practice, 1999

35. Holden EW, De Carolis G, Huff B: Policy implications of the national
    evaluation of the comprehensive community mental health services for
    children and their families program. Children's Services: Social
    Policy, Research, & Practice 5:57-65, 2002

36. Bruns EJ, Suter J, Burchard J, Leverentz-Brady KM: A national
    portrait of wraparound implementation: Findings from the Wraparound
    Fidelity Index, in The 16^th^ Annual Research Conference
    Proceedings, A System of Care for Children's Mental Health:
    Expanding the Research Base. Edited by Newman C, Liberton CJ, Kutash
    K, et al. Tampa, FL, University of South Florida, The Louis de la
    Parte Florida Mental Health Institute, Research and Training Center
    for Children's Mental Health, 2003

37. Drake RE, Goldman HH, Leff HS, et al: Implementing evidence-based
    practices in routine mental health settings. Psychiatric Services
    52:179-182, 2001

38. Hernandez M, Gomez A, Lipien L, et al: Use of the system-of-care
    practice review in the national evaluation: evaluating the fidelity
    of practice to system-of-care principles. Journal of Emotional &
    Behavioral Disorders 9:43-52, 2001

Author Note

The authors gratefully acknowledge the contributions of many individuals
and agencies who have donated their time and efforts to the development
of the Wraparound Fidelity Index. The development of the WFI would not
have been possible without the help and support of our collaborating
sites. Thanks also to Karen Mehrtens of Johnson, Vermont, for serving as
a parent consultant to the research team that developed the WFI.

This project receives support from several sources, including the Center
for Mental Health Services, Substance Abuse and Mental Health Services
Administration; the Child, Adolescent, and Family Unit of the Vermont
Department of Developmental and Mental Health Services; and ORC Macro,
Inc.

Table 1

Administrative and System Characteristics of Study Sites

+----------------------------------------------+-----------+----------+
| **Program/system support variable**          | **Mean    | *        |
|                                              | (SD) / N  | *Range** |
|                                              | (%) of    |          |
|                                              | sites**   |          |
+----------------------------------------------+-----------+----------+
| **Years implementing wraparound**            | 3.93      | 2.5 - 7  |
|                                              | (1.45)    |          |
+----------------------------------------------+-----------+----------+
| **Staff turnover**                           |           |          |
+----------------------------------------------+-----------+----------+
| > Number of resource facilitators            | 18.2      | 8 - 35   |
|                                              | (10.2)    |          |
+----------------------------------------------+-----------+----------+
| > Number hired in last 12 months             | 6.00      | 1 - 12   |
|                                              | (3.2)     |          |
+----------------------------------------------+-----------+----------+
| > Turnover rate                              | 0.34      | .13 -    |
|                                              | (0.12)    | .50      |
+----------------------------------------------+-----------+----------+
| **Resource facilitator caseload**            |           |          |
+----------------------------------------------+-----------+----------+
| > Caseload size                              | 9.38      | 6 - 15   |
|                                              | (2.72)    |          |
+----------------------------------------------+-----------+----------+
| **Interagency coordination and funding**     |           |          |
+----------------------------------------------+-----------+----------+
| > Percent with community collaborative team  | 6 (75%)   |          |
+----------------------------------------------+-----------+----------+
| > Percent serving children referred by:      |           |          |
+----------------------------------------------+-----------+----------+
| > Special Education                          | 4 (50%)   |          |
+----------------------------------------------+-----------+----------+
| > Mental Health                              | 6 (75%)   |          |
+----------------------------------------------+-----------+----------+
| > Juvenile Justice                           | 6 (75%)   |          |
+----------------------------------------------+-----------+----------+
| > Child Welfare                              | 5 (63%)   |          |
+----------------------------------------------+-----------+----------+
| > Percent with \"decategorized\"/\"blended\" | 2 (25%)   |          |
| > funding                                    |           |          |
+----------------------------------------------+-----------+----------+
| > Agencies contributing to blended funding   |           |          |
| > pool:                                      |           |          |
+----------------------------------------------+-----------+----------+
| > Special Education                          | 2 (25%)   |          |
+----------------------------------------------+-----------+----------+
| > Health                                     | 1 (13%)   |          |
+----------------------------------------------+-----------+----------+
| > Mental Health                              | 2 (25%)   |          |
+----------------------------------------------+-----------+----------+
| > Juvenile Justice                           | 1 (13%)   |          |
+----------------------------------------------+-----------+----------+
| > Child Welfare                              | 1 (13%)   |          |
+----------------------------------------------+-----------+----------+
| **Family centeredness**                      |           |          |
+----------------------------------------------+-----------+----------+
| > Mean percent of community collaborative    | 16.2      | 0 - 55   |
| > teams that are consumers, parents, or      | (22.1)    |          |
| > family advocates                           |           |          |
+----------------------------------------------+-----------+----------+
| > Percent allowing parent/advocate referral  |  4 (50%)  |          |
+----------------------------------------------+-----------+----------+
| **Community based**                          |           |          |
+----------------------------------------------+-----------+----------+
| > Mean percent of community collaborative    | 2.50      | 0 - 10   |
| > teams that are non-agency community        | (4.18)    |          |
| > representatives                            |           |          |
+----------------------------------------------+-----------+----------+
| Mean percent of children served in community | 81.0      | 65-100   |
|                                              | (16.5)    |          |
+----------------------------------------------+-----------+----------+
| **Natural supports**                         |           |          |
+----------------------------------------------+-----------+----------+
| > Percent with mandated percent of natural   | 5 (63%)   |          |
| > supports on teams                          |           |          |
+----------------------------------------------+-----------+----------+
| > Percent allowing non-professional          | 5 (63%)   |          |
| > providers                                  |           |          |
+----------------------------------------------+-----------+----------+
| **Strengths focus**                          |           |          |
+----------------------------------------------+-----------+----------+
| > Percent with strengths-based assessment    | 8 (100%)  |          |
+----------------------------------------------+-----------+----------+
| > Percent with mandates for strength-based   | 6 (75%)   |          |
| > services                                   |           |          |
+----------------------------------------------+-----------+----------+
| **Individualization**                        |           |          |
+----------------------------------------------+-----------+----------+
| > Percent mandating recording goals          | 8 (100%)  |          |
+----------------------------------------------+-----------+----------+
| > Percent mandating recoding action steps    | 8 (100%)  |          |
+----------------------------------------------+-----------+----------+
| > Percent with flexible funds                | 7 (88%)   |          |
+----------------------------------------------+-----------+----------+
| > Percent with flexible funds immediately    | 7 (88%)   |          |
| > available                                  |           |          |
+----------------------------------------------+-----------+----------+
| > Percent requiring pre-approval for         | 5 (63%)   |          |
| > flexible funds                             |           |          |
+----------------------------------------------+-----------+----------+
| > Percent with limits to flexible funds      | 5 (63%)   |          |
+----------------------------------------------+-----------+----------+
| **Outcome orientation**                      |           |          |
+----------------------------------------------+-----------+----------+
| > Percent collecting and reviewing data      | 7 (88%)   |          |
+----------------------------------------------+-----------+----------+
| > Percent assessing family satisfaction      | 6 (75%)   |          |
+----------------------------------------------+-----------+----------+
| > Percent reviewing data in team meetings    | 5 (63%)   |          |
+----------------------------------------------+-----------+----------+

Table 2.

*Site-Level Ratings on WFI and WFI-PA*

+---------+---------------+----------------+--------------+----------+
| Site    | WFI-RF        | WFI-CG         | WFI-Y        | WFI-PA   |
|         |               |                |              |          |
|         | # M (SD)      | *M (SD)*       | *M (SD)*     | supports |
+---------+---------------+----------------+--------------+----------+
| 1       | \-            | 5.89 (1.44)    | 5.67 (1.51)  | 3        |
|         |               |                |              |          |
|         |               | n=43           | n=40         |          |
+---------+---------------+----------------+--------------+----------+
| 2       | 6.54 (.64)    | 5.92 (1.58)    | 6.30 (1.58)  | 2        |
|         |               |                |              |          |
|         | n=32          | n=26           | n=29         |          |
+---------+---------------+----------------+--------------+----------+
| 3       | 6.12 (1.00)   | 6.50 (.58)     | 6.37 (1.16)  | 5        |
|         |               |                |              |          |
|         | n=11          | n=11           | n=6          |          |
+---------+---------------+----------------+--------------+----------+
| 4       | 6.58 (.71)    | 5.66(1.52)     | 5.69 (1.44)  | 4        |
|         |               |                |              |          |
|         | n=46          | n=37           | n=25         |          |
+---------+---------------+----------------+--------------+----------+
| 5       | 6.61 (.66)    | 6.33 (.94)     | 6.05 (1.06)  | 7        |
|         |               |                |              |          |
|         | n=20          | n=12           | n=19         |          |
+---------+---------------+----------------+--------------+----------+
| 6       | 6.95 (.82)    | 6.52 (1.19)    | 7.01 (.74)   | 8        |
|         |               |                |              |          |
|         | n=26          | n=30           | n=20         |          |
+---------+---------------+----------------+--------------+----------+
| 7       | 6.88 (.57)    | 6.12 (1.44)    | 6.30 (.81)   | 7        |
|         |               |                |              |          |
|         | n=40          | n=30           | n=19         |          |
+---------+---------------+----------------+--------------+----------+
| 8       | 6.47 (.91)    | 6.81 (1.28)    | \-           | 7        |
|         |               |                |              |          |
|         | n=18          | n=32           |              |          |
+---------+---------------+----------------+--------------+----------+
| **Mean  | **6.59        | **6.22 (.38)** | **6.19       | **5.38   |
| for all | (.27)**       |                | (.46)**      | (2.2)**  |
| sites** |               | **N=221**      |              |          |
|         | **N = 193**   |                | **N=158**    | **N=8**  |
+---------+---------------+----------------+--------------+----------+

*Note.* WFI = Wraparound Fidelity Index; RF = resource facilitator; CG =
caregiver; Y = youth; PA = program administrator

![](media/image1.wmf)

*Figure 1*

Mean wraparound adherence scores by number of total administrative and
system supports