**POLICY AND PROGRAM STUDIES SERVICE**

**REPORT HIGHLIGHTS**

**Implementation Study of Smaller Learning Communities: Final Report**

The Smaller Learning Communities (SLC) program was established in
response to growing national concerns about students too often lost and
alienated in large, impersonal high schools, as well as concerns about
school safety and low levels of achievement and graduation for many
students. Authorized under the *Elementary and Secondary Education Act*
(Title V, Part D, Subpart 4, Section 5441(b)), the SLC program was
designed to provide local educational agencies with funds to plan,
implement, or expand SLCs in large high schools of 1,000 students or
more. The SLC legislation allows local educational agencies to implement
the most suitable structure or combination of structures and strategies
to meet their needs.

**Study Design**

The U.S. Department of Education contracted with Abt Associates to
conduct the Implementation Study of Smaller Learning Communities. The
primary purpose of the study was to evaluate the implementation of the
federal education law that authorizes funding for the SLC program, by
describing the strategies and practices used in implementing SLCs. The
report is based on findings from the first group (first cohort) of
grantee schools funded under this program in 2000. This first cohort of
119 SLC schools was surveyed at two points in time (spring 2002 and fall
2003). From among those freshman and career academies with the highest
student participation and degree of SLC implementation, 18 schools were
purposively selected for case studies.[^1] The study addresses three
major research questions:

-   How are schools implementing SLCs---what are the principal
    strategies, models, and practices implemented?

-   What are the factors facilitating and inhibiting implementation in
    SLC schools?

-   How do outcomes for SLC schools, as measured by student achievement
    and school behavior, change over time?

This study relied on three major sources of data: (1) Annual Performance
Reports (APRs), completed by all grantees/schools funded through the SLC
program, (2) a Periodic Implementation Survey (PIS), and (3) in-depth
case studies of 18 SLC schools that intended to implement a freshman or
career academy.

Major Study Findings

The study findings primarily concern the status of SLC implementation in
the Cohort 1 schools and factors facilitating and inhibiting
implementation. The study also examined in a limited manner how outcomes
as self-reported in the APR data changed for Cohort 1 schools over time.

Implementation Findings

***How are schools implementing SLCs---what are the principal
strategies, models, and practices implemented?***

-   **The most prevalent structures were freshman and career
    academies.** Schools changed over time, in both the number and types
    of SLC structures they were implementing. Freshman academies showed
    the most growth. In 2001--02, 38 percent of SLC schools had freshman
    academies; by 2002--03, the number had risen to 55 percent. Career
    academies showed some growth (from 38 percent of schools to 42
    percent).

```{=html}
<!-- -->
```
-   Schools chose to implement one or more SLC strategies, with block
    scheduling (58 percent of schools) and teacher teams (52 percent)
    the most popular choices. However, schools over time were gradually
    shifting from the use of SLC strategies to a greater use of SLC
    structures.

-   Schools with freshman academies reported a high level of
    participation (78 percent on average) among their 9th-grade
    students. For house plans, average student participation was 77
    percent during the 2002--03 school year.

-   All but two schools reported undertaking efforts to increase
    personalization. The most popular mechanisms for enhancing
    personalization were school or classroom-based and involved
    providing individual assessments (76 percent), a cooperative
    learning focus (63 percent) or formal mentoring programs (47
    percent).

```{=html}
<!-- -->
```
-   **SLC-related professional development, although provided by nearly
    all schools, was not very extensive.** SLC teachers received a
    little more than three days of professional development per year. In
    close to half of Cohort 1 schools (45 percent) teachers received
    less than 16 hours of SLC-specific professional development during
    the 2002--03 school year.

```{=html}
<!-- -->
```
-   **Schools reported success in involving community representatives in
    their SLC activities, with four-fifths of schools (82 percent)
    working with an external partner in 2002--03, up from two-thirds of
    schools (65 percent) in the previous year.** Partners included
    businesses, institutions of higher education, and community based
    organizations. Most schools used partners on advisory committees and
    as in-school volunteers. Those schools engaging external partners
    with their SLCs reported that they derived specific benefits for
    their students, including a range of career-related opportunities
    such as community service learning, internships, and job shadowing.

-   **The demographics of career and freshman academies often did not
    match the demographics of the school or freshman class.** The law
    authorizing SLCs mandates that the "method of placing students in
    the smaller learning community or communities \[shall be\] such that
    students are not placed according to ability or any other measure,
    but are placed at random or by their own choice, and not pursuant to
    testing or other judgments" (P.L. 107-110, Section 4441). **About
    half of the schools with either career or freshman academies had
    their enrollments in each academy match the racial composition of
    the school as a whole (for career academies) or the freshman class
    (for the freshman academies).** About half the schools with freshman
    academies had matched enrollments for limited English proficient
    (LEP) students and 38 percent of schools had similar LEP
    demographics for career academies. Three-quarters of the freshman
    academies had matched enrollments by gender, compared with 29
    percent of schools with career academies. Because the data reported
    in the APR do not distinguish between enrollments based on school
    random assignment or student's choices, it is not possible to
    ascertain the extent to which the differences in demographics are
    based on student choice rather than school assignment; however these
    comparisons suggest that schools are clearly challenged to create
    academies that match the population from which the academies are
    drawn. As the data reveal, schools find it less difficult to have
    freshman academy groupings similar to the freshman class than to
    have career academies that mirror the demographics of the school.

***What are the factors facilitating and inhibiting implementation in
SLC schools?***

-   **SLC respondents reported** that these factors to facilitate
    implementation: professional development specifically focused on
    SLCs; the availability of resources, including instructional
    materials; and a variety of teacher-related variables (e.g.,
    attitudes toward reform, pedagogical practices, and expertise).
    Schools also reported a number of factors to have a negative
    influence on SLC implementation, including scheduling and logistical
    issues, physical space, and school staffing needs, especially in
    terms of core academic teachers and guidance counselors.

-   A common set of **factors affecting academy implementation**
    included strong school leadership, involved and supportive
    districts, high levels of staff buy-in, and sufficient space to make
    programs separate. Inhibiting factors included staff and
    administrative turnover, weak school leadership, prescriptive
    district oversight of SLC reforms, and limited resources on the part
    of the school.

**APR Data on Outcomes**

The section below presents a comparison of the reported APR data related
to key program outcomes in the period just prior to program
implementation and just after program implementation. The data are based
on the SLC schools' self reported data through Annual Performance
Reports (APR). **Schools first completed the APR during the 2000--01
school year, at which time they also provided retrospective data for
school years 1996--97 through 1999--2000.** APR data were also collected
annually for school years 2001--02 and 2002--03. The APR data includes
information on academic achievement, school-related behaviors, and the
achievement of academic milestones at the school level.

**Limitations of the APR Outcome Analysis**

While analysis of the APR data give some self-reported information on
how schools were trending over time before receiving SLC funding and
whether or not there was a measured shift in trends when schools
received SLC funds, absent a valid comparison group, any inferences from
this data about the impacts of SLC funding and implementation on those
outcomes are clearly inappropriate. In addition, there are a number of
very important caveats and limitations that also make use of this data
for evaluation of outcomes or impact analysis inappropriate. These are
summarized below.

-   Many schools were engaged in implementing SLCs structures and
    strategies prior to receiving their federal grants, which could
    potentially have affected their pre-grant status on outcomes.

-   In many cases the SLC feature being implemented only directly
    affected a subset of students in the school, while outcomes were
    reported for the school as a whole.

-   The data collection period did not cover a sufficient period of time
    to adequately capture changes in end of high school outcomes where
    implementation activities may have focused primarily on 9th-grade
    students.

-   The dynamics of the SLC implementation process may have affected
    short-term school outcomes as schools adjusted to the task of
    restructuring. That is, restructuring such a large institution as a
    high school may not only lead to no immediate changes, but there may
    actually be a temporary worsening of outcomes as school staff take
    on and become accustomed to their new roles.

-   Results are based on school-reported data, which varied greatly in
    quality and accuracy. Specifically, there is a serious measurement
    issue in terms of the lack of data comparability (both between
    districts and states).

    Keeping in mind these limitations, the APR data revealed the
    following:

    ***Short-Term Outcomes***

```{=html}
<!-- -->
```
-   **As measured by APR data, early changes in schoolwide reported
    outcomes after receiving SLC funding were modest or neutral, with a
    good deal of variation between schools.**

-   Where there is evidence of change, trends appear to be moving in the
    right direction for school-related behaviors. **Specifically, the
    APR data suggest an upward trend in student extracurricular
    participation and promotion rates from 9th to 10th grades.** The
    trend for extracurricular involvement in SLC schools showed a
    substantial and statistically significant increase of five
    percentage points in participation after receipt of SLC funding.

```{=html}
<!-- -->
```
-   **There was a statistically significant positive trend in the
    percentage of 9th-grade students being promoted to 10th grade during
    the post-grant period.** This trend also held for SLC schools
    implementing freshman academies, which have as an expressed focus
    reducing the 9th-grade dropout rate. In addition, mean estimates
    were similar to the national average for large high schools by the
    end of data collection (85 percent).

-   **There was also a downward trend in the incidence of violence in
    SLC schools over time.** The data suggest that, on average, SLC
    schools experienced a statistically significant 1.4-point drop in
    the number of violent incidents (per 100 students) during the
    post-grant period.

Longer-Term Outcomes

-   **The data suggest increases in the percentage of graduating
    students planning to attend either two- or four-year colleges.**
    Between the pre- and post-grant periods, this percentage increased
    by about four percentage points, which is statistically significant.
    The absence of comparative national data, however, makes it
    difficult to infer whether this is due to receipt of the SLC grant
    rather than part of a more general national trend.

```{=html}
<!-- -->
```
-   There were **no significant trends in academic achievement**, as
    measured by either scores on statewide assessments or college
    entrance exams.

Sustainability of SLCs

The data suggest a serious commitment on the part of most SLC schools to
sustain structural changes in the way their school and classrooms are
organized. Specifically, close to three-quarters of those schools that
report having made changes using SLC funding expect to sustain those
changes after their grants end. For example, almost all (96 percent) of
the schools that reported making their schoolwide core curricula more
academically rigorous are committed to sustaining those changes even
after their SLC funding has run out. Similarly, 94 percent of the
schools that reported using more varied student assessments for grading
and promotion decisions expect to sustain those changes in the future.

Although schools were less likely to report classroom-level changes with
the federal SLC funding, at least 80 percent of the schools that had
implemented classroom-level changes also reported that they would
sustain them. One exception is reduced class size, a change that may not
be within the power of the school to sustain.

[^1]: This report does not include findings from the second cohort of
    222 SLC schools funded in 2002. These schools were surveyed at only
    one time and did not have any case study visits. Findings for this
    cohort of SLC schools are summarized in the unpublished *Cohort 2
    Follow-up Report* (Bernstein, Millsap, & Schimmenti, 2005) available
    upon request.