POLICY AND PROGRAM STUDIES SERVICE

REPORT HIGHLIGHTS

**This is some**

**Second Evaluation of the Improving Literacy Through School Libraries
Program**

**[Background]{.underline}**

The Improving Literacy through School Libraries (LSL) program was
established under Title I, Part B, Subpart 4 of the *Elementary and
Secondary Education Act (*ESEA) by the *No Child Left Behind Act of
2001(NCLB)*. The purpose of the program is to improve the literacy
skills and academic achievement of students by providing them with
increased access to up-to-date school library materials; well-equipped,
technologically advanced school library media centers; and well-trained,
professionally certified school library media specialists.

Included in the legislation was a requirement for an evaluation of the
program to be conducted no later than three years after the enactment of
NCLB and biennially after that. This report provides findings from the
second evaluation of the LSL program. It contains new survey data on the
2005-06 school year and an analysis of the relationship between student
test scores and the receipt of LSL grants in 2003-04.

The following four data sources were used in the evaluation:

-   **A survey of school libraries.** The survey was sent in the fall of
    2006 to a sample of 400 school libraries served by the grant in
    2005-06 (grantees) and to a matched comparison sample of 400 schools
    in districts that were eligible for the grant in that year
    (nongrantees).

-   **The district performance reports.** Each grant recipient must
    submit a report to the Department of Education (ED), components of
    which include a description of the project, highlights of key
    accomplishments, a report on how the district met each of its
    project objectives, a project evaluation, and information on
    expenditures, schools served, and professional development.
    Performance reports from districts receiving the grant in 2004-05
    and 2005-06 were analyzed for this evaluation.

-   **Case study site visit reports.** Site visits were made to nine
    school districts, including one or two schools within the districts
    visited. The visits focused on promising school library practices in
    districts that had received LSL grants in 2003 through 2005. Site
    selection was based on performance and demographic data. Site visits
    were conducted by one- or two-person teams between November 2006 and
    January 2007.

-   **Test scores.** Annual school-level test score data on students'
    proficiency in reading/language arts were obtained from the National
    Longitudinal School-Level State Assessment Database (NLASSAD). This
    database is a compilation of test results from each state's
    proficiency exams. NCLB uses these results for accountability
    purposes. The most recent test score data available at the time of
    this report were for 2004-05, which was prior to receipt of the LSL
    grants by the grantees surveyed in this evaluation. Therefore, the
    test score data were merged with the survey data from the first LSL
    evaluation in order to measure whether changes in school libraries
    were associated with changes in student test scores. Of the 701
    respondents to the 2004 survey, test scores were available for 553
    schools (79 percent).

# [Key Findings:]{.underline}

-   Generally, the approaches used by districts to select schools for
    participation in the grant have been the same each year between
    2003--04 and 2005--06. Districts often reported selecting schools to
    participate in the grant based on various kinds of disadvantages at
    those schools: 36 percent chose schools based on a lack of library
    resources, 22 percent based on the poverty level, and 20 percent
    based on those identified for improvement under *NCLB*.

-   The survey of library media centers confirmed that on many
    characteristics the grantee schools started with a relative
    disadvantage as compared to nongrantee schools.

```{=html}
<!-- -->
```
-   Among those districts conducting needs assessments, grantees were
    significantly more likely than nongrantees to identify needs for
    more library staff (55 percent versus 33 percent), more up-to-date
    materials (95 percent versus 85 percent), and more space (51 percent
    versus 33 percent). Additional needs identified more frequently by
    grantees were opening the library for more hours (75 percent versus
    41 percent), more time for planning with teachers (64 percent versus
    40 percent), and more professional development (75 percent versus 57
    percent). Overall, the percentage of grantees and nongrantees
    identifying various needs was about the same in the first and second
    evaluations.

-   Prior to the grant, grantees were more likely than nongrantees to
    report their holdings were inadequate in all four general areas that
    were examined in the survey: the overall reading/English collection
    (34 percent versus 22 percent), print materials (35 percent versus
    23 percent), video/ audiovisual materials (52 percent versus
    37 percent), and computer software (57 percent versus 44 percent).

```{=html}
<!-- -->
```
-   Grantees roughly tripled their expenditures on books, subscriptions,
    and computer hardware, while nongrantees showed little change. In
    the first evaluation, grantees roughly doubled their expenditures on
    these items.

-   Of those schools that conducted needs assessments, the grantees were
    often more likely to make changes, including getting more up-to-date
    materials (92 percent versus\
    78 percent), providing more time for planning with teachers
    (39 percent versus\
    21 percent), increasing professional development (68 percent versus
    41 percent), and increasing the hours the library was open (62
    percent versus 24 percent). Overall, the percentage of grantees and
    nongrantees making changes to address needs was about the same in
    the first and second evaluations.

-   The first evaluation found that, overall, the grants seemed to
    compensate for grantees' earlier disadvantages and bring them to
    rough equality with the nongrantees, whereas the second evaluation
    found that the grants enabled the grantees to surpass the
    nongrantees. In spring 2005, prior to the grant, significantly fewer
    grantees considered their reading/English materials to be adequate
    or excellent compared to nongrantees for all types of materials,
    i.e., overall reading/English literature, print materials,
    video/audiovisual materials, and computer software. In contrast, in
    spring 2006, during the grant year, significantly more of the
    grantees considered their overall reading/English literature, print
    materials, and computer software to be adequate or excellent
    compared to nongrantees.

-   In both the first and second evaluations, grantees often were more
    likely to provide new or expanded programs than nongrantees. In the
    second evaluation, these included providing instruction on
    information skills (52 percent versus 36 percent), working with
    classroom teachers on selecting reading or English resources (49
    percent versus 33 percent), holding family literacy nights (38
    percent versus 13 percent), and holding after-school programs
    offering an orientation to the library (34 percent versus
    8 percent).

-   Grantees increased access to libraries in several ways. During the
    school year, 39 percent of grantees provided access during nonschool
    hours in 2005, and 72 percent in 2006. Among nongrantees, the
    statistics were 54 percent in 2005 and 56 percent in 2006 (Exhibit
    E-3). Grantees increased the number of days the library was open
    during the summer from a mean of 2.9 in 2005 to 7.7 in 2006,
    compared with no significant change among nongrantees in the amount
    of summer access. In the first evaluation, grantees showed
    significant increases in the hours they were open in a typical full
    week in spring, as well as in the days that the libraries were open
    in the summer, while the grantees showed no significant change.

-   To the extent that libraries increased the hours they were open, one
    might expect that student usage of the libraries would
    increase---e.g., in the number of students using the libraries in a
    typical week or in the number of materials that were checked out. As
    in the first evaluation, grantees showed an increase in usage per
    week but no significant change in number of materials checked out.

-   Grantees were more likely than nongrantees to report establishing
    new programs or expanding existing programs that involved
    collaboration. Specifically, these include programs to have library
    media staff assist teachers in designing, implementing, and
    evaluating research projects for students (42 percent versus 24
    percent), coordinate training programs about integrating educational
    technology into the curriculum for teachers and other staff (42
    percent versus 22 percent), work with the principal and/or teachers
    on curriculum issues (40 percent versus 23 percent), and participate
    in team meetings (36 percent versus 23 percent). In the first
    evaluation, grantees were also more likely than nongrantees to have
    increased services involving collaboration.

-   In schools that participated in LSL in 2003--04, the percentage of
    students who met or exceeded the proficiency requirements on state
    reading assessments increased by an extra 2.7 percentage points over
    the increase observed among nonparticipating schools during the same
    time period. However, some or all of the increase may be associated
    with other school reform efforts that also appeared in the schools.
    Thus, no definitive statement can be made based on these data on
    whether LSL participation was associated with improved test scores
    that was separate from these other programs.

The findings from the analyses of student reading achievement should be
interpreted with considerable caution. Although the LSL and non-LSL
schools are similar demographically, they have not been randomly
assigned to treatment and control groups. Therefore, we cannot assume
that the two groups of schools are equivalent but for the fact that one
group received LSL funds while the other did not. This means that the
findings cannot support causal inferences that attribute observed
differences in student reading achievement between LSL and non-LSL
schools to the LSL program.

This report is available online at
www.ed.gov/about/offices/list/opepd/ppss/reports.html.