Library and Information Services

Policy Forum Proceedings

Topic: Library Services and

Technology Act (LSTA)

State Grant Program:

Implications for Use of and

Additions to National Library Data

September 15-16, 1997

Prepared by Denise Glover at Westat

1650 Research Boulevard

Rockville, Maryland 20850

(301) 251-1500

for

National Center for Education Statistics

555 New Jersey Avenue, N.W.

Washington, DC 20208-5652

May 7, 1998
TABLE OF CONTENTS

INTRODUCTION.............................................. 1

Background on Library and Information

Services Policy Forums................................ 1

LSTA – 1997 Library and Information Policy

Forum ......................................................... 2

Objectives and Format of the Forum............ 3

Forum Overview ........................................ 4

SUMMARY OF PRESENTATIONS............... 5

Welcome and Introductions

Pat Forgione............................................ 5

Jeanne Hurley Simon............................... 6

Diane Frankel.......................................... 6

Sandy Cooper.......................................... 7

Tom Sloan .............................................. 7

PRESENTATIONS......................................... 9

Assessing Progress Towards LSTA’s Purposes.. 8

Mary Jo Lynch ........................................ 8

Status Report on COSLA Evaluation Committee 8

Amy Owen.............................................. 8

Electronic Linkages and Services: Panel on

Methods for Measurements and Reporting ......... 9

Gerry Rowland........................................ 9

Aleck Johnson......................................... 10

John Bertot ............................................. 11
Determining Underserved Populations............... 13

Steve Fischer and Neal Hohmann ............. 13

Gary Strong ............................................ 14

LSTA Connection to Library Services to

Children........................................................... 16

Mary Somerville...................................... 16

Carol Rasco ............................................ 16

Using Qualitative Information in a Quantitative

Environment .................................................... 18

David Penniman...................................... 18

Douglas Zweizig ..................................... 18

Joey Rodger............................................ 19

Importance of LSTA Reporting for National

Understanding.................................................. 19

Carol Henderson ..................................... 19

Telecommunication Rates—Update and

Impact ............................................................. 24

Linda Roberts.......................................... 24

Martha Hogerty....................................... 25

SMALL GROUP REPORTS......................... 28

Final Reports and Recommendations from the

Small Group Discussions.................................. 28

Group 1: Tom Sloan, Chair ..................... 28

Group 2: Rod Wagner, Chair................... 31

Group 3: Bridget Lamont, Chair.............. 32

Group 4: James Scheppke, Chair ............. 34
APPENDIXES OF SELECTED

PRESENTATIONS AND PAPERS

A A Strong Working Partnership Toward

Quality Data: NCES and the Library

Communities

Pascal D. Forgione, Jr., Ph.D........................... 37

B Assessing the Progress Towards LSTA’s

Purposes: How The NCES Library

Surveys Can Help

Mary Jo Lynch............................................... 45

C To Boldly Go—Searching for Output

Measures for Electronic Services

Mark Smith and Gerry Rowland ........................ 57

D Using Qualitative Information in a

Quantitative Environment--Overheads

David Penniman ............................................ 63

E The Importance of LSTA Reporting and

Evaluation for Continued Federal Funding

and Reauthorization

Carol C. Henderson........................................ 71
Librarand Information Services Page 1

Policy Forum Proceedings

September 15-16, 1997

Library and Information Services

Policy Forum Proceedings

Topic: Library Services and Technology Act (LSTA)

State Grant Program: Implications for Use of and

Additions to National Library Data

INTRODUCTION

he 1997 Library and Information Services Policy Forum

was sponsored by the National Center for Education

Statistics (NCES), in cooperation with the U.S. National

Commission on Libraries and Information Science (NCLIS),

with the assistance of the Institute of Museum and Library

Services (IMLS), the Chief Officers of the State Library

Agencies (COSLA), and the National Institute on

Postsecondary Education Libraries and Lifelong Learning.

These proceedings summarize the presentations and

recommendations from the meeting held on September 15-16,

1997, in Arlington, Virginia. The proceedings provide

background information on the Forum, including an overview

of the meeting content, objectives, and format, and conclude

with recommendations from the small group discussions.

Attachments include the text of presentations available from

some Forum speakers.

Background on Library and Information

Services Policy Forums

NCES has sponsored Library and Information Services Policy

Forums on various topics. The purpose of the forums is to

provide input to NCES on the collection, dissemination, and

T
Page 2 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

use of national library statistics. The meeting is generally an

invitational forum of approximately 50 professionals

representing library researchers, and practitioners from

academic, public, and school library communities, and

representatives from IMLS, the Federal-State Cooperative

System (FSCS), the American Library Association (ALA) and

its affiliate associations, NCLIS, COSLA, as well as staff from

NCES’ Library Statistics Cooperative Program.

LSTA – 1997 Library and Information

Services Policy Forum

The subject of the 1997 Forum was the Library Services and

Technology Act (LSTA) State Grant Program: Implications for

Use of and Additions to National Library Data. The LSTA was

selected as the topic for the 1997 Forum because it requires the

library community to balance the need to collect national, state,

and local statistics. National statistics, such as those collected

by NCES, can reflect a common vision, purpose, and goals for

all library programs. State or local statistics can demonstrate

the effectiveness of unique programs designed to meet needs

identified by individual states in their state plans. The 1997

Forum provided an opportunity for librarians, researchers, and

statisticians to develop recommendations concerning the

content areas for data collection in support of LSTA goals.

The purposes of the funding as identified in LSTA, Section

231, is to:

• Establishing or enhancing electronic linkages among or

between libraries;

• Electronically link libraries with educational, social, or

information services;
Library and Information Services Page 3

Policy Forum Proceedings

September 15-16, 1997

• Assisting libraries in accessing information through

electronic networks;

• Encouraging libraries in different areas, and encouraging

different types of libraries, to establish consortia and share

resources;

• Paying costs for libraries to acquire or share computer

systems and telecommunications technologies;

• Targeting library and information services to persons

having difficulty using a library and to underserved urban

and rural communities, including children (from birth

through age 17) from families with incomes below the

poverty line; and

• Targeting individuals with disabilities and persons with

limited functional literacy or informational skills.

Objectives and Format of the Forum

The specific Forum objectives were to:

• Review library statistics and research information to

determine existing data items that may help local, state, and

federal participants in the Library Services and Technology

Act of 1996 to report on LSTA programs;

• Recommend new data elements and methodologies that

may be useful to meet the measurement and reporting

requirements of LSTA goals; and

• Contribute to the capabilities of state and national reporting

agencies to provide the statistics and information that will

reflect LSTA program performance and results.

The meeting format included a combination of presentations in

plenary sessions and small group discussions. The

presentations informed participants about various

methodologies, results from research projects, and programs
Page 4 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

related to the purposes of LSTA. In small groups, participants

reviewed current NCES library surveys and other research

projects, considered new data elements, and identified

methodologies for data collection.

Presentations

The Forum included welcome and remarks from:

Pat Forgione, NCES

Jeanne Hurley Simon,

NCLIS

Diane Frankel, IMLS

Sandy Cooper, COSLA

Tom Sloan, COSLA

In addition, the following topics were presented:

Assessing Progress Towards LSTA’s Purposes

Mary Jo Lynch

Status Report on COSLA Evaluation Committee

Amy Owen

Electronic Linkages and Services

Gary Rowland, Aleck Johnson, John Bertot

Determining Underserved Populations

Steve Fischer, Neal Hohmann, and Gary Strong

LSTA Connection to Library Services to Children and America

Reads!

Mary Somerville and Carol Rasco

Using Qualitative Information in a Quantitative Environment

David Penniman

Evaluation Techniques through the “Tell It” Method

Douglas Zweizig

Importance of LSTA Reporting for National Understanding

Carol Henderson

Telecommunications Rates: Update and Impact

Linda Roberts and Martha Hogerty

Each of the presentations is summarized on the following

pages.
Library and Information Services Page 5

Policy Forum Proceedings

September 15-16, 1997

SUMMARY OF PRESENTATIONS

Welcome and Introductions

at Forgione, NCES Commissioner, provided an overview

of NCES’s mission and its library surveys and discussed

how baseline data collected in several of NCES’s surveys can

be used for the LSTA. (Refer to the full text of this speech in

appendix A.)

The Center’s mission is to collect, analyze, and disseminate

statistics and other data related to education in the United

States. Forgione indicated that the Center has been collecting

data on academic libraries since 1966 as part of the general

higher education data collection. Library data are collected for

the different types of libraries through a variety of surveys.

Public library statistics have been collected through the FederalState Cooperative System (FSCS) since 1988. In 1993, the

Center surveyed school libraries and media centers. More

recently, the Center has surveyed state library agencies and

federal libraries. In addition, the Center’s National Household

Education Survey (NHES) included several questions on public

library use. NCES is also planning a library cooperatives

survey this year. This survey will provide key national and state

indicators of progress toward meeting the LSTA objective of

linking libraries by monitoring changes in the number and types

of library cooperatives and in the number and types of the

libraries they serve.

Forgone emphasized that the NCES role is to collect basic

information, not just data on the hot issues of the day. Data

collection cannot be driven solely by evaluation needs or

legislative program needs; however, NCES is making progress

P
Page 6 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

toward responding to new needs (e.g., the public library survey

is adding electronic services questions to the 1997 survey).

Analyzing data on libraries that are collected in conjunction

with other data sets is another strategy for developing national

and state indicators. For example, NCES is supporting a study

to map public libraries’ service areas and to geocode library

buildings. Once completed, a merged data file containing the

mapping data, the public libraries survey data, and Census data

will be created. This type of analysis will be invaluable to the

states in their need to know the level of LSTA funding received

by each library in the state and how the observed changes fit

into the state plan.

Jeanne Hurley Simon, Chair, U.S. National Commission on

Libraries and Information Science (NCLIS) welcomed

attendees and honored John Lorenz for his distinguished service

to the Library Statistics Program and to the larger library and

information service community.

Diane Frankel, Director, Institute of Museum and Library

Services (IMLS), discussed the implications of the

Government Performance and Results Act (GPRA) for the

continuation of LSTA funds. Due to GPRA, which went into

effect in 1993, budget allocations are based on the ability to

achieve results. Frankel also reviewed the purposes of the

LSTA and described the process for coordinating data

collection and evaluation efforts with COSLA. She suggested

that it is critical for funding agencies and the library community

to answer two questions about evaluation:

1. What difference is federal funding making to our

communities? We must support the anecdotal stories with

more concrete data.

2. How are we going to measure services to underserved

populations?
Library and Information Services Page 7

Policy Forum Proceedings

September 15-16, 1997

Sandy Cooper, Forum Chair and COSLA Liaison to IMLS,

reviewed the forum objectives, including:

• Discussing what is currently being collected;

• Identifying new data elements, definitions, and

methodologies needed to support LSTA; and

• Discussing how NCES, COSLA, IMLS, and NCLIS can

work together to contribute to local and national efforts.

Tom Sloan, COSLA representative, explained the small

group process. The groups were tasked to:

1. Refer to proposed FSCS data definitions, but not limit

discussion to these definitions or to public libraries;

2. Capture what we would like to collect as well as what we

can collect;

3. Identify methodologies for measurement; and

4. Identify data items that would measure the value of LSTA

in terms of impact and outcomes for our customers, and

identify key outcomes we would like to accomplish with

LSTA.
Page 8 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

PRESENTATIONS

Assessing Progress Towards LSTA’s Purposes

Mary Jo Lynch

Mary Jo Lynch of the American Library Association

provided highlights from her paper “Assessing Progress

Towards LSTA’s Purposes: How the NCES Library Surveys

Can Help.” She characterized her work as a narrative

description of the existing structure in NCES for collecting data

on libraries in the United States. Lynch’s paper reflects on how

these surveys relate to LSTA. (Refer to full text of document in

appendix B.)

Status Report on COSLA Evaluation Committee

Amy Owen

Amy Owen, Chair, COSLA Evaluation Committee,

described some of the challenges of LSTA, provided a brief

overview of the state plans, and presented a status report on the

Committee’s work. One of the challenges of LSTA is to

demonstrate through measurement and evaluation that services

have been improved as a result of the funding. Owen shared

the following thoughts about the state plans (which are

available on the World Wide Web): (1) plans show great

diversity among states, (2) goals and objectives of the plans are

not tied to dollars, (3) plans vary on the extent to which they

will be state-driven versus locally driven, (4) plans demonstrate

evidence of baseline, formative, summary, or impact data, and

(5) definitions used in and provided by LSTA may be

interpreted differently by local folks.

The task of the COSLA Evaluation Committee is to develop a

model to be used by COSLA to gather data useful at the state
Library and Information Services Page 9

Policy Forum Proceedings

September 15-16, 1997

level and at IMLS. The committee tentatively has identified 50

data elements and has recognized the following data collection

needs:

• To develop key performance indicators that are easy to

collect and useful at state and national levels;

• To develop ways to identify and share information about

best or promising practices, and to consider creating

templates for reporting specific types of projects;

• To identify baseline data within states and within NCES;

and

• To develop ways to measure impact and outcomes.

The next phase of COSLA’s work involves:

1. Creating a subgroup to examine the collectability of the

data;

2. Mailing a draft of the data elements to the chief officers

before the fall 1997 meeting;

3. Refining data elements and reducing the number of data

elements before ALA midwinter; and

4. Providing IMLS with definitions before spring 1998.

The Fast Response Survey System (FRSS) or NHES could

possibly include the new data elements.

Electronic Linkages and Services: Panel on Methods

for Measurements and Reporting

Gerry Rowland, Aleck Johnson, and John Bertot

The panel members discussed evaluation and measurement

(inputs and outputs) of electronic services in libraries and the

challenges of measuring these services. Gerry Rowland, a
Page 10 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

library researcher with the Iowa State Library and a

member of the FSCS Steering Committee, provided the context

for the discussion. Rowland indicated that a panel of experts

discussed Internet use about three years ago. The problems

they encountered with definitions of the number of “hits” and

with state librarians accepting output measures about Internet

use prompted them to focus on input measures. (Gerry

Rowland’s article on Output Measures appears in appendix C.)

Aleck Johnson of the American Library Association, Office

of Information Technology, talked about evaluation and how

ALA is looking at this issue. He reported on the results of a

meeting with Markel Foundation with researchers from Rand,

American Institutes for Research, and the U.S. Department of

Education. The purpose of the meeting was to come up with a

“dream list” of information they would like to retrieve from the

ideal process for evaluating library services. This group

identified four categories of information: (1) access and equity,

(2) usage issues and questions, (3) impact measurements and

qualitative measures, and (4) big-picture issues.

Johnson proposed guiding questions under each of the four

categories. Under access and equity issues he asked:

• Are more libraries getting access to the Internet? Is Internet

access spreading to traditionally underserved areas? How is

Internet access changing in these areas?

Under usage, Johnson asked:

• Are patrons using technology and how are they using it?

What new services are patrons using? What is the next

wave of services? How can we assure that libraries have

access to this next wave of services? Is usage spreading to

patrons as well as to library staff?
Library and Information Services Page 11

Policy Forum Proceedings

September 15-16, 1997

For impact and qualitative measures, Johnson posed the

following questions:

• What is the impact of access on learning? How do you

measure it? What are the right questions to ask about

impact on learning?

Johnson suggested that policymakers need answers to the

question: “What has been the substantive effect of this policy?”

Finally, Johnson provided examples of big-picture questions:

• What are the effects of universal service on the budgets of

libraries? Does universal service mean budgets are

skewed? Does universal service mean that funds currently

allocated for technology uses get reallocated to other areas?

Or does universal service mean that libraries can expand

services?

Johnson concluded by indicating that the data collected to

respond to the above questions are very useful for Congress and

for local and state agencies.

John Bertot, a professor from the University of Maryland

at Baltimore County, reported on the 1997 Public Libraries

and the Internet Survey sponsored by the American Library

Association and the U.S. National Commission on Libraries

and Information Science. Bertot described the tension between

specific data collection activities that consider current and

future trends, as opposed to ongoing data collection needs that

are part of established data collection activities. Bertot

indicated that he saw a challenge in changing established data

collection to meet the needs of the future.

The 1997 Public Libraries and the Internet Survey revealed that

data could be collected on the following: connectivity, services,

costs, uses, and impact. Bertot described the challenges of

collecting these data to arrive at comparable costs: (1) rural
Page 12 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

areas connect through dial access (since rural librarians do not

have direct access, it is difficult to compute their

telecommunications costs), (2) equipment costs and

telecommunications costs are subsidized in rural areas, and (3)

public libraries (regionals, consortia, etc.) do not know how

much they are receiving in subsidies..

Bertot indicated that data on connectivity, services, costs, uses

and impact need to be broken down in several ways: by city,

suburban, rural areas; population served; state level; and

branch/outlet level. He stated that questions such as, “How

pervasive is Internet activity?” need to be answered. For

example, Bertot mentioned that we do not know how many

branches are connected to the Internet in a public library

system.

Bertot concluded by underscoring the need for all data

collection entities—NCES, NCLIS, ALA, Public Libraries

Association—to coordinate their data collection activities to

avoid duplication and to examine how these activities fit

together into a coherent plan about Internet connectivity.

Participant Comment

Based on his experience, one participant determined that cost

data are hard to collect; and cost data are so ill-defined that

some states do not follow definitions and therefore information

cannot be used.
Library and Information Services Page 13

Policy Forum Proceedings

September 15-16, 1997

Determining Underserved Populations

Steve Fischer and Gary Strong

Public Library Geocoding Demonstration

Steve Fischer from Westat, with the assistance of Neal

Hohmann (Geographic Information Systems USA),

provided an overview of the NCES geomapping project for

public libraries, which began in October 1996. The purpose of

this project is to geocode FSCS public library locations and

identify and map associated library legal service areas. Once

completed, a merged data file containing the mapping data, the

public library FSCS data, and selected Census data will be

created for use in a geographic information system application.

This type of capability will be invaluable since it will enable

libraries and agencies to readily perform analysis previously

possible only through the use of multiple data sources and

complex analytical techniques.

The project encompasses all public library legal service

jurisdictions and over 17,000 public library service outlets,

nationwide. The project involves (1) geocoding all library

outlets, (2) mapping and digitizing library legal services

jurisdictions, and (3) providing documentation and a user’s

guide.

The project tasks are divided into five phases: (1) geocoding

“readily processable” “ and moderately difficult” addresses, (2)

geocoding “significantly difficult” addresses (e.g., post office

boxes), (3) processing conforming legal service area boundaries

(county, school district, etc.), (4) producing the initial data file,

and (5) determining the method and process for nonconforming legal service area boundaries. Currently, 80

percent of the outlet records are in the file. Subsequent

geocoding of difficult addresses (20 percent) have been placed

in seven categories: post office box number, no identifying

street number, box number only, an incorrect address, no

address, duplicate address, and other.
Page 14 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

Fischer concluded his presentation with a demonstration of

what can be done with the data. He showed maps of library

outlet locations and indicated how FSCS data and Census data

can be related. For example, the system could be used for

determining and depicting an underserved population based on

the parameters and definitions established by a library or library

system.

Participant Comments

One participant expressed concern about who would have

access to the data and how it was going to be used. A second

participant suggested that the file itself, the user’s guide, and/or

the documentation should describe ways for interpreting the

data.

Gary Strong, Director of Queens Borough Public Library,

presented their geomapping project. Strong’s project focuses

on identifying immigration patterns and projections and

languages spoken to determine where the library needs to

concentrate services. The data collected and analyzed from the

demographic project are used to plan, design, and make

decisions about public library programs, services, and

collections. Strong described some of the unique characteristics

of their library and the library population:

• Urban public library with 62 branches, population served is

1,950,000;

• Forty percent of population born outside the United States;

• Forty-four percent speak a language other than English;

over 80 languages are spoken;

• Over 800,000 registered borrowers; and
Library and Information Services Page 15

Policy Forum Proceedings

September 15-16, 1997

• Major collection development projects in Spanish, Chinese,

Korean, and South Asian languages; smaller projects in

Russian, Hebrew, and Greek.

Queens Borough Public Library has three projects underway.

Strong distributed three booklets containing demographic

statistics illustrative of the first project, “Mapping

Demographics of Queens Ethnic and Language Groups from

Public Statistics.” Sources of data for this project are the 1990

U.S. Census, New York City Board of Education, and U.S.

Immigration and Naturalization Service.

For the second project, “Mapping Demographics of Registered

Library Customers,” which is in progress, they will geocode

800,000 records and overlay a table with Queens Borough

census maps. This project helps librarians determine the

percentage of the population registered, average number of

transactions per person, gender compared to general population,

language, occupational characteristics, and education. Strong

emphasized that they were not as concerned with economic

characteristics. Based on the Queens Borough experience,

economics and social situation do not have much of an impact

on whether people use the library. Their most important

concern: Does the library have materials, services, and

programs they want? If so, people will use the library.

The third project, “Mapping Library Operational Statistics,”

allows library administrators to address the following concerns:

operating costs at community level, discretionary funds

assigned by legislators, collections, circulation at branches,

what programs are held in what languages, what new borrowers

are registered, and incidence of complaints. Queens Borough

Public Library’s demographer aggregated 1990 Census data to

construct a profile of ethnic and language characteristics for

each service area constituency. The language data came from

Board of Education survey.
Page 16 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

Some of the specific ways these data are used to plan library

services, collections, and budgets include:

• Identifying languages spoken to create an international

resource center in one of libraries and to display different

languages on computer screens for Internet access;

• Altering collection development priorities and budgets

based on concentration of where languages are located in

the geographic area (materials budget is $9.2 million); and

• Linking circulation statistics with budget (e.g., Asian

language materials circulate more than any other language)

so the library can reallocate its budget accordingly.

Strong concluded with some examples of how this project

influenced them to change the way they were doing business.

They now hold community meetings to discuss expectations of

library services; they no longer make assumptions about what

people in the community want.

LSTA Connection to Library Services to Children

Mary Somerville and Carol Rasco

Mary Somerville, Past President, American Library

Association, discussed the importance of measuring the impact

of library services on children. She recommended that the

library community measure the impact of acculturation on the

lives of children and families and study the correlation between

reading, library use, and SAT scores.

Carol Rasco, from the U.S. Department of Education,

provided an overview of the America Reads! Challenge,

proposed and supported by the Clinton Administration.

America Reads is a challenge issued by the President to each

citizen in the United States to see that all children read well and

independently by the end of the third grade. The
Library and Information Services Page 17

Policy Forum Proceedings

September 15-16, 1997

Administration has submitted legislation to Congress for two

components of this program: to examine gaps in early

childhood education programs and the role of parents; and to

provide more extended learning time and train tutors to work

with children.

Rasco informed the committee about the Gorton Amendment

passed by the Senate in November 1997. The effects of this

amendment, according to Rasco, are to take federal funding

(including a substantial amount of Title I funds) from K-3 and,

with a few exceptions, put it into a block grant directly to the

local education agency:

• Local districts must use funds as they deem appropriate.

• One possible implication is that money that currently goes

to districts with higher concentrations of bilingual children

will be spread in a block grant across the United States.

There will be no special consideration for districts like Los

Angeles and Miami.

• Wealthy school districts with greater parental involvement

will receive much of the funding previously targeted for

children who have the greatest need.

• Half of the Vocational-Rehabilitation money currently

spent at the federal level is put into block grants.

[Update: This Amendment was not part of the final

appropriations bill.]

The Web site for America Reads! Challenge is www.ed.gov.

Click on America Reads Challenge.
Page 18 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

Using Qualitative Information in a Quantitative

Environment

David Penniman, Douglas Zweizig, and Joey Rodger

David Penniman from the University of Tennessee discussed

ways to use anecdotal information (stories) so they are collected

objectively and scientifically. Statistical requirements involve a

different approach to collecting anecdotal information in order

to satisfy statistical needs. The approach includes:

• Analyzing stories to make projections;

• Systematically sampling and reporting data by collecting

anecdotes to extrapolate and report them as comprehensive

through a random or stratified random sampling technique

that also allows for incidents with no consequence (that’s

OK); and

• Conservatively adjusting for anecdotes that are anomalous

(e.g., throw out extreme responses).

In reporting anecdotal information, Penniman discussed using a

critical incident technique or model. He defined a critical

incident as an observable activity complete enough in itself to

make inferences about the system that one is observing. The

event is critical if it occurs in a situation where the purpose and

intent are clear. The steps in collecting and reporting critical

incidents are to determine the objectives, develop a systematic

plan for collection, and collect, analyze, and interpret data.

(Refer to appendix D for copies of the overheads.)

Douglas Zweizig from the University of Wisconsin presented

a brief overview of the “Tell It” Method, an evaluation model

developed under the sponsorship of the U.S. Department of

Education, Office of Educational Research and Improvement,

Library Programs. Zweizig introduced the model by explaining

that it is light on planning because librarians need to spend their

time implementing services, not collecting data. In
Library and Information Services Page 19

Policy Forum Proceedings

September 15-16, 1997

summarizing the approach, Zweizig indicated that the method

supports purposeful data collection and focuses on the need for

a clear vision in the evaluation process. Steps of the approach

include (1) talk about the vision, (2) explore alternatives, (3)

design your approach, (4) learn from what’s happening, (5) let

people know what happened, (7) integrate results with ongoing

services, and (8) think about how it all worked. The book also

includes a section on training others and a chapter on data

collection methods. (Zweizig, D., et al. 1996. The TELL IT!

Manual: the Complete Program for Evaluating Library

Performance. Chicago: American Library Association.)

Joey Rodger from the Urban Libraries Council concluded

this section with a report on current projects that evaluate the

impact of public library services. Rodger reported on two

programs that measure the impact of education on preschoolers.

Both projects presented a new model for library services that

transforms the library from a deliverer of direct services to a

facilitator of services delivered to the children (train-the-trainer

model). The major impact of this model is to leverage staff

time and to positively affect the lives of more children. In the

Free Library of Philadelphia’s project, library staff teach

daycare workers to work more effectively with children and

books. The evaluation, conducted by Susan Newman from

Temple University, is based on prereading literacy skills. The

second project, funded by the U.S. Department of Education,

was awarded to the St. Louis Public Library (Glenn and Leslie

Holt are the contacts).

Importance of LSTA Reporting

for National Understanding

Carol Henderson

Carol Henderson, Executive Director, Washington Office of

ALA, described why reporting and evaluation are so important

to the LSTA. Henderson indicated that one of the primary

purposes for reporting and evaluation for LSTA is for
Page 20 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

continued funding and authorization. She provided an

overview of the history and significance of LSTA, and

identified lessons learned from the Library Services and

Construction Act (LSCA). She concluded with a discussion of

the challenges of collecting evaluative data for Congress and

for the people served by these programs. (The full text of

Carol Henderson’s speech can be found in appendix E.)

LSTA and the Need for Thorough

Data Collection on Libraries

Library statistics, if not thoroughly reported, can be

misinterpreted. For example, data collected by John Bertot in

his 1997 National Survey of U.S. Public Libraries and the

Internet indicate that although 60 percent of public libraries

offer World Wide Web access, only 9 percent of public libraries

have Web access throughout their systems. However, if only

the 60 percent figure had been reported, without the qualifying

data regarding system-wide access, readers could have been

misled about the availability of Web access. By showing both

figures, Bertot's findings suggest that libraries need discounted

rates or LSTA funds to provide Web access at all library

outlets, not just at central libraries or selected branches.

Current Challenges under LSTA

Henderson indicated that in a changing federal climate of

GPRA, Congress is not interested in funding programs simply

because they were funded last year. They want each funding

agency to answer the question, “Why should this program

continue to be funded?” The special challenges of producing

evaluative data that are realistic and meaningful include the

following:
Library and Information Services Page 21

Policy Forum Proceedings

September 15-16, 1997

• Traditionally, it has been difficult to quantify results of a

public good.

• Library users are voluntary rather than a captive audience

which makes it difficult to survey them and follow up on

nonresponses.

• Librarians generally inquire into user’s motives only

enough to assist them in their searches, not, for example, to

ask them if their search was successful.

• The impact of using the library is not immediately apparent

to the user. Often users do not know how useful the

information is until after they have completed the project or

task.

• LSTA is but one small funding stream among larger

sources of support. This begs the question about whether

new users are served better if the program is supported

with: (1) federal funds alone, (2) state funds combined

with federal funds, or (3) either of the first two in

combination with local matching funds. These issues make

it difficult to understand the impact of LSTA funds alone

on services.

Conclusion

Henderson’s final comments focused on balancing national,

state, and local needs for data collection. She indicated that

there is flexibility in LSTA to adapt the ACT’s purposes to

local needs: Henderson underscored the need for the library

community to have a handful of nationwide indicators of LSTA

effectiveness. Comparability and consistency of these

indicators must be balanced with the changing role of libraries.

Henderson challenged the participants to be careful what is

measured, and whether it goes far enough in supporting the

impact of the funding.
Page 22 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

Henderson concluded that we must have national-level

reporting and evaluation, carefully planned and measured

judiciously since future funding depends on it.

Comments and Questions from Participants

This presentation by Henderson inspired a lively discussion.

The questions and answers and comments are summarized

below.

Question #1: How does the LSTA define underserved?

Answer from Henderson: LSCA defined underserved by

characteristics such as: difficulty in using libraries, people with

disabilities, indicators of services to the elderly, people for

whom English is a second language. The LSTA does not

provide a laundry list of definitions or characteristics. Instead,

the Act provides the flexibility for states to address those needs

by giving them wide latitude to design services in a variety of

ways, such as using electronic services to reach rural

communities.

Participant Comments on Definitions

There might be a problem in defining who is underserved

nationally and who is underserved in the state. Consider

creating an evaluation template that states can adapt locally.

We need more than just state definitions of the underserved.

Henderson’s comments:

We need meaningful reporting on what has happened as a result

of improved services to the underserved, regardless of how it is

defined. Technology can help serve the underserved. We also

need to focus on evaluating the service itself rather than

identifying the subgroups that use it.
Library and Information Services Page 23

Policy Forum Proceedings

September 15-16, 1997

Question #2: What about thinking about the underserved this

way: all people have access to a specific set of services so we

should report on the services as a whole. Can we consider

identifying national program effectiveness and index successful

programs (as determined through evaluation) by congressional

district? Would a national document that defines access for all

people for these services work?

Answer from Henderson: How are you showing progress and

how are you continuing to show progress over time? We

cannot just involve new users; we must think about

improvements for all users.

Question from Henderson directed to the participants: How

do we provide better tracking of exact amounts we are spending

to support and subsidize access, such as telecommunications

and hardware costs?

Answer: Maybe these costs can only be identified or collected

at the state level through the state library agency, not by the

local library receiving the services. Perhaps we can capture

costs by sampling state library agencies.

Final Participant Comments

We should be reminded that improving access to electronic

services is one of many ways to improve services to the underserved.

Technology is not always the answer to serving the underserved. For example, screen displays with graphics are

problems for the visually impaired.

In response to a question about whether LSTA encourages

participation from all types of libraries (e.g., academic, public,

school), Henderson replied that there was an explicit

recognition that Congress was encouraging that.
Page 24 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

Telecommunication Rates—Update and Impact

Linda Roberts and Martha Hogerty

Linda Roberts, Director, Office of Educational Technology

at the Department of Education, updated the participants on

the new Telecommunications Act of 1996 and its impact.

Roberts summarized her message to the audience: the

education, library, and museum communities are all in this

world of education together. Our biggest challenge, she

indicated, is to bring the world of information (books and online resources) into the hearts and minds of every person in this

country.

In her update, Roberts indicated that the purpose of the Act is to

make access to telecommunication services affordable for every

community in America through the E-rate, a universal service

funds subsidy. The Act affects entire communities by helping

them build an infrastructure for services and use the

infrastructure to gain access to the content and services they

need.

Roberts discussed the guiding principles from which the

Department of Education worked in giving advice to the

Federal Communications Commission (FCC) about how the

Act should be implemented. The first principle was that there

would be a range of discounts, from 20 to 90 percent. The

second principle recognized the need to educate schools,

communities, and states about competitive bidding. The third

principle is that the E-rate must make a difference. Roberts

noted the education and library communities must be clear

about the goals of the program and the benefits to learners of all

ages (formal and informal, and lifelong learning), including

distance learning opportunities, reading, etc.

In order to reflect the interests of all entities affected by the Act,

the Department of Education consulted with librarians, cable

companies, school districts, state education agencies, and

intermediate service companies. They learned that one size
Library and Information Services Page 25

Policy Forum Proceedings

September 15-16, 1997

does not fit all. Therefore, the worse mistake they could have

made was to come up with a template that would apply to

everyone. Roberts identified two of the challenges inherent in

implementation: (1) different ways to consider populations in

need, and (2) who gets the services—individual school districts

or combinations of school districts; individual schools or

individual libraries.

Roberts concluded her presentation by sharing an experience

that illustrates technology is for everyone, not just for the

“enlightened.” On a recent trip to Seattle, Roberts visited a

public library that was located in the High Point Public Housing

Project. The librarian, Erica Sternin, was the expert on

technology, as well as teacher, social worker, and community

advisor. The library is the center of the community for children

and for their parents and is in full use during operating hours.

Young and old use the technology and sometimes the library

becomes so full that it becomes necessary to lock the door.

Through a grant from Microsoft, the whole library also has five

multi-media computers, typically surrounded and used by

several children at one time. They were reading, writing, and,

most importantly, learning and getting ready for their futures.

The library offers an oasis from the surrounding community and

an opportunity for children to learn in a safe, nurturing

environment.

In order to see a copy of the Working Group report on how the

Telecommunications Act should be implemented, see ED,

Office of Educational Technology Web site: www.ed.gov/tech.

One participant asked whether Roberts thought unfiltered

access to the Internet for children was a threat to the E-rate.

Roberts responded that the Internet was not yet a threat to the

E-rate, but it could become an issue with some of the members

of Congress. Most schools do provide some filtered access.

Martha Hogerty, Public Counsel, Jefferson City, Missouri,

is an advocate for the public before the Missouri Utilities
Page 26 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

Commission and the courts. Hogerty discussed the specifics of

the Telecommunications Act including governance and board

structure. From her position as an advocate, Hogerty ensures

that consumers are treated fairly and they have just and

reasonable rates.

Hogerty summarized some of the main provisions of the

Telecommunications Act. The Act permits competition in local

telephone monopoly, which eventually will help create a

telecommunications network throughout the country. Second,

the Act codifies universal service and promises consumers

lower rates and better services. Third, the Act permits

assistance to high cost areas and low-income consumers.

Finally, the Act includes a $2.25 billion annual program for

schools and libraries to purchase telecommunication services,

Internet access, and internal connections at discounts from 20 to

90 percent.

Structure for Implementation

The FCC determined that the National Exchange Carriers

Association (NECA), which currently administers universal

service funds, will be the temporary administrator of the

universal service funds. An advisory committee will

recommend that a neutral third-party administration be

established on a permanent basis. Goals of the program are to

be: (1) technologically neutral, (2) competitively neutral, and

(3) efficient, i.e., satisfy the need for fiscal responsibility

without micromanaging schools and libraries.

The first meeting of the Universal Service Administrative

Corporation (USAC) board (board of directors of 17

representatives including three school members, one library

representative, one rural health care member, one low-income

member, one state regulator, and one consumer advocate) was

September 22, at which time the board named the service

member representative for the schools and library corporation.
Library and Information Services Page 27

Policy Forum Proceedings

September 15-16, 1997

This board of schools and libraries is responsible for

administration, Web sites, outreach, reviewing bills, doing

audits.

Hogerty provided a recent update on implementation activities.

The FCC issued notice seeking comments as to whether firstcome-first-served approach to reviewing applications for the

rate subsidy should be altered or if there should be a window of

opportunity where everyone is on equal footing. In addition, the

FCC asked for comments on aggregation proposals that were

due September 25. Hogerty concluded her comments by stating

that the goal is to set up a system so consumers can receive best

rates that will work for the whole. Everyone involved wants

this program to be a success.

Participant Questions

Question: What is the status of the Southwestern Bell case?

Answer from Hogerty: Southwestern Bell challenged the

Telecommunications Act in federal district court in Texas on

constitutional grounds. They are currently waiting for a

decision from the court.

Question: What has to be included over next three months and

presented to the library community in order for this program to

begin on Jan. 1? What odds are you giving that it will actually

start on Jan. 1?

Answer from Hogerty: The boards have been set up, which is a

hopeful sign. FCC and others are committed to getting things

running on Jan. 1.

Question: What group is in charge of producing the final

application?

Answer from Hogerty: The board under the USAC board

(school and library board) will produce the application.
Page 28 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

SMALL GROUP REPORTS

Final Reports and Recommendations from the

Small Group Discussions

Each of the four groups discussed and made recommendations

for a specific set of LSTA objectives.

Group 1: LSTA Objective—Establish or enhance electronic

linkages among and between libraries; link libraries

electronically with education, social, or information services;

encourage libraries in different areas, and encourage

different types of libraries to establish consortia and share

resources.

Tom Sloan, Chair; Gerry Rowland, Recorder

Group 1 recommended a framework for identifying national

and state goals that are responsive to the LSTA and to

Congress. The specific framework and examples are provided

below.

Data Collection Needs: Collecting National

and State Data

Group 1 discussed the need for reporting and collecting both

national and state data. The group concluded that there was a

need to establish national goals developed by the library

community (e.g., professional library associations, etc.). Each

state will then develop measurable indicators based on these

national goals. National goals are voluntary, like Goals 2000,

but give the library community leverage with Congress and

satisfy requirements of LSTA. States can then demonstrate

how their indicators meet national goals, or, based on state
Library and Information Services Page 29

Policy Forum Proceedings

September 15-16, 1997

plans, identify additional needs or priorities not included in the

national goals.

Examples of national goals and state indicators are provided

that relate specifically to LSTA objectives.

Example #1:

National Goal: Residents will have graphical access to the

Internet through their local library. State Indicators might

include:

1. Number of graphical terminals

2. Number of graphical terminals per capita

3. Training of staff and customers

Example #2:

National Goal: Each resident will have access to holdings and

resources to all libraries in the state. State Indicators might

include:

1. All on union list of holdings

2. All on OCLC

3. Web-based union catalog

4. Z39.50 for all systems

5. Digitized content of unique materials

6. Training

Example #3:

National Goal: Each resident has access to educational, social,

or information services and resources through the local library.

State Indicators might include:

1. Library is a community information center (paper or

electronic linkage)
Page 30 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

2. Hosting Web sites for other communication organizations

3. Linking to local Web sites

4. Providing transactional level services for others

Example #4:

National Goal: All libraries share resources to the benefit of

library customers. State Indicators might include:

1. Electronic interlibrary loan

2. Sharing networks; infrastructure

3. Integrated Library Systems

4. Union List of Serials

5. Cooperative cataloging/acquisitions

6. Consortia access to commercial databases

7. Training

Additional Comments from Group 1:

1. State agencies should involve state data coordinators in

evaluating LSTA.

2. State plans should include measurable objectives.

3. Sources for evaluating programs include “Tell It”; NCES

publications, Public Libraries Association data book, state

and local reports (demographic and economic reports),

state data centers, state demographer; private contractors,

library school students; local educational institutions;

online and end user surveys, staff, etc.
Library and Information Services Page 31

Policy Forum Proceedings

September 15-16, 1997

Group 2: LSTA Objective—Assisting libraries in accessing

information through electronic networks and paying cost for

libraries to acquire or share computer systems and

telecommunications technologies.

Rod Wagner, Chair; Keith Lance, Recorder

Group 2 identified six possible studies, two measurement

methodologies, and two practical suggestions.

1. Develop studies through IMLS on the most common uses

of LSTA funds.

2. Study of value, effects, efficiencies, impact, etc., of the

leveraging of LSTA funding.

3. Study effects of electronic access to information on

interlibrary loan and other services.

4. Develop methodologies for systematic observation, i.e.,

collecting anecdotal information, including national

sampling. Capture results in a searchable Web-accessible

format.

5. As a high priority, recommend that IMLS contract for a

study of the return on investment of a public access

terminal: (1) Internet access to free information sources,

and (2) Added value—commercial or locally provided

content.

6. Conduct parallel studies to determine the effect of

technology use in libraries on other library functions (e.g.,

the relationship between circulation, interlibrary loan, and

reading).

7. Assess the extent to which libraries engage in digitization

of electronic content and finding tools.

8. Develop through IMLS templates or criteria to collect data

and information from LSTA funded projects, including

identified data elements.
Page 32 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

9. Address concerns that the evaluation process not

overburden staff.

10. Create a Web-accessible/searchable clearinghouse to

identify studies addressing LSTA funded programs and

projects and other related research.

Group 3: LSTA Objective—Target library and information

services to people of diverse geographic, cultural, and

socioeconomic backgrounds emphasizing underserved urban

and rural communities including children (from birth

through age 17) from families with incomes below the poverty

line.

Bridget Lamont, Chair; Mary Jo Lynch, Recorder

Group 3 made recommendations in three areas: (1) philosophy,

(2) starting points, and (3) approaches.

Philosophy

1. Acknowledge the value of libraries but understand we must

determine how to communicate that value in a tangible and

meaningful manner.

2. What do we need to know?

3. What do we need to learn: (1) ways to improve practice, (2)

what happened as a result?

Starting points

1. IMLS content analysis of needs, goals, and objectives of

the 55 state plans.

2. Survival statistics organized by district funded, then by

themes.

3. Impact: people served and project outcomes.
Library and Information Services Page 33

Policy Forum Proceedings

September 15-16, 1997

Approaches

1. Rely on state library agencies to identify common tools to

evaluate programs and services.

2. States are in best position to define what underserved

means.

3. Laud effective evaluation at state library agencies and at

local level.

4. Acknowledge focus LSTA has given to evaluation.

5. Encourage training on evaluation methods at state and local

levels.

6. National evaluation of what really works in thematic areas.

Group 4: LSTA Objective—Targeting library and

information services to individuals with disabilities and people

with limited functional literacy or information skills.

James Scheppke, Chair; Susan Paznekas, Recorder

Group 4 provided specific recommendations for each area

covered in LSTA. They first described their vision of services

to these populations as equity of access to information.

The group’s definitions of these populations follow:

• Persons with disabilities: blind/visually impaired, deaf/

hard of hearing; mobility impaired/homebound; developmentally disabled; learning disabled, functionally disabled.

• Persons with limited literacy skills: English as-a-second

language; lack of functional literacy; lack of work-related

literacy skills.

• Persons with limited information skills: problems in

formulating questions, searching for information; problems
Page 34 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

with effectively analyzing the information, and applying

information.

The group’s inventory of current measurement in these areas

revealed that measures are very limited, of little relevance, or

survey libraries, not library users. Current surveys of some

relevance include NCES Public Library Survey, NCES State

Library Agencies Survey, The Library of Congress Survey of

National Library Services for the Blind Regional Libraries,

NCES Integrated Postsecondary Education Data System

(IPEDS), the National Household Education Survey (NHES),

and the National Adult Literacy Survey.

Data Collection Recommendations for

Persons with Disabilities

1. National survey of persons with disabilities in order to

assess needs and barriers to their use of library services

(possibly by including questions on the NHES).

2. National sample survey of libraries to assess level of

services provided to persons with disabilities (possibly

through NCES Fast Response Survey).

3. Use national survey methodologies to conduct surveys at

the state level on this issue (possibly by including questions

on the State Library Agencies Survey).
Library and Information Services Page 35

Policy Forum Proceedings

September 15-16, 1997

Data Collection Recommendations for Persons with

Limited Skills Literacy

1. Conduct the NCES National Adult Literacy Survey again

incorporating library use questions.

2. Conduct NCES surveys at the state level (state library

agencies and partners).

3. Followup State Library Agencies Surveys with state level

assessments of library services, including adult

literacy/family literacy services and services to

preschoolers.

4. Analyze existing NCES National Adult Literacy survey

data on library use.

Data Collection Recommendations for Persons with

Limited Information Skills

1. Assess information competencies of library users.

2. Add questions to the NHES about information behaviors.

3. Assess how information provided by libraries affects

people’s lives (use anecdotal information collected by

focus groups).

General Recommendations

1. Support use of template for state reports to IMLS on

LSTA-funded projects; states would report exemplary

projects in these areas and develop template.

2. Conduct thorough inventory of existing measurement

activities that might have relevance to use of libraries by

these populations.
Page 36 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

Questions and Comments on Small Group Reports

Question: What happens with these recommendations?

Participant Comment: They should feed into COSLA

evaluation committee.

Participant Comment: Look at all possible funding

mechanisms, not just one source such as IMLS.

Participant Comment: COSLA needs to share their document

with ALA, Forum, etc., and with others for feedback.

Closing Comments: One participant suggested the group

consider taking out the word “national,” and simply call them

LSTA goals approved by the library profession. A Forum

planning committee member commented that these

recommendations will inform our national surveys and that

excellent progress has been made.
Library and Information Services Page 37

Policy Forum Proceedings

September 15-16, 1997

Appendix A

A STRONG WORKING PARTNERSHIP TOWARD

QUALITY DATA: NCES AND THE LIBRARY

COMMUNITIES

Pascal D. Forgione, Jr., Ph.D.

Commissioner of Education Statistics

National Center for Education Statistics

U.S. Department of Education

555 New Jersey Avenue, N.W.

Washington, D.C. 20208

Telephone: 202-219-1828

Fax: 202-219-1736

Email: pascal_forgione@ed.gov

http://www.ed.gov.NCES/

September 15, 1997

I am very pleased to be with you this morning and to welcome

you to the fifth annual Library and Information Services Forum.

At the outset, I would like to thank the planning committee that

developed the structure and content that brought this forum into

focus. This was a joint effort among many offices and

organizations including NCES, NCLIS, the Postsecondary and

Library Institute in OERI, the Institute of Museum and Library

Services, COSLA, and State Data Coordinators for the public

library survey. It was coordinated and brought to life by a

gentleman who has been doing this sort of thing for many years,

Mr. John Lorenz. John, congratulations on a job well done.

In the five years that the Forum has been jointly sponsored by

the Center and the National Commission on Libraries and

Information Sciences, the Forum has become a focal point of

the Center’s Library Cooperative Program. The past four
Page 38 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

forums have focused on identifying emerging trends that were

occurring in libraries and the data that were needed to address

them. Last year’s forum, for example, was concerned with

issues surrounding the measurement of library programs that

are targeted toward populations with special needs. Two other

forums delved into data that were needed to assess the growth

in the use of technology and the provision of electronic

services. The outcomes of these previous meetings provided a

framework for the consideration of new data elements by the

various library working groups that are convened by NCES and

for consideration by the Federal State Cooperative Systems

Steering Committee for public libraries.

This year’s forum is somewhat of a departure from previous

forums. Rather than keying in on one specific issue area, it will

focus on the data implications of a major new federal-state

library and information services development initiative as

provided for by the new Library Services and Technology Act

(LSTA). The LSTA has two primary objectives: hastening the

development of and strengthening electronic linkages between

libraries, and improving information services both to persons

having difficulty using a library and to underserved urban and

rural communities. As I understand it, the LSTA is not just

concerned with public libraries, but it also affords the

opportunity for states to build networks and cooperative

agreements among all types of libraries, that is, the full range of

public libraries, academic libraries and school libraries.

To meet its objectives, the LSTA provides funds to state library

agencies. State library agencies can expend the funds either

directly or through subgrants. This money is not, ‘free’,

however. State agencies are charged with devising a plan that

describes how they will expend the funds to meet LSTA

objectives. Further, in this era of accountability, states must

decide how they will evaluate their programs to provide

evidence that the state is making progress toward meeting its

LSTA objectives.
Library and Information Services Page 39

Policy Forum Proceedings

September 15-16, 1997

Each state plan will be unique in that it will reflect specific state

interests, needs and priorities; and, the state’s evaluation of

progress mandated by the LSTA will most certainly be tied to

the state’s individual objectives as delineated in its state plan.

The urgency for assessing progress towards LSTA state grant

objectives is reinforced at a national level by the Government

Performance and Results Act (GPRA). This act makes it clear

that continued federal funding of programs such as the LSTA

will depend on evidence of progress toward meeting the

objectives of the relevant federal legislation.

The Center’s mission, on the other hand, is to collect, analyze,

and disseminate statistics and other data related to education in

the United States. Since some libraries are important

components of educational institutions (i.e., academic libraries,

and school libraries and media centers), and public and other

libraries have an educational mission as well as a service

mission, the Center has broadly interpreted its mission and has

been collecting data on the status of libraries in the United

States for many years. We have been collecting data on

academic libraries since 1966 as part of our general higher

education data collection; and we have regularly collected data

on public libraries since the formation of the Federal State

Cooperative System in 1988. In 1993 we surveyed school

libraries and media centers; recently we have surveyed state

library agencies and federal libraries; and we are currently

planning a survey of library cooperatives. We have also looked

at libraries from the perspective of users of libraries; that is, the

parents, other adults, and children, through the Center’s

national household survey.

We have some recent estimates of just how far public libraries

reach. From our National Household Education Survey

(NHES) conducted in 1996, NCES has found that at least one

individual in 65 percent of households in the Nation has used

one or more library services in the year prior to the survey, and

that 44 percent had used them in the last month. In addition, in
Page 40 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

those households with children, 61 percent had used at least one

library service in the past month. While most people use

libraries to borrow or return books or tapes, 18 percent reported

using them to hear a lecture or to attend a story hour, and many

reported using them to hear a lecture or to attend a story hour,

and many reported using them to get information. This

extraordinarily high use of public libraries is indicative of the

significant role they play in American life.

For all of our data collection activities, NCES works with the

appropriate communities to determine those data that are

needed at the national level for reporting on current status and

trends. Most of our data bases are developed to address longstanding issues of national concern as identified by our working

groups and cooperatives. While we do have fast response

surveys that address current ‘hot topics’, most of our surveys

are intended to collect basic information that will stand the test

of time and that has become extremely valuable as tools for

looking at changes over time.

This is not to say that our basic databases do not change. As

the phenomena we are measuring change, our surveys are

modified to reflect these changes. In doing this, however, we

do everything possible to maintain historical trends. For

example, there have been radical changes in the accounting

standards for non-profit higher education institutions. The

Center has collected finance data on these schools for the past

20 years under the old accounting standards model. Now,

however, recognizing that these new standards will govern

finance reporting for many years to come, we are completely

revising our finance survey for non-profit postsecondary

institutions to conform to these new standards. At the same

time, the Center is funding a study to develop a bridge between

the old finance data and the new finance data to ensure that we

can continue to report on changes in institution finances over

time.
Library and Information Services Page 41

Policy Forum Proceedings

September 15-16, 1997

Given our charge for providing consistent and reliable data over

time, the national data that NCES collects cannot, and should

not, be driven by solely program evaluation needs. As you are

well aware, legislative programs are somewhat transitory in

nature. What is important in one legislative session or in one

administration is not necessarily a priority as political winds

change. Thus, as exciting and significant as the LSTA is to the

entire library community, we must be very careful not to fall

into the abyss of substituting basic statistics on libraries and

school media centers for program-specific data.

I am delighted to report, however, that all is not lost. As Mary

Jo Lynch points out in her paper that you received as part of

your materials for this meeting, some of the data the Center

already collects on public libraries, state libraries, and academic

libraries are quite relevant to the objectives of the LSTA.

Several items that were added to the 1997 public library survey,

for instance, are concerned with electronic services provided by

libraries. Last week Tom Sloan of Delaware, with the support

of NCES and NCLIS, organized a special meeting of the data

definitions subcommittee of the (Federal State Cooperative

System). The purpose of this meeting was to identify additional

data items on electronic services and outcomes that might be

added to the public library survey. I am sure you will hear more

about these matters in the next two days. Since the public

library survey is a universe survey, data collected through this

survey are available at a state level and will prove very useful in

establishing baselines against which progress can be measured.

It is a happy coincidence that NCES’ library statistics program

collect information from public libraries, academic libraries,

and school libraries and media centers, all the types of libraries

mentioned in the LSTA. It is also fortunate that the public

library and academic library collections are universe surveys

and are done sufficiently often to provide good measures of

change as the LSTA is implemented. In the case of the school

library and media center survey, it is a sample survey, but the
Page 42 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

sample is a state-level sample so state estimates are possible. It

is also done on a periodic basis, about once every 5 years which

coincides quite well with the 5 year cycle of the LSTA. We

plan the next school library media center survey for 1999.

Like the public library survey, each of the other library surveys

in NCES’ portfolio has begun to include items on the

availability of technology in the library, the electronic services

offered by libraries, and, in a limited way, information on the

extent of usage of these services. These items are beginning to

appear on these surveys because you, the library community,

has recognized the emerging importance of technology to the

way the libraries do business, a phenomenon that is likely to

grow in importance as technology advances.

Further, the survey of library cooperatives that NCES has been

planning since 1994 will provide a key national and state

indicator of progress toward meeting the LSTA objective of

linking libraries by monitoring changes both in the number and

types of library cooperatives and in the number and types of the

libraries they serve.

Identifying individual data elements is not the only tool

available for looking at progress in meeting LSTA objectives.

Analyzing the data that are collected on libraries in conjunction

with other data sets is another strategy for developing national

and state indicators. NCES, for example, is supporting a study

to map public libraries service areas and to geocode library

buildings. Once this activity is completed, a merged data file

containing the mapping data, the public library survey data and

Census data will be created. This file will allow analysts to

relate the economic and demographic characteristics of the

population served by a library to the characteristics and service

measures of that library. To the extent that Census data contain

the population characteristics that are relevant to the LSTA, it

will be possible to assess how services to the populations

mentioned in the act change as the LSTA is implemented. This

type of analysis, while of interest at the national level, will be
Library and Information Services Page 43

Policy Forum Proceedings

September 15-16, 1997

invaluable to the states since states will know the level of LSTA

funding received by each library in the state and how the

observed changes fit into the state plan.

Additionally, the Census Bureau, at the behest of NCES, has

already added an SMSA code the public library survey data file

so services provided by libraries in central cities, suburban

areas, and rural locations can be studied.

Libraries in the United States are on the edge of a revolution

both in the types of services they provide and in the way these

services are provided. I suspect that the LSTA will help

libraries move more quickly into an electronic future. Your

challenge over the next two days is to chart out strategies for

capturing this movement through our national data collections

and through creative uses of the these data. We at NCES look

forward to working cooperatively with you on this exciting and

important undertaking.
Page 44 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997
Library and Information Services Page 45

Policy Forum Proceedings

September 15-16, 1997

Appendix B

ASSESSING THE PROGRESS TOWARDS LSTA’S

PURPOSES: HOW THE NCES LIBRARY

SURVEYS CAN HELP

Mary Jo Lynch

Director, Office for Research Statistics

American Library Association

NCES/NCLLIS Policy Forum September 15-16, 1997

I. Introduction

The Library Services and Technology Act (LSTA) begins a

new era in federal funding for library services in this country.

Funds now go to states for use on behalf of all libraries and are

to be used on any or all of six purposes according to a plan

devised by each state. Sharing resources and using networked

information are strongly encouraged. At the same time, there is

a strong focus on assessment of progress, largely due to the

Government Performance and Results Act (GPRA) which takes

effect in FY98. State library agencies must not only change

from the purposes and procedures of the state grant program of

the Library Services and Construction Act (LSCA) to those of

LSTA but must also figure out how to assess progress towards

the LSTA state grant objectives included in their state plans.

How can this be done in a way that respects state sovereignty,

makes sense nationally, and takes advantage of existing systems

for collecting data? This paper describes how to do that by

using the library surveys already in place at the National Center

for Education Statistics (NCES). I’ll be the first to admit that

these surveys won’t do it all -- at least with the current content

and existing time constraints. But they provide a solid base to

build on and change is always possible.
Page 46 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

It is important to remember that NCES does not do these

surveys in a vacuum. For each one there is an advisory

committee of librarians and NCES has a strong commitment to

providing data for policy purposes. An important result of the

Forum could be the recommendation of change in one or more

of the NCES library surveys. The rest of the paper describes

those existing surveys and how they relate to LSTA state grant

objectives. Attached to this paper are several useful

documents:

1. Section 231 of LSTA which lists objectives of the grants to

states.

2. Abstract of study by Christine Koontz funded by OERI for

$422,559

3. FSCS data items 44-49.

4. IPEDS Academic Libraries Survey, Part G, Electronic

Services, 1996.

5. Table 6-1 from The Condition of Education, 1997.

Also useful are several other enclosures sent to Forum

Participants:

• An eight page pamphlet describing the six library surveys

in the NCES Library Statistics Cooperative Program.1

• An article by Mark Smith and Gerry Rowland (“To Boldly

Go”) that describes work on the measurement of electronic

services by the Federal-State Cooperative System for

Public Library Data (FSCS).2

 

1 Library Statistics Cooperative Program. NCES 97-407. Prepared for the

National Center for Education Statistics by the U.S. National Commission on

Libraries and Information Science. Washington, DC: U.S. Department of

Education, 1997.

2 Smith, Mark and Gerry Rowland. “To Boldly Go: Searching for Output

Measures for Electronic Services.” Public Libraries. May/June 1997:168-172.
Library and Information Services Page 47

Policy Forum Proceedings

September 15-16, 1997

II. Federal-State Cooperative System for Public

Library Data (FSCS)

Several of the NCES surveys described in the pamphlet on the

Library Statistics Cooperative Program will be mentioned in

this paper but the one that merits attention first is the public

libraries survey popularly known as FSCS. Although the

brochure shows it as one of six, it is really the most important

for our purposes. FSCS is:

• collected electronically (diskette, FTP, WWW).

• heavily based in the states.

• highly dependent on the work of 50 state data coordinators.

• annual.

• already working on measuring electronic services.

• already working on analysis of community characteristics.

Unlike any other NCES library survey, the data for FSCS are

collected by the states using their own forms -- not forms

created by NCES. The state forms may ask for other items, as

needed by an individual state, but they all contain a core set of

items -- 49 at this writing --- agreed upon by the states through

the 50 state data coordinators (SDCs). Those 50 meet annually

to discuss and improve FSCS and elect five members to a

steering committee that involves NCES, the National

Commission on Libraries and Information Services (NCLIS),

and others but is always chaired by an SDC. One of the major

functions of the Steering Committee and the annual meeting is

to consider changes, deletions, or additions to the list of items

all states will collect from public libraries. The article by Smith

and Rowland describes work in progress to identify measures

for electronic services. That work will eventually result in ways

to measure progress toward at least some of objectives 1A-1E

of the LSTA grants to states. Before discussing data for those
Page 48 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

objectives further, let us turn briefly to objective 2, where FSCS

also has something to offer.

III. Targeting Services to Specific Groups

Objective 2 involves targeting services to a number of different

groups of people. Although some of those groups will be hard

to define, many of them have characteristics regularly counted

by the Census of Population. Linking census data to library

service areas can now be accomplished using the software

known as Geographic Information Systems (GIS). This has

been done successfully by individual libraries in making

decisions about branch locations3and is now being used in 93

cities by Dr. Christine Koontz in work sponsored by the OERI

Institute on Postsecondary Education, Libraries, and Lifelong

Learning. Attachment 2 describes this project. Koontz used

the FSCS file of library outlets in the early stages of her work.

Key to the use of GIS is “geocoding” the library location. It is

relatively easy to geocode the address of a single library outlet

but harder to define the market area around that address. The

Koontz project will solve that problem for outlets in her study.

The FSCS outlet file has also been used by Koontz’s colleague,

Dean Jue, to help ALA’s Office for Information Technology

Policy (OITP) identify library outlets serving low income

people in connection with ALA’s testimony to the FCC on

implementing universal service discounts. For the OITP

project, Dean Jue and Andrew Magpantay (OITP Director) are

experimenting with different ways of determining which libaray

outlets in the nation serve areas of high poverty and are

therefore entitled to special discounts. The FSCS Metropolitan

State Code -- “central city,” “metropolitan area - not central

city,” “not metropolitan area” -- has been a key factor in that

work.

 

3 Ottensmann, John R. “Using Geographic Information Systems to Analyze

Library Utilization.” Library Quarterly. January 1997:24-49.
Library and Information Services Page 49

Policy Forum Proceedings

September 15-16, 1997

Both of the projects just described are based on library outlets

(the 8,879 centrals and 7,025 branches as of the 1994 FSCS

report4). But FSCS gathers those 49 data items from 8,921

public library “administrative entities” where the appropriate

geographic factor is not address but legal service area -- defined

as “the geographic area for which a public library has been

established to offer services and from which (or on behalf of

which) the library derives income, plus any areas served under

contract for which the library is the primary service provider.”

Recently the FSCS Steering Committee decided to enhance the

usefulness of the FSCS data file by providing geocodes for each

of the 8,921 public library “administrative entities,” 20% of

which have multiple outlets and many of which include nonstandard political jurisdictions in their “legal service area”.

This would enable anyone with appropriate software and skill

to produce maps showing demographic characteristics of the

people potentially or actually served by a public library

administrative entity. NCES recently contracted with Westat to

provide those codes. That project is well underway at this

writing and is on the Forum agenda. Once this project is

completed, the demographics of public library use can be

analyzed by legas service area nationwide. All the tools will be

available: GIS software, FSCS boundary and data files and

Census data files.

IV. Sharing

It seems to me that the first five objectives of LSTA grants to

states are highly interrelated and that the relationships focus on

two themes:

• electronic linkages (mentioned in 1A-1C and implied in

1E).

 

4 Public Libraries in the United States: 1994. NCES 96-213. Washington, D.C.:

U.S. Department of Education, 1997.
Page 50 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

• sharing (mentioned in 1D and 1E -- implied in 1A).

How can NCES surveys help to measure the extent to which

libraries share and the extent to which they offer electronic

services? Sharing is something libraries have been doing for a

long time. The most widely known manifestation of that

sharing is interlibrary loan -- a system which began long before

librarians began using computers. The annual figures for

interlibrary loan in public libraries collected by FSCS and the

biennial figures for interlibrary loan in academic libraries

collected by the Integrated Postsecondary Education Data

System (IPEDS) show increases in the electronic era. Data on

interlibrary loan is already a regular component of NCES

library surveys and will no doubt be continued. But other

dimensions of sharing are harder to measure. The pamphlet on

the Library Statistics Cooperative Program notes that a survey

of library cooperatives is in progress now. The Advisory

Committee for that survey meets the week before the Forum to

consider results of the pretest of the survey instrument. LSTA

will be a factor as the Advisory Committee makes

recommendations for the final questionnaire.

NCES did earlier surveys of the topic in 1978 and in 1986. For

those surveys, as for the current attempt, a major problem is

establishing a valid and reliable universe file, i.e. a list of

existing consortia/cooperatives/networks that meet a common

definition. Once completed, however, the current survey will

be a useful benchmark especially for objective 1D. The current

schedule calls for repetition every 5 years but it may need to be

done more often.

The annual State Library Agency survey is likely to be even

more useful in measuring the extent of electronic sharing.

Three tables in the latest report5look important in that regard:

 

5 State Library Agencies, Fiscal Year 1995. NCES 97-434. Washington, D.C.:

U.S. Department of Education, 1997.
Library and Information Services Page 51

Policy Forum Proceedings

September 15-16, 1997

• Table 3, “Electronic network functions supported at the

state level by state library agencies. . .” shows yes or no

responses by state to the following nine items:

Electronic network

functions supported at

the state level

• Electronic network

planning or monitoring

• Electronic network

operation

Database development

• Bibliographic data

bases

• Full text or data files

Library access to

the Internet

• Training or consultation for

participation

• Subsidy for participation

• Providing equipment

• Mounting directories, data

bases, or online catalogs

• Managing gophers, file

servers, bulletin boards, or

listservs

Currently all states report “Yes” for the first item and many

more report “Yes” than “No” on the other items. Clearly state

library agencies are already active in this arena. In future years,

Table 3 can be used to monitor these important functions -- or

changed to include others as the technology for sharing

changes.

• Table 12c, “Number and percentage distribution of

budgeted full-time equivalent (FTE) positions in

automation/electronic network development in state library

agencies,” shows for each state the number of “Librarians

with ALA-MLS,” “Other Professionals,” and “Other paid

staff” involved in those activities. This may need to be

changed with the new focus of LSTA.
Page 52 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

• Table 25d, which has covered LSCA Title III expenditures

(library cooperation) will obviously have to be changed

along with 25a, b, and c which covered other aspects of

LSCA. The state library agency survey has now been

conducted for three years (1994, 95, 96) when LSCA was

in place. The three reports provide a solid base from which

to monitor the changes that will occur in state library

agencies because of LSTA. The Advisory Committee for

the State Library Agencies survey meets the week before

the September Forum and we can expect a report on their

work along those lines.

V. Electronic Services

The state library agency survey captures the statewide

dimensions of electronic services but how these services reach

citizens can be seen best in the surveys of public and academic

and school libraries. As was indicated earlier, the FSCS is

already working on measurement of electronic service delivery.

The article by Smith and Rowland indicates that this is not easy

and is far from finished. The FSCS Steering Committee

struggled mightily before adding the five items shown on

Attachment 3 to the set of items states were asked to collect

from public libraries and report to NCES. The first reports that

include these items are due in NCES by the end of August

1997. Already, experience with those data items indicates that

there may be a need to clarify the definitions. Part of the

problem is that the same database may be available several

different ways as described in Carol Tenopir’s first annual

survey of “The Data Dealers.”6The Steering Committee is

working on those data items and also considering the additional

measures as described by Smith and Rowland. A special

meeting of the FSCS definitions sub-committee will take place

 

6 Tenopir, Carol and Jeff Barry. “The Data Dealers.” Library Journal. May 15,

1997:28-36.
Library and Information Services Page 53

Policy Forum Proceedings

September 15-16, 1997

a week before the September Forum and it is hoped that a

progress report will be available then.

Once those FSCS electronic measures are part of the annual

data collection it will be possible to use some of them to assess

progress towards several LSTA objectives. Because FSCS data

are collected and compiled at the state level before they are sent

to NCES, there is no need for a state to wait for the national

report which admittedly comes out long after data are collected.

These data can be used at the state level at a much earlier date.

Because all states use the same items and definitions, reports

made to the Institute of Museum and Library Services (IMLS)

and/or to Congress will present comparable data. There is

much work to be done but FSCS is well on the way to

providing the metrics for measuring the results of LSTA for

electronic services in public libraries.

Academic Libraries

What about academic libraries -- those in two year and four

year colleges and graduate universities? The NCES survey

covering academic libraries is one of nine surveys that comprise

the Integrated Postsecondary Education Data Surveys (IPEDS).

It will be less useful than FSCS because it is:

• biennial.

• imbedded in the IPEDS -- a complex system involving state

higher education agencies and campus institutional

research officers.

• not really “owned” by the academic library community.

For those reasons and others the IPEDS Academic Libraries

Survey (ALS) will probably not be as useful as FSCS in

assessing progress toward LSTA objectives. However, there is

good news as well in that:
Page 54 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

• the FSCS State Data Coordinator in each of 21 states is

also the IPEDS Library Representative (in other states the

library representative is in an academic library or

cooperative).

• the academic library sector is way ahead of other parts of

the library community in making full use of electronic

services. A status report on this topic was published by

ALA in 1996.7

The most recent published report of the IPEDS ALS is dated

1992,8 but a 1994 report is expected by the end of 1997.

Neither has any useful data on electronic services, but the 1996

survey had a new section (G) focusing on electronic services --

see attachment 4. Several items in Section G could be used to

assess progress toward LSTA Objectives. Although NCES

collects these data only every two years, a state could easily use

Section G to collect in other years. (Note: attachment 4 is a

revision of Section G for the 1998 survey. It has not yet been

approved by OMB.)

Results of Section G will be quite different from those of the

new FSCS items on electronic services in that they measure

capacity for service, not service volume. Measuring service

volume has been discussed many times by the Advisory

Committee for the Academic Libraries Survey. For many

reasons, however, the Committee has not moved in the

direction taken by FSCS and seems unlikely to do so in the near

future.

 

7 Lynch, Mary Jo. Electronic Services in Academic Libraries. Chicago: American

Library Association, Office for Research and Statistics, 1996

8 Academic Libraries: 1992. NCES 95-031. Washington, D.C.: U.S. Department

of Education, 1994.
Library and Information Services Page 55

Policy Forum Proceedings

September 15-16, 1997

School Library Media Centers

The pamphlet on the Library Statistics Cooperative Program

shows that NCES collects data from school library media

centers every five years, but the most recent full report covers

data for 1985-86.9 A small amount of data, primarily on LMC

staffing, was collected in 1990-91 as part of a school survey

and described in a separate report.10 Results from the most

recent survey of school library media centers, done in 1993-94,

have not yet been fully reported at this writing although data on

staffing items were included in a general report on the 1993-94

Schools and Staffing Survey (SASS),

11 and results for items on

electronic services were included in The Condition of

Education 1997.

12 Table 6-1 from that volume is Attachment 5

to this paper. When full results of the 93-94 survey are

released, they will show results by states. This will be done for

public schools only as the sample was not large enough to allow

reporting for private schools. It seems unlikely that NCES data

on school library media centers will be useful in measuring

progress on LSTA objectives because the Library Media Center

survey is:

• a sample survey.

• conducted every five years.

States will need to find another way to monitor this sector of the

library community. A small committee has just been formed to

help NCES revise the School Library Media Center

 

9 Statistics of Public and Private School Library Media Centers, 1985-86 (with

historical comparisons from 1958-1985). Washington, D.C.: U.S. Department

of Education, 1987.

10 School Library Media Centers in the United States: 1990-91. NCES 94-326.

Washington, D.C.: U.S. Department of Education, 1994.

11 SASS by State: 1993-94 Schools and Staffing Survey: Selected State Results.

NCES 96-312. Washington, D.C.: U.S. Department of Education, 1994.

12 The Condition of Education, 1997. NCES 97-388. Washington, D.C.: U.S.

Department of Education, 1997.
Page 56 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

questionnaire used in 1993-94 for a survey in 1998-99. It

seems likely that the technology items will be changed.

Individual states may want to do their own surveys using this

common set of data items.

VI. Conclusion

NCES surveys are essential to the work of monitoring progress

toward LSTA objectives for at least three reasons:

1. they already collect some of what is needed.

2. they can be changed to collect more.

3. they provide a common framework for other data

collection efforts – national, regional, and state.

Because of the confluence of LSTA and GPRA, if the NCES

Library Statistics Program did not exist, we would have to

invent it. But it does exist. There is much work to be done and

we have the tools to begin.
Librarand Information Services Page 57

Policy Forum Proceedings

September 15-16, 1997

Appendix C

This article is reprinted by permission of the American Library Association.

Public Libraries, May/June, Vol. 36, No. 3, ISSN 0163-5506, 1997, pp. 168-172.
Page 58 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

This article is reprinted by permission of the American Library Association.

Public Libraries, May/June, Vol. 36, No. 3, ISSN 0163-5506, 1997, pp. 168-172.
Library and Information Services Page 59

Policy Forum Proceedings

September 15-16, 1997

This article is reprinted by permission of the American Library Association.

Public Libraries, May/June, Vol. 36, No. 3, ISSN 0163-5506, 1997, pp. 168-172.
Page 60 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

This article is reprinted by permission of the American Library Association.

Public Libraries, May/June, Vol. 36, No. 3, ISSN 0163-5506, 1997, pp. 168-172.
Library and Information Services Page 61

Policy Forum Proceedings

September 15-16, 1997

This article is reprinted by permission of the American Library Association

. Public Libraries, May/June, Vol. 36, No. 3, ISSN 0163-5506, 1997, pp. 168-172.
Page 62 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997
Library and Information Services Page 63

Policy Forum Proceedings

September 15-16, 1997

Appendix D

USING QUALITATIVE INFORMATION

IN A QUANTITATIVE ENVIRONMENT

W. David Penniman

University of Tennessee

LIS Policy Forum

September 15, 1997
Page 64 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997
Library and Information Services Page 65

Policy Forum Proceedings

September 15-16, 1997
Page 66 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997
Library and Information Services Page 67

Policy Forum Proceedings

September 15-16, 1997

Flanagan, 1954, p. 327
Page 68 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

\
Library and Information Services Page 69

Policy Forum Proceedings

September 15-16, 1997
Page 70 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997
Library and Information Services Page 71

Policy Forum Proceedings

September 15-16, 1997

Appendix E

THE IMPORTANCE OF LSTA REPORTING AND

EVALUATION FOR CONTINUED FEDERAL

FUNDING AND REAUTHORIZATION

Carol C. Henderson

Executive Director, Washington Office,

American Library Association

September 16, 1997

Presentation to the NCLIS/NCES

Library and Information Services Policy Forum

on LSTA State Grant Program: Data Needs and Implications

John Lorenz, in his wonderful diplomatic way, has given this

presentation a fine sounding title -- "Importance of LSTA

Reporting for National Understanding." But let's face it -- the

reality is much more mundane, although absolutely crucial. It's

the "Importance of LSTA Reporting and Evaluation for

Continued Federal Funding"! And a few years down the road,

it's the "Importance of LSTA Reporting and Evaluation for

Reauthorization"!

Any of you who were involved in our 4-year reauthorization

project that resulted in the new Library Services and

Technology Act may groan to even think about reauthorization again. And hopefully when that time comes LSTA will

need, at most, a little fine tuning, and not the wholesale

rewriting that culminated less than a year ago.

Reporting and evaluation are important to the federal process

every year. Why? A bit of history first, and a cautionary tale.

We have had very little national level evaluation of LSCA. One

of the significant studies a few decades back showed that,
Page 72 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

thanks in large part to the Library Services Act and the Library

Services and Construction Act, public library service had been

extended to 96 percent of the country, a dramatic improvement

since 1957 when LSA was first funded.

Now, that data could have been used to claim credit, inquire

into the quality of service in many parts of the country, and

refocus the Act on the changed nature of the job to be done.

But Administrations at that time simply used the data to say the

federal task had been accomplished, and the program was no

longer needed. For many years after that, zero budget

justifications were based on that piece of data.

What lesson might we learn from this tale? We will soon be at

the point where more than 90% of public libraries will have

some kind of connection to the Internet. What then? We could

well be in danger of a similar phenomenon -- the federal job is

done! Why do we need LSTA? Why do we need discounts on

telecommunications rates?

That's why I'm pleased that the new data from ALA's OITP and

NCLIS, as previewed for you yesterday by John Bertot, paint a

much more complex and sophisticated picture. Yes, we've

made significant progress in libraries connected to the Internet,

up from 44 to 72% in a year.

But no, the job is by no means done. The new data also show

that almost half of public libraries do not yet offer any WWW

access, and only 9% provide web access to the public at every

outlet in their systems.

Perhaps because of the impression that the job was done, there

were no further comprehensive national evaluations of the

federal library program. Administrations continued to give it

benign neglect. The program continued to be popular with

Congress because legislators liked libraries, because library

constituencies were active and supportive, and because the

program caused no controversy.
Library and Information Services Page 73

Policy Forum Proceedings

September 15-16, 1997

And it wasn't that there was no evidence of what LSCA

accomplished. The Office of Education and then the

Department of Education would (less often than we would like)

issue reports, some of them quite attractive, giving examples of

activities funded by LSCA. State library agencies turned in the

required reports, and considerable evaluative activity took place

within states.

Further, there was never any hint of funds being misused.

There were a few disputes over whether a rare state that could

not maintain effort deserved a waiver. And in the years before

LSCA Title I was made more flexible, some auditors objected

to public library projects that involved library databases with

school and college library holdings. And there were (and

always will be) disagreements between individual libraries and

state library agencies on the use of federal funds.

But had there been any hint of scandal or abuse in this program,

unfavorable Administrations would have exploited it. It is a

tribute to libraries everywhere that the program remained lean,

highly leveraged, effective, and popular with Congress. So

what effect did the lack of national level evaluation have?

Since most Administrations gave LSCA little attention,

Congress was free to tinker with the program on its own,

although never moving far from the focus on technological

innovation for sharing library resources and for sparking

innovative services to the underserved.

Given the lack of a firm evaluative basis for revising the Act,

congressional friends added complicating provisions to the

state-based titles based on their own interests or those of key

constituents, and added several special focus titles. Some of

these gave attention to important issues such as literacy, and

accomplished much good. Some were so specialized that they

were funded only minimally or not at all.

Over a surprisingly long period, LSCA received no thorough

review. But it became what one memorable assistant secretary
Page 74 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

called "encrusted with barnacles." By the early 90s it was

apparent that LSCA needed to be hauled into dry dock and

given a good cleaning.

It is to the great credit of leaders in the library field that they

decided to do this themselves. Evaluative studies would have

helped immensely with this process. But over a two-year

period, associations representing libraries came up with a good

plan for a Library Services and Technology Act. And just in

time!

The 1994 congressional elections changed the party in power in

Congress and put new chairs at the helms of committees and

subcommittees. The new congressional leaders set out to

change everything -- just as LSCA was about to sunset.

I'm convinced we would not have gotten a multiple-year

renewal of the existing LSCA. But a streamlined and refocused

LSTA could be adopted by a new Congress as a new program.

During the two-year roller coaster ride to passage, LSTA was

streamlined more than we bargained for, and acquired a new

home. But in the end it was enacted.

It was the only major program of many expiring programs

housed in the Department of Education to be reauthorized in the

104th Congress, and, ironically, the only expiring program for

which the Administration had made no recommendations to

Congress.

But the last Congress and this one have also taken to heart the

Government Performance and Results Act. Congress is in no

mood to fund a program simply because they renewed it or

funded it last year, or even because the White House asked for

it. Congress still thinks kindly about libraries, but they want to

know exactly what the federal program has accomplished and

why continued funding is needed.

I've already been told in meetings with appropriations staffers

that they expect the constituencies of the programs to cooperate
Library and Information Services Page 75

Policy Forum Proceedings

September 15-16, 1997

fully with this need for results. They know the federal agencies

can't do it alone; Congress wants and expects cooperation from

the constituencies.

I'm pleased that IMLS is committed to this process. Diane

Frankel has made quite clear that evaluation is a high priority

for her, not just to comply with the law, but to have ammunition

to take before Congress. This is a refreshing change for the

library program! And IMLS is getting substantive help from

COSLA.

Yet we have special challenges in producing evaluative data

that are realistic and meaningful. We have the traditional

difficulties of producing quantifiable results for a public good

such as library service. We have the difficulty that library users

are voluntary, not captive clientele; that librarians inquire into

their motives only enough to help them; and that the impact of

using a library may not be immediately apparent.

And with the federal program, we have the problem that it is

one small funding stream combined with much larger sources

of support. LSTA may leverage new and improved services,

but are new users served or served better due to the federal

stimulus alone? or to the state money added to the federal? or

to local matching funds? or to a combination of all of these?

We need a way to measure federal leverage.

Another challenge comes from a desirable feature of LSTA --

flexibility. States have much more flexibility to adapt the

federal purposes to state and local needs. And states have a

major role in evaluation. But we must have at least a handful of

nationwide indicators of LSTA effectiveness. And because

technology and services evolve more quickly now, we must

make sure our indicators evolve as well. Consistency and

comparability over time must be balanced with the changing

role of libraries in our data gathering and reporting.
Page 76 Library and Information Services

Policy Forum Proceedings

September 15-16, 1997

Finally, we must be careful what we measure and whether it

goes far enough. The purposes for which states can use LSTA

funds list electronic linkages, electronic networks, sharing of

resources and computer systems, and targeting services to the

underserved. We can indeed measure how many libraries are

linked and to what, and how much sharing is going on. But

frankly, what does that mean to a congressional appropriator?

These mechanisms help libraries put certain infrastructure in

place so that service can be improved. And this is an

appropriate federal role; much of that infrastructure would be

hard for the individual library to acquire on its own. But it's

technical and not very exciting in and of itself.

More important is -- what do electronic linkages and the sharing

of library resources mean for the library user? How do the

improved services the infrastructure makes possible really

affect the legislator's constituents?

Whatever the challenges, we must have national level reporting

and evaluation on what LSTA is accomplishing. We must do

it; we must do it well; we must do it carefully; and we must use

the results judiciously. Future LSTA funding depends on it.

Continuation of LSTA beyond the current six year

reauthorization depends on it.