Environmental Protection Agency: EPA Needs to Complete a Strategy for Its Library Network to Meet Users' Needs
Published: 2010-09-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Environmental Protection Agency's (EPA) library network provides agency staff and the public with access to environmental information. A 2006 attempt by EPA to reorganize its network by consolidating libraries and making more materials and services available online caused concern among users, and in 2007, EPA put a moratorium on its reorganization plans. Congress requested that GAO report on the reorganization and has again requested a follow-up on these issues. Accordingly, GAO reviewed (1) the status of EPA's overall strategy for its library network, (2) the status of EPA's plan to reopen the libraries it closed and other actions planned or taken, (3) EPA's efforts to digitize printed documents to make them electronically available, and (4) EPA's efforts to communicate with staff and other stakeholders about its library network. GAO reviewed regulations and agency funding and inventory documents and interviewed EPA staff and contractors, as well as independent library professionals. GAO also assessed the reliability of EPA's data on library holdings and from EPA's staff survey on library use and needs.
section_title: What GAO Found
paragraphs: Although EPA has taken a number of steps to meet the needs of library users, it has not completed a plan identifying an overall strategy for its library network, with implementation goals and a timeline of what it intends to accomplish. Scheduled for completion in 2008, the strategic plan was to provide EPA staff and the public a detailed view of EPA's library operations and future direction. The draft outline of the strategic plan, however, is largely a placeholder list of current and planned EPA activities. For example, while it emphasizes the central role to be played by electronic library resources, the draft outline does not contain goals or a timeline for completing an inventory of holdings or digitizing those holdings. The draft outline also does not set out details of how funding decisions are to be made. Given the current economic environment, without a completed strategic plan, including a detailed strategy for acquiring, deploying, and managing funding, EPA may find itself hard-pressed to ensure that the network can meet its users' needs. The agency has reopened libraries closed during reorganization, although about half the network's 10 regional libraries are operating with reduced hours. EPA has also developed standards for the regional and headquarters libraries' use of space, on-site collections, staffing, and services. The agency has also hired a national library program manager to carry out day-to-day activities and bring focus and cohesion to the network. Working closely with EPA management and library staff, the national library program manager, who is responsible for library network strategic planning, has set in motion a number of actions meant to improve library network operation and communication, including working closely with internal and external advisory boards and creating a library policy and related procedures. EPA has resumed digitizing some of its libraries' documents, although it has not inventoried the network's holdings. The agency is digitizing documents in three phases. Phase 1 was completed in January 2007, phase 2 is scheduled for completion in December 2010, and planning has begun for phase 3. Because EPA has not taken a complete inventory of its library holdings, however, it cannot determine which documents, or how many, will need to be digitized and, consequently, cannot accurately estimate the total cost of digitization or how long it will take. Since we reported on the library network reorganization in 2008, EPA has taken steps to communicate with staff and other stakeholders about its library network, including providing information about the libraries and soliciting information from library users. EPA has also made improvements to the main Internet gateway to the network, making more documents available electronically and providing better access to electronic documents and services. Nevertheless, because EPA's 2009 survey of the information needs and library use of its staff had methodological flaws--similar to those GAO identified in 2008--the agency is unlikely to obtain accurate information that would enable it to make appropriate decisions on the corrective actions that would best address library users' needs.
section_title: What GAO Recommends
paragraphs: GAO recommends, among other actions, that EPA complete its strategic plan for the library network and ensure that survey methods provide reliable data on which to base decisions. With clarifications, EPA concurred with our recommendations.

section_title: Letter
paragraphs: In 1971, soon after its creation, the Environmental Protection Agency  (EPA) established a library network to provide agency staff and the public  with access to environmental information. Comprising libraries and  repositories at EPA headquarters and in regional EPA offices, research  centers, and laboratories across the nation, this network houses  information on a broad range of subjects, including science, environmental  protection and management, and environmental laws. In 2006, partly in  anticipation of fiscal year 2007 budget reductions, EPA began a  reorganization effort meant to consolidate libraries in the network and  make more materials and services available online. The reorganization  plan proposed closing libraries; dispersing, disposing of, or digitizing some  collections; cutting operating hours at certain libraries; and altering  librarian services. In 2006, EPA closed 5 out of the 26 libraries it operated  at the time.
As EPA was implementing this reorganization, Congress, professional  library associations, labor unions, and other library users raised several  concerns. In response to congressional and other concerns, in January  2007 EPA placed a moratorium on its reorganization activities, directing its  staff to make no further changes in library services until library policy and  procedures were adopted. During the moratorium, 5 libraries remained  closed; no additional closures were made; there were no additional  reductions in hours of operation, services, or resources; and the disposal  of library materials and document digitization stopped.
At your request, we reported on the reorganization in 2008. In our report,  we found that EPA had not followed its own recommended steps to  prepare for reorganizing the libraries. Neither did the agency develop  procedures to fully inform stakeholders, such as library users and library  professionals, of the final configuration of the library network; rather,  communication with stakeholders varied from library to library. We also  found that EPA lacked an effective strategy or central leadership to ensure  the continuity of library services after reorganization, and each library  decided on its own whether to close its doors and how to disperse or  dispose of its materials. Moreover, EPA did not specifically allot funds to  help closing libraries manage their collections, instead leaving each  program or regional office to use its annual funding for closure costs.
In light of continuing congressional and public concern over EPA’s plans  for its library network, you asked us to follow up on these issues.  Accordingly, this report reviews (1) the status of EPA’s overall strategy for  its library network, (2) the status of EPA’s plan to reopen the libraries it  closed and other actions planned or taken, (3) EPA’s efforts to digitize  printed documents to make them electronically available, and (4) EPA’s  efforts to communicate with staff and other stakeholders about its library  network.
To address these objectives, we reviewed relevant EPA funding and  inventory documents, policies, plans, and guidance, as well as related  regulations and requirements pertinent to the library network and efforts  to improve its operations. We focused our review on EPA’s Office of  Environmental Information’s headquarters library in Washington, D.C.; the  10 regional EPA libraries; and the two Office of Administration and  Resources Management libraries, in Cincinnati, Ohio, and Research  Triangle Park, North Carolina. We compared library operations before,  during, and after the reorganization; obtained and reviewed the library  network’s policy and procedures; reviewed the agency’s draft outline of a  strategic plan for the library network; obtained and reviewed documents  on EPA’s digitization process; and reviewed EPA’s efforts to communicate  with and solicit input from users. We interviewed EPA librarians and  library managers in selected EPA libraries, as well as EPA officials  knowledgeable about EPA’s library network and budget. We also  interviewed management officials from the federal employees’ union  representing EPA staff and spoke with representatives from EPA’s  regional science councils, which consist of EPA scientists and technical  specialists. We further sought information from library professionals,  including representatives from the American Library Association and from  contractors involved in digitizing EPA documents. After limited testing  and discussions with EPA officials, we determined that EPA’s data on  library funding and on the number of digitized documents and those  scheduled to be digitized were not sufficiently reliable for our purposes.  Because these data were the only data available, however, we used them  to discuss in broad terms EPA’s funding history and the status of its  digitization efforts, noting the data’s limitations as appropriate. Appendix I  describes our scope and methodology in greater detail.
We conducted this performance audit from October 2009 through  September 2010, in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: EPA was established in December 1970 to consolidate a variety of federal  research, monitoring, standard-setting, and enforcement activities into one  agency to ensure the protection, development, and enhancement of the  total environment. To help accomplish its mission, EPA in 1971  established a library network that came to comprise 26 libraries located  across the country. The library network functions as a collection of  independent local libraries, catering primarily to the needs of local EPA  staff and walk-in public patrons. The libraries are funded and managed by  several different regional and program offices at EPA (see fig. 1). EPA  defines network libraries as those of its libraries with an official  membership presence in the global Online Computer Library Center  system.
The combined EPA library network collection contains a wide range of  general information on environmental protection and management and on  basic and applied sciences, as well as extensive coverage of topics related  to the statutory mandates that EPA must meet. Several of the libraries  maintain collections focused on special topics to support specific regional  or program office projects. The libraries thus differ in function, scope of  collections, extent of services, and public access. In addition to its  physical locations and holdings, the EPA network provides staff and  public access to its collections through the following:  its online library system, a Web-based database of all of EPA library  holdings, also known as EPA’s online “card catalog”;  interlibrary loans to another library within the network or to other  libraries;    a Web site combining two databases—EPA’s National Service Center for  Environmental Publications (NSCEP) and its National Environmental  Publications Internet Site (NEPIS)—which provide an online gateway for  access to available print and digital documents, respectively; and    desktop access for staff to online journals, the Federal Register, news,  databases of bibliographic information, and article citations.
In addition, librarians are available in each library to catalog and maintain  collections and to assist EPA staff and the public with research.
In 2003, EPA began studying options for operating its library network in  the future. In August 2006, the agency issued a reorganization plan, titled  EPA FY 2007 Library Plan: National Framework for the Headquarters  and Regional Libraries. The focus of this plan was a reorganization of the  headquarters library and the 10 regional libraries, all of which received  substantial funding from EPA’s Office of Environmental Information. The  2007 library plan identified a new model for library services, which  consisted of three components: a coordinated library network, instead of  stand-alone operations; more electronic delivery of services; and  maintenance of existing essential services. During implementation of this  plan, EPA closed the Chicago, Dallas, and Kansas City regional libraries  and closed its headquarters library to physical access, although the  headquarters library remained as one of three repositories for storing print  collections. Another library located at EPA headquarters within the Office  of Prevention, Pesticides, and Toxic Substances (referred to as the  “chemical library”) was not subject to budget reductions and was not  discussed in the reorganization plan, but, like the headquarters library,  was also closed to physical access. EPA also reduced or eliminated the  library staff at the closed libraries. Several other libraries reduced their  operating hours, and some libraries disposed of their materials or  dispersed them to other EPA libraries or to non-EPA libraries. EPA also  began to digitize EPA documents not currently in NEPIS, beginning with  documents in the libraries being closed. EPA’s reorganization plan also  discussed how the closed libraries were to handle their collections,  directing them primarily to disperse the collections to other libraries.
EPA’s implementation of its reorganization plan caused widespread  concern among its staff, the public, interested parties, and Congress. In  response to these concerns, congressional committees directed $1 million  in funding to restore the libraries recently closed or consolidated, asked us  to review EPA’s reorganization plan and its implementation, and directed  EPA to prepare a report regarding actions to restore publicly available  libraries. In addition, EPA in January 2007 imposed a moratorium on its  reorganization efforts.
Until April 2007, EPA’s library network operations had been guided by  EPA’s Information Resources Management Policy Manual, which stated  that the library network was to provide EPA staff with access to  information to carry out the agency’s mission and that the libraries were to  provide state agencies and the public with access to the library collection.  The Policy Manual also defined the role of a national library program  manager, who was to have responsibility for coordinating major activities  of the library network, although not budget authority for the libraries. EPA  replaced this manual in April 2007 with an interim library policy and, in  May 2009, with its final library policy. The final May 2009 policy also  defined key roles and responsibilities, including those of the national  library program manager and those of “federal library managers,” who  were to have first-line responsibility for operating the physical libraries  and providing services.
EPA lifted the moratorium in June 2009, following implementation of its  May 2009 policy and 6 of 12 proposed procedures for the library network.  After we issued our report in February 2008, Congress held hearings on  EPA’s library network reorganization efforts, which were followed by the  release of EPA’s March 2008 report addressed to Congress, in which EPA  stated that it would reopen the closed libraries by September 30, 2008. In  our 2008 report on EPA’s library network reorganization, we assessed the  reorganization effort against our past work on key practices and  implementation steps to assist mergers and organizational  transformations. These key practices include ensuring that top leadership  drives the transformation, establishing a coherent mission and integrated  strategic goals to guide the transformation, and setting implementation  goals and a timeline to show progress from day one.
section_title: EPA Has Not Completed a Strategic Plan for Its Library Network Identifying an Overall Strategy for the Network
paragraphs: Although it has been preparing a strategic plan for its library network for  3 years, EPA has not completed a plan identifying its overall network  strategy, with implementation goals and a timeline for what it seeks to  accomplish. In our 2008 report, we stated that EPA was developing a  library strategic plan for 2008 and beyond, which was to detail library  services for staff and the public and lay out a vision for the library  network’s future. EPA has had a draft outline of this strategic plan since  July 2007. We reported that in October 2007, EPA’s Office of  Environmental Information asked local unions throughout the agency to  comment on a draft of the plan.
The draft outline of the strategic plan envisions the library network as “the  premier environmental library network that provides timely access to  information and library services to its employees and the public” and  proposes to realize this vision by increasing emphasis on electronic  resources and using new information technologies. The draft outline of  the strategic plan also lists several principles as the foundation for present  and future directions of the library network: setting overall goals and  objectives and a direction for implementation; periodic review of the plan  to evaluate progress and update strategies to respond to new  opportunities or challenges; soliciting input from internal and external  stakeholders; and developing the plan in a transparent manner by  reporting progress and soliciting input from interested parties. According  to EPA officials, since 2007, EPA has been in the process of assessing  library users’ needs, which is to be completed before they believe they can  finish the strategic plan. EPA officials have stated that a working group led  by the national library program manager is to resume work on the plan  later in 2010.
The draft outline of the strategic plan is largely a list of current and  planned EPA activities—primarily placeholders to be completed. For  example, under the heading, “Digitization,” the text states that the  digitization procedures will outline the methods to be used by EPA  libraries to prepare and send EPA documents for digitization and inclusion  in NEPIS; no goals or timeline for implementing these activities—which  we previously reported are among the key components of successful  organizational transformations—are given. We have found that an  organization’s transformation is strengthened when it publicizes  implementation goals and a timeline to build momentum and show  progress. Despite an emphasis on the central role to be played by  electronic library resources, the draft outline of the strategic plan briefly  describes procedures for packing and shipping documents to be digitized,  without describing actions to be taken to digitize holdings or target dates  for accomplishing those actions. EPA holds an enormous amount of  environmental information, including publications generated by its  program offices, as well as research publications generated under  contracts, grants, and cooperative agreements. A large portion of this  information exists only in print form. As part of its vision for the library  network, according to the draft outline, EPA is seeking to convert this  information into a digital format to make it more widely available and  readily accessible to users. Yet the draft outline of the strategic plan does  not describe criteria for deciding what documents should be digitized, for  deciding whether or how to digitize copyrighted documents of value, or  for scheduling the funding needed and a time frame for completing the  digitization. Without such criteria, EPA cannot ensure that it is digitizing  the most valuable or important documents or providing users with  information most relevant to them.
Furthermore, although the draft outline of the strategic plan includes a  placeholder for a section describing a funding model for the network, it  contains no detail. Under the heading, “Funding Model,” the text states  that this section in the plan will address how EPA will ensure that the  network libraries have “adequate funding” and will discuss how funding  decisions are made, along with the Office of Environmental Information’s  role in the funding process. But the draft outline of the strategic plan does  not define what constitutes adequate funding, although inadequate funding  has been a concern for the library network since fiscal year 2007. Until  then, library spending had remained relatively stable, ranging from a high  of $9.2 million in fiscal year 2002 to a low of $8.2 million in fiscal year  2006. In fiscal year 2007, when EPA’s budget was reduced, library  spending was $6.3 million.
The draft outline of the strategic plan also does not set out the details of  how funding decisions are made. Setting out details for how such  decisions are to be made, to ensure that they are informed and  transparent, is especially important because of the decentralized nature of  the library network. The library network’s funding remains subject to  uncertainty in the future because the several different program and  regional offices responsible for EPA’s libraries generally decide how much  to spend on their libraries out of funding available in larger accounts that  support multiple activities. EPA’s Office of Environmental Information, the  primary source of funding for the regional libraries, typically provides  funding through each region’s support budget and generally gives regional  management officials discretion on how to distribute this funding among  the library and other support services, such as information technology,  utilities, and mail room support. EPA officials told us that, starting in fiscal  year 2010, they are increasing the amount of funding allocated to the  libraries in the regions. The regions also obtain a much smaller portion of  their library funding from other program offices. For example, the  Superfund program office funds the storage and maintenance of  information on the National Priorities List, EPA’s list of the most seriously  contaminated sites in the United States. The extent to which other  program offices provide funding to the regional libraries varies. Thus,  without a detailed strategy for how decisions are made to acquire, deploy,  and manage funding resources across the library network, EPA may find it  difficult, particularly in an era of declining budgets and competing national  priorities, to achieve its vision for the library network and to fully meet the  needs of library users.
Moreover, although the draft outline of the strategic plan contains a  section on communication among network libraries, it makes no mention  of a strategy or a formal outreach plan to ensure that EPA communicates  with and obtains feedback from users about improvements to its library  network in a way that would allow it to measure whether such  improvements are meeting users’ needs. The section lists communication  methods EPA is using, such as monthly network teleconferences among  staff and federal managers. In addition, the section identifies comment  cards, questionnaires, and a Web presence for how it solicits users’  feedback, but there is no mention of how EPA plans to assess feedback on  what is important to users or what improvements are working well or  poorly. EPA has another draft document, titled “EPA Library Network  Communication Strategies,” whose purpose is to establish procedures by  which libraries in the network are to communicate both internally and  externally. Most of this document focuses on communication within the  library network itself, explaining how the library network is coordinated  and detailing mechanisms for internal communication, including annual  meetings for library network staff and federal library managers. One of the  final sections in this procedures document lists several means of  communicating externally, including Web sites and various local options  for libraries to reach out to local patrons, such as tours, signs, comment  cards, and online feedback mechanisms. Beyond listing such mechanisms,  however, this document, like the draft outline of the strategic plan, does  not lay out a systematic communication strategy with communication and  feedback performance goals that can be measured to determine progress.  Without such a strategy, communication with library users is likely to  remain piecemeal and reactive.
section_title: Since 2008, EPA Has Reopened Closed Libraries and Taken Other Actions
paragraphs: EPA has reopened all of the libraries it closed in 2007 and has taken other  actions to improve library operations. In its 2008 report addressed to  Congress, the agency stated its commitment to have libraries in each  region and at headquarters open to the public. EPA also committed to  reestablishing on-site libraries for its staff and the public in the three  regions where the libraries had been closed and at the combined  headquarters and chemical library. EPA reopened all five closed libraries  by September 2008, although the agency had to find new space for two of  the three closed regional libraries and their collections, and all three of  these regional libraries are operating on reduced schedules. Each of the  reopened libraries was staffed with a professional librarian and required to  maintain a collection of core reference materials and additional library  resources to meet local needs and to ensure that staff had access to core  library services and the public had access to the library and its collections.  With the reopening of the closed libraries, one other regional library that  had not been closed also began operating on reduced schedules, as  compared with its hours before reorganization (see table 1). As of  September 2010, about half of the 10 regional libraries were operating  4 days a week, rather than 5 days as they were before EPA’s reorganization  efforts—a reduction in hours due largely to funding constraints, according  to library officials. All of the libraries are staffed with at least one full-time  or part-time librarian, with several libraries having more than one librarian  or additional library staff.
In addition to reopening the closed libraries, according to EPA officials we  spoke with, EPA developed standards for the regional and headquarters  libraries’ use of space, on-site collections, staffing, and services. These  standards specify, for example, that the libraries make adequate space  available for in-person interactions between library staff and users, that  on-site collections and materials should address local and regional needs,  that staff and the public have certain minimum hours of access per week  (at least 4 days per week on a walk-in or appointment basis in the regional  libraries and at least 5 days per week on a walk-in or appointment basis in  the headquarters library), and that the libraries provide interlibrary loans  and reference or research assistance.
One of the key actions taken by EPA in May 2007 was to hire a national  library program manager, a position that had been vacant since 2005.  Housed in EPA’s Office of Information Analysis and Access, within the  Office of Environmental Information, the national library program  manager is charged with carrying out day-to-day activities of the library  network and with bringing focus and cohesion to the network. Part of this  charge involves agencywide responsibility for public information access,  including strategic planning for the library network, and participating in  policy formulation regarding access to EPA’s public information. To fulfill  this leadership role, EPA officials said, the national library program  manager is to work closely with the management of EPA’s Office of  Environmental Information to set in motion a number of actions meant to  improve library network operation and communication. To communicate  with and gather feedback internally, the national library program manager  initiated monthly teleconferences and annual meetings for all library  managers and staff. Seeking to get the most out of the experience and  knowledge of these library managers, librarians, and staff, the national  library program manager established internal working groups to research  improvement activities, address concerns, and recommend improvements.  For example, the national library program manager established working  groups on digitization, staff information needs, and development of the  final May 2009 library policy and related procedures (see table 2). In  addition, the national library program manager serves as the EPA- appointed representative in working with outside library professionals,  specifically an external board of advisors created by the Federal Library  and Information Center Committee, which advises EPA on future  directions for the library network.
section_title: EPA Has Resumed Digitizing Unique EPA Documents but Has Not Inventoried Its Holdings
paragraphs: EPA has restarted its process of digitizing some of its libraries’ holdings,  but because the agency has not completed an inventory of its holdings, it  does not know the total number of documents to be digitized. According  to EPA data, which our limited testing found to be insufficiently reliable,  EPA had digitized 16,175 documents from its libraries as of January 2010.  Creating an electronic copy of a document by means of digitization is  relatively simple—essentially the same scanning process for making  photocopies—although it can be time-consuming and expensive if the  document contains special features such as foldout pages, cannot be taken  apart, or needs to be digitized at a high level of resolution or in color. After  the resulting electronic files are uploaded to EPA’s Web databases, the  administrator of EPA’s online library catalog is to ensure that links to the  digitized documents are included in the bibliographic records for each  document. According to EPA officials, the present digitization effort will  expand NEPIS, EPA’s sole electronic archive of published material, which,  according to EPA officials, contains 40,000 publicly accessible digital  documents as of June 2010, up from 4,000 in 1996.
According to EPA documents, the digitization process is to take place in  three phases:    Phase 1 covered unique EPA documents held by the libraries that were  closed under the reorganization plan. EPA data show that 15,260  documents were digitized during this phase, which was completed in  January 2007.
Phase 2 is to cover all remaining unique EPA documents except those  larger than 11 by 17 inches. According to EPA officials, this phase is  scheduled for completion in December 2010 and should produce 10,102  additional digitized documents, bringing the total number of digitized  library documents available to the public to over 25,000.
Phase 3 is to include EPA documents of which more than one copy exists  in the library network, plus unique EPA documents larger than 11 by  17 inches. As of July 2010, EPA was beginning to plan this phase.
As of September 2010, the total estimated cost for digitizing EPA’s library  holdings remained unclear, in large part because EPA has not completed  an inventory of its holdings and has therefore not determined the total  number of documents that need to be digitized. When it began digitizing  documents from the closed libraries in 2006 under phase 1, EPA estimated  that the project would cost $80,000—$78,000 for scanning and $2,000 for  uploading the digital files to the Web databases—although, according to  agency officials, the agency did not track the actual costs. For phase 2  digitization, which began in fiscal year 2009, EPA estimated the cost at  about $327,000. EPA has not estimated the cost or a completion date for  its final, phase 3 digitization effort, in part because the agency is still  cataloging all its library holdings in a single database so it can inventory all  the documents that need to be digitized. One regional librarian we spoke  with, for example, told us that about 2,000 documents in the regional  library’s catalog were not in EPA’s online library system, and it is still  unknown which or how many of these documents will need to be digitized.  Without a complete catalog or inventory of its holdings, EPA cannot  determine which documents, or how many, will need to be digitized and,  consequently, cannot accurately estimate the total cost of digitization or  how long it will take. According to the national library program manager,  an EPA workgroup is currently drafting a new cataloging procedure for  the libraries and expects the procedure to be approved and implemented  before the end of 2010. This procedure requires each network library to  inventory its collection on a regular basis, either the entire collection  every 3 years or one-third of the collection each year.
In addition, EPA library officials observed that a significant number of the  documents in EPA libraries are copyrighted, and to date EPA does not  plan to digitize them. EPA, like other federal agencies, often contracts with  entities in the private sector to do work. In addition, EPA provides  financial assistance in the form of grants and cooperative agreements to  various recipients, such as state, local, and tribal governments;  educational institutions; hospitals; and nonprofit organizations. Such  assistance is documented in an assistance agreement. Both contracts and  assistance agreements may result in the production of copyrighted  documents. In the case of contracts, federal regulations provide that when  a contractor is permitted to assert a copyright in any document(s)  produced, the government has a license to display the copyrighted work  publicly, which includes posting it on a Web site accessible to the public.  In the case of works produced under assistance agreements, on the other  hand, the government has a right to reproduce, publish, or otherwise use a  copyrighted work for federal purposes, but EPA’s Office of General  Counsel has determined that inclusion in EPA’s online public library  would not constitute a federal government purpose.
On the advice of EPA’s general counsel, EPA’s digitization workgroup has  recommended digitizing documents created under contract but not those  created under EPA’s assistance agreements. According to EPA’s grant  awards database, these agreements have resulted in more than 21,000  grants valued at over $40 billion in taxpayer dollars. Some of these grants  led to publications, resulting in a substantial body of publicly funded  written material. According to EPA’s general counsel, EPA may digitize  such documents so that staff and others may use them for federal  government purposes but may not disseminate them for other purposes.  EPA may also seek permission from copyright holders to digitize and  disseminate online copyrighted documents produced under assistance  agreements, although some costs may be associated with obtaining such  permissions—tracking down copyright holders after years, or even  decades, have passed, for example—further complicating any estimation  of total digitization costs. An alternative practice has been in use by the  Federal Library and Information Network, the business arm of the  Federal Library and Information Center Committee: permission to use  copyrighted material produced under assistance agreements is sought at  the time an agreement is established. If the prospective copyright holder  grants permission, then a statement to that effect is incorporated into the  assistance agreement, incurring minimal, if any, additional costs. Without  permission from copyright holders, however, documents prepared under  EPA assistance agreements, using taxpayer dollars, will remain  unavailable online to the public.
section_title: EPA Has Taken Steps to Communicate with Staff and Other Stakeholders about Its Network, but Its Staff Survey Was Flawed
paragraphs: EPA has taken steps to communicate with staff and other stakeholders  about its library network—including providing information about the  libraries as well as soliciting information from library users—but a 2009  survey about its staff’s information needs was flawed. In general, EPA staff  and external stakeholders told us the agency is doing a better job of  communicating with them and soliciting input on the operations and  future direction of the library network, particularly at the local level.  Representatives from EPA’s employees’ unions and regional science  councils stated that communication about the library and its services— such as new resources, training, and open houses—is primarily done at the  regional level, either through e-mail or the region’s intranet page. Although  staff have not been directly solicited for feedback, according to the  representatives, no outstanding issues regarding the libraries have been  raised, except that a few representatives said they would like to see the  libraries open 5 days and 40 hours per week.
To keep library managers and staff engaged in improving library  operations, EPA has adopted a number of techniques to communicate with  them and solicit their input. These techniques have allowed EPA to gather  staff input for policies and procedures, operational issues, and Web page  improvements. Examples include the following:    The national library program manager holds monthly network  teleconferences with library managers and staff on matters of interest to  the entire network or on operational topics, such as the library policy and  procedures.
The national library program manager also holds ad hoc teleconferences  with library managers elsewhere in the network to discuss their libraries’  needs.
Managers and staff use mailing lists to communicate with one another  about daily library operations or requests for assistance.
For the last 3 years, EPA has held an annual network meeting in different  locations for library managers and staff to foster collaboration, provide  training, and share information about the network. At the last meeting, in  October 2009, participants discussed ways to address results of the 2009  staff survey, prepared for the next round of digitization, and discussed  ways to improve library services. The next annual meeting is scheduled for  March 2011.
EPA and the union representing EPA staff agreed to create a union- management advisory board with six members—three union  representatives and three from EPA management. The board reviews and  makes recommendations on library network policy and procedures and  will review the library network strategic plan once it is completed.  In December 2009, EPA instituted a pilot program, an “ask a librarian” live  chat. Ten libraries are participating in the program, which lets users  contact a librarian through an electronic link to request services. As of  July 2010, the “ask a librarian” pilot was available only to EPA staff.
To begin to realize its vision of effectively implementing new information  technologies and making documents readily available electronically, EPA  in 2007 engaged a contractor to review the user-friendliness of the  combined Web page, or gateway, to the NSCEP and NEPIS online  databases. The review identified ways to improve the site’s effectiveness  and overall functionality for users. EPA implemented many of these  improvements. For example, the gateway home page now describes the  purpose of each database (NSCEP for print materials and NEPIS for  electronic documents) and what types of publications they contain, noting  that they contain only EPA publications. EPA made several changes to the  document display page as well, such as placing a navigation bar at the top  and bottom of the document with large icons and providing a button that  allows users to obtain a copy in one of three formats. The display page  allows users to put a copy of frequently used documents in a holding area  for later retrieval. Work is also under way to integrate Google search  capabilities into this gateway, as well as the ability to refine the precision  of searches with user-friendly “clouds” of related keywords. In addition,  EPA has added easy ways for users to offer feedback, which EPA may  then incorporate to make improvements. For example, the navigation bar  on the document display page now includes a “report an error” button, and  every page has a “contact us” link. Furthermore, the left frame of the site  contains a link to a customer satisfaction survey, and the site also has a  separate page for user feedback.
EPA has also made outreach efforts to library professionals outside EPA— primarily by presenting and exhibiting at professional library association  trade shows and conferences, attending external training, visiting other  federal national libraries, and interacting with its external board of  advisors. Ties with the external board have been a particularly important  part of EPA’s response to concerns over the agency’s library  reorganization. From June 2007 through February 2010, the national  library program manager met with the board approximately 20 times,  working with it on the full range of key issues, from policy development to  funding models to communication with stakeholders. The external board  also advised EPA on the development of a survey to assess the information  needs of EPA staff.
One of the principles in the draft outline of the strategic plan is soliciting  feedback from internal and external stakeholders about their information  needs. To solicit such feedback from staff, EPA in early 2008, under the  direction of the national library program manager, engaged a contractor to  conduct interviews, hold focus groups, and conduct a Web-based survey.  The survey was made available for approximately 1 month in 2009 via a  secure Web site only to EPA staff (about 17,000 individuals), not all of  whom were library users. After the survey was completed, the contractor  conducted a series of focus groups and one-on-one interviews with EPA  managers, focusing on significant issues identified in the Web-based  survey; according to EPA officials, these in-person discussions were to  help ensure that a comprehensive perspective of user needs was captured.  EPA officials stated that the Web-based survey results corroborated what  the agency learned in an earlier survey, done in 2004 to 2005. On the basis  of the survey results, focus group discussions, and management  interviews, the contractor developed recommendations for EPA’s  consideration. EPA received the results of the survey and discussions,  along with the contractor’s recommendations, in August 2009 and has  assigned working groups of library staff to review the findings and suggest  how EPA could address them.
We found, however, that this survey had flaws, similar to those we  identified in the 2004 to 2005 survey and discussed in our 2008 report,  which greatly reduce its usefulness. First, in both the earlier and the 2009  surveys, the response rate was 14 percent, far lower than the 80 percent  response rate that Office of Management and Budget guidance  recommends for a survey to increase the likelihood that it adequately  represents a universe of respondents. Neither EPA nor the contractor for  the 2009 survey analyzed the results for the nonresponse bias that may  occur at response rates lower than 80 percent, particularly if the group of  respondents differs significantly in relevant ways from nonrespondents.  Thus, EPA cannot be assured that either survey accurately described staff  needs for information or their uses of the library. Second, respondents to  both the 2009 and the earlier survey were self-selected, a methodology that  often leads to biased samples, since the traits that affect a person’s  decision to participate in the survey—such as strong opinions or  substantial knowledge—may be correlated with traits that affect survey  responses; the result is an unrepresentative sample of possible  respondents. The risk of potential bias through self-selection is increased  by the fact that neither EPA nor the contractor for the 2009 survey  instituted any safeguards to prevent respondents from submitting more  than one survey each. Thus, there is no assurance that the survey results  are unbiased and reflect a broad range of EPA staff perspectives and  experiences. Third, in neither survey did EPA gather views from or  determine the needs of other significant users of EPA libraries, such as  state and local environmental organizations or the public at large.  Although EPA officials told us that EPA is planning to assess the needs of  public patrons, an assessment that does not correct the methodological  weaknesses we found in EPA’s two previous surveys of its staff is unlikely  to produce results that accurately reflect the needs of public patrons.
section_title: Conclusions
paragraphs: In the 4 years since EPA issued a reorganization plan for its library  network, the agency has taken a number of steps to better communicate  with, and meet the needs of, library users. EPA’s lack of a completed  strategic plan identifying its overall strategy for the network, however,  leaves unclear how the agency will translate into reality its vision of a  “premier environmental library” with an “emphasis on electronic  resources.” Steps the agency has taken, including hiring a national library  program manager and establishing a uniform policy and some procedures  for the libraries, have led to some improvements in library services and  will undoubtedly enhance network cohesion. But without a completed  strategic plan that contains implementation goals and timelines, neither  EPA nor users of its libraries can have a clear view of what EPA plans to  do, when EPA plans to do it, and whether EPA’s actions will ultimately  meet users’ needs. In particular, without a strategy for acquiring,  deploying, and efficiently allocating library funding, the library network  could have difficulty maintaining high-quality service in the digital age.  Moreover, EPA’s approach to digitizing copyrighted works in the future— as well as the fact that the agency has not yet inventoried all library  holdings—could, if not revisited, detract significantly from the utility of  the library network. Specifically, unless EPA revisits its decision not to  digitize documents prepared with taxpayer dollars under assistance  agreements, it may be missing opportunities to make these documents  more readily available to users, including other federal users, who need  them to better carry out their work. Finally, improvements to the library  network’s Internet gateway offer new means of seeking feedback from  library patrons about their use of and need for library services.  Nevertheless, EPA does not have a valid method for assessing those  library users’ needs. If, in future assessments of users’ needs, EPA fails to  correct the flawed methods of its previous staff surveys, the agency is  unlikely to obtain accurate information that would enable it to make  appropriate decisions on the corrective actions that would best address  those needs.
section_title: Recommendations for Executive Action
paragraphs: To ensure that EPA’s library network continues to meet its users’ needs,  we recommend that the Administrator of EPA take the following six  actions:    Complete EPA’s strategic plan for the library network, including  implementation goals and timelines. In so doing, EPA should outline  details for how funding decisions are to be made, to ensure that they are  informed and transparent.
Complete an inventory of the library network’s holdings to identify what  items remain to be digitized.
For assistance agreements already in place, EPA should digitize  documents produced under the agreements and make them available to  federal employees and other authorized users for federal government  purposes.  In future assistance agreements, make explicit that EPA can include in the  agency’s public online database, without obtaining prior permission from  the copyright holder, any documents produced under the agreements.
For future assistance agreements where EPA cannot make such an  arrangement, EPA should digitize documents produced under the  agreements and make them available to federal employees and other  authorized users for federal government purposes.
Ensure that the data analysis protocols used for conducting surveys of  users’ needs—including sampling procedures and response rates—are  sufficiently sound methodologically to provide reliable information on  which to base decisions and allocate resources efficiently.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided EPA with a draft of this report for review and comment. With  clarifications, the agency concurred with our recommendations. EPA  acknowledged that the planning document available on the agency’s Web  site—which our report refers to as the draft outline of the strategic plan— has provided more of a working agenda than a strategic plan to guide the  rebuilding of the library program. In responding to our recommendations,  EPA wrote that it believes it now has enough information to develop an  effective strategic plan for the library network and that it is time to  complete and publish a formal plan identifying an overall network  strategy, with implementation goals and a timeline for future  accomplishments. The agency stated that it is moving forward with the  strategic plan, which it aims to complete in fiscal year 2011. In addition,  EPA said it will address the inventory of library holdings, completing a  schedule for cataloging the inventory by November 1, 2010, and striving to  complete the cataloging by September 30, 2011. The agency further agreed  to take the necessary steps to ensure that any future assessments of users’  needs employ methodologically sound data protocols and provide reliable  information.
Regarding our recommendations on the digitization of copyrighted  documents produced under assistance agreements, EPA said it would  address the feasibility and legality of digitizing products resulting from  such agreements. For future assistance agreements, the agency said it will  develop options for gaining advance permission to digitize products from  these agreements and take these options to senior agency managers by  mid-2011 for consideration and action. For existing assistance agreements,  however, EPA wrote that, because of legal and technical constraints, it  does not plan to digitize products produced under existing agreements. In  further clarifying the agency’s written comments, EPA officials told us that  because the documents produced under existing assistance agreements  are copyrighted, the agency cannot include them in its public online  database. In the agency’s view, EPA would therefore need to develop a  forum for disseminating the documents to EPA staff and determine  whether other federal employees needed access to the documents for  federal government purposes. EPA officials also said that digitizing these  documents was constrained by several factors, including agency priorities  for which documents are to take precedence and efforts to identify which  of the many types of assistance programs are likely to produce documents  of most value to EPA staff. We have clarified the wording of our  recommendations to eliminate any implied reference that EPA should  make the copyrighted documents available to the general public in its  online database, and we maintain that making copyrighted documents  resulting from assistance agreements available solely for federal  government purposes is permissible, feasible, and desirable.
EPA’s written comments appear in appendix II. EPA also provided  technical corrections, which we incorporated.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the  appropriate congressional committees, Administrator of EPA, and other  interested parties. in addition, this report will be available at no charge on  the GAO Web site at http://www.gao.gov.
If you or your staff members have any questions on this report, please  contact me at (202) 512-3841 or stephensonj@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to  this report are listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To complete our work, we reviewed relevant Environmental Protection  Agency (EPA) funding and inventory documents, policies, plans, guidance,  and procedures, as well as related regulations and requirements pertinent  to the library network and efforts to improve its operations. We limited  our review to the 26 libraries belonging to EPA’s library network, that is,  libraries that are members of the Online Computer Library Center system.  We focused on EPA’s headquarters library, the 10 regional libraries funded  in part by EPA’s Office of Environmental Information, and the Office of  Administration and Resources Management libraries in Cincinnati, Ohio  (which is responsible for EPA’s digitization and Web site maintenance),  and Research Triangle Park, North Carolina. We compared library  operations before, during, and after attempted reorganization in fiscal year  2007; obtained and reviewed library network policy and procedures;  reviewed the agency’s draft outline of a strategic plan for the library  network; obtained and reviewed documents on EPA’s digitization process;  and reviewed EPA’s efforts to communicate with and solicit input from  users. We interviewed EPA librarians and library managers in selected  EPA libraries, as well as Office of Environmental Information officials  knowledgeable about EPA’s library network and budget; when possible,  we corroborated information provided to us during interviews with  relevant documentation. We also interviewed management officials from  the federal employees’ union representing EPA staff and spoke with  representatives from EPA’s regional science councils, which consist of  EPA scientists and technical specialists. We further sought information  from library professionals, including representatives from the Library of  Congress; the National Agriculture Library; and, through visits and  interviews, from Lockheed Martin and Integrated Solutions and Services,  contractors involved in digitizing EPA documents.
In addition, we obtained information on library funding from each of the  26 libraries in the network from fiscal year 2002 to fiscal year 2010.  Because EPA does not specifically track funding for the libraries, the  information provided contained a mix of outlays for some fiscal years and  budget authority for other fiscal years. In addition, the information  provided by each of the libraries reflected only spending by the library and  not funding sources. For example, a large portion of funding for regional  office libraries comes from the Office of Environmental Information, but  these libraries also receive funding from other EPA program offices, such  as Superfund. Also, funding data from the libraries contained a mix of  funding for contract support; library staff salaries; and acquisition costs  for books, journals, and other materials. We interviewed EPA budget  officials to assess data reliability and performed a limited test to verify the  accuracy and completeness of the data provided by the libraries. On the  basis of this test and discussions with EPA officials, we concluded that the  data were not reliable enough to include in our report.
We also obtained data on the number of EPA and other documents that  have already been digitized and the number still to be digitized. After  limited testing and discussions with EPA officials, we determined that  EPA’s data on library funding and on the number of digitized documents  and those scheduled to be digitized were not sufficiently reliable for our  purposes. Because these data were the only data available, however, we  used them to some extent, noting their limitations in our report as  appropriate. We also reviewed documents about EPA’s digitization  process, guidance on what documents should or should not be digitized,  and digitization contracts, and we discussed the contents of these  documents with EPA and digitization-contractor officials. We also  discussed EPA’s future digitization plans with Office of Environmental  Information officials.
In addition, we assessed EPA’s survey of library users, examining the  adequacy of the survey methodology, including response rate, sampling  methodology, security measures, survey questions, and processes. To  determine the adequacy of the response rate to EPA’s survey, we followed  an 80 percent response rate as a criterion, as Office of Management and  Budget guidance recommends and we apply in our own surveys to  increase the likelihood of sufficiently representing a universe of  respondents. For surveys with response rates lower than 80 percent, we  also perform an analysis to determine the existence of nonresponse bias.  To generate its survey sample, however, EPA relied on self-selection, using  a Web site to make the survey available to approximately 17,000 EPA staff;  the response rate achieved was 14 percent. We performed a limited  nonresponse analysis of EPA’s survey data and determined that some  staffing categories were represented in proportions different from those  found in the population of EPA staff. Given the 14 percent response rate to  EPA’s survey, the nonrandom methodology that generated the sample, and  the results of our limited analysis for nonresponse bias, we found EPA’s  survey results to be inadequate for EPA’s purpose of obtaining a  representative view of EPA library users. We also interviewed local union  representatives from headquarters and some of EPA’s regional offices.  Furthermore, we interviewed regional science council representatives  from some of the regional offices. The science councils are located in each  regional office and consist of EPA scientists and technical specialists. To  determine the extent to which EPA communicated with and solicited  views from outside stakeholders, we interviewed representatives from  several professional library associations and other external stakeholder  groups, such as the American Library Association, the Library of Congress,  the Federal Library and Information Center Committee, and the Union of  Concerned Scientists. We also reviewed information EPA provided to the  public via the EPA Web site or, when applicable, Federal Register notices.
We conducted this performance audit from October 2009 through  September 2010, in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Environmental Protection Agency
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact above, Ed Kratzer, Assistant Director; Ellen W.  Chu; Pamela Davidson; Les Mahagan; John C. Martin; Ben Shouse; and  Jeannette Soares made key contributions to this report.