NTIS’s basic statutory function is to collect research reports, maintain a bibliographic record and permanent repository of these reports, and disseminate them to the public. In addition, NTIS has developed a variety of information-related services. NTIS charges user fees for the sale of its products to the public and services to federal agencies. Under statutory authority enacted in 1950, NTIS collects reports containing scientific, technical, and engineering information from both domestic and foreign sources in a repository and makes the information available to (1) business and industry, (2) state and local governments, (3) other federal agencies, and (4) the general public to increase U.S. competitiveness in the global economy. The statute does not define scientific, technical, and engineering information. However, the Secretary of Commerce has interpreted this to include “all types of information which have a more or less direct bearing on business and industry generally.” including “economic information, market information, and related information so long as it is reasonably specific and bears some direct relationship to the organization and operation of industrial or business enterprise.” NTIS’s enabling legislation authorized it to charge fees for its The Secretary of Commerce described such information as products and established a policy to recover all costs, as feasible, through the fees. NTIS’s authority was revised in the National Technical Information Act of 1988. cooperative agreements, joint ventures, and other transactions as necessary in the conduct of the business of NTIS, and declared the NTIS repository to be a permanent federal function that could not be transferred to the private sector without congressional approval. The act was subsequently amended by the American Technology Preeminence Act of 1991, This act gave the agency authority to also enter into contracts, required all costs associated with acquisition, processing, storage, bibliographic control, and archiving to be recovered primarily by fees; required agencies to transfer unclassified scientific, technical, and engineering information which results from federally funded research and development to NTIS; and provided that NTIS’s use of new methods or media for information dissemination should include producing and disseminating information in electronic format. Further, the Commerce, Justice, and State, the Judiciary, and Related Agencies Appropriations Act of 1993 established a revolving fund for the payment of all expenses incurred by NTIS and gave it the authority to use that fund without further appropriations action by Congress. Pub. L. No. 100-519 (Oct. 24, 1988); 15 U.S.C. 3704b. million. Accordingly, in August 1999, the Secretary of Commerce proposed closing NTIS by September 30, 2000, because he believed that declining sales revenues would not continue to be sufficient to recover all of the agency’s operating costs. The Secretary attributed this decline partly to other agencies’ practice of making their research results available to the public for free through the Web. He also proposed transferring NTIS’s research report archives to the Library of Congress and requiring federal agencies to give the public free online access to new research reports. GAO-01-490. consideration that Congress look at how this information was defined; whether there was a need for a central repository of this information; and if a central repository was maintained, whether all information should be retained permanently, and what business model should be used to manage it. The Secretary of Commerce agreed with our assessment and raised as a primary question whether there was a need for a central repository in view of the increasing availability of newer publications from sources other than NTIS. The Secretary also noted that the need for a central repository depended on whether the information would be permanently maintained by agencies and whether the information would be easy to locate without the kind of bibliographic control that NTIS provides. Subsequent to the issuance of our reports, in December 2003, Congress passed the 21st Century Nanotechnology Research and Development Act, which provided a coordinated federal approach to stimulating nanotechnology research and development. The act directed the Secretary of Commerce to establish a clearinghouse for information related to the commercialization of nanotechnology research using the resources of NTIS to the extent possible. As of September 2012, NTIS noted that it held over 700 publications in its nanotechnology collection. NTIS currently operates as one of 12 independent bureaus within Commerce, with the mission to help promote the nation’s economic growth by providing access to information that stimulates innovation and discovery. Headquartered in Alexandria, Virginia (with warehouses in Alexandria, Virginia and in Brandywine, Maryland), the agency is organized into five primary offices. To carry out its statutory functions of collecting and maintaining a permanent repository and bibliographic record of research reports, and to disseminate them, the agency offers a variety of products, such as fee-based access to the reports in its repository. In addition, NTIS offers information-related services to federal agencies, such as distribution and order fulfillment, web hosting, and e- training, that are less directly related to its basic statutory function. While NTIS’s service offerings have resulted in increased revenues, allowing the agency to remain financially self-sustaining, it has experienced a net cost relative to its products, calling into question whether the agency’s function of acting as a self-financing repository of technical information is still viable. NTIS is led by a director, who is aided by two executives—a chief information officer and a chief financial officer. In addition, three operational offices have a variety of responsibilities for providing products and services that include collecting and disseminating technical reports, offering access to other information sources, and providing information- related services to federal agencies. Figure 1 displays the NTIS organization. NTIS operates as a unit within Commerce and receives oversight from the Deputy Secretary of Commerce, the Director of NIST,advisory board. In this regard, the NTIS Director communicates progress toward agency goals to the Deputy Secretary of Commerce. For example, the Director participates in biweekly executive management meetings that are held with the Deputy Secretary. At these meetings, executive leads from each of Commerce’s components report on the status of performance and strategic goals within their offices, among other things. and an In addition, under a Commerce departmental order,also reports to the Director of NIST. As outlined in the Commerce organization chart and stated in the Commerce departmental organization order, the NTIS Director is to report and be responsible to the Director of NIST. In turn, the Director of NIST prepares the NTIS Director’s annual performance appraisal. Beyond this, the Director of NIST stated that the agency does not provide any other operational or financial oversight functions for NTIS. For example, NIST does not approve NTIS’s budget (although it does coordinate NTIS’s budget for final inclusion in the department’s overall budget, which is approved by Commerce). Further, NTIS receives guidance on its operations from the NTIS Advisory Board, which was established by law policies and operations of NTIS, including policies related to fees and charges for its products and services. The board, comprised of a chairperson and four members appointed by the Secretary of Commerce, is required to meet at least every 6 months to discuss NTIS’s activities. The board’s last meeting took place in late October 2012, and, according to the notice of open meetings from the Federal Register, the intended focus was on the agency’s strategic business plan. The board submits an annual report to the Secretary of Commerce, which includes strategic and tactical recommendations regarding NTIS’s future operations. in 1988 to review the general As of late October 2012, NTIS was supported by 181 staff, all except 6 of which held full-time positions. These included 103 NTIS employees and 78 contractors. Table 1 shows the number of staff dedicated to the Office of the Director and each of the agency’s other primary offices as of October 2012. 15 U.S.C. 3704b(c). Among these offices, executive functions, such as directing NTIS activities and developing NTIS budgets and policies for the use of information technology, are carried out by the Office of the Director, along with the offices of the Chief Financial Officer and the Chief Information Officer. The three other operational offices have a variety of responsibilities for providing NTIS’s products and services. Table 2 summarizes the specific responsibilities and functions of these offices. As part of its basic statutory function to collect and disseminate technical reports, NTIS offers a variety of fee-based products. According to the agency, its customer base for these products include scientists, engineers, the business community, librarians, information specialists in government, academia, and the general public. The agency has organized its products into three lines of business—”Technical Reports,” “Clearinghouse,” and “Publishing”—as described below. To carry out its statutory function of collecting and disseminating such information, NTIS maintains a searchable repository that contains bibliographic records for the over 2.5 million scientific, technical, engineering, and business research reports it has acquired from federal government agencies, state and local governments, international organizations, and private-sector organizations. NTIS’s bibliographic database provides for the classification and cataloging of the records in this database. For example, according to the NTIS Database Search Guide, NTIS classifies its records into 39 categories. These categories can be used for searching the contents of NTIS’s database. The database covers a host of scientific and technical subjects, such as biology, chemistry, physics, transportation, health care, and the environment. Of the 2.5 million reports, NTIS noted that approximately 700,000 reports have been digitized, with the remainder in physical form, such as paper or microfilm. Access to the reports is provided both through the direct sale of individual reports and by subscription. Individual reports can be purchased via postal mail; phone; e-mail; and NTIS’s online ordering system, accessed through its website, http://www.ntis.gov. Subscription-based access to the reports is obtained through, among other things, NTIS’s National Technical Reports Library, which provides subscribers with the ability to search the bibliographic records repository and to access the approximately 700,000 digitized reports in portable document format (PDF). In addition, NTIS’s Selected Research Service allows a subscriber to select from more than 378 subcategories and automatically receive reports tailored to that area of interest. According to NTIS, reports that have not been digitized can be provided in a digital format when a customer purchases a copy. Prices for individual reports and subscriptions vary. For example, an electronic copy of a report from the Economics and Statistics Administration, Benefits of Manufacturing Jobs: Executive Summary, can be purchased for $15; a “customized CD” for this report can be purchased for $30. Further, an electronic copy of a report from the National Aeronautics and Space Administration (NASA) Marshall Space Flight Center, NASA Robotics for Space Exploration, can be purchased for $15; a “customized CD” for this report can be purchased for $30. With regard to subscriptions, access to the National Technical Reports Library is sold as an annual subscription to institutions based on the number of individuals accessing the library. For example, as of October 2010, an annual subscription providing access for up to 3,000 individuals costs $2,100, while an annual subscription providing access for 18,001 to 28,000 individuals costs $11,200. In addition, subscriptions can be purchased for a specific number of issues of a particular document type. For example, six issues of the Livestock, Dairy, and Poultry Outlook can be purchased for $91. In addition to the technical reports that it collects and disseminates, NTIS disseminates publications covering a wide array of topics on behalf of other federal agencies. According to NTIS, these agencies request that NTIS distribute the publications in print or electronically. The following are examples of the federal products distributed: Standard Occupational Classification Manual. A manual containing information on all occupations in the national economy classified according to the system used by federal statistical agencies for the purpose of collecting, calculating, analyzing, or disseminating occupational data. Food and Drug Administration Food Code Manual. A code and reference document that provides technical and legal information about the regulations of the retail and food service industry. North American Industry Classification System. A publication that details a system for the collection, analysis, and dissemination of industrial statistics used by government policy analysts, academics, researchers, the business community, and the public. Export Administration Regulations. A compilation of regulations issued by Commerce’s Bureau of Industry and Security relating to the control of certain exports, re-exports, and activities. National Correct Coding Policy Manual. A manual developed by the Centers for Medicare and Medicaid Services to control improper coding leading to inappropriate payment in Medicare Part B claims. The manual provides guidance for providers on the correct coding of claims being submitted for reimbursement. Through a memorandum of understanding or interagency agreement, NTIS also provides access to information collected from federal agencies, which it refers to as its “Publishing” line of business. In some instances, NTIS repackages the information with additional features. According to NTIS, agencies initiate the request for these services. These offerings include the following: Drug Enforcement Administration database. The Drug Enforcement Administration database identifies persons and organizations authorized to handle certain controlled drug substances and chemicals under the Controlled Substance Act. NTIS is the authorized official distributor of the database. NTIS provides online subscription access to the database on a weekly, monthly, or quarterly basis. NTIS also provides this information via a searchable CD-ROM. Death Master File. This file, maintained by the Social Security Administration (SSA), contains approximately 85 million records of deaths that have been reported since 1936. The file is used by government; credit reporting organizations; and financial, investigative, medical research, and other industries to verify deaths. Through an agreement with SSA, NTIS is the only authorized official distributor of the Death Master File. In this regard, NTIS provides access to this information, including on DVD, and provides the means to search and download the Death Master File online. World News Connection. NTIS offers access to this online news service, which provides translated and English-language news and information from non-U.S. media sources. The information is obtained from full-text, and summaries of newspaper articles, conference proceedings, television and radio broadcasts, periodicals, and nonclassified technical reports. The material in World News Connection is provided to NTIS by the Open Source Center, a U.S. government agency that provides analysis of foreign open source intelligence. In addition to its product offerings, NTIS offers a variety of fee-based services to federal agencies that are less directly related to its basic statutory function of collecting and disseminating scientific and technical information. These include services that leverage capabilities NTIS has developed in the course of carrying out its mission. Its five service offerings are distribution and order fulfillment, web-based services and federal cloud computing, brokerage services, e-training and knowledge management services, and digitization and scanning services. To provide its services, NTIS enters into memorandums of understanding or interagency agreements with agencies. Further, NTIS offers some of these services through public-private partnerships with private industry, which it refers to as “joint venture partners.” The five types of services are described below. Through a memorandum of understanding or interagency agreement, NTIS distributes large amounts of information products for federal agencies. According to NTIS officials, these services differ from that provided by its Clearinghouse in that they are used for distributing large quantities of agencies’ products rather than selling individual copies of publications. NTIS identified five primary clients for these services: the Internal Revenue Service, the Department of Agriculture, the Department of Education, the Pension Benefit Guaranty Corporation, and the Administrative Office of the U.S. Courts. For example, NTIS currently has an agreement with the Department of Agriculture to distribute on its behalf health and nutrition educational materials in the form of brochures, posters, and similar nutritional products. It also has an agreement with the Department of Education to perform similar services. In 1988, Congress required NTIS to implement new methods or media for disseminating technical information; a 1992 amendment specified that this should include producing and disseminating products in electronic formats. According to NTIS, this has been a primary basis for NTIS’s transformation from a static paper-based distribution operation to a modern, computer-based model and also a basis for NTIS to provide information-dissemination services to other agencies. For example, NTIS’s information systems infrastructure enables it to host federal agencies’ applications and websites. The agency currently has an agreement with the Federal Insurance and Mitigation Administration to provide hosting for two of its websites and associated systems and to host its web-based tool for reviewing underwriting and claims operations. NTIS has also expanded its infrastructure to provide cloud computing services and, according to the agency, is currently offering infrastructure- as-a-service and software-as-a-service. For example, along with joint venture partner Carney, Inc., NTIS had an agreement with the National Archives and Records Administration and currently has an agreement with the Social Security Administration to configure and host the “Jive” platform. According to NTIS, the agency has eight primary clients for its web-based services and federal cloud computing offerings, including the Department of Homeland Security, the Internal Revenue Service, and other federal agency initiatives. NTIS provides billing and collection services on a reimbursable basis to other agencies that, like itself, charge for products and services but which lack the necessary financial infrastructure to do their own billing and collecting. The agency refers to these as its “Brokerage Services.” For example, NTIS had an agreement with the National Agricultural Library to develop, implement, and operate account maintenance, invoicing, and collection procedures for the fees charged by the National Agricultural Library to users of its photocopy and loan services. In addition, the agency had an agreement with the National Library of Medicine to perform invoicing, accounting, and collection services for its Interlibrary Loan services. NTIS officials stated, however, that the agency plans to stop marketing its brokerage services due to the decrease in demand for this service. NTIS’s service offerings have also been expanded to e-training and knowledge management. Specifically, in conjunction with joint venture partners, the agency provides collaborative software solutions, learning management systems and support services, training evaluation software, and talent management applications. For example, NTIS entered into an agreement with Booz Allen Hamilton to provide, among other things, program management; secure Internet hosting; and operations, maintenance, and support services for the Defense Manpower Data Center’s enterprise training program. According to NTIS, as of May 2012, it had 28 primary government clients for this service offering, including the Departments of Justice, the Interior, and the Treasury, and the U.S. Patent and Trademark Office. As another service, NTIS digitizes various document types, such as microfilm or microfiche and paper forms, to assist agencies in complying with section 508 standards. Further, it offers storage and distribution for the documents that it digitizes. For example, NTIS has an agreement with SSA to provide alternative modes of receiving SSA notices and other communications. NTIS provides this service with the assistance of its joint venture partners, Vastec, Inc. and Braille Works, Inc. NTIS offers this information in data compact disc, large print, and audio compact disc. As of May 2012, the agency had six primary clients for its digitization and scanning services, including SSA and the Department of Justice. As a fee-for-service entity, NTIS’s revenues are generated exclusively from its products and services, and all its revenues, expenses, and capital expenditures are deposited into and paid out of its revolving fund. Overall, NTIS had net earned revenues for 8 of the last 11 fiscal years. For example, for fiscal year 2011, the agency reported that net earned revenues from all its functions (products and services) totaled about $1.5 million. According to NTIS’s Financial Report for fiscal year 2011, the was approximately $7.4 revolving fund ending unobligated balancemillion. However, over most of the last decade, the agency has incurred net costs for its products. Specifically, NTIS product expenditures exceeded revenues for 10 out of the past 11 fiscal years. The agency lost, on average, about $1.3 million over the last 11 years on its products. In contrast, NTIS’s overall financial performance has been supported by revenues from its service offerings. The agency’s service revenues increased, on average, about $1.8 million over the last 11 years. In particular, for fiscal year 2011, revenues were about $53.5 million, costs incurred were about $52 million, and the overall net earned revenue was approximately $1.5 million from its service offerings. NTIS officials attribute most of the net earned revenue to the agency’s agreements with the Departments of Agriculture and Education and the Social Security Administration for various service offerings. Table 3 identifies the net earned revenue or net cost for NTIS products and services over the last 11 fiscal years, as reported by NTIS. In those years in which NTIS had net costs, the agency was sustained by cumulative net earned revenues from previous years’ operations. Figure 2 illustrates the net earned revenues and net costs associated with NTIS’s products and services from fiscal year 2001 through fiscal year 2011. The decline in revenue for its products continues to call into question whether NTIS’s basic statutory function of acting as a self-financing repository and disseminator of scientific and technical information is still viable. This is further highlighted by the fact that the services which are financially sustaining the agency are less directly related to this function. Recognizing its financial stance, NTIS has conducted analyses, and identified in its 2011-2016 Strategic Plan actions to help address net costs from its products, including its technical reports. The plan emphasizes that the agency’s collection, culture, and information technology infrastructure are its main strengths; that continued use of less robust business systems and an aging work force are its primary weaknesses; that growth opportunities still exist in the various sectors served by NTIS, whether through products or services; and that NTIS is threatened by and will have to overcome a shrinking customer base for its products. The plan identifies three strategic initiatives to guide NTIS during this period: 1. Increase revenue by enhancing the number of acquisitions, creating new products, reaching more customers, and adding value to what NTIS collects, and to reduce costs by reviewing and improving key business processes. 2. Improve NTIS’s utilization by other agencies by increasing the breadth and depth of its own collection and enhancing the suite of information management services that it can provide. 3. Achieve workforce excellence by focusing on identifying and acquiring the critical workforce skills required to accomplish the agency’s mission in a rapidly changing world. Beyond these initiatives identified in the strategic plan, the Director of NTIS also provided information on several other initiatives under way to address the budget shortfalls from products. These initiatives include the following product and organizational improvements: Enhancing the accessibility of federal science content by shifting from a pricing model for stand-alone products (e.g., paper/print, microfiche, and compact disk media) to one that is subscription-based. Repositioning NTIS to support open government initiatives in science—meeting with agencies such as NIST, the National Oceanic and Atmospheric Administration, and the Government Printing Office to address how NTIS can reposition its programs to support current science information needs. Building collections of reports based on themes and categories that will be supported through subscriptions. Adjusting the NTIS business model to support the increased demand for subscriptions to customers. Reducing staff—for example, the agency has received authorization to provide early retirement for eligible employees and has stopped hiring additional staff. The agency anticipates employee attrition to further reduce current staffing levels. Notwithstanding these efforts, NTIS could likely continue to face challenges in recouping the costs of its products given the increasing availability of technical information from other sources. From fiscal year 1990 through 2011,repository were older reports published in the year 2000 or before; however, the greater demand was for more recently published reports. In this regard, the agency added 841,502 reports to its repository from 1990 through 2011. Of the reports added, approximately 62 percent, or 524,256, had publication dates of 2000 or earlier, while approximately 38 percent, or 317,246, of the reports were published from 2001 to 2011. Specifically, reports that were added to the repository during this period were as follows: most of the additions to NTIS’s 79,943 reports published in 1989 and prior years, 444,313 reports published from 1990 through 2000, 129,591 reports published from 2001 through 2004, 126,225 reports published from 2005 through 2008, and 61,430 reports published from 2009 through 2011. Figure 3 shows the distribution of reports that were added by each publication date age group from 1990 through 2011. With regard to demand for the reports, we estimate that, during fiscal years 2000 through 2011, NTIS distributed (sold) one or more copies of about 419,657, or almost 50 percent, of the 841,502 reports added to its repository during fiscal years 1990 through 2011. Of these 419,657 reports, approximately 78 percent were distributed through a subscription only. NTIS officials attributed this to the fact that subscriptions are a cost-effective way for libraries to meet their collection development requirements within a specific or broad area of interest. The officials noted that direct sales are generally to customers interested in specific topics. However, as shown in figure 4, the agency was more likely to distribute a higher percentage of more recently published reports than older ones. For example, we estimate that between 96 and 100 percent of the reports published from 2001 through 2011 had been distributed, while only about 21 percent of reports published in 1989 or earlier were distributed during this time period. That is, the demand for older holdings in the NTIS repository is lower than for newer publications. Based on our sample, we estimate that most (about 74 percent) of the reports added to NTIS’s repository during fiscal years 1990 through 2011 were readily available from other public websites, and nearly all of these (95 percent) could be obtained for free. Specifically, we estimate that approximately 621,917, or about 74 percent, of the 841,502 reports added to NTIS’s repository from fiscal years 1990 through 2011 are readily available from one of the other four publicly available sources we searched (i.e., the issuing organization’s website; the Government Printing Office’s Federal Digital System website; the U.S. government’s official web portal, USA.gov; or from another website located through a Google search).searching for was another website located through http://www.Google.com The source that most often had the report we were . In addition, of the reports added since fiscal year 1990, those with more recent publication dates were more likely to be available from other public sources than older ones. For example, approximately 87 percent of the reports with publication dates from 2009 to 2011 were available elsewhere, while 55 percent of those published in 1989 or earlier were. Figure 5 shows the estimated availability of reports added to NTIS’s repository since fiscal year 1990 by date of publication. As shown in figure 6, of the reports that were found to be readily available from one of the other four sources that we searched, about 61 percent of those reports had been distributed (sold) by NTIS. Conversely, of the reports that were not found to be readily available from one of the other four sources, most, or about 82 percent, had not been distributed by NTIS. Not only were most of the reports in our sample available from sources other than NTIS, but about 95 percent of the reports available elsewhere could be obtained free of charge from one of the four other sources we searched. The remaining 5 percent were available from the public sources for a fee. Moreover, the year of publication did not appear to have an effect on whether a report was available free of charge. For example, the following reports available for a fee from NTIS were available free of charge from the issuing organization’s website: Hazardous Waste Characteristics Scoping Study, November 1996, Environmental Protection Agency, 278 pages. (At NTIS, print on demand costs $73, electronic $25.) “Homeland Security: Intelligence Indications and Warning,” December 2002, Naval Postgraduate School, 5 pages. (At NTIS, print on demand costs $17, electronic $15.) Export Controls: System for Controlling Exports of High Performance Computing Is Ineffective, 2000, GAO, 60 pages. (At NTIS, print on demand costs $48, electronic $15.) FDA Enforcement Report: July 20, 2011, July 2011, Food and Drug Administration, 28 pages. (At NTIS, print on demand costs $33, electronic $15.) Principal Rare Earth Elements Deposits of the United States: A Summary of Domestic Deposits and a Global Perspective, 2010, Geological Survey, 104 pages. (At NTIS, print on demand costs $60, electronic $25.) Of those reports we found available elsewhere, figure 7 shows the estimated percent that were available elsewhere for free by year that the document was published. The Director of NTIS acknowledged two factors that contributed to the free and public availability of reports from other sources: Federal agencies are providing information, including their federal scientific, technical, and engineering information products, on their websites in electronic format and on central federal information websites such as http://www.data.gov and http://www.science.gov. The information is available for immediate download at no cost. Federal agencies are participating in programs with Internet search engines that permit the public to locate their products for free or for less than they are when purchased from NTIS. In addition, commercial vendors are also able to obtain these information products from agency websites or through Internet search and are able to make these products available for free or at a price lower than that offered by NTIS. Further, NTIS acknowledged in its strategic plan that because the Internet continues to change the way people acquire and use information and permits federal agencies to make their information products available for free, NTIS is challenged to meet its statutory mandate as a self-financing repository and disseminator of technical information. Notwithstanding these acknowledgments, NTIS continues to charge for reports that are freely available from other public sources. NTIS serves as a permanent repository and disseminator of technical information, and by statute, is required to be financially self-sustaining, to the fullest extent feasible, by charging fees for its products and services. While the agency had cumulative net earned revenues as of September 30, 2011, its costs exceeded revenue by an average of about $1.3 million over the last 11 years from the sale of technical information. The agency’s net revenue now comes primarily from services that are less directly related to its basic statutory function. The decline in sales of technical information is due in part to the increasing availability of this information from other sources, including websites and Internet search tools, and often at no charge. Charging for information that is freely available elsewhere is a disservice to the public and may also be wasteful insofar as some of NTIS’s customers are other federal agencies. Taken together, these considerations suggest that the fee-based model under which NTIS currently operates for disseminating technical information may no longer be viable or appropriate. In light of the agency’s declining revenue associated with its basic statutory function and the charging for information that is often freely available elsewhere, Congress should consider examining the appropriateness and viability of the fee-based model under which NTIS currently operates for disseminating technical information to determine whether the use of this model should be continued. The Acting Secretary of Commerce provided written comments on a draft of this report, which are reprinted in appendix II. In its comments, Commerce expressed appreciation for our study and for our focus on the initiatives that NTIS has undertaken. However, the department said NTIS did not believe our conclusions (that the fee-based model under which it operates for disseminating technical information may no longer be viable or appropriate) fully reflect the additional value that NTIS provides with the work that it performs. Commerce stated that, through its federal clearinghouse and repository, the agency provides federally funded reports that are not otherwise readily available, such as most of those issued prior to l989. Additionally, Commerce stated that NTIS recognizes that it cannot remain financially solvent solely through sales and subscriptions of technical reports with expectations that these products will be widely available for free. It added that NTIS is moving in the direction of “open and improved access” to information, but recognizes that it needs to maintain a sustainable financial model and continue providing enhanced value to the information generated by other federal agencies. In this regard, Commerce described features that it believes have added value to the technical reports that NTIS maintains, how these features improve access to the documents and related data, and specific information services that the agency provides to science information professionals. Our report highlighted various initiatives that NTIS has undertaken to provide older reports that might not otherwise be readily available, and to increase the value of its technical reports, information management services, and technology transfer capabilities. However, as discussed in the report, we found that the demand for older holdings in the agency’s repository is lower than for new publications. For example, we estimate that between 96 and 100 percent of the reports published from 2001 through 2011 had been distributed, while only about 21 percent of reports published in l989 or earlier were distributed during this time period. Further, as the agency acknowledged, its financial health is based on both its information product and service missions. Also, as we state in our report, the agency’s net revenue now comes primarily from services that are less directly related to its basic statutory function, while sales of its technical information products have resulted in net losses. This decline in sales of NTIS’s technical reports is due in part to the increasing availability of this information from other sources, including websites and Internet search tools, and often at no charge. With these factors in mind, we stand by our conclusion that the fee-based model under which NTIS currently operates for disseminating technical information may no longer be viable or appropriate. NTIS also provided technical comments on the report via e-mail, which we have incorporated as appropriate. We are sending copies of this report to interested congressional committees. We are also sending copies to the Secretary of Commerce; the Director, NTIS; the Director, NIST; and other interested parties. Copies of this report will also be available at no charge on GAO’s website at http://www.gao.gov. Should you or your staffs have any questions on information discussed in this report, please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III. Our objectives were to determine (1) how the National Technical Information Service (NTIS) is currently organized and operates, including its various functions, current staffing level, reported cost of operations, and revenue sources; (2) the age of and demand trends for reports added to NTIS’s repository; and (3) the extent to which these reports are readily available from other public sources. To determine how NTIS is organized and operates, we reviewed the agency’s strategic plan, and documentation on its organizational and reporting structure, office staffing level and assigned responsibilities, and types of products and services offered. We also reviewed cost data contained in the agency’s annual financial reports. In addition, we reviewed relevant laws and regulations on NTIS’s authority and responsibilities and our previous reports that discussed its mission and operations. We supplemented our analyses with interviews of the Director of NTIS and other relevant agency officials; we also interviewed officials of the Department of Commerce and its National Institute of Standards and Technology (NIST), which have specific reporting relationships with NTIS. We did not include reports added for fiscal year 2012 because our study only focused on those fiscal years that had been completed at the time that our study was initiated. sample size of 384 reports. All of the estimates made with this sample are weighted to reflect the stratified design. NTIS provided us with the full bibliographic data for each document in our sample. To determine the age of reports added to NTIS’s repository since fiscal year 1990, we used the year of publication for the reports in our sample to estimate the age range (prior to 1990; 1990-2000; 2001-2004; 2005- 2008; and 2009-2011) for all documents added from fiscal year 1990 through fiscal year 2011. To determine the demand trends for reports added to NTIS’s repository during fiscal years 1990 through 2011, we requested the sales data from fiscal year 2000 through fiscal year 2011 for the 384 reports in our stratified sample. NTIS provided the distribution data for both direct sales and subscriptions for all of these documents. We then used our sample and the sales data to estimate the extent and prevalence of the sales among all reports added to the NTIS repository. Specifically, we used these data to estimate the (1) total number of reports distributed by direct sale and subscription, (2) total number of reports distributed one or more times, (3) percentage of reports distributed relative to the year the reports were published, and (4) percentage of reports distributed relative to their availability elsewhere. The observations were statistically weighted in the estimation process to reflect the stratified sample design that we used. To determine the extent to which reports in the repository are readily available from other public sources, we first developed a methodology for conducting systematic Internet searches to determine availability elsewhere. More specifically, as part of this methodology, we searched the Internet to determine if each of the reports included in our sample of 384 reports could be found elsewhere and at no cost. Using a tiered approach, we searched the following four sources in the order shown: (1) the issuing organization’s website; (2) the U.S. Government Printing Office’s Federal Digital System website—http://www.gpo.gov/fdsys; (3) a web search conducted using the federal government Internet portal USA.gov—http://www.USA.gov; and (4) a web search conducted using Specifically, with the commercial search engine http://www.Google.com.this methodology, we determined whether each report was first available at no cost on the issuing organization’s website and, if so, concluded the Internet search at this point. However, if the report was not available, then the search continued to the second source, and so on, until either the report was found to be available at one of the remaining sources, or all sources were exhausted. We then used our results to estimate the percentage of the total population of NTIS reports added to the repository during fiscal years 1990 through 2011 that was available from other public sources. All of the results derived from the sample analyses constituted estimates that are subject to sampling errors. These sampling errors measure the extent to which samples of this size and structure are likely to differ from the population they represent. Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results as a 95 percent confidence interval. This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. To determine the reliability of the data provided from NTIS’s repository of reports, we performed basic steps to ensure the data provided were valid, and reviewed relevant information describing the database supporting the repository. We tested for duplicate records, missing values, and out-of- range values in the data received from NTIS. We did not assess the reliability of the system used to maintain these data or the processes used in extracting the data for our engagement purposes. To determine the reliability of the sales data provided by NTIS, we conducted interviews with agency officials to gain an understanding of the process by which accounts receivable records are added and managed within NTIS’s system of accounts receivable–”CIS.PUB.” Further, we asked cognizant agency officials specific questions to understand the controls in place for ensuring the integrity and reliability of the data contained in CIS.PUB. In addition, we met with NTIS officials to discuss data collected from NTIS and obtained their assertions regarding the data it provided. Based on the results of these efforts, we found the data sources to be sufficiently reliable, given the way they are reported herein. We conducted this performance audit from February 2012 to November 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In addition to the contact named above, Cynthia Scott (Assistant Director), Carl Barden, Virginia Chanley, Elena Epps, Nancy Glover, Alina J. Johnson, Lee McCracken, Constantine Papanastasiou, David Plocher, Bradley Roach, and Tina Torabi made key contributions to this report.

Summary: NTIS was established by statute in 1950 to collect scientific and technical research reports, maintain a bibliographic record and repository of these reports, and disseminate them to the public. NTIS charges fees for its products and services and is required by law to be financially self-sustaining to the fullest extent possible. GAO was mandated by Congress to update its 2001 report on aspects of NTIS's operations and the reports in its collection. Specifically, GAO's objectives were to determine (1) how NTIS is currently organized and operates, including its functions, current staffing level, reported cost of operations, and revenue sources; (2) the age of and demand trends for reports added to NTIS's repository; and (3) the extent to which these reports are readily available from other public sources. To do this, GAO reviewed agency documentation, analyzed a sample of reports added to NTIS's collection from fiscal years 1990 through 2011 (reports from the period since GAO's last study and other older reports), and interviewed relevant agency officials. As a component of the Department of Commerce, the National Technical Information Service (NTIS) is organized into five primary offices that offer the public and federal agencies a variety of products and services. As of late October 2012, NTIS was supported by 181 staff, all except 6 of which held full-time positions. NTIS reports its progress toward agency goals to the Deputy Secretary of Commerce, and the Director of NTIS reports to the Director of Commerce's National Institute of Standards and Technology. In addition, NTIS receives oversight of its functions and strategic direction from an advisory board with members appointed by the Secretary of Commerce. NTIS's product and service offerings include, among other things, subscription access to reports contained in its repository in both print and electronic formats, distribution of print-based informational materials to federal agencies' constituents, and digitization and scanning services. NTIS revenues are generated exclusively from direct sales or subscriptions for its products and services. NTIS reported that net revenues from all its functions (products and services) totaled about $1.5 million in fiscal year 2011. However, over most of the last 11 years, its costs have exceeded revenues by an average of about $1.3 million for its products. While NTIS has not recovered all of its costs for products through subscriptions and other fees, it has been able to remain financially self-sustaining because of revenues generated from its services such as distribution and order fulfillment, web hosting, and e-training. The NTIS strategic plan states that the electronic dissemination of government technical information by other federal agencies has contributed to reduced demand for NTIS's products. As a result, the agency is taking steps to reduce its net costs, such as improving business processes and increasing the breadth and depth of its collection. NTIS's repository has been growing with mostly older reports, but the demand for more recent reports is greater. Specifically, NTIS added approximately 841,500 reports to its repository during fiscal years 1990 through 2011, and approximately 62 percent of these had publication dates of 2000 or earlier. However, the agency was more likely to distribute (by direct sale or through a subscription) reports published more recently. For example, GAO estimated that 100 percent of the reports published from 2009 through 2011 had been distributed at least once, while only about 21 percent of reports published more than 20 years ago had been. Of the reports added to NTIS's repository during fiscal years 1990 through 2011, GAO estimates that approximately 74 percent were readily available from other public sources. These reports were often available either from the issuing organization's website, the federal Internet portal (http://www.USA.gov), or from another source located through a web search. Reports published from 1990 to 2011 were more likely to be readily available elsewhere than those published in 1989 or earlier. Further, GAO estimated that 95 percent of the reports available from sources other than NTIS were available free of charge. NTIS's declining revenue associated with its basic statutory function and the charging for information that is often freely available elsewhere suggests that the fee-based model under which NTIS currently operates for disseminating technical information may no longer be viable and appropriate. GAO is suggesting that Congress reassess the appropriateness and viability of the fee-based model under which NTIS currently operates for disseminating technical information to determine whether the use of this model should be continued. In comments on a draft of this report, the Department of Commerce stated that NTIS believes GAO's conclusions do not fully reflect the value that the agency provides. However, GAO maintains that its conclusions and suggestion to Congress are warranted.