High Performance Computing and Communications: New Program Direction Would Benefit from a More Focused Effort
Published: 1994-11-04T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed the status of the High Performance Computing and Communications (HPCC) program, focusing on: (1) the effectiveness of the program's management structure in setting goals and measuring progress; and (2) how extensively private industry has been involved in program planning and execution.
section_title: What GAO Found
paragraphs: GAO found that: (1) the Administration is broadening the HPCC role in developing new technology in support of the National Information Infrastructure (NII); (2) industry and academic researchers believe that specific technology areas will need to be targeted to develop support for NII; (3) a more focused HPCC management approach could help ensure that program goals are met; (4) a detailed technical agenda will be needed to identify HPCC priority areas and commit resources to them; (5) inconsistent budget information has made tracking HPCC investments difficult, since participating agencies have diverse methods of identifying and categorizing their HPCC spending; (6) industry participation in HPCC is more important now that the Administration has linked HPCC to the planned NII; and (7) industries that could capitalize on HPCC technologies to create new products and services for NII should be better represented among HPCC program participants.

section_title: Letter
paragraphs: This report responds to your request that we evaluate the status of the federal High Performance Computing and Communications (HPCC) program. Specifically, you asked that we assess (1) the effectiveness of the program’s management structure in setting goals and measuring progress and (2) how extensively private industry has been involved in the planning and execution of the program. The report identifies opportunities to improve the program and makes recommendations to the Director of the Office of Science and Technology Policy to focus the program more explicitly on its new role in developing new technology in support of the national information infrastructure.
We are sending copies of the report to appropriate congressional committees, the Director of the Office of Science and Technology Policy, the Secretaries of the Departments of Defense and Energy, the Administrator of the National Aeronautics and Space Administration, and the Directors of the National Science Foundation and the National Coordination Office for High Performance Computing and Communications. Copies will also be made available to others upon request.
This work was performed under the direction of David O. Nellemann, Director, Information Resource Management/National Security and International Affairs, who can be reached at (202) 512-2666. Other major contributors are listed in appendix II.
subsections: section_title: Introduction
subsections: section_title: Background
paragraphs: The federal High Performance Computing and Communications (HPCC) program began in fiscal year 1992 as a joint effort among nine federal agencies to significantly accelerate the availability and utilization of the next generation of high performance computers and networks. The overall goals of the program are to extend U.S. technological leadership in high performance computing and provide wide dissemination and application of the technologies to speed the pace of innovation and to improve national economic competitiveness, national security, education, health care, and the global environment; and provide key parts of the foundation for the national information infrastructure (NII) and demonstrate selected NII applications.
Four agencies—the Advanced Research Projects Agency (ARPA), the Department of Energy (DOE), the National Aeronautics and Space Administration (NASA), and the National Science Foundation (NSF)—developed the original program plan for HPCC in 1989, and they remain the program’s dominant participants. In fiscal year 1995 these agencies together will spend more than $900 million, or 81 percent of the official budget. Led by the White House Office of Science and Technology Policy (OSTP), programs at each of these agencies were drawn together to form the governmentwide HPCC program. Ten federal agencies currently participate in the HPCC program.
OSTP has designated the National Science and Technology Council (NSTC) to oversee the HPCC program, through its Committee on Information and Communications. The NSTC, a cabinet-level organization created by the President in November 1993, is intended to serve, in part, as a mechanism for coordinating research and development strategies across the government and for monitoring agency research and development spending plans. Since the NSTC has only recently been established, it is still too early to gauge its impact on the HPCC program. Table 1.1 presents an overview of reported HPCC spending to date and budgeted amounts for fiscal year 1995 by participating agency.
The 1989 program plan laid out the original framework and parameters for the government’s investment in HPCC, proposing that the program grow in even increments from a base of approximately $500 million to approximately $1.1 billion in its fifth year. In budgeting for the actual program, HPCC managers have adhered closely to these original targets. Spending is anticipated to continue at over $1 billion annually until 1998.
To date, the HPCC program and its predecessor agency programs have been highly successful. Participating agencies have been instrumental in establishing more than a dozen high performance computing research centers throughout the U.S. Efforts to provide nationwide access to these centers through interconnected high-speed data networks have led to dramatic increases in the use of those networks. The computing research centers and networks have, in turn, allowed scientists to make significant advances in addressing the highly complex, scientific problems that are collectively referred to as “grand challenges.” Grand challenges include such problems as understanding global climate change, analyzing nuclear reactions, and mapping the human genetic structure.
In September 1992, OSTP established a National Coordination Office (NCO) to coordinate the activities of the agencies participating in HPCC and to serve as liaison to Congress, industry, academia, and the public. The office’s director serves part-time; this individual is also director of the National Library of Medicine. The NCO provides administrative support, disseminates information, and chairs coordination meetings attended by officials of the participating agencies. The NCO does not assess agency HPCC programs or provide guidance to the agencies on their programs. It also does not review or have approval authority regarding agency HPCC budgets.
section_title: Agency HPCC Programs Vary Considerably
paragraphs: Since HPCC is structured as a consortium of federal research agencies with independent programs and budgets, participating agencies can—and do—have widely varying approaches to research and development. The programs of the four major participants reveal the diversity of these agencies’ approaches. ARPA and NSF—the major participants in terms of expenditures—are quite different from NASA and DOE. ARPA and NSF concentrate more heavily on basic research, although all four agencies fund scientists working on practical applications of HPCC technologies.
ARPA has been at the forefront of research into critical technologies, such as computer time-sharing, computer graphics, computer networks, and artificial intelligence, for many years. The agency has had a high performance computing program since the early 1980s. ARPA funds some 200 or more HPCC projects, most of which are relatively small-scale efforts costing between $100,000 and $500,000. Having no laboratories or centers of its own, ARPA funds projects that are run half by academic researchers and half by industry and other government researchers. It also funds the placement of HPCC computers and networks at research sites for use on a variety of research problems.
NSF, like ARPA, funds a large number of relatively small-scale research projects in a wide range of scientific disciplines. NSF also is similar to ARPA in providing HPCC computing and communications infrastructure for a range of research uses. NSF does this by providing base funding for four national supercomputer centers that, in turn, support research in a range of disciplines, such as biotechnology, global change studies, and manufacturing design.
NASA and DOE, in contrast to ARPA and NSF, are involved in HPCC primarily because of the potential for HPCC technology to enhance their ability to carry out agency missions. NASA’s projects, for example, are all linked to either (1) design and simulation of aerospace vehicles or (2) earth and space sciences research. Rather than investing heavily in research to design new computer architectures and build new systems, NASA concentrates on the use and evaluation of HPCC systems in the context of its mission needs. DOE similarly emphasizes the role of being an early user of advanced systems and providing feedback to the systems’ developers, rather than attempting to develop new system architectures on its own. Both NASA and DOE have laboratories and centers with extensive HPCC resources. Much of their HPCC funding goes to projects at these sites.
section_title: Recent Changes in Program Direction
paragraphs: In February 1993, the new administration issued a document outlining its strategy for investing in advanced technology. In the document, the administration rejected the traditional approach of limiting the federal government’s technology development spending to support of basic science and mission-oriented research in the Department of Defense (DOD), NASA, and other agencies. The document stated that challenges facing the U.S. were too profound to rely on the government’s investments in defense and space technology to trickle down to the private sector. Instead, it called for direct support of private sector technology development efforts.
In keeping with this new thinking, the administration sought to align the HPCC program more closely with broader applications that could be developed and commercialized in the private sector. Specifically, HPCC was linked to the development of a national information infrastructure (NII). OSTP envisions the NII, which is commonly referred to as the “information superhighway,” as a nationwide infrastructure of high performance computing hardware and massive computer databases, all linked together by high-speed communications networks and new software that allows trained users to access and use the information contained therein.
The HPCC program’s technology support for the NII is contained in a new program component added for fiscal year 1994, called Information Infrastructure Technology and Applications (IITA). The new component is intended to (1) develop the technology base for the NII and (2) work with industry in using this technology to develop and demonstrate new applications for the NII. The IITA component is also expected to broaden the market for HPCC technologies and accelerate industry development of the NII.
section_title: HPCC’s Technical Approach
paragraphs: In addition to the new IITA component, the HPCC program includes efforts undertaken in four other broad areas described below.
subsections: section_title: High Performance Computing Systems
paragraphs: The High Performance Computing Systems component concentrates on the development of the underlying technology required to build scalable,parallel computer systems capable of sustaining trillions of operations per second on large problems. Most traditional computers have one computational processor, and traditional computer development has focused on making this processor faster and more efficient. However, the potential for continued increases in speed is reaching the limits imposed by the physical properties of the materials used to build the processor. Consequently, an entirely new kind of computer design is needed if speed and performance improvements are to continue.
Computer scientists see development of parallel processing systems as the only way to achieve the dramatic improvements in computer speed that will be needed to address large, complex scientific problems. Parallel processing means breaking computational problems into many separate parts and having a large number of processors tackle those parts simultaneously. Greatly increased processing speed is achieved largely through the sheer number of processors operating simultaneously, rather than through any exceptional power in each processor. Massively parallel processing (MPP) refers to large machines that include many cooperating processors. Other approaches to parallel processing include clustering large numbers of independent workstations together or developing ways to link together a number of completely different computer systems to address a single complex problem in parallel.
section_title: Advanced Software Technology and Algorithms
paragraphs: The primary justification for developing increasingly more powerful parallel computer systems is to address the large, complex scientific problems, commonly known as the “grand challenges.” The grand challenges are fundamental problems in science and engineering that require significant increases in computational capability to address, such as predicting global climate change or testing advanced aircraft designs. The Advanced Software Technology and Algorithms component of the HPCC program targets software development to make MPP and other high performance computer systems useful in addressing grand challenges.
Radically new system software and software tools are needed to operate MPP and other parallel systems. Most potential users have not yet adopted parallel systems because of the high cost and risk of developing software for their specific applications, and because system software for current parallel systems is still rather primitive. Major workshops on HPCC software convened in 1992 and 1993 agreed that greater focus on research to improve system software and software tools is critical if the HPCC program is to succeed.
section_title: National Research and Education Network
paragraphs: The National Research and Education Network segment of the program focuses on the development of a national high-speed communications infrastructure to enhance the ability of U.S. researchers and educators to perform collaborative research and education activities, regardless of their physical location or available local computational resources. This segment has two parts: (1) development of an interagency internetwork and (2) gigabit research and development. The interagency internetwork program will upgrade the networks of participating agencies to higher speeds than are currently available and ensure their interconnection. The gigabit research and development program will develop new high-speed communications technologies through basic research and through experimentation with testbed networks located at various sites around the country.
section_title: Basic Research and Human Resources
paragraphs: The Basic Research and Human Resources segment supports long-term research by individual investigators in scalable high performance computing. It is also intended to increase the pool of trained personnel by enhancing education and training in HPCC. Finally, this segment provides computing and communications resources needed to support these research and education activities.
section_title: Applications
paragraphs: With the recent addition of the IITA component, the HPCC program now targets two kinds of applications as ultimate beneficiaries of the technology being developed in the program. Program managers refer to these two groups of applications as “grand challenges” and “national challenges.”
Grand challenges, mentioned above, are aimed primarily at the scientific research community. National challenges, on the other hand, are defined in HPCC program documentation as major societal needs that HPCC technology can address, such as the civil infrastructure, digital libraries, education and lifelong learning, energy management, the environment, health care, manufacturing processes and products, national security, and public access to government information. While grand challenges address complex scientific questions, national challenges involve making use of large stores of data and information to enhance everyday activities. The national challenges are an identified subset of the wide range of potential applications that may be developed for the NII.
section_title: Objectives, Scope, and Methodology
paragraphs: In April 1993, the House Committee on Armed Services requested that we evaluate the status of the HPCC program. On the basis of subsequent discussions with committee staff, our specific objectives were to assess (1) the effectiveness of the program’s management structure in setting goals and measuring progress and (2) how extensively private industry has been involved in the planning and execution of the program.
To meet our objectives, we reviewed official HPCC program documentation of the participating agencies and the NCO. We also reviewed the administration’s statements regarding technology policy and the creation of the National Information Infrastructure. We discussed these issues with government, academic officials, and private industry from a broad range of organizations.
Specifically, with regard to the program’s management, we interviewed government officials at
Office of Science and Technology Policy, Executive Office of the President, Washington, D.C.,
National Economic Council, Executive Office of the President,
Office of Management and Budget, Executive Office of the President,
National Coordination Office for HPCC, National Library of Medicine,
ARPA, Computing Systems Technology office, Arlington, Virginia,
Department of Energy, Office of Energy Research, Gaithersburg, Maryland,
NASA, High Performance Computing and Communications Office,
National Science Foundation, Directorate for Computer and Information Science and Engineering, Washington, D.C.,
National Security Agency, Ft. Meade, Maryland, and
National Institutes of Health, Bethesda, Maryland.
We also interviewed officials from government laboratories, including
Oak Ridge National Laboratory, Oak Ridge, Tennessee,
Cornell Theory Center, Ithaca, New York,
National Center for Supercomputer Applications, Urbana-Champaign,
Pittsburgh Supercomputing Center, Pittsburgh, Pennsylvania, and
San Diego Supercomputing Center, San Diego, California.
We interviewed members of the academic community from
National Research Council, Computer Science and Telecommunications Board, Washington, D.C.,
Syracuse University, Syracuse, New York,
California Institute of Technology, Pasadena, California,
Rice University, Houston, Texas,
University of Washington, Seattle, Washington,
Stanford University, Stanford, California,
University of California, Berkeley, California, and
University of Colorado, Boulder, Colorado.
Regarding industry’s participation in the program, we reviewed reports prepared by industry associations and interviewed representatives of these associations, including
Computing Research Association, Washington, D.C.,
EDUCOM, Washington, D.C.,
American Electronics Association, Washington, D.C., Information Technology Association of America, Washington, D.C., and
Computer Systems Policy Project, Washington, D.C.
We also interviewed industry officials representing
Electronic Data Systems Corporation,
Eastman Kodak Company,
Microelectronics and Computer Technology Corporation,
Boeing Computer Services,
Visual Numerics, Inc.,
Tera Computer Company,
Schlumberger Well Services,
General Motors Research Corporation,
MasPar Computer Corporation,
Silicon Graphics, Inc.,
Sun Microsystems, Inc.,
Eli Lilly & Company,
Cray Research, Inc.
A detailed audit of the funding of the HPCC program was beyond the scope of this review. Accordingly, we did not attempt to determine the appropriateness of funding for any specific HPCC projects or the merits of proposals that have not been funded. However, we did collect budget information from each of the six agencies included in the review in order to assess the program’s management processes for tracking and reporting how funds are spent.
We conducted our review from May 1993 to June 1994, in accordance with generally accepted government auditing standards. The Assistant to the President for Science and Technology provided written comments on a draft of this report. These comments are presented, along with our evaluation, in appendix I.
section_title: New Program Direction Could Benefit From a More Focused Management Approach
paragraphs: The HPCC program is a loosely coordinated group of research and development activities sponsored by a variety of federal agencies. To date, the program’s broad technical goals have been driven by scientists’ need for ever-increasing computer power to address the grand challenges. Now, however, the administration is also counting on the HPCC program to help develop the new technology that will be needed to make the NII successful and to give the nation a competitive economic edge.
In order to best ensure that it stays focused on achieving these more immediate goals, the HPCC program could use more explicit management controls. First, the program will need to set more specific, measurable technical goals by developing a prioritized technical agenda. Such a document would serve as a master program plan, identifying and prioritizing specific technical challenges and establishing a framework for managing costs and evaluating results. Second, the program could make HPCC budget and expenditure information more consistent and meaningful across participating agencies to improve public visibility into program funding patterns.
subsections: section_title: Current HPCC Management Approach
paragraphs: As discussed in chapter 1, the HPCC program involves 10 federal agencies that have a wide variety of missions and approaches to research and development. A representative from the White House Office of Science and Technology (OSTP) stated, and researchers we contacted agreed, that this diversity of management approaches is a valuable asset in a research environment because it allows a variety of technical approaches to be explored. In addition, the major participating agencies—ARPA, DOE, NASA, and NSF—had conducted successful research and development programs related to HPCC for some years prior to the establishment of the joint program. As such, the OSTP representative stated that designating a strong central manager for the HPCC program would not be appropriate and that it would be disruptive to the programs to impose outside control over them.
Instead of taking a centralized approach, the OSTP representative said participating agencies should be seen as members of a consortium, each pursuing their own objectives but coordinating their efforts. The agency program managers are members of a committee called the High Performance Computing, Communications, and Information Technology (HPCCIT) committee. This committee, which is chaired by the NCO, meets on a monthly basis to coordinate HPCC activities. A number of researchers told us that, to date, this arrangement has worked reasonably well. HPCC managers are generally given high marks by researchers for sharing information and coordinating their activities. Figure 2.1 shows the organization of the HPCC program.
The program originally operated under the assumption that the advances it pioneers in high performance computing would eventually work their way down to widespread use for everyday activities throughout the private sector. Indeed, much computer research funded by ARPA and DOD in the past for military applications has been the foundation for technology widely used today in personal computers and communications networks. However, the administration now argues that the challenges facing the U.S. are too profound to rely on the government’s investments in defense and space technology to trickle down to the private sector. The administration intends the HPCC program to play a key role in a more focused approach to stimulating commercial development and application of new technologies.
Measuring progress within the program remains an informal process. In October 1992, OSTP established guidelines for the formal ongoing evaluation of federal research programs. Although the guidelines require that a program such as HPCC submit a plan for continual and thorough evaluation of progress and outcomes, no such plan has yet been prepared.
section_title: The Program’s Technical Agenda Could Be Better Defined
paragraphs: Potential NII applications will require specific new technologies that the HPCC program has not yet identified and prioritized as technical goals. For example, users of the NII will need to access and manipulate databases of information that are much larger than can be handled efficiently by today’s systems. Although some large database technology research is going on within the HPCC program, no determination has yet been made about whether it is a priority area that should be emphasized. Outside commentators on the HPCC program have proposed a range of specific technology areas, such as this, that could be targeted as a way of accelerating development of the NII.
Rather than targeting specific technology areas for accelerated development, the HPCC program has pursued research in many different aspects of advanced parallel computing. The program has had two broad technical goals, which it originally set out to achieve by 1996. One is to gain a thousand-fold improvement in useful computing capability and the other is to achieve a hundredfold improvement in computer communications capability. The program has aimed to address the full spectrum of hardware, software, networking, and training issues associated with developing this radically new breed of parallel computers.
Although much faster computers and networks are certainly a basic need, particularly to enable scientists to address grand challenge problems, these goals are all-encompassing and do not give enough technical focus to the program. Because they are so broad, controversy and confusion have sometimes arisen as to what the “real” goals of the program are. For example, university and industry experts have observed that, in its original form, the program appeared to be concentrating heavily on developing new hardware architectures, with relatively little attention being paid to software issues, thus leaving systems difficult to use. More recently, the addition of the NII-oriented IITA component has further broadened the technology spectrum to be addressed by HPCC. Both participants and outside observers have questioned the extent to which the program is actually shifting its emphasis toward NII technology issues, given that the level of funding for IITA projects to develop applications in areas such as education and health care is minimal compared with funding for hardware systems development.
No official prioritization has yet been made. The program’s annual report to the Congress describes ongoing work in a number of technical areas, but does not prioritize among competing technical goals. For example, the annual report states that the five broad component areas of the program are considered equally important. Within the new IITA component, the document identifies a range of technologies that will be needed for the NII, but does not prioritize them or offer an overall strategy for developing them.
An explicit technical agenda, identifying and prioritizing specific technology challenges and establishing a framework of expected costs and results, could go a long way toward better defining the program’s direction. This agenda could also provide the needed management framework for focusing on technologies in support of the NII. Although it could take a variety of forms, an official technical agenda would specify a target amount of resources to be invested in each priority area and the major results that are expected. Subject to periodic review and adjustment, this document would clarify the program’s goals and objectives, focus efforts on critical areas, and serve as a baseline for measuring program progress and results.
One potential model for identifying and prioritizing technology challenges is a draft prepared by the Computer Systems Policy Project, an affiliation of American computer companies that have an interest in the national information infrastructure. The document identifies nine technology areas that will be critical to the success of the NII. For each of these nine areas, the document lists a number of specific technologies that need to be researched and developed and suggests which of these should receive priority attention.
section_title: Inconsistent Budget Information Has Made Tracking HPCC Investments Difficult
paragraphs: Budgets and expenditures for HPCC activities, both inside and outside the program, have not been accounted for in a uniform and easily understood way. Accordingly, it is unclear how much money is actually being spent on advanced computing and communications and on what projects.
Spending for the formal HPCC program has closely followed its original plan of expanding in even increments from a $500 million base program to approximately $1.1 billion in its fifth year. However, the program budget, which is often cited publicly as a measure of the federal government’s investment in HPCC, actually offers little insight into how the federal government is investing in total in HPCC research and development. This is because participating agencies have diverse research programs and equally diverse ways of identifying and categorizing their HPCC spending. There are no uniform guidelines for determining what projects to include within the HPCC program or for categorizing those projects within the five major components of the program.
subsections: section_title: Budget Levels Do Not Reflect All HPCC-Related Funding
paragraphs: According to the official summary documents for the HPCC program that accompany the President’s budget request each year, nearly $3 billion has already been spent on HPCC, and, beginning in fiscal year 1995, annual budgets will top $1 billion. However, these figures do not reflect the total federal investment in HPCC. Several types of research and infrastructure activities are not consistently included or excluded from the program.
For example, preexisting government supercomputer centers have sometimes been included in the HPCC program and sometimes not. Four supercomputer centers supported by NSF are included, as is the National Cancer Institute’s center; however, the supercomputer center at the National Center for Atmospheric Research, also funded by NSF, is not included. Similarly, NASA includes some of its supercomputer facilities but not others. In each case, program managers have made their own judgments on what to include under HPCC since no programwide guidelines were available.
Advanced computer research that is not directly related to development of scalable parallel computers is another area that is neither clearly within nor clearly excluded from HPCC. NSF includes research into advanced optical computing, for example, whereas ARPA keeps its optical computing research separate from HPCC. NSF’s HPCC program also supports fundamental research in areas such as the theory of computing, software engineering, and the theoretical aspects of computer systems, while ARPA funds this type of research outside the HPCC program.
section_title: Categorization by Program Components Has Not Been Consistent
paragraphs: HPCC program documentation uses five component categories to describe the types of research and development that are funded within the program (these five categories are defined in chapter 1). Although this categorization could be helpful in understanding how HPCC funds are spent, its value is diminished by discrepancies in the way agencies categorize their official HPCC spending. Currently, no uniform method for categorizing projects is used. Relying on the personal judgment of HPCC managers and coordinators, participating agencies group similar projects differently within the five program categories.
For example, program documentation generally describes High Performance Computing Systems as the hardware component of the program. However, hardware spending also shows up in the Advanced Software Technology and Algorithms and Basic Research and Human Resources categories.
The Basic Research and Human Resources component, in particular, overlaps all the other categories, since program managers have to determine whether research is “basic” and then categorize their projects accordingly. Program managers have listed a full spectrum of research activities under this component, from research on architectures and systems, to software, algorithms, and applications. Because of these inconsistent classifications, it is difficult to determine what areas HPCC is really emphasizing—developing hardware platforms, writing systems software and tools, developing software applications, or none of these—or how much effort is being expended on each.
section_title: No Explicit Budget Guidelines Exist
paragraphs: Explicit guidelines for preparing HPCC budgets across agencies, which do not currently exist, would afford greater visibility into the overall federal investment and would facilitate more informed assessments of whether appropriate emphasis is being placed on areas that need greatest attention. Such guidelines should include new, more precise budget categories that would provide visibility into how much is to be spent on operating supercomputer centers, placement of computer systems, and other activities that support researchers but may not be research per se.
In April 1994, the NCO issued a document providing a detailed analysis of the types of activities that each HPCC agency funds and how much is being spent for them. The new document is a step in the right direction in that it sets a standard format for all participating agencies to use in presenting budget information and presents more detailed information than has been publicly available before. However, the document does not resolve the discrepancies in how various agencies account for their HPCC activities.
In addition to increasing visibility into the government’s investment, more open and consistent reporting of HPCC funding could also broaden industry support for the program, because the program’s major interest areas and priorities for funding would be clearer.
section_title: Conclusions
paragraphs: While continuing to foster basic research to address scientists’ need for ever-increasing computer power to address grand challenge problems, the HPCC program is also taking on the task of developing the specific technologies that will be needed for the NII. In order to be successful at that new task, the program could benefit from a detailed technical agenda, identifying and prioritizing the kinds of technologies it will develop in support of the NII. Such a document would better define the program’s direction and also serve as a baseline for measuring future progress.
The budget information annually reported to Congress on HPCC does not provide enough visibility into how much the government is investing in HPCC or what kinds of research and other activities are being funded. Much of the problem is due to the fact that no precise guidelines exist for determining what activities to include within the HPCC program. Also, the program’s five component categories, while useful in describing the program generally, are not helpful in revealing the specific kinds of activities that are being funded.
section_title: Recommendations
paragraphs: We recommend that the Director of OSTP direct the HPCC program managers, in consultation with industry and academic representatives, to develop an explicit HPCC technical agenda, delineating the program’s overall strategy and setting development priorities for specific technology areas. This document should specify target amounts of resources to be invested in each priority area and the major results that are expected, so that it can be used as a baseline for measuring progress and controlling costs.
We also recommend that the Director of OSTP develop, in consultation with the Office of Management and Budget and the Congress, detailed guidelines for preparing HPCC budgets, including guidance on the types of activities to include in the program and how they should be categorized. OSTP may wish to delegate this task to the NSTC Committee on Information and Communications.
section_title: Agency Comments and Our Evaluation
paragraphs: In his September 1994 comments, the Assistant to the President for Science and Technology (Science Advisor) concurred with our findings that a more focused management approach is appropriate, given the new direction of the HPCC program. He said that this more focused approach will include improved consistency in preparation of HPCC budgets within participating agencies as well as a more detailed and prioritized technical agenda to ensure that the goals of the program are clearly defined and success is clearly measurable.
The Science Advisor disagreed with what he perceived as our view that the program be centrally managed and that it have a centrally controlled budget. However, we did not recommend centralizing the program’s management or budget; instead, we discussed the advantages of a coordinated approach as well as the drawbacks of central management. We agree that HPCC program goals can be met within the framework of the existing program structure. However, achieving and sustaining the kind of targeted effort now envisioned for HPCC must begin with the identification of specific technical goals and priorities. These specific goals and priorities, once established, can then form an objective framework for making decisions about the type of activities to be funded within the program and the amount of funding to be allocated for each.
In July 1994, a committee of the National Research Council issued an interim report on the HPCC program that raised concerns in many of the same areas that we addressed. The committee, whose study is still ongoing, said it would continue to examine areas such as the potential for developing standard program performance measures for HPCC and the need for greater budget consistency.
section_title: Greater Industry Involvement Could Help the HPCC Program
paragraphs: Close collaboration with industry is essential to ensure that the HPCC program meets its goal of accelerating the development and widespread use of HPCC technologies. While industry has been extensively involved in the actual execution of HPCC projects, as the program moves forward it would benefit from partnerships with key industries that could capitalize on HPCC technologies to create new products and services for the NII.
Representatives from a variety of companies with a potential interest in HPCC told us they remain uninvolved in the program for several important reasons. They expressed the belief that the program does not address their needs and interests, largely because HPCC managers have not solicited their input in program planning. Also, the NCO, which was established in part to foster industry participation, has not provided industry representatives with needed information or responded to industry initiatives to improve communications between the program and potential industry participants. Given that the administration sees the HPCC program as playing an important role in developing key technologies for the NII, HPCC managers must more effectively promote industry participation.
subsections: section_title: Achieving NII-Related Goals Requires Government Researchers to Collaborate Closely With Industry
paragraphs: Since the program’s inception, HPCC program documentation has emphasized that industry participation is critical to meeting the program’s goals of accelerating the development and widespread application of high performance computing and networks. Now, industry’s collaborative role has become even more important in the context of HPCC’s new role of supporting development of the NII.
Specifically, the HPCC program is now committed to helping the private sector develop new technologies, including applications and services, that will maximize the value of the NII to a broad base of users. These applications include remote medical diagnosis by specialists and experts anywhere in the nation; the delivery and use of environmental information for a broad range of users, such as agriculture workers and truckers; and enhanced educational opportunities in which students could perform science experiments in collaboration with scientists at the national laboratories or visit museums and research centers without leaving their classrooms. In each case, it is envisioned that these applications will be developed by the private sector, with some level of government support. One goal of government collaboration will be to help ensure that issues of accessibility, security, and reliability are addressed.
section_title: Industry Has Focused on Execution Over Program Planning
paragraphs: Industry involvement in the actual execution of the HPCC program has been extensive. At ARPA alone, for example, 43 percent of the HPCC budget goes to companies that have successfully responded to ARPA’s requests for research proposals in specific technological areas. DOE also has established cooperative agreements with numerous partners from industry. Nevertheless, HPCC managers have generally not involved industry in planning the HPCC program. At the governmentwide level, a mechanism for obtaining nonfederal advice and evaluation was mandated by the High Performance Computing Act, which directed the President to establish an advisory committee including representatives from industry. According to OSTP officials, the administration is working to get the advisory committee appointed, although concerns about potential conflict-of-interest have slowed the effort.
Many HPCC agencies have their own advisory committees that review their HPCC programs. These committees have been helpful in planning effective agency programs. A case in point is NASA’s program, which was reviewed in 1993 by a NASA Advisory Council Task Force. The task force reported that the priorities in the agency’s HPCC plan did not address the research problems that the aerospace industry considered most critical. NASA responded by soliciting direct industry involvement in reworking its program plan for aerospace. Aerospace industry representatives told us they are encouraged that a revised plan will more fully reflect their interests and concerns.
The NCO, which was established in part to serve as a point of entry for industry into the program, disseminates general information about the program as well as funding opportunities. The NCO recently made this information available electronically over the Internet. In addition, the NCO has been involved in numerous liaison activities with industry, academia, and the public. These activities have included meetings, workshops, and conferences. The NCO has also allowed groups of industry representatives to attend certain designated portions of the HPCC program managers’ regular meetings and give brief presentations of their views.
Industry representatives whom we contacted agreed that all of these activities are valuable. However, they seek greater opportunities for close collaboration between government and industry in planning program direction. They have proposed that the NCO cooperate in arranging for the HPCC program to participate officially in symposia, in order for industry and academic representatives to meet with program managers to air their views on the direction and priorities of the program. They emphasize that these meetings should provide for a full discussion and consideration of issues of importance to industry, such as how best to invest limited resources. The NCO could exercise this function until a permanent advisory committee, which will maintain a more substantial, ongoing dialogue with program management, is appointed.
section_title: Greater Industry Involvement Depends on Increasing Software Emphasis
paragraphs: A major roadblock to broader industry utilization and commercialization of high performance computing technologies is the lack of software and software development tools to take advantage of the power of high performance computers. Currently, only a limited range of applications software is available, and development tools, which are needed to write new applications software, are primitive. Moreover, a lack of standards discourages industry from investing in software development projects that may have a limited market. A greater emphasis by the HPCC program on software could reduce some of the risks for potential industry participants and increase their involvement.
subsections: section_title: Existing Software Applications and Tools Are Inadequate
paragraphs: HPCC so far has focused on the grand challenges as target applications. While the grand challenges are important scientific problems, they involve only small communities of scientists working in specialized areas. For example, applications developed in NASA’s HPCC program are targeted at aerospace engineers designing and simulating new aircraft. Earth and environmental scientists, likewise, will profit from various HPCC projects supported by NASA, NSF, DOE, and ARPA. As valuable as these lines of effort are, they do not directly address broad areas where HPCC technology can benefit the NII, and industry tends to view them as offering little opportunity for commercialization.
One of the most important industry applications of HPCC on the NII will be information processing and management. A core set of generic software for processing, storing, searching, and retrieving multiple data types from very large databases would have a broad range of commercial applications, ranging from health care to banking. For example, software for handling databases of imagery would enable applications as diverse as remote medical consultations or law enforcement.
Software development tools, which would make it easier for software companies to design and develop new applications, might offer a particularly good opportunity to leverage government investment in HPCC. A series of reports by groups of HPCC researchers has identified and prioritized the tools that would be needed to facilitate the development of a broader range of applications software. These include debugging tools, memory management tools, and performance analysis tools, all of which would help to create a more productive software environment.
The HPCC program already supports some research in these areas. However, by establishing software development as a priority and devoting more resources to it, HPCC would encourage industry to invest in the development of a wide range of specialized NII applications.
section_title: Standards-Setting Activities Need Greater Support
paragraphs: Developers have identified the lack of standards as an impediment to more intensive commercial development of HPCC applications software. Agreement on standards would permit commercial software developers to build programs that work on a variety of high performance computers, rather than on only one specific hardware system, which may or may not do well in the marketplace. Broadening the base of computers on which the software will run would expand its potential commercial market, thereby allowing developers to put a much greater effort into building applications software.
However, setting standards is a difficult process, requiring a great deal of interaction over time within the HPCC community. Industry representatives agree that the government should not set standards. Industry, they believe, must lead this effort. Nevertheless, the government can play a practical role in supporting standards-setting efforts.
The HPCC program already provides funding for several standards-setting activities. For example, several agencies support a project to establish a standard HPCC version of the Fortran programming language. However, industry representatives have urged greater government support for standards-setting activities in order to stimulate commercial software development. Specifically, the HPCC program could fund more workshops where government, academia, and industry can come together to discuss and collaborate on emerging standards. The program could also provide more direct support for researchers to work with industry on evaluating potential standards.
section_title: Conclusions
paragraphs: It is widely recognized that the HPCC program needs a standing advisory committee that includes representatives from a broad range of potential industry participants. Such a committee would provide the mechanism to sustain an ongoing dialogue between the program and industry. However, in addition to establishing this committee, program officials can take additional steps to promote industry involvement, through cosponsoring symposia with industry and involving industry representatives in the program planning process, in order to forge a true partnership between government and industry.
section_title: Recommendations
paragraphs: We recommend that the Director of OSTP (1) take steps to expedite the appointment of an advisory committee whose membership includes representatives from a wide range of industries, and (2) delegate to the NCO the role of sponsoring symposia where industry can meet with program officials and academia to help define the research priorities of the program.
We also recommend that OSTP direct the Director of the NCO to take additional steps to promote industry participation, including involving industry representatives in the program planning process, and providing greater support for software development and standards-setting activities to make it easier for industry to develop applications for deployment on the NII.
section_title: Agency Comments and Our Evaluation
paragraphs: In his formal comments, the President’s Science Advisor strongly concurred with the recommendation that a private sector advisory committee be established and noted that OSTP was taking the initial steps to do so. The Science Advisor did not comment on our recommendation that the NCO sponsor symposia involving industry, academia, and HPCC program managers.
In preliminary discussions on a draft of the report, HPCC program managers maintained that the program has already implemented our recommendation to place greater emphasis on developing software tools and sponsoring standards-setting activities, as documented in the fiscal year 1995 Implementation Plan. We, however, do not agree that a significant shift in emphasis has yet occurred. While the implementation plan recognizes that greater focus on software tools will be required to encourage industry involvement in developing applications, a small percentage of the budget for the advanced software technology and applications component is allocated to this area. We believe that the program could better leverage federal funding by devoting more resources to activities that would make it easier for private industry to develop a broader range of applications.
In its interim report, the National Research Council’s HPCC study committee expressed concerns similar to ours. The committee recommended that an HPCC Advisory Council be appointed immediately to provide broad-based, active input to the HPCC program from industry and academia as well as government. The committee also expressed concerns about the need for software development to catch up with advances that have been made in HPCC hardware development.