Space Station: Plans to Expand Research Community Do Not Match Available Resources
Released: 1994-12-27T00:00:00

Letter

Background

As part of its mandate to guide the nation’s civil space program, NASA is to preserve U.S. preeminence in critical aspects of space science, technology, and applications. The goal of life and microgravity sciences is to study gravity-dependent physical phenomena and those phenomena obscured by the effects of gravity in biological, chemical, and physical systems. Research is conducted in biotechnology (e.g., protein crystal growth), combustion science, fluid physics, life and biomedical sciences, and materials science. Life science research in space biology studies the effects of gravity on living systems by using acceleration environments across the “gravity continuum”—micro, earth-normal, and hypergravity.

NASA’s Office of Life and Microgravity Sciences and Applications, which was formed in 1993, funds this type of research. Between fiscal years 1989 and 1994, the annual budget authority for life and microgravity research increased by 114 percent, from $222.4 million to $476.3 million.

Not all aspects of life and microgravity sciences research require a space-based environment. Short duration, low acceleration environments can be created in drop towers (2 to 5 seconds of free fall), aircraft flying a distinctively curved flight path (up to 23 seconds of low gravity), and suborbital rockets (over 300 seconds). Hypergravity can be created by a centrifuge.

Space-based research is principally conducted in pressurized and nonpressurized facilities on the Space Shuttle. The centerpiece for this research is a 23-foot by 16-foot pressurized module—Spacelab—that fits in the Space Shuttle payload bay. Spacelab was developed by the European Space Agency and contains utilities, computers, work areas, and instrument racks for experiments. An exterior cutaway view of Spacelab is shown in figure 1.

The most recent Spacelab flight was the second International Microgravity Laboratory (IML-2), which ended a 15-day mission on the Space Shuttle on July 23, 1994. IML-2 was a collaborative effort by NASA; the European Space Agency; and the national space agencies of Canada, France, Germany, and Japan. According to NASA officials, IML-2 provided a preview of the science operations to come on the space station.

The IML flights, which began in January 1992, gave the U.S scientific community access to foreign-developed flight hardware while providing the international research community with access to the Space Shuttle/Spacelab. Approximately 80 investigations were performed on IML-2, including 15 U.S. experiments—11 in the biotechnology, fluid physics, and materials science and 4 in the life sciences. IML-2 was the last flight of this international series of spacelabs before the station era begins in 1997. An interior view of IML-2 is shown in figure 2.

NASA publicly solicits research proposals from investigators in the life and microgravity research communities. The funding decision is principally based on an evaluation of the project’s scientific merit by a peer review panel. NASA’s peer review and other quality assurance procedures are outlined in appendix I.

Developing a Larger Research Community Will Require Adjusting Funding Priorities

NASA intends to build a space station-era research community from the ground up. To do so, a larger cadre of ground-based researchers than currently available will be needed to adequately support U.S. research on the station. A NASA official estimates that the number of ground-based microgravity researchers needs to increase from 73 to 240 between fiscal years 1992 and 1998. NASA officials have not made comparable estimates for life science researchers. To accomplish this goal, NASA has abandoned its tradition—principally associated with life science research—of soliciting research proposals for general and specific space flight opportunities. Although this approach appears reasonable, the planned funding levels do not match the program’s objective, and funding priorities may need to be reassessed if the number of life and microgravity ground-based investigators is to be significantly increased.

Selecting IML-2 Investigators Illustrated Two Basic Approaches for Developing Research Community

In recent years, NASA has used two approaches for developing life and microgravity science research communities—“select for flight” or “select for science.” In the select-for-flight approach, all of the U.S. life science and most of the U.S. microgravity investigators on IML-2 were selected from proposals submitted in response to flight-related announcements. In the select-for-science approach, two IML-2 microgravity investigators were selected from researchers who submitted proposals in response to two 1991 discipline-related (“fundamental science” and “biotechnology”) research announcements.

A NASA program scientist considers the IML-2 flight to have been a programmatic success and, in some respects, a model for the international space station. According to a NASA official, one indication of the flight’s success was the amount of good research generated from the many proposals submitted in response to a mix of science and flight-related research announcements. Additionally, two of NASA’s recent research announcements were in the select-for-flight tradition: its July 1993 announcement soliciting proposals for research on a 1998 space life sciences flight (Neurolab) and its February 1994 announcement soliciting proposals for life science research on the Russian space station Mir from 1995 to 1997. Presumably then, one effective way to develop a research community would be to solicit specific proposals for research that are directly related to the space station. NASA, however, has chosen to move toward exclusive use of “select for science,” as discussed below.

“Select for Science” Has Become Preferred Approach

Although NASA recently solicited proposals specifically for research on Shuttle flights to Mir, NASA’s life science office changed from the widespread use of the select-for-flight approach in December 1993. At that time, it solicited proposals for ground-based research in space biology focused on the hypergravity effects that can be induced by NASA’s centrifuges. NASA’s shift to ground-based research did not stifle competition for funding: it received 650 responses to the December 1993 announcement. Although “select for science” is relatively new to life science research, all microgravity research announcements since 1990 have focused on research opportunities in one or more science disciplines. And, as if to emphasize the independence of microgravity research from space station development, NASA changed the fiscal year 1992 goals for the microgravity program. The previous goals referred to developing and using the space station, whereas the current, more general goal is to “enable research . . . by choosing the carrier most appropriate for the experiment.”

Physical events, unlike biological processes, can be meaningfully observed under the short-duration microgravity conditions afforded by ground-based facilities, aircraft, and suborbital rockets. Consequently, a ground-based microgravity research investigator does not always have to conduct experiments in a space environment, and many do not. For example, of the 51 principal investigators who conducted such research at NASA’s Lewis Research Center from fiscal years 1989 through 1993, only 7 have been principal investigators on space-based experiments, including a microgravity Spacelab flight in September 1995. “NASA establish a vigorous ground based research program focussing on gravitational biology in which centrifuge facilities at NASA centers are utilized for exploring science programs aimed at forces greater than 1g .”

Select-for-Science Approach Reasonable

NASA’s strategy for using the select-for-science approach to further develop a life science research community in the station-era appears reasonable based on the experience of the microgravity sciences community.

First, the microgravity sciences research community has been growing. Principal investigators funded for microgravity sciences research increased by 120 percent—from 89 in calendar year 1989 to 196 in fiscal year 1993. The budget authority for microgravity sciences increased by 130 percent, from $75.6 million to $173.9 million during this period. The number of proposals submitted in response to research announcements also generally increased during this period. For example, although proposals submitted in response to materials, fluids, and fundamental (benchmark) physics research announcements decreased from 397 in 1991 to 217 in 1993, those responding to combustion physics announcements increased from 65 in 1989 to 98 in biotechnology research announcements increased from 94 in 1991 to 141 materials and fluids research announcements increased from 69 in 1990 to 346 in 1991.

Second, the microgravity research community is stable but not stagnant. Fifty-five percent of all microgravity sciences investigators that were funded in 1989 were also funded in fiscal year 1993. This core group represents 25 percent of the investigators funded in fiscal year 1993. On the other hand, 44 percent of the investigators funded in 1993 were not funded in 1992.

Third, NASA is attracting new investigators to its microgravity sciences program. The decline in proposals (from 397 to 217) submitted in response to the 1993 materials, fluids, and fundamental physics announcement may have been partly caused by NASA’s stated purpose of encouraging new investigators, and most of the 55 investigators funded from this announcement were new to the program. Only 15 of them had been previously funded by NASA. In June 1994, NASA’s life sciences advisory subcommittee recommended that NASA use this approach and establish appropriate categories within life science research announcements that recognize and encourage new investigators.

Finally, for those proposals we reviewed, the select-for-science approach produced relatively fewer low peer review scores than the select-for-flight approach. Figure 3 shows that 8 percent of the select-for-science proposals received peer review scores in the bottom category, while 32 percent of the select-for-flight proposals received scores in the bottom category.

Similarly, as figure 4 shows, of all the proposals in the bottom category, only 16 percent were in the select-for-science tradition, while 84 percent were in the select-for-flight tradition.

Many of the proposals submitted in response to NASA’s research announcements were not considered scientifically meritorious. For example, peer review panels gave 129, or 44 percent, of the 290 proposals we reviewed relatively low scientific merit scores.

Expected Funding Will Not Support NASA’s Approach

NASA’s plans to expand its ground-based research program are not realistic based on planned funding. A NASA microgravity research official estimates that NASA will need to fund about 240 ground-based investigators to support a station-based microgravity sciences research program. In fiscal year 1992, NASA funded 73 ground-based investigators in microgravity sciences, only about 30 percent of the future need. Ground-based research is funded from NASA’s research and analysis budget. However, NASA does not anticipate that this budget will increase for fiscal years 1995 through 1999. Annual life science research and analysis appropriations are estimated to be about $51 million and microgravity sciences at $21.7 million. To deal with this potential mismatch between plans and resources, NASA’s microgravity sciences office has proposed that the research and analysis budget be augmented by research and development funds used to support NASA’s space-based research program. The proposed amounts are $4.7 million for fiscal year 1996, $12.2 million for 1997, and $22.2 million for 1998. According to NASA, shifting resources in this way would not increase its overall budget authority. “where they are proven to be inadequate to support the intellectual underpinning of the flight program, even if this means a transfer from the budget so as to comply with overall budget constraints.”

Selecting Quality Experiments

NASA’s quality assurance procedures start with a series of external and internal reviews designed to evaluate the merits of research proposals. Peer review is a crucial part of this consensus-building process. The process starts with individual reviewers independently evaluating each proposal assigned to peer review panels. The reviewers then resolve any differences by consensus within the peer review panel. The panel’s final determinations are not binding on NASA’s selection officials, and NASA can choose proposals other than those highly recommended by the panel.

In June 1994, we reported that the peer review processes at the National Institutes of Health (NIH), National Science Foundation, and National Endowment for the Humanities appear to be working well and that intrinsic qualities of a proposal (e.g., research design), and not characteristics of reviewers or applicants (e.g., applicant’s region, academic rank, or employing academic department’s prestige) were important factors in reviewers scoring.

In 1993, the Senate Committee on Appropriations directed NASA to model its peer review standards after NIH. Based on the Committee’s direction, NASA requires that all research proposals be reviewed by peers for scientific merit and relevance (previously, some life science research conducted by NASA scientists was not subject to peer review); all research be reviewed by peers at least every 3 years; all research be reviewed for progress annually and for the performance of its objectives at least every 3 years; peer review be performed by the best-qualified individuals available in the peer review scores provided by external peer review groups be critical factors determining the priority for initial and continued funding of research projects and programs.

Individual Peer Reviewers’ Scores Are Usually Consistent

The logic of peer review, in our opinion, rests, in part, on the assumption that two or more peers can independently agree on a research experiment’s scientific merits. For example, they should agree on the testability of the proposed hypothesis and the relevance and appropriateness of the experimental design. As such, peers’ scores for scientific merit of any given proposal ought to be the same or similar.

Peers agreed on the scientific merit of 73 percent of the proposals that we reviewed, including all but 1 of the 15 U.S. experiments selected for IML-2. Table 1 shows the distribution of the reviewers’ scores. Peer reviewers were better able to agree on proposals having top scientific merit scores than on proposals having middle or bottom scientific merit scores. Peers gave only 11 percent (11 of 99) of the top proposals dissimilar scores; in contrast, they gave 35 percent (44 of 126) of proposals dissimilar scores in the middle category and 38 percent (23 of 60) scores in the bottom category.

NASA Selected Proposals Despite Peer Review Concerns

Table 2 shows that NASA’s selecting officials’ funding decisions were generally congruent with the findings of the peer review panel. Of the 84 proposals funded, 73, or 87 percent, were in the top category for scientific merit scores, and the other 11 proposals funded were in the middle category.

Determinations of peer review panels are not binding on NASA’s selection officials. For example, NASA selected four proposals that received mid-level scores by the peer review panel. Based on an average of peers’ individual scores, three of them would have been in the top category. However, in subsequent deliberations, the peer review panel members placed three proposals in the middle category because the need for the microgravity environment of space was not compelling, experiment-related issues could be resolved using the ground-based appropriateness of analytical techniques was questionable.

No peer review panel was convened for the fourth proposal because the number of proposals in the specific area of investigation was too small. Generally, the peer reviewers found the proposal to be of high quality, but they also noted that the research objectives, although compatible with the life science program, were inconsistent with the microgravity science program. In this case, the investigator did not propose to use microgravity to study phenomena whose understanding is obscured on earth by the presence of gravity.

After the peer review panel completed its deliberations, a NASA categorization committee made category assignments that were forwarded to a steering committee. The categorization committee determined that the four proposals were, in the words of a NASA official, “sound but not exceptional science”—the second highest of four possible categories.The steering committee assessed these categorizations and recommended funding the proposals, but committee members noted that one investigation resembled a “fishing expedition,” another had “similar weaknesses” to proposals that were rejected, a third would require too much time to conduct on a Spacelab mission, and a fourth should only be partly funded. Despite these views, these four proposals were funded for IML-2.

Assessing the Impact of Canceled Spacelab Flight on the Research Community

NASA’s efforts to develop a research community are not likely to be adversely affected by the February 1994 cancellation of SLS-3. The U.S. principal investigators on the Spacelab flight stated that they will be able to meet their experiment objectives on other missions, including multiple Shuttle flights to Mir. They plan to submit proposals for future NASA research opportunities.

NASA Advisory Committee Concerned About Cancellation of Spacelab Program and Usefulness of Russian Space Station

NASA planned to fly a collaborative U.S.-French SLS-3 mission in February 1996. The purpose of the mission was to study the effects of microgravity on the musculoskeletal system of humans, Rhesus monkeys, and rats. The French were responsible for developing the Rhesus research facility. Planning for the mission began in the late 1970s. On February 18, 1994, however, the NASA Administrator notified his French counterpart that the flight was canceled because of budget limitations and NASA’s commitment to the international space station. In November 1993, as part of the agreement between NASA and the Russian Space Agency to bring Russia into the space station program, the United States and Russia agreed to fly up to 10 Space Shuttle flights to the Russian space station Mir. “cancellation of mission or substitution of middeck experiments for a dedicated Spacelab mission would have serious consequences for . . . the continued participation of the mainstream life sciences community that NASA seeks to attract.”

NASA responded to these concerns in April 1994, stating that all experiments from SLS-3 have been accommodated on other missions, including Shuttle flights to Mir. NASA also noted that although Mir is not a substitute for Spacelab, it will augment and enhance on-orbit science capabilities because experiments requiring more than 30 days of microgravity cannot be performed on Spacelab.

In late 1994, NASA announced a new life and microgravity sciences spacelab mission for July or August 1996. This mission will provide a flight opportunity for some experiments that were scheduled to fly on SLS-3.

U.S. Investigators Not Adversely Affected by Spacelab Cancellation

We discussed the cancellation of SLS-3 with 13 of the 15 U.S. investigators who were scheduled to fly experiments on this flight. Their views are summarized, as follows:

Nine investigators were generally satisfied with the way NASA handled the cancellation. NASA never formally notified investigators about its decision to cancel the flight. Consequently, most investigators learned of the cancellation from rumors or other informal communication. One investigator said investigators should have been consulted before NASA canceled the mission. One investigator questioned why SLS-3, a relatively near-term mission, was canceled rather than a later one such as the Neurolab flight in 1998.

Eleven investigators said that their experiments will be accommodated on other missions, including the Russian Biosatellite, another Spacelab mission, or Space Shuttle flights to Mir. Two investigators said they have not been assigned to specific missions.

Ten investigators currently scheduled for other missions said they will be able to meet their basic experiment objectives. However, three of them said they will not necessarily be able to obtain the same amount of information as they would have on SLS-3. Their experiments involved the use of Rhesus monkeys, and even though they will fly on the Biosatellite, in-flight biological measurements cannot be done. Three other investigators said that their experiments on a substitute mission would be adversely affected by hardware limitations or the loss of opportunities to efficiently collaborate with other investigators.

All 13 investigators said they will continue to submit proposals for future NASA research opportunities, and at least 6 have already done so.

Scope and Methodology

To accomplish our objectives, we obtained documents from and interviewed officials at NASA headquarters in Washington, D.C., and at NASA’s Lewis, Johnson, and Marshall field centers in Cleveland, Ohio; Houston, Texas; and Huntsville, Alabama, respectively. In May 1994, we attended the IML-2 mission simulation and science review conference and observed crew training exercises prior to launch.

To review the further development of NASA’s life and microgravity sciences research community, we obtained information on research announcements issued between 1988 and 1994 and on the principal investigators who conducted ground- and space-based experiments. We examined peer review-related information on 319 proposals submitted in response to the 4 research announcements related to IML-2. We categorized the scores of all proposals that peer review panels considered responsive to the objectives of the announcements, as shown in appendix II.

To assess the possible impact of the cancellation of the SLS-3 Spacelab flight on the further development of NASA’s research community, we interviewed 13 of the 15 U.S. principal investigators on that mission.

We performed our work between November 1993 and September 1994 in accordance with generally accepted government auditing standards. As requested, we did not obtain agency comments on this report. However, we discussed the issues in this report with NASA officials and incorporated their comments where appropriate.

As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 15 days from its issue date. At that time, we will send copies to the NASA Administrator and other appropriate congressional committees. Copies will also be made available to other interested parties on request.

Please contact me on (202) 512-8412 if you or your staff have any questions concerning this report. Major contributors to this report are listed in appendix III.

Experiment Quality Assurance Procedures

The National Aeronautics and Space Administration’s (NASA) guiding principle for quality assurance is periodic review over the lifetime of an experiment. Figure I.1 depicts the major science and engineering review milestones.

The steps or actions involved throughout this process are outlined below.

Preparing Research Announcements and Soliciting Research Proposals

NASA’s Discipline Working Group(s) evaluates research program’s strengths and weaknesses and makes recommendations to the program scientist, who defines areas of investigation for forthcoming announcement.

NASA conducts workshops for prospective investigators from the scientific community to develop interest in forthcoming announcement.

NASA solicits research proposals by issuing announcement.

Assessing Research Proposals

Peers are selected by contractor (life sciences) or NASA (microgravity sciences) to evaluate proposals’ scientific merit. Peers should be leading researchers in their field and free from conflicts of interest (e.g., a current or recent professional collaboration with an applicant), and not currently receiving research funds from NASA.

For proposals receiving strong science reviews, the appropriate NASA field center assesses a proposal’s estimated cost and engineering feasibility. For example, Lewis Research Center is a “center of excellence” for two microgravity science disciplines: combustion science and fluid physics.

NASA program scientist recommends principal investigators’ proposals for funding to senior NASA management.

Reassessing Experiments

Science Concept Review

Principal investigator and project scientist describe science scope and feasibility for evaluation by Science Review Board.

Project manager describes conceptual design of experiment-related hardware for evaluation by Engineering Panel.

Requirements Definition Review

Project manager describes cost and schedule estimates.

Engineering panel assesses design of hardware.

Science panel assesses compatibility of science requirements with design of hardware.

Developing Flight Experiments

Preliminary design review assesses the compatibility of science requirements with a preliminary engineering model (“breadboard”) of hardware.

Critical design review assesses complete engineering model of hardware.

Preshipment review consists of experiment simulations, integration with hardware, and testing prior to sending the hardware to the launch site.

Peer Review Panel Scores for Selected Projects

To determine the similarity/dissimilarity of peers’ perception of a proposal’s scientific merit, we defined similar scores on the five-point scales as same or adjacent scores (for example: “3” and “3”, or “3” and “4”); and on the nine-point scales as same, adjacent, and next scores (for example: a “3” and “3”, or “3” and “4”; or “3”, “4”, and “5”).

Major Contributors to This Report

National Security and International Affairs Division, Washington, D.C.

Dave Warren Frank Degnan Thomas Mills James Berry Kimberly Carson Jeffrey Knott Larry Kiser The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (301) 258-4097 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Address Correction Requested