International Space Station: Measurable Performance Targets and Documentation Needed to Better Assess Management of National Laboratory
Published: 2015-04-27T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The U.S. has spent almost $43 billion to develop, assemble, and operate the ISS over the past two decades. The NASA Authorization Act of 2010 required NASA to enter into a cooperative agreement with a not-for-profit entity to manage the ISS National Laboratory and in 2011 did so with CASIS. CASIS is charged with maximizing use of the ISS for scientific research by executing several required activities. Recently, questions have arisen about the progress being made to implement the required activities and the impact it has had on ISS's return on the investment.
GAO was asked to report on the progress of CASIS's management of the ISS National Laboratory. GAO assessed the extent to which (1) CASIS has implemented the required management activities, and (2) NASA and CASIS measure and assess CASIS's performance. To perform this work, GAO reviewed the cooperative agreement between NASA and CASIS, CASIS's annual program plans, and other documentation and interviewed ISS, CASIS, and NASA officials.
section_title: What GAO Found
paragraphs: The Center for the Advancement of Science in Space (CASIS), manager of the International Space Station (ISS) National Laboratory, has taken steps to fulfill its management responsibilities contained in its cooperative agreement with the National Aeronautics and Space Administration (NASA), and has initiated the activities required by the NASA Authorization Act of 2010. GAO found that CASIS implemented procedures for prioritizing research; evaluated 206 proposals and awarded approximately $20 million in grants to 77 research projects through January 2015; and cultivated relationships with academic institutions, research-specific organizations, and other entities. CASIS, however, has not been able to fulfill its responsibility in the cooperative agreement to interact with the ISS National Laboratory Advisory Committee, which NASA was statutorily required to establish under the Federal Advisory Committee Act, because NASA has yet to staff the committee as required by the NASA Authorization Act of 2008. As a result, CASIS is not able to fulfill its responsibility in the cooperative agreement that requires it to coordinate with this committee and review any report or recommendations it originates.
CASIS has established fiscal year 2015 metrics that meet most of GAO's key attributes for successful performance measures (see figure below); however, NASA and CASIS did not establish measurable targets for these performance metrics, and NASA's annual assessment of CASIS was not documented.
Source: GAO analysis. | GAO-15-397
GAO's work on best practices for measuring program performance has found that performance metrics should have quantifiable targets to help assess whether goals and objectives were achieved by easily comparing projected performance and actual results. CASIS officials told GAO in July 2014 that setting measurable targets would be arbitrary because CASIS processes and metrics are still evolving. In January 2015, however, the Chairman of the CASIS Board of Directors told GAO that setting measurable targets is a priority for the board. CASIS, however, has yet to establish a date by which measurable targets will be developed. Using the established metrics, NASA is required by the cooperative agreement to perform an annual program review of CASIS's performance. This review is informal and not documented as ISS program officials provide the results to CASIS orally. This approach is inconsistent with federal internal control standards, which call for information to be recorded and communicated to those who need it to manage programs, including monitoring performance and supporting future decision making. Although NASA officials reported that they were generally satisfied with CASIS's performance, CASIS officials said a formal summary of the results would make the information more actionable.
section_title: What GAO Recommends
paragraphs: GAO recommends NASA fully staff the ISS National Laboratory Advisory Committee; NASA and CASIS work together to develop measurable targets for CASIS's metrics; and NASA begin documenting its annual review of CASIS's performance. NASA partially concurred and CASIS did not concur with the first recommendation, but concurred with the other two. GAO continues to believe the first recommendation is valid, as discussed further in the report.

section_title: Letter
paragraphs: The International Space Station (ISS) has been used as a manned  research outpost continuously for over 14 years. Much of the National  Aeronautics and Space Administration (NASA) ISS research has focused  on reducing the known risks related to long-duration human spaceflight as  the agency plans for missions beyond low-Earth orbit. The United States  has spent over $43 billion to develop, assemble, and operate the ISS  over the past two decades and NASA plans to spend about $17 billion  over the next 5 years to enable further scientific research the agency  views as critical to future human space activities. In January 2014, the  Administration proposed extending the life of the ISS by a minimum of 4  years to at least 2024. To take further advantage of the investment in the  ISS, Congress enacted several laws to increase utilization of the ISS by  commercial and academic researchers, first with the designation of the  ISS as a National Laboratory, and then later by directing NASA to enter  into an agreement with a not-for-profit entity to manage the ISS National   In 2011, NASA selected the Center for the Advancement of  Laboratory.Science in Space (CASIS), a non-profit entity, to manage non-NASA  commercial and academic research aboard the ISS National Laboratory.
In March 2014, you stated that the continued operations of this unique  research facility was a priority for the committee, but also expressed  concern about the progress being made by CASIS to manage the facility.  You requested that we review and report on CASIS’s progress in  managing the ISS National Laboratory, including a review of the  execution of the activities required by the NASA Authorization Act of  2010. We assessed the extent to which (1) CASIS has initiated and  implemented the required management activities for non-NASA research  aboard the ISS National Laboratory, and (2) NASA and CASIS measure  and assess CASIS’s performance.
To assess the extent to which CASIS has implemented required  management activities for non-NASA research aboard the ISS National  Laboratory in the NASA Authorization Act of 2010, we reviewed the  cooperative agreement between NASA and CASIS and the fiscal year  2014 and 2015 Annual Program Plans. We also examined documentation  on CASIS’s portfolio management, research prioritization and proposal  review and evaluation processes, business development efforts and  outreach efforts, and education initiatives. To obtain additional information  on CASIS’s implementation of some of the required activities, we  conducted structured interviews with 14 of 20 randomly sampled  researchers who had submitted proposals to CASIS through July 2014.  While our sample was, in part, representative of the population of 172  researchers, the results from the relatively small sample size are not  generalizable to the population. To determine the extent NASA and  CASIS measure and assess CASIS’s performance, we evaluated  CASIS’s fiscal year 2014 and 2015 metrics to determine whether they  adhered to GAO’s key attributes of successful performance measures.  We also reviewed CASIS documentation related to measuring and  reporting the results of these metrics. We reviewed the cooperative  agreement between NASA and CASIS to determine relevant NASA  responsibilities. In addition, we interviewed NASA and CASIS officials to  understand the processes involved with developing CASIS’s metrics and  assessing its performance. See appendix I for a detailed description of  our scope and methodology.
We conducted our review from April 2014 to April 2015 in accordance  with generally accepted government auditing standards. These standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings based on our audit  objectives.
subsections: section_title: Background
paragraphs: The ISS supports research projects with state of the art facilities for Earth  and space science, biology, human physiology, physical science, and  materials research, and provides a platform to demonstrate new space- related technologies.
The facilities include modular multipurpose payload racks and external  platforms to store and support experiments, refrigerators and freezers for  biological and life science samples, research lockers or incubators, and a  combustion chamber to observe combustion patterns in microgravity,  among other research equipment. The ISS currently has three crew  members in the U.S. operating segment who, according to NASA officials,  devote a total of approximately 35 hours per week to conduct research.  The remaining crew time is used for operations and maintenance of the  ISS, training, exercise, and sleep. NASA plans to increase the number of  astronauts in the U.S. operating segment of the ISS from three to four  once a U.S. capability to transport crew to and from the ISS is available.  Cargo transportation to the ISS is done through a commercial resupply  services contract that was signed with Orbital Sciences Corporation  (Orbital) and Space Exploration Technologies Corporation (SpaceX) in  2008. SpaceX currently has a capsule that can also return significant  amounts of cargo to Earth and is the only vehicle currently servicing the  ISS that has this capability. Orbital and SpaceX are scheduled to provide  8 and 15 resupply flights, respectively, through December 2017. As of  January 2015, SpaceX has launched five successful resupply missions  and Orbital has launched two successful resupply missions. Orbital  resupply flights to the ISS were deferred pending a review of a mishap  that occurred during a resupply launch in October 2014, which resulted in  the loss of that mission. According to NASA officials, a “return to flight”  plan was submitted by Orbital and accepted by the ISS program in  January 2015.
subsections: section_title: Legislation on Management of the ISS National Laboratory
paragraphs: Since 2005, Congress has directed several changes regarding the  management and utilization of the ISS. The NASA Authorization Act of  2005 designated the U.S. segment of the ISS as a National Laboratory.The 2005 act directed the NASA Administrator to seek to increase ISS  utilization by other federal entities and the private sector through  partnerships, cost-sharing agreements, and other arrangements that  would supplement NASA funding of the ISS. It also allowed the  Administrator to enter into a contract with a nongovernment entity to  operate the ISS National Laboratory.
The NASA Authorization Act of 2008 further directed NASA to establish  the ISS National Laboratory Advisory Committee, which was to be  composed of individuals representing organizations that had formal  agreements with NASA to utilize the U.S. portion of the ISS. act stated that the committee shall monitor, assess, and make  recommendations regarding effective utilization of the ISS as a national  laboratory and platform for research, and submit a report containing these  assessments and recommendations at least annually to the NASA  Administrator.
National Aeronautics and Space Administration Authorization Act of 2008, Pub. L. No.  110-422, § 602. designate a NASA liaison, with whom the selected not-for-profit entity  would cooperate and consult with in carrying out its responsibilities under  the agreement. An individual in the Space Life and Physical Sciences  Research and Applications Division of the Human Exploration and  Operations Mission Directorate is currently serving as the NASA liaison.  The 2010 act outlined seven management and research and  development activities that NASA was required to provide funding for the  not-for-profit entity to initiate. Those activities stated briefly, are to:
Plan and coordinate ISS National Laboratory research activities;
Develop and implement guidelines, selection criteria, and flight  support requirements for non-NASA utilization of the ISS research  capabilities and available facilities;  Interact with the ISS National Laboratory Advisory Committee and  review recommendations provided by that committee;
Coordinate transportation requirements in support of the ISS research
Cooperate with NASA, other departments and agencies of the U.S.  government, and commercial entities to sustain ground support  facilities for the ISS;
Develop and implement scientific outreach and education activities  designed to ensure effective utilization of ISS research capabilities;  and
Address other matters relating to the utilization of the ISS National  Laboratory for research and development as the Administrator may  consider appropriate.
The 2010 act also requires the ISS National Laboratory-managed  experiments to be guaranteed access to and use of at least 50 percent of  the U.S. research capacity allocation including power, facilities to keep  experiments cold, and requisite crew time onboard the ISS through  September 30, 2020. The Administrator can allocate additional capacity  to the ISS national laboratory if this capacity is in excess of NASA  research requirements. If any NASA research plan requires more than the  at least 50 percent of the U.S. research capacity allocation of ISS  resources, the plan should be submitted for consideration of proposed  research to be conducted within the ISS National Laboratory capacity of  ISS resources. The person designated as the NASA liaison to the not-for- profit entity has the authority to provide those resources beyond its 50  percent allocation on an exception basis if a proposed experiment is  considered essential for purposes of preparing for exploration beyond  low-Earth orbit, based on a joint agreement between the NASA liaison  and the not-for-profit entity.
section_title: CASIS and NASA Responsibilities Outlined in Cooperative Agreement
paragraphs: In August 2011, after a competitive process, NASA signed a cooperative  agreement with CASIS, a not-for-profit entity, to manage the activities of  the ISS National Laboratory through September 30, 2020. Cooperative  agreements differ from contracts. Generally, cooperative agreements are  used when the principal purpose of a transaction is to stimulate or support  research and development for a public purpose, and substantial  involvement is expected between the executive agency and the award  recipient when carrying out the activity identified in the agreement. In  contrast, contracts are used when the principal purpose is acquisition of  property or services for the direct benefit or use of the federal  government.
CASIS is bound by the responsibilities outlined in the cooperative  agreement, which tasks CASIS with maximizing the value of the ISS  National Laboratory by stimulating interest and use of the ISS for  scientific research by directly soliciting potential users and fostering a  market to attract others. CASIS is also charged with maximizing the use  of the ISS for advancing science, technology, engineering, and  mathematics (STEM) education. Pursuant to the cooperative agreement,  NASA will provide CASIS $15 million annually through 2020, of which it  will seek to award at least $3 million for research grants. CASIS officials  said that the remainder of NASA funding is used for infrastructure and  direct costs such as labor and travel-related expenses. According to the  cooperative agreement, CASIS will solicit non-NASA funding for research  by targeting various sources—such as government grants, foundation  funding, charitable contributions, private equity, venture financing, and  private investors—and facilitate matching of projects that meet the  research objectives with those qualified funding sources. Additionally, the  cooperative agreement requires the development of an annual program  plan, which includes a detailed plan of CASIS’s proposed activities for the  following year, which CASIS must meet using its “best efforts,” and  annual and quarterly performance metrics.
The cooperative agreement outlines responsibilities for NASA such as  providing resources and accommodations to CASIS to meet ISS National  Laboratory requirements and performing the payload operations  integration to ensure safe and effective flight readiness and vehicle  integration. The Cooperative Agreement Technical Officer, a NASA  employee within the ISS Program Office at Johnson Space Center, is  charged with oversight of the cooperative agreement. The Cooperative  Agreement Technical Officer is to coordinate the approval of the Annual  Program Plan and track performance to the plan using the metrics  reflected in CASIS’s quarterly reports.
section_title: CASIS Implemented Most of the Required Activities, but Is Not Able to Interact with Advisory Committee
paragraphs: CASIS has taken steps to carry out its responsibilities to manage and  promote research activities on the ISS National Laboratory as outlined in  its cooperative agreement. For example, CASIS identified key research  areas and released seven requests for proposals to solicit interest for  research projects. Our survey of a sample of researchers who had  submitted proposals to CASIS revealed generally positive comments  about CASIS’s management effort. For example, many respondents  indicated that CASIS’s processes were clear and that it evaluated their  proposals fairly. CASIS, however, has not been able to coordinate with  the ISS National Laboratory Advisory Committee (INLAC), as required,  because NASA has yet to staff the committee.
subsections: section_title: CASIS Took Steps to Implement Management Activities
paragraphs: CASIS has taken steps to fulfill its responsibilities contained in its  cooperative agreement with NASA, and has initiated the activities  required by the NASA Authorization Act of 2010.the activities contained in the 2010 act as well as the corresponding   Table 1 summarizes  responsibilities for CASIS and NASA outlined in the cooperative  agreement.
To determine its research and technology development objectives in  accordance with the cooperative agreement, CASIS identified and  prioritized the most promising research areas—which CASIS refers to as  pathways—with guidance from the Science and Technology Advisory  Panel, a CASIS committee comprised of both academic and commercial  experts. These pathways are identified by compiling a list of research  categories and determining financial feasibility. According to CASIS and  NASA officials, research pathways are generated from various sources  such as the Decadal surveys—studies conducted once every decade by  the National Research Council that prioritize the research focus for the  next 10 years in various scientific disciplines—and past NASA studies. To  date, CASIS identified protein crystal growth, stem cell research,  materials science, enabling technology to support science in space, Earth  imaging, and remote sensing as key research pathways and developed a  request for proposals (RFP) for each of these research pathways. CASIS  released seven RFPs since it was established with the first occurring in  June 2012, about 10 months after it was established. CASIS also accepts  unsolicited proposals from researchers and other sources such as   As of January 2015, CASIS  partnership accelerators and competitions.had received 206 proposals from all sources and awarded approximately  $20 million in research grants to 77 projects, and paid almost $13 million  to the awarded grants. Table 2 shows information related to the types of  proposals CASIS has received and the number of grants awarded.
CASIS-sponsored research investigations awarded through its first RFP  in 2012—involving protein crystal growth and microgravity—flew to the  ISS National Laboratory in April 2014 and were returned to Earth in  October 2014. These research investigations are currently in post flight  analysis. As of December 2014, there were 11 CASIS-sponsored  research investigations being conducted aboard the ISS National  Laboratory.
According to NASA and CASIS officials, as CASIS increases the number  of experiments for the ISS National Laboratory, the demand for crew time  and certain research facilities aboard the ISS is expected to increase and  they project the ISS National Laboratory will be challenged with meeting  that demand. NASA officials explained that while the demand for crew  time is currently manageable, it remains allocated at or near 100 percent,  as the three crew members on the U.S. segment of the ISS utilize most of  the 35 hours scheduled per week to conduct research. Crew time is  expected to double on the ISS National Laboratory once the crew  increases from three to four astronauts in fiscal year 2018 because,  according to NASA officials, the additional crew member would be able to  devote most of his or her time to research. NASA officials stated they are  also working with CASIS to build automation into research experiments to  reduce the monitoring time by crew members. Both CASIS and NASA  expect increased demand for facility resources such as the Animal  Enclosure Module used for rodent research and the remote sensing  cameras used for Earth observation. Sharing of the ISS National  Laboratory facilities requires considerable communication and  agreement. NASA and CASIS officials said both organizations have on- going discussions about how to share resources, coordinate research and  ensure all users are represented when meeting the demand for crew time  and ISS National Laboratory facilities and hardware. NASA officials  explained that they reprioritize as necessary to ensure resources are not  overstressed.
subsections: section_title: Develop and Implement Guidelines, Selection Criteria, and Flight Support Requirements
paragraphs: To initiate the development of guidelines and selection criteria, CASIS  implemented procedures for prioritizing research, guidelines for proposal  development, and evaluation and selection of research proposals, in  accordance with the cooperative agreement.
Procedures for prioritizing research: CASIS has implemented a  multi-layer review process to identify and develop the overall research  portfolio and prioritize future research pathways. See figure 2 for the  process CASIS follows to prioritize research pathways.
Guidelines on proposal development and flight support  requirements: CASIS established guidelines that are incorporated in  the applicable RFP for researchers to follow as they develop their  proposals. The RFPs include specific criteria for proposal that CASIS  uses as a basis for initial acceptance or denial of proposal  submissions. For example, one RFP issued in 2014 contained  minimum eligibility criteria such as the research being flight ready  within 12 months of award and the research having secured funding,  and included provisions that excluded the use of new sensors or  instruments for remote sensing, and required that selected proposals  be completed by 6 months post-flight. Each RFP also had unique  criteria which can be dependent upon the research pathway and the  facilities available on the ISS National Laboratory in the proposed time  line. CASIS has separate guidance for unsolicited proposal  submissions.
CASIS has documented the specific activities for meeting flight  requirements, which includes the role of implementation partners and  NASA in meeting these requirements. Implementation partners are  subcontractors to CASIS and specialize in aerospace technologies  and services. They have an integral role in providing hardware, flight  integration services, and ground services to support CASIS- sponsored research. NASA performs the activities necessary to  incorporate the research on a flight vehicle, such as providing the  resources and accommodations to meet ISS National Laboratory  requirements, and managing launch operations through payload  return to Earth.
Evaluation and selection of research proposals: CASIS  implemented a policy that documents the submission and general  review process for solicited and unsolicited proposals as well as  proposals that it evaluates as part of agreements with outside  organizations such as partnerships or subcontracts. The process  begins with a two-step initial submission review for solicited proposals  and a preliminary review for unsolicited proposals, then a five-step  evaluation process. Figure 3 details the CASIS proposal evaluation  process.
We surveyed a random sample of 14 researchers who submitted  proposals to CASIS from 2012 through 2014 to obtain their perspectives  on CASIS’s performance in this and other areas. Although the results of  this survey are non-generalizable because of our small sample size,  overall the respondents were generally positive about their interaction  with CASIS. For example, 11 of the 14 respondents indicated that  CASIS’s evaluation criteria were clearly articulated and 12 of 14  respondents believed their proposals were evaluated fairly. Of the 14  respondents, 13 said they were likely to submit future proposals to  CASIS. In addition, all 14 respondents indicated that they were notified in  a timely manner of the disposition of their proposal. CASIS declined  proposals for 8 respondents. Of these 8, 7 said that they were provided  feedback concerning why their proposal was declined. Several  respondents, however, said that they were provided only a short bulleted  response that fell short of addressing the scientific merit of the proposal.  One respondent said they received a letter summarizing reviewers’  comments that had several good points and was fair, but it was not  detailed as it contained less information than what other grantors provide.  According to CASIS guidelines, researchers whose projects are not  selected for award are provided feedback and, are invited to revise and  resubmit their projects as an unsolicited proposal. Of those that we  surveyed, only 1 of 8 respondents who had a proposal declined had  resubmitted the proposal, while another respondent said that it was not  made clear that proposals could be resubmitted.
section_title: Coordination of Transportation Requirements
paragraphs: Under the cooperative agreement, NASA is required to provide the ISS  National Laboratory research facilities and resources and coordinate with  CASIS when preparing CASIS-sponsored research for launch. CASIS  has an integral role in the payload development and integration process  during three distinct phases—pre-flight, operations, and post-flight. During  the pre-flight phase, the CASIS operations team works with the  researcher and implementation partners to understand project objectives  and requirements such as power, crew and hardware compatibility needs,  flight integration time frames, and design and integration support. CASIS  submits the science objectives, requirements, and a development  schedule to NASA. The NASA ISS National Laboratory Office also  assigns staff to each CASIS-sponsored researcher to help coordinate and  navigate the payload development and integration process and ensure  that flight planning remains on track. During the operations and in-flight  phase, CASIS provides operational support by collaborating with the  implementation partner or the researcher to oversee NASA’s integration  of the research project or hardware into the flight vehicle. The post-flight  phase involves the return of payload samples or hardware from the flight  vehicle to the researcher to begin post-processing activities, which CASIS  monitors.
The ability to secure transportation for selected research investigations to  the ISS facility is outside of CASIS’s control and has presented  challenges. NASA provides launch services to the ISS National  Laboratory through its commercial resupply services contracts and CASIS  receives cargo allocations for its sponsored research. Launch failures and  delays, however, have resulted in cost increases. For example, the recent  rocket launch failure to the ISS in October 2014 resulted in the loss of  several CASIS-sponsored research investigations at a total cost of almost  $175,000 which includes hardware and materials, labor consulting and  grants. In addition, launch delays for another cargo resupply mission  resulted in over $300,000 in cost increases for several researchers for  additional materials and samples. CASIS officials explained that the  majority of cost increases are related to biological research, which  represents approximately 50 percent of the CASIS-sponsored research.  These biological payloads have a limited viability or very specific  requirements associated with the timing of the payload flight and often  require consumables such as gas, nutrients, and water that must be  replenished when a launch is delayed. Absorbing the increased cost has  been a challenge for CASIS, but it is addressing the increased costs of  delays by asking researchers that have biological payloads to identify the  impact and associated costs for launch delays in their budgets so it can  plan for budget reserves, if necessary.
section_title: Cooperative Efforts and Partnerships
paragraphs: The cooperative agreement requires CASIS to manage planning and  coordination of research activities for both ground and on-orbit execution.  According to CASIS officials, CASIS is addressing this requirement by  leveraging the resources of companies that provide hardware, technical  expertise and ground support. Eleven implementation partners have  received over $5.4 million in funding from CASIS or its sponsored  researchers since the establishment of CASIS through September 2014  to provide hardware, flight integration, and ground services for 58  research investigations. CASIS officials reasoned that by leveraging  existing companies that can provide specialized hardware and integration  capabilities on an as required basis, CASIS can effectively manage the  ISS National Laboratory without having to maintain all the requisite skills  or capabilities within its organization. CASIS-sponsored researchers are  encouraged to select an implementation partner during the proposal  submission process from a list of preferred partners. CASIS officials said  that they assembled this list of implementation partners beginning with  companies that had relationships with NASA for ISS-related operations  and expanded the list through its own business development operations.  These partners can provide hardware, and technical services, and  consultation to researchers that address the project’s science  requirements and research needs aboard the ISS National Laboratory.  Although CASIS provides a list of implementation partners, the  researchers are responsible for entering into formal business  arrangements with these partners and including the costs of the  implementation partner support in their proposed budget. CASIS officials  noted the cost can vary based on the amount of involvement required by  the implementation partners and can range from $50,000 to $300,000 per  flight.
section_title: Scientific Outreach and Educational Initiatives
paragraphs: In accordance with the cooperative agreement, CASIS is building a  geographic network to facilitate outreach initiatives and cultivate new  partnerships and has implemented educational initiatives that provide  opportunities for educators and students to learn about and have access  to the ISS National Laboratory. Specifically,
Network outreach: CASIS has organized its outreach to scientific  and academic communities in seven geographical areas. These areas  are supported by more than 30 CASIS employees and consultants  and each area has a research emphasis. See figure 4 for the  locations of CASIS’s networks and the research emphasis for each.
The outreach efforts conducted through CASIS’s networks are  primarily relationship-based and focused on engaging financial  support, forging long term partnerships, and ultimately generating  potential research and technology development projects for flight on  the ISS National Laboratory. According to CASIS officials, academic  institutions, research-specific organizations, philanthropic entities, and  industry partners that CASIS identified through this network can  benefit from the CASIS-sponsored research and technology  development aboard the ISS National Laboratory. For example,  Boston was identified as one of the geographic areas because it has  over 100 universities and over 300 biotech companies that can  support the commercialization of life sciences research and CASIS’s  mission.
CASIS is working to expand its network. CASIS has developed 45  new partnerships to date and is leveraging a variety of new  partnership opportunities. For example, in 2014, CASIS initiated two  strategic campaigns, Good Earth—an international collaboration  seeking to maximize ISS Earth observation capabilities—and Good  Health—an effort to capitalize on the unique benefits of the  microgravity environment so interventions can be developed to  preserve health on Earth. CASIS officials expect both campaigns to  bring together large scale collaborations to stimulate ISS utilization  over the coming years. CASIS also supported the Rice Business  Competition by providing a $25,000 grant during 2014 to a startup  company that showed the most promise for developing a technology  or business that would benefit from access to the ISS National  Laboratory. This partnership also gives CASIS access to many forum  events and panels.
According to CASIS officials, it has been challenging to raise  additional funding from external sources to supplement the amount of  funding provided by NASA to support and sustain its operations  because CASIS is a new non-profit entity. Although CASIS’s business  development team is actively identifying partnerships and funding  opportunities with commercial and non-profit granting organizations,  CASIS officials said that it takes time to identify, develop, and mature  these partnerships. CASIS and NASA officials said that the value of  doing research aboard the ISS National Laboratory has to be further  demonstrated so commercial industries can be convinced it is worth  the high investment. Both NASA and CASIS officials said that  demonstrating the value of research on the ISS as a substitute for  ground-based research is a tremendous and important effort that is  necessary to open a marketplace for space research. NASA officials  stated that doing research aboard the ISS National Laboratory can  take upwards of 2 to 3 years to plan and execute, time lines that are  generally not acceptable to commercial companies that desire a more  rapid return on their investments.
Ten of the 14 respondents to our survey reported that CASIS was  effective in reaching out to the research community. For example,  several researchers were made aware of CASIS opportunities by  attending presentations from CASIS staff at industry meetings or  campus visits. Respondents also offered areas for improvement for  CASIS to increase utilization of the ISS National Laboratory. For  example, five respondents said that CASIS could increase its visibility  by attending more conferences, using more print ads, and working  more with NASA on joint RFPs.
Education: CASIS established its education strategic plan, which  included building education programs that promote the ISS as a  science, technology, engineering, and mathematics (STEM) learning  platform; partnering with existing education entities such as schools,  universities, and other educational foundations and associations; and  reaching out to underrepresented and nontraditional demographics.  CASIS also implemented various educational initiatives that it  developed both internally and externally in conjunction with its  partners. For example, in fiscal year 2014, CASIS supported 12  educational initiatives. CASIS sponsored the Space Station Academy,  a 4-week online program designed to take participants on a simulated  mission to the ISS as “virtual astronauts.” This pilot program involved  25 students and 25 educators. In addition, CASIS supports its  educational efforts through education grant funding and partnerships.  See appendix III for more information on additional CASIS educational  initiatives.
section_title: NASA Has Not Staffed Advisory Committee with Which CASIS Is Required to Interact
paragraphs: The one required activity in the cooperative agreement that CASIS has  been unable to address is its interaction with the ISS National Laboratory  Advisory Committee (INLAC) because the committee has not been  staffed by NASA. The NASA Authorization Act of 2008 required NASA to  establish the INLAC under the Federal Advisory Committee Act. The  INLAC was required to include membership from organizations that have  formal agreements with NASA to utilize the U.S. portion of the ISS. As  outlined in the 2008 act, this committee is required to exist for the lifespan  of the ISS and is to function in an advisory capacity to the NASA  Administrator by assessing and monitoring ISS National Laboratory  resource utilization and reporting its assessments and recommendations  at least annually. According to the cooperative agreement, CASIS will  coordinate with the INLAC as established under section 602 of the NASA  Authorization Act of 2008 and review recommendations provided by the  INLAC. Although NASA formally established the committee in 2009,  NASA has not fully implemented the 2008 act because the committee has  yet to be staffed. NASA officials told us that with CASIS in place, the  great majority of non-NASA ISS users do not have an agreement with  NASA because they work with CASIS. They added that there are  exceptions where NASA works with other agencies, but those are  typically for exploration technology or defense-related projects. In  addition, NASA officials indicate that the INLAC has not been staffed  because they believe that the structure and function of the current CASIS  Board of Directors has proven to be a better alternative to a NASA  advisory committee since the CASIS board represents a broad  experience base including military, medical research, strategic  partnerships, and engineering, among others. Further, NASA officials said  that the Research Subcommittee of the Human Exploration and  Operations Committee to the NASA Advisory Council also provides  research advisory oversight of the ISS National Laboratory. This  subcommittee’s objectives, however, have a focus on human spaceflight  and the membership of this subcommittee is to consist of individuals from  the research committee with a broad awareness of human spaceflight  related activities.
CASIS officials also believe that their board is performing some of the  INLAC’s advisory duties, but acknowledge that the board does not meet  the section 602 requirements under the 2008 act—to monitor and report  annually to the NASA Administrator its assessments and  recommendations of ISS National Laboratory utilization—nor does its  membership meet the criteria specified in the act. Without a staffed  INLAC, NASA currently lacks a single advisory committee that represents  all users of the ISS National Laboratory and provides ongoing monitoring  and assessments and recommendations of ISS National Laboratory  resource utilization, as required by the charter. As a result, CASIS is not  able to fulfill its responsibilities as outlined in the cooperative agreement  and as established under section 602 of the NASA Authorization Act of  2008.
section_title: CASIS and NASA Have Taken Steps to Measure and Assess CASIS Performance, but Measurable Targets Needed
subsections: section_title: CASIS’s Metrics Consistent with Most Key Attributes, but Lack Quantifiable Goals
paragraphs: CASIS has established metrics, but not targets against which its  performance can be measured by NASA. The metrics CASIS developed  in collaboration with NASA for fiscal year 2015 meet most key attributes  of successful performance measures. These metrics are based on CASIS  responsibilities outlined in the cooperative agreement and are related to  CASIS strategic goals and objectives. Metrics are included in an Annual  Program Plan, which CASIS prepares with input from NASA.
We have previously reported that successful performance measures as a  whole should have four general characteristics: demonstrate results, be  limited to a vital few, cover multiple priorities, and provide useful  information for decision making. We cited specific attributes as key to  successful performance measures, such as linkage, clarity, measurable  targets, objectivity, and balance. The four characteristics are overarching,  thus they do not necessarily directly link to the attributes. Furthermore,  the attributes may not be equal, and a noted weakness does not mean  that a measure is not useful. Weaknesses identified should be considered  areas for further refinement. Table 3 defines the key attributes of  successful performance measures.
We assessed CASIS’s fiscal year 2015 metrics, and found that the  metrics met almost all of these key attributes. The results of our  assessment are shown in table 4. We also assessed the metrics CASIS  had developed for fiscal year 2014, and similarly found that the metrics  met most of the key attributes. The results of our assessment of CASIS’s  fiscal year 2014 metrics for key attributes of successful performance  measures can be found in appendix IV.
Our analyses indicated that CASIS did not establish measurable targets  or goals for either fiscal year 2014 or 2015 metrics, which limits its ability  to use these metrics to assess performance. We have previously reported  that performance metrics should have quantifiable, numerical targets or  other measurable values, which help assess whether overall goals and  objectives were achieved.it is unclear how NASA objectively assesses CASIS’s performance.
Without defined measurable targets or goals,  CASIS officials noted that operating as a new entity with no history made  it difficult to establish performance targets, but this is beginning to  change. CASIS officials initially told us in July 2014 that establishing  targets would be arbitrary because CASIS processes and metrics are still  evolving. Subsequently, in January 2015, they indicated that since CASIS  now has some operating history, they will be able to do so. The Chairman  of the CASIS Board of Directors told us that measurable targets should  be developed and that this is a priority for the Board. However, CASIS  has not established a date by which measurable targets will be  developed. Further, CASIS officials indicated that not all metrics will have  measurable targets initially because some metrics are subjective, such as  those that attempt to measure the quality of research or a new technology  generated by CASIS-sponsored research. The Chairman said that the  CASIS Board of Directors is also working to develop targets for subjective  measures, and they hope to have them in place in the next several years.
Although the ability to objectively measure performance is limited without  measurable targets, CASIS and NASA officials generally agreed about  how long-term success for CASIS will be defined. According to CASIS  officials, success would ultimately be defined by demonstrating that the  research and technology development performed aboard ISS National  Laboratory benefits Earth and that commercial markets can be sustained  in low-Earth orbit. NASA officials similarly said that developing  commercial markets in space and bringing products back to Earth will  determine success.
section_title: NASA Assesses CASIS Annually, but Performance Assessment Is Not Documented
paragraphs: NASA performs an annual assessment of CASIS’s performance  consistent with its responsibilities in the cooperative agreement, but this  assessment is not documented. The Cooperative Agreement Technical  Officer (CATO) uses the metrics in CASIS’s quarterly and annual reports  to monitor CASIS’s efforts. The cooperative agreement also requires  CASIS to propose an adjustment to the metrics if performance is not  going to be met. However, without performance targets, CASIS cannot  determine whether the metrics need to be adjusted. Further, without  these targets, NASA and CASIS cannot conduct assessments that are  measurable or conclusive and, therefore, the assessments are subjective.  According to the CATO, during the annual program review, he assesses  CASIS metrics for trends, looking for improvements over time and  questioning any perceived lack of progress. The CATO added that he  discusses any issues identified during the annual review with CASIS  officials, NASA management, and stakeholders. CASIS officials  concurred, and told us this discussion with NASA highlights areas for  further refinement. For example, as a result of such discussion, CASIS is  now more proactively engaging NASA technical expertise on available  flight hardware, and has broadened business development efforts aimed  at attracting new commercial users of the ISS National Laboratory.
Both CASIS and NASA officials told us that NASA does not document its  annual program review of CASIS performance. Federal standards for  internal controls call for information to be recorded and communicated to  management and others who need it to carry out their responsibilities.  This type of documented information is important to support decision  making and conduct assessments. CASIS officials have not asked for a  formal summary of the results of NASA’s annual program review because  CASIS receives informal feedback on quarterly reports provided to NASA.  CASIS also maintains minutes of regularly scheduled meetings with  NASA where any issues that need to be discussed between CASIS and  NASA are addressed. While NASA does not document this annual  assessment, NASA officials told us that they were generally satisfied with  CASIS performance. CASIS officials, however, said that the results of the  annual review should be reported in some sort of formal manner to make  the information more actionable.
Because CASIS is allocated at least 50 percent of ISS research capacity,  future success of the ISS as a research platform is partially dependent on  the efforts CASIS has undertaken. However, without definitive and  documented assessment factors, NASA will be challenged to take action  in response to CASIS performance. For example, without documentation,  NASA lacks support to terminate the cooperative agreement, if deemed  necessary. Conversely, NASA also would have no record to justify  extending the cooperative agreement to support a possible ISS life  extension. The cooperative agreement will expire at the end of fiscal year  2020, but includes a provision for an extension.
section_title: Conclusions
paragraphs: The ISS offers the potential for scientific breakthroughs, a unique test bed  for new technologies and applications, and a platform for increased  commercial and academic research. Achieving greater utilization of the  ISS and its unique capabilities, showing the benefit of commercial and  academic research, and demonstrating success to generate increased  interest from potential users could help NASA get a better return on its  significant investment in the ISS. NASA currently lacks an advisory  committee established under the Federal Advisory Committee Act that is  composed of individuals representing organizations who have formal  agreements with NASA to use the U.S. portion of the ISS. As a result,  CASIS is not able to fulfill its responsibility as outlined in the cooperative  agreement that requires it to coordinate with INLAC as established under  the NASA Authorization Act of 2008 and review recommendations  originated by the INLAC. A fully staffed and operational INLAC could  provide information to senior NASA management on how to better utilize  the constrained resources of the ISS—which could affect how CASIS  attracts new users and fulfills its responsibility to increase utilization of the  ISS National Laboratory.
In addition, clearly defined measurable targets are essential for CASIS to  demonstrate results, allow NASA to objectively assess CASIS  performance, and help stakeholders assess whether overall goals and  objectives for the ISS National Laboratory are achieved. Finally, NASA’s  annual performance assessment of CASIS is not documented and the  results are provided to CASIS on an informal basis. Not documenting the  results of the annual program assessment is a practice contrary to good  internal controls, which call for information to be recorded and  communicated to management and others who need it to carry out their  responsibilities, to include taking appropriate corrective actions. Without a  clear, well-documented assessment of CASIS performance, NASA  management and stakeholders could also be missing information  important for decision making, for example, deciding to extend the  cooperative agreement with CASIS beyond the September 2020  expiration if the service life of the ISS is extended or terminate the  agreement, if necessary.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the NASA Administrator take the following three  actions:  In order for NASA to fully implement the NASA Authorization Act of  2008 and for CASIS to fulfill its responsibility as outlined in the  cooperative agreement, direct the Associate Administrator for the  Human Exploration and Operations Mission Directorate to fully staff  the INLAC.
In order to set clear goals to allow NASA to objectively assess CASIS  performance, require the ISS Program Manager work with CASIS to  collectively develop and approve measurable targets for metrics for  fiscal year 2016 and beyond.
In order to provide CASIS management actionable information to  better fulfill its responsibilities and NASA management with additional  information by which to make future decisions concerning the  extension of the agreement with CASIS, require the ISS Program  Manager to document the annual program assessment of CASIS  performance.
section_title: Agency and Third- Party Comments and Our Evaluation
paragraphs: NASA and CASIS each provided written comments on a draft of this  report, which are reprinted in appendix V and appendix VI, respectively.  NASA and CASIS also provided technical comments, which have been  incorporated into the report, as appropriate. NASA partially concurred and  CASIS non-concurred with one of our recommendations and both NASA  and CASIS concurred with the other two recommendations.
NASA partially concurred and CASIS non-concurred with our  recommendation directing the Associate Administrator of the Human  Exploration and Operations Mission Directorate within NASA to staff the  INLAC. In response to this recommendation, both NASA and CASIS  raised concerns that the current requirements for membership of the  INLAC would create a conflict of interest. Specifically, NASA stated that  the individuals who would make up the committee would likely have user  agreements with CASIS and, in many cases, would be receiving funding  from CASIS and NASA. Furthermore, because these entities would be  competing for CASIS resource allocations, CASIS believes that they  would not be sufficiently independent to perform the functions required of  the committee. In response to these concerns, CASIS indicated the  composition of membership as defined in the NASA Authorization Act of  2008 should be amended. NASA also responded that while meeting  statutory obligations and obtaining knowledgeable input and  recommendation to achieve optimal utilization of the ISS is important, it is  the agency’s position that the CASIS Board of Directors serves the intent  of the INLAC charter by providing recommendations regarding effective  utilization of the ISS. As a result, NASA indicated that it plans to work with  the Congress to adjust the INLAC requirement to address these  concerns. We continue to believe our recommendation is valid. We do not  see that staffing the INLAC as directed in the 2008 act would necessarily  result in a conflict of interest and that the entities would be competing for  CASIS resource allocations. The act required an advisory committee that  represents all users of ISS National Laboratory and that provides ongoing  monitoring and assessment and makes recommendations. According to  the cooperative agreement between CASIS and NASA, CASIS is directed  to coordinate with the INLAC and review the committee’s  recommendations. The INLAC, however, functions only in an advisory  capacity; therefore we do not see how a conflict of interest would be  created by the membership of the INLAC. Furthermore, according to the  NASA Authorization Act of 2010, CASIS shall be guaranteed access to  not less than 50 percent of the United States research capacity allocation.  Because CASIS has to agree with NASA for an allocation of resources at  a level below 50 percent, we do not see how the composition of the  INLAC would create a competition for resource allocation with CASIS. In  addition, it was not clear to us in our review that the existing mechanisms  in place accomplish these requirements. If NASA were to seek relief or  changes to this requirement, it should clearly outline how these  requirements can be met through existing bodies and processes.
NASA and CASIS concurred with our recommendation directing the ISS  Program Manager to work with CASIS to collectively develop and  approve measurable targets for metrics in fiscal year 2016 and beyond. In  response to this recommendation, NASA stated that fiscal year 2016 is a  reasonable time to establish measurable targets with CASIS because the  non-profit will be entering its fourth full year of operations. Similarly,  CASIS responded that it is now in a position to develop targets for key  metrics and plans to formalize the process in fiscal year 2016. NASA  indicated that these targets should be established by December 31, 2015.  Once complete, this action should address our recommendation to  develop and approve measurable targets for CASIS’s metrics.
NASA and CASIS also concurred with our recommendation directing the  ISS Program Manager to document the annual program assessment of  CASIS performance. In response to this recommendation, NASA said that  it would begin documenting the agency’s annual program assessment in  response to CASIS’s 2015 annual report. Once complete, this action  should address our recommendation to document NASA’s annual  assessment of CASIS’s performance.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution of it until 30 days from  the date of this letter. We will send copies of the report to NASA’s  Administrator and interested congressional committees. We will also  make copies available to others upon request. In addition, the report will  be available at no charge on GAO’s website at http://www.gao.gov.
Should you or your staff have any questions on matters discussed in this  report, please contact me at (202) 512-4841 or makm@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix VII.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to assess the extent to which (1) the Center for the  Advancement of Science in Space (CASIS) has initiated and  implemented the required management activities for research aboard the  International Space Station (ISS) National Laboratory, and (2) the  National Aeronautics and Space Administration (NASA) and CASIS  measure and assess CASIS’s performance.
To determine the extent to which CASIS is performing the required  management activities for non-NASA research aboard the ISS National  Laboratory, we obtained and reviewed relevant legislation and  documentation, and interviewed ISS program and CASIS officials. We  reviewed the NASA Authorization Act of 2005, which designated the U.S.  Operating Segment of the ISS as a National Laboratory; the NASA  Authorization Act of 2008, which directed NASA to establish an ISS  National Laboratory Advisory Committee; and the NASA Authorization Act  of 2010, which required NASA to enter into a cooperative agreement with  a nonprofit organization to manage the activities of the ISS National  Laboratory. We also reviewed the cooperative agreement between NASA  and CASIS and the CASIS fiscal year 2014 and 2015 Annual Program  Plans for CASIS responsibilities related to the required activities outlined  in Section 504(c) of the NASA Authorization Act of 2010. We examined  the CASIS portfolio management and research prioritization process and  various market analyses and studies that CASIS considered in  establishing research areas. We reviewed the CASIS proposal review  and evaluation process for solicited and unsolicited proposals as well as  the Requests for Proposals that CASIS had issued to solicit research  proposals. We studied fiscal year 2014 quarterly and annual reports to  gain insight into the activities CASIS had undertaken to meet its  responsibilities. We reviewed CASIS business development efforts,  including funding and marketing processes and outreach efforts. We  reviewed the partnerships CASIS has established with philanthropic  institutions that could provide additional resources to sponsor research  aboard the ISS National laboratory and implementation partners that  provide logistical assistance to researchers. Additionally, we reviewed  CASIS education efforts, particularly science, technology, engineering,  and mathematics activities. We also reviewed GAO, NASA Inspector  General, and NASA reports on sustaining the ISS. We interviewed  several ISS program officials including the ISS Program Director, ISS  Program Manager, ISS Program Scientist, and the Cooperative  Agreement Technical Officer to gain their perspectives on the work  CASIS was performing. We also interviewed officials in the Space Life  and Physical Sciences Research and Applications division, including the  NASA Liaison to CASIS, to gain perspective on the work NASA is  sponsoring aboard the ISS. In addition, we interviewed the CASIS  President and Executive Director, the CASIS Chief Operating Officer, the  CASIS Chief Financial Officer, and the Chairman of the CASIS Board of  Directors to better understand the processes and procedures being  implemented, how proposals are evaluated, and the challenges that  CASIS faces to further implement the responsibilities outlined in the  cooperative agreement.
To obtain additional information on CASIS’s performance and the  effectiveness of its implementation of some of the required activities, we  used information provided to us by CASIS to select a random sample of  20 principal investigators who had submitted either a solicited or  unsolicited research proposal to CASIS. Of the 20 researchers selected,  we conducted structured interviews with 14 researchers to obtain  additional insights into CASIS’s performance. Although the randomly  selected researchers are, in part, representative of the population of 172  researchers who had submitted proposals to CASIS through July 2014,  the descriptive nature of the responses and the relatively small sample  size does not permit the development of reliable, quantitative estimates  that are generalizable to the population. However, we believe our  interview results provide us with valuable information about researcher’s  experiences and perspectives on CASIS’s performance in the area of  soliciting, reviewing and providing feedback on proposals.
To determine whether CASIS, in collaboration with NASA, has  established performance metrics, we reviewed CASIS metrics as  presented in its fiscal years 2013 to 2015 Annual Program Plans. We  concentrated on fiscal year 2014 and 2015 metrics, but examined the  previous metrics to determine how performance measures evolved. We  also reviewed CASIS quarterly reports for fiscal year 2014 and the first  quarter of fiscal year 2015 and the fiscal year 2014 annual report to  determine how performance was measured and reported to NASA. We  analyzed CASIS’s fiscal year 2014 and 2015 metrics to evaluate whether  they adhered to GAO’s key attributes of successful performance  measures, which were identified in previous work. Judgment was required  to determine which attributes were applicable to assess and whether the  performance measures met the definition of the attributes selected. To  determine how NASA assesses CASIS performance, we reviewed the  cooperative agreement to determine relevant NASA responsibilities,  including the roles of the Cooperative Agreement Technical Officer and  NASA Liaison. We also interviewed the NASA Liaison to CASIS, the  Cooperative Agreement Technical Officer, and CASIS officials to gain  their perspective on the evolution of metrics and how they are used to  assess CASIS’s performance.
Our work was performed at NASA Headquarters in Washington, D.C.,  and Johnson Space Center in Houston, Texas. We also visited CASIS  headquarters in Melbourne, Florida.
We conducted our review from April 2014 to April 2015 in accordance  with generally accepted government auditing standards. These standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings based on our audit  objectives.
section_title: Appendix II: International Space Station National Laboratory Management Activities Required in the NASA Authorization Act of 2010
paragraphs: The National Aeronautics and Space Administration (NASA) Authorization  Act of 2010 directed that the Administrator shall provide initial financial  assistance to the organization with which the Administrator enters into a  cooperative agreement to manage the International Space Station (ISS)  National Laboratory. In August 2011, after a competitive process, NASA  signed a cooperative agreement with the Center for the Advancement of  Science in Space, Inc. (CASIS), a not-for-profit entity, to manage the  activities of the ISS National Laboratory through September 30, 2020.  The 2010 act outlined several management and research and  development activities for CASIS, as the not-for-profit entity selected, to  initiate, as follows:  1.  Planning and coordination of the ISS national laboratory research  activities.  2.  Development and implementation of guidelines, selection criteria, and  flight support requirements for non-NASA scientific utilization of ISS  research capabilities and facilities available in United States-owned  modules of the ISS or in partner-owned facilities of the ISS allocated  to United States utilization by international agreement.  3.  Interaction with and integration of the International Space Station  National Laboratory Advisory Committee established under section  602 of the National Aeronautics and Space Administration  Authorization Act of 2008 (42 U.S.C. 17752) with the governance of  the organization, and review recommendations provided by that  Committee regarding agreements with non-NASA departments and  agencies of the United States Government, academic institutions and  consortia, and commercial entities leading to the utilization of the ISS  national laboratory facilities.  4.  Coordination of transportation requirements in support of the ISS  national laboratory research and development objectives, including  provision for delivery of instruments, logistics support, and related  experiment materials, and provision for return to Earth of collected  samples, materials, and scientific instruments in need of replacement  or upgrade.  5.  Cooperation with NASA, other departments and agencies of the  United States Government, the States, and commercial entities in  ensuring the enhancement and sustained operations of non- exploration-related research payload ground support facilities for the  ISS, including the Space Life Sciences Laboratory, the Space Station  Processing Facility and Payload Operations Integration Center.  6.  Development and implementation of scientific outreach and education  activities designed to ensure effective utilization of ISS research  capabilities including the conduct of scientific assemblies,  conferences, and other fora for the presentation of research findings,  methods, and mechanisms for the dissemination of non-restricted  research findings and the development of educational programs,  course supplements, interaction with educational programs at all  grade levels, including student focused research opportunities for  conduct of research in the ISS national laboratory facilities.  7.  Such other matters relating to the utilization of the ISS national  laboratory facilities for research and development as the Administrator  may consider appropriate.
section_title: Appendix III: The Center for the Advancement of Science in Space (CASIS) Educational Activities
subsections: section_title: Educational Initiatives The Center for the Advancement of Science in Space (CASIS) Developed Educational Initiatives National Design Challenge Pilot Project
paragraphs: A national education campaign that provides  educators and their students the opportunity to  design and implement an authentic research  experiment on the International Space Station  (ISS).
Houston, Tex. – Six educators and 220  students completed experiments to fly to the  ISS on Orb-3 in October 2014.  Denver, Colo. – Three schools are currently  developing experiments that will be sent to  the ISS in spring on 2015. The pilot includes  105 middle and high school students.
Brings middle and high school students to the  Kennedy Space Center Visitor Complex and the  Space Life Science Lab to interact with an  astronaut and research scientist to send their  experiment to the ISS.
Six CASIS Academy Live events have been  held at the Space Life Sciences Lab and the  Kennedy Space Center for 390 Central  Florida middle and high school students.
Created to educate middle school students about  the ISS.
There have been nearly 15,000 total views of  the CASIS Academy student website with the  monthly average of 2,492 views. The  educators’ webpage has a total of 1,530  views averaging 255 monthly.
Work with volunteers across the nation who  communicate the CASIS mission and information  about recent research conducted on board the ISS  National Laboratory. Volunteers serve as pilot  tester, focus group and provide local training on  CASIS education programs.
Program brought awareness of the ISS and  CASIS science, technology, engineering, and  mathematics (STEM) activities to 450  educators and students at various workshops  and presentations.
Partnership with the Professional Golfers’  Association of America Center for Golf Learning  and Performance, Cobra Puma Golf and St Lucie  County Schools to bring together science and golf  by offering a 5-day golf summer cap to  underprivileged middle school students teaching  them math and physics .
Sixty-three middle school students  participated in the Professional Golfers’  Association of America STEM Camp in  summer 2014.
section_title: CASIS Educational Partnership Programs Zero Robotics Middle School Program
paragraphs: Five-week summer program for middle school  students to work in teams with program staff,  mentors, and scientists to learn about  programming, robotics and space engineering  while getting hands-on experience working with  and programming Synchronized Position Hold,  Engage, Reorient, Experimental Satellites.
There were 550 students and 110 teachers  from 9 different states who participated in the  program in summer 2014.
Offers students the ability to participate in near  real-time life science research onboard the ISS to  study foraging ant behavior.
The ant experiment was flown to the ISS in  December of 2013. A total of 8,814 students  in 32 states participated in program in  FY2014.
section_title: Educational Initiatives Story Time From Space
paragraphs: Objectives  Videotapes of astronauts reading selected stories  from the ISS.
Status   The videotapes were downloaded in January  2014. Fundraising efforts continue for Phase  2 in parallel with the development of the  demonstration kit of materials that will  complement the science content in the books.  There have been 6,500 student s and  educators participating in the program in  2014.
Students engage in the experiments design and  proposal writing process that culminates in flying an  experiment on the ISS.
CASIS is a national sponsor of Missions 5  and 6 in fiscal year 2014. This represents  more than 8,000 students actively engaged in  authentic research experiences. CASIS  presented to 400 of these students and their  parents at the Student Spaceflight  Experiments Program National conference in  Washington D.C.
A 4-week online program designed to take  participants on a simulated mission to the ISS as  “virtual astronauts”. Offered to middle and high  school students and children and adults outside of  the school system.
A total of 25 students and 25 educators  participated in the prototype version of the  Space Station Academy in July 2014.
The High School Students United with NASA to  Create Hardware program is a partnership between  high schools and NASA where students design,  build and implement an experiment in microgravity.
The experiment is being developed by a team  of students at Lakewood High School in  Colorado.
CASIS entered into a partnership with National  Geographic Learning/Cengage to help develop an  online interactive science program for grades K-6.
section_title: Appendix IV: Assessment of the (CASIS) FY 2014 Metrics Against GAO’s Key Attributes of Successful Performance Measures
paragraphs: Performance Metric  (18) Number of total flight projects  manifested as a result of solicited proposals  or investments  (20) Describe intended impacts/ outcomes of  ISS NL research and development to life on  Earth  (21) Report scientific or technological  breakthroughs related to use of the ISS NL  (22) Report transformational/ translational  science  (23) Report projects or activities contributing  to national scientific, educational, or  technology initiatives  (26) Report new initiatives to solicit interest  in/engagement with CASIS toward broader  utilization of the ISS  (27) Number of awards given to unsolicited  proposals  (28) Dollar ($) amount given to unsolicited  proposals  (29) Number and dollar ($) amount of  awards by type of responding organization  (other government agencies, academic,  individual, commercial, other)  (30) Dollar ($) amount contributed to  projects by non-CASIS sources, and their  origins (including targeted giving,  commercial entities, private investments)  (31) Dollar ($) amount and description of  flight projects provided by other government  agencies  (32) Describe actual impacts of ISS NL  research and development to life on Earth  (specific examples, as they occur)
section_title: Appendix V: Comments from the National Aeronautics and Space Administration
section_title: Appendix VI: Comments from the Center for the Advancement of Science in Space
section_title: Appendix VII: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Shelby S. Oakley, Assistant  Director; Richard A. Cederholm; Virginia Chanley; Maria Durant; Laura  Greifner; Ralph Roffo; Sylvia Schatz; and Roxanna T. Sun made key  contributions to this report.