Report Brief: 

NASA STEM Engagement Institutional Research Capacity Evaluation

October 2022

In order to inspire an informed society, NASA has invested considerably into various programsthat engage

the public in science, technology, discovery and exploration through a number of STEM programs and 

activities. These efforts align with their goals to create unique opportunities for students of diverse 

backgrounds to contribute to NASA’s work in scientific exploration and discovery to foster economic 

growth, embody American ingenuity, and serve as a magnet for the STEM workforce. 

Crucial to these efforts is a programmatic structure designed to enable authentic and relevant student 

contributions to NASA’s missions and work. The architecture aligns STEM engagement programs, present 

and future projects, and activities across NASA into an overarching framework and strategy. (See Figure 1 

for more details) One important aspect of this work is institutional research. 

To ensure that NASA continues to build and maintain institutional research capacity in pursuit of these 

goals, NASA’s OSTEM has engaged in an evaluation of its current programs (e.g., Space Grant, EPSCoR, 

and MUREP) to define institutional research capacity and identify avenues of growth.

Figure 1. Office of STEM Engagement (OSTEM) architecture.

Purpose of the Report Brief 

Building on findings from the EPSCoR Literature Review and Benchmarking study as well as the MUREP 

Program Level Evaluation, this report brief discusses the results of the NASA STEM Engagement 
Institutional Research Capacity Evaluation which investigated ways NASA can build institutional research 

capacity in pursuit of its strategic goals. The purpose of this evaluation was to:

1. Understand the ways EPSCoR, Space Grant, and MUREP activities collectively define and 

operationalize research capacity. 

2. Identify common definitions, metrics, goals, and practices to build institutional research capacity. 

3. Provide a baseline understanding of the alignment of institutional research capacity across Space 

Grant, EPSCoR, and MUREP activities and identify evidence-based practices. 

4. Make recommendations about external and internal performance measures to finalize learning 

questions and future evaluation activities. 

Evaluation Study Participants

NASA has a number of programs dedicated to inspiring STEM exploration. They range from programs that 

study the role of the oceans’ role in the Earth system to programs that focus on STEM curriculum 

development and providing opportunities for students to be involved in competitive research. This 

evaluation study focused specifically on the EPSCoR, Space Grant, MIRO, MUREP High Volume, OCEAN, 

MUREP INCLUDES, and M-STAR programs. (See Table 1 for descriptions of each program) 

Participants in this study included funded program leadership, staff, and stakeholders. Interviews were 

conducted with the participants to gather information from each regarding their perceptions of what 

institutional research capacity is, their experiences with activities that led to growing institutional research 

capacity, and their thoughts on what the products of the NASA investment produced deliverable-wise that 

supported increasing institutional research capacity within their organization. 

Table 1. Overview of Participating Programs and Activities

Programs/Activities Description

Established Program to 

Stimulate Competitive 

Research (EPSCoR)

EPSCoR establishes partnerships with government, higher education, and 

industry to improve research infrastructure and research and development 

capacity. The goal of EPSCoR is to provide funding to schools and 

organizations across the country that have not had the opportunity to 

engage in competitive research and technology development projects in 

areas critical to NASA’s mission. 

Space Grant The Space Grant Consortia fund fellowships and scholarships for students 

pursuing STEM careers. Additionally, the consortia focus on curriculum 

enhancement and faculty development. The goal of Space Grant is to 

contribute to the nation’s science enterprise through the funding of 

education, research, and public engagement projects through the network 

of university-based Space Grant consortia.

MUREP Institutional 

Research Opportunity 

MIRO strengthens and develops the research capacity and infrastructure of 

MSIs in areas aligned with NASA’s mission and national priorities. MIRO 

WHO PARTICIPATED IN STUDY?
(MIRO) awards aim to promote STEM literacy and enhance/sustain the capability of 

institutions to perform NASA-related research within the mission 

directorates.

MUREP Aerospace 

High-Volume 

Manufacturing and 

Supply Chain 

Management 

Cooperative

(MUREP High Volume)

High Volume seeks to address the need for a high-volume manufacturing 

and supply chain ecosystem in aerospace and future needs of 

manufacturing ecosystems by introducing new and diverse ecosystem 

networks. High Volume aims to reach undergraduate and graduate students 

from community colleges and four-year universities. In addition, High 

Volume collaborates with several business partners to create new 

opportunities for students. 

MUREP Ocean Biology 

and Biogeochemistry

(OCEAN)

OCEAN awards grants to MSIs to support NASA’s Science Mission 

Directorate in seeking a better understanding of the ocean’s role in the 

Earth system. This is done by enhancing the capability of MSIs to participate 

in funding opportunities and to support achieving a deep scientific 

understanding of planet Earth. 

NASA MUREP Inclusion 

Across the Nation of 

Communities of 

Learners of 

Underrepresented 

Discovers in 

Engineering and Science

(MUREP INCLUDES)

MUREP INCLUDES is an activity created in partnership with National Science 

Foundation INCLUDES. Through this partnership, MUREP INCLUDES seeks to 

broaden participation in the STEM workforce, with a focus on diversifying 

the engineering workforce. MUREP INCLUDES leverages the talents of 

students and researchers at MSIs to fund partnerships between 

organizations that have the capacity to broaden participation in the 

engineering workforce. 

MUREP Space 

Technology Artemis 

Research

(M-STAR) 

M-STAR provides awards to MSIs to promote STEM literacy and boost 

capabilities to engage in NASA’s Space Technology Mission Directorate

research. M-STAR seeks to increase institutional awareness of NASA 

competitive resources that can build the capacity of MSIs to offer and 

conduct STEM undergraduate and graduate research.

Details of the 2021-22 Evaluation

The 2021-22 OSTEM Institutional Research Capacity evaluation was guided by the following evaluation 

questions: 

HOW WAS STUDY CONDUCTED?

1. How do Space Grant, EPSCoR, and MUREP activities collectively define and operationalize 

institutional research capacity?

2. What are the common goals and objectives regarding institutional research capacity across 

Space Grant, EPSCoR, and MUREP activities?
3. In what ways are Space Grant, EPSCoR, and MUREP activities using evidence-based practices to 

support and build institutional research capacity?

a. What are the perceptions of impact from program/activity stakeholders?

This study used an exploratory qualitative design that used three modes of data collection:

1. Document analysis

2. Interviews with project managers

3. Interviews with a sample of external stakeholders from EPSCoR, Space Grant, and MUREP. 

Existing documents from each program/activity were collected as well as documents from NASA OSTEM 

Performance and Evaluation Team. These included project and activity solicitations, program evaluation 

reports, project planning documents, presentation slides, and any other documents identified as 

potentially relevant to the evaluation. 

Interviews were conducted with project managers, program staff, and technical/reporting monitors of 

Space Grant, MUREP, and Space Grant activities to provide information regarding the intent of each 

program. Interviews were also conducted with a sample of external stakeholders such as Directors, and 

Principal Investigators from MUREP, Space Grant, and EPSCoR. Three Space Grant and EPSCoR Directors, 

three MIRO Principal Investigators, and three High Volume Principal Investigators, for a total of nine 

interviews.

Data Analysis

This study used qualitative analysis methods such as document and interview analysis. Documents were 

analyzed using Bowen’s (2009) methodology of skimming, reading, and interpretation to identify codes 

and categories aligned to the evaluation questions. Themes from the document analysis were then used 

to determine if similar findings emerged in the interviews. 

Table 2. Evaluation Questions Aligned to Data Sources and Analysis Procedures

Data Collection and Analysis

Evaluation Question Data Source Analysis Procedure

How do Space Grant, EPSCoR, and MUREP 

activities collectively define and operationalize 

institutional research capacity?

• Document Analysis

• Interviews with Project 

Managers

Qualitative analysis 

What are the common goals and objectives 

regarding institutional research capacity across 

Space Grant, EPSCoR, and MUREP activities?

• Document Analysis

• Interviews with Project 

Managers

Qualitative analysis

In what ways are Space Grant, EPSCoR, and 

MUREP activities using evidence-based practices 

to support institutional research capacity?

a. What are the perceptions of impact from 

program/activity stakeholders?

• Document Analysis

• Interviews with Project 

Managers

• Interviews with 

External Stakeholders

Qualitative analysis 
Summary of Findings

Findings from the first question of the evaluation study suggested that: 

1. There is no common definition of institutional research capacity among NASA program staff.

2. There are three common components of institutional research capacity across programs – (a) 

infrastructure development, (b) building knowledge of faculty and students, (c) sustaining 

research through partnerships. 

3. Several metrics are useful for measuring the research capacity of awarded institutions – (a) 

faculty, student, and program development metrics, (b) physical infrastructure metrics, (c) 

partnerships and sustainability metrics, and (d) research-based metrics.

Findings from the second question of the evaluation study revealed common goals and objectives for 

institutional research capacity across Space Grant, EPSCoR, and MUREP programs. These goals included: 

1. Building institutions research infrastructure

2. Developing partnerships and securing additional sources of funding

3. Collaborating and aligning with NASA missions

4. Engaging with students to build the STEM workforce

5. Contributing to the broader research community

Findings from the third question of the evaluation study identified evidence-based practices being used 

to build institutional research capacity. These included: 

1. A focus on building partnerships 

2. Using funding for necessary equipment and resources

3. Sustaining research efforts through the development of curriculum, coursework, internships, 

fellowships, and scholarships that engage students in research and hands-on experiences

4. Engaging faculty in professional development trainings to support building research capacity. 

As indicated in the evaluation report findings, there are several recommendations for continuous 

program improvement to increase institutional research capacity moving forward which included:

• Adopting a common definition for institutional research capacity using the three components 

identified (infrastructure development, building knowledge of faculty and students, and 

sustaining research through partnerships) and the federal definition. 

• Developing common metrics for measuring the research capacity of institutions. Several metrics 

were identified in this study and can be used to assess institutional research capacity:

o Faculty, student, and program development metrics:

 Number of new coursework and curricula

 Number of workshops, trainings, and/or seminars

 Number of degrees awarded

o Physical infrastructure metrics:

 Number of new technologies, inventions, and/or techniques

 Number of patents and licenses

 Number of new research infrastructure developed or enhanced

 Number of new/updated websites or internet sites

o Partnerships and sustainability metrics:

 Number of new partnerships

WHAT CAN THE PROGRAM DO TO IMPROVE?
 Additional funding secured

o Research-based metrics:

 Number of conferences and presentations

 Number of books or other non-periodical publications

 Number of journal publications

 Number of conference proceeding publications

• Creating a preliminary logic model for building institutional research capacity that would allow 

for a shared understanding of the definition, goals, and overall purpose of institutional research 

capacity across programs. 

• Developing an infographic of the findings in this study to distribute to external stakeholders to

provide institutions with evidence-based practices to build institutional research capacity. These 

practices include:

o Encouraging collaboration and partnerships

o Using funding for infrastructure enhancement (equipment, resources, renovations)

o Sustaining research funding through non-NASA funding

o Developing new coursework and curriculum

o Creating internships, fellowships, and scholarships

o Engaging students in research

o Engaging students in hands-on experiences

o Conducting professional development and training

• Developing and administering a survey to capture the broader perceptions regarding institutional 

research capacity enabled from Space Grant, EPSCoR, and MUREP funding. 

WHAT ARE THE KEY TAKEAWAYS?

Although there is currently no common definition of institutional research capacity across NASA’s 

programs, there are common components and metrics that could be used to develop a common 

understanding in conjunction with the federal definition of institutional research capacity—"efforts to 

increase the ability of individuals and institutions to undertake high-quality research and to engage with 

the wider community of stakeholders.” 

NASA’s programs (e.g., Space Grant, EPSCoR, and MUREP) share common goals and objectives related to 

institutional research capacity and use several evidence -based practices to build institutional research 

practices.

The report recommended that NASA continue to build institutional research capacity by: 

• adopting a common definition of institutional research capacity and using common metrics across 

programs and activities

• creating a logic model for building institutional research capacity that would allow for shared 

understanding across programs and activities

• building partnerships, developing coursework and curriculum, engaging students in research and 

hands-on experiences that support NASA’s missions.