National Aeronautics and 

Space Administration 

FY 19/20 Performance Measurement Strategy, Framework, and 

Measures 

August 27, 2018 
 

 

 

Performance Measure Development Key Takeaways 

Four key takeaways from the development of candidate performance measures: 

1. Comprehensive Performance Management Strategy 

2. Supports Agency Level STEM Engagement (SE) Strategy 

3. Rigorous Performance Monitoring 

4. Collaborative and Evidence-Based Approach 

2 
 

 

AGENDA

• Performance Assessment Strategy Development 

• Office of STEM Engagement Performance Assessment Framework 

• FY19 and FY20 Candidate Performance Goals and Annual Performance Indicators 

3
Performance Assessment Strategy 

Development 

4
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 •Review Past Performance 

External and Internal Performance Measures 

5 

Development Process 

STEP ONE STEP TWO STEP THREE 

Agencies, Literature Review, •Benchmark Other Federal 

& Stakeholder Discussions 

A brief history of NASA’s Office of 

Education performance measures 

• FY 2014 – FY 2017 Performance Goals, 

Annual Performance Indicators, Results 

• Strategic Object Annual Reviews (SOAR) 

• OMB and BSA guidance and 

recommendations: Concerns, need for 

change/reformed measures, processes, 

and tools 

Summary of guidance from 

literature and practice 

• NASA’s Goals and Priorities, including 

BSA priorities 

• Administrative Priorities and 

Legislative Directives 

• Literature review findings 

• Benchmarking findings 

• Internal stakeholder discussions 

• Mission Directorate discussions 

• Expert Review Panel - External 

stakeholder discussions 

Finalized Candidate 

Performance Measures 

Recommended External and Internal 

Performance Measures 

• Recommended external and internal 

performance measures 

• Recommended data collection sources 

• Recommended data collection processes 

• Recommended data collection tools 

• Recommended strategies for assessing and 

analyzing performance data 

• Recommended strategies for reporting 

performance data 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

External and Internal Performance Measures 

Development Schedule 

Performance Measures Development 

Milestones 

Date of completion 

Oct. Nov. Dec. Jan. Feb. Mar. Apr. May Jun. Jul. Aug. Sep. 

Internal Assess of the Office of Completed October 2017 

Education’s Evaluation & Performance 

Measurement including pervious APR’s, 

SOAR assessments, & OMB guidance 

Review of relevant literature (evidence- Completed March 2018 

based practices & current trends in 

STEM education and evaluation) & 

Benchmarking of Federal Agencies’ 

STEM education performance 

measurement strategies and tools 

Expert Review Panel Completed May/June 2018 

Draft External and Internal Performance Completed June 2018 

Measures Document, including data 

collection tools and processes 

Final External and Internal Performance Completed July 2018 

Measures Document, including data 

collection tools and processes 

(Candidate FY19 & FY20 Performance Completed July 2018 

Goals and Annual Performance 

Indicators due to OCFO-SID (July 2018). 

1 

2 

3 

4 

5 

6 

Initial Input 

Requested 

Follow-Up Input 

Requested 

Alignment with OSE Strategy Development 

July FY20 APP & Revised FY19 APP Due to OCFO – SID 

6 
 

 

Office of STEM Engagement 

Performance Assessment Framework 

7
 

 

 

8

FY 2019 – 2022 OE/OSE Performance Assessment Strategy 

Strategic Goals 

Strategic Objectives 

2018 NASA Strategic Plan 

Multi-year 

Performance Goals Annual Performance 

Indicators 

Agency 

Priority Goals 

Cross-Agency 

Priority Goals 

Mission Directorates/Mission Support Offices 

Traditional Performance Assessment Strategy 
FY19 & FY20 Office of Education/STEM Engagement 

Performance Assessment Model 

9 
 

 

 

 

 

 

 

 

 

PG: 3.3.5: Provide opportunities for students to contribute to NASA’s aeronautics, space, and science missions 

and work in exploration and discovery. 

Assessment Focus Area Example Performance Measures 

(Internal and External) 

Strategic Assessment Functions 

Question 

1 2 3 4 

Overarching Research 

Question 

To what extent are NASA’s 

STEM engagement 

investments contributing to 

NASA’s missions and work? 

10
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PG: 3.3.5: Provide opportunities for students to contribute to NASA’s aeronautics, space, and science missions 

and work in exploration and discovery. 

Assessment Focus Area Example Performance Measures 

(Internal and External) 

Strategic Assessment Functions 

Question 

1 2 3 4 

Overarching Research Types of activities that 

Question will be studied to answer 

the question. 

To what extent are NASA’s 

STEM engagement 

investments contributing to 

NASA’s missions and work? 

Higher Education 

Design Challenges and

Competitions 

Higher Education 

Internships and

Fellowships 

Higher Education 

Institutions Grants and 

Awards 

11
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PG: 3.3.5: Provide opportunities for students to contribute to NASA’s aeronautics, space, and science missions 

and work in exploration and discovery. 

Assessment Focus Area Example Performance Measures 

(Internal and External) 

Strategic Assessment Functions 

Question 

1 2 3 4 

Overarching Research Types of activities that Types of information and 

Question will be studied to answer evidence that will be collected 

the question. to answer the question. 

To what extent are NASA’s 

STEM engagement 

investments contributing to 

NASA’s missions and work? 

Higher Education 

Design Challenges and

Competitions 

Higher Education 

Internships and

Fellowships 

Higher Education 

Institutions Grants and 

Awards 

Quality and Alignment of 

Student and Institution R&D 

Products to NASA Mission 

Priorities, with an Emphasis 

on Assessing Student R&D 

Products 

Quality and Reach of Higher 

Education Student STEM 

Experiences 

12
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PG: 3.3.5: Provide opportunities for students to contribute to NASA’s aeronautics, space, and science missions 

and work in exploration and discovery. 

Assessment Focus Area Example Performance Measures 

(Internal and External) 

Strategic Assessment Functions 

Question 

1 2 3 4 

Overarching Research Types of activities that Types of information and Specific performance measures that 

Question will be studied to answer evidence that will be collected will be used to answer the question 

the question. to answer the question. (output, outcome, or milestone) 

To what extent are NASA’s 

STEM engagement 

investments contributing to 

NASA’s missions and work? 

Higher Education 

Design Challenges and

Competitions 

Higher Education 

Internships and

Fellowships 

Higher Education 

Institutions Grants and 

Awards 

• Number and quality of student research (peer- Quality and Alignment of reviewed publications and technical presentations) Student and Institution R&D and work products produced. Products to NASA Mission • Multiple case study to characterize the NASA 

Priorities, with an Emphasis student research experiences’ contributions to 

on Assessing Student R&D missions and positive impact to the student. 

Products 

• Alignment of higher education challenges, 

Quality and Reach of Higher competitions, and internships and fellowships to 

Education Student STEM evidence-based effective practices. 

Experiences • Number and academic level of student participants in 

STEM experiences. 

• Geographic distribution of participants. 

13
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

-

-

-

PG: 3.3.3: Provide opportunities for students to engage with NASA’s aeronautics, space, and science people, 

content, and facilities in support of a diverse future NASA and aerospace industry workforce. 

1 Strategic Assessment 2 Functions 3 Assessment Focus Area 4 Example Performance Measures 

Question 

NASA Provided K12 

Student STEM 

Experiences 

Higher Education 

Internships, and

Fellowships 

Partner Delivered and 

Collaborator Delivered 

Student Experiences 

Higher Education 

Challenges and

Competitions Diversity of Higher Education 

Challenges, Competitions, 

Internships, and Fellowships 

To what extent are NASA’s Participants 

STEM engagement 

investments contributing to 

the diversity of the future 

aerospace industry’s STEM Quality and Reach of K12 

workforce? Student STEM Experiences 

Strategic Alignment to NASA 

and External Stakeholders 

Needs and Priorities 

(Internal and External) 

• Meet or exceed demographics for national enrollment 

of traditionally underrepresented audiences. 

• The average annual percentage gaps between higher 

education interns and fellows from groups 

traditionally underrepresented and not traditionally 

underrepresented in STEM fields decrease. 

• Results of the feasibility analysis and benchmarking 

study to assess the possibilities and constraints of 

implementing a robust longitudinal study of NASA’s 

Office of Education/STEM Engagement’s contributions 

to the aerospace industry workforce. 

• Alignment of K12 student STEM experiences and 

internships and fellowships to evidence-based 

effective practices. 

• Number and academic level of student participants in 

STEM experiences. 

• Geographic distribution of participants. 

• Internal and external stakeholder satisfaction survey 

results. 

14
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

PG: 3.3.4: Enhance the effectiveness of education investments using performance assessment and 

evaluation-driven processes. 

Question 

To what extent have 

enhancements to STEM 

engagement performance 

assessment and evaluation 

been implemented? 

Learning Agenda 

New or Enhanced Data 

Management System 

Outcome Evaluation 

1 Strategic Assessment 2 Functions 3 Assessment Focus Area 4 Example Performance Measures 

(Internal and External) 

• Define requirements and pilot test a performance 

system to align with new measures. 

• Develop a Learning Agenda that includes: a) enhanced 

performance assessment strategy, b) business process 

for third party evaluations, c) business process for 

evidence-based project-level decision making and d) 

business process for compliance with federal 

mandates on data collection. 

• Establish a process to refine strategic assessment 

questions and performance measures as a result of 

findings from data collection cycle and maturation of 

Agency SEC Portfolio. 

• Conduct a pilot outcome assessment of higher 

education challenges and competitions. 

Quality of Performance 

Assessment and Evaluation 

15
 

 

 

 

 

 

 

 

 

 

 

 

 

16

FY19 & FY20 Office of Education/STEM Engagement Performance Assessment Model 

Diversity of Higher Education 

Challenges, Competitions,

Internships and Fellowships 

Six Assessment

Participants 

Focus Areas

Quality and Alignment of 

Student and Institution R&D 

Products to NASA Mission 

Priorities, with an Emphasis on 

Assessing Student R&D Products 

Quality and Reach 

of K12 Student 

STEM Experiences 

Strategic Alignment 

to NASA and External 

Stakeholders Needs 

and Priorities 

Quality of Performance Assessment 

and Evaluation 

Quality and Reach 

of Higher Education

Student STEM 

Experiences 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Assessing the Impact of NASA’s STEM Engagement Investments:

Performance Assessment & Evaluation Overview 

•Broad strategic goals designed to advance NASA’s mission and

address relevant national problems, needs, challenges and 

opportunities. 

•Timeframe: 2018 - 2022 

Strategy 

Performance 

Framework 

Performance 

Assessment 

FY19 and FY20 

Annual 

Performance 

Indicators 

•Ongoing monitoring and reporting of program accomplishments,

particularly progress toward pre-established goals. 

•Includes outputs such as: higher education interns and fellows 

demographics and number of paper presentations and peer-reviewed

publications; and outcomes such as developing higher education

students’ science or engineering identity. 

•Timeframe: Short term (Annual) 

•Systematic study using research methods to collect and analyze

data to assess how well a program is working and why. 

•Includes outcomes such as: developing higher education 

students’ science or engineering identity, cognitive understanding

of research processes and skills, or longitudinal study of interns. 

•Timeframe: Long term 

Evaluation 

Evaluation 

Assessment 

Performance 

FY19 Outcome Evaluation 

Planning & Feasibility 

Analysis for Longitudinal 

Study 

FY20 Conduct Outcome 

Evaluation 

17 
Questions

18