Introduction to NASA Education

Performance Measurement

Patricia Moore Shaffer

Evaluation Manager

NASA Office of Education

July 16, 2015
NASA EDUCATION

STRATEGIC FRAMEWORK 

& PERFORMANCE 

REPORTING 

2
3

Strategic Objective 2.4: Advance the Nation’s STEM 

education and workforce pipeline by working 

collaboratively with other agencies to engage students, 

teachers, and faculty in NASA’s missions and unique assets.

NASA’s Strategic Goal 2: Advance understanding of Earth 

and develop technologies to improve the quality of life on 

our home planet.

2014 NASA Strategic Plan and 

STEM Education

3
NASA Annual Performance Plan*

• To support the measurement of Strategic Objective 2.4, 

Performance Goals and Annual Performance Indicators (APIs) 

with clear success criteria and evidence established

• Performance Goals remain stable throughout the Performance 

Plan Period (2014-2017)

• Annual Performance Indicators (APIs) are annual targets 

developed in consultation with OE leadership; “green,” 

“yellow,” and “red” ratings are assigned quarterly using predetermined success criteria

* Agency FY2016 annual performance plan/FY2014 annual performance report: 

http://www.nasa.gov/sites/default/files/files/NASA_FY14_APR-FY16_APP_Complete.pdf

4
FY2016 Performance Goals/APIs

5

Performance Goals (FY2014-17) Annual Performance Indicators (FY15)

2.4.1: Assure that students participating in NASA higher 

education projects are representative of the diversity of 

the Nation. 

ED-16-1: Provide significant, direct student awards in 

higher education to (1) students across all institutional 

categories and levels (as defined by the U.S. 

Department of Education); (2) racially or ethnically 

underrepresented students, (3) women, and (4) 

persons with disabilities at percentages that meet or 

exceed the national enrolled percentages for these 

populations, as determined by the most recent, publicly 

available data from the U.S. Department of Education’s 

National Center for Education Statistics for a minimum 

of two of the four categories.

2.4.2: Continue to support STEM educators through the 

delivery of NASA education content and engagement in 

educator professional development opportunities.

ED-16-2: Engage with at least 80,000 educators in 

NASA-supported professional development, research, 

and internships that use NASA-unique STEM content.

2.4.4: Continue to provide opportunities for learners to 

engage in STEM education through NASA unique 

content provided to informal education institutions 

designed to inspire and educate the public. 

ED-16-4: Maintain the NASA Museum Alliance and/or 

other STEM education strategic partnerships in no 

fewer than 30 states, U.S. Territories and/or the District 

of Columbia.

2.4.5: Continue to provide opportunities for learners to 

engage in STEM education engagement activities that 

capitalize on NASA unique assets and content. 

ED-16-5: Engage with at least 750,000 elementary and 

secondary students in NASA STEM engagement 

activities.
Office of Education Performance Reporting

A single Office of Education performance measurement 

system (OEPM) is used for reporting and survey 

administration.

All NASA activity managers and PIs (awardees) are 

required to report performance data into this system.

As of FY2014, NASA Education reporting utilized prior 

year data (i.e. FY2013) across the entire portfolio. 

6
On the Immediate Horizon

• New FY2017 API focused on ensuring that 

independent evaluations are conducted and 

reported by awardees 

• New agency strategic plan in place by FY2018, 

with ECC-level planning in FY2016 for new 

performance goals and outcome-focused APIs 

beginning this fall

7
FY2018 PERFORMANCE MEASUREMENT:

FROM OUTPUTS TO STEM-RELATED OUTCOMES

8
From Outputs to Outcomes Measurement:

What we’ve done so far

 Development of a logic model for each business line 

 Identification of outputs and short-term outcomes 

from logic models for performance indicators across 

business lines and projects that are aligned with 

Federal CAP goal and FC-STEM investment priority 

areas

 Development / identification of instruments to assess 

participant-level outcomes has begun

 Paperwork Reduction Act clearance from OMB for 

methodological testing of data collection instruments. 

 Improved OEPM reporting functionality

9
From Outputs to Outcomes Measurement:

What’s ahead

 Revision of OEPM data model/adjustment of OEPM data 

collection fields to align with output-focused performance 

indicators identified by the LOBs and funding programs

 Performance data review currently underway for Space Grant

 Instrument identification/development, testing and OMB 

Paperwork Reduction Act clearance to collect participantlevel outcomes

 Outcome measures are research literature based and STEMrelated

 First draft surveys prepared for NIFS with technical review panel 

held last March

 Development of outcome-focused APIs in FY2018

10
Paperwork Reduction Act and its Impact 

on Outcome Measurement

• All Federal public information collections require 

Paperwork Reduction Act clearance; each information 

collection clearance requires a minimum of 6-9 months 

not including testing/development

• OMB requires validity and reliability testing of 

instruments prior to clearing instruments

• Current clearance packages in process include:

– Individual activity and project-level data fields clearance

– STEM attitudinal surveys for middle school students for 

use in STEM Challenge activities

11
Visit NASA Education Performance 

Assessment

http://www.nasa.gov/offices/education/performance/

12