# Presentation: 871361

## GPRA Performance Reporting: Views from an S&T Bureau

- Paul DoremusStrategic Planning & Economic Analysis GroupNational Institute of Standards and Technology
- American Association for Budget and Program AnalysisFall Symposium
- 15 November 1999

## Two observations

- Mission matters
- Discrete services provided in large volumes
- Distributed services and/or intangible outputs
- Issues in measuring the generation and dissemination of knowledge
- Need for multiple measurement methods
- Size matters
- Lots of filters in the information pipeline
- What is the optimal level of resolution for GPRA?

**Notes:**
To understand performance you have to know a fair amount about the agency, its functions, and its management systesm.  
APR will not tell you very much.  Information filters tend to select quantitative data that can be reported annually.  For NIST, that means a partial set of key output metrics.  

## Size:  No small issue

## Department of Commerce

## National Institute of Standards and Technology

**3300 employees**
**$760 million annual budget**
**1200 industrial partners**
**2000 field agents**
**1550 guest researchers**
**$1.4 billion co-funding of industry R&D**
**national measurement standards**

***Helping America***
***Measure Up***

- National Institute of Standards and Technology

## NIST programs include

**Measurements and Standards**
**Laboratories**

**Advanced Technology Program**

**Manufacturing Extension**
**Partnership**

**Baldrige National**
**Quality Program**

- Unique co-funding partnership between NIST and private industry to accelerate the development of high-risk, enabling technologies with broadbenefits for the entire
-  economy and for society.

- Nation’s ultimate reference point for measurements, standards, and technology research to support industry, science, health, safety, and the environment.

- Nationwide network of locally managed extension centers offering technical assistance and best business practices to the Nation’s 380,000 smaller manufacturers.  Centers in all 50 state, DC, and Puerto Rico.

- 				Outreach program to 	promote business performance excellence and quality achievement by U.S. companies.  Annual Baldrige awards in manufacturing, service, small business, education, and health care.

## The mission drives the measures

**Goal**: Provide technical leadership for the Nation’s measurement and standards infrastructure, and assure the availability of essential reference data and measurement capabilities.

## The measurement challenge

**The measurement challenge**:  Developing quantifiable, verifiable, cost-effective outcome measures that accurately and comprehensively capture the impact of measurement and standards research over long time periods.

## Impacts are project specific

## Overview of NISTMSL performance measures

**External Review**:  NRC Board on Assessment (expert peer-review panels for each lab); direct feedback from industry clients; additional guidance from VCAT
**Output Metrics**:  Tracking and analysis of core products and services--SRMs, SRDs, calibrations and tests, staff technical publications, etc.
**Outcome Assessment**:  Microeconomic impact assessments provide quantitative and qualitative estimates of long-term outcomes from individual projects.
**Benchmarking**:  Comparisons with other National Measurement Institutes to assess “best in world” status

## NIST approach to GPRA reporting

## Strengths & weaknesses of measurement methods

## Performance reporting challenges

- Mission requires complex evaluation system
- No “vital few” quantitative measures
- Performance data not synchronized with budget cycle
- Measurement challenges combined with small size lead to information filters at higher reporting levels

## Performance reporting: filters

- Peer and/or external review a rich source of performance information...but unwieldy for GPRA reporting
- Impact studies ultimately are illustrative; results are not cumulative, and cover diverse funding periods
- MSL benchmarking data a key metric for top-level performance goal...but difficult to report concisely and/or comprehensively

## Laboratory benchmarking data

## The result:  Output metrics

- Capture partial reporting year outputs
- Quantity of products and services is not always the issue
- Strategy for SRM and calibrations:  focus on points of greatest leverage in measurement supply chain
- Publications:  goal is constant level of high quality output

## So in the end...

- ...just how informative will the FY 1999 Annual Performance Report be for an S&T bureau?
- Performance measurement and evaluation are central to how NIST operates
- GPRA provides additional motivation as well as a communication channel
- But how useful will the APR be for our stakeholders?