# Presentation: 472717

## Metrics Planning & Reporting Study               H. K. Ramapriyan

- Metrics Planning and Reporting Study
- Status Overview

**Study Team:  H. K. Ramapriyan (Rama), NASA GSFC; **
**                     ****Bud Booth, Greg Hunolt, SGT, Inc.**
**“****Community” **
**Participants: Don Collins, JPL; **
**                     ****Frank Lindsay, U of MD; **
**                     ****Hank Wolf, GMU**
**                                    ****3/4/02**

## Metrics Planning & Reporting                        H. K. Ramapriyan

**Purpose of Study**
**Identify various types of institutions to be funded and appropriate funding mechanisms for participants**
**Define appropriate metrics collection and monitoring mechanisms for reporting (publicizing) performance (accomplishments)**
**Recommend, to Earth Science Enterprise, appropriate language for inclusion in various types of solicitations **

## Metrics Planning & Reporting                       H. K. Ramapriyan

**Approach**
**Engage community through workshops and survey interviews**
**Survey sponsoring and implementing organizations**
**Identify/Define “classes” of participants (data service provider classes similar to types of ESIPs; Program and Project offices) and define reporting requirements**
**Survey existing mechanisms for metrics planning and reporting, and their pros and cons**
**Contact projects (e.g., HST, ESSP), ESIP federation members and other entities to learn about mechanisms being used**
**Obtain opinions of sponsoring organizations about metrics information they are getting (and missing)**
**Identify metrics planning and reporting requirements for announcement opportunities and funding instruments**
**Identify requirements mandated by the government (NPGs etc.) as appropriate to different classes of participants and dollar levels**
**Identify documentation requirements for different classes of participants (Grants, Cooperative Agreements, Working Agreements, Contracts, IRDs, ICDs, Operations Agreements, etc.)**
**Identify requirements/funding flow options for the different classes of participants**

## Metrics Planning & Reporting                         H. K. Ramapriyan

**Community Workshop, Feb 5 – 7, 2002**
- ~15 individuals attended breakout session
- Representatives from HQ, DAACs, ESIPs and SEEDS team
- 3 new participants added to team, all 3 participate in weekly telecon with Ramapriyan, Greg Hunolt and Bud Booth:
- Don Collins, Manager, JPL PODAAC
- Hank Wolf, Assistant Director of CEOSR, George Mason University; Member, Seasonal to Inter-annual ESIP-2 Project
- Frank Lindsay, Manager, Global Land Cover Facility ESIP-2, University of Maryland
- Reinforced multiple viewpoints for metrics planning and reporting.  This will provide a basic framework for the study since it defines the relationships among the various “classes” of participants.
- Currently looking at 5 classes for SEEDS:
- NASA HQ, End Users, NASA (and Non-NASA) project sponsors, Data Providers, and Provider internal organizations.
- Accountability and metrics management, including specification of “value” and “success” measures all depend on what class you are considering.
- General consensus was that current metrics only partially reflect a provider’s performance, e.g., measures of utilization of data and products by the science community are currently not reflected in metrics collection.  The solution to this is not easy.

**Status**

## Metrics Planning & Reporting                         H. K. Ramapriyan

**Survey sponsoring and implementing organizations**
- Draft question list sent to Don Collins, Hank Wolf and Frank Lindsay (Focus Review Team) for review and comment, resulting in excellent feedback:
- Current draft largely ignored direct questions on accountability.
- Need to differentiate between a project’s organization and its institutional organization – two very different views (e.g., task-reporting and administrative-reporting).
- To improve clarity and simplicity,  like questions were grouped under their own headings, several questions were merged and simplified, and several new questions were suggested.
- Based on this feedback, a revised question list was sent to the Focus Review Team for further review and comment prior to delivery to participants.
- Additional participants were suggested:  NSF, ASF DAAC, SEDAC, DOE and NOAA facilities, financial community, commercial applications developers, geophysical community.  All will be given consideration as the study progresses.
- Survey questionnaire scheduled to be distributed to participants week of March 4.
- Each participant will be personally contacted and interviewed as part of the survey process.
- There are 26 participants on the initial “visit list.”
- The Focus Review Team recommended that the Questionnaire be posted to the SEEDS Web Site.

**Status, Cont’d**

## Metrics Planning & Reporting                         H. K. Ramapriyan

**Identify metrics planning and reporting requirements for various announcement opportunities and funding instruments**
- The goal is to define appropriate solicitation opportunities and funding instruments for the various “classes” of SEEDS participants and dollar levels
- Currently reviewing funding instruments:  grants, contracts, cooperative agreements, interagency agreements, NASA internal (Centers and JPL), Space Act Agreement.
- Currently reviewing various NASA opportunities:  AO, NRA, CAN, and NASA Announcements.
- Draft report scheduled for April 1, 2002

**Status, Cont’d**

## Metrics Planning & Reporting                         H. K. Ramapriyan

**Schedule**
**Task Start – December 2001**
**Draft questions to send to sponsors and implementing organizations – January 4, 2002 ****(*****completed)***
**Community Workshop - February 5-7, 2002 *****(completed)***
**Refine questions and “visit list” - February 15, 2002 *****(completed)***
**Distribute questionnaires to “visit list” - March 8, 2002**
**Draft report on metrics planning and reporting as a function of “class” – April 1, 2002**
**Obtain responses and conduct follow-up interviews – March – May 2002**
**Preliminary report – June 30, 2002**
**Further contacts with sponsors and implementing organizations as needed - July - October 2002**
**Recommendations to  ESE about metrics planning and reporting mechanisms - December 2002**

## Backup charts

## Metrics Planning & Reporting                         H. K. Ramapriyan

**Initial Visit List**

## Metrics Planning & Reporting                         H. K. Ramapriyan

**Questionnaire (page 1 of 5)**

## Metrics Planning & Reporting                         H. K. Ramapriyan

**Questionnaire (page 2 of 5)**

## Metrics Planning & Reporting                         H. K. Ramapriyan

**Questionnaire (page 3 of 5)**

## Metrics Planning & Reporting                         H. K. Ramapriyan

**Questionnaire (page 4 of 5)**

## Metrics Planning & Reporting                         H. K. Ramapriyan

**Questionnaire (page 4 of 5)**