# Presentation: 187370

## James Avery, Ph.D.
Research Coordination Staff
Office of Science Policy

January 19, 2005

***Research Coordination Staff***
***Office of Science Policy***
***January 19, 2005***

- Overview of Multi-Year Plans

## What are Multi-Year Plans?

- Provide a framework that integrates research across ORD’s laboratories and centers and Government Performance and Results Act (GPRA) goals in support of the Agency’s mission to protect human health and the environment
- Describe the direction of ORD’s research program in a topic area over the next 5-10 years

## Structure of Multi-Year Plans

- Narrative
- Introduction and background that familiarize the reader with MYPs and Agency goals and objectives
- Progress to date from previous versions
- Overview of the Long Term Goals (LTGs)
- Description of the flow diagrams and the rationale used to sequence the performance measures
- Relationship to other research, e.g., work in other MYPs or by other Federal entities
- Performance and accountability information
- Logic Model
- Performance measures
- Flow diagrams that illustrate steps to attainment of the LTGs

## ORD Multi-Year PlansCurrent Topic Areas

- Goal 1
- PM & Tropospheric Ozone
- Air Toxics
- Goal 2
- Drinking Water
- Water Quality
- Goal 3
- Contaminated Sites & RCRA

- Goal 4
- Safe Pesticides/Safe Products
- Ecological Research
- Global Change
- Mercury
- Human Health
- Endocrine Disruptors
- Goal 5
- Pollution Prevention and New Technologies (transitioning to Science and Technology for Sustainability)
- Economics and Decision Sciences

## ORD’s Inclusive MYP Process

**EPA Strategic Plan**
***Government Performance and Results Act Goals***
**Clean Air  **
**Clean and Safe Water          **
**Preserve and Restore the Land**
**Healthy Communities and Ecosystems**
**Compliance and Environmental Stewardship**

**Customer/User Needs**
**EPA Program Offices**
**Regions/States**
**Federal research partners**
**Private Sector**

**Outside Peer Advice**
**Science Advisory Board**
**Board of Scientific Counselors**
**National Research Council**
**Scientific Peer Reviews**

**  **

**ORD Strategic Planning**
**ORD Strategic Plans**
**Peer Reviewed Research****       Strategies and Plans**

**Multi-Year Research Plans**

## How are MYPs developed?

- Written by a lead author and writing team
- Writing team consists of
- Laboratory and center experts
- Program and regional office representatives
- Office of Science Policy (OSP) staff
- Office of Resource and Management Acquisition (ORMA) staff
- Assume level resources using the most recent President’s budget
- Updated biennially or on an as needed basis

## MYP Development

**Agency Strategic Direction**
- GPRA goals/objectives/outcomes
- Program/Regional science needs

**Develop key science questions to be addressed by research**

**Develop long-term goals (LTGs)**
- Identify time-frame to deliver work
- Determine ORD role and role of others

**Determine annual performance goals to reach LTGs**
- Identify sequence to provide results** **
- Integrate research from all sources

**Develop annual performance measures**
- Determine who will accomplish work (in-house lab or center or STAR research)
- Ensure work can be done with available resources

## How are MYPs used?

- MYPs are used to
- Provide the link between Agency and ORD Strategic Plans and Research Strategies and Laboratory Implementation Plans
- Develop performance goals and measures
- Provide critical evidence on implementation of the Research and Development Investment Criteria and for the Program Assessment Rating Tool (PART)
- Plan the annual budget and explain the context and impact of budget decisions
- Communicate the direction of ORD’s research program internally and externally

## Annual

- Planning

- Multi-Year
- Planning

- Research
- Strategy
- (if available)

- Lab/Center
- Research Plans

- EPA
- Priorities &
- Budget Guidance

- EPA
- Strategic Plan

- ORD
- Strategic Plan

**EPA $**

## Link between Agency and ORD Strategic Plans 

- Agency and ORD Strategic Plans
- Set major areas of emphasis
- Provide GPRA goals that define environmental outcomes and ORD’s interpretation of science needed to achieve them
- Inform research strategies and plans, as well as MYPs, that translate overall direction into more specific strategic approaches, priorities and outcomes

## Develop Performance Goals and Measures

- Outcomes
- External to the research program
- Relevant to, and important for, the intended client, the agency, and the public (e.g., reduction in exposure to pollutants)
- Performance goals & measures:
- Describe the most important aspects of the program’s mission and priorities
- Include clear links to a program outcome
- Include a logic model, long term goals, annual performance goals and annual performance measures
- MYP guidance on performance measures is evolving based on continuing dialogue with OMB

## Develop Performance Goals and Measures:  Logic Model

- Logic Model links
- Specific environmental problems
- EPA’s goals and objectives
- Environmental decisions needed
- To the research required to strengthen decision-making

##  Logic Model:  Communicating How Environmental Research Contributes to Outcomes

*** ******Logic Model:  ****Communicating How Environmental Research Contributes to Outcomes*

## Develop Performance Goals and Measures:  Long Term Goals

- Long Term Goals
- Define the endpoints to be achieved from ORD research
- Should be
- Outcome-oriented and directly relevant to the research program in the MYP
- Not overly broad (e.g., “improve science”), but narrow enough to show contribution of the research
- Achievable by ORD in 5-8 years
- Defined so that success can be measured

## Develop Performance Goals and Measures:  APGs and APMs

- Annual Performance Goals
- Major steps necessary to achieve the LTG
- Described in the context of the outcome to which the research outputs contribute
- Typically multi-laboratory/center
- Annual Performance Measures
- Most important research outputs that measure the progress to achieve APGs and LTGs
- Examples include models, reports and publications, etc.
- Laboratory/center-specific for accountability purposes
- APGS and APMs should represent all of ORD’s projected significant accomplishments in a research program

## Develop Performance Goals and Measures:  Flow Diagram

## ORD Multi-Year Research Plans: Prospective Evidence that the Research Program Responds to the OSTP/OMB R&D Investment Criteria

***ORD Multi-Year Research Plans: Prospective Evidence that the Research Program Responds to the OSTP/OMB R&D Investment Criteria***

**Criterion 1:  **_**Relevance**_**. **R&D programs must be able to articulate _*why*_ their investment is important, relevant, and appropriate.  Programs must have well-conceived plans that identify program goals and priorities and identify linkages to national and client needs and to agency mission.

**Advances in research & scientific knowledge**

## ORD Multi-Year Research Plans: Prospective Evidence that the Research Program Responds to the OSTP/OMB R&D Investment Criteria

***ORD Multi-Year Research Plans: Prospective Evidence that the Research Program Responds to the OSTP/OMB R&D Investment Criteria***

**Criterion 2: **_**Performance**_**.**  R&D programs must have the plans and management processes in place to monitor and document *how well* their investment is performing.  Program managers must define appropriate outcome measures and milestones that can be used to track progress towards goals, and assess whether funding should be enhanced or redirected

- *Please note that Performance Goals include key research outputs , a synthesis product, & plans for effective transfer to intended clients to bring about short-term outcomes.  The synthesis product addresses a key research question, and compares the accomplishment represented by the Performance Goal to baseline conditions and to related Performance Goals in future years needed to achieve outcomes.

## ORD Multi-Year Research Plans: Prospective Evidence that the Research Program Responds to the OSTP/OMB R&D Investment Criteria

***ORD Multi-Year Research Plans: Prospective Evidence that the Research Program Responds to the OSTP/OMB R&D Investment Criteria***

_**Resources**_

_**Research**_
_**Activities**_

_**Research**_
_**Outputs**_

_**Short-Term**_
_**Outcomes**_

_**Effective Transfer**__** **_

_**Intermediate**_
_**Outcomes**_

_**Long-Term**_
_**Outcomes**_

**8.  **. ... which contribute to measurable changes in environmental contaminants, stressors, or exposures...

**Strategic**
**Objectives**

**Strategic**
**Goals**

**7****.**** **... which result in changes in client decisions or  actions ...

**6****.** client reactions and to changes in knowledge, attitudes, skills, or aspirations ...

**5.  **... and to prepare for the effective transfer of key research outputs that will lead to . . .

**3.** ... to create the research knowledge, tools, processes, technologies etc. and the key outputs that must be developed over time ...

**2.  **...to  coordinate and conduct the research, development, & administrative activities that  will be needed  ...

**1.** We use resources ($, FTE, & infrastructure),  partnerships, and guidance from stakeholders  ...

_**Specific Clients**_

**4.**  ... for intended clients (such as senior scientists or decision makers in government, the regulated community, or the public) ...
- ...

**Research Program **

**Results**

** ****Clients **

**9.** to demonstrate measurable long-term improvements in human or ecosystem health

**Criterion 3: **_**Quality**_**.**  R&D programs must justify how funds will be allocated to ensure quality R&D.  Programs allocating funds through means other than a competitive, merit-based process must justify these exceptions and document how quality is maintained.

- Examples of *Quality* Indicators:  	external expert review of STAR research;                                     			external expert review of scientific progress & “performance”

## MYPs Provide Critical Evidence for OMB’s Program Assessment Rating Tool (PART)

***MYPs Provide Critical Evidence for OMB’s Program Assessment Rating Tool (PART)***
- Agencies must submit evidence that support a research program’s: design and purpose, strategic planning, management, and results

- OMB requires agencies to submit results from independent, expert, evaluation

_**Research**_
_**Activities**_

_**Outputs**_

_**Outcomes**_

_**Clients Reached**__** **_

_**Outcomes**_

_**Outcomes**_

**Section I:  Program Purpose & Design**

**Section II:  Strategic Planning**

**Section III:  Program Management**

**Section IV:  Program Results**

**PART**

**Program Design**

**    ****Performance Goals**

- APMs

-    _Resources_

- 10 % of total score

- 20 % of total score

- 20 % of total score

- 50 % of total score

**        ****Program Results**

## MYPs and the Budget

- Serve as the basis for ORD’s budget request
- Provide information for the research budget Congressional justification
- Description of the planned research and application of the R&D investment criteria
- Used to prioritize work within and across MYPs for budget planning
- Provide information on the context and impact of budget decisions (increases or decreases)
- Form the structure for ORD accounting

## Communication

- Used to communicate with the program and regional offices about the research program
- Planned work
- Progress to date
- Used to communicate with external audiences, e.g., public and other research organizations
- Available on the Internet
- http://www.epa.gov/osp/myp.htm