# Strategic Use and Implementation

of EPA OAR Voluntary

, Partnership

, and Community

Based Programs

A Report of the Clean Air Act Advisory Committee

October 25 , 2010Strategic Use a nd Implementation of Partnership Programs i



# Partnership and Community

- Based Programs Work Group

The CAAAC would like to recognize and thank the members of the CAAAC Partnership and Community

- Based Programs Work Group for their time and efforts in developing this report.



### S. William Becker

Nat ional Association of Clean Air Agencies Carolyn L. Green EnerGreen Capital Management, LLC

Richard Bolton Center for Toxicology and Environmental Health, LLC Brian Higgins
Nalco Mobotec

John Crouch Hearth, Patio & Barbeque Association,

Inc. Gary A. Jone s Printing Industries of America Graphic Arts Technical Foundation

Anthony John DeLucia, Ph.D East Tennessee State University Ursula Kramer Pima County Department of Environmental Quality

Michael Formica
National Pork Producers Council Jeffry C. Muffat

3 M Corporation

Jack Gehring Capterpillar, Inc. Frank Spassaro SDG&E/SoCal Gas (Sempra Energy Utilities)

Jack H. Goldman Hearth, Patio & Barbeque Association,

Inc. Eddie Terrill Oklahoma Department of Environmental Quality

Lisa P. Gomez Sempra Energy Uti lities Kathryn Watson, Work Group Chair Improving Kids’ EnvironmentStrategic Use a nd Implementation of Partnership Programs i 

### National Air Program Vision Statement

Achieve and maintain air quality that protects and enhances public health, welfare, and the

environment

. 

# Strategic Use and Implementation
of EPA OAR Voluntary, Partnership

, and Community

- Based Programs

Executive Summary
A Report of the Clean Air Act Advisory Committee

The Clean Air Ac t Advisory Committee (CAAAC) has developed this report to the U.S. Environmental Protection Agency’s ( EPA ) Office of Air and Radiation (OAR) to provide advice on the selection, implementation, and management of voluntary, partnership, and community

based p rograms. The observations and recommendations in this report fully align with and support the CAAAC’s Vision and Guiding Principles for the National Air Program that the CAAAC issued in February 2009. Key observations and recommendations in this report i nclude:

1. Partnership programs 1 are an essential component of EPA and OAR’s work to
protect human health and the environment.

Partnership programs are an important complement to OAR’s regulatory

programs.
OAR should finalize the revised draft principles ( included in this report) and use them to guide the design, implementation, and management of its partnership

programs. The “what to do” principles should be used by OAR in deciding whether to invest resources in a program , along with potential cost

effecti veness , and potential to cause long

- term sustainable change. The “how to do it” principles should be considered when developing and operating partnership programs. The CAAAC also recommends some modifications to the wording of the principles.

1 Throughout this report, the CAAAC uses the term “ partnership p rograms” to refer to voluntary,

p artnership , and c ommunity

- b ased p rograms that are managed or supported by OAR

.Strategic Use a nd Implementation of Partnership Programs ii 

### Possible Partnership Model Roles

EPA : technology, best practices, expertise, other support
NGO/Academia/Industry Association : princip al “community” contact, leadership, grant applicant

Private Sector : pilot efforts for market

transformation

Private F o undation: partial or full funding

Other F ederal/State/local/ T ribal
government

/Utility : expertise, funding ; community contact and other support 2. Proposed an d existing partnership programs should have clear, written plans, followed by a regular strategic review to continually improve the program

. Every proposed program should have a clear plan that includes the following elements:

1. Goals, objectives, and miles tones (including quantifiable goals and qualitative benefits);

2. A description of how the program will achieve results (i.e., a logic model or theory of change);

3. A plan for measuring and communicating performance and results;

4. An assessment of other EPA or th ird party programs that address the same issue so as to justify the proposed scope and scale of OAR’s program and role;

5. A funding plan that describes how the proposed program will consider
opportunities to leverage external resources ; and

6. Projected progra m lifespan and/or evolution.

All OAR partnership program plans should use a common planning template, which will allow OAR leadership to compare the merits of various programs and assist OAR in performing strategic reviews following implementation. The planning template
should enable assessment of how each program addresses the partnership program principles and incorporate the best practices discussed in this report, where appropriate.

Every program should undergo a strategic review following implement ation to: Identify and share “lessons learned”
within OAR, EPA, and with program partners and stakeholders;
Allow OAR to look strategically across its portfolio of programs;
Assess cost effectiveness; and
Determine appropriate program lifespan or evolution

.

3. Partnership programs should maximize opportunities to leverage resources.

OAR management should ensure that partnership program plans address opportunities to leverage resources, including money, in

- kind contributions, human resources, publicity, and c redibility.
OAR should explore opportunities to leverage resources from a variety of sources, including other EPA programs, federal programs, state/local/tribal
government programs, NGOs, trade associations, environmental groups, privateStrategic Use a nd Implementation of Partnership Programs iii companies, and fou ndations. Special consideration should be given to opportunities to leverage resources from non

- traditional partners.
OAR should look for opportunities to strategically deploy resources to support other organizations’ efforts to reduce emissions and impro ve air quality.

4. OAR should identify and share best practices within EPA and with outside

partners.

OAR should c reate mechanisms for identifying and diffusing lessons, best practices, and

tools to improve the design, implementation, and management of p artnership programs. The CAAAC has identified a number of best practices and lessons which EPA should consider in its development, implementation, and review of partnership
programs. Where appropriate, best practices should be incorporated into the common planning template (recommended above), as well as into each program plan. The CAAAC identified several best practices and lessons (see table below) during its
discussions; however, the CAAAC also considers this report to be a collection of lessons

and be st practices for OAR partnership programs.

Selected Best Practices and Lessons for OAR Partnership Programs

Use a more collaborative design process that identifies and engages key stakeholder groups and their leaders early in the process. “Buy

- in” and the establishment of a trust relationship with these groups is a key element of success.

Accommodate flexibility in the implementation of partne rship programs.

Expand and enhance use of social media in partnership programs, and other
computer

- based technology such as webinars.

Look for opportunities to enhance participation of disadvantaged communities in community

- based programs, including provid ing mentoring, training, and network development for leaders within those communities.

Set reasonable and achievable goals for partnership programs that are easy to assess and report.

Establish partnership programs with stages or milestones at which an ass essment is made to determine whether to continue or adapt the program.

Perform a regular strategic review of the entire portfolio of partnership programs to ensure they support OAR’s goals and priorities.

Create opportunities for participants to share thei r experiences internally and with others outside the partnership or program.

Encourage and implement peer

- to

- peer learning models in program design.

Focus on empowering communities to achieve sustainable, lasting change or improvement

.
Promote f riendly com petition and awards as useful tool s for engaging participants and sharing best practices.Strategic Use and Implementation of Partnership Programs 1 

# Strategic Use and Implementation
of EPA OAR Voluntary, Partnership

, and Community

- Based Programs

A Report of the Clean Air Act Advisory Committee

Final Report

– October 2 5 , 2010

The CAAAC strongly supports the use of partnership programs and encourages OAR to adopt the recommendations in this report to help ensure the strategic focus and implementation of voluntary, partnership, and community

- based programs.

In May 2010, the Clean Air Act Advisory Committee (CA AAC) was asked to provide the U.S. Environmental Protection Agency ( EPA ) with advice and recommendations regarding the strategic use and implementation of voluntary, partnership and community

- based programs to achieve EPA’s Office of Air and Radiation (OAR ) goals. 2 This report summarizes key observations and recommendations identified during the
CAAAC’s review and provides advice to OAR regarding the strategic use and implementation of partnership programs

.

The observations and recommendations in this rep ort fully align with and support the Vision and Guiding Principles for the National Air Program that the CAAAC issued in its February 2009 report. The Vision and Guiding Principles encourage EPA and its partners in the National Air Program to:

D raw on a diverse collection of regulatory and non

- regulatory tools and techniques

, including innovative approaches, to make rapid progress toward air quality and National Air Program goals in a cost

- effective manner. Careful consideration is
needed to ensure that tools and techniques , including regulatory, voluntary and partnership approaches, are deployed appropriately and at the right level of
government.

At the same time, the CAAAC recognizes in the Visions and Guiding Principles that “go vernment agencies fac e unprecedented funding and resource challenges du e to current economic conditions,” new challenges, and other factors. These factors can constrain EPA’s ability to invest in partnership programs that are not mandated by

2 Throughout this report, the CAAAC us es the term “ partnership p rograms” to refer to voluntary,

p artnership , and c ommunity

- b ased p rograms that are managed or supported by OAR

. The CAAAC believes this term is appropriate, given that nearly every voluntary, partnership, and community

- based prog ram involves partnership among a variety of entities.Strategic Use and Implementation of Partnership Programs 2

statute, even if they play a signi ficant role in achieving the Agency’s mission of protecting human health and the environment.



# Scope of this Report

This report is structured to provide strategic advice to OAR on the selection, implementation, and management of partnership programs. In d oing this, the CAAAC has addressed the five questions outlined in the charge to the CAAAC. These questions

are:

1. 

### What general principles should guide OAR investments in partnership and community

- based programs ? (Do you have suggestions for improving the proposed principles developed by OAR? )

2. What types of best practices should be considered when designing, implementing, and operating partnership programs? What best practices are relevant for community

- based programs?

3. 

### What improvements would create synerg ies and improve coordination across OAR’s and other EPA partnership and community

- based programs, including grant programs?

4. How can OAR better leverage opportunities to partner with others (e.g., other federal, state, local and tribal organizations, NGOs, foundations, industry associations, and others) to implement, operate, and evolve its partnership and community

- based programs? What partnership models could enable OAR to significantly expand progress towards its goals in light of limited resources?

5. How can OAR best understand if its partnership and community

- based programs are achieving results commensurate with the scale of the investment?

This report focuses on partnership programs in which OAR plays or has potential to play a significant role . The C AAAC believes that partnership programs complement EPA’s regulatory programs and should not replace or be undertaken at the expense of regulatory programs. Partnership programs, as discussed in this report, have several

purposes.

OAR uses partnership pro grams to cooperatively engage individuals, businesses, schools, communities and organizations to work together in mutually beneficial ways to solve environmental problems and foster sustainability. Partnership programs complement regulations, enforcement, and other Agency activities by providing information, assistance, or resources to enhance capacity, encourage behaviors, or
facilitate technology adoption. Partnership programs can also enable OAR to enhance capacity or encourage behavior s , technology ad option, or other actions in areas where the Agency may not have regulatory authority but that are important to meeting EPA air
and climate

- related goals. In other cases, partnership programs address problems not amenable to regulatory approaches, such as providing information directly to individuals, households, and assistance providers. Ultimately, OAR partnershipStrategic Use and Implementation of Partnership Programs 3

programs must be able to demonstrate cost

- effective results for achieving environmental and public health goals. 

# Framework for this Report

I n response to the charter for this effort and questions posed by EPA, the CAAAC identified five areas in which it provides summary observations and recommendations. 3 These five areas are summarized in the box below. This report is structured in sections
around each of these five areas.

3 The text box references the specific questions from EPA’s charter to the CAAAC for this effort that are addressed in each of the five areas used to structure this report. 

## Observation and Recommendation Areas

I. Principles and Leverage

– The principles that guide EPA OAR’s creation and use of partnership and community

- based programs a nd how these programs can be used to strategically leverage results which complement those achieved through regulatory programs

. (Questions 1 and 4)

II. Program Design

– The approach and process for designing partnership and
community

- based programs, incl uding the configuration of activities and roles to

achieve results. (Questions 2 and 4)

III. Program Implementation and Operations

– Best practices for how programs are implemented and coordinated to maximize effectiveness and efficiency in achieving

resu lts. (Questions 2 and 3)

IV. Measuring Results and Evaluating Performance

– The ways in which program performance and results are measured to inform program management, strategic planning, and transparent accountability. (Question 5)

V. Learning and Conti nuous Improvement

– The process and practice of capturing lessons learned and using information to improve and evolve partnership and
community

- based programs. (Questions 2 and 3)Strategic Use and Implementation of Partnership Programs 4



# Observations and Recommendations Addressing the Strategic Use and Implementation of Partnership Programs

I

. Principles and Leverage

1.1 OAR’s principles provide an effective tool for guiding design, implementati on, and management of partnership programs, although improvements can be made to enhance the principles and ensure their effective use.

Overall, the draft partnership program principles developed by OAR provide an important tool for guiding the developmen t and review of partnership programs. The draft principles should be organized in two groups

– one set focused on guiding OAR considerations of “what to do” (e.g., what programs to develop and/or invest in) and another set focused on “how to do” program d evelopment and implementation. The CAAAC also recommends some slight modifications to the language and focus of several of the principles. These suggested changes are reflected in the text box below. The CAAAC recommends that the revised draft principle s be finalized by OAR, communicated within OAR, and shared with key partners, as appropriate.

“What to Do” Principles

1. Address a priority environmental issue

2. Com plement (and not undermine) core regulatory and enforcement responsibilities, including filling regulatory and compliance gaps and building capacity to increase compliance

3. Enhance EPA’s relationship with the public and key stakeholders

4. Ascertain the appropriate EPA role along with those of other stakeholders (EPA could be considered for a lead role if that role cannot easily be filled by other levels of government or other organizations, at least at this time)

5. Effectively leverage other resources i n OAR, EPA, and other external organizations

6. Help build expertise and capacity within EPA related to a specific environmental issue or

to w orking with a group of partners

7. Help build capacity and effectiveness in communities, companies, and other organi zations to address environmental needs

“How to Do It” Principles

8. Set measurable goals, collect performance information, and report on environmental and public health results in a manner that supports accountability

9. Identify the appropriate duration of EPA involvement

– from helping convene partners, to launching and piloting new programs, to long

- term operation of a program

10. Be designed and operated with transparency to ensure clear communication with stakeholders and the public

11. Comply with the EPA partnership program guidelines and process requirementsStrategic Use and Implementation of Partnership Programs 5

The or iginal list of draft principles are provided in Appendix A, along with the CAAAC’s observations and comments on them. It will be important for OAR to def ine and articulate the process by which these principles should be used or considered. Additional context and editing would likely be needed when communicating these principles to stakeholders outside of EPA.

1.2 Encourage opportunities for collaboration and leveraging of resources with
traditional and non

- traditional partners.

OAR’s efforts to provide support to programs run by other EPA offices and federal agencies

– primarily in the form of expertise, advisory input, and
technical assistance

– represen t an important area for OAR to achieve results
with limited resources. Given the large number of potential opportunities for OAR involvement, OAR will need to prioritize resources to focus on those areas that promise and demonstrate the greatest returns t o protecting human health
and the environment. Important air quality and climate change co

- benefits can be realized by close
coordination with initiatives and grant programs managed both in and outside of OAR. For example, EPA has participated in the Sustainable Communities Grant Program managed by HUD (under the auspices of the HUD

- DOT

- EPA

Sus tainable Communities Partnership), the federal Task Force on Childhood Obesity , and in the Middle Class Task Force Recovery through Retrofit initiative

. Stimulus (ARRA) funding also provides a unique opportunity for building cross

program connections and synergie s while leveraging resources.
OAR should continue and expand the use of non

- traditional partners for collaboration and leveraging of resources. For example, academics can be perceived as more approachable and impartial as messengers for a program.

News media and other publicity outlets can also be invaluable resources to tap when conducting program

- related communication and outreach. Private companies can relate positive stories of profitable air pollution reductions that can encourage other companies. By educating partnership program staff in other federal agencies, EPA offices, and stakeholder groups about air quality issues, tools

, and resources, program participants and grantees can reach and influence broader audiences in a cost

effective, efficien t manner.

1.3 OAR should look for more opportunities to leverage partnership programs to accomplish regulatory objectives.

OAR should c ontinue and expand efforts to encourage and enable well

- designed partnership programs to receive “credit” under the Stat e Implementation Planning (SIP) process

. This might also include potential “credits” through activities in other media (e.g., green infrastructure, waste minimization). The CAAAC recognizes the important work that OAR has done to develop approaches and gu idance to assist State and tribal agencies to receive “credit” under the SIP process for non

- regulatory and partnership programs that addressStrategic Use and Implementation of Partnership Programs 6 air quality needs. See Appendix B for a list of EPA SIP “credit” guidance for partnership programs. There may be additional opportunities to expand awareness and use of existing guidance, to fill gaps in existing guidance, and to improve guidance periodically based on field experience. In addition, SIP credit opportunities should be explored during the design of

ne w OAR partnership programs, where appropriate. However, it is important to recognize that not all partnership programs may be well

- suited to accommodate SIP credits, and that developing viable pathways (e.g., guidance) for programs to provide SIP credits can require resources.



# II

. Program Design

2.1 Limited EPA resources require proposed partnership programs to have clear, written plans at the outset, followed by a regular strategic review to continually improve the program.

OAR should ensure that al l partnership programs, including major new enhancements to existing programs, have a clear plan which includes the following elements:

1. Goals, objectives, and milestones (including quantifiable goals and qualitative benefits);

2. A description of how the prog ram will achieve results (i.e., a logic model or theory of change);

3. A plan for measuring and communicating performance and results;

4. An assessment of other EPA or third party programs that address the same issue to justify the proposed scope of OAR’s progr am and role and identify possible leveraging opportunities;

5. A funding plan that describes how the proposed program will consider opportunities to leverage external resources ; and

6. Projected program lifespan and/or evolution.

Additional observations and re commendation related to program plans include:

Develop a common “planning template” (including the elements listed above)
for all OAR supported partnership programs that w ould allow OAR leadership to

compare programs across OAR’s portfolio and support OAR in performing strategic program reviews following implementation. The planning template should enable assessment of how each program addresses the partnership program principles and incorporate the best practices, where appropriate. While it is important that all programs have clear plans, the CAAAC recognizes that program plans may need to change rapidly as programs learn, adapt and evolve. Planning efforts should accommodate the flexibility needed to allow new types of partnership programs to address en vironmental needs and challenges in creative ways. Program plans should take a holistic or systems approach to environmental protection by looking for opportunities to support complementary initiativesStrategic Use and Implementation of Partnership Programs 7 and results in other environmental media areas aside f rom air, such as water and waste (where appropriate). Program designs that foster peer

- to

- peer learning opportunities are particularly useful in disseminating and communicating information and best practices among program participants and beyond.
OAR shou ld ensure that program plans do not become lengthy, bureaucratic documents that divert staff attention from important work. The value of program plans lies in ensuring important information is considered during program design and implementation and in bei ng able to clearly communicate and discuss this information with OAR managers and leadership. As reference, the draft December 2007 EPA document Guidelines for Designing EPA Partnership Programs was identified by several OAR program representatives as a us eful resource, and it should be made broadly available to
OAR offices. Every program should undergo a regular (e.g., annual) strategic review following implementation to:

o Identify and share “lessons learned” within OAR, EPA, and with program partners and s takeholders;

o Allow OAR to look strategically across its portfolio of programs;

o Assess cost effectiveness; and

o Determine appropriate program lifespan or evolution. While any formal policy on “sun

- setting” programs may be problematic and inflexible, a regula r internal strategic review across the portfolio of programs can help ensure that tough questions are asked that can help to improve programs, identify gaps and challenges, and ensure that OAR’s resources are directed where they can achieve the greatest be nefit. Careful attention will likely be needed to ensure that such a review process does not become a cumbersome bureaucratic exercise that inhibits candor and productive internal
deliberation. When OAR opts to disinvest in a specific partnership program, discussions with program participants can be useful to determine whether the program, or specific elements of it, can be effectively transferred to and operated by other

entities.

2.2 Explicitly consider alternative program models and different EPA roles during program design to maximize results and ensure program sustainability.

Programs do not always need to be designed with a lead role for EPA. In addition, more opportunities may exist for EPA to evolve its role over time, freeing resources to focus on other priorities. In many cases, an appropriate long

- term goal may be to have effective programs operating that require little or no investment by EPA. Alternative models for addressing a need should be actively considered before EPA invests in new progr am development. This typically requires assessing external organizations and initiatives that are currently working to address

— or who are well

- suited to address

— the problem or need (referenced above as a recommended part of a program plan).Strategic Use and Implementation of Partnership Programs 8 A range of mod els exist for engaging partners and for ensuring longer

- term program effectiveness and sustainability. For example, EPA was successfully able to develop and incubate the Hospitals for a Healthy Environment (H2E) program before launching it as a separate n ot

- for

- profit organization. In other cases, EPA has worked to support other organizations’ efforts to develop programs as an alternative to EPA developing a new program with the Agency in a lead role. In such cases, OAR may be particularly well

- suited in some areas to provide technical expertise and informational resources, tools, and templates to external organizations.
There can be divergence between OAR Headquarters programs and how they are translated at the regional level. Enhanced c larity around ho w OAR program s should be supported or implemented at the EPA Regional level should be included in program plans.

2.3 E ngage key stakeholder representatives earlier in the program design phase

. Some work group members observed that EPA can sometimes have an “I’m in charge” mentality when working with program s takeholders

. Partnerships necessarily involve give and take between parties as they jointly implement programs. It was noted that community

- based programs work best when partners have equal footing and where there are ample opportunities for communication and for participants to engage in meaningful ways that help build trust. Defining the roles and responsibilities of the partners up front is an identified best practice. Engaging key stakeholders

— such as community leaders, environmental groups, environmental justice groups, businesses, trade associations and academic and
research institutions

— during program design has the potential to strengthen shared ownership, enhance potential project impacts, increase long

- term investment by project partners, and open significant opportunities for OAR. Using more collaborative design approaches may require cultural changes within OAR which would require support from OAR leadership.



# III

. Program Implementatio n and Operations

3.1 Explore opportunities (in collaboration with EPA Regional Offices) to reach and support small, resource

- constrained, and/or impacted communities.

When EPA offers competitive grant opportunities to fund partnerships and community

- b ased projects , it is very difficult for small agencies to compete due

to limited staff resources and the lack of staff grant

- writing expertise . This is
also true for many community

- based and tribal organizations, which constrains the ability of some commu nities to participate in OAR programs. OAR should consider opportunities to enhance the capacity and competitiveness of such small and/or resource

- constrained agencies, organizations and impacted communities to apply for or participate in OAR

programs. Among other things, outreach and workshops for communitiesStrategic Use and Implementation of Partnership Programs 9 conducted by some EPA Regional Offices may provide useful assistance for overcoming these constraints.

3.2 Create opportunities to increase sharing of information and best practices among
EPA staf f.

Create opportunities to enhance information and skill sharing among OAR staff supporting similar functions (e.g., marketing, communications, recruitment, performance measurement) for different OAR partnership programs. For example, informal, cross

- OAR work groups of partnership program staff could
support cross

- program learning, while capturing lessons learned for future
benefit. Build into the design, implementation, and strategic review of partnership programs a methodology to identify, extract and s hare best practices within EPA and with outside partners. While identifying a full inventory of “best practices” for partnership programs is beyond the scope of this effort and report, the CAAAC has identified a number of “best practices” and lessons (summ arized in the table below) which EPA is encouraged to consider in its development, implementation, and review of partnership programs. The CAAAC also considers this full report to be a collection of best practices for OAR partnership programs.

Selected Best Practices and Lessons for OAR Partnership Programs

Use a more collaborative design process that identifies and engages key stakeholder groups and their leaders early in the p rocess. “Buy

- in” and the establishment of a trust relationship with these groups is a key element of success.

Accommodate flexibility in the implementation of partnership programs.

Expand and enhance use of social media in partnership programs, and other

c omputer

- based technology such as webinars.

Look for opportunities to enhance participation of disadvantaged communities in community

- based programs, including providing mentoring, training, and network development for leaders within those communities.

Set reasonable and achievable goals for partnership programs that are easy to assess and report.

Establish partnership programs with stages or milestones at which an assessment is made to determine whether to continue or adapt the program.

Perform a regular st rategic review of the entire portfolio of partnership programs to ensure they support OAR’s goals and priorities.

Create opportunities for participants to share their experiences internally and with others outside the partnership or program.

Encourage and implement peer

- to

- peer learning models in program design.

Focus on empowering communities to achieve sustainable, lasting change or improvement

.
Promote f riendly competition and awards as useful tool s for engaging participants and sharing best practices.Strategic Use and Implementation of Partnership Programs 10

3.3 Cont inue efforts to expand and improve use of social media to engage partners
and to communicate.

Several OAR programs already appear to be using social media in significant ways. OAR should expand efforts to encourage voluntary programs to use social
media t ools and technical information delivery mechanisms in appropriate ways that support program communication, outreach, and implementation goals. OAR should ensure that program staff members have access to training and resources to equip them to effectively u se social media and other technical information delivery mechanisms.

3.4 Create broad

- based support for partnership programs by involving a balanced set of key stakeholders, including industry, trade associations, community

- based organizations, environmen tal justice groups and environmental groups.

The credibility of all programs is increased when there is balanced representation and participation from a range of key stakeholders. Balanced stakeholder participation can help programs communicate with and s ecure program support from important constituencies, thereby enhancing participation in and performance of programs. Significant opportunities exist to involve a balance of stakeholders in partnership programs. For example, t rade associations can provide credibility to industry

- focused programs and mitigate potential concerns industry participants may have in working collaboratively with a government agency. Trade associations also have
well established networks which can enhance communication and outreac h to commercial and industrial target audiences. Significant opportunities exist to involve a balance of stakeholders in partnership programs. For example, as part of the Great American Wood Stove Change

- out Program, OAR is working with the Hearth, Patio , and Barbeque Association (HPBA) , state agencies, tribes, utilities, environmental groups and others to promote the replacement of old stoves with new, cleaner burning alternatives using creative partnership and communication approaches

. While state, loc al, and tribal government organizations are typically viewed as

co

- regulators by EPA, their involvement in the design and implementation of partnership programs can help broaden programs’ reach, effectiveness, and
results.



# IV

. Measuring Results and Evalu ating Performance

4.1 All OAR partnership programs should have a clear plan for measuring results and performance.

Evaluation and measurement are critical to being able manage programs effectively and for knowing whether to continue, terminate or revise them. Evaluation and measurement are also necessary to ensure accountability. As mentioned, all OAR partnership programs should have a clear plan forStrategic Use and Implementation of Partnership Programs 11 evaluating and measuring program performance and results. Regular program
review will also allow OAR t o look strategically across its entire portfolio of

programs. EPA’s draft June 2006 Guidelines for Measuring the Performance of EPA Partnership Programs is a useful resource to guide plan development

. The
CAAAC further recommends that the draft June 2006 Guidelines for Measuring the Performance of EPA Partnership Programs be finalized and periodically refreshed by EPA

. During the program development phase, participants should agree on

outcomes

— including environmental and human health outcomes

— and metrics to assess program performance and results. There are a variety of tools (e.g., “scorecards,” “dashboards”) that help program managers focus on a small number of key measures that can help program managers adapt and improve program results and performance. While measuring and evaluating program results and performance is important even for small programs, OAR needs to balance investment in quantitative measurement and evaluation activities with other important program needs
that can be more difficult to me asure such as community outreach, community education and community empowerment. While OAR needs to quantify the environmental or emissions reduction results achieved through program activities, OAR also needs to measure progress
relative to the scale of t he need or opportunity to address the problem. This could involve consideration of the total emissions reduction potential in the area being targeted by a program as well as the total population (e.g., individuals, schools, grocery stores) being targeted by the program. By assessing progress relative to the scale of the opportunity or need, OAR will be better equipped to determine whether a program is appropriate, or has sufficient resources, to adequately address the scale and scope of the challenge.

4. 2 Partnership programs have important benefits that are not easily measured quantitatively; these should be qualitatively assessed and considered during program reviews.

Partnership programs often have important qualitative benefits that may not be
easy to measure quantitatively. These benefits may include capacity building among communities, organizations, and within EPA as well as influences on behavior or technology adoption beyond participants directly involved in a program. OAR should consider these benefits when reviewing programs. In some cases, partnership programs provide valuable learning and capacity

building opportunities in areas that may eventually be addressed (at least partially) by regulatory programs. For example, OAR programs such as Cl imate Leaders have helped develop capacity and experience within EPA and among external audiences that have supported emerging regulatory initiatives such as EPA’s Greenhouse Gas Mandatory Reporting Rule.

M ore qualitative measure s and approach es for unders tanding how a program is achieving results should be developed i n cases where direct cause and effect (attribution) cannot easily be measured.Strategic Use and Implementation of Partnership Programs 12



# V

. Learning and Continuous Improvement

5.1 Catalyze information sharing and collaboration within and among programs by supporting peer

- to

- peer networks.

OAR should explore opportunities to catalyze greater information sharing and collaboration among program participants, partners, and grantees. This can also support efforts to harvest and disseminate lessons learned, case examples, and best practices more broadly. For example, OAR’s SmartWay Transport partnership program publishes case study information and OAR’s Climate Showcase Communities grant program periodically convenes grantees to share
information on common challenges, strategies, tactics, and resources.
Web

- based tools open new opportunities for program participants and grantees to directly share information peer

- to

- peer in real

- time and to collaborate in new tool development.

5.2 Create mechanisms for identifying and sharing lessons learned, best practices, and
tools.

As mentioned previously, OAR should develop efficient mechanisms to identify and share lessons, tools, and any materials developed across and among OAR partnership and community

- based program teams. OAR does not appear to have taken many steps to identify, collect, and share lessons learned across its partnership programs. Effective sharing of current and past program best practices, resources and tools is well worth the effort. Best practices should directly inform the development of the program planning template discussed earlier in this report. A mechanism or process for capturing lessons learned in partnership programs could be incorporated into the significant amount of evaluatio n and reporting that is currently being done within OAR. Periodic program evaluations can complement direct information sharing among program staff, as evaluation results in one program may be informative to other programs as well. Combining the two com plementary efforts can also avoid creating a large burden on staff time and resources.Strategic Use and Implementation of Partnership Programs 13



# Concluding Thoughts

Partnership programs are a vital component of OAR’s work and they complement regulatory programs in achieving EPA’s mission and the vision of the N ational Air Program. While these programs may not always be required by statute, they typically address important and timely needs and gaps

— enabling EPA and its partners to reduce emissions and better protect human health and the environment.

As disparit ies between needs and available resources grow, OAR must confront several challenges related to its support for partnership programs.

First, OAR must carefully select where to invest its resources, prioritizing areas that promise and demonstrate significa nt returns in achieving OAR’s goals and
EPA’s mission.

Second, OAR must intensify efforts to leverage resources and scale impact and results in its partnership programs, engaging non

- traditional partners and creative approaches.

Third, OAR must seek effi ciencies in implementing its partnership programs. Rapid learning and adaptation will help EPA and its partners get better results
faster.

The CAAAC recognizes that designing, implementing, and managing partnership programs is complex. As a result, it i s vital for OAR to engage with external partners and key stakeholders on these programs. The CAAAC applauds OAR’s willingness to seek advice on this important topic and hopes that this report advances OAR’s efforts to continually improve its programs.Strategic Use and Implementation of Partnership Programs 14



# A ppendix A: Draft OAR Principles for Partnership and Community

- Based Programs

The CAAAC has provided review and comment on the following draft principles provided by EPA OAR in the charter to the CAAAC for this effort.

OAR investments in partnerships and community programs must:

a. Address a priority environmental problem
b. Set measurable goals, collect performance information and report on environmental and public health results

c. Complement (and not undermine) core regulatory and enforcement responsibilities, i ncluding filling regulatory and compliance gaps
d. Enhance EPA’s reputation with the public

e. Justify the unique need for an EPA lead role, rather than other levels of government or other organizations f. Plan for the appropriate duration of EPA involvement

– from helping convene partners, to launch and piloting of new programs, to long

- term operation of a

program

g. Effectively leverage other resources in OAR, EPA and other organizations
h. Be designed and operated with transparency and accountability

i. Help build interna l environmental management capacity in communities, companies and other organizations j. Comply with the EPA voluntary program guidelines and process requirements

The CAAAC had the following specific comments related to the principles listed above.

As a re sult of the comments below, the CAAAC revised the draft principles accordingly (these revised principles are included in the report).

Principles “a, c, d, e, g, and i” are principles that should be considered when OAR is deciding what programs to develop or invest in. Principles “b, f, h, and j” relate more to how a program should be managed or operated to ensure its effectiveness. The CAAAC recommends having a set of principles that should be considered when assessing what programs to develop or invest in, or “what to do”. A second set of principles that focuses on “how to do it” should be considered during program design and periodic reviews to ensure each program is being operated in a manner that addresses these important considerations. Adjust princ iple “a” to use the word “issues” rather than “problems.” This allows for consideration of programs that seek to prevent areas from becoming environmental problems. For example, partnership efforts designed to prevent areas from being designated as non

- a ttainment for the NAAQS should be considered as important too.Strategic Use and Implementation of Partnership Programs 15 Principle “c” should also mention increasing compliance awareness and

assistance. Principle “d”, related to e nhancing EPA’s reputation with the public, will occur naturally if the program addre sses the other principles. The way it is worded, “d” appears too self

- serving and should be focused more on building trust with the public than building EPA’s reputation. Instead, more of an emphasis should be placed on communication and outreach with a focus on communicating about the program and its benefits for the community and the public. The long term goal for any partnership or program is to change behavior with an emphasis on a building capacity and launching a cultural change in communities,
bus inesses, and corporate practices that will sustain for the long term. To the extent that the principles are communicated broadly outside of EPA, OAR should consider making adjustments to principle “d” to be less “EPA

- centric.” The language in principle “e ” should be modified; “demonstrate” is a better word than “justify”. Or it could be modified to say “ascertain the appropriate EPA role along with that of other stakeholders. ” The language suggested helps to clarify that principle “e” should not discoura ge OAR from participating in and or supporting partnership programs in which it does not have a lead role. In principle “f”, “plan” should be changed to “identify. ” Expand principle “g”, related to the opportunity to leverage external resources,
to inclu de the ability to transition the program to a more susta inable funding
model over time that does not rely s olely on OAR budget allocations . While this should clearly not be a litmus test for new programs, it is an important area of
consideration. In princ iple “h”, it was unclear to the CAAAC how closely transparency and accountability are related and which was the main focus for this principle. The CAAAC observed that it would be useful to separate these concepts or to more clearly identify the connection between them. One approach would be to link “accountability” to principle “b”, as measurement and reporting are often designed to ensure accountability (in addition to enabling results

- based management). Clarify what is meant by the EPA guidelines and pr ocess requirements referred to in principle “j”. OAR could refer to the series of draft publications that were developed in 2006 and 2007 by EPA’s National Center for Environmental
Innovation with cross

- EPA input, however it would be important to note tha t the focus of these materials is on partnership programs and may not fully address some unique features of community

- based programs

. In addition to the principles listed above, OAR should consider adding a principle that states that partnership and commun ity

- based programs should create a long term benefit over the course of their lifespan. In addition, it may be useful to indicate that some consideration should be given to a program’s benefits and effectiveness in relationship to the program’s cost. At the same time, the CAAAC recognizes that this may not need to be a separate principle, and that these considerations could be incorporated into the process in which programs (and potential programs) are periodically considered in relation to these principl es.Strategic Use and Implementation of Partnership Programs 16



# Appendix B: EPA Guidance Related to SIP Credit for Partnership Programs

The CAAAC identified the following EPA guidance documents addressing opportunities to obtain “credit” in State Implementation Plans (SIPs) for partnership programs.

Incorporatin g Emerging and Voluntary Measures in State Implementation Plans: http://www.epa.gov/ttn/oarpg/t1/memoranda/evm_ievm_g.pdf

Guidance on Incorporating Bundled Measures in a State Implem entation Plan: http://www.epa.gov/ttn/oarpg/t1/memoranda/10885guideibminsip.pdf

Guidance on SIP Credits for Emissions Reductions from Electric

- Sector Energy Efficiency and Renewable Energy Measures: http://www.epa.gov/ttn/oarpg/t1/memoranda/ereseerem_gd.pdf

Guidance on Incorporating Voluntary Mobile Source Emission Reduction Programs in SIPs: http ://www.epa.gov/oms/stateresources/policy/general/vmep

- gud.pdf

Guidance for Quantifying and Using Emissions Reductions from Voluntary Woodstove Changeout Programs in State Implementation Plans: http://www.epa.gov/ttncaaa1/t1/memoranda/guidance_quantfying_jan.pdf

The SmartWay Program issued SIP guidance for states that are reducing long duration truck and locomotive idling. These resources include:

o Guidance for Quantifyin g and Using Long Duration Truck Idling Emission Reductions in State Implementation Plans and Transportation Conformity : http://www.epa.gov/smartway/transport/documents/tech/4 20b04001.pdf

o Guidance for Quantifying and Using Long Duration Switch Yard Locomotive Idling Emission Reductions in State Implementation Plans : http://www.epa.gov/smartway/t ransport/documents/tech/420B09037.pdf