## Department of Energy Washington, DC 20585



### November 29, 1995 MEMORANDUM FOR: Paul Brodie Rick Fedrizzi David Goldstein Joe Mattingly Bill Prendergast Edward Reid Chuck Samuels Dick Stillwell 

### FROM: Christine A. Erv Assistant Secret ohn Wil.on Energy Efficienckantenewable Energy SUBJECT: Appliance Standards Process Improvement Thank you for all of your hard work at the first meeting of our process improvement on Tuesday, November 21, 1995.. It was heartening to come back to the meeting in the afternoon and to see the progress that the group had made-in developing a framework for future discussions. I want you to know that we appreciate your commitment to work with us on improving this process. 

### Our facilitator, Doug Brookman, has faithfully transcribed and made sense of our notes from the meeting. You can see how productive the meeting was when you review the attached documents. The group suggested a very aggressive time line for completion of our process tasks and has scheduled the following dates for meetings of the full stakeholder group: January 3, 1996 Planning "Basket" January 17-18, 1996 Data/Analysis "Basket" February 13, 1996 Decision making "Basket" In order to stay on this tight timef.rame, several members of the group volunteered to work with our staff, through a series of conference calls, to develop a draft agenda/work plan for each of these meetings. Attached is a matrix with the dates and times of these calls. The first "Planning" conference call has been scheduled for December 7, 1995, the first "Input-Analysis" conference call has been scheduled for December 12, 1995 and the first "Decision making"conference call has been scheduled fora January 30, 1996.
Pratte i.ph soy ink am VSC CI w pgp i Matt Chwalowski Gene Foley Steve Nadel Kyle Pitsor 

### W.A. "Buzz" Reed Joseph Romm Douglas Smith



### coon "T1T-T •- .:......«. 

### ,**coo~** 

### Department of Energy Washington, DC 20585
MEMORANDUM FOR: 

### Paul Brodie Rick Fedrizzi David Goldstein Joe Mattingly Bill Prendergast Edward Reid Chuck Samuels Dick Stillwell FROM: 

### Christine A. 

### Erv Assistant Secret Energy Efficienc November 29, 1995 Matt Chwalowski Gene Foley Steve Nadel Kyle Pitsor W.A. "Buzz" Reed Joseph Romm Douglas Smith

ohn**W~l** 

### SUBJECT: 

### Appliance Standards Process Improvement Thank you for all of your hard work at the first meeting of our process improvement on Tuesday, November 21, 1995." It was heartening to corne back to the meeting in the afternoon and to see the progress that the group had made" in developing a .framework for future discussions. I 

### want you to know that we appreciate your commitment to work with us on improving this process. Our facilitator, Doug Brookman, has faithfully**tr~nscribed** 

### and made "sense of our notes from the meeting. You can see how productive the meeting was when you review the attached documents. The group suggested a 

### very aggressive time line for completion of our process tasks and has scheduled the following dates for meetings of the full stakeholder group: January 3, 

### 1996 January 17-18, 1996 February 13, 

### 1996 Planning "Basket" Data/Analysis "Basket" Decision making "Basket" In order to stay on this tight t.imeframe, several members of the group volunteered to work with our staff, through a series of conference calls, to develop a draft agenda/work plan for each of these meetings. Attached is a matrix with the dates and times of these calls. The first "Planning" conference call has been scheduled for December 7, 

### 1995, the first "Input-Analysis" conference call has been scheduled for December 12, 1995 and the first "Decision making" conference call has been scheduled for January 30, 1996.**~T"T'"** .". ........_ ..._. 

# ..All calls will begin at 2:30 pm. Mike Rivest of the A.D. Little

staff will. be in touch with the volunteers-to see if these dates and times work for the majority of the group. He also will 

### provide the participants with*the.teleconference numbers. An important reminder. If there are other stakeholders who you believe should be invited to participate in the process improvement, please contact Deborah,Miller at 202/ 586-8888 as 'soon as possible. 

### Again, thank you for your participation. I look forward to seeing you on January, 3, 1996! Attachments

tool 31-TTI'I 'a ?Iti$LW C SCSC98ZOZI ST:6T S6/OC/TT**too~** 

### All calls will begin at 2:30 

### pm. Mike Rivest of the A.D. Little staff will. be in touch with the volunteers to see if these dates and times work**~or** 

### the ma10rity of the group. He also will provide the participants with-the teleconference numbers. An 

### importa:1t reminder. If there are other**stakehol~ers** 

### who you believe should be invited to participate in the process improvement, please contact Deoorah'Miller at 202/586-8888 as ·soon as possible. Again, thank you for your participation. I look forward to seeing you on January, 3, 1996! Attachments

:nu11 .

## a lIl1HJ.HV 

## n :BY S6/0Clttn 0 0 

# Task Force/Stakeholder Group Milestones for Process Improvement Draft Schedule U L 

# 1.46 Om ^3-i-^i • ^^'rs'.[ -'Lay'1 . ^'=: ^ O

R :3»L ,^>c'L-ii•`r -3i ^+ -.,° _^=

^^

_ ^}=r^S^^s '" a:aoT^'35 -c. `'rz':...s^cr

^" +^ r;ui ^u• 3.r ^R*s^, ssc 2-; tir ii F. e: •'_ <$- g^r '^Sx.-r.i-- =a __ .^i :^-= ^K=`".".,^`:=^c.S<:=^:•-e..^a;.s,^.z.$„}•^_'.;s^_'s•:r^c,f^, ::s ^,';^ •c>;cZ- .Jt?Tn . ^!'.p, "^a-ts' .p^E .-'^c^^sc^: A 

## uecernber 7 J December 14 2:30 pm to 3:30 pm 

# 1 2:30 pm to 3:30 pm
January 30 February 6 

## 2:30 pm to 4:30 pm 2:30 pm to 3:30 pm Planning Task Force: Input/ Anaylsis Task Force: 

## February 9 February 13
David Goldstein, Ed Reid, Chuck Samuels Matt Chwalowski, Gene Foley, Joe Mattingly, Kyle Pitsor, Bill Prendergest, Chuck Samuels 

### Decesion Making Task Force: Steve Nadel, Bill Predergest, Chuck Samuels, Dick Stillwell ? Q ? Q ? I) o 

### ... :> 

### ... l



# . I) of 

# TaskForce/StakeholderGroup Milestones for ProcessImprovement

2:30pm to

## 4:30 PJn 

# January 30 2:30 pmto4:30 pm Planning TaskForce: 

## Itlptltl Anaylsis'

# TaskForce:

.. 

### , 

### Decesion Making Task Force:

. " 

# DraftSchedule

December 15 

## January 3 

## 2:30pm to

## 3:30

## pm January 

# 9 January 12 

## January 17-18 

## 2:30 pm to 3:30pm 

# February 6 February 

# 9 Februruy 13 

## 2:30 pm to 3:30pin David Goldstein, Ed Reid, ChuckSamueJs 

## Mati**Chwalowski~** 

### Gene Foley, Joe Mattingly, Kyle Pitsor, 

## Bill Prendergest, ChuckSamuels 

### Steve Nadel. BiD Predergest. Chuck Samuels, Dick Stillwell

## OVERVIEW MEETING SUMMARY 11/21/95 STAKEHOLDER GROUP ON RULE MAKING



### PURPOSE OF MEETING • Begin to explore range of diverse views [and ultimately collect advice] on. Rule-making Implementation Process Define goals, major outcomes, elements and a pathway to initiate a Rule-making Implementation Process



### GOALS/OUTCOMES GOAL: Improve quality and clarity of communication with consumers

and stakeholders Look at flaws in the law or application of the law (Look at this first) OUTCOME: Define what process should be to improve S/H participation

Recommendations on how to improve the process Get input to DOE about how to do this Rule-making better



## OVERALL GOAL
-j PLANNING: Time - 80% DOE 20% Other -?j INPUT DATA ANALYSIS ^j COMMENT DECISION MAKING Outcome = Substantive Input That Leads to Process Improvements
1 

# OVERVIEW· MEETING SUMMARY 11/21/95 STAKEHOLDER GROUP ON RULE MAKING PURPOSE OF MEETING 

# v • Begin to explore range ofdiverse views [and ultimately collect advice] onRule-making Implementation Process 

# v • Define goals. majoroutcomes.elements and a pathway to initiate a 

### GOAL:Rule-making Implementation Process 

### GOALS/OUTCOMES Improve quality and clarity ofcommunication with consumers and stakeholders Look at flaws in the law or application ofthe law (Look at this first) 

### OUTCOME: Define what process should be to improve 

### SIH participation Recommendations on how to improve the process Get input to DOE abouthow to do this Rule-making better 

## OVERALL GOAL 

## c**~** PLANNING: Time - 80% DOE 20% Other 0 

# e**~** INPUfDATA M M 

# -+ ANALYSIS E**~** COMMENT N 

# -+ DECISION MAKING Outcome

# = Substantive InputThatLeads to Process Improvements
1

## OVERVIEW ROLE OF GROUP • Inform DOE's Rule-Making Process • Part of planning process - Makes sense (what we do) - Identify ways to proceed (guidance) - Vehicle to provide information to Stakeholder groups - Identify areas to improve - Results in research agenda • Provide actual substantive advice • Advise on internal review process NEXT STEPS Stakeholder group will spend I day per issue area Meeting Schedule: January 3 Schedule Planning (I Day) Stakeholder Group Reps: Chuck Samuels, Ed Reid, David Goldstein January 17-18 Input - Analysis (1.5 - 2 Days) Stakeholder Group Reps: Buzz Reed, Joe Mattingly, Gene Foley Bill Prendergest, Kyle Pitsor, Matt Chwalowski, Chuck Samuels February 13 Decision Making (1 Day) Stakeholder Group Reps: Steve Nadel, Bill Prendergest, Dick Stillwell, Chuck Samuels STRAWMAN: • DOE/ADLJB-K plus members create strawman prior to each meeting - Conference call with members to scope strawman - Drafting - Conference call to refine strawman

2 

# OVERVIEW ROLE OF GROUP
• Inform DOE's Rule-Making Process • Part ofplanning process

- Makes sense (what we do)

- Identify ways to proceed (guidance)

- Vehicle to provide infonnation to Stakeholder groups

- Identify areasto improve

- Results in research agenda • Provide actual substantive advice • Advise on internal review process 

## NEXT STEPS Stakeholder group will spend 1day per issue area Meeting Schedule: 

### January 3 Stakeholder Group Reps: 

### -January 17·18 Stakeholder Group Reps: 

### February 13 Stakeholder Group Reps: STRAWMAN: Schedule Planning (1 Day) Chuck Samuels,Ed Reid, David Goldstein Input - Analysis (1.5 - 2 Days) Buzz Reed,Joe Mattingly, Gene Foley Bill Prendergest, Kyle Pitsor, Matt Chwalowski, Chuck Samuels - .Decision Making (1 Day) Steve Nadel,Bill Prendergest,Dick Stillwell, - Chuck Samuels

• DOEIADUB-Kplus members create strawman priorto each meeting

- Conference call with members to scope strawman

- Drafting

- Conference call to refine strawman
2

### ONE THING TO CHANGE

If you could change one thing about DOE's rule-making process, what would it be?

• Done faster

• Broader array of inputs into process
• Multi-directional dialogue at early stages in process before ANOPR and between ANOPR and NOPR 14

• More participation by all groups in all stages of process -- especially at the beginning A
• Re-focus process on developing and communicating rational basis for consumers to choose (the process does not accomplish this at all) 

### 1^M

• Clearer definition of ground rules, such as at is economically justifiable to ?wA

• Defined end point for product when reach diminishing returns
• More realistic better weighting of input values that lead to results
• Better input as labs develop their models (models not developed in a vacuum) SA
• Find outcomes that maximize benefits to all 

# PVA • All be happy!
• Faster + DOE should have more discretion - Especially regarding the statute
• End products should be a better communication vehicle for consumer to make rational decisions
• More use of negotiation - Eliminate/limit the points of disagreement 9)4/_ZA 1,vM
• Use better judgment in development of technical assumptions ?A
• Ongoing, open dialogue, faster 01,t 

## 1. / t t1 '

• Incorporate more opportunity for give-and take. Create opportunities to bargain 

## ONE THING TO CHANGE If 

### you could change one .thing about DOE's rule-making process, what would it 

### be? • Done faster • Broader array ofinputs into process • Multi-directional dialogue at eady stages in process before ANOPR and between ANOPR and NOPR • More participation by all groups in all stages ofprocess -- especially at the beginning • Re-focus process on developing and communicating rational basis for consumers to choose (the process does not accomplish this at all) 

### IA

• Clearer definition ofground rules, such as what is economicallyjustifiable to consumer • Defined end point for product when reach diminishing returns • More realistic better weighting of input values that lead to results • Better input as labs develop their models (models not developed in a vacuum) • Find outcomes that maximize benefits to all • All be happy! • Faster 

### + DOE should have more discretion - Especially regarding the statute • End products should be a better communication vehicle for consumerto make rational decisions • More use ofnegotiation - Eliminatellimitthe points ofdisagreement • Usebetterjudgmentin development oftechnical assumptions • Ongoing, open dialogue, faster 

### lA

• Incorporate more opportunity for give-and take. Create opportunities to bargain tV'i)':

# /0/., sV'


### 3• Make sure the process is explicitly public '!, j S A ( h1M

• Technical support documentation should be subject to peer review early on, especially prior to being published in record zA • Phase down mandatory standards and move toward market-based standards PC • Cumulative regulatory burden taken into account FL • Impacts on small (uneducated?) manufacturers 16 

### PRELIMINARY LIST KEY ISSUES/PROBLEMS WITH REGULATORY FRAMEWORK • User friendliness is missing i'-j • Is simplification possible? • Some key factors are over-looked A

Example:' Heat pumps

(ambient air temperature was not considered in the analysis) • Cumulative burden not considered DA • Cost differential for-confidence level to market and manufacture a new product /A0 • Need to compare data against established criteria 1'\ Specific models' adequacy: - Should they be used at all? - Which individual contractors should be used - How to interpret the outputs? • DOE over-relies on these computer models rather than weigh expert judgments, and non-quantitative analysis • Make sure the process is explicitly public • Technical support documentation should be subject to peer review early on, especially prior to being published in record 'IA • Phase down mandatory standards and move toward market-based standards 

# n • Cumulative regulatory burden taken into account fL • Impacts on small (uneducated?) manufacturers**~** 

## PRELIMINARY LIST KEY ISSUES/PROBLEMS WITH REGULATORY FRAMEWORK • User friendliness is missing • Is simplification possible? • Some key factors are over-looked f'A Example: Heat pumps (ambient air temperature was not considered in the analysis) • Cumulative burden not considered 

## vf'1 • Cost differential for·confidence level to market and manufacture a new product 

## :r lAM • Need to compare data against established criteria ::' ()" • Specific models' adequacy: 

### ;i

## Pt

- Should they be used at all?

- Which individual contractors should be used

- How to interpret the outputs? • DOE over-relies on these computer models rather than weigh expertjudgments, and non-quantitative analysis j**~**



### 4

### PLANNING KEY ISSUES Overall picture needs to be looked at • Prioritization of products/actions • Test procedures must be revised • Consideration of non-regulatory alternatives • Public input and dialogue • Consideration of regulatory alternatives <'Realty = DOE cannot do all that is prescribed > <DOE must be practical about what DOE tries to accomplish>

- Schedule: Must be reasonable * Congress sets, they are frequently missed - Regulatory burden (other Federal requirements) 

### EXAMPLE(S) OF PLANNING BREAKDOWNS A. Regulatory agenda published every year

- Is total failure By the time it hits the street, it is out of date, not useful B. Input to climate action plan

S/H input was not used much 

### PLANNING KEY ISSUES Overall picture needs to be looked at • Prioritization of products/actions • Test procedures must be revised • Consideration ofnon-regulatory alternatives • Public input and dialogue • Consideration ofregulatory alternatives <Realty

# = DOE cannotdo all that is prescribed>· <DOE must be practical about what DOE tries to accomplish>

- Schedule: Must be reasonable 

## * Congress sets, they are frequently missed

- Regulatory burden (other Federal requirements) 

### EXAMPLE(S) OF PLANNING BREAKDOWNS A.Regulatory agenda published every year

- Is total failure By the time it hits the street, it is out ofdate, not useful B.Input to climate action plan



### - SIH input was not used much



### 5

### DATA KEY ISSUES INPUT AND OPINION ACCUMULATION • Evaluate use of contractors t A • Assure data is useful/quality 

## z& • Workshops to gather and describe type of data pathway `i f • Stakeholders should be involved in selecting design options • Intentional/unintentional gaming of the system °

Example: Usefulness

<Questions about which elements to use <How to evaluate quality of data by divergent groups ° Show range of uncertainty ° Make these judgments, and show how they were arrived at transparent • Address confidentiality issues followed by workshop on "Proposed" inputs to model j- • Mechanisms to decide which data is missing 7- A • Key drivers for which no good data

Example: Mark-up incrementalism (GIST to elasticity of demand) fA



### 6 

### DATA KEY ISSUES INPUT AND OPINION ACCUMULATION • Evaluate use ofcontractors • Assure data is useful/quality • Workshops to gather and describe type ofdata pathway /1 l • Stakeholders should be involved in selecting design options ! ',..

, • Intentional/unintentional gaming ofthe system**~** 

### r Example: . 

### Usefulness**~Questions** about which elements to use**~How** to evaluate quality ofdata by divergent groups o Show range ofuncertainty o Make thesejudgments. and show how they were arrived at transparent • Address confidentiality issues followed by workshop on "Proposed" inputs to model .1**~** • Mechanisms to decide which data is missing 

### ;p/>< • Key drivers for which no good data Example: 

### Mark-up incrementalism (GST to elasticity ofdemand)



### 6

### DECISION MAKING KEY ISSUES

• It takes too long now • If participants and department get agreement along the way , then this problem should be diminished • There is insufficient attention to advanced technologies " 1 • Don't get maximum buy-in = no sense that Stakeholder were a part of making decision • Decision-making process is not apparent -- understandable • The end points not described • There is no strategic plan government-wide -- no large overview
• Special consideration should be given to small/medium size manufacturers

- Technical issues don't spread evenly among all sizes of manufacturers • Special consumer classes need to be included ? consulted? • Increased flexibility in statutory requirements -- especially implementation period for STDs 

### FACA CONSTRAINTS GOAL: NOT GET ENTANGLED IN FACA Ground Rules =

1. Group not to make decisions 2. Group not to create consensus decisions/recommendations to DOE DOE's Expectations of Group: • Government will get a balanced view, such as "Who is missing?" • These are open meetings that provide an open process • The process conforms to FACA guidelines



### 7 

### DECISION MAKING KEY ISSUES
• It takes too long now • Ifparticipants and department get agreement along the way, then this problem should be diminished • There is insufficient attention to advanced technologies 

## :r IA • Don'tget maximum buy-i-n 

### = no sense that Stakeholder were a part ofmaking decision • Decision-making process is not apparent -- understandable • Theend points not described • There is no strategic plan government-wide -- no large overview • Special consideration should be given to small/medium size manufacturers

- Technical issues don't spread evenly among all sizes ofmanufactl,lrers • Special consumerclasses need to be included? consulted? • Increased flexibility in statutory requirements -- especially implementation period for STDs 

### GOAL: FACA CONSTRAINTS NOT GETENTANGLED INFACA Ground Rules 

### = 1 . Group not to make decisions 2. Group not to create consensus decisions/recommendations to DOE DOE's Expectations ofGroup: • Government will get a balanced view, such as "Who is miSSIng?" • These are open meetings that provide an open proc'ess • The process confonns to FACA guidelines



### 7LIST TO REVISIT



### ISSUE A CTION 3 FACA ISSUE Evaluate this as we go "No assumption is written in stone" 3 "What is said in this room" -- -i No attribution to assure free Ways that all can speak freely flowing exchange 3 • Consumer interest not represented * Key persons group identified Key issues group identified * Interviewed by ADL to see if they need to be included --► ADL interviews to assess if they need to be included
STORAGE BIN • "Nitty gritty" details of what doe does (after this improvement effort,-- 1 year +) • Public interaction process needs Public hearing * Etc.



### 8 LIST TO REVISIT ISSUE ..; • FACA ISSUE "No assumption is written in stone" ..; • "What is said in this room" -- Ways that all can speak freely ..; • Consumer interest not represented 

## * Key persons group identified 

## * Key issues group identified 

## * Interviewed by ADL to see ifthey need to be induded 

### ACTION

- Evaluate this as we go

- No attribution to assure free flowing exchange

- ADL interviews to assess ifthey need to be included


### STORAGE BIN • "Nitty gritty" details ofwhat doe does (after this improvement**effort,~-** [ year +) • Public interaction process needs 

## * Public hearing 

## * Etc. .



### 8

### APPENDICES TRUISMS AND KEY ASSUMPTIONS • REALISTICALLY DOE'will not, cannot do everything • Budgets constrain us (among other reasons) • There is no such thing as an 'average product • One size does not fit all • 50 sizes does not fit better 

### DEFINITIONS DIALOGUE • DOE does not dictate this plan. Have input to the plan not input on the plan • Interactive NOT Reactive NEGOTIATED = Does not mean consensus at every step -- attempts to narrow differences and limits the factors/range in dispute
• Quality of data • Source of data 

### WAYS TO CATEGORIZE PROBLEMS

t p, • Analysis of data • Logistics of public input • DOE decision-making `%'^ • Statutory constraints 

### APPENDICES TRUISMS AND KEY ASSUMPTIONS • REALISTICALLY DORwill not. cannot do everything • Budgets constrain us (among otherreasons) • There is no such thing as an 'average product • One size does not fit all 

### • 50 sizes does not fit better 

### DEFINITIONS DIALOGUE**~** • DOE does not dictate this plan. Have input to the plan not input on the plan • Interactive 

### NOT Reactive NEGOTIATED 

### = Does not mean consensus at every step -- attempts to narrow differences and limits the factors/range in dispute 

### WAYS TO CATEGORIZE PROBLEMS • Quality ofdata • Source ofdata • Analysis ofdata "\-: • Logistics ofpublic

## inp~ti j

## ~

# 1 \!I"A .,.}, • DOEdecision-making 'i' , • Statutory constraints :;

## ~l.r \ 1i' \

### \)'fV'**~**


### 9

### INITIAL BASKETS

CREATING ENOUGH STRUCTURE SO THAT COMMENTS ARE USEFUL

• DOE planning and prioritization (Macro)
• Quality of data
• Consumer input
• Source of data
• Complements/Supplements to STDs VI-
• Analysis of data z N
• Process input logistics of public input QL^St Ins
• DOE decision making o
• Application of DOE discretion 

### • Statutory issues iI tip` t off` • Need for public comment throughout the entire process rt'^"^' hM



### 10 

### INITIAL BASKETS CREATING ENOUGH STRUCTURE SO THAT COMMENTS ARE USEFUL • DOE planning and prioritization (Macro) • Quality ofdata • Consumer input • Source ofdata • Complements/Supplements to SIDs l(\.- • Analysis ofdata**~** (It • Process input logistics ofpublic input

## ?L.\J~\n"'" • DOE decision making 

### \)tA • Application ofDOEdiscretion \I'M • Statutory issues 

### 1\.-1

### ~,.\ ot!- • Need for public comment throughout the entire process 

### ?v\ \:

## ~\\'lP"



### 10

### ADOPTED (PROPOSED) AGENDA

END TIME 9:15 Opening Remarks:

Introductions: Christine Ervin

All 9:30 Agenda Review:

Housekeeping: Ground Rules: FACA: AD HOC Group Doug Brookman

All Doug Brookman Doug Smith 

### 10:30 Role, Purpose of Stakeholder Group

"One thing you would like to change."

BREAK 12:00 How to approach -- Path and Goals, Outcomes, Elements 

### LUNCH 2:45 Interview Guide 

### BREAK 4:00 Next Steps



### 11 

## · 

# .
ENPTIME 9:15 9:30
10:30
12:00 2:45 4:00 

## ADOPTED (PROPOSED) AGENDA Opening Remarks: Introductions: Agenda Review: Housekeeping: Ground Rules: FACA: AD HOC Group Christine Ervin 

### All Doug Brookman 

### All Doug Brookman Doug Smith Role, Purpose ofStakeholderGroup "One thing you would like to change." BREAK tIow to approach -- Path and Goals, Outcomes,Elements LUNCH Interview Guide BREAK Next Steps



### 11