PUBLIC VERSION (11.18.2013)

Second Annual 

Energy Efficiency Benchmarking Report 

(Based on EIA 2011 data) 

Energy Efficiency Working Group

Benchmarking Subcommittee: Data Analysis Team

Benchmarking Subcommittee Chair:		

Norman	Muraya	

Report Authors (Data Analysis Team within the Benchmarking Subcommittee):		

Annika	 Brink,	 Paul	 Davis,	 Tom	 Gross,	 Michael	 Little,	 Nicole	 Moreland,	 Norman	 Muraya,	

Larry	Philbin,	Jeff	Stafford,	Don	Wucker,	and	Steve	Zach		

October 11, 2013
LPPC EEWG Benchmarking Subcommittee   

Page i
LPPC EEWG Benchmarking Subcommittee   

Page ii

Table of Contents

Executive	Summary	................................................................................................................................................................	1

Introduction.....................................................................................................................................................................................1

Outline of the Report....................................................................................................................................................................1

 Analysis	of	LPPC	Data	.......................................................................................................................................	1

 General	and	Metric‐Specific	Data	Issues	...................................................................................................	2

 Improved	Data	Collection,	Quality,	and	Usefulness	.............................................................................	2

 Conclusions	and	Recommendations	for	Future	Research	.................................................................	2

 Charts	......................................................................................................................................................................	3

Introduction	...............................................................................................................................................................................	4

The Large Public Power Council and the Energy Efficiency Working Group....................................................4

Benchmarking Subcommittee: Purpose and Activities................................................................................................4

Analysis	of	LPPC	Data	............................................................................................................................................................	6

Description of LPPC Utility Population...............................................................................................................................6

Table	1‐A:	LPPC	“Retail	Equivalent”*	Utilities	Included	in	this	Report	(2011	Data)	........................	7

Table	 1‐B:	 Remaining	 LPPC	 Utilities	 –	 Primarily	Wholesale	 Utilities	 not	 Otherwise	 Found	 in	

this	Report,	and	Summary	Statistics	(2011	Data)	...........................................................................................	8

Energy Efficiency Program Performance Metrics and Measures...........................................................................9

Cost	Measures	.................................................................................................................................................................	9

Savings	Measures	and	Demand	Response	Indicators	..................................................................................	10

Cost‐to‐Savings	Rates	................................................................................................................................................	10

Trends	..............................................................................................................................................................................	11

Short‐Term	and	Long‐Term	Goals	.......................................................................................................................	11

Metrics	Not	Included	in	This	Report	...................................................................................................................	11

General	Data	Issues	..............................................................................................................................................................	12

Metric‐Specific	Data	Issues	................................................................................................................................................	13

Cost.................................................................................................................................................................................................... 13
LPPC EEWG Benchmarking Subcommittee   

Page iii

Savings............................................................................................................................................................................................13

Cost‐to‐Savings Rates...............................................................................................................................................................13

Improving	Data	Collection,	Quality,	and	Usefulness	...............................................................................................	14

Conclusions	and	Recommendations	for	Future	Research	....................................................................................	16

Charts..........................................................................................................................................................................................	17

Cost Measures..............................................................................................................................................................................17

Chart	1:	Total	EE	&	DR	Spending,	2011	(in	millions	of	$)	..........................................................................	17

Chart	2:		Total	EE	Spending	Relative	to	Retail	Sales,	2011	(in	$/MWh)	..............................................	17

Chart	3:		Total	DR	Spending	Relative	to	Peak	Demand,	2011	(in	$/MW)	...........................................	18

Chart	4:		Total	EE	&	DR	Spending	Relative	to	Retail	Revenue,	2011	(in	%)	.......................................	18

Chart	5:	Total	EE,	DR,	&	Renewable	FTE	Employees	by	Type	(External	vs.	Internal),	2011	.......	19

Chart	 6:	 Total	 EE,	 DR,	 &	 Renewable	 FTE	 Employees	 Relative	 to	 MWh	 Retail	 Sales,	 2011	 (in	

FTEs	per	million	MWh)	............................................................................................................................................	20

Savings Measures.......................................................................................................................................................................21

Chart	7:		Incremental	Energy	Savings	from	EE,	by	Sector,	2011	(in	MWh)	........................................	21

Chart	8:	Incremental	Energy	Savings	from	EE	Relative	to	Retail	Sales,	2011	(in	%)	.....................	21

Chart	9:	Incremental	Energy	Savings	from	Sector‐Specific	EE	Relative	to	Retail	Sales	from	that	

Sector,	2011	(in	%)	....................................................................................................................................................	22

Chart	10:	Peak	Demand	Reduction	from	EE	&	DR,	2011	(in	MW)	..........................................................	22

Chart	11:	Peak	Demand	Reduction	from	EE	&	DR,	by	Sector,	2011	(in	MW)	....................................	23

Chart	 12:	 	 Peak	Demand	Reduction	 from	 EE	&	DR	Relative	 to	Annual	System	 Peak	Demand,	

2011	(in	%)	....................................................................................................................................................................	23

Chart	13:		Peak	Demand	Reduction	from	EE	&	DR	Relative	to	Annual	System	Peak	Demand,	by	

Sector,	2011	(in	%)	....................................................................................................................................................	24

Cost‐to‐Savings Rate Measures............................................................................................................................................25

Chart	14:		EE	Spending	Relative	to	Incremental	Energy	Savings,	2011	(in	$/MWh)	.....................	25

Chart	15:		EE	Spending	Rel.	to	Incremental	Energy	Savings,	by	Sector,	2011	(in	$/MWh)	.........	25

Chart	16:	EE	&	DR	Spending	Relative	to	Peak	Demand	Saved	(in	$/kW)	...........................................	26
LPPC EEWG Benchmarking Subcommittee   

Page iv

Chart	17:	Incremental	Energy	Savings	from	EE	Relative	to	#	of	(Internal	&	External)	EE	FTE’s	

&	from	DR	Relative	to	#	of	(Internal	&	External)	DR	FTE’s,	2011	(in	MWh/FTE)	..........................	26

Trends: 2009‐2011 ....................................................................................................................................................................27

Chart	18:	2009‐2011	Total	Retail	Sales	(in	MWh)	........................................................................................	27

Chart	19:	2009‐2011	Retail	Revenue	(in	billions	of	$)	................................................................................	27

Chart	20:	2009‐2011	Total	EE	&	DR	Spending	(in	millions	of	$)	............................................................	28

Chart	21:	2009‐2011	Total	EE	Spending	(in	millions	of	$)	........................................................................	28

Chart	22:	2009‐2011	Total	DR	Spending	(in	millions	of	$)	.......................................................................	29

Chart	23:	2009‐2011	Total	EE	&	DR	Spending	Relative	to	Retail	Revenue	(in	%)	..........................	29

Chart	24:	2009‐2011	Total	Incremental	Energy	Savings	from	EE	(in	MWh)	....................................	30

Chart	25:	2009‐2011	Total	EE	&	DR	FTE’s	(Internal	and	External/Contracted)	.............................	30

Chart	26:	2009‐2011	Total	Incremental	Savings	from	EE	Rel.	to	MWh	Retail	Sales	(in	%)	........	31

Chart	27:	2009‐2011	Total	 Incremental	EE	Spending	Relative	 to	 Incremental	EE	MWh	Saved	

(in	millions	of	$/MWh)	.............................................................................................................................................	31

Chart	 28:	 2009‐2011	 Total	 Incremental	 EE	Spending	Relative	 to	 Incremental	 Peak	 kW	 Load	

Reduction	from	EE	(in	$/kW)	................................................................................................................................	32

Chart	29:	2009‐2011	Load	Factor	(MWh	per	Peak	kW	x	8,760)	.............................................................	32

Short‐Term and Long‐Term Goals.....................................................................................................................................33

Chart	30:	Short‐Term	Goals	for	Energy	Savings	(MWh	as	%	of	Total	MWh	Retail	Sales)	............	33

Chart	31:	Long‐Term	Goals	for	Energy	Savings	(MWh	as	%	of	Total	MWh	Retail	Sales)	.............	33

Chart	32:	Short‐Term	Demand	Goals	for	Energy	Efficiency	and	Demand	Response	(MW	as	%	of	

Peak	Demand)	..............................................................................................................................................................	34

Chart	33:	Long‐Term	Demand	Goals	for	Energy	Efficiency	(MW	as	%	of	Peak	Demand)	............	34

Chart	34:	Long‐Term	Demand	Goals	for	Demand	Response	(MW	as	%	of	Peak	Demand)	..........	35

Chart	35:	Long‐Term	Demand	Goals	 for	Demand‐Side	Renewables	(MW	 Installed	Capacity	as	

%	of	Peak	Demand)	....................................................................................................................................................	35

Chart	36:	Long‐Term	Goals	for	Demand‐Side	Renewables	(MWh	Annually	as	%	of	Total	MWh	

Retail	Sales)	...................................................................................................................................................................	36
LPPC EEWG Benchmarking Subcommittee   

Page v

Chart	37:	Long‐Term	Demand	Goals	for	Supply‐Side	Renewables	(MW	Installed	Capacity	as	%	

of	Peak	Demand)	.........................................................................................................................................................	36

Chart	 38:	 Long‐Term	Goals	 for	Supply‐Side	Renewables	 (MWh	Annually	as	%	 of	 Total	MWh	

Retail	Sales)	...................................................................................................................................................................	37

National Comparison ...............................................................................................................................................................38

Data	Table	for	Five	Key	EE	Performance	Measures	.....................................................................................	39

Chart	40:	Percentiles	for	Total	EE	Spending	Relative	to	Retail	Revenue	(in	%)	..............................	41

Chart	41:	Percentiles	for	Total	EE	Spending	Relative	to	Retail	Sales	(in	$/MWh)	..........................	42

Chart	42:	Percentiles	for	EE	Spending	(in	thousands	of	$)	........................................................................	43

Chart	43:	Percentiles	for	Incremental	Energy	Savings	from	EE	Relative	to	Retail	Sales	(in	%)	44

Data	Table	for	Four	Key	DR	Performance	Measures	....................................................................................	45

Chart	44:	Percentiles	 for	Actual	Annual	Demand	Reduction	 from	DR	as	a	Percent	of	Summer	

System	Peak	Demand	(%)	.......................................................................................................................................	46

Chart	 45:	 Percentiles	 for	 Actual	 Annual	 Demand	 Reduction	 from	 DR	 as	 a	 Percent	 of	Winter	

System	Peak	Demand	(%)	.......................................................................................................................................	46

Chart	46:	Percentiles	for	Direct	&	Incentive	Costs	Rel.	to	Actual	Peak	Reduction	(in	$/MW)	...	47

Appendix	A:	Benchmarking	Data	by	Utility	(Main	Source:	EIA	Form	861,	2011)	......................................	49

Appendix	B:	Overview	of	DEED	Grant	Activities	......................................................................................................	53

Appendix	C:	EEWG	Benchmarking	Subcommittee’s	Public	Comments	to	the	U.S.	Energy	Information	

Administration	regarding	EIA	Form	861	(Submitted	May	6,	2013)	................................................................	55
LPPC EEWG Benchmarking Subcommittee   

Page 1

Executive Summary

Introduction

This	report	is	a	project	of	the	Large	Public	Power	Council	(LPPC)	Energy	Efficiency	Working	Group	

(EEWG),	 a	 collaboration	 among	 energy	 efficiency	 professionals	 drawn	 from	 the	 nation’s	 largest	

public	power	utilities.	The	EEWG’s	Benchmarking	Subcommittee	identified	 the	limited	availability	

of	 dependable	 energy	 efficiency	 program	 data	 from	 peer	 utilities	 as	 a	 barrier	 to	 LPPC	 utilities’	

ability	to	judge	and	improve	energy	efficiency	program	performance.	

This	 second	 annual	 benchmarking	 report	 documents	 the	 continuation	 of	 the	 EEWG	 data	 analysis	

benchmarking	 initiative	 and	 provides	 a	 foundation	 for	 developing	 useful	 energy	 efficiency	

performance	 data	 and	 information.	 This	 report	 uses	 data	 from	 the	 Energy	 Information	

Administration’s	 (EIA)	 Form	 861	 to	 provide	 a	 snapshot	 of	 LPPC	 utilities’	 energy	 efficiency	

performance	from	2009	through	2011.	In	some	cases,	this	data	was	supplemented	with	additional	

information	 provided	 by	 LPPC	 utilities.	 The	 Benchmarking	 Subcommittee	 will	 produce	 annual	

reports	and	use	findings	from	each	year’s	report	to	support	best	practices	conversations	within	the	

EEWG,	 to	improve	subsequent	annual	reports,	 to	continually	influence	EIA	 to	improve	Form	861,	

and	 to	 standardize	 LPPC	 data	 reporting.	 Wherever	 possible,	 the	 Subcommittee	 will	 draw	 on	

existing	initiatives	in	this	arena.	

Outline of the Report

 Analysis of LPPC Data

The	report	begins	by	listing	the	data	sources	used	and	explaining	which	twenty‐one	utilities	are	

included	in	this	report.	It	further	outlines	concerns	about	the	small	sample	of	utilities	and	areas	

of	variation	among	utilities	that	affect	data	comparability.	For	example,	the	size	of	the	included	

LPPC	 utilities	 ranges	 from	 46,000	 to	 1.48	 million	 customers.	 The	 report	 goes	 on	 to	 explain	

which	 energy	 efficiency	 and	 demand	 response	 metrics	 and	 measures	 were	 included	 in	 the	

analysis.	These	metrics	and	measures	were	chosen	based	on	usefulness	and	perceived	lack	of	

bias,	but	also	based	on	the	data	available	through	EIA	Form	861.They	are,	briefly:	

Cost Measures ‐	describes	the	annual	expenditures	and/or	resources	consumed	by	EE	and	

DR	programs.	1	These	include	relative	cost	measures	that	adjust	for	utility	size.	

Savings Measures ‐	describes	MWh	and	MW	reductions	achieved;	also	adjusted	for	size.	

Cost‐to‐Savings Rates ‐	describes	the	reductions	achieved	relative	to	their	costs.	

Demand Response Indicators	‐	describes	demand	response	capabilities.	

																																																													

1 In this report, indirect costs were allocated to EE or DR according to the relative weight of direct spending in

these categories.
LPPC EEWG Benchmarking Subcommittee   

Page 2

 General and Metric‐Specific Data Issues

Next,	the	report	explores	general	as	well	as	metric‐specific	data	issues	that	affect	the	ability	to	

draw	 comparisons	 among	 utilities.	 Data	 reporting	 content	 and	 consistency	 have	 surfaced	 as	

issues	 primarily	 due	 to	 different	 ways	 each	 utility	 may	 classify	 its	 data	 and	 due	 to	 lack	 of	

definitional	 precision	in	 the	 Form	 861	itself.	 Areas	 of	 uncertainty	include	 utility	methods	 for	

measuring	 and	 reporting	 savings	 and	 for	 defining	 cost	 categories.	 Additional	 comparability	

issues	 occur	 because	 LPPC	 utilities	 vary	 in	 the	 percent	 of	 wholesale	 power	 sold	 (although	

primarily‐wholesale	LPPC	utilities	have	largely	been	excluded	from	this	report)	and	in	whether	

or	not	they	provide	energy	efficiency	services	to	both	wholesale	and	retail	customers.	

Measures	in	the	Cost	metric	are	affected	primarily	by	imprecise	definitions	of	the	data	elements	

comprising	 them	and	a	lack	 of	direction	 on	how	 to	apportion	costs.	Additionally,	it	is	 unclear	

how	 utilities	 count	 employees	 working	 part‐time	 on	 DSM	 activities	 and	 whether	 utilities	 are	

including	 fully‐burdened	rates	when	reporting	labor	costs.	Measures	in	the	Savings	metric	are	

affected	by	the	type	of	savings	reported	(gross,	net,	etc.).	Measures	in	the	Cost‐to‐Savings	Rate	

metric	 are	 affected	 by	 the	 fact	 that	 Form	 861	 does	 not	 collect	 information	 on	 the	 lifetime	 of	

energy	 efficiency	 measures.	 Without	 this	 information,	 cost‐effectiveness	 cannot	 truly	 be	

measured.		

 Improved Data Collection, Quality, and Usefulness

Since	issuing	the	inaugural	benchmarking	report	in	June	2012,	the	LPPC	EEWG	Benchmarking	

Subcommittee	 has	 made	 a	 significant	 effort	 to	 improve	 the	 quality	 of	 energy	 efficiency	 data	

generated	via	the	U.S.	Energy	Information	Administration’s	(EIA)	data	collection	efforts.		

Further	 discussion	 on	improving	 utility	 data	 reporting	 on	EIA	 Form	 861	appears	later	in	 this	

report,	 including	 Appendix	 B	 –	 Overview	 of	 DEED	 Grant	 Activities.	 Major	 recommendations	

include:	 provide	 clear	 definitions	 of	 specific	 terms	 and	 data	 inputs,	 provide	 examples	 of	

calculation	 methodologies,	 provide	 more	 opportunities	 for	 transparency	 of	 calculation	

methodologies,	 ask	 for	 information	 about	 the	 longevity	 of	 savings,	 standardize	 reporting	 of	

labor	 costs,	 and	 break	 out	 additional	 cost	 categories.	 Ceres	 and	 the	 Consortium	 for	 Energy	

Efficiency	 have	 each	 also	 produced	 useful	 recommendations	 for	 how	 EIA	 can	 improve	 data	

collection	and	reporting.	

 Conclusions and Recommendations for Future Research

The	 written	 analysis	 concludes	 with	 a	 summary	 of	 challenges	 encountered	 in	 producing	 the	

report	 and	 a	 list	 of	 recommendations	 for	 future	 research	 and	 reports.	 	 In	 the	 process	 of	

soliciting	and	reviewing	participant	data,	 the	Benchmarking	Subcommittee	discovered	several	

issues	related	 to	data	accuracy	and	precision;	however,	it	also	generated	many	questions	and	

ideas	 that	 could	 pave	 the	 road	 to	 ongoing	 and	 dynamic	 performance	management	 tools.	 The	

challenge	going	forward	will	be	to	categorize	and	prioritize	which	areas	and	activities	have	the	

best	cost‐benefit	potential	given	limited	resources	to	pursue	them.		

Areas	of	note	include	(but	are	certainly	not	limited	to):	developing	or	adopting	data	definitions	

and	 reporting	 protocols,	 creating	 performance	 analyses	 databases	 to	 track	 trends	 over	 time,	

bringing	in	data	from	utility‐generated	benchmarking	reports,	creating	a	cross‐tabbed	database	
LPPC EEWG Benchmarking Subcommittee   

Page 3

of	 energy	 efficiency	 program	 characteristics	 in	 order	 to	 statistically	 correlate	 these	

characteristics	to	performance	results,	and	capturing	information	on	measure	longevity.	In	the	

future,	 the	 Benchmarking	 Subcommittee	 may	 want	 to	 create	 its	 own	 data	 definitions	 and	

reporting	protocols	or	it	may	adopt	guidelines	and	protocols	created	by	others.	

 Charts

Finally,	 the	 report	 presents	 a	 series	 of	 charts	 showing	 the	 results	 of	 the	 Benchmarking	

Subcommittee’s	data	analysis:	these	charts	are	presented	largely	without	commentary.	The	first	

set	 of	 charts	 uses	 2011	 data	 to	 explore	 the	 primary	measures	 chosen	 for	 analysis.	 These	 are	

followed	by	charts	showing	a	2009	to	2011	comparison	for	several	of	these	measures.	The	next	

set	of	charts	shows	short‐term	and	long‐term	LPPC	utility	energy	savings	goals.	The	last	section	

of	charts	brings	in	additional	EIA	data	to	present	a	national	comparison	among	LPPC	and	other,	

similarly‐sized	utilities.	
LPPC EEWG Benchmarking Subcommittee   

Page 4

Introduction

The Large Public Power Council and the Energy Efficiency Working Group

The	Energy	Efficiency	Working	Group	(EEWG)	is	made	up	of	energy	efficiency	professionals	 from	

members	of	the	Large	Public	Power	Council	(LPPC),	an	organization	comprised	of	twenty‐six	of	the	

nation's	 largest	 locally	 owned	 and	 controlled,	 not‐for‐profit	 power	 systems.	 The	 LPPC	 EEWG	 is	

working	 to	 move	 the	 public	 power	 sector	 toward	 providing	 more	 robust	 energy‐efficiency	

programs	and	services.	Members	of	the	EEWG	have	focused	on	breaking	down	many	of	the	barriers	

to	collaboration	among	utilities	in	order	to	foster	ongoing	communication	and	to	create	a	forum	in	

which	members	can	share	information	and	resources.	In	2012,	the	EEWG	has	focused	its	efforts	in	

three	areas:	benchmarking,	retailer	partnerships,	and	emerging	issues	and	programs.	

Since	2011,	the	EEWG	has	been	assisted	in	its	efforts	by	the	Clean	and	Efficient	Energy	Program	for	

Public	Power	(CEEP),	a	project	staffed	by	the	Alliance	to	Save	Energy	in	cooperation	with	the	Large	

Public	Power	Council	and	 the	American	Public	Power	Association.	CEEP	 facilitates	peer	exchange	

and	collaboration	among	LPPC	energy	efficiency	program	managers.	In	addition,	CEEP	maintains	a	

website	 of	 energy	 efficiency	 resources,	 CleanEfficientEnergy.org,	 and	 has	 worked	 with	 the	

American	Public	Power	Association	(APPA)	to	develop	resources	and	trainings.	

Benchmarking Subcommittee: Purpose and Activities

The	Benchmarking	Subcommittee	identified	the	limited	availability	of	dependable	energy	efficiency	

program	 data	 from	 peer	 utilities	 as	 a	 barrier	 to	 LPPC	 utilities	 judging	 and	 improving	 energy	

efficiency	program	performance.	The	subcommittee	decided	to	analyze	LPPC	energy	efficiency	data	

to	 identify	 high‐performing	 utilities,	 areas	 for	 specific	 utilities	 to	 target	 for	 improvement,	

shortcomings	of	the	existing	data,	and	future	research	needs.		

First,	the	subcommittee	aggregated	LPPC	energy	efficiency	data	as	reported	to	the	U.S.	Department	

of	Energy’s	Energy	Information	Administration	(EIA)	through	Form	861.	It	then	chose	several	data	

points	 from	Schedule	6,	“Demand‐Side	Management	 Information,”	intended	 to	provide	a	snapshot	

of	 LPPC	 utilities’	 performance	 for	 2009	 through	 2011.	 The	 subcommittee	 also	 issued	 a	

supplemental	data	survey	to	collect	information	from	LPPC	utilities	on	a	few	additional	data	points.	

The	subcommittee’s	goal	is	to	produce	annual	benchmarking	reports	that	provide	insights	to	LPPC	

utilities.	LPPC	utilities	will	use	 these	insights	 to	initiate	conversations	with	management,	 to	share	

best	 practices	with	 other	 LPPC	 utilities,	and	 to	modify	 their	energy	efficiency	 programs	 to	 better	

match	LPPC	best	practices	and	benchmarks.	The	subcommittee	will	also	use	the	findings	from	each	

year’s	 report,	 including	 emerging	 data	 challenges,	 to	 improve	 subsequent	 reports.	 The	

subcommittee	has	worked	 (and	will	continue	 to	work)	with	EIA	 to	improve	its	data	collection	by	

improving	 the	 instructions	 and	 format	 for	 Form	 861.2	 It	 also	 works	 across	 LPPC	 utilities	 to	

standardize	 the	way	 that	LPPC	utilities	(and	other	public	power	utilities)	report	energy	efficiency	

data	on	Form	861:	this	occurs	naturally	as	a	result	of	Subcommittee	member	conversations	and	has	

																																																													

2 The EIA’s Form 861 is subject to periodic review and public comments, per the Paperwork Reduction Act of 1995.

Other groups, including the Consortium for Energy Efficiency, have also commented on Form 861.
LPPC EEWG Benchmarking Subcommittee   

Page 5

been	supplemented	in	2012	by	one‐on‐one	technical	assistance	to	public	power	utilities	that	CEEP	

data	analysis	identified	as	having	potentially	erroneous	EIA‐861	data.	

As	a	follow‐up	to	its	2012	Inaugural	Benchmarking	Report,	the	Subcommittee	has	dug	deeper	into	

the	annual	EIA‐861	data	to	produce	a	2013	report	with	a	finer	level	of	detail.	This	includes	breaking	

down	 data	 to	 the	 sector	 level	 (residential	 vs.	 commercial/residential),	 collecting	 more	 detailed	

information	on	types	of	energy	efficiency	and	renewable	energy	employees,	and,	for	the	first	time,	

requesting	that	utilities	provide	the	Subcommittee	with	information	on	spending	and	savings	at the

program or measure level.	 To	 some	 extent,	 this	 year’s	 data	 collection	 efforts	 have	 been

experimental,	as	the	Subcommittee	makes	an	effort	to	understand	which	types	of	data	are	easy	to	

produce,	 seeking	 a	 balance	 between	 collecting	 useful	information	 and	minimizing	 data	 reporting	

requests	to	LPPC	peer	utilities.	For	example,	because	the	Subcommittee	is	just	beginning	to	explore	

peers’	 reporting	 systems,	 this	 year’s	 request	 for	 program/measure‐level	 information	 did	 not	

request	specific	data	points	and	asked	utilities	to	provide	this	information	in	whichever	format	was	

most	convenient,	including	by	simply	providing	a	link	to	existing	reports.		

In	 future	years,	the	Subcommittee	would	like	to	start	conversations	with	high‐performing	utilities	

identified	via	its	Benchmarking	Report	analysis.	These	conversations	will	help	LPPC	utilities	access	

peer	 knowledge,	 identify	 best	 practices,	 and	 improve	 their	 own	 programs	 going	 forward.	 The	

Subcommittee	will	also	draw	on	existing	initiatives	in	this	arena,	such	as	those	being	conducted	by	

DOE	through	the	State	and	Local	Energy	Efficiency	Action	Network	(SEE	Action),	by	the	Northwest	

Regional	 Technical	 Forum,	 and	 by	 Northeast	 Energy	 Efficiency	 Partnerships.	 As	 this	 initiative	

continues,	the	Subcommittee	will	also	place	more	emphasis	on	tracking	trends	over	time.	

Start: Unify LPPC EIA‐861 Data in a 2013 report (2011 data)

Oct 2012 Nov‐Dec 2012 Jan‐Sep 2013 Oct‐Nov 2013 Nov‐Dec 2013

Identify	

barriers	

Collect	

data	

Analyze	

data	

Produce	report Identify	findings:

 High	performers	

 Areas	to	improve	

 Data	challenges	

 Future	research	needs	

Apply	findings:

 Bring	results	to	

management	

 Share	best	practices	

 Modify	programs	

Continue: Re‐initiate process to produce improved		

2014 report with finer level of detail
LPPC EEWG Benchmarking Subcommittee   

Page 6

Analysis of LPPC Data

Description of LPPC Utility Population

The	 subcommittee	 began	 its	 data	 analysis	 with	 EIA‐861	 data	 from	 the	 population	 of	 twenty‐six	

LPPC	 utilities.	 The	 subcommittee	 determined	 that	 seven	 utilities	 (ElectriCities	 of	 North	 Carolina,	

Inc.;	Grand	River	Dam	Authority;	Lower	Colorado	River	Authority;	MEAG	Power;	Nebraska	Public	

Power	 District;	 New	 York	 Power	 Authority;	 and	 Platte	 River	 Power	 Authority)	 were	 wholesale	

power	 providers,	 and	 should	 be	 excluded	 from	 this	 analysis.	 However,	 Nebraska	 Public	 Power	

District	 and	 Platte	River	 Power	Authority	 subsequently	 adjusted	 their	 data	 so	 that	 they	 could	 be	

meaningfully	compared	with	the	data	of	the	largely	retail	utilities.3

Of	 the	 remaining	 twenty‐one	 utilities,	 several	 have	 substantial	 wholesale	 power	 sales,	 but	 were	

retained	because	 they	seem	 to	have	more	in	common	(e.g.	number	of	customers)	with	retail	 than	

with	wholesale	utilities.	These	utilities	were	invited	 to	participate	in	a	supplemental	data	survey,	

which	 several	 did.	 Robust	 benchmarking	 requires	 enough	 similarity	 to	 be	 able	 to	 compare	

performance	 attributes	 on	 an	 “apples	 to	 apples”	 basis:	 purely	 wholesale	 utilities	 could	 create	 a	

subset	 for	 future	analysis.	Of	 these	 twenty‐one	members,	only	nineteen4	provided	energy	savings	

data.	This	number	is	too	small	to	justify	high	confidence	in	statistical	comparisons	and	analyses,	but	

still	useful	for	providing	a	snapshot	of	the	utilities	in	question.		

There	 is	 a	 wide	 range	 of	 variation	 in	 participant	 utility	 makeup	 including	 size,	 MWh	 sales,	

geographic	 location,	 customer	 class	 distribution,	 savings	 goals	 and	 expenditures,	 peaking	

characteristics,	maturity	 of	 EE	programs,	 and	 EE	 program	 portfolio	 content	 and	 design,	 etc.	 This	

exacerbates	 the	 problem	 of	meaningful	and	 representative	 comparisons.	 For	example,	 the	 size	 of	

the	included	LPPC	utilities	ranges	from	46,000	to	1.47	million	customers.	Where	the	EIA‐861	data	

allowed,	this	report	includes	measures	that	attempt	to	compensate	for	these	varied	attributes—for	

example,	by	relating	spending	to	a	utility’s	size.	However,	the	data	are	not	sufficient	to	compensate	

for	every	variation.	

The	 table	 that	 follows	 outlines	 specific	 utility	 characteristics	 that	 make	 it	 difficult	 to	 draw	

meaningful	comparisons	among	LPPC	utilities.	

																																																													

3 For the purposes of this report, NPPD data include NPPD’s seventy‐seven wholesale customers’ retail customers,

as NPPD’s energy efficiency program serves all end‐use customers. For the purposes of this report, Platte

River’s customer base characterization (e.g., peak load, annual load, number of customers) is based on the

customer base of its four member municipal utilities. For DSM programs, only those programs managed and

funded primarily by Platte River are included. Its member utilities also offer efficiency and load management

programs, but they are not included in this report. 4 Puerto Rico Electric Power Authority did not report information on MWh savings from energy efficiency

programs, although it did report MWh savings from its demand response programs.
LPPC EEWG Benchmarking Subcommittee   

Page 7

Table 1‐A: LPPC “Retail Equivalent”* Utilities Included in this Report (2011 Data)

CLIMATE SIZE

(by MW)

SIZE

(by # customers)

SIZE

(by MWh Sold) 	 	

% Customer Mix

by MWh (R/C/I) HDD / CDD Peaking

System

Peak

(MW)

Rank Number of

Customers Rank

Annual MWh Sold

(Retail +

Wholesale)

Rank

%

Whole

‐sale

**Avg

price,

$/kWh

Austin	

Energy	 36	 /	 45 /	 19	 2,039	 / 3,791 Summer 2,714 10	 417,191	 9	 13,101,834	 12	 0%	 0.09	

Clark	PUD	 53 /	 29	 /	 17	 4,884 /	 477 Winter	 956 17	 185,035	 16	 4,871,980	 19	 8%	 0.08	

Colorado	

Springs	 33	 /	 24	 /	 44 6,226 /	 1,005	 Summer 878 19	 211,188	 15	 4,748,463	 20	 4%	 0.09	

Imperial	

Irrig	Dist	 47	 /	 53 /	 0	 1,574	 /	 4,409 Summer 1,000 16	 148,196	 19	 3,313,144	 22	 1%	 0.12	

JEA	 43 /	 33	 /	 24	 1,599	 /	 2,791 Winter	 3,062 8	 419,992	 8	 14,784,868	 11	 18%	 0.11	

Long	Island	

Pwr	Auth	 50 /	 50 /	 0	 4,618 /	 1,214 Summer 5,771 4	 1,118,202	 4	 20,248,464	 6	 0%	 0.18	

LA	Dept.	

Water&Pwr	 32	 /	 59 /	 9	 1,753 /	 415	 Summer 5,907 3	 1,463,076	 3	 24,178,842	 5	 4%	 0.13	

NE	Pub.	

Pwr	Dist	 32	 /	 41 /	 27	 6,942 /	 1,194	 Summer 2,646 11	 371,907	 11	 19,869,177	 7	 N/A	 0.08	

Omaha	

Pub.	Pwr	

Dist	

34 /	 32	 /	 34 6,207 /	 1,452	 Summer 2,468 12	 352,182	 12	 15,328,409	 10	 30%	 0.07	

Orlando	

Util	Comm	 39	 /	 55 /	 7	 624	 /	 3,461 Winter	 1,276 15	 223,618	 14	 7,350,359	 18	 18%	 0.11	

Platte	River	

Power	Auth	 36 /	 33	 /	 31	 6,846 /	 1,022 Summer 640 21	 146,198	 20	 4,118,996	 21	 N/A	 0.07	

PUD	No.	1	

of	Chelan	 49 /	 30	 /	 20	 7,039 /	 777	 Winter	 634 22	 47,636	 21	 11,825,032	 15	 87%	 0.03	

PUD	No.	2	

of	Grant	 19	 /	 18	 /	 63 6,488 /	 890	 Summer 664 20	 46,351	 22	 11,844,013	 14	 66%	 0.03	

Puerto	Rico	 36	 /	 48 /	 16	 0	 /	 5,308 Summer 3,303 7	 1,475,196	 2	 18,251,359	 8	 0%	 0.27	

SMUD	 44	 /	 6	 /	 49 3,443 /	 1,327 Summer 2,840 9	 598,681	 7	 12,870,474	 13	 19%	 0.12	

Salt	River	

Project	 47 /	 41	 /	 12	 1,301	 /	 5,068 Summer 6,493 2	 965,644	 5	 32,885,786	 2	 19%	 0.09	

San	

Antonio	 45	 /	 50 /	 4	 1,623	 /	 3,882 Summer 4,911 6	 723,541	 6	 28,448,864	 3	 24%	 0.08	

Santee	

Cooper	 16	 /	 18	 /	 66 2,795 /	 2,200 Winter	 5,676 5	 164,677	 18	 27,551,663	 4	 59%	 0.07	

Seattle	City	

of	 33	 /	 55 /	 12	 5,488 /	 251	 Winter	 1,731 13	 400,351	 10	 17,058,087	 9	 44%	 0.07	

Snohomish	

PUD	No.	1	 52 /	 35	 /	 13	 5,750 /	 130	 Winter	 1,375 14	 323,460	 13	 9,034,481	 16	 24%	 0.08	

Tacoma	

City	of	 41	 /	 7	 /	 52 5,637 /	 187	 Winter	 931	 18	 168,286	 17	 7,515,807	 17	 36%	 0.06	

TVA	 36 / 24 / 40 3,674 /	 2,003	 Summer 31,157	 1	 4,615,606	 1	 165,449,755	 1	 N/A	 0.07	

TOTAL 87,032	 14,586,214 474,649,857

Legend																HDD	–	Heating	Degree	Day															CDD	– Cooling	Degree	Day 				 			 					R	– Residential 							 	 	 	 C	– Commercial															I	– Industrial

																														**	Calculated	by	dividing	Retail	Revenue	by	Electricity	Retail	Sales	

*	For	the	purposes	of	this	report	the	group	of	“Retail	Equivalent”	utilities	also	includes	primarily	wholesale	utilities	that	provided	data	adjustments	to	

the	Benchmarking	Subcommittee.	For	these	utilities,	we	list	as	“retail	customers”,	“retail	MWh	sold”,	and	“retail	revenues”	data	from	all	customer	

groups	to	which	the	utility	provides	energy	efficiency	services.	Thus,	the	data	shown	here—and	elsewhere	in	this	report—differs	from	EIA‐861	data	

for	the	following	utilities:	Nebraska	Public	Power	District,	Platte	River	Power	Authority,	and	Tennessee	Valley	Authority.	
LPPC EEWG Benchmarking Subcommittee   

Page 8

Table 1‐B: Remaining LPPC Utilities – Primarily Wholesale Utilities not Otherwise Found

in this Report, and Summary Statistics (2011 Data)

CLIMATE SIZE

(by MW)

SIZE

(by # customers)

SIZE

(by MWh Sold) 	 	

% Customer Mix

by MWh (R/C/I)

HDD /

CDD Peaking System Peak(MW)

Number of

Customers

Annual MWh

Sold (Retail +

Wholesale)

%

Whole

‐sale

**Avg

price,

$/kWh

Grand	Riv.	Dam	Auth.	 	 Summer    996 81	 7,893,613	 	 	

Lower	CO	Riv.	Auth.	 Winter	    3,945 Not	Avail.	 17,412,038	 	 	

Mun.	Elec.	Auth.	GA	 Summer    2,093 Not	Avail.	 12,840,991	 	 	

ElectriCities***	 	 Summer    2,745 Not	Avail.	 14,316,360	 	 	

NY	Power	Auth.	 	 Summer    3,730 163	 39,919,170	 	 	

Subtotal LPPC Primarily

Wholesale 244 92,382,172 	 	

Total, Tables 1‐A & 1‐B	 	 14,586,458 567,032,029 	 	

U.S. Total  151,707,966 3,716,376,665 	 	

Share from LPPC + TVA		 			9.65%** *** 9.61% 15.3%

**	Calculated	by	dividing	Retail	Revenue	by	Electricity	Retail	Sales	

***	The	ElectriCities	numbers	were	arrived	at	by	adding	the	entries	for	N	Carolina	Mun	Pwr	Agency	#1	and	North	Carolina	Eastern	MPA.	

 	In	order	to	avoid	double‐counting,	Tables	1‐A	and	1‐B	list	the	combined	Retail	and	Wholesale	MWh	Sold	for	the	LPPC	utilities	and	for	TVA;	

however,	Table	1‐B	lists	only	Retail	MWh	Sold	for	the	U.S.	Total.	

** ***	Percentage of	combined	Summer	Peak	Demands	reported	to	EIA
LPPC EEWG Benchmarking Subcommittee   

Page 9

Energy Efficiency Program Performance Metrics and Measures

The	 EEWG	Benchmarking	Subcommittee	 selected	 three	metric	 categories	 for	immediate	analysis:	

cost,	savings,	and	cost‐to‐savings	ratios.	Because	there	is	no	perfect	measure	to	assess	performance	

within	 any	 of	 these	 metric	 categories,	 this	 report	 includes	 several	 measures	 to	 give	 a	 global	

perspective	 of	 utility	 performance.	 Below	is	 the	list	 of	 these	Energy	 Efficiency	 (EE)	 and	 Demand	

Response	 (DR)	measures.		The	biases	of	 these	measures	as	well	as	inherent	data	weaknesses	are	

assessed	as	they	arise	throughout	the	report.	

With	the	exception	of	data	relating	to	EE	and	DR	Full	Time	Employees	(FTEs)	and	short‐term	and	

long‐term	 EE	 and	 renewable	 energy	 goals,	 EIA‐861	 reporting	 is	 the	 source	 of	 the	 data	 used	 to	

generate	the	measures	listed	below.	We	also	gave	utilities	the	opportunity	to	correct	or	update	the	

data	 from	 their	 EIA	 submissions.	 In	 cases	 where	 the	 values	 reported	 to	 the	 Benchmarking	

Subcommittee	 differ	 from	 those	 reported	 to	 EIA,	 this	 report	 uses	 the	 values	 provided	 directly	 by	

the	utility.	These	data,	as	well	as	additional	utility‐reported	information,	are	detailed	in	Appendix	A	

and	are	available	in	spreadsheet	format	upon	request.		

Metric	categories	and	associated	measures	are:	

Cost Measures ‐	describes	the	annual	expenditures	and/or	resources	consumed	by	EE	and	

DR	programs.	5

1. Energy	 efficiency	 and	 demand	 response	 spending	 (Total	 EE	 and	 DR	 Actual	

Expenditure).	Unit	in	$.6

2. Energy	 efficiency	 spending	 relative	 to	 retail	 sales	 (Total	 EE	 Actual	 Expenditure	in	

$/Electricity	Retail	Sales	in	MWh).	Unit	in	$/MWh.	

3. Demand	response	spending	relative	to	peak	demand	(Total	DR	Actual	Expenditures	

in	$/Annual	System	Peak	Demand	in	MW).	Unit	in	$/MW.	

4. Energy	efficiency	and	demand	response	spending	relative	to	electric	retail	revenues	

(Total	 EE	 Actual	 Expenditure	 in	 $/Retail	 Revenue	 in	 $	 and	 Total	 DR	 Actual	

Expenditure	in	$/Retail	Revenue	in	$).	Unit	in	%.	

5. FTE’s	 allocated	 to	 energy	 efficiency,	 demand	 response,	 and	 renewables,	 by	 type	

(both	internal	and	external/outsourced).	Unit	in	#.	

6. FTE’s	 allocated	 to	 energy	 efficiency,	 demand	 response,	 and	 renewables	 relative	 to	

retail	sales	(FTE’s/Electricity	Retail	Sales	in	MWh).	Unit	in	#/MWh.	

																																																													

5 In this report, indirect costs were allocated to EE or DR according to the relative weight of direct spending in

these categories. 6 The descriptions in parentheses generally refer back to EIA data category names. One change has been made

consistently throughout this report: whereas EIA uses “Load Management” and “LM,” this report uses

“Demand Response” and “DR.” For the purposes of this report, these terms are interchangeable. Additionally,

the original EIA cost data have been altered to include indirect costs.
LPPC EEWG Benchmarking Subcommittee   

Page 10

Savings Measures and Demand Response Indicators ‐	 describes	 MWh	 and	 MW	

reductions	achieved.		

7. Incremental7	 energy	 savings	 from	 energy	 efficiency,	 by	 sector	 (Sector‐Specific	

Incremental	Energy	Effects	from	EE	in	MWh).	Unit	in	MWh.

8. Incremental8	energy	savings	from	energy	efficiency	as	a	percent	of	retail	sales	(Total	

Incremental	Energy	Effects	from	EE	in	MWh/	Electricity	Retail	Sales	in	MWh).	Unit	

in	%.

9. Incremental9	 energy	 savings	 from	 sector‐specific	 energy	 efficiency	 as	 a	 percent	 of	

retail	sales	 from	that	sector	(Sector‐Specific	Incremental	Energy	Effects	 from	EE	in	

MWh/Sector‐Specific	Electricity	Retail	Sales	in	MWh).	Unit	in	%.

10. Peak	demand	reduction	from	energy	efficiency	and	demand	response	(Total	EE	and	

DR	Incremental	Peak	Demand	Saved).	Unit	in	MW.

11. Peak	 demand	 reduction	 from	 energy	 efficiency	 and	 demand	 response,	 by	 sector	

(Sector‐Specific	Incremental	EE	and	DR	Peak	Demand	Saved).	Unit	in	MW.

12. Peak	demand	reduction	as	a	percent	of	annual	system	peak	demand	(Total	EE	and	

DR	Peak	Demand	Saved	in	MW/	Annual	System	Peak	Demand	in	MW).	Unit	in	%.

13. Peak	demand	reduction	from	sector‐specific	activities	as	a	percent	of	annual	system	

peak	 demand	 (Sector‐specific	 EE	 and	 DR	 Peak	 Demand	 Saved	 in	 MW/Annual	

System	Peak	Demand	in	MW).	Unit	in	%.

Cost‐to‐Savings Rates ‐	describes	the	reductions	achieved	relative	to	their	costs.	

14. Current	year	energy	efficiency	spending	per	incremental10	savings	(Total	EE	Actual	

Expenditure	 in	 $/Total	 Incremental	 Energy	 Effects	 from	 EE	 in	 MWh).	 Unit	 in	

$/MWh.11

15. Current	 year	 energy	 efficiency	 spending,	 by	 sector,	 per	 incremental12	 savings,	 by	

sector	 (Sector‐Specific	 EE	 Actual	 Expenditure	 in	 $/Sector‐Specific	 Incremental	

Energy	Effects	from	EE	in	MWh).	Unit	in	$/MWh.13

16. Energy	efficiency	and	demand	response	spending	per	peak	demand	reduction	from	

energy	 efficiency	 and	 per	 peak	 demand	 reduction	 from	 demand	 response,	

																																																													

7 The instructions to EIA Form 861 define incremental effects as: “those changes in energy use (measured in

megawatt hours) and peak load (measured in megawatts) caused in the current reporting year by new

participants in DSM programs that already existed in the previous reporting year, and all participants in your

new DSM programs that existed for the first time in the current reporting year.”

8 See earlier footnote for definition of “incremental effects.”

9 See earlier footnote for definition of “incremental effects.”

10 See earlier footnote for definition of “incremental effects.”

11 Please note that this measure does not accurately reflect cost‐effectiveness, because EIA does not track data

such as measure life, lifetime savings, or expected useful life.

12 See earlier footnote for definition of “incremental effects.” 13 Please note that this measure does not accurately reflect cost‐effectiveness, because EIA does not track data

such as measure life, lifetime savings, or expected useful life.
LPPC EEWG Benchmarking Subcommittee   

Page 11

respectively	(Total	EE	Actual	Expenditures	in	$/EE	Peak	Demand	Saved	in	MW	and	

Total	DR	Actual	Expenditures	in	$/DR	Peak	Demand	Saved	in	MW).	Unit	in	$/kW.	

17. Incremental	 energy	 savings	 from	 energy	 efficiency	 per	 FTE	 allocated	 to	 energy	

efficiency	and	incremental	energy	savings	from	demand	response	per	FTE	allocated	

to	 demand	 response14	 (Total	 Incremental	Energy	Effects	 from	 EE	in	MWh/EE	 FTE	

and	Total	Incremental	Energy	Effects	from	DR	in	MWh/DR	FTE).	Unit	in	FTE/MWh.	

Trends

Charts	18‐29	depict	changes	over	time	for	the	above,	and	related,	measures:	2009‐2011.	

Short‐Term and Long‐Term Goals

Information	on	goals	was	collected	independently	by	the	Benchmarking	Subcommittee	and	

addresses	short‐term	and	long‐term	goals	for	energy	efficiency,	demand	response,	demand‐

side	 renewables,	 and	 supply‐side	 renewables.	 Some	 utilities	 did	 not	 have	 goals	 in	 all	 of	

these	areas,	and	some	utilities	did	not	have	both	MWh	and	MW	goals.	Charts	30‐38	cover	

this	information.	

Metrics Not Included in This Report

The	 Benchmarking	 Subcommittee	 also	 recognizes	 the	 following	metrics	 as	important,	 but	

does	not	have	the	data	to	include	them	in	the	current	analysis.	These	metrics	are	potential	

targets	for	future	research.	

 Quality‐shows	how	good	the	EE	process15	is	and	how	well	it	meets	customer	needs.	

 Speed‐shows	how	long	it	takes	the	EE	process	to	produce	output	results.

 Financial Value‐shows	the	return	in	dollars	to	stakeholders	of	the	EE	process.

 Milestone‐shows	the	accomplishment	of	an	activity	or	goal	of	the	EE	process.	

 Innovation‐shows	 the	 impact	 of	 new	 ideas	 or	 ways	 of	 doing	 things	 for	 the	 EE	

process.	

																																																													

14 See earlier footnote for definition of “incremental effects.” 15 Process refers globally to the development and deployment of energy efficiency portfolios, programs, and

measures.
LPPC EEWG Benchmarking Subcommittee   

Page 12

General Data Issues

The	Benchmarking	Subcommittee	chose	the	metrics	and	measures	listed	above	based	on	usefulness	

and	perceived	lack	of	bias,	but	also	based	on	 the	data	available	 through	EIA	Form	861.	As	shown	

above,	the	initial	focus	was	on	cost	and	efficiency	of	the	EE	process	and	this	is	to	be	expected	in	the	

early	 stages	 of	 the	 subcommittee’s	 work	 as	 these	 are	 the	 areas	 that	 management	 typically	 is	

interested	 in	 on	 a	 short‐term	 basis.	 There	 is	 ample	 room	 to	 expand	 this	 effort	 into	 balanced	

performance	management	assessment	tools.	

Data	 reporting	 content	 and	 consistency	 have	 surfaced	 as	 issues	 primarily	 due	 to	 different	 ways	

each	utility	may	classify	its	data	and	due	to	lack	of	definitional	precision	in	the	Form	861	itself.	As	

an	example,	we	are	not	sure	if	reported	EE	savings	are	based	on	engineering	calculations,	deemed,	

verified,	M&V	 gross,	M&V	 net,	 or	 produced	 by	 a	 combination	 of	 these	 methods.	 Uncertainty	 also	

exists	regarding	which	costs	are	reported	by	each	utility,	and	how	cost	categories	are	defined.	The	

Benchmarking	Subcommittee	has	worked	with	EIA	 over	 the	course	 of	 the	past	 year	 to	clarify	 the	

inputs	 to	 EIA	 cost	 categories,	 both	 for	 current	 and	 future	 reporting	 formats.	 In	 the	 future,	 the	

Subcommittee	may	nevertheless	wish	to	issue	guidelines	to	help	public	power	utilities	report	their	

costs	to	EIA.	One	option	is	the	adoption	of	guidelines	and	protocols	created	by	others,	such	as	DOE’s	

SEE	 Action	 Work	 Group,	 the	 Northwest	 Regional	 Technical	 Forum,	 or	 the	 Northeast	 Energy	

Efficiency	Partnerships.	In	2012‐13,	LPPC	utilities	worked	diligently	with	EIA	staff	and	consultants	

to	 improve	 the	 quality	 of	 data	 used	 in	 this	 and	 upcoming	 Benchmarking	 Reports:	 for	 additional	

details	see	Appendix	B	–	Overview	of	DEED	Grant	Activities.	

As	 mentioned	 in	 “Description	 of	 LPPC	 Utility	 Population,”	 comparing	 retail	 to	 wholesale	 utilities	

also	 produces	 uncertainty,	 since	 utilities	 do	 not	 report	 whether	 they	 provide	 energy	 efficiency	

services	to	both	wholesale	and	retail	customers.	This	report	generally	assumes	that	utilities	direct	

energy	 efficiency	 spending	 toward	 retail	 customers	 only,	 and	 thus	 uses	 retail	 sales	 (MWh)	 and	

retail	 revenues	 ($)	 wherever	 sales	 and	 revenue	 numbers	 are	 called	 for.	 In	 cases	 where	 utilities	

direct	 energy	 efficiency	 spending	 toward	 retail	 and	 wholesale	 customers,	 the	 report	 bases	 its	

analysis	 on	 retail	 plus	 wholesale	 sales	 and	 on	 retail	 plus	 wholesale	 revenues.	 In	 this	 report,	

however,	 this	 accommodation	 has	 only	 been	 made	 for	 NPPD	 and	 Platte	 River,	 which	 drew	 the	

report	authors’	attention	to	this	situation	(see	earlier	footnote	for	additional	clarification).	

Small	sample	size	is	also	an	issue,	as	discussed	elsewhere	in	 this	report.	The	charts	in	 this	report	

will	 help	 LPPC	 utilities	 compare	 themselves	 to	 their	 peers,	 but	 the	 small	 sample	 size	 limits	 the	

conclusions	 that	 LPPC	 utilities	 can	 draw	 from	 these	 comparisons.	 The	 “National	 Comparison”	

section	brings	in	data	 from	 IOU’s	and	cooperatives	 to	provide	a	more	statistically	valid	picture	of	

utility	performance	relative	to	a	broader	population	of	similarly	sized	utilities.	
LPPC EEWG Benchmarking Subcommittee   

Page 13

Metric‐Specific Data Issues

Cost

Measures	in	 the	 Cost	metric	 are	 affected	 primarily	 by	imprecise	 definitions	 of	 the	 data	 elements	

comprising	 them	 and	 a	 lack	 of	 direction	 on	 how	 to	 apportion	 costs.	 For	 example,	 utilities	 may	

report	 marketing	 costs	 as	 direct	 costs	 or	 as	 indirect	 costs,	 depending	 on	 how	 internal	 cost	

categories	 are	 structured	 and	 allocated.	 Or,	 marketing	 activities	 may	 be	 handled	 by	 another	

department	 and	 entirely	 excluded	 from	 EIA	 reporting.	 Similar	 situations	 may	 occur	 with	 EM&V	

costs.	Additionally,	it	is	unclear	how	utilities	count	employees	working	part‐time	on	DSM	activities	

and	whether	utilities	are	including	fully‐burdened	rates	when	reporting	labor	costs.	Some	utilities	

do	 not	 separately	 report	 indirect	 costs,	 and	 for	 2011	 data,	 EIA	 did	 not	 ask	 utilities	 to	 allocate	

indirect	costs	between	EE	and	DR	programs.	 In	 this	report,	indirect	costs	were	allocated	 to	EE	or	

DR	according	 to	 the	relative	weight	of	direct	spending	in	 these	categories.	LPPC	utilities	reported	

between	 0‐25%	 of	 total	 expenses	 as	 indirect	 costs,	 with	 eleven	 utilities	 reporting	 0‐2%	 of	 total	

expenses	as	indirect	costs.	

Additionally,	some	utilities	report	expenses	only	 for	EE	or	DR,	but	not	both.	 It	is	unclear	whether	

this	reflects	a	lack	of	programmatic	activity	in	one	of	these	areas,	or	whether	the	utility	is	reporting	

combined	expenses	under	one	category.		

As	a	result	of	LPPC	EEWG	efforts	with	the	EIA,	the	current	form	planned	for	2014	will	eliminate	the	

five	cost	categories	of	“Direct	Costs	EE”,	“Direct	Costs	LM”	(DR),	“Incentive	Payments	EE”,	Incentive	

Payments	 LM”	 (DR),	 and	 “Indirect	 Costs”.	 For	 2013	 data,	 the	 categories	 of	 “customer	incentives”	

and	 “all	 other	 costs”	 will	 be	 listed	 separately	 for	 energy	 efficiency	 programs	 and	 for	 demand	

response	programs,	 for	a	 total	of	 four	cost	categories.	This	will	aid	consistency	and	ensure	better	

accuracy	 in	 future	 reports	 by	 eliminating	 the	 need	 to	 allocate	 indirect	 costs	 based	 on	 relative	

weighting.	

Savings

As	 noted	 previously,	measures	in	 the	Savings	metric	are	affected	 by	 the	 type	 of	 savings	 reported	

(gross,	net,	etc.).	In	this	report,	we	have	made	an	effort	to	normalize	savings	achieved	by	utility	size,	

either	 on	 a	 “per	MWh	 sales”	 or	a	 “per	 revenues”	 basis.	Normalization	 by	 revenues	 can	introduce	

bias	based	on	how	retail	electric	rates	differ	among	utilities.	Additionally,	certain	savings	categories	

had	 few	 reported	 data	 points:	 many	 utilities	 reported	 neither	 MW	 peak	 demand	 reductions	 nor	

incidental	MWh	savings	from	load	management	(DR)	programs.	

Cost‐to‐Savings Rates

The	biggest	barrier	to	using	Form	861	data	to	determine	cost‐to	savings	rates	is	the	fact	that	it	does	

not	 collect	 information	 on	 lifetime	 savings	 of	 energy	 efficiency	 measures.	 The	 form	 collects	

information	 on	 incremental	 effects—that	 is,	 current‐year	 savings	 from	 all	 participants	 in	 new	

programs	and	new	participants	in	old	programs—without	soliciting	the	longevity	information	that	

would	 enable	 a	 true	 measure	 of	 cost‐effectiveness.	 The	 Benchmarking	 Subcommittee	 and	 other	

stakeholders	 have	 discussed	 this	 issue	 with	 EIA,	 which	 has	 responded	 by	 publishing	 proposed	
LPPC EEWG Benchmarking Subcommittee   

Page 14

changes	 to	 Form	 EIA‐861	 that	 would	 solicit	 information	 on	 longevity	 via	 a	 request	 for	 the	

“weighted	average	life”	of	savings.	

Measures	in	the	Cost‐to‐Savings	Rate	metric	are	also	affected	by	the	same	issues	existing	in	the	Cost	

and	 Savings	 metrics,	 along	 with	 the	 fact	 that	 actual	 reporting	 of	 demand	 data	 in	 the	 survey	 is	

sparse.	As	noted	above,	customer	contribution	is	not	collected	by	Form	861,	which	means	that	the	

full	 cost	 of	 incremental	 energy	 saved	 is	 not	 reflected.	 Similarly,	 the	 Cost‐to‐Savings	 Rate	 metric	

focuses	on	 the	cost	of	saved	energy	and	does	not	address	 the	monetary	value	of	savings	benefits,	

such	 as	 bill	 savings,	 increased	 customer	 comfort,	 reduced	 emissions,	 etc.	 These	 benefits	 can	 be	

estimated,	but	this	is	beyond	the	scope	of	this	report.	

Improving Data Collection, Quality, and Usefulness

Since	 issuing	 the	 inaugural	 benchmarking	 report	 in	 June	 2012,	 the	 LPPC	 EEWG	 Benchmarking	

Subcommittee	 has	 made	 a	 significant	 effort	 to	 improve	 the	 quality	 of	 energy	 efficiency	 data	

generated	 via	 the	 U.S.	 Energy	 Information	 Administration’s	 (EIA)	 data	 collection	 efforts.	 The	

Subcommittee	has	partnered	closely	with	the	EIA	to:	

 Submit	feedback	on	the	EIA‐produced	User	Guide	for	EIA	Form	861,	Schedule	6.	

 Plan,	develop,	and	conduct	two	webinars	on	EIA	energy	efficiency	data,	which	were	

attended	by	167	utility	staff,	include	86	public	power	utility	staff.	

 Provide	70	public	power	utilities	with	technical	assistance	on	EIA	Form	861,	Schedule	6.	

 Submit	feedback	on	EIA’s	proposed	changes	to	their	data	collection	for	EIA	Form	861,	

Schedule	6	for	the	2014	data	collection	cycle	(2013	data	year).

While	impacts	 of	 this	effort	will	 not	 be	 seen	immediately,	 these	investments	in	generating	 better	

data	 will	 yield	 better	 intelligence	 in	 years	 to	 come.	 Further	 discussion	 on	 improving	 utility	 data	

reporting	on	EIA	Form	861	appears	later	in	this	report,	including	Appendix	B	–	Overview	of	DEED	

Grant	 Activities.	 The	 Benchmarking	 Subcommittee	 has	 submitted	 recommendations	 to	 EIA	 that	

would	resolve	some	of	these	issues	of	concern:	visit	www.eia.gov/survey/changes/electricity/	 for	

EIA’s	 proposed	 revisions	 to	 Form	 EIA‐861,	 Schedule	 6	 2014	 form	 (for	 2013	 data).	 	 The	

subcommittee’s	 main	 categories	 of	 recommendation,	 many	 of	 which	 would	 be	 addressed	 by	 the	

changes,	include:	

 Definitions	of	specific	terms	for	consistent	understanding	and	application16

 Clarification	on	 the	data	inputs	and	what	 they	do	or	do	not	include	–	some	improvements	

are	currently	planned	for	inclusion	in	the	2014	form.	

 Providing	examples	of	how	some	calculations	should	be	done	–	EIA	has	proposed	to	provide	

a	calculation	worksheet	for	certain	data	requested	by	the	2014	form.	

 Providing	more	opportunities	for	transparency	of	utility	calculation	methodology.	

																																																													

16 EIA provides a glossary, available here: http://www.eia.gov/tools/glossary/index.cfm, and NEEP has created a

glossary, available here: http://neep.org/emv‐forum/project‐committees/protocol‐development‐projects.
LPPC EEWG Benchmarking Subcommittee   

Page 15

 Asking	 for	information	 about	 the	longevity	 of	 EE	 savings	 (e.g.	weighted	 average	measure	

life,	expected	useful	life,	etc.)	–	currently	planned	for	inclusion	in	the	proposed	2014	form.	

 Standardization	of	reporting	“burdened”	costs	for	all	labor	costs	–	not	fully	addressed	by	the	

proposed	2014	form.	

 Reporting	 of	 Administrative	 and	 Marketing	 costs	 separately	 –	 not	 addressed	 by	 the	

proposed	2014	form.	

 Appropriate	 allocation	 of	 “indirect	 costs”	 to	 either	 EE	 programs	 or	 DR	 programs.17		

Currently,	 indirect	 costs	 (sometimes	 substantial)	 are	 combined	 into	 only	 one	 category.	 –	

Currently	planned	for	inclusion	in	the	2014	form,	via	a	simplification	of	the	cost	categories.	

The	EEWG	Benchmarking	Subcommittee	is	by	no	means	the	first	group	to	suggest	improvements	to	

EIA	data	collection	and	usefulness.	Others	have	developed	similar	recommendations	in	the	past	and	

the	Subcommittee	 assumes	 that,	in	 addition	 to	 the	Subcommittee,	 several	 other	 groups	 provided	

feedback	 to	EIA	 via	 the	public	comment	period	on	its	proposed	2014	 form	 (comments	submitted	

March	6	to	May	6,	2013).	The	Subcommittee’s	public	comments	are	available	in	Appendix	C	–	EEWG	

Benchmarking	 Subcommittee’s	 Public	 Comments	 to	 the	 U.S.	 Energy	 Information	 Administration	

regarding	EIA	Form	861	(Submitted	May	6,	2013).	

																																																													

17 Currently, indirect costs (sometimes substantial) are combined into only one category.
LPPC EEWG Benchmarking Subcommittee   

Page 16

Conclusions and Recommendations for Future Research

This	 report	 builds	 on	 the	 Subcommittee’s	 inaugural	 report,	 including	 a	 modest	 expansion	 of	 the	

type	of	data	collected	and	analyzed.	This	report	is	intended	to	provide	a	foundation	for	developing	

useful	energy	efficiency	performance	data	and	indicators.	Measures	contained	herein	are	based	on	

a	very	small	participant	sample	size	and	have	some	issues	as	noted	and	 for	 this	reason	should	be	

viewed	as	a	very	high	level	snapshot	of	what	the	actual	population	of	energy	efficiency	performance	

actually	is.		Data	accuracy	exists	only	within	the	participant	reported	data	range	and	should	not	be	

extrapolated	beyond	this.	Data	precision	is	low	due	to	the	small	sample	size.	

In	the	process	of	soliciting	and	reviewing	participant	data,	more	questions	and	ideas	have	surfaced	

that	 could	 pave	 the	 road	 to	 ongoing	 and	 dynamic	 performance	management	 tools.	This	 potential	

will	 be	 constrained,	 however,	 by	 limited	 resources	 and	 potentially	 by	 demands	 for	 narrow	 and	

quick‐response	cost	and	savings	measurements	that	could	be	used	too	literally.	

Many	 ideas	 for	 future	 research	 and	 development	 have	 surfaced	 during	 various	 Subcommittee	

discussions	 and	 this	 is	 natural	 for	 the	 early	 stages	 of	 an	 initiative	 to	 exchange	 performance	

information	and	develop	knowledge	and	intelligence	for	improvement	of	processes.	The	challenge	

going	 forward	 will	 be	 to	 categorize	 and	 prioritize	 which	 areas	 and	 activities	 have	 the	 best	 cost‐

benefit	potential	given	limited	resources	to	pursue	them.	

Areas	of	note	include	(but	are	certainly	not	limited	to):	

 Addressing	identified	data	issues	via	increasing	participant	sample	size	and	developing	(or	

adopting/borrowing)	 precise	 data	 definitions	 and	 reporting	 protocols	 along	 with	 peer	

review	of	responses	(DEED	grant	activities	included	review/error	analysis	of	2011	data).	

 Using	EIA	data	for	IOU’s	and	cooperatives	to	expand	and	contrast	LPPC	performance	data.	

 Creating	performance	analyses	databases	for	each	specific	year	along	with	combined	years	

and	producing	summary	data	reports.	

 Obtaining	and	reviewing	extant	EE	performance	benchmarking	reports	and	data	(first	steps	

have	been	taken	to	do	this	in	the	current	report).	

 Developing	a	hierarchy	 of	measure	usefulness	 to	reduce	 the	amount	of	data	collected	and	

processed.	 For	instance,	 the	 80/20	 Pareto	 principle	 holds	 that	80%	 of	important	 outputs	

are	the	result	of	20%	of	the	inputs.	

 Developing	natural	subsets	of	utilities	with	similar	characteristics,	so	 that	more	useful,	in‐

depth	 comparisons	 can	 be	 conducted	 (e.g.	 primarily	 wholesale	 utilities	 or	 utilities	 with	

similar	peaking	characteristics).	

 Creating	 a	 cross‐tabbed	 database	 of	 energy	 efficiency	 portfolios,	 including	 distinguishing	

characteristics	and	programs,	and	statistically	correlating	them	to	performance	results.	

 Adding	case	studies	to	the	website	to	allow	utilities	to	describe	their	use	of	benchmark	data.	

 Capturing	 information	 on	 measure	 longevity,	 perhaps	 in	 the	 form	 of	 a	 weighted	 average	

measure	life	across	each	utility’s	energy	efficiency	program	portfolio	(will	be	facilitated	by	

EIA‐861	data	starting	with	2013	data	year).	
LPPC EEWG Benchmarking Subcommittee   

Page 17

Charts

Cost Measures

Chart 1: Total EE & DR Spending, 2011 (in millions of $)

Description:		Includes	direct,	incentive,	and	indirect	spending	for	both	EE	and	DR	programs.	

Chart 2:		Total EE Spending Relative to Retail Sales, 2011 (in $/MWh)

Description:	Relates	direct,	incentive,	and	indirect	EE	spending	 to	retail	sales,	normalizing	 for	utility	size.	 “Retail	Sales”

numbers	 for	 Tennessee	 Valley	 Authority,	Nebraska	 Public	 Power	District,	 and	 Platte	 River	 Power	 Authority	 have	 been	

adjusted	to	reflect	the	full	customer	base	to	which	these	utilities	provide	EE	services.	
LPPC EEWG Benchmarking Subcommittee   

Page 18

Chart 3:		Total DR Spending Relative to Peak Demand, 2011 (in $/MW)

Description:	Relates	direct,	incentive,	and	indirect	DR	spending	to	peak	capacity,	normalizing	for	utility	size.	

Chart 4:		Total EE & DR Spending Relative to Retail Revenue, 2011 (in %)

Description:		Relates	direct,	incentive,	and	indirect	EE	and	DR	spending	to	each	utility’s	total	retail	revenues,	normalizing	

for	utility	size.	This	measure	is	skewed	by	a	utility’s	retail	electric	rates.	
LPPC EEWG Benchmarking Subcommittee   

Page 19

Chart 5: Total EE, DR, & Renewable FTE Employees by Type (External vs. Internal), 2011

Description:	 	 Shows	 the	 full‐time	 employees	 each	 utility	 has	 allocated	 as	 resources	 for	 energy	 efficiency,	 demand	

response,	 and	 renewable	 program	 activities.	 Data	 on	 FTE’s	 is	 not	 collected	 by	 EIA‐861	 and	 was	 provided	 directly	 by	

utilities	 to	 the	 LPPC	 EEWG.	 No	 FTE	 data	 was	 provided	 for	 Tennessee	 Valley	 Authority,	 Puerto	 Rico	 Electric	 Power	

Authority,	and	Imperial	Irrigation	District.
LPPC EEWG Benchmarking Subcommittee   

Page 20

Chart 6: Total EE, DR, & Renewable FTE Employees Relative to MWh Retail Sales, 2011

(in FTEs per million MWh)

Description:	 Relates	 the	 full‐time	 employees	 each	 utility	 has	 allocated	 as	 resources	 for	 energy	 efficiency,	 demand	

response,	and	renewable	program	activities	 to	retail	sales,	normalizing	 for	utility	size.	Data	on	FTEs	is	not	collected	by	

EIA‐861	and	was	provided	by	utilities	directly	to	the	LPPC	EEWG.	“Retail	Sales”	numbers	for	Tennessee	Valley	Authority,	

Nebraska	Public	Power	District,	and	Platte	River	Power	Authority	have	been	adjusted	to	reflect	the	full	customer	base	to	

which	these	utilities	provide	EE	services.	No	FTE	data	was	provided	for	Tennessee	Valley	Authority,	Puerto	Rico	Electric	

Power	Authority,	and	Imperial	Irrigation	District.
LPPC EEWG Benchmarking Subcommittee   

Page 21

Savings Measures

Chart 7:		Incremental Energy Savings from EE, by Sector, 2011 (in MWh)	 	

Description:	 Shows	 energy	 savings	 achieved	 by	 EE	 in	 current	 year	 (excludes	 energy	 effects	 of	 DR).	 Includes	 only	 new	

participants.18		

Chart 8: Incremental Energy Savings from EE Relative to Retail Sales, 2011 (in %)

Description:		“Shows	 the	energy	savings	achieved	by	EE	in	 the	current	year	(excludes	effects	of	DR).	 Includes	only	new	

participants.19	 Relates	 energy	 savings	 to	 retail	 sales,	 normalizing	 for	 utility	 size.	 Retail	 Sales”	 numbers	 for	 Tennessee	

Valley	Authority,	Nebraska	Public	Power	District,	and	Platte	River	Power	Authority	have	been	adjusted	to	reflect	the	full	

customer	base	to	which	these	utilities	provide	EE	services.	Green	shading	indicates	better	performance.	

																																																													

18 See earlier footnote for definition of “incremental effects.” 19 See earlier footnote for definition of “incremental effects.”
LPPC EEWG Benchmarking Subcommittee   

Page 22

Chart 9: Incremental Energy Savings from Sector‐Specific EE Relative to Retail Sales from

that Sector, 2011 (in %)

Description:	 	Shows	 the	energy	 savings	achieved	 by	EE	in	 the	 current	 year	 (excludes	effects	 of	DR).	 Includes	 only	new	

participants.20	Relates	energy	savings	at	the	sector‐level	to	retail	sales	from	that	sector,	normalizing	for	utility	size.	“Retail	

Sales”	numbers	 for	Tennessee	Valley	Authority,	Nebraska	Public	Power	District,	and	Platte	River	Power	Authority	have	

been	adjusted	to	reflect	the	full	customer	base	to	which	these	utilities	provide	EE	services.	Green	shading	indicates	better	

performance.	

Chart 10: Peak Demand Reduction from EE & DR, 2011 (in MW)

Description:		Shows	peak	demand	saved	by	EE	and	DR	programs.	Only	seven	utilities	reported	DR	peak	demand	savings	

from	DR,	a	very	small	sample	size	for	comparison	purposes.	

																																																													

20 See earlier footnote for definition of “incremental effects.”
LPPC EEWG Benchmarking Subcommittee   

Page 23

Chart 11: Peak Demand Reduction from EE & DR, by Sector, 2011 (in MW)

Description:		Shows	peak	demand	saved	by	EE	and	DR	programs.	Only	seven	utilities	reported	peak	demand	savings	from	

DR,	a	very	small	sample	size	for	comparison	purposes.	

Chart 12:	 	 Peak Demand Reduction from EE & DR Relative to Annual System Peak

Demand, 2011 (in %)

Description:		Shows	peak	demand	saved	by	EE	and	DR	programs.	Relates	peak	demand	savings	 to	annual	system	peak	

demand,	normalizing	 for	utility	size.	Depicts	 the	relative	aggressiveness	of	DSM	programs.	Only	seven	utilities	reported	

peak	demand	savings	from	DR.	Green	shading	indicates	better	performance.	
LPPC EEWG Benchmarking Subcommittee   

Page 24

Chart 13:	 	 Peak Demand Reduction from EE & DR Relative to Annual System Peak

Demand, by Sector, 2011 (in %)

Description:		Shows	peak	demand	saved	by	EE	and	DR	programs.	Relates	peak	demand	savings	 to	annual	system	peak	

demand,	normalizing	for	utility	size.	The	ranking	depicts	the	relative	aggressiveness	of	DSM	programs.	Again,	only	seven	

utilities	reported	peak	demand	savings	from	DR.	Green	shading	indicates	better	performance.	
LPPC EEWG Benchmarking Subcommittee   

Page 25

Cost‐to‐Savings Rate Measures

Chart 14:		EE Spending Relative to Incremental Energy Savings, 2011 (in $/MWh)

Description:		Relates	EE	spending	to	incremental	MWh	saved.		Measures	cost‐effectiveness	achieved	on a first year basis;	

does	 not	 account	 for	 measure	 life	 or	 levelized	 costs.	 Includes	 only	 new	 participants.21	 Green	 shading	 indicates	 better	

performance.		

Chart 15:		EE Spending Rel. to Incremental Energy Savings, by Sector, 2011 (in $/MWh)

Description:		Relates	EE	spending	to	incremental	MWh	saved.		Measures	cost‐effectiveness	achieved	on a first year basis;	

does	 not	 account	 for	 measure	 life	 or	 levelized	 costs.	 Includes	 direct	 spending	 and	 incentives.	 Includes	 only	 new	

participants.22	Green	shading	indicates	better	performance.		

																																																													

21 See earlier footnote for definition of “incremental effects.” 22 See earlier footnote for definition of “incremental effects.”
LPPC EEWG Benchmarking Subcommittee   

Page 26

Chart 16: EE & DR Spending Relative to Peak Demand Saved (in $/kW)

Description:	Relates	EE	spending	to	peak	demand	reductions	from	EE	programs	(blue)	and	relates	DR	spending	to	peak	

demand	reductions	from	DR	programs	(red).	This	is	useful	for	utilities	that	measure	demand	acquisition	against	the	cost	

of	installed	generation.	Utilities	with	high	 $/kW	numbers	generally	have	excess	capacity	and/or	little	emphasis	 on	DR.	

Green	shading	indicates	better	performance.		

Chart 17: Incremental Energy Savings from EE Relative to # of (Internal & External) EE

FTE’s & from DR Relative to # of (Internal & External) DR FTE’s, 2011 (in MWh/FTE)

Description:		Relates	energy	savings	to	the	number	of	EE	and	DR	FTEs.	Data	on	FTEs	is	not	collected	by	EIA‐861	and	was	

provided	by	utilities	directly	 to	 the	LPPC	EEWG.	 Includes	only	new	participants	in	old	programs	and	all	participants	in	

new	programs.23	No	FTE	data	was	 provided	 for	Tennessee	Valley	Authority,	Puerto	Rico	Electric	Power	Authority,	and	

Imperial	Irrigation	District.

																																																													

23 See earlier footnote for definition of “incremental effects.”
LPPC EEWG Benchmarking Subcommittee   

Page 27

Trends: 2009‐2011

Chart 18: 2009‐2011 Total Retail Sales (in MWh)

Description:	 “Retail	 Sales”	 numbers	 for	 Tennessee	 Valley	 Authority,	 Nebraska	 Public	 Power	 District,	 and	 Platte	 River	

Power	Authority	have	been	adjusted	to	reflect	the	full	customer	base	to	which	these	utilities	provide	EE	services.

Chart 19: 2009‐2011 Retail Revenue (in billions of $)

Description:	“Retail	Revenue”	numbers	for	Tennessee	Valley	Authority,	Nebraska	Public	Power	District,	and	Platte	River	

Power	Authority	have	been	adjusted	to	reflect	the	full	customer	base	to	which	these	utilities	provide	EE	services.
LPPC EEWG Benchmarking Subcommittee   

Page 28

Chart 20: 2009‐2011 Total EE & DR Spending (in millions of $)		

Chart 21: 2009‐2011 Total EE Spending (in millions of $)
LPPC EEWG Benchmarking Subcommittee   

Page 29

Chart 22: 2009‐2011 Total DR Spending (in millions of $)

Chart 23: 2009‐2011 Total EE & DR Spending Relative to Retail Revenue (in %)

Description:	:	“Retail	Revenue”	numbers	for	Tennessee	Valley	Authority,	Nebraska	Public	Power	District,	and	Platte	River	

Power	Authority	have	been	adjusted	to	reflect	the	full	customer	base	to	which	these	utilities	provide	EE	services.	Green	

shading	indicates	better	performance.		
LPPC EEWG Benchmarking Subcommittee   

Page 30

Chart 24: 2009‐2011 Total Incremental Energy Savings from EE (in MWh)

Chart 25: 2009‐2011 Total EE & DR FTE’s (Internal and External/Contracted)

Description:	 No	 FTE	 data	 was	 provided	 for	 Tennessee	 Valley	 Authority,	 Puerto	 Rico	 Electric	 Power	 Authority,	 and	

Imperial	Irrigation	District.
LPPC EEWG Benchmarking Subcommittee   

Page 31

Chart 26: 2009‐2011 Total Incremental Savings from EE Rel. to MWh Retail Sales (in %)

Description:	 “Retail	 Sales”	 numbers	 for	 Tennessee	 Valley	 Authority,	 Nebraska	 Public	 Power	 District,	 and	 Platte	 River	

Power	Authority	have	been	adjusted	to	reflect	the	full	customer	base	to	which	these	utilities	provide	EE	services.	Green	

shading	indicates	better	performance.	

Chart 27: 2009‐2011 Total Incremental EE Spending Relative to Incremental EE MWh

Saved (in $/MWh)

Description:	Green	shading	indicates	better	performance.		
LPPC EEWG Benchmarking Subcommittee   

Page 32

Chart 28: 2009‐2011 Total Incremental EE Spending Relative to Incremental Peak kW

Load Reduction from EE (in $/kW)

Description:	 Green	 shading	 indicates	 better	 performance.	 Utilities	 with	 high	 $/kW	 numbers	 generally	 have	 excess	

capacity	and/or	little	emphasis	on	DR.	

Chart 29: 2009‐2011 Load Factor (MWh per Peak kW x 8,760)
LPPC EEWG Benchmarking Subcommittee   

Page 33

Short‐Term and Long‐Term Goals

The	Benchmarking	Subcommittee’s	supplemental	data	survey	asked	utilities	to	report	their	short‐

term	and	long‐term	energy	 savings	goals.	These	goals	are	depicted	below	 on	a	percent	 of	 system	

basis.	

Chart 30: Short‐Term Goals for Energy Savings (MWh as % of Total MWh Retail Sales)

Chart 31: Long‐Term Goals for Energy Savings (MWh as % of Total MWh Retail Sales)

Note:	All	energy	savings	are	from	energy	efficiency,	except	for	Austin	Energy,	which	includes	a	small	amount	

from	demand	response.		
LPPC EEWG Benchmarking Subcommittee   

Page 34

Chart 32: Short‐Term Demand Goals for Energy Efficiency and Demand Response (MW as

% of Peak Demand)

Note:	Goals	over	multiple	years	were	averaged	and	are	represented	here	on	a	one‐year	basis.	

Chart 33: Long‐Term Demand Goals for Energy Efficiency (MW as % of Peak Demand)
LPPC EEWG Benchmarking Subcommittee   

Page 35

Chart 34: Long‐Term Demand Goals for Demand Response (MW as % of Peak Demand)

Chart 35: Long‐Term Demand Goals for Demand‐Side Renewables (MW Installed

Capacity as % of Peak Demand)
LPPC EEWG Benchmarking Subcommittee   

Page 36

Chart 36: Long‐Term Goals for Demand‐Side Renewables (MWh Annually as % of Total

MWh Retail Sales)

Chart 37: Long‐Term Demand Goals for Supply‐Side Renewables (MW Installed Capacity

as % of Peak Demand)
LPPC EEWG Benchmarking Subcommittee   

Page 37

Chart 38: Long‐Term Goals for Supply‐Side Renewables (MWh Annually as % of Total

MWh Retail Sales)
LPPC EEWG Benchmarking Subcommittee   

Page 38

National Comparison24

The	national	comparison	percentile	table	and	charts	that	follow	are	based	on	2011	EIA	data	from	all	

utilities	 that	 filled	 out	 EIA	 Form‐861,	 Schedule	 6,	 including	 LPPC	 members.	 Charted	 data	 may	

exclude	extreme	values	 for	visual	clarity.	A	percentile	is	a	value	on	a	scale	of	0‐100	 that	indicates	

the	percent	of	a	distribution	above	or	below	it.		

As	 an	 example,	 Austin	 Energy	 experienced	 a	 cost‐effectiveness	 rate	 of	 $125/MWh	 saved	 by	 EE,	

which	 resulted	 in	 a	 percentile	 rank	 of	 28%.	 This	 means	 that	 about	 72%	 of	 the	 EIA	 respondents	

were	 as	 high	 or	 higher	 in	 terms	 of	 $/MWh	 saved	 by	 EE.	 Conversely,	 about	 28%	 of	 the	 EIA	

respondents	were	as	low	or	lower.	Depending	on	the	measure,	a	low	percentile	can	indicate	good	or	

bad	relative	standing.

The	national comparison data	(in	contrast	to	the	data	in	the	rest	of	the	report)	are	taken	directly	

from	EIA	and	do	not	reflect	accommodations	or	corrections	made	by	LPPC	utilities	that	are	

included	elsewhere	in	this	report.	Platte	River	Power	Authority’s	four	member	utilities	(City	of	Fort	

Collins,	City	of	Longmont,	City	of	Loveland,	and	Town	of	Estes	Park)	are	included	here	in	lieu	of	

Platte	River	Power	Authority,	which	does	not	report	on	Schedule	6.	Each	point	on	a	chart	

represents	one	of	these	LPPC	or	LPPC‐related	utilities.	

The	data	for	TVA,	NPPD,	and	Platte	River	in	the	national	comparison	may	be	misleading,	especially	

when	compared	to	data	earlier	in	the	report,	because	these	data	have	not	been	adjusted	to	account	

for	its	wholesale	customers’	sales	and	revenues.

																																																													

24 Data in the National Comparison section is drawn directly from EIA’s 2011 data and does not include any self‐

corrected data from LPPC utilities. Also, in contrast to the rest of the charts in this report, spending numbers

do not include weighted indirect costs for EE and DR.
LPPC EEWG Benchmarking Subcommittee Page 39

Data Table for Five Key EE Performance Measures

LPPC Utility Name

Cost Effectiveness EE

Spending $/MWh Saved Percentile

EE Spending $ as a %

of Retail Sales $ Percentile

EE Spending $/Retail

Sales MWh Percentile EE Spending $ (,000) Perc

Austin Energy $ 125 28% 1.30% 70% 1.13 $ 67% 14,768 $ 

City of Colorado Springs - (CO) $ 80 12% 0.40% 35% 0.34 $ 31% 1,544 $ 

City of Fort Collins - (CO) $ 188 44% 2.88% 90% 1.98 $ 83% 2,870 $ 

City of Longmont $ 185 43% 1.19% 65% 0.80 $ 55% 637 $ 

City of Loveland - (CO) $ 123 27% 0.76% 53% 0.50 $ 41% 351 $ 

City of San Antonio - (TX) $ 337 69% 1.30% 70% 1.09 $ 65% 23,468 $ 

City of Seattle - (WA) $ 304 65% 4.99% 97% 3.41 $ 92% 32,707 $ 

City of Tacoma - (WA) $ 203 48% 4.54% 96% 2.62 $ 88% 12,647 $ 

Imperial Irrigation District $ 629 87% 1.65% 77% 1.92 $ 82% 6,307 $ 

JEA $ 141 32% 0.71% 50% 0.80 $ 56% 9,760 $ 

Long Island Power Authority $ 242 55% 1.33% 71% 2.51 $ 87% 47,725 $ 

Los Angeles Department of Water & Power $ 143 33% 1.69% 78% 2.14 $ 84% 49,530 $ 

Nebraska Public Power District $ 173 39% 1.72% 78% 1.39 $ 73% 4,533 $ 

Omaha Public Power District $ 120 26% 0.36% 32% 0.26 $ 27% 2,823 $ 

Orlando Utilities Comm $ 319 67% 0.67% 49% 0.71 $ 53% 4,316 $ 

PUD No 1 of Chelan County $ 145 33% 3.34% 92% 1.10 $ 65% 1,728 $ 

PUD No 1 of Clark County - (WA) $ 172 38% 3.30% 92% 2.59 $ 88% 11,583 $ 

PUD No 2 of Grant County $ 81 12% 2.98% 91% 1.00 $ 63% 4,019 $ 

Sacramento Municipal Util Dist $ 186 43% 2.56% 87% 3.05 $ 90% 31,707 $ 

Salt River Project $ 77 11% 1.20% 66% 1.12 $ 67% 29,961 $ 

Snohomish County PUD No 1 $ 250 58% 3.90% 95% 2.95 $ 89% 20,298 $ 

South Carolina Pub Serv Auth $ 590 86% 1.31% 70% 0.89 $ 59% 9,997 $ 

Town of Estes Park $ 286 62% 1.41% 72% 1.35 $ 72% 172 $ 

Note: all EE spending is incremental

Note:	The	following	cities	are	included	here	because	they	are	served	by	Platte	River	Power	Authority:	Fort	Collins,	Longmon
LPPC EEWG Benchmarking Subcommittee Page 40

Chart 39: Percentiles for “Cost‐Effectiveness,” or EE Spending Relative to Incremental Energy Savings (in $/MW
LPPC EEWG Benchmarking Subcommittee Page 41

Chart 40: Percentiles for Total EE Spending Relative to Retail Revenue (in %)
LPPC EEWG Benchmarking Subcommittee Page 42

Chart 41: Percentiles for Total EE Spending Relative to Retail Sales (in $/MWh)
LPPC EEWG Benchmarking Subcommittee Page 43

Chart 42: Percentiles for EE Spending (in thousands of $)
LPPC EEWG Benchmarking Subcommittee Page 44

Chart 43: Percentiles for Incremental Energy Savings from EE Relative to Retail Sales (in %)
LPPC EEWG Benchmarking Subcommittee Page 45

Data Table for Four Key DR Performance Measures

Utility Name

% Summer Peak (MW) 

of Actual Annual DR Percentile

% Winter Peak (MW) 

of Actual Annual DR Percentile

Redundancy (Ratio of 

Potential to Actual) Percentile

Direct and Inc

Costs per A

MW ($/MW

Austin Energy 0.54% 36% 3.42% 45% 0.01 2% 3

$ 

City of Colorado Springs - (CO) 1.03% 20% 3

$ 

City of Fort Collins - (CO) 2.71% 37% 2

$ 

City of LongmontCity of Loveland - (CO) 7.48% 100% 7.03% 68% 11

$ 

City of San Antonio - (TX) 0.01% 2% 2.28% 34% 14

$ 

City of Seattle - (WA) 2.90% 72%City of Tacoma - (WA) 0.17% 20%Imperial Irrigation DistrictJEA 0.15% 16%Long Island Power AuthorityLos Angeles Department of Water & PowerNebraska Public Power District 0.41% 31% 26.36% 97% 0.02 9% 

$ 

Omaha Public Power District 3.06% 42% 2

$ 

Orlando Utilities CommPUD No 1 of Chelan CountyPUD No 1 of Clark County - (WA)PUD No 2 of Grant CountySacramento Municipal Util DistSalt River Project 7.77% 71% 1

$ 

Snohomish County PUD No 1South Carolina Pub Serv Auth 0.11% 13% 0% 6

$ 

Town of Estes Park 29.63% 

$ 

Note:	The	following	cities	are	included	here	because	they	are	served	by	Platte	River	Power	Authority:	Fort	Collins,	Longmon
LPPC EEWG Benchmarking Subcommittee Page 46

Chart 44: Percentiles for Actual Annual Demand Reduction from DR as a Percent of Summer System Peak DemChart 45: Percentiles for Actual Annual Demand Reduction from DR as a Percent of Winter System Peak Dema
LPPC EEWG Benchmarking Subcommittee Page 47

Chart 46: Percentiles for Direct and Incentive Costs Relative to Actual Peak Reduction (in $/MW)
LPPC EEWG Benchmarking Subcommittee Page 48
LPPC EEWG Benchmarking Subcommittee Page 49

Appendix A: Benchmarking Data by Utility (Main Source: EIA Form 861, 2011)25Page	1	of	4	25 LPPC utilities were given the opportunity to correct their data. These Excel snapshots reflected the corrected data.
LPPC EEWG Benchmarking Subcommittee Page 50

Page	2	of	4	
LPPC EEWG Benchmarking Subcommittee Page 51

Page	3	of	4	

Page	4	of	4	
LPPC EEWG Benchmarking Subcommittee Page 52
LPPC EEWG Benchmarking Subcommittee

Page 53

Appendix B: Overview of DEED Grant Activities

In	the	Fall	of	2012	Santee	Cooper	worked	with	five	other	DEED	utilities	(Austin	Energy,	Nebraska	

Public	 Power	 District,	 Orlando	 Utilities	 Commission,	 Omaha	 Public	 Power	 District,	 and	 Tacoma	

Public	Utilities)	and	CEEP	staff	to	apply	for	a	Demonstration	of	Energy	and	Efficiency	Developments	

(DEED)	 grant	 from	 the	 American	 Public	 Power	 Association.	 The	 group	 was	 awarded	 a	 grant	 of	

$75,000	for	work	beginning	in	January	2013	and	running	through	April	2014,	to	be	performed	by	

CEEP	staff	with	in‐kind	staff	time	contributed	by	the	participating	utilities.	

The	DEED	activities	are	designed	to	enable	public	power	utilities	to	make	better‐informed	decisions	

about	how	to	spend	limited	energy	efficiency	resources	by	improving	quality,	access	to,	and	ease	of	

interpretation	 of	 peer	 data.	Officials	 constantly	 demand	 data	 from	 public	 power	 utilities	 to	 show	

how	their	energy	efficiency	(EE)	program	performance	stacks	up	to	peer	performance.	Yet,	the	best	

source	of	national	EE	data,	EIA’s	861	survey,	is	confusing	and	filled	out	inconsistently	from	utility	to	

utility.	Resource‐constrained	utilities	find	it	difficult	to	access	reliable	data,	make	apples‐to‐apples	

comparisons,	 confidently	 rank	 their	 EE	 performance,	 and	 decide	 how	 to	 allocate	 resources.	 The	

project	 works	 to	 reduce	 utilities’	 time	 burden	 to	 report	 EIA‐861	 Schedule	 6	 data	 and	 generate	

benchmark	comparisons.	

To	date,	 the	LPPC	EEWG	Benchmarking	Subcommittee	has	partnered	closely	with	 the	U.S.	Energy	

Information	Administration	to:	

 Submit	feedback	on	the	EIA‐produced	User	Guide	for	EIA	Form	861,	Schedule	6.	

 Plan,	develop,	and	conduct	two	webinars	on	EIA	energy	efficiency	data,	which	were	

attended	by	167	utility	staff,	include	86	public	power	utility	staff	(Spring	2013).	

 Provide	70	public	power	utilities	with	technical	assistance	on	EIA	Form	861,	Schedule	6.	

 Submit	feedback	on	EIA’s	proposed	changes	to	their	data	collection	for	EIA	Form	861,	

Schedule	6	for	the	2014	data	collection	cycle	(2013	data	year).
LPPC EEWG Benchmarking Subcommittee

Page 54

In	Winter/Spring	2013,	CEEP	staff	analyzed	the	2011	EIA‐861,	Schedule	6	data	to	identify	potential	

data	measurement	and	reporting	errors	made	by	public	power	utilities.	CEEP	staff	then	contacted	

staff	at	 75+	 utilities	in	 order	 to	 discuss	 these	 potential	errors	and	 help	 utility	 staff	improve	 their	

reporting	practices	for	subsequent	years.	

Table B‐1: Summary of DEED Activities and Timeline

Activity Description Status

1) Create	Interactive	

Web	Benchmarking	

Tool	

We	will	create	a	user‐friendly	online	tool	that	generates	

customizable	peer	comparison	charts	for	indicators	such	

as	energy	savings,	potential	demand	reduction,	and	

spending	relative	to	size.	Users	may	filter	by	utility	size,	

region,	etc.	The	tool	will	include	data	from	500+	utilities	

that	submit	EIA‐861,	including	154+	public	power	

utilities.	Future	year	uploads	will	require	marginal	effort.	

Anticipated	Completion	in	

Autumn	2013	

2) Develop	and	Conduct	

Webinars	

To	improve	national	data	quality,	we	will	conduct	

webinars	on	how	to	generate	reliable,	comparable	data	

for	EIA‐861,	Sched.	6.		

Two	webinars	held	Spring	

2013;	Two	more	webinars	

planned	for	Spring	2014	

3) Write	Practical	Guide	

to	EIA‐861	

This	guide	will	give	examples	and	outline	consensus	

reporting	practices	clarified	via	project	outreach.	

Activity	scheduled	for	

Winter	2013/2014	

4) Conduct	Technical	

Assistance	Outreach	

Through	outreach,	we	will	identify	and	correct	mistakes	

in	EIA‐861,	Sched.	6	data,	spur	webinar	participation	and	

resource	use,	and	publicize	the	benchmarking	tool.	We	

expect	one	year	of	quality	control	outreach,	together	with	

new	resources,	to	result	in	sustained	improvement	in	

reporting	practices.	

2013	outreach	conducted	

Spring	2013;	minimal	

additional	outreach	

planned	for	Spring	2014	

5) Provide	

Benchmarking	

Access	for	Small	

Utilities	

Small	DEED	utilities	not	required	to	report	EIA‐861,	

Sched.	6	may	voluntarily	submit	data	to	the	online	tool	to	

generate	comparisons.	They	will	have	full	access	to	

technical	assistance	and	webinars.	

Benchmarking	

opportunity	to	be	

announced	to	small	

utilities	in	Autumn	2013	
LPPC EEWG Benchmarking Subcommittee

Page 55

Appendix C: EEWG Benchmarking Subcommittee’s Public

Comments to the U.S. Energy Information Administration regarding

EIA Form 861 (Submitted May 6, 2013)

To:	Rebecca	Peterson,	ERS2014@eia.gov

Re:	Public	Comments	on	Form	EIA–861,	‘‘Annual	Electric	Power	Industry	Report’’	

From:

Volunteer	 members	 of	 the	 Large	 Public	 Power	 Council	 Energy	 Efficiency	 Working	 Group	 (LPPC	

EEWG)	Benchmarking	Subcommittee,	led	by:	

 Subcommittee	Chair	Norman	Muraya	(Austin	Energy)	norman.muraya@austinenergy.com,	

 Member	Tom	Gross	(Orlando	Utilities	Commission)	tgross@ouc.com,	and	

 Facilitated	by	Annika	Brink	(Alliance	to	Save	Energy/Clean	and	Efficient	Energy	Program	

for	Public	Power)	abrink@ase.org.		

Over	the	course	of	the	past	year,	the	LPPC	EEWG’s	Benchmarking	Subcommittee	has	leveraged	data	

from	Form	EIA‐861,	Schedule	6	 to	benchmark	 the	energy	efficiency	activities	and	performance	of	

LPPC	member	utilities.	This	work	has	included	ongoing	discussions	among	Subcommittee	members	

and	 with	 the	 Energy	 Information	 Administration	 regarding	 the	 quality	 and	 consistency	 of	 data	

collected	via	Form	EIA‐861.	Questions	and	comments	on	the	public	comments	submitted	below	can	

be	directed	toward	the	three	working	group	participants	listed	above.	

Comments on Form EIA‐861: Excel Version of Form

Incorrect	terminology	in	Tab	“ENERGY	EFFICIENCY	6a”	

1. Tab	“ENERGY	EFFICIENCY	6a”	

a. In	the	title	of	the	last	section,	the	word	“Weighed”	in	“Weighed	Average	Life	for	

Portfolio	(Years)	‐	Use	Spreadsheet	to	Calculate”	should	be	replaced	with	

“Weighted.”	

b. In	line	10,	the	word	“Weighed”	in	“Weighed	Average	Life”	should	be	replaced	with	

“Weighted.”	

2. Tab	“Weighed	average	life	calculator”	

a. In	the	name	of	the	tab,	the	word	“Weighed”	should	be	replaced	with	“Weighted.”	

3. To	verify	that	the	term	“Weighed	average	life”	is	not	widely	used,	try	an	internet	for	this	

term,	which	retrieves	zero	results.	A	search	for	“Weighted	average	life”	on	the	other	hand,	

returns	over	3	million	results.	To	take	things	a	step	further:	we	prefer	“Weighted	average	

measure	life,”	which	is	widely	used	in	utility	energy	efficiency	and	returns	over	7,000	

results,	most	of	which	seem	to	relate	directly	back	to	energy	efficiency.	

©	used	instead	of	(c)	

 In	Tabs	“ENERGY	EFFICIENCY	6a,”	“DEMAND	RESPONSE	6b,”	and	“DYNAMIC	PRICING	

PROGRAMS	6c,”	Excel	autocorrect	has	mistakenly	replaced	(c)	with	©	for	the	“Industrial”	

columns.	

Inconsistency	between	“whom”	and	“that”	
LPPC EEWG Benchmarking Subcommittee

Page 56

 In	Tab	“ADVANCED	METERING	AND	CUSTOM	6d,”	Line	6	asks	for	the	“Number	of	

Customers	for	whom...”	while	Line	9	asks	for	the	“Number	of	customers	that	can...”		

In	the	tab	labeled	“Weighed	average	life	calculator,”	the	title	“Residential	Customers”	links	back	to	

the	 “ENERGY	 EFFICIENCY	 6a”	 Tab,	 while	 the	 titles	 for	 “Commercial	 Customers,”	 “Industrial	

Customers,”	and	“Transportation	Customers”	do	not	link	back	to	anything.	This	is	confusing.	

In	the	tab	labeled	“Weighed	average	life	calculator,”	the	terminology	“Average	mean	life	program”	is	

unclear.	 This	 should	 read	 “Average	 measure	 life,”	 or	 “Average	 life	 of	 program	 or	 measure,”	 or	

“Average	life	of	measure	or	measures	in	program”	(Columns	D,	I,	N,	and	S).	

Comments on Form EIA‐861: Pdf Version of Instructions

On	Page	2,	Point	 9,	where	 the	instructions	 state	 “See	 the	Glossary	 for	 terms	used	in	 this	survey,”	

please	provide	a	link	to	the	Glossary.	

On	Page	2,	Point	9	where	the	instructions	state	“The	financial	and	accounting	terms	are	consistent	

as	outlined	in	the	Uniform	System	of	Accounts	for	Public	Utilities	and	Licensees	(U.S.	of	A.)	(18	CFR	

Part	101),”	please	provide	a	link	to	this	source.	

On	 Page	 9‐10,	 under	 “SCHEDULE	 6.	 DEMAND‐SIDE	 MANAGEMENT	 AND	 SMART	 GRID	

INFORMATION,”	 additional	 guidance	 is	 needed	 on	 the	 reporting	 responsibilities	 of	 wholesale	

utilities	and	joint	action	agencies,	etc.	that	conduct	demand‐side	management	activities	on	behalf	of	

distribution	utilities.	The	instructions	note	that	“Power	supply	cooperatives,	municipal	joint	action	

agencies,	 and	 Federal	 Power	 Marketing	 Administrations	 should	 coordinate	 the	 reporting	 of	 DSM	

information	with	their	power	purchasing	utilities	to	avoid	double	counting	the	effects	and	costs	of	

DSM	programs.”	In	this	section,	EIA	should	also	state	its	preference	for	which	entity	should	report	

this	information.	

Typo	on	Page	10:	Where	the	instructions	state	“An	energy	efficiency	resource	must	achieve	a	long	

term	continuous	reduction	in	demand	for	electricity	and	are	available...”	the	word	“are”	should	read	

“be.”	

On	 Page	 10	 where	 the	 instructions	 state	 “Examples	 are	 replacing	 light	 bulbs	 with	 more	 efficient	

technology	 or	 replacing	 older	 HVAC	 systems	 with	 high	 efficiency	 systems,”	 we	 recommend	

replacing	this	with	better	examples:	“Examples	are	adding	additional	insulation	in	older	homes	or	

replacing	 an	 HVAC	 system	 with	 a	 higher	 efficiency	 system	 that	 exceeds	 current	 codes	 and	

standards.”	

On	Page	11,	under	“1.	Incremental	Annual	Savings—Reporting	Year,”	the	two	bullets	are	confusing	

and	could	be	eliminated	 to	simply	read:	“Participants	in	DSM	programs.”	 If	EIA	insists	on	keeping	

both	bullets,	then	we	recommend	replacing	“DSM	programs	that	operated	in	the	previous	reporting	

year”	with	“Existing	DSM	programs.”	
LPPC EEWG Benchmarking Subcommittee

Page 57

On	Page	11,	under	“Incremental	Savings—Life	Cycle,	we	recommend	striking	“DSM	programs	have	

a	useful	life,	and	 the	net	effects	of	 these	programs	will	diminish	over	 time.	To	 the	extent	possible	

consider	the	useful	life	of	efficiency	by	accounting	for	building	demolition,	equipment	degradation,	

and	program	attrition.”	This	guidance	is	directly	in	conflict	with	EIA	guidance	on	Page	10	to	report	

Gross	Savings.	As	a	reminder:	EIA	has	decided	to	switch	from	Net	Savings	to	Gross	Savings	in	order	

to	achieve	better	consistency	of	reporting	across	different	utilities.	This	guidance	would	introduce	a	

new	 opportunity	 for	 inconsistency	 by	 asking	 utilities	 to	 self‐define	 a	 type	 of	 Net	 Savings.	 	 We	

recommend	replacing	the	phrase	quoted	above	with,	“Gross	savings	do	not	consider	free	ridership,	

spillover	effects,	building	demolition,	equipment	degradation,	and	program	attrition.”	

Page	 11:	 It	 might	 be	 better	 if	 the	 terminology	 did	 not	 vary	 slightly	 between	 the	 form	 and	 the	

instructions:	

 “Reporting	Year	Incremental	Annual	Savings”	vs.	“Incremental	Annual	Savings—Reporting	

Year”	

 “Incremental	Life	Cycle	Savings”	vs.	“Incremental	Savings—Life	Cycle”	

Page	 11:	 In	 general,	 the	 proposed	 instructions	 use	 the	 word	 “program”	 where	 “measure”	 or	

“project”	would	be	more	appropriate.	

 Here,	and	throughout	the	instructions,	it	would	be	more	accurate	to	refer	to	“DSM	program	

measures”	rather	than	“DSM	programs”	when	discussing	the	useful	life	of	savings.	

 The	instructions	also	state	that	a	“Life	cycle”	mean	the	number	of	years	the	program	is	

planned	to	exist...”	First,	there	is	a	typo:	it	should	read	“means”	rather	than	“mean.”	Second,	

this	is	inaccurate:	a	program	may	exist	for	2	years,	but	install	measures	that	last	30	years.	

Clearly,	EIA	wants	to	collect	information	on	savings	from	the	installed	measures	over	the	

course	of	30	years	rather	than	from	the	two	years	the	program	is	in	existence,	actively	

implementing	new	measures.	

 Building	off	the	example	outlined	above,	we	recommend	replacing	“the	number	of	years	the	

program	is	planned	to	exist”	with	“the	number	of	years	the	effects	of	the	program	measures	

are	planned	to	exist.”	

 For	“Incremental	Savings—Life	Cycle,”	instead	of	“the	life	cycle	of	the	incremental	programs	

and	participants,”	it	would	be	better	to	say,	“the	life	cycle	of	the	program	measures	

implemented	in	the	reporting	year.”	

Page	12:	Point	3,	Costs	

 We	recommend	that	this	section	offer	guidance	on	what	“all	other	costs”	should	include,	e.g.	

utility	DSM	staff	time,	administration,	marketing,	contractual	obligations,	EM&V	costs,	etc.	

 We	recommend	that	this	section	offer	guidance	on	whether	or	not	costs	should	be	

“loaded”/“fully	burdened”	i.e.	include	fringe,	overhead,	etc.	

 We	think	EIA	should	consider	pulling	out	marketing	costs	into	their	own,	third,	separate	

category.	Marketing	often	composes	a	large	percentage	of	DSM	budgets,	making	this	data	

useful	for	benchmarking	purposes.	(If	EIA	makes	this	change,	it	should	provide	guidance	on	

whether	or	not	to	include	marketing	staff	labor	costs	as	part	of	marketing	DSM	costs.)		
LPPC EEWG Benchmarking Subcommittee

Page 58

 The	instructions	designate	“in‐kind	services	(e.g.	design	work)”	as	part	of	customer	

incentives.	However,	the	inclusion	here	of	“in‐kind	services”	could	be	misconstrued	to	

include	many	types	of	activities	traditionally	undertaken	by	utility	energy	efficiency	staff	

and	not	generally	counted	as	“customer	incentives.”	Additional	clarification	is	required.	

 We	recommend	not	switching	back	and	forth	between	“reporting	year”	and	“incremental	

year”	if	these	terms	are	meant	to	be	used	interchangeably.	It	would	be	clearer	to	choose	one	

term	and	use	it	consistently:	we	prefer	“reporting	year.”	

 The	proposed	instructions	state	that,	“Reporting	Year	Incremental	costs	should	include	all	

costs	for	the	programs	for	years	prior	to	the	incremental	year	if	these	costs	were	incurred	at	

part	of	the	start	up	of	the	program.”	

o “costs”	should	be	capitalized	to	read	“Costs.”	

o “at”	should	read	“as”	

o As	the	proposed	instructions	currently	read,	it	is	not	clear	whether	or	not	start	up	

costs	are	meant	to	be	re‐reported	each	year	that	the	program	is	in	existence.	Should	

the	utility	report	the	entire	start	up	cost	again	each	year?	Should	it	only	re‐report	a	

certain	portion	that	it	considers	attributable	to	program	activities	that	occurred	in	

the	reporting	year?	

o Please	include	an	example	that	shows	how	to	count	start	up	costs	incurred	in	

previous	years,	while	avoiding	double‐counting	across	years.	

Page	12:	SCHEDULE	6.	PART	B.	DEMAND	RESPONSE	PROGRAMS	

 For	“Demand	Response	Programs”	we	recommend	that	EIA	replace	“dimming	of	lights”	with	

“cycling	HVAC”	(because	it’s	a	more	common	DR	program).	

 Instead	of	“shutting	down	industrial	processes”	we	recommend	“shifting	industrial	

processes.”	

 For	“Potential	peak	demand	savings”	we	are	not	in	agreement	as	to	whether	or	not	EIA	

should	be	collecting	information	on	energy	savings	in	MWh	from	demand	response	

programs.	It	is	not	clear	that	there	are	significant	energy	savings	from	demand	response,	

because	energy	use	is	generally	shifted	to	non‐peak	hours.	(Industrial/production	

processes	are	shifted,	generators	are	less	efficient,	HVAC	is	deferred	momentarily,	etc.)	

 For	“Potential	peak	demand	savings”	EIA	requests	information	on	grid	interactive	water	

heaters.	Please	add	a	link	to	the	DOE	definition	or	provide	a	definition.	Why	just	Water	

Heaters	and	not	HVAC,	pool	pumps,	or	thermostats?	

Page	12:	Point	4,	Weighted	Average	Life	for	Portfolio	(Years)	

 Here,	the	instructions	correctly	use	the	word	“weighted”	rather	than	“weighed.”	However,	

we	still	recommend	use	of	the	term	“Weighted	Average	Measure	Life”	rather	than	

“Weighted	Average	Life.”	This	should	be	standardized	across	the	instructions	and	the	form	

itself.