1

Comments by S. Clayton Palmer

on the

FINAL REPORT OF THE GTMAX MODEL REVIEW PANEL: 

Report	of	a	Workshop	held	August	31	and	September	1,	2011	in	Flagstaff,	Arizona

Draft	of	Tuesday,	September	04,	2012

Background Information

In	recent	years,	the	Glen	Canyon	Adaptive	Management	program	committees,	specifically	the	Glen	

Canyon	Adaptive	Management	Work	Group	(AMWG)	and	the	Technical	Work	Group	(TWG)	have	been	

interested	in	an	expanded	and	more	transparent	program	of	gathering	and analyzing	economic	data	

related	to	the	operation	of	Glen	Canyon	Dam	(GCD).	The	purpose	of	these	committees	is	to	make	

recommendations	to	the	Secretary	of	the	Department	of	Interior	(DOI)	regarding	the	operation	of	this	

dam.	Economic	analysis	is	key	to	making	informed	recommendations.	

In	order	to	begin	the	development	of	this	expanded	economic	program,	the	Grand	Canyon	Monitoring	and	

Research	Center (GCMRC) convened	a	workshop	on	behalf	of	the	TWG	that	included	a	panel	of	

economists	This	workshop	occurred in	December,	2009.	The economic	panel	prepared	a	report	with	a	set	

of	recommendations.	Subsequently,	the	TWG	established	a	subcommittee	to	review	the	

recommendations	of	this	panel	and	make	its	own	recommendations	regarding	implementing	the	panel’s	

report:	the	Socioeconomic	Ad	Hoc	Group	(SEAHG).	

Western	Area	Power	Administration	(Western)	sponsored	the	development	of	a	simulation	model	– the	

GT	Max	– as	a	representation	of	the	production	of	hydropower	in	the	Upper	Colorado	River	System.	The	

model was	developed	by	Argonne	National	Laboratory.	The	model	had	two	purposes: to	assist	Western	in	

making		its	operation	of	the	CRSP	powerplants	more	efficient,	and	to	estimate	the	electrical	power	

production	impacts	of	different	operational	regimes.	For	the	latter	purpose,	the	U.S.	Bureau	of	

Reclamation	has	also	made	use	of	the	GT	Max	model	– specifically	for	estimating	the	impact	of	proposed	

changes	to	the	operation	of	CRSP	powerplants	in	environmental	planning	documents.	

The	purpose	for	my	recitation	of	this	history	is	to	explain	why	the	GCD	adaptive	management	program	has	

taken	an	interest	in	this	model.	At	one	of	its	recent	meetings,	the	AMWG	recommended	that	Western	use	

the	GT	Max	model	for	evaluating	the	electrical	power	system	impacts	of	proposed	changes	in	GCD	

operations.	The	GCMRC,	the	AMWG	and	TWG	became	interested	in	considering	the	suitability	of	the	GT	

Max	model:		is	it	a	suitable	model	– properly	specified	and	sufficiently	robust	– to	address	the	information	

needs	of	the	stakeholders?		

The	purpose	of	the	GCD	AMP’s	interest	in	considering	the	GT	Max	model	for	estimating	the	economic	

impacts	on	the	electrical	power	system	should	be	made	clear.	The	GCD	AMP	program	requires	

information	on	the	economic	impact	on	the	electrical	power	system,	along	with	other	economic	data	and	

analysis,	in	order	to	make	recommendation	on	the	operation	of	the	GCD.	The	SEAHG,	following	up	on	its	

task	to	consider	the	economic	panel’s	report,	prepared	its	recommendations.	These	were	adopted	by	the	

TWG	and	AMWG1.	The	information	required	is	economic	valuation	of	proposed	changes	to	GCD	operation	

– one	component	of	which	is	in	regard	to	the	use	values	related	to	electrical	power.	

																																																												 1 RECOMMENDED INFORMATION NEEDS AND PROGRAM ELEMENTS FOR A PROPOSED AMP 

SOCIOECONOMIC PROGRAM, APPROVED BY AMWG, FEBRUARY 23, 2012
2

The	GT	Max	Panel	Report

As	reflected	by	the	subject	report,	the	GT	Max	review	panel	appears	to have	a	set	of	objectives	that	differ	

from	the	information	needs	of	the	GCD	AMP.	In	addition	to	a	review	of	the	GT	Max	model,	the	panelists	

provide	a	set	of	recommendations	related	to	improving	Western’s	position	in	the	electrical	utility	industry	

[page	30]:

It	is	also	right	and	proper	for	Western	to	investigate	opportunities	presented	by	

its	geography,	that is,	the	capability	to	bid	into	CAISO	markets	and	enter	into	

bilateral	contracts	with	other	entities.	For example,	Western	should	investigate	

energy	arbitrage	between	day-ahead	and	real-time	markets.

In	addition	to	advocating	that	Western	modify	its	position	vis-à-vis	real-time	markets,	the	panelists	

propose	that	Western	participate	further	in	ancillary	service	markets	[page	29]:	

Western	should	evaluate	their	participation	in	ancillary service	markets.

Moreover,	in	various	places	throughout	the	report,	the	authors	suggest	that	Western should	seek	to	

improve	its	profit position,	increasing net	revenues	by bidding its	generation	into	California’s	ancillary	

services	market.

Not	only	do	these	recommendations	seem	to	have	little	or	no	relationship	to	the	goals	of	the	GCD	AMP,	it	

crosses	from	positive	analysis	to	normative	analysis.		An	advocacy	of	a	modified	set	of	operating	policies	

and legal	framework	seems	foreign	from	an	advocacy	of	unfettered	and	objective	scientific	inquiry.	It	

further	indicates	a	panel	that	is	unfamiliar	with	the	legal	framework	in	which	GCD	electrical	power	is	

marketed.	

Another	important	mistake	of	the	panel’s	report	on	the	GT	Max	model	is	this:	the	GT	Max	model	is	

routinely	used	by	Western	to	estimate	the	impact	of	short-term	experiments,	to	provide	an	indication	of	

direction	and	magnitude	of	a	proposed	GCD	operational	change	where	an	impact is	required	quickly	or	to	

evaluate existing	scheduling	practices.	The	presentations	made	by	the	GT	Max	model	creators	and	users	

during	the	workshop	were	geared	to	explaining	these	routine	uses.	As	a	presenter,	I	could	have	explained	

how	the	GT	Max	model	might	be	reconfigured	to	address	the	issues	in	the	panel	report.	After	all,	the	GT	

Max	model	has	been	used	to	model	substantially	larger	electrical	systems,	both	in	geography	and	in	type,	

than	the	CRSP	power	system.	2 Questions	related	to	how	the	GT	Max	model	– a	production	cost	model	–

might	fit	into	a	process	of	a	more	geographically	or	temporally	robust	framework	was	not	discussed	during	

the	workshop.	In	this	response	to	the	panel	report,	I	will	attempt	to	illustrate	an	example	of	how	this	has	

been	done.	

The	GT	Max	Model	and	the	GCD	AMP	Relevant	Questions	

The	relevant	questions	for	the	GCD	AMP,	its	committees,	stakeholders	and	the	GCMRC	is	this:	

• For	developing	economic	analyses	of	the	electrical	power	system,	can	the	GT	Max	be	a	useful	tool	

and/or	how should	the	GT	Max	be	configured	and	calibrated,	what	data	should	be	used	as	inputs	and	

																																																												 2 I	will	go	into	more	detail	about	this	subject	in	subsequent	pages.
3

how	should	the	GTMax	model	outputs	be	evaluated	to	develop	scientifically	adequate	analysis	to	meet	

the	information	needs	of	the	GCD	AMP?			

Economic	use	Values	for	the	Electrical	Power	System

The	determination	of	economic	use	values	for	the	electrical	power	system	for	changed	operations	at	the	

GCD	requires	the	following:	

• The	identification	of	the	relevant	market.	

This	issue appears	to	be	elusive	and	has	been	considered	(or	ignored)	over	several	years.	In	order	to	

determine	the	economic	value	of	a	good	or	service,	the	market	within	which	the	good	or	service	is	

produced	and	consumed	needs	to	be	identified.	In	the	current	context: is	electrical	production	at	GCD	

exchanged	and	consumed	within	a	North	American	market,	within	the	“western	grid”	or	is	production,	

consumption	and	exchange	confined	within	the	CRSP	system	and	the	electrical	systems	to	which	it	is	

directly	tied?	

“Everything	affects	everything	else”	is	an	oft	used	phrase.		Never-the-less,	everything	doesn’t	affect	

everything	else	in	equal	proportions	and	some	effects	are	irrelevant	or	inconsequential.	In	the	December	

2009, economic	panel	report,	the	economists	that	served	on	that	panel	were	also	interested	in	

establishing	economic	values	for	use	in	evaluating	changes	in	operations	at	Glen	Canyon	Dam.	They	were	

well	aware	that an	electrical	“grid”	exists	in	the	Western	United	States.	As	they	noted3:	

[the]	GCD	and	the	CRSP	system	are	embedded	in	the	larger	western	power	grid	(the	WECC).	

Similarly,	the	utilities	to	which	CRSP	sells	power	are	embedded	in	the	WECC.	Therefore,	in	principle,	

the	market	by	reference	to	which	the	economic	value	of	GCD	power	is	determined	is	not the	CRSP	

system	but	the	WECC.	At	any	point	in	time,	it	is	the	marginal	price	of	electricity	in	the	WECC	that	

determines	the	economic	value	of	power	generated	at	GCD.	

However,	the	panelists	also	suspected	that	the	electrical	power	output	of	GCD	is	small relative	to	the	total	

electrical	capacity	in	this	“grid”.		The	panelists	suggested	a	study	to	determine	the	extent	to	which	a	

change	in	the	operation	of	GCD	“spill	over” into	the	WECC	“grid”.	 The	proposal	from	the	panelists	is as	

follows:

Given	the	alternatives,	existing	models	used	by	WAPA	to	optimize	the	operation	of	the	

integrated	system	of	generation	resources	should	be	used	to	determine	if	all	

consequences	of	changed	operations	can	be	managed	within	the	WAPA	marketing	area,	

or	if	electrical	(and	thus	economic)	“spill-over”	effects	will	alter	generation	patterns,	

market	prices	or	transmission	bottlenecks	elsewhere	in	the	WECC	system.	If	the	effects	of	

changed	operations	at	Glen	Canyon	can	be	managed	by	WAPA	without	economically	

significant	changes	in	the	rest	of	the	western	U.S.,	then	the	economic	consequences	of	

such	operations	will	be	limited	to	WAPA’s	customers,	and	the	modeling	effort	limited.

The	logic	is	this;	if	the	electrical	production	at	Glen	Canyon	Dam	is	“small”	relative	to	other	resources	

in	the	western	electrical	grid,	a	change	in	its	operation	will	not	affect	exchange	rates	(prices)	in	the	

																																																												 3 Final	Report	of	the	GCMRC	Socioeconomic	Research	Review	Panel	Report	of	a	Workshop	held	December	2	&	3,	2009	

Phoenix,	Arizona,	Hamilton,	J,	Hanemann,	M.,	Loomis,	J.,	&	Peters, L.,	Grand	Canyon	Monitoring	and	Research	

Center,	February	26,	2010
4

WECC	system.	If	this	is	the	case,	then	the	so	called	“WAPA	models”	that	are	configured	for	the	CRSP	

electrical	system	and	its	contractors	is	sufficient	to	establish	economic	values.

If	GCD,	the	CRSP	electrical	system	and	its	use	in	a	contractor’s	resource	stack	doesn’t	affect	market	

prices,	it	therefore	follows	that	the	electrical	production	at	the	GCD	is	“small”	in	the	economic	sense	

and	WECC market	price	are	exogenously	determined4.	

The	resultant	estimates would	be	the	appropriate	economic	values	provided	that	a	change	in	the	

operation	of	that	resulted	in	a	loss	of	the	electrical	power	resource	at	GCD	would	give	rise	to	the	

need	for	additional	resource	to	come	into	being	to	replace	the	loss.	As	the	panelists	noted:	

.	.	.the	marginal	cost	of	this	extra	capacity	would	count	as	a	real	economic	cost.	It	

would	not	necessarily	be	the	cost	of	additional	capacity	in	CRSP	– it	would	be	the	cost	

of	additional	capacity	anywhere	in	the	WECC	system	to	which	WAPA	and/or	WAPA	

contractors	have	access.

• Dealing	with	Institutional	Constraints:

A	second	aspect	of	establishing	economic	values	is	to	determine	whether	institutional	constraints	distort	

economic	values.	More	specifically,	if	market	prices	are	to	be	used	as	a	representation	of	economic	cost,	

are	there	institutional	constraints	that	cause	market	prices	to	significantly	differ	from	economic	costs?	If	

this	is	the	case,	do	these	“distortions”	cause	a	difference	that	is	so	great	as	to	cause	differences	in	the	

magnitude	of	economic	impact	of	ordering	of	the	impact	of	various	alternative	GCD	operational	regimes?		

If	so,	what	modifications	to	the	GT	Max	model,	to	the	input	data	or	to	the	analysis	of	model	output	should	

occur?	

For	clarity,	institutional	constraints	differ	from	limitations	on	infrastructure.	The	legal	mandate	that	

requires	that	Western	market	the	GCD	electrical	output	to	the	“preference”	customers	rather	than	to	“the	

market”	is	independent	of	whether	unencumbered	transmission	systems	exist	such	that	Western	could	

sell	electricity	to	buyers	in	Northern	California.	

There	are	two	reasons	why	institutional	constraints	may	not	fatally	encumber	the	economic	evaluation	

that	would	be	needed	the	GCD	AMP	for	appropriate	decision	making.	In	the	EIS	prepared	by	the	DOI	and	

Reclamation	on	the	operation	of	the	GCD	in	19955,	two	estimates	are	made	on	the	economic	impact	to	

electrical	power	for	each	alternative.	One	approach	(CROD)	assumes	the	continuation	of	existing	federal	

contracts	for	electricity.	The	other	approach	(Hydro)	assumes	that	GCD	power	is	allowed	to	be	sold	to	the	

WECC	market	and	is	dispatch	for	“peak	shaving”.		In	essence,	the	Hydro	approach	disregards	existing	legal	

requirements	for	the	sale	of	electrical	power.	The	results	showed	that,	while	the	magnitude	of	the	

economic	impact	differed	(the	Hydro	approach	showed	a	lesser	impact),	the	differences	between	the	

																																																												 4 “Everything	affects	everything	else”.	So,	a	determination	that	a	change	in	the	operation	at	GCD	doesn’t	affect	

market	prices	in	the	WECC	would	have	to	be	a	determination	made	based	on	statistical	significance.	There	is	also	the	

possibility	that	WECC	market	prices	could	be	statistically	significantly	different,	given	a	change	in	GCD	operations,	but	

not	consequential	and/or	that	the	added	precision	is	not	worth	the	added	expense.	I	will	have	more	to	say	on	this	in	

due	course.	

5 Operation	of	Glen	Canyon	Dam,	Final	Environmental	Impact	Statement,	U.S.	Bureau	of	Reclamation	and	

Department	of	Interior,	November,	1995
5

CROD	approach	and	the	Hydro	approach	were	proportional	and	maintained	the	relative	ranking	of	the	

alternatives	in	terms	of	impact.	

If	this	example	is	illustrative	of	the	effect	of	institutional	constraints	on	economic	impact	analysis,	it	may	

be	that	market	prices,	even	in	light	of	institutional	and	legal	constrains,	within	the context	of	GCD	AMP	

decision	making,	may	serve	to	estimate	the	economic	value	of	proposed	changes	to	GCD	operation.	

Another	important	consideration	regarding	institutional	and	legal	constraints	is	this;	the	GCD	AMP	stake	

holders	are	often	interested	in	the	distribution of	economic	impacts.	For	example,	the	SEAHG	report	(cited	

above)	describes	an	information	need	related	to	the	economic	impact	of	modified	GCD	operations	on	

Native	American	Tribes.	

For	distributional	impacts	to	be	considered,	it	is	important	to	identify	and	have	a	proper	specification	of	

institutional	constraints	and	of	the	legal	framework	in	which	federal	power	from	the	GCD	is	marketed.	For	

example,	federal	power	from	the	GCD	power	plant	is	required,	under	the	law,	to	be	sold	as	firm	electrical	

power	and	energy,	preferentially,	to	certain	utilities,	state	and	federal	facilities,	irrigation	districts	and	

Native	American	tribes.	

The	GT	Max	model	in	consort	with	capacity	expansion	and other	models

If	a	change	in	operation	at	the	GCD	gives	rise to a	consequential	change	in	electrical	exchange	and/or	

prices	within	the	WECC	area,	that	would	indicate	that	the	relevant	market	for	GCD	electrical	power	is	at	

least	the	“western	grid”.	A different	modeling	process	would	be	needed	to	estimate	economic	values, one	

that	includes	the	electrical	resources	in	the	WECC	region.	This,	more	robust	modeling	would	be	necessary	

to	estimate	economic	values.			

Figure	1	below,	is	an	illustration	of	a	modeling process	for	a	more	robust	and	geographically	more	

complete	modeling	process	6.		This	is	a	flow	chart	that	illustrates	a	modeling	process	undertaken	by	

Western	for	an	environmental	impact	statement	on	one	of	its	marketing	programs7.	This	process	includes	

the	several	models:	one	that	determines	hydrological	conditions	and	GCD	release	volumes	on	a	monthly	

basis	(Hydro	Forecast/Condition)8,	a	production	model	(Hydro	Dam	Operational	Restrictions),	a	capacity	

expansion	model	(Utility	Supply-Side	Expansion) and	a dispatch	model	(Utility	Dispatch).		This	flow	chart	

illustrates	how	several	models	– including	a	models	that	deal	with	capacity	expansion	and	production	cost,	

have	worked	together	to	achieve	an	appropriate	estimate	of	economic	values.	

When	the	GT	Max	model	– a	production	cost	model	– is	included	in	a	modeling	effort	that	includes	a	

capacity	expansion	model,	the	lack	of	a	capacity	expansion	algorithm	described	in	the	panel	report	(page	

19) becomes	a	mute	point.	The	panel	report	states:

The	GTMax	model	and	Western’s	analytical	framework	is	designed	primarily	to	

evaluate	short-term operations.	The	authors	of	GTMax	excluded	long-term	planning	

features,	like	long-term	capacity expansion	capability.	

																																																												 6 Salt	Lake	City	Area	Integrated	Projects	Electric	Power	Marketing	Environmental	Impact	Statement,	Volume	3:	

Appendix	A,	U.S.	Department	of	Energy,	Western	Area	Power	Administration,	February,	1994

7 The	EIS	was	prepared	by	Argonne	National	Laboratory	for	Western.	It	was	the	Post	1989	Marketing	and	Allocation	

Program.

8 This	model	(and	others	in	this	flow	chart)	were	replace	with	the	GT	Max	model.	Therefore,	in	describing	this	flow	

chart,	I	use	the	term	production	cost	model.	
6

This	statement	indicates	a	lack	of	understanding.	The	GT	Max	model	is	production	cost	model	which	can	

be,	and	is	often,	configured	for	uses	beyond	estimating	short-term	financial	costs	in	a	closed	electrical	

system.	It	can	be	sufficiently	detailed	so	that	it	can	estimate	power	production	on	an	hourly	basis,	

dynamically,	from	a	set	of	three	conjoined	dams	in	Colorado	(the	Aspinall	Units)	and,	when	combined	with	

other	models		can	serve	to	estimate	long-run	marginal	cost	within	a	geographically	large	and	complex	set	

of	electrical	systems9.	

A	production	cost	model	(as	shown	in	this	flow	chart) is	also	useful	for	another	reason.	In	addition	to	

economic	values,	this	model,	in	addition	to	other	considerations,	such	as	Western	contracts	and	

commitment	levels,	allows for	estimates	of	how	economic	impacts	are	distributed	to	various	contractors.	

The	distribution	of	economic	effects	is	often	of	interest	to	stakeholders	and	decision	makers	and	would	be	

less	precisely	determined	if	a	capacity	expansion	model	or	WECC-wide	model	worked	alone.	

Estimates	of	economic	values	related	to	the	change	in	operation	at	the	GCD	was	prepared	for	the	

Bureau	of	Reclamation	(Reclamation)	for	the	EIS	on	the	operation	of	GCD	published	in	199510.		At	the	

time	the	analysis	was	done	for	this	EIS,	electrical	capacity	in	the	WECC	region	was	in	surplus. In	

addition,	the	EIS	required	a	20-year	evaluation	of	the	affected	resources.	The	consulting	firm	of	Stone	

and	Webster,	in	consultation	with	the	cooperating	agencies,	prepared	economic	impact	analysis	for	

the	EIS	alternatives	– all	of	which	were	changes	in	the	operation	of	GCD.	

																																																												 9 In	the	SLCA/IP	Post	89	Marketing	EIS,	a	forerunner	of	the	GT	Max	model	was	used	within	a	set	of	interactive	models	

to	analyze	15	electrical	systems	in	the	Rocky	Mountain	West	along	with	the	CRSP	power	system.	

10 Operation	of	Glen	Canyon	Dam,	Final	Environmental	Impact	Statement,	U.S.	Department	of	the	Interior, Bureau	of	

Reclamation,	March,	1995.
7
Comments by Tom Veselka and Les Poch

on the

FINAL REPORT OF THE GTMAX MODEL REVIEW PANEL: 

Report of a Workshop held August 31 and September 1, 2011 in Flagstaff, Arizona

Draft of Tuesday, September 04, 2012

Argonne National Laboratory (Argonne) staff would like to thank the Grand Canyon Monitoring and 

Research Center (GCRMC) for hosting the “GTMax Model Review and Knowledge Assessment for 

Hydropower” workshop conducted in Flagstaff, Arizona on August 31 and September 1, 2011.

Participants included staff from GCRMC, the Western Area Power Administration (Western), Argonne, 

members of the review panel, and other interested individuals. 

The comments below are made by Les Poch and Tom Veselka from Argonne who presented GTMax 

materials at the workshop. Comments are mainly in response to information contained in the document 

entitled “Final Report of the GTMax Model Review Panel” that Western management received for 

comment on October 10, 2012. Note the review document was distributed to presenters more than 13 

months after the conclusion of the workshop. Therefore, instead of relying on our recollection of what 

transpired during the workshop, the comments below are primarily based on the review document and 

PowerPoint presentations given at the workshop.

Reviewer comments and suggestions address GTMax general modeling frameworks and methodologies 

as well as more specific aspects of hydropower modeling. Below we remark on several facets of 

reviewer comments, add background information about modeling, and provide justification for the 

GTMax structure, scope, and methodology.

Production Cost Modeling and Mixed Integer Linear Problem Formulation

In the 1st paragraph of the executive summary the review document states “GTMax belongs to a class 

of models known as production simulation models. The model’s primary objective is to simulate the most 

efficient operation of the Colorado River Storage Project (CRSP) hydroelectric power facilities at least 

cost. As is typical of such models, it simplifies what is in reality a very complex system of interrelated 

electrical generation facilities and decision factors, and therefore, like all such models, it is an imperfect 

representation of reality and has limitations in how it can be applied.” 

In general, we agree with the above reviewer comments as individual isolated statements. As modelers 

with a combined working experience of almost 70 years in power system modeling and analysis, we

recognize that no model is perfect. Furthermore, we acknowledge that proper model application and 

interpretation of results are of utmost importance. However, we would like to clarify that GTMax is 

more accurately classified as a production cost optimization model as opposed to a simulation model.

Also the objective of GTMax as it is applied to the CRSP system is to maximize the economic value of 

federal power resources using market price signals, not to minimize hydropower production costs as 

stated by the review panel. In the main body of the final report, reviewers state that “If purchases are 

less expensive than dispatching a Western unit, the model will displace the unit.” This is an inaccurate 

portrayal of GTMax and the dispatch of hydropower resources in general. CRSP marginal production 

costs are miniscule – there are no fuel costs and variable operating and maintenance (O&M) costs are 

for all practical purposes either zero or very small. Therefore, CRSP hydropower production costs are 
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

2

almost always less than market prices. Instead of using a production cost minimization objective, GTMax 

schedules limited water resources to maximize hydropower water resources. 

Readers of the external review document should also be aware that, although not perfect, production 

costs models are extremely useful. Readers should also realize that the optimization techniques that 

GTMax utilizes have been extensively used by other models in the past and continue to be widely 

applied by utility companies and academia throughout the world. 

The GTMax model is comprised of a large set of equations that are formulated as a network problem 

which describe the operation of one or more interdependent systems. In addition, it includes equations 

that represent physical processes, institutional limitations, and time sensitive scheduling problems. 

Although the GTMax model possesses several aspects of a network flow optimization problem, there are 

some fundamental differences from a pure network problem. Unlike traditional network flow models, 

GTMax contains more complex features. Properties of a resource may be altered as it flows through a 

network and/or over time and there are numerous temporal constraints, which are more common in 

sequencing and scheduling problems. 

Core mathematical relationships in GTMax require that equations are linear functions. In addition some 

variables are declared as “integers”; that is, solutions for these variables are required to be 

discontinuous integer number values. Solutions for other variables, referred to as “real variables” in the 

mathematical problem, are allowed to be continuous real number values. These two requirements place 

GTMax core functions into a classification that is commonly referred to as a “mixed-integer linear 

program (MILP)” problem. The solver that GTMax uses has been shown to solve real-life problems such 

as the ones required to accurately optimize Colorado River Storage Project (CRSP) operations with tens 

of thousands of variables and constraints. 

Water and hydropower systems are inherently nonlinear and discontinuous, seriously challenging the 

limitations of leading optimization software packages and the computational capabilities of our most 

advanced computer systems. While a Mixed Integer Non-Linear Program (MINLP) can most accurately 

represent the problems modeled by GTMax, the current state of MINLP technology limits its use to tiny 

models only. GTMax problems typically require thousands of variables and constraints. Considering the 

complex structure of GTMax and the large problem sizes it solves, we use a general-purpose MILP solver 

to find the solution.

Linear Programming (LP) techniques in electricity systems modeling have been widely used by 

academics and electric utility companies since the 1950’s. One study stated that the MILP approach 

shows the greatest potential for addressing the unit commitment problem for power systems that have 

combined-cycle generating units (Kinetrics, Inc. 2005). LP and MILP problem formulations have been 

used to determine Security Constrained Unit Commitment (SCUC) and Security Constrained Economic 

Dispatch (SCED) for the largest power grids in the world. Some electricity market simulation models 

using LP and/or MILP include AURORAxmp, PLEXOS, GridView, PROMOD, GE MAPS, and GenTrader.

AURORAxmp was developed by EPIS, Inc., in 1997 to model competitive wholesale electricity markets. It 

has been licensed to over 60 users such as electric utilities, independent power producers (IPPs), 

government agencies and regulators, electricity traders, hedge funds, and energy industry consultants in 

North America, Europe and Asia.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

3

PLEXOS was developed by Energy Exemplar to model electricity markets. By the beginning of 2012 

PLEXOS was installed at over 135 sites worldwide. It is currently being used by National Renewable 

Energy Laboratory (NREL) to study the impact of a proposed energy imbalance market (EIM) in the

Western Interconnect (WI).

GridView was developed by ABB and is a security-constrained unit commitment and economic dispatch 

model. It has been used by the California Independent System Operator (ISO), Southwest Power Pool, 

the New York ISO, NREL, and others. It was used to perform a cost/benefit analysis of a proposed EIM 

for WECC in 2011.

PROMOD was developed by Venytx and has been used by energy companies for 30 years. It is currently 

being used by WECC to perform integrated energy and capacity analyses.

The Multi-Area Production Simulation Software (MAPS) is a chronological simulation model developed 

by GE. It has been used by numerous transmission companies, PJM, the New York ISO, the New England 

ISO, and the Midwest ISO to perform reliability studies, and transmission and generator expansion 

studies.

GenTrader was developed by PCI in 1999. It has been used by utilities such as South Carolina Electric & 

Gas, Mirant, Kansas City Power and Light, and utilities in the deregulated Texas market such as Reliant 

Energy, TXU, and Calpine Corp.

Last, but not least, GTMax and its predecessor, the Hydro LP model, have been used for approximately 

20 years. When it was first developed no production cost model was available to adequately address 

hydropower issues at a level of granularity that Western needed to conduct analyses in support of a 

Power Marketing Environmental Impact Statement (EIS). Western funded Argonne to create a model 

that mimics its core business function – the dispatch of the Colorado River Storage Project (CRSP). The 

basic design however is very flexible and can also be used to optimize operations of other systems. 

Consequently GTMax projects have been sponsored by the Bureau of Reclamation (Reclamation),

International Atomic Energy Agency (IAEA), USAID, World Bank, Fichtner GmbH & Co KG (Germany), 

Adica Consulting, ENRON, and Chubu Electric Power Company (Japan). These studies have been applied 

to hydropower-centric problems in the United States, Asia, Africa, and several regions in Europe.

The review panel concluded that among production cost models GTMax is exceptional in its 

representation of hydroelectric systems. We are pleased that reviewers gave GTMax high marks for the 

purpose it was designed; that is, optimizing the economic value of hydropower plants under a large 

range of operating criteria and constraints. Determining economic value is a core task that is performed

under EISs that examine operational changes in electric power systems. 

GTMax has a hydropower-centric design and is tailored to perform specific tasks. It focuses on what is 

important for problems that are of interest to Western and CRSP stakeholders. This design helps it 

overcome some of the shortcomings of dispatch models that are designed for more general and broader 

applications such as imperfect modeling of “inter-related electricity faculties” and “decision factors” that 

review committees members listed in the 1st paragraph of the executive summary.

Not only does GTMax better dispatch hydropower plants than the vast majority of production cost 

models, it also better reflects actual “decision factors” regarding water operating criteria as it respects 

dispatch guidelines and goals that are specific to the CRSP system. It also addresses “inter-related 
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

4

electricity facilities.” The GTMax CRSP topology links CRSP power facilities together as an interconnected 

system and explicitly simulates the connectivity of water resources, the flow of water throughout the 

systems, and water related impacts on power resources. In addition to water resources, GTMax

simultaneously schedules Western’s federal hydropower resources and engages in short-term firm 

bilateral power market transactions and spot market activities to meet its obligations to serve firm 

loads. 

The review panel recognized that GTMax is able to account for and simulate the effects of these 

relationships among CRSP facilities. However, reviewers stated “Glen Canyon Dam is just one component 

of the CRSP system, and decisions made at one facility affect all the others, so there are significant 

challenges associated with extracting effects of Glen Canyon Dam operations from the rest of the system 

using this model.” Reviewers failed to mention that Western and Argonne staff devised an elegant and 

effective methodology for isolating and assigning these effects to Glen Canyon Dam. The technique was 

described in the workshop presentation titled, Methods for Representing CRSP Resources.

Reducing Operating Objectives and Constraints to a Single Number

GTMax determines values for “decision variables” that maximize a given function, called the “objectivefunction,” while satisfying given inequalities of other functions called “constraint-functions.” The 

executive summary of the review panel document discusses the “single number” that the model 

produces. This number is essentially the GTMax objective function value. For CRSP applications, the 

objective function maximizes the economic value of CRSP energy resources over a user-specified time 

period. This “bottom-line” economic value is comprised of a multitude of additive components. Readers 

not knowledgeable in LP and MILP modeling should realize that the objective function in large problems

such as those posed by the CRSP system is dependent on numerous variables. 

The GTMax model determines values of decision variables for all chronological hourly time steps during 

the model timeline. This timeline typically spans a one-week period. This time chronology feature of 

GTMax allows it to simulate a host of operational aspects, such as generator ramp-rate limitations, unit 

commitment schedules, thermal unit starts and stops, and multi-time-period constraints, including 

maximum and minimum water releases from a reservoir over the simulated period. When optimizing, 

the model recognizes that the solved state of the system at any point in time affects operations at all 

other points during the optimized time period. Simulated time is also critical for solving network 

interdependency problems, such as cascaded reservoirs, because GTMax recognizes reservoir 

connectivity and the time it takes water to flow through a system.

Therefore, in addition to producing a value for the objective function, GTMax also solves for and 

produces literally thousands of other results in a typical CRSP optimization run. Some key model outputs 

include hourly generation levels for each hydropower plant, power and non-power reservoir water 

releases, reservoir storage volumes and associated forebay elevations, downstream water flow rates, 

river stage, and marginal values for both water and energy throughout the interconnected network. 

Therefore, GTMax should not be viewed as a “black-box” model that produces a single inexplicable

number. Instead each component of the objective function value is transparent, explainable, and 

traceable through an analysis of model outputs, constraints, scalars, and equation constants and 

coefficients. 

Workshop Purpose and Proposed Markets

One section of the review panel report describes the purpose of the workshop. We agree with the 

review panel’s first sentence that states “The purpose of this workshop was to review the capabilities 
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

5

and uses of the GTMax model toward the investigation of possible changes in the operating regime of 

the Glen Canyon Dam.” However, in the text that follows we find a disconnect between what we were 

asked to present by GCRMC organizers and several of the statements made by the review panel. The 

review panel report discusses “proposed and future uses of GTMax and assess the applicability of the 

model to anticipated future economic and financial issues.” Reviewers then briefly describe how the 

power sector is evolving and different types of power sector business models. Then in the sections that 

follow reviewers are critical of GTMax in its present form since it may be ill-equipped to model potential 

changes in the Western Interconnect (WI) such as evolving market structures and future supply resource 

mix – in particular, an increase in variable resources. 

Regarding the applicability of GTMax for future studies we would like to point out that GTMax is not a 

static model. Instead, it has been extensively modified and enhanced to meet the evolving needs of 

Western to reflect changes in reservoir operating criteria and alterations in Western’s marketing goals 

and objectives. In our opinion it is premature for Western to expend some of its limited resources to 

either upgrade GTMax or develop new models to better represent potential market structures and 

business models that are not yet applicable to the CRSP system. Reviewers specifically refer to the 

proposed WI Energy Imbalance Market (EIM). At the time the workshop was conducted EIM market 

rules and structures were under development. Although more than16 months have passed, EIM rules 

have not yet been fully defined and agreed upon. Basic economic analyses continue to be conducted 

and it is not certain if the envisioned structure of the WI-wide EIM will materialize in the foreseeable 

future. Western has generously funded GTMax upgrades on an as-needed basis to meet specific needs 

or in support of well-defined projects. It is anticipated that future upgrades will be made as deemed 

necessary by Western or in support of GTMax projects funded by other organizations.

Argonne and Western staff are fully aware of changes in the WI markets and are engaged in EIM related 

activities via participation in webinars, workshops, internal meetings, and through extensive reviews of 

EIM reports and presentations. In addition, at the request of Western, Argonne has performed an 

extensive analysis of EIM models, methods, and results to help Western make more informed EIM 

decisions (Veselka et al. 2012); the report is available to the public at the Department of Energy (DOE) 

Information Bridge web site at http://www.osti.gov/bridge/.

In addition, several years ago Argonne developed the Electricity Market Complex Adaptive System 

(EMCAS) to model the behavior of participants in electricity markets under a well-defined set of market 

rules (DIS 2007). The model has been applied to studies in both the U.S. and Europe. Through this work 

and our extensive study of both domestic and international markets we disagree with reviewer 

statements that suggest that “the fundamental economic behavior driven by cost minimization provides 

a common foundation that underlies the behavior of market participants in both market and non-market 

segments of the industry.” In contrast we find that in a competitive marketplace, market players acting 

as autonomous agents, are driven by corporate objectives such as profit maximization and/or to 

increase market share. This is in stark contrast to traditional markets in which the cost-minimization

objective is the primary force behind utility behavior. In a perfectly competitive market in which there 

are no barriers for new market entrance, the end result in theory may be cost minimization. However, 

no such market exists – especially in the utility sector in which market players under some conditions

can influence market clearing prices. Later in the review document it states “Formal electricity markets 

sharpen the profit incentive of all participants ....” The statement appears to conflict with prior 

statements since there are stark differences between cost minimization and profit maximization 

objectives.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

6

If we had been informed that the workshop would focus more on how Western may model future 

markets and conditions, the content and focus of workshop presentations would have been different. 

We would have discussed market issues and ways in which EMCAS algorithms could potentially be 

leveraged to better represent anticipated changes in the WI and Western’s response to these changes.

However, it is premature to develop a model for a voluntary EIM that is not yet fully defined and may 

never be implemented. 

Review panelists also suggest that Western engage in marketplace activities that overstep its statutory 

boundaries and in some instances are physically impossible. Some of the statements include 

“opportunities for western generation asset owners to increase returns on operation of their plants”,

“the potential to expand the opportunities for profitable operation”, “bid into California markets when 

either energy or ancillary services prices make it financially attractive”, and engage in day-ahead and 

real-time market “arbitrage.” Western is a U.S. government agency that schedulers federal resources; it 

is not a publicly-traded company that is obligated to maximize shareholder value. Instead its primary

mission is to maximize the economic value of federal resources. Ultimately these federal facilities belong 

to the citizens of the U.S. Western must abide by the Colorado River Storage Project Act, its statutory 

and delivery obligations, and WAPA’s prevailing contractual delivery obligations to existing customers.

CRSP resources are already fully dedicated to existing contractual obligations, regardless of specific 

Colorado River Basin hydrological condition. This includes Western’s statutory obligation to market 

federal project hydropower at the lowest possible rates to consumers consistent with sound business 

practices. In our view, Western has neither the resource flexibility nor the discretion to pursue reviewer 

suggestions.

Ancillary Services and Higher Penetration of Variable Resources

Regarding comments concerning the model’s treatment of ancillary services we find that reviewers were 

not adequately informed about or did not fully grasp the model’s capabilities. The reviewers state that

“GTMax de-rates hydroelectric generation capability to capture the cost to provide for these services.” 

This statement is correct as it applies to the treatment of ancillary services in the CRSP system. We 

utilized this methodology for CRSP because it adequately captures the impacts of providing ancillary 

services on hydropower plants operations and system economics. However, reviewers also state that 

“GTMax fails to make commitments for security reserves, even at the hourly time step” and later in the 

text “the absence of ancillary service valuation capacity is conspicuous.” These statements are incorrect.

Not only can GTMax explicitly model ancillary services, it also contains options to represent spinning 

reserves on an hourly basis as spare generating capacity. Units that provide spinning reserve services 

must be synchronized to the grid (i.e., are in generation mode in GTMax). The regulation service is used 

to compensate for (or react to) very short-term (in the range of seconds) changes in the grid through the 

use of unit automatic generation controls (AGCs). A powerplant providing this service must also be in 

generation mode in GTMax. 

In GTMax units that supply ancillary services affect the range and flexibility of operation to serve load 

and/or for energy sales. Regulation up and spinning reserves reduces maximum energy schedules. That 

is, the sum of those two ancillary services plus scheduled energy production at all times must be less 

than or equal to the maximum operational capacity of the units. For hydropower operations GTMax also 

ensures that hydropower plants release enough water from reservoirs to accommodate the sum of the 

minimum flow rate requirement plus regulation services down. The GTMax user sets ancillary service 

limits by unit/plant and the model then optimizes ancillary service assignments to meet total system 

and/or subsystem requirements. We find that other production cost models use a similar approach.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

7

Although the current GTMax treatment of ancillary services is adequate under current conditions, the 

situation may change in the future. The panel review states “With Western and the WECC entering into 

a period of increasing generation from renewables and therefore also possibly the need for ancillary 

services....” We agree that there is a trend toward more renewables (more specifically variable 

resources) in the WI. However, it should be noted that Western’s current ancillary service obligations 

are based on its participation in a reserve sharing group and it will sell additional services through 

bilateral agreements. Also, Western will not construct any variable resources in the foreseeable future 

and at this point in time there are no plans for forming ancillary services markets in the larger WI

footprint. Ancillary services markets are outside the realm of the proposed WI EIM. It is prudent for 

Western to wait and gain more information about how the system will evolve and about future 

institutional arrangements before making modifications to either GTMax or adopting a new model. 

Regarding ancillary services, reviewers also state “GTMax cannot simulate operational periods of less 

than one hour.” We acknowledge that it cannot solve for movements at sub-hourly time intervals such 

as instantaneous variations in load, continuous changes in generation as a unit ramps between set 

points, and second-by-second fluctuations in production as units respond to AGC signals. Improvements 

can be made in this area. To increase and more effectively utilize hydropower in the U.S. the 

Department of Energy (through Western) entered into a memorandum of understanding with the 

Department of Interior (through Reclamation) and the Department of the Army (through the Army Corp 

of Engineers). This partnership is assisting a team of specialists from 4 DOE laboratories to demonstrate 

a newly developed Water Use Optimization Toolset (WUOT) at the CRSP system. One component of 

these toolset models is a very detailed, high fidelity hydropower day-ahead scheduling and real-time 

dispatch tool that operates at user-specified time steps. At present it is designed to model operations at 

time steps as short as 5 minutes. Another tool deals with inflow water forecasts, longer-term reservoir

water routing, and the environment.

The WUOT is also being demonstrated at the Oroville-Thermalito Complex which is in the California ISO 

and at the Conowingo/Muddy Run pumped storage facility located in the PJM footprint. Western’s 

participation in the WUOT project requires minimal expense and will help prepare for its next 

generation modeling needs. Application of the day-ahead scheduling and real-time dispatch tool to the 

CRSP system is ongoing. Preliminary results have been documented and sent to DOE for review.

Modeling Scope and Fidelity

Reviewers suggested that the scope of GTMax be expanded in time and space with a more refined level 

of granularity and fidelity. To some degree we appreciate this broader perspective and the notion that in 

principle, everything in this universe and beyond is connected. Sir Arthur Eddington wrote, “You cannot 

disturb the tiniest petal of a flower without the troubling of a distant star.” Unfortunately given our 

present state of technology and knowledge, there are practical limits to achieving this goal. No single

model can do everything and answer every question perfectly with a high degree of fidelity. Therefore, a 

crucial part of any modeling task is to determine what is important and what is not to answer a specific 

question within the scope and budget allocated to a project. We chose to concentrate on the central 

problem when applying GTMax. This strategy has been very successful and useful. For example, 

schedulers in the CRSP Energy Management and Marketing Office located in Montrose, Colorado review 

GTMax model results each month and use model outputs to aid in their decision making. This is a strong 

testimony that the model accurately represents of the dispatch of the CRSP system and provides both 

realistic and meaningful results.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

8

Our approach has been and continues to be to improve the water management and power dispatch 

functions in GTMax, rather than to expand its scope. As a model grows in scope, the problem size often 

becomes overwhelming and forces modelers to make compromises in terms of spatial and temporal 

fidelity. This tactic could potentially compromise the current integrity of the model in terms of its ability 

to produce realistic CRSP dispatch results. Furthermore, Western has no legal authority to build new 

capacity, such as wind and solar, or to serve load growth. 

To investigate some of the broader issues we leverage a set of specialized tools to feed pertinent 

information into GTMax. Some of the toolset models (and/or model outputs) that we have used in 

conjunction with GTMax in the past include: (1) WASP - capacity expansion results provide GTMax with 

new thermal and hydropower plant additions; (2) Aurora model - results supply GTMax with estimates 

of future locational marginal prices (LMPs); (3) Riverware - provides GTMax with estimates of future 

monthly water releases and reservoir storage levels; and, (4) VALORAGUA - provides GTMax with 

hydrological information in a multi-country region. This approach allows GTMax to focus solely on 

weekly CRSP operations based on a set of operating limits, goals, and guidelines to evaluate operations 

in terms of (1) the economics/financial value of energy; and, (2) plant-level “operating” capacity while 

leveraging results and insights produced by other models to help guide the hourly dispatch from a 

longer term and broader perspective. 

In our opinion, using the toolset approach is a practical and reasonable methodology for conducting 

studies within the confines of project resources. However, the review panel report states “The reviewers 

had concerns, however, with the model’s application for other purposes for which it was not originally 

designed and is not well-suited. Specifically, the model is not well-suited for forecasting economic 

implications of long-term operational scenarios.” We disagree with the reviewers on this point. Without 

challenge from numerous affected parties over the last 20 years, GTMax and GTMax-light have been 

used for this exact purpose. This encompasses numerous studies in support of EISs, and both economic 

and financial analyzes of CRSP facilities. GTMax performs an economic dispatch under a very specific set 

of conditions. This set of conditions may be for today, next week, a year from now, or 50 years into the 

future. Basic dispatch principles are not expected to change; however, model inputs as guided by 

specialized long-term tools will vary during the course of a long study time horizon. For example, for 

long-range studies, GTMax relies on results from the Riverware model for future estimates of reservoir 

water releases and reservoir storage volumes. This allows GTMax to alter its dispatch in response to 

anticipated changes in hydrological conditions over time. To the extent that the Riverware run has 

incorporated the effects of global warming into its projections, these will also be reflected in GTMax 

model results. In this context, GTMax is one piece a model toolset; it is not the sole analytical tool.

Reviewers are concerned about the applicability of GTMax for long-term studies because it does not 

expand system capacity. The review document states “There are other reasons for having capacity 

expansion capability in a simulation model. Without such capability, simulated market prices for 

wholesale firm energy would reach unsustainable levels ....” We agree that new capacity will be required 

in the future to maintain reasonable prices. However, embedding MILP equations in a production cost 

model to expand capacity is not the only approach or always the best method of estimating future WI 

prices. Reviewers point to research that “argues that the perfect information assumption of production

simulation models and other simplifications make them a poor predictor of electricity market prices.” For 

CRSP applications, we typically project prices based on futures market indices. As required we can 

extend forecasts using Energy Information Administration electricity prices projected by the National 

Energy Modeling System (NEMS) through 2040. This model has a very broad national perspective that 
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

9

goes beyond the WI and the electric sector. We have also used projections made by other models such 

as Aurora.

We acknowledge that using a fixed price vector has a conceptual weakness since market prices do not 

react to actions taken at Glen Canyon Dam. One approach that we used to model hydropower systems 

in the Iberian Peninsula is to use a market price curve instead of a static value because alternative 

actions often resulted in significant changes in system marginal production costs. However, changes in 

Glen Canyon Dam operating criteria affects only 200 MW to 400 MW of capacity; that is only 0.1 % to 

0.2% of the total WI capacity. Therefore, in past model applications we assumed that capacity swings of 

this magnitude would have only miniscule impacts on market clearing prices. We based this assumption 

not only on the small change in generation swings, but the fact that the WI marginal fuel is 

predominately natural gas. Nevertheless, Argonne is conducting a simplified WI analysis to further study 

the validity of this assumption. 

It should be noted that GTMax has the capability to compute LMPs instead of using exogenous market 

prices. This modeling option has been exercised in GTMax applications in Europe, Asia and Africa. For 

long-term applications, we always incorporated exogenous capacity expansion schedules into GTMax 

topologies. Also, in each of these studies there were adequate data available and sufficient support from 

utility system experts to perform realistic price calculations.

From comments in the review panel report, it appeared reviewers did not fully understand and

appreciate the approximation methods that were used for evaluating capacity costs when time and 

money do not allow for a more thorough analysis. One of these approximation methods was used for 

two Reclamation EISs and approved by Reclamation, Western, and Western customer representatives. 

During the Power Marketing EIS, which was led by Western, and the Glen Canyon Dam EIS, which was 

led by Reclamation, a more rigorous approach was implemented. Modelers and analysts prefer a more 

detailed approach, but it is also very expensive and time consuming. In these studies capacity expansion 

models were run for each of Western’s largest customers for each EIS alternative. Experts from each of 

the utilities modeled provided advice and supplied detailed information about systems operations, 

objectives, and the slate of candidate expansion technologies. 

It is our impression that reviewers would like to employ an even larger and broader perspective by 

estimating the capacity expansion impacts on the Western Interconnect (WI). Such a model may 

generate numbers, but it is important to recognize when model “answers” are meaningful. We contend 

that capacity expansion for the entire WI is based on judgments made by a multitude of independent 

decision makers. Each has unique objectives, operating conditions, and financial status. Coupled with 

federal and state laws, renewable portfolio standards (RPS), wind and solar integrations, uncertainty, 

unique system-level hydropower/water conditions and regulations, national policies etc. makes 

modeling realistic capacity expansions on a WI level extremely difficult.

Connection issues in the WI are extremely complex and interwoven which makes some of the broader 

issues beyond the mission and capacity of Western. For example, reviewers suggested that GTMax 

should quantity the grid benefits of ancillary services. This one task by itself is an enormous problem. 

Over the last several years, DOE and other organizations such as EPRI have sunk many millions of dollars 

investigating the grid value of hydropower. Argonne staff members have participated in several 

hydropower grid services meetings and have reviewed project documents. More recently DOE has 

funded Argonne to lead a team comprised of national laboratory experts, prominent hydropower 

consultants, software developers, and utility experts to perform further modeling and analysis on this 
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

10

topic. This project is in addition to the WUOT mentioned earlier. Similar massive studies, years in the 

making, investigate WI wind and solar integration, energy markets, the system-wide level 

representation of hydropower, global warming effects, etc. These investigations are ongoing and will 

continue into the foreseeable future. To the extent possible, it may be possible to leverage information 

and/or products generated by these projects. 

These WI power issues and very long-term trends such as climate change are of great interest, but in our 

opinion the marginal value of incorporating these factors into CRSP studies for evaluating the power 

economics of Glen Canyon Dam under alternative operating criteria are very small, exorbitantly 

expensive, and the confidence level of the model results would be low. We base this option on the 

following rationale. 

(1) Alternative operating criteria imposed at Glen Canyon Dam has very little or no impact on the 

annual amount of power it produces. In an average hydrological year this accounts for less than 

0.6% of WI generation and will progressively decrease in percentage terms over time.

(2) Alternative operating criteria typically increases or decreases Glen Canyon Dam operating 

capacity by only 200 to 400 MW. This represents only 0.1% to 0.2% of WI generating capacity 

and it will progressively decrease in percentage terms over time.

(3) The marginal fuel in WI interconnect is predominately natural gas which is abundant and 

projected to continue to be inexpensive. Therefore, in theory any change in production cost as a 

result of Glen Canyon operations is primarily a function of changes in heat rate making the cost

curve relatively flat. This is the primary reason that pumped storage power plants are rarely 

used for arbitrage in current WI markets.

(4) Since economic analyses focus on differences among alternatives and not absolute values, the 

effects of long-term large scale issues such as climate change on the decision-making processes 

are significantly diminished compared to changes in absolute values.

(5) WI operations are influenced by a multitude of autonomous decision makers in a massive 

system that is not only exceptionally diverse and complex, but interwoven with water, 

economic, and political/legal considerations that transcend even the most sophisticated models. 

Although models may produce “numbers” based on economic theory, this result is very difficult 

to defend given the simplifying assumptions that are required and the quality and granularity of 

model input data. Especially when operational changes at Glen Canyon Dam comprise a small 

fraction of the total grid resource.

Therefore, it is our opinion that it would be a mistake for GTMax to shift focus away from the “flower 

petal” and on to the “distant star.” At the CRSP level, Western has adequate information and data along 

with the expertise to produce high quality results. In addition, CRSP relationships are well defined and 

understandable with a high degree of fidelity. To the extent that changes in long-term large scale grid 

operations can be measured with an acceptable level of accuracy we rely on other models to provide

GTMax with direction. 

Concluding Remarks

In the introduction the review states that the objectives of the report are “to review the GT Max model 

and its uses for economic analysis of issues relevant to the Glen Canyon Dam Adaptive Management 

Program.” It appears the focus of the report significantly strayed from this objective throughout much of 

the text. Instead, it discusses market issues that are not yet applicable to Western and futures that may 

or may not come into fruition. We are pleased that reviewers found GTMax exceptional in its ability to 

perform hydroelectric analyses. Unfortunately, most of the review discussed issues that are beyond the
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

11

current design and scope of the model; that is, it mainly discussed what GTMax does not do. It focused 

on the inability of GTMax to adequately represent potential WI-wide changes that, from their viewpoint,

may be important in the future. It also suggests the model include actions that are outside the 

boundaries of Western’s statutory obligations.

The reader of the review should recognize that GTMax is not a stagnant model, but it evolves to reflect 

changing conditions. As instructed by workshop organizers, our presentations were geared toward 

describing the model and past applications. If informed prior to the workshop that the focus was on 

issues such as global warming, different utility business models, capacity expansion, and ancillary 

services, we would have structured our presentations accordingly to describe how GTMax can be 

integrated with other tools to address these issues. 

While we have an appreciation for longer-term and broader perspectives discussed by the reviewers, it 

should be recognized that no model can do everything. Therefore given limited resources we have 

developed GTMax to addresses critical operational issues as it pertains to the CRSP system. The 

narrower scope of GTMax should be viewed as a strength, not as a weakness. It allows GTMax to 

produce realistic results that are both useful and used with acceptable run times. A testimony to the 

validity is that it is used on a monthly basis by the CRSP Energy Management and Marketing Office to 

make CRSP-related decisions. By broadening the scope of GTMax this strength may become diluted. 

Although there are some very important and interesting topics raised by the reviewers, it is our opinion 

that some modeling suggestions are extremely costly to implement and of small marginal benefit in 

terms of addressing pertinent questions related to the economic costs of altering Glen Canyon Dam 

operations.

As with any model, it is important to make decisions regarding what issues are of importance to the 

problem it addresses. Reviewers however, discuss broad WI issues that span multiple decades, yet 

advocate a modeling time step of 5 minutes or less over a vast and complex system. It is our opinion 

that these issues are beyond the direct scope of issues related to the Glen Canyon Dam Adaptive 

Management Program and best addressed in other forums.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

12

References

DIS (Decision and Information Sciences Division), 2007, Electricity Market Complex Adaptive System 

(EMCAS) User’s Manual – Version 2.0, Argonne National Laboratory Report, December.

Kinetrics, Inc., 2005, Commitment Techniques for Combined-Cycle Generating Units, prepared for CEA 

Technologies, Inc. (CEATI), CEATI Report No. T053700-3103, Dec. 

[http://www.nyiso.com/public/webdocs/markets_operations/committees/bic_mswg/meeting_material

s/2006-02-15/3103_Commitment_Techniques_for_CCGUs.pdf], accessed January 3, 2013.

Veselka, T.D., L. Poch, and A. Botterud, 2012, Review of the WECC EDT Phase 2 EIM

Benefits Analysis and Results Report, ANL/DIS-12-2, Argonne National Laboratory, April.

Bibliography

Abhishek, K., S. Leyffer, and J.T. Linderoth, 2010, “FilMINT: An Outer Approximation-Based Solver for 

Convex Mixed-Integer Nonlinear Programs,” INFORMS Journal on Computing, vol. 22, pp. 555–567.

Achterberg, T. 2009, “SCIP: Solving Constraint Integer Programs,” Mathematical Programming 

Computation, vol. 1, no. 1, p. 1–41.

Ahuja, R.K., T.L. Magnanti, and J.B. Orlin, 1988, Network Flows: Theory, Algorithms, And Applications, 

Prentice Hall, NJ.

Belotti, P., 2009, COUENNE: A User’s Manual, Department of Mathematical Sciences, Clemson 

University, Clemson, SC, available at http://www.coin-or.org/Couenne/couenne-user-manual.pdf, 

accessed April 23, 2012.

Bonami, P., L.T. Biegler, A.R. Conn, G. Cornuejols, I.E. Grossmann, C.D. Laird, J. Lee, A. Lodi, F. Margot, N. 

Sawaya, and A. Waechter, 2008, “An Algorithmic Framework for Convex Mixed Integer Nonlinear 

Programs,” Discrete Optimization, vol. 5, no. 2, pp. 186–204.

FICO, 2012, “Test Drive the Xpress-Optimization Suite,” available at 

http://www.fico.com/en/Products/DMTools/xpress-overview/Pages/Xpress-Optimizer.aspx, accessed 

April 23, 2012.

Forrest, J.J., 2006, “Coin Branch-and-Cut Solver,” July 18, available at 

http://dimacs.rutgers.edu/Workshops/COIN/slides/forest2.pdf, accessed April 23, 2012.

Forrest, J.J., and E.M.L. Beale, 1976, “Global Optimization Using Special Ordered Sets,” Mathematical 

Programming, vol. 10, pp. 52–69.

Fourer, R., D.M. Gay, and B.W. Kernighan, 2002, AMPL: A Modeling Language for Mathematical 

Programming, Brooks/Cole Publishing Company.

GUROBI Optimization, undated, “An Easier Way to Better Decisions,” available at 

http://www.gurobi.com, accessed April 23, 2012.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

13

IBM, undated, “IBM ILOG CPLEX Optimizer: High-Performance Mathematical Programming Solver for 

Linear Programming, Mixed Integer Programming, and Quadratic Programming,” available at 

http://www-01.ibm.com/software/integration/optimization/

cplex-optimizer/, accessed April 23, 2012.

Mahajan, A., S. Leyffer, J.T. Linderoth, and J. Luedtke, 2012, “MINOTAUR wiki page,” Argonne National 

Laboratory, April 12, available at http://wiki.mcs.anl.gov/minotaur/

index.php/Main_Page, accessed April 23, 2012.

Papadimitriou, C. H., 1981, “On the Complexity of Integer Programming,” Journal of the Association for 

Computing Machinery, vol. 28, pp. 765–768.

Ralphs, T., M. Guzelsoy, and A. Mahajan, 2011, SYMPHONY Version 5.3 User’s Manual, COR@L 

Laboratory, Industrial and Systems Engineering Department, Lehigh University, Bethlehem, PA. 

Sahinidis, N.V., 1996, “BARON: A General Purpose Global Optimization Software Package,” Journal of 

Global Optimization, vol. 8, pp. 201–205.

Tomlin, J.A., 1981, “A Suggested Extension of Special Ordered Sets to Non-Separable Non-Convex 

Programming Problems,” in Annals of Discrete Mathematics (11) Studies on Graphs and Discrete 

Programming, P. Hanson (Ed.), North-Holland Publishing Company, Amsterdam, NY.

Wang, J., X. Wang and Y. Wu, 2005, Operating Reserve Model in the Power Market, IEEE Transactions on 

Power Systems, Vol. 20, No. 1, February.
1

Comments	by	Sam	Loftin

on	the

FINAL	REPORT	OF	THE	GTMAX	MODEL	REVIEW	PANEL:	

Report	of	a	Workshop	held	August	31	and	September	1,	2011	in	Flagstaff,	Arizona

My	comments	are	based	on	my	use	of	the	GT-Max	and	predecessor	models	for	the	last	decade	or	so,	

and	on	my	33	years	of	experience	as	an	electrical	engineer	at	Western	Area	Power	Administration’s	

CRSP	office.

Executive	Summary,	Pages	1-3

The	 purpose	of	the Report	by	the	reviewers was	to	assess	Western’s	and	Argonne’s current	GT-Max	

model	usage,		usage in	the	LTEMP	EIS,	and	in	large	geographic	and	time-scale	power	system	studies.		In	

this	section,	the reviewers summarize some	limitations	of the	current	model	in its present configuration

– some	of	which	 I	agree	with.		Addressing	some	if	not	most of	these limitations	to the	current	GT-Max

model	is	the	purpose	of	the	current	Argonne-led	DOE	lab	effort	to	create	a	new	hydro	modeling	system.

As	is	typical	of	such	models,	it	simplifies	what	is	in	reality	a	very complex	system	of	inter-related	

electrical	generation	facilities	and	decision	factors,	and	therefore,	like	all	such	models,	it	is	an	

imperfect	representation	of	reality	and	has	limitations	in	how	it	can	be	applied.

I	agree	with	the	above	statement	from	the	Report,	and	especially	the	underlined	portion (my	

emphasis).		We	need	to	keep	in	mind	that	computer	models	are	meant	to	help	us	inform	decisions,	

but	are	limited in	applicability,	scope, and	accuracy.		The	statement	applies	not	only	to	GT-Max	but	

to	all	computer	models,	including	river	system	models like	Riverware,	sediment	models,	beach	

building	models,	and	climate	change	models.

In	summary,	the	GTMax	model	and	Western’s	analytical	framework	are	designed	primarily	to

evaluate	short-term	operations.	The	authors	of	the	GTMax	model	did	not	include	long-term

planning	features,	such	as	long-term	capacity	expansion	algorithms,	which	limits	its	utility	for

forecasting	economic	implications	of	long-term	operations.	The	model	is	not	capable	of	modeling

operations	of	less	than	one-hour	duration,	which	means	that	the	potentially	significant	economic

value	of	ancillary	services	cannot	be	adequately	represented	in	the	modeling	results.

I	think	that	this	statement	is	accurate.		In	the	body	of	the	report	the	reviewers	discuss	how	GT-Max

was	designed	primarily	to	inform	short-term	hydroelectric	operational	needs,	and	only	secondarily	

for	longer-term financial	and	economic	studies.	Western	and	Argonne	designed	GT-Max	to	do	some	

things	very	well,	and	that	is	what	GT-Max does	best.		That	doesn’t	mean	that	GT-Max can’t be	used	

for	other	modeling	problems,	only	that	it	may	be more	complex or	time	consuming	to	use	it	for	

longer	time	duration	or	larger	geographical	area	studies. Or	that	GT-Max might	need	model	input	

from	other	model	outputs.

The	model	relies	on	inputs	from	the	RiverWare	model,	which	are	provided	by	Reclamation;	this	

constrains	Western’s	ability	to	modify	hydrogeneration	scenarios	for	assessing	impacts	of	climate	
2

change	or	river	diversions.	Thus,	the	model	does	not	facilitate	convenient	or	efficient	evaluation	of	

alternative	assumptions	that	may	be	substantially	different	in	the	future.

The	reviewers	discussion	of	Riverware	here	and	in	the	body	of	the	Report	leads me	to	think	 that	

they	don’t	really	understand	how	Western	uses	Riverware	output	in GT-Max	studies.		Riverware	

output	(more	specifically	the	multi-trace [ISM] version	of	CRSS)	is	used	in some	GT-Max	studies	

Western	has	performed	or	participated	in,	particularly	the	longer-term	studies like	the	Shortage

Criteria	EIS,	the	Colorado	River	Basin	Water	Supply	and	Demand	Study,	or	in	projecting	purchase	

power	expenses	for	Western’s SLIP	Power	Repayment	Studies.		Western	uses	Reclamation’s	24-

month	study	output	more	often than	Riverware,	particularly	in	the	monthly	prechedule	studies	that	

I	do.		There	is	no	requirement	to	use	any	Reclamation	model	output	in	GT-Max	– we	could	easily	

use	other	sources	of	water	release	data	as	long	as	those data	are in	a	format that	GT-Max	can	use

(for	instance,	monthly	reservoir	releases	in	acre	feet	and	reservoir	elevation	in	feet).

Furthermore,	the	model	does	not	have	the	geographic	scope	or	an	adequate	representation	of	

transmission	to	study	possible	consequences	of	policy	changes	in	other	parts	of	the	WECC.	The	

strength	of	the	model	as	currently	formulated	lies	in	its	ability	to	examine	the	consequences	of	

following	specified	management	regimes	over	short	periods	of	time	when	water	conditions,	

electricity	prices,	and	other	variables	are	reasonably	stable.

Again,	true,	but	that	wasn’t	what	the	GT-Max model was	designed	to	do.		GT-Max	is	unrivalled	in	

my	experience in	modeling	the	complex	operational	and	environmental	constraints	that	we	

encounter	at	CRSP	power	plants.		That	is	where	its	true	usefulness	lies,	and	that	is	where	other	

power	system	models	that	I	have	encountered	are	lacking.

Three	Business	Models,	pages	7-9

The	reviewers	present	an	overview	of	the	business	models	found	in	North	American	power	

markets;		ISOs	like	the	California	ISO,	traditional	bilateral	markets,	and	a	mixture	of	the	two	like	is	

found	in	the	Western	Interconnection.	They	then	discuss	Western	business	model.

With	available	excess	capacity,	Western	could	bid	into	California markets	when	either	energy	or	

ancillary	services	prices	make	it	financially	attractive.

Reading	these	sections	suggests that	the	reviewers	prefer	the	ISO	model	and	would	like	Western	to	

move	its	business	model	in	that	direction.	They	advocate	for	models	that	can	better	characterize	

sub-hourly	hydroelectric	operations	for	ancillary	services. This	advocacy	continues	in	other	sections	

of	the	report	and	in	the	Conclusions	and	Recommendations.		This	business	model	doesn’t	really	fit	

with	the	current	model	that	Western	operates	under.		Our	statutory	requirements	and	limitations	

make	Western	significantly	different	from	a	typical	Investor-Owned	or	Publicly-Owned	Utility and	

limits	our	ability	to	do	as	the	reviewers	propose.

Typical	Application	and	Unique	Features,	Page	12

Most	commercially	available	production	cost	models	have	very	simple	representations	of	the
3

hydroelectric	generation	system.	The	standard	approach	is	to	divide	hydro	resources	into	“base	

load” and	“load-following”	segments	for	each	simulation	period.	The	base	load	segment	is

dispatched	in	all	hours	of	the	period.	The	load	following	energy	is	used	to	“peak	shave”	the

anticipated	load	profile	up	to	the	limit	of	hydroelectric	capacity.	The	result	of	this	is	a	flattened	load

shape	that	is	used	to	simulate	the	operation	of	thermal	power	plants.

The	reviewers	compare	GT-Max	with	other	commercially	available	production	cost	models.		They	

correctly	emphasize	the	superiority	of	GT-Max	for	modeling	complex	hydro	operations.		This	is	a	

topic	that	the WECC	Hydro	Modeling	Task	Force	(HMTF),	that	I	participate	in,	spends	a	great	deal	of	

its	time	on. WECC	is	using	commercial	products such	as	Promod	and	Gridview to	simulate	WECCwide	operations	while	simulating	widely	varying	hydro	plants,	each	with	its	own	set	of	constraints.		

The	hydro	operations	of	individual	members’ systems	are	much	more	complex	than	can	be	

characterized	in	such	models,	so the	HMTF	continues	to	experiment	with, develop,	and	implement

methods	to	better	characterize	hydro	operations.

Objective	function,	solution	algorithms,	fundamental	period,	pages	12-15

In	any	case,	all	objectives	have	to	be	reduced	to	a	single	number	for a	computer	model	or	analyst	to	

compare	and	rank	alternative	plans.

In	my	modeling work	at	Western,	I	never	look	at	the	“single	number”.		The	useful	output	from	the	

model	is	the	modeled hourly	operations in	megawatts	at	each	power	plant,	total	generation	over	

the	month, and	the	overall	purchase/sales dollar information.

Key	Exogenous	Data	and	Constraints,	pages	17, 18

Western	relies	on	inputs	from	Reclamation’s	RiverWare model	for	hydrologic	modeling.	The

general	structure	of	RiverWare	is	described	in	Zagona,	et	al	(2001).	As	we	understand	it,

Reclamation	uses	this	model	to	provide	Western	with	forecasts	of	monthly	water	quantities

available for	discharge	through	the	Glen	Canyon	Dam.	Reclamation	may	revise	these	forecasts	over

the	year	as	hydrologic	conditions	unfold.	With	a	monthly	water	allocation	as	input	GTMax	then

simulates	operation	of	the	Glen	Canyon	Dam	and	associated	facilities	according	to	various	economic

objectives	and	operational	constraints.	The	economic	factors	that	might	affect	the	monthly

allocation	of	water	across	the	year	are	opaque.	Casual	observation	of	various	GTMax	results

suggests	that	RiverWare	has	some	notion	that	water	is	more	valuable	for	power	generation	during

the	summer	and	winter	months	than	during	the	spring	and	fall.

See	my	comments	on	Riverware	in	the	Executive	Summary section.	

The	reviewers	appear to	have	the	relationship	between	Riverware	and	water	release	decisions	

reversed.		Reclamation,	at	Western’s	request,	programs	additional	water	into	the	summer	and	

winter	months	because it	is	more	valuable	to	Western then.		Riverware	then	takes	those	release	

preferences	and	incorporates	them	into	future	release	patterns.	In	this	section,	the	reviewers	imply

that	water	is	not	more	valuable	for	generation	in	the	summer	and	winter than	the	spring	and	fall.		

Western’s	power	allocations	to	its	customers	peak	in	the	winter	months	(primarily	December,	

January,	and	February)	and	summer	months	(primarily	July	and	August),	so	that	is	why	we	ask	
4

Reclamation	to	program	higher	water	releases	in	those	months. Additionally,	prices	for	firming	

power	purchases	also	peak	in	those	months so	we	try	and	avoid	making	purchases	then	when	

possible.

This	procedure	ignores	the	possibility	of	climate	change.	Under	a	climate	change	hypothesis,	some	

sample	selection	method	would	need	to	be	used	to	determine	a	going	forward	relevant	range	of	

hydrologic	variation	that	would	be	different,	arguably drier,	than	what	would	be	reflected	using	the	

complete	hydrologic	record.	We	discuss	this	point	further	below.

Reclamation	has	looked	at	potential	climate	change	effects	on	Colorado	River	water	resources	in	

the	recently	completed	Colorado	River	Basin	Water	Supply	and	Demand	Studyi,	done	in	conjunction	

with	the	seven Colorado	Basin	States.		DOE	has	also	recently	completed its	report	on	the	effect	of	

potential	climate	change	on	Federal	Power	Marketing	Administrations,	the	Section	9505	Report ii.		

Both	of	these	reports	specifically	address	potential	changes	to	Colorado	River	runoff	and	its	effect	

on	hydropower	operations.		Reclamation	can	incorporate	information	from	those	studies	into	

Riverware,	and	that	information	can	be	incorporated	into	GT-Max	modeling.

It	is	interesting	to	note	in	particular	that	the	Section	9505	Report	looked	at	projections	of	climate	

change	in	the	next	several	decades.		The	projections were	prepared	by	Oak	Ridge	Laboratory	on		

behalf	of	the	Department	of	Energy	and	showed	that	the	potential	changes	in	runoff	are	well	within	

the	historical	climate	variability	that	Western	has	experienced	over	the	last	20	to 30	years.		The	

projections	actually	showed	an	increase	in	Western	generation	in	several	of	the	river	basin	regions

of	our	service	area,	contradicting	the	drying	trend	in	the	western	states	that	is	generally	assumed	to	

be	the	result	of	global	warming,	and	is	predicted	in	the	Colorado	River	Basin	Water	Supply	and	

Demand	Study.	This	result	led	Western	to	conclude	in	the	report	that	our	existing	authorities	and	

operations	and	marketing	programs	were	adequate	to	accommodate	any	potential	changes due	to	

climate.

Capacity	Expansion	Algorithm,	pages	19,	20

This	section	notes	the	lack	of	capacity	expansion	planning	in	GT-Max.		Since	Western	has	no	legal	

authority	to	add	capacity	or	serve	load	growth,	it	is	understandable	why	GT-Max	would	not	include	

this	feature.		It	might	be	easier	to	just	access	capacity	expansion	data and	model	output and	import	

it	into	GT-Max	when	a	longer	term	study	is required,	rather	than	adding	the	bulk	and	complexity	to	

the	existing	model.

Reserves	commitment	and	ancillary	services	algorithms,	pages	20	and	21

See	my	comments	to	the	Three	Business	Models	Section.	The	reviewers	again	advocate	for	using	

Western’s	hydro	system	for	providing	ancillary	services	to	renewable,	non-dispatchable	generation.		

This	advocacy	seems	misplaced	in	a	review	of	the	GT-Max	model.	 Perhaps	the	reviewers	saw	this	

report	as	their	only	opportunity to	influence	Western’s	policy	direction.

Unit	reliability	algorithm,	page	21
5

The	reviewers	are	correct	to	note	that	GT-Max	treatment	of	unit	outages	is	less	that	optimal.		GTMax	simulates	power	plants	as	a	lumped	total,	rather	than	at	a	unit	level,	and	any	plant	deratings	

are	in	monthly	increments.		In	my	experience	using	GT-Max,	accurately	modeling	unit	outages	is	

one	of	the	more	challenging	and	time	consuming	aspects	of	the	job.		It	is	my understanding	that	the	

new	Hydro	model	Argonne	is	creating	will	significantly	improve	on	this	aspect,	being	able	to	model	

individual	units	on	time	scales	of	less	than	a	month.

Long-term	risk	algorithm,	pages	21	and	22

I	disagree	with	the	reviewers thoughts	that	GT-Max	should	include	long-term	risk	algorithms.		I	

think	that	computer	models	that	become	too	large	and	complex	become	unwieldy	to	use	and	more	

prone	to	giving	inaccurate	results. If	you	want	a	good	laugh,	go	back	and	read	some	of	the	reports	

that	were	produced	20	years	ago	using	those	expansion	models	and	see	what	they	were	projecting.		

I	agree	that	long-term	risk	is	a	worthy	subject	to	study,	but	think	that	it	should	be	done	separately	

from	GT-Max.

Energy	Market	Prices,	pages	22	and	23

All	of	the	economic	studies	assume	the	same	prices	regardless	of	operational changes	at	the	Glen	

Canyon	Dam.	The	underlying	assumption	is	that	whatever	changes	might	possibly	occur,	they	are	

too	small	to	matter	given	the	large	size	of	the	market.

I	suspect that	generation	changes	at	Glen	Canyon and	CRSP	in	general	could,	in	the	most	extreme	

circumstances	(High	Flow	Experiments	or	very	low	flow	drought	scenarios), affect	local power	prices	

by	flooding	the	market with	surplus	generation,	or	by	buying	most	available	generation	surpluses.		I	

don’t	have	any	evidence,	however,	it	is	just	a	hunch.

I	agree	with	reviewers	that	the	ability	to	incorporate	a	probability	distribution	of	power	prices	

rather	than	a	single	price	would	be	an	improvement	to	GT-Max.		That	would	require	multiple	runs	

of	the	model	to	look	at	the	entire	range	of	prices,	which	would	add	considerably	to	the	complexity	

and	model run	time.

Ancillary	Services,	pages	25	and	26

The	reviewers	again	advocate that Western	enter		the ancillary	services	market in	the	Western	

Interconnection. See	my	comments	above in	the	Reserves	commitment	and	ancillary	services	

algorithms section	and	the	Three	Business	Models section.

Capacity	value—lack	of	conceptual	clarity,	pages	26	and	27

I	agree	with	the	reviewers	that	Western’s	treatment	of	capacity	and	its	value	is	inconsistent.		Most	of	

the	monthly	GT-Max	operational	modeling that	 I	perform doesn’t	take	capacity	value	into	account	at	

all,	since	we	firm	our hydro	capacity	almost	completely	with	firm	energy	purchases rather	than	capacity	

purchases.		In	other studies Western	has	done, capacity	value is	included,	but	the	values	vary	 based	on	

what	 information	we	have	available	at	the	time.		I	think	the	lack	of	good	sources of	knowledge	of	what	
6

capacity	sells	for	in	the	Western	Interconnection	is	a	big	impediment – more	so	than	any	deficiency	in	

GT-Max.

Riverware,	page	28

See	my	comments	on	Riverware	in	the	Executive	Summary section.	

Large-Scale	and	Long-Term	Risk,	page	28

See	my comments	in	the	Key	Exogenous	Data	and	Constraints section	above.		The	reviewers	

misunderstand that	Western,	as	a	wholesale	supplier	of	a	defined	resource	base, has	no	responsibility	

for	load	growth	or	changes	in	retail	loads.		Western	can’t	just	decide	to	change	its	business	model	

without	change	to	the	legislation	that	governs	our	operations.

Conclusions	and	Recommendations,	pages	30	through	32

I	think	all	these	subjects	have	been	covered	in	my	comments	above.

																																																												 i

http://www.usbr.gov/lc/region/programs/crbstudy/finalreport/index.html

ii Department	of	Energy	(DOE)	Report	to	Congress	on	Climate	Change	and	Federal	Hydropower

(Section	9505	Report)	http://nhaap.ornl.gov/content/climate-change-impacts
 

ARIZONA

Arizona Municipal Power Users Association

Arizona Power Authority

Arizona Power Pooling Association

Irrigation and Electrical Districts 

Association

Navajo Tribal Utility Authority

(also New Mexico, Utah)

Salt River Project

COLORADO

Colorado Springs Utilities

Intermountain Rural Electric Association

Platte River Power Authority

Tri-State Generation & Transmission 

Association, Inc.

(also Nebraska, Wyoming, New Mexico)

Yampa Valley Electric

Association, Inc.

NEVADA

Colorado River Commission

of Nevada

Silver State Energy Association

NEW MEXICO

Farmington Electric Utility System

Los Alamos County

City of Truth or Consequences

UTAH

City of Provo

City of St. George

South Utah Valley Electric Service District

Utah Associated Municipal Power Systems

Utah Municipal Power Agency

WYOMING

Wyoming Municipal Power Agency

Leslie James

Executive Director

CREDA

10429 S. 51st St., Suite 230

Phoenix, Arizona 85044

Phone: 480-477-8646

Fax: 480-477-8647

Cellular: 602-469-4046

Email: creda@qwest.net

Website: www.creda.org

CREDA

Colorado River Energy Distributors Association

January 7, 2013 VIA EMAIL

David Lytle - USGS

Jack Schmidt - GCMRC

RE: Final Report of the GT Max Model Review Panel (September 2012)

Dear David and Jack:

CREDA members appreciated the opportunity to participate in the August 

31-September 1, 2011 workshop held at GCRMC’s offices, and during day two of the 

workshop, presented information reflecting work undertaken by a CREDA planning, 

operational and analysis committee over the past couple of years. As a member of 

the committee, Dave Slick presented a utility perspective and there was discussion 

of the information amongst the panelists and participants. As some of the 

information contained in the September 4, 2012 Final Report reflects the utility 

perspective discussion, CREDA is offering the following observations on the report. 

As you read the following observations, please keep in mind that the 

stakeholder participants in the workshop have not been provided the specific charge 

or assignment given to the panel participants, so the following comments are based 

on our assumptions derived from reading the final report. We would welcome an 

opportunity to discuss the report with you or your staff to have a better 

understanding as to the task(s)/question(s) the panelists were asked to consider 

during the workshop and in preparation of the report. We are offering the 

following technical comments on the report, as well as offering a perspective on 

how the report may affect collaboration within the Glen Canyon Dam Adaptive 

Management Program (AMP).

1. Stated Objective Versus Resultant Work Product

In the last paragraph of the introduction on page 6, the author(s) state that their 

role was “to review the GT Max model and its uses for economic analysis of issues 

relevant to the Glen Canyon Dam Adaptive Management Program.” We are unable 

to understand how the report addresses that objective; rather, the report 

emphasizes a formal, centralized electric market advocacy position. 

Rather than being a constructive, useful tool to assess applicability of GT Max for 

AMP applications, the tone and content of the report are quite different than the 

actual workshop discussions among the participants. The report appears to focus 

on what the model doesn’t do, and what WAPA doesn’t do in fulfilling its statutorily 

mandated mission, as opposed to assessing the model’s applicability to AMP 

applications.

2. Concept of Peer Review 

It was our understanding that a purpose of the workshop was to provide a peer 

review of the GT Max model. We think that a peer is a person who has equal 

standing in some respect. We believe that the object of the federal government’s 

peer review process is, generally, to obtain an objective assessment of a body of 
work by a person who has a similar expert understanding about the subject, or body of work, as the 

person or persons who perform the work that is being peer reviewed. 

As stated in the report, GT Max is a production cost simulation model. Employees of WAPA and 

Argonne who presented information at the workshop and use the model in the conduct of daily 

business responsibilities are production cost simulation experts, with very specific, special knowledge 

and experience in dealing with the complicated nuances associated with CRSP hydro system 

operation. 

Based on many statements/conclusions/recommendations contained in the report, it appears to us 

that the author(s) may not be production cost simulation experts. Consequently, we are not 

convinced that an actual peer review has occurred. As pointed out by the author(s), there are a 

number of production cost models being used in the United States, and a large body of literature 

about the subject. 

We may be misinformed as to the purpose and scope of the workshop and role/charge to the 

panelists, but possibly that could be resolved if such information is made available.

3. WAPA’s Statutory and Contractual Obligations

In various places throughout the report, the author(s) suggest that WAPA should seek to improve its 

profit position, increase net revenues, bid its generation into California’s ancillary services market, 

maximize revenues, and other similar ideas. 

These suggestions signal a lack of understanding about the Colorado River Storage Project Act, 

WAPA‘s statutory obligation, delivery obligation concepts such as contract rate of delivery and 

sustainable hydropower, and WAPA’s contractual delivery obligations to existing customers.

In short, WAPA’s CRSP resources are already fully dedicated to existing contractual obligations, 

regardless of any year’s specific Colorado River basin hydrology (wet or dry). And WAPA’s statutory 

obligation is to market federal project hydropower at the lowest possible rates to consumers 

consistent with sound business practices. Those rates are structured to recover 100% of the 

construction and O&M costs allocated to power AND to repay a significant amount of the investment 

in irrigation projects in the upper Colorado River basin. That irrigation repayment obligation alone sets 

WAPA apart from any other power marketing administration.

In our view, WAPA has neither the resource flexibility nor the discretion needed to pursue the 

author(s)’ suggestions in a way that would have a material impact on either WAPA or western region 

electric business economics.

It is our opinion that these comments/recommendations are out of scope (as we understand it) and 

unfortunately could lead to a serious misunderstanding of these issues by AMP stakeholders. If there 

is a basic misunderstanding of these issues amongst the AMP stakeholders, it makes collaboration and 

consensus building, already a challenge, much more elusive.

4. Centralized Market Effectiveness in the Western U.S.

As reflected in the report, a viewpoint of at least one panelist during the workshop was that formal, 

centralized markets are the panacea solution for all electric business problems, and that WAPA can 

materially improve its economic welfare through increasingly robust participation in such markets.

An assessment of actual historic experience in the Western U.S. suggests that retail customers in 

California have not benefited from the presence of the centralized market in that state. 
The analysis included with this email transmittal (attachments 1-3) compares the change in prices for 

the California Independent System Operator (CAISO) participants between 1995 (the year prior to 

CAISO launch) and 2010 (the latest year for which information was available at the time the analysis 

was done) versus the change in prices in other Western States where no centralized markets exist. 

The essential message of all three attachments is that the organized wholesale market in California has 

not served to provide bottom line retail cost benefits to consumers relative to electric business 

management in other Western states during this 15 year period.

While conceding that near-term costs for consumers will go up, a March 16, 2012 DOE Secretary of 

Energy Chu memorandum expressed the belief that in the long run, consumer costs would go down as 

a result of centralized market formation. From the perspective of many of WAPA’s customers, 

centralized markets may not be “the” answer.

As with item 3 above, we believe that this portion of the report is out of scope and could lead to a 

flawed understanding by AMP stakeholders and readers of the report.

5. Revenue Versus Cost

In addition to numerous references to maximizing revenues and improving profits, the author(s) state 

that “minimizing costs may not be the most important objective”, and proceed to suggest that 

maximizing revenues would constitute a mathematically equivalent approach.

From a WAPA customer perspective, we believe that there is a significant policy-level difference 

between a statutory obligation that requires WAPA to market federal project hydropower at the lowest 

possible rates to consumers consistent with sound business practices, and a revenue maximization goal.

On one hand, the author(s) correctly observe that the objective function of the GT Max model is to 

minimize cost. In this regard, we believe that WAPA and Argonne have correctly oriented the tool so as 

to be reflective of WAPA’s legal obligation. 

On the other hand, the author(s) describe the business area where 100% of WAPA’s customers reside 

as “the regulated IOU business model”, which suggests to us that the author(s) may not understand 

the public power characteristics of WAPA’s customers.

The foregoing observations notwithstanding, the real damage in the revenue versus cost discussion is 

the promulgation of the misleading suggestion that utility business entities (WAPA, WAPA’s customers, 

other public power entities, IOUs, etc.) are exclusively focused on maximizing profits. This is 

particularly harmful in the AMP arena, where the majority of participants are not utility business 

professionals, and a perception that “profits trump natural resources” could be reinforced. 

As in items 3-4 above, the promotion of false perceptions such as this serves only to establish or 

strengthen barriers among AMP stakeholders who might otherwise be able to collaborate more 

effectively. 

6. Capacity Expansion Algorithms

The report’s author(s) suggest that the absence of a capacity expansion algorithm is a weakness of GT 

Max for the “types of analyses that Western is considering”.

In contrast, our understanding of WAPA’s scope of federal responsibility and mission is such that we do 

not understand why such modeling capability would ever be needed. WAPA has no retail obligation to 

serve, and no load growth responsibility. In analyses of alternative Glen Canyon and other CRSP hydro 
system flow regimes, the prospect of federal construction of new generating capability to offset capacity 

lost due to flow regime changes has, to our knowledge, never been contemplated.

Institutions that are in the business of using production cost models to forecast and study projected 

future generating system operation and costs (because they have the responsibility to plan for the 

future electric service requirements of their retail customers) do not use capacity expansion algorithms, 

and consider doing so to be an academic exercise that is not particularly helpful to the conduct of real 

world decision-making.

Finally, we think the suggestion that a capacity expansion algorithm is necessary to the production of 

superior electric price forecast information in future years for the valuation of capacity is not valid.

7. Other

There are other technical errors in the report. For example, the statement on page 21 that “the failure 

of a substation transformer or transmission line at one of the dams” (referring to Hoover and Glen 

Canyon) “takes out the entire project” is incorrect. The CRSP (Glen Canyon) and Boulder Canyon 

Project (Hoover) are separately authorized and marketed projects. Further, the Boulder Canyon 

Project does not have associated transmission, and the transmission system(s) used to market its 

output are not automatically “taken out” by a transformer failure or transmission line in the CRSP 

project (and vice versa). 

We are available to discuss these observations with you or your staff, the panelists, and/or the 

author(s) of the report. Please don’t hesitate to contact me at 480-477-8646, or Dave Slick at 602-236-

2082.

Sincerely,

/s/ Leslie James

Leslie James

Executive Director

Cc: CREDA Board

 Darren Buck – WAPA

Att: 4 attachments to email transmittal
Western States Retail Electric Price Information 

Western States Retail Electric Price Information Fig 1 May 2012.docx

Figure 1: Cent / KWh Increase Between 1995 and 2010

Description

Figure 1 displays the retail electric price increase in each state between 1995 and 2010 as measured 

in cents per kilowatt hour (Cents/KWh). For example, in Idaho the retail electric price in 2010 was 

2.45 cents/KWh higher than the retail electric price in 1995. 

Message

The retail electric price increase in California between 1995 and 2010 was significantly higher than 

every other Western state. The organized wholesale market in California (CAISO) did not serve to 

provide bottom line retail cost benefits to consumers relative to electric business management in other 

Western states during this 15 year period. 

Details

The Energy Information Administration (EIA) was the source for all data. California prices represent 

the average of those California entities that are CAISO transmission owners and serve retail 

customers (PG&E, SCE, SDG&E, Anaheim, Azusa, Banning, Pasadena, Riverside and Vernon). The 

investor owned utilities (PG&E, SCE, SDG&E) represent 96% of retail sales made by these entities.

2.56

2.89 2.45

3.63

4.33

3.23

1.88

1.64

2.07

3.03

1.63
Western States Retail Electric Price Information 

Western States Retail Electric Price Information Fig 2 May 2012.docx

Figure 2: Cent/KWh Differences Between California Electricity Prices and Other States

Description

Figure 2 displays retail electric price differences between California and other Western states during 

1995 and 2010 as measured in cents per kilowatt hour (Cents/KWh). For example, in 1995 the retail 

electric price in Utah was 5.10 cents/KWh lower than California, and in 2010 the retail electric price in 

Utah was 7.79 cents/KWh lower than California. 

Message

In 1995 retail electric prices in other Western states were significantly lower than retail electric prices

in California, and by 2010 the gap between retail electric prices in other Western states and California 

had widened. The organized wholesale market in California (CAISO) did not serve to provide bottom 

line retail cost benefits to consumers relative to electric business management in other Western states 

during this 15 year period. 

Details

The Energy Information Administration (EIA) was the source for all data. California prices represent 

the average of those California entities that are CAISO transmission owners and serve retail 

customers (PG&E, SCE, SDG&E, Anaheim, Azusa, Banning, Pasadena, Riverside and Vernon). The 

investor owned utilities (PG&E, SCE, SDG&E) represent 96% of retail sales made by these entities.

Cents / KWh

1995: 10.40 

2010: 14.73

1995: 6.30 

2010: 8.07

1995: 5.10

2010: 7.79

1995: 5.73

2010: 7.17

1995: 5.75

2010: 6.85

1995: 6.08

2010: 8.53

1995: 6.31

2010: 8.19

1995: 4.30

2010: 5.00 1995: 4.28

2010: 5.58

1995: 2.78

2010: 5.04

1995: 3.63

2010: 6.33
Western States Retail Electric Price Information 

Western States Retail Electric Price Information Fig 3 May 2012.docx

Figure 3: Annual Savings For A Family of Four Relative to California Electricity Prices

Description

Figure 3 displays the dollars that a family of four in other Western states saved relative to a family of 

four in California that consumed the same amount of electricity in 1995 and 2010. For example, in

Colorado a family of four saved $1,883 in 1995 and $2,678 in 2010 relative to a family of four in 

California that consumed the exact same amount of electricity in those years.

Message

For the same amount of electricity, the cost for a family of four in 1995 in all other Western states was 

lower than that for a family of four in California, and by 2010 the gap between California and other 

Western states had widened. The organized wholesale market in California (CAISO) did not serve to 

provide bottom line retail cost benefits to consumers relative to electric business management in other 

Western states during this 15 year period. 

Details

The Energy Information Administration (EIA) was the source for all price data. California prices 

represent the average of those California entities that are CAISO transmission owners and serve retail 

customers (PG&E, SCE, SDG&E, Anaheim, Azusa, Banning, Pasadena, Riverside and Vernon). The 

investor owned utilities (PG&E, SCE, SDG&E) represent 96% of retail sales made by these entities.

1995: $2,772 

2010: $3,874

1995: $2,244

2010: $3,739

1995: $2,521

2010: $3,442

1995: $2,530

2010: $3,288

1995: $2,675

2010: $4,094

1995: $2,776

2010: $3,931

1995: $1,892

2010: $2,400 1995: $1,883

2010: $2,678

1995: $1,223

2010: $2,419

1995: $1,597

2010: $3,038