FINAL PRODUCTS

OF THE 2011 HYDROPOWER KNOWLEDGE ASSESSMENT

AND GTMAX MODEL REVIEW WORKSHOP

January 14, 2012
TABLE OF CONTENTS

Introduction and Workshop Summary 

Part 1: 

Final Report of the GTMax Model Review Panel

Appendix A: Professional biographies of the Review Panel

Part 2:

Comments by S. Clayton Palmer, Western Area Power 

Administration

Comments by Sam Loftin, Western Area Power 

Administration

Comments by Tom Veselka and Les Poch, Argonne National 

Laboratory

Letter from Leslie James, CREDA
Introduction and Summary of Workshop Results

This document presents final written products that resulted from the Hydropower 

Knowledge Assessment and GTMax Model Review Workshop hosted by the Grand 

Canyon Monitoring and Research Center (GCMRC) at the US Geological Survey campus 

in Flagstaff, Arizona, on August 31-September 1, 2011.

Part 1 contains the report prepared by an independent review panel consisting of three 

experts in the fields of energy systems economics and modeling: Drs. Edward Kahn, 

Verne Loose, and Michael Schilmoeller. The panel’s report summarizes their

independent review of the GTMax model. Appendix A provides brief professional 

biographies of the panelists.

Representatives from Western Area Power Administration (Western), Argonne National 

Laboratories (ANL), and the Colorado River Energy Distributors Association (CREDA)

provided key presentations at the workshop that formed the foundation for the 

independent panel’s review. These representatives were invited by the GCMRC to 

review the report presented in Part 1 and their written comments are included in Part 2. 

Comments were provided by S. Clayton Palmer (Western), Sam Loftin (Western), Tom 

Veselka and Les Poch (ANL), and Leslie James (CREDA).
Part 1
Final Report of the GTMax Model Review Panel:

Report of a Workshop held August31and September 1, 2011

in Flagstaff, Arizona. Report dated September 4, 2012

by

Edward Kahn

Vernon Loose

Michael Schillmoeller
 

FINAL REPORT OF 

THE GTMAX MODEL 

REVIEW PANEL

Tuesday, 

September 

04, 2012 

Report of a Workshop held August 31 and 

September 1, 2011 in Flagstaff, Arizona 

At the invitation of the Grand Canyon Monitoring and Research 

Center (GCMRC), three independent experts reviewed the 

GTMax power operation model and its uses for economic 

analysis of issues relevant to the Glen Canyon Dam Adaptive 

Management Program (GCDAMP). This document presents 

their findings and recommendations. 
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 1

FINAL REPORT OF THE GTMax 

MODEL REVIEW PANEL

REPORT OF A WORKSHO P HELD AUGUST 31 AND 

SEPTEMBER 1, 2011 IN FLAGSTAFF, ARIZONA 

EXECUTIVE SUMMARY 

GTMax belongs to a class of models known as production simulation models.  The model’s primary

objective is to simulate the most efficient operation of the Colorado River Storage Project (CRSP)

hydroelectric facilities at least cost.  As is typical of such models, it simplifies what is in reality a very

complex system of inter‐related electrical generation facilities and decision factors, and therefore,

like all such models, it is an imperfect representation of reality and has limitations in how it can be

applied.   

As explained during the workshop, and discussed in the report, the GTMax model was developed by

Argonne National Laboratory at the request of Western Area Power Administration (hereafter,

Western) to assist Western with daily scheduling of CRSP operations and for use in special purpose

studies to evaluate various operating policies.  The model assesses the operational efficiencies and

trade‐offs associated with various operating scenarios for the CRSP system as a whole. Glen Canyon

Dam is just one component of the CRSP system, and decisions made at one facility affect all the

others, so there are significant challenges associated with extracting effects of Glen Canyon Dam

operations from the rest of the system using this model.  This is further complicated by the use of

RiverWare simulation output as an input to GTMax; since RiverWare software was not part of our

review we are unable to make further statements about how RiverWare output affects GTMax

analyses and results.

The model can be constrained according to various assumptions about how the system has been or

will be operated.   It relies on various data as inputs, uses a suite of software tools to solve a

complex system of equations to reduce all operating objectives and constraints to a single number.

That number is the total cost of system operation to meet the specified load for the designated

time period. This single number permits comparisons between different operating scenarios.  The

report describes the various components of the GTMax model and discusses these attributes in

relation to its original intended purpose, as well as in relation to other models of its type and recent

applications of the model.
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 2

The research review panel members (hereafter “reviewers”) conclude that GTMax is generally well‐

suited to the original purpose for which it was designed, i.e., to provide comparative assessments of

various operational options over relatively short‐time frames—one year or less.   The reviewers had

concerns, however, with the model’s application for other purposes for which it was not originally

designed and is not well‐suited.   Specifically, the model is not well‐suited for forecasting economic

implications of long‐term operational scenarios.  The reviewers express reservations with how the

model addresses capacity issues, reserve commitments, the valuing of ancillary services, and long‐

term risk factors, noting that proper valuation of these factors is essential for making accurate long‐

term economic forecasts. The reviewers point out that because the GTMax model was designed

primarily to evaluate short‐term operations, developers of the model did not include long‐term

planning features such as capacity expansion algorithms and algorithms for valuing ancillary

services, reserve commitments and various risk factors, which limit its utility for long‐term

forecasting purposes and raises questions about the accuracy of some previous study results.    

The report explores some of the data inputs used by the GTMax model and discusses how the

choices made about those inputs influence subsequent model outcomes.  For example, the model

relies on inputs from a separate hydrological model, RiverWare, which is used by the US Bureau of

Reclamation (hereafter, Reclamation) to forecast annual water volume distributions.  The

assumptions used to create the outputs from the RiverWare model that are subsequently used as

inputs to the GTMax model were unclear to the reviewers, but they noted that RiverWare appeared

to take into account the fact that the value of the hydropower resource is generally greater in

summer and winter vs. spring and fall, which in turn had implications for the results generated by

GTMax.  The reviewers also had questions concerning Western’s choices of prices for valuing

capacity in some of their “economic” analyses, noting that the basis for those choices was unclear

but had significant implications for modeled outcomes, such as those presented in Argonne’s 2010

Post‐ROD analysis report. They suggested that more transparency surrounding such choices was

needed in the future.   They recommend that Western generate diagnostic reports with its

modeling runs to clarify the effects of the underlying assumptions and inputs used for generating

model results.  They also recommended that the GTMax model itself should be documented in

writing for the benefit of those who wished to understand its structure and functions in more

detail.

Considerable discussion of the role of pricing, and its consequences for modeling, is included in the

report.  The reviewers note that choices made in valuing certain factors, such as capacity, have

significant effects on modeling outcomes.  The reviewers expressed concern that there was

insufficient clarity in past studies as to why some prices for power replacement costs were chosen

over others. They also questioned the proposed use of an expanded version of the model for

predicting prices at trading hubs.  They suggest that other methods, such as econometric modeling,

may be better suited to that purpose.

Below are some additional observations and recommendations from the report:
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 3

1) Western needs to develop the conceptual framework and quantitative tools to better prepare for

and be able to characterize its participation in ancillary services markets.  

2) For relatively modest outlay Western could conduct an econometric analysis of pricing at the

major trading hubs it employs in GTMax scenarios.

3) The review panel members observed that Western uses the terms “financial” and “economic” to

characterize two types of analyses involving very specific and somewhat limited applications of the

GTMax model (see pages 28‐29 for further details.)   These analyses do not encompass the full suite

of analyses typically associated with use of these two terms.

4) The reviews suggest that using GTMax to model the interface of the CRSP with the WECC system

would require significant enhancement, particularly in terms of its transmission typology to

effectively represent the combined system. The GTMax model does not have the geographic scope

to study possible consequences of policy changes on other parts of the WECC.  They suggested that

it might be possible to conduct such an analysis using GTMax if it were supplemented with more

extensive models to properly assess both the short‐term and long‐term effects of actions by

Western and the rest of the WECC on each other.  

In summary, the GTMax model and Western’s analytical framework are designed primarily to

evaluate short‐term operations.  The authors of the GTMax model did not include long‐term

planning features, such as long‐term capacity expansion algorithms, which limits its utility for

forecasting economic implications of long‐term operations.  The model is not capable of modeling

operations of less than one‐hour duration, which means that the potentially significant economic

value of ancillary services cannot be adequately represented in the modeling results.  The model

relies on inputs from the RiverWare model, which are provided by Reclamation; this constrains

Western’s ability to modify hydrogeneration scenarios for assessing impacts of climate change or

river diversions.  Thus, the model does not facilitate convenient or efficient evaluation of alternative

assumptions that may be substantially different in the future.  Furthermore, the model does not

have the geographic scope or an adequate representation of transmission to study possible

consequences of policy changes in other parts of the WECC.  The strength of the model as currently

formulated lies in its ability to examine the consequences of following specified management

regimes over short periods of time when water conditions, electricity prices, and other variables are

reasonably stable.
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 4

Contents 

EXECUTIVE SUMMARY ............................................................................................. 1

INTRODUCTION ........................................................................................6 

PURPOSE OF THE WORKSHOP .............................................................................. 6

THREE BUSINESS MODELS ..................................................................................... 7

ISO/RTO with Formal Wholesale Markets ................................................................ 7

Pure Traditional Scheduling Area .............................................................................. 8

Mixed Business Model among NERC Regional Entities ........................................... 8

Western’s Institutional Context .................................................................................. 9

REVIEW OF GTMAX ................................................................................................. 10

Typical application and unique features .................................................................. 12

Objective function, solution algorithms, fundamental period ................................... 12

Solution period, state variables, and solution variables .......................................... 15

Study period ............................................................................................................ 15

Types of units represented ...................................................................................... 16

Transmission topology ............................................................................................ 16

Principal output ........................................................................................................ 17

Key exogenous data and constraints ...................................................................... 17

Typical Production Cost Model Features Not in GTMax ......................................... 19

Capacity Expansion Algorithm .............................................................................. 19

Reserves commitment and ancillary services algorithms ..................................... 20

Unit reliability algorithm ......................................................................................... 21

Long-term risk algorithm ....................................................................................... 21

OBSERVATIONS AND FINDINGS ........................................................................... 22

Energy Market Prices .............................................................................................. 22

Transmission topology and the WECC ................................................................... 25

Ancillary Services .................................................................................................... 25

Capacity value—lack of conceptual clarity .............................................................. 26

Large-Scale and Long-Term Risk ........................................................................... 28

Economic and Financial Studies ............................................................................. 29

The User Community and Process Issues .............................................................. 29

Non-power values: reducing these to hydro constraints is the best that it can do. . 30

CONCLUSIONS AND RECOMMENDATIONS ......................................................... 30

Anticipated operations and modeling implications .................................................. 30

Econometric model of Palo Verde Prices ................................................................ 31

Long-term, large-scale risk and modeling implications ........................................... 31

The User Community and Process Issues .............................................................. 32
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 5

REFERENCES .......................................................................................................... 32

FIGURES 

Figure 1: Dispatch Follows Contract Load ............................................................................ 13 

Figure 2: Disptach Maximizes Resource Value .................................................................... 14 

Figure 3: Colorado River Storage Project Topology in GTMax ............................................ 17 

Figure 4: Southwest WECC and Western Purchase Prices ................................................. 24 

TABLES 

Table 1: GTMax Features and the Means for Implementing Each ....................................... 11

Table 2: GTMax Data and Modeling Constraints ................................................................. 18

Table 3: Production Cost Model Features Not Contained in GTMax ................................... 19

Table 4: Western Capacity Replacement Sheet .................................................................. 27
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 6

INTRODUCTION 

The Grand Canyon Monitoring and Research Center (GCMRC) convened a workshop on August 31

and September 1, 2011 in Flagstaff, Arizona to discuss the GTMax model.  The model has been used

by the Western Area Power Administration (Western) to model the economic effects of the

operation of a variety of river systems including the Colorado River Storage Project, the Glen

Canyon Dam and other facilities on the Upper Colorado River Basin.  Representatives attended the

workshop from Western, GCMRC, Argonne National Laboratory (ANL), developers of GTMax, and

various stakeholders with interests in the operation of Glen Canyon Dam. The panel, designated as

outside independent experts, was also invited to participate in the discussions:

 Dr. Edward Kahn, Special Advisor, Analysis Group, Inc., San Francisco, California

 Dr. Verne Loose, Senior Economist & Principal, Verne W. Loose Associates, LLC,

Albuquerque, New Mexico

 Dr. Michael Schilmoeller, Senior Power Systems Analyst, Power Planning

Division, Northwest Power and Conservation Council, Portland, Oregon

Our role as independent experts was to review the GTMax model and its uses for economic analysis

of issues relevant to the Glen Canyon Dam Adaptive Management Program (GCDAMP). This

document represents our findings and recommendations from the workshop.

PURPOSE OF THE WORKSHOP 

The purpose of the workshop was to review the capabilities and uses of the GTMax model toward

the investigation of possible changes in the operating regime of the Glen Canyon Dam (GCD).

Development of wholesale markets for electricity in various regions of the country provides

opportunities for timely bidding of generation resource owners to improve their profit position.

These new markets introduced “value‐based” compensation for generation, replacing the

“historical cost‐based” compensation of the regulated investor‐owned utility (IOU) model. The

evolution to multilateral wholesale markets is not complete. In particular, in the Rocky Mountain

west, the regulated IOU business model predominates. This is also true in the larger Western

Electricity Coordinating Council region (a.k.a., “Western Interconnection”), with the single exception

of a large portion of California in which the wholesale market business model is in effect. The

existence of this wholesale market in the WECC region presents opportunities for western

generation asset owners to increase returns to operation of their plants. The combination of

potential rapid response to fluctuations in the electricity supply and demand balance and the state‐

mandated renewable portfolio standards (RPS) has the potential to expand the opportunities for

profitable operation.
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 7

These institutional changes are important to Western, its customers, and the GCMRC, because they

mean that traditional methods of economic evaluation may need to be modified to reflect new

economic products and markets.  For a number of years, Western has been conducting much of its

economic analysis using the GTMax model. The purpose of the Flagstaff workshop was to review

the previous and proposed future uses of GTMax and assess the applicability of the model to

anticipated future economic and financial issues. To put these questions in proper perspective, it is

useful to first review briefly the nature of the changes in the organization of the electricity industry.

The next section, Three Business Models, provides this overview. Next, the section entitled

Review of GTMax reviews the basic structure of the GTMax model.  The section entitled

Observations and Findings discusses the suitability of GTMAX for the purposes identified here.  

Finally, the Conclusions and Recommendations section provides suggestions for enhancing the

analytical strength of Western’s tools and future studies for application in the Glen Canyon Dam

Adaptive Management Program.

THREE BUSINESS MODELS 

The exchange of electricity products occurs through various market structures that overlay physical

and operational organization of the grid. Areas that operate under the bilateral transaction

paradigm (i.e., the regulated IOU business model) are referred to as Traditional Scheduling Areas. In

contrast, formal markets generally operate in conjunction with an Independent System Operator

(ISO) or Regional Transmission Organization (RTO). The southeastern United States contains the

Southeast Reliability Corporation (SERC) and the Florida Reliability Coordinating Council (FRCC)

regions that have no formal markets, ISOs, or RTOs. The Western Interconnection represents a mix

of the two approaches since it contains two ISOs (the California ISO and the Alberta ISO) with

formal markets, while the rest of the Western Interconnection functions under the bilateral

transaction paradigm.

ISO/RTO with Formal Wholesale Markets 

ISOs and RTOs manage grid operations within their territories and also operate markets through

which energy, ancillary services, and capacity resources are procured.  Seven ISOs operate at the

present time in the U.S. (several extend operations into Canada). Like all Transmission Service

Providers, ISOs are required to file Open Access Transmission Tariffs (OATTs) with the Federal

Energy Regulatory Commission (FERC). OATTs define and implement market definitions and

operations and specify details of accounting and settlement procedures. Market participants within

these areas must file substantial technical and financial paperwork with the ISO, have their

generation assets tested and approved to meet the technical engineering standards required to
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 8

implement North American Electric Reliability Corporation (NERC) reliability standards, and

demonstrate that they are financially sound. Individual generator owners operating in formal

markets bid their services pursuant to a financial incentive as opposed to an obligation to serve

load.

Pure Traditional Scheduling Area 

The SERC and the FRCC regions are examples of typical traditional scheduling areas.  Both are

comprised of utilities that have more‐or‐less maintained their historical organizational. They are

regulated by state public utility regulatory authorities as well as by FERC and have an obligation to

serve the load within their defined territory. They schedule energy and power transactions, and

coordinate operations as well as system expansion planning to maintain grid reliability following the

same NERC criteria. Entities may engage in bi‐lateral trades with other adjacent utilities in order to

meet their obligations. No formal markets exist in these traditional scheduling areas.

Mixed Business Model among NERC Regional Entities 

The mixed business model is characteristic of NERC regional entities that contain a combination of

market and non‐market areas within their boundaries. The WECC represents a mixed model due to

the presence of formal markets under the California Independent System Operator (CAISO) and

Alberta Electric System Operator (AESO) with the balance governed by a system very much like that

in the southeast. The presence of formal markets in WECC, particularly the CAISO market, presents

opportunities and obligations to generation asset owners whose assets exist outside of the CAISO

borders. They are still used to meet load‐serving obligation within their balancing areas, but they

now also can bid their assets into the California market. There are also informal bilateral wholesale

markets in WECC at trading hubs such as Palo Verde and Mid‐Columbia.

As the competitive business model continues to expand into or influence non‐market areas, change

will continue to take place. This variety of physical, operational, and market organization structures

makes it difficult to generalize about the methods, procedures, and effects of hydro generation

participation now and in the future. The same can be said for integrating large amounts of variable

generation capacity into the existing capacity mix. However, one ameliorating factor tending to

simplify this otherwise complex situation is that fundamental economic behavior driven by cost

minimization provides a common foundation that underlies the behavior of participants in both

market and non‐market segments of the industry.

Formal electricity markets sharpen the profit incentives of all participants, including those that are

subject to some, or even considerable, regulation.  The role of formal markets is likely to grow in
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 9

the WECC region (E3, 2011), and Western will need to respond to these developments.1 One basic

set of questions addressed in our review concerns both how and how well GTMax can represent

both traditional cost minimization objectives as well as new market opportunities for increasingly

monetizing the value of the hydroelectric assets which it markets.   

Western’s Institutional Context 

The macro and regional economic and business context outlined above interacts with Western’s

business context. In addition, Western must operate within certain other constraints that affect its

relationship to its customers and stakeholders.  

Most of the hydroelectric facilities in the western U.S.—built with federal funding—were intended

to serve multiple purposes including irrigation, flood control, recreation, and electric power

production. These facilities are today managed by a number of federal agencies so as to achieve

these multiple purposes. All of the dams that comprise the CRSP are considered Bureau of

Reclamation (Reclamation) facilities. Management of the water flow is also conducted by

Reclamation but is constrained by water delivery and storage requirements, and by purposes

consistent with laws and statutes collectively known as “The Law of the River.” Reclamation

consults with the Colorado River Basin states in its setting of annual and monthly release volumes.

Western schedules hourly releases in conformance with the monthly volumes set by Reclamation

and to meet contractual obligations for the delivery of electrical power. Western’s power marketing

responsibility begins at the switchyard of the federal hydroelectric facilities and includes the federal

transmission system to interconnected utility systems with the rest of WECC.

CAISO allows generation asset owners outside the CAISO footprint to bid into its markets with

appropriate physical and financial representations. Indeed, California requires the energy and

capacity from these resources since it is short of generation from within its footprint.  

This presents an opportunity for generation resource owners in WECC with available capacity

beyond their “native” load. With available excess capacity, Western could bid into California

markets when either energy or ancillary services prices make it financially attractive. Timing is

critical in this. Generator asset owners must be prepared to react when day‐ahead and real‐time

prices diverge, presenting the opportunity for energy “arbitrage.” It is also the case that California

Ancillary Services (AS) market prices are among the highest of the seven ISOs, with regulation (i.e.

adjustment of electricity supply and demand variations over intervals up to five minutes) being the

highest prices of the services that include spin and non‐spin reserves. Thus, Western has the

opportunity to earn additional revenue if it can respond in a timely fashion.

However, this is not a totally unmitigated opportunity as there is concern among hydroelectric

engineers that cycling of turbine generator sets as required by AS market provision might result in

extra wear and tear upon the equipment and result in higher operation and maintenance costs as

 

1

 Recent discussion of an “Energy Imbalance Market” within WECC exemplifies such developments. 
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 10

well as shortened lifetimes. Thus, it is possible that costs might also increase with increasing

revenue. Also, decreased hydrology due to drought conditions and environmental constraints

implemented in the last 10 to 20 years have limited Western’s ability to generate enough electricity

to meet even its contractual obligations with its customers, much less have available capacity to sell

into energy or AS markets. Nevertheless, drought conditions can clearly change and if increased

water releases for power production can be accomplished within existing constraints on releases,

reservoir volumes, and pondage constraints and matched to price spikes in relevant markets,

Western has the potential to increase net revenues.

REVIEW OF GTMAX

GTMax belongs to a class of models representing the operation of electricity systems generally

known as production cost simulation models. There is a large literature on these models; see, for

example, Foley et al., 2010; Kahn, 1995; Arnedillo, 2011. Commercial examples of such models

include PROMOD, PLEXOS, and U‐PLAN. There are others. Reviewing and discussing this literature

or any of these commercial products directly is well beyond the scope of this study, Instead we list

the important features of GTMax and how those features are implemented in the model. See Table

1. All production cost models have these features but may represent them by different means. For

example, most production cost models do not have a very developed hydroelectric analysis

algorithm. GTMax is exceptional in this respect.

 
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 11

TABLE 1: GTMAX FEATURES AND THE MEANS FOR IMPLEMENTING EACH 

Capability or Feature How GTMax Implements Each Feature 

Typical application Determining the Incremental cost and river operation implications to 

Western and the CRSP of alternative policies and contracts 

Unique features Hourly integrated hydro-thermal model with stream and transmission 

linkages 

Objective Function Minimum variable system cost 

Solution Algorithms Mixed-Integer Linear Program optimized on chronological weekly 

operations 

State Variables Pumped Storage and forebay water levels 

Solution variables Thermal and hydro unit loading; transmission flows; market electricity 

purchases; thermal unit start-up; dam spill 

Fundamental period One hour 

Solution Period One week 

Study period User specified-1 week to multiple years typically using representative 

weeks for each month 

Types of Units 

represented 

Reservoirs with and without generation; run-of-river generation; pumped 

storage; fixed heat rate coal and gas-fired boilers and turbines; 

cogeneration 

Topology Zonal, at user’s option-typically limited to CRSP 

Transmission Algorithm Economic transport 

Principal output Total variable system cost and revenues from both Western customer 

and societal perspectives; unit operation, shadow price for hydroelectric 

generation; system lambda (short-term marginal power prices) 

Documentation Unknown 

Support and training Undetermined 

Cost Undetermined; free for federal agencies 
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 12

Typical application and unique features 

Developers create computer simulation models to solve particular problems or to address specific

inadequacies in existing models.  For example, the GTMax model evaluates weekly Colorado River

operation in the context of the regulated wholesale firm power markets.  Several commercial

models were available in the mid‐1990s that represented the participation of the thermal systems

in deregulated markets.  Such models had sophisticated calculations for transmission and unit

commitment.  None of them, however, had adequate representation for the Colorado River system

projects (CRSP).

Most commercially available production cost models have very simple representations of the

hydroelectric generation system.  The standard approach is to divide hydro resources into “base‐

load” and “load‐following” segments for each simulation period. The base load segment is

dispatched in all hours of the period. The load following energy is used to “peak shave” the

anticipated load profile up to the limit of hydroelectric capacity. The result of this is a flattened load

shape that is used to simulate the operation of thermal power plants. This standard heuristic is

sensible for systems with relatively small amounts of hydro generation. In systems like the CRSP,

however, this simple representation is inadequate.  Moreover, the developers of commercial

models have not provided the kind of detailed hydroelectric generation representation that

Western needed.   

Rather than any inherent difficulty, it is perhaps the unique nature of each system that makes the

task unprofitable for commercial developers.  Not only are the power generation characteristics

among dams dissimilar, but also each hydroelectric generation system is constrained by very

different non‐power considerations.  Both the flow through and the level of the forebay and tailrace

of each dam are constrained.  These elements are constrained for compliance with requirements

for fish and wildlife protection, flood control, navigation, preservation of archeological and cultural

treasures, and recreation, to name a few. Moreover, dams and thermal generation are tightly

coupled.  Energy cascades from a dam to all downstream dams and, in some systems; even

upstream dams can be affected.  The dams can store energy purchased or produced by any thermal

generation.   

Consequently, when Western was confronted with the need to evaluate and model the CRSP, they

collided with the hydroelectric generation limitations of existing models.  GTMax was designed to

address that inadequacy.

Objective function, solution algorithms, fundamental period 

By selectively purchasing and selling power, dispatching or refraining from use of certain resources,

and by storing or drawing energy from hydro‐electric generation, system operators can provide a

given level of service (e.g., energy generation) at least cost.  Minimizing costs, however, may not be

the most important objective.  In any case, all objectives have to be reduced to a single number for
a comput

value of t

GTMax is

possible o

shown to

Western

refer to a

customer

exposing

graphical

When We

permits t

this as “e

the perfe

dispatchi

prices are

idle, the m

 

2 Source:

ter model or

the objective

s typical in th

objective ref

o be essentia

constrains G

as a “financia

r loads.  The

Western to t

ly.2

estern does n

he model to

conomic mo

ect foresight a

ng a Western

e higher than

model will di

 

Veselka, 201

analyst to co

e function, a f

at it minimiz

erenced in p

lly the same

GTMax differe

l” study, the

purpose of t

the vicissitud

 FIGUR

not constrain

purchase an

odeling.”  Rem

assumption i

n generation

n the generat

ispatch the u

 

11, Slide 13

FIN

ompare and

function of a

zes cost subje

published des

problem.

ently for diffe

y are constra

his kind of st

des of marke

RE 1: DISPATCH

n net generat

nd sell in the

moving const

implicit in th

n unit, the mo

tion cost of a

unit to sell int

NAL REPORT

rank alternat

all the variou

ect to constr

scriptions of

erent studies

aining net ge

tudy is to find

et purchases

H FOLLOWS CO

tion and pur

market to fu

traints can on

ese models.

odel will disp

a Western ge

to the whole

T OF THE GT

tives plans.  T

s variables a

aints.  Maxim

the GTMax m

s.  When We

eneration and

d the most e

and sales.  F

ONTRACT LOA

chases to me

urther minim

nly improve

If purchases

place the unit

eneration uni

esale market.

TMax MODEL

That single n

nd constrain

mizing revenu

model, can m

stern perfor

d purchases t

economic ope

igure 1 illust

D 

eet custome

mize cost.  We

the econom

s are less exp

t.  If wholesa

it that would

.  This is illust

L REVIEW PA

Pag

number is the

nts.

ues, another

mathematical

ms what the

to meet

eration witho

rates this

r loads, West

estern refers

ic outcome, g

pensive than

ale power ma

d otherwise b

trated in Figu

ANEL 

ge 13

e

r

lly be

y

out

tern

to

given

arket

be

ure 2
(Veselka,

which is a

constrain

Figure 1.

Comparin

the policy

Many alg

programm

suitable o

function o

cases, suc

address r

A particu

power pla

efficiency

offset any

 

3 Source:

2011, Slide 1

above deman

nt that genera

ng the results

y that isolate

gorithms exis

ming (LP) dat

only for parti

of the variab

ch as quadra

remains limit

lar attribute

ants incur sig

y.  If they can

y losses due

 

Veselka, 201

14) below. 3

nd. This capt

ation should

s of “financia

es Western fr

t for minimiz

tes from the

cular kinds o

bles with line

tic objective

ted.

of the CRSP

gnificant star

n make enoug

to their start

FIGURE

 

11, Slide 14.

FIN

Figure 2 show

ures the valu

match dema

al” and “econ

rom market p

zing the obje

1940s and is

of problems;

ar constraint

functions, b

does not len

rtup cost and

gh money ov

tup or opera

E 2: DISPTACH 

NAL REPORT

ws Western’

ue of high pe

and results in

nomic” studie

price excursi

ctive functio

s among the

specifically,

ts.  Techniqu

but the kinds

nd itself well

d, once comm

ver high‐valu

tion during lo

MAXIMIZES RE

T OF THE GT

’s generation

eak prices. By

n the lower p

es gives Wes

ons.   

on of such co

most familia

when the va

ues were dev

of problems

to simple lin

mitted, may n

ue hours, how

ow‐value ho

ESOURCE VALU

TMax MODEL

n going as hig

y contrast, im

peak generat

stern an idea

mplex system

ar of these.  H

alue of the ob

eloped for ad

s that unador

ear program

need to run a

wever, their o

urs.

UE 

L REVIEW PA

Pag

gh as 900 MW

mposing the

tion shown in

a of the cost o

ms.  Linear

However, it is

bjective is a l

ddressing sp

rned LP can

mming.  Certa

at less than i

operation ca

ANEL 

ge 14

W,

n

of

s

inear

ecial

ain

deal

n
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 15

The commitment of generating units is not a linear or even a continuous variable.  The problem falls

under the category of "integer programming” which seeks to minimize or maximize the objective

function of integer variables; in this case, variables having only two values: 0 (not committed) and 1

(committed).

Mixed integer LP algorithms provide the capability to minimize costs when some of the variables

are integers.  GTMax uses the most popular mixed integer LP algorithm, the Lagrangian relaxation

technique.  As we discuss shortly, this algorithm is often used in power generation production cost

simulation models for other purposes.  For example, it can be used to find the least cost solution for

security reserves commitment.  GTMax, nevertheless, uses the mixed integer LP only for minimizing

dispatch costs.

Finally, the topic of security reserves commitment is closely tied to the fundamental simulation

period in this model.  Dispatch commitments are made for the full hour.  One hour is also the

shortest time period the GTMax model "sees."  This means that spinning and operating reserve for

contingencies lasting less than one hour, sub‐hourly load following, regulation capability,

incremental and decremental reserves, and many other ancillary services are outside the scope of

GTMax simulation and evaluation.

Solution period, state variables, and solution variables 

Because the GTMax model can represent commitment of power generating units in certain hours to

capitalize on opportunities in other hours, it is natural to ask over what period costs are minimized.  

For GTMax, this period is one week, which corresponds to a cycle for loads and market prices.  It is

typical to commit thermal generation on Monday morning for use over the weekdays.  Wholesale

power prices typically have their highest values during weekday on‐peak hours.  At the end of the

workweek, these power generation units might be taken off‐line to save money on the fuel that

would otherwise be consumed to maintain the power plants in a standby mode.

However, the total cost is also affected by the initial conditions for certain variables, referred to

here as "state variables."  Attention must be given to how much energy is behind each of the dams

and pumped storage facilities at the beginning of a simulation.  At the end of each solution period,

the model must perform necessary bookkeeping to assure that the initial conditions in the

subsequent solution period are correct. Therefore, at the end of each solution period the GTMax

model performs necessary bookkeeping.

Study period 

Typically, analysts and decision makers are concerned with operation of the river over time periods

of weeks to years.  Seasonal effects and cycles in commodity prices and weather and probable

changes in other future circumstances compel problem solvers to consider the long‐term

implications of short‐term decisions.
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 16

When Western considers questions that extend from months to years, GTMax takes operation over

the typical week as representative for the month.  Consequently, 12 one‐week simulations from

hourly data comprise an annual study.  For studies over multiple years these annual representations

are taken from either continuous years or from years at regular intervals over the study horizon.

This approach is one of the standard techniques for reducing computational burden.  Other models

might elect to sample between three and eight hours in a day, but that approach sacrifices the

ability to represent ramp rate and other dispatch constraints.  As long as operation over all the

weeks in a month is uniform and state variables are updated correctly, the GTMax approach should

be reasonable.

Types of units represented 

GTMax appears to have adequate representation for the kinds of generation in the CRSP system.  It

is not clear whether the model can represent the full complement of resource technologies in the

WECC.  There appears to be very little representation of energy efficiency or demand side resources

beyond those that are captured through simple adjustments to hourly loads.

Transmission topology 

Western has the capability to model transmission within its service area.  GTMax typically

represents the interface of Western with the rest of the WECC as purchases and sales of wholesale

electricity at exogenous prices provided by the model user.  The topology of the model is illustrated

in Figure 3 below.  As a practical matter, however, the topology in Figure 3 appears to be used

infrequently at best in the studies conducted by Western and ANL. Instead, the default

representation is simply a single price node representing the Palo Verde market hub where all

purchases and sales take place. The ongoing usefulness of this simplified default representation was

discussed at the workshop, and is addressed in more detail in the next section.  
Princi

GTMax p

optimize

added re

informati

sold on a

power pla

managem

determin

capacity c

Key e

The list in

amount a

impossib

FIG

ipal outp

roduces fina

the value of

porting capa

ion such as w

n hourly bas

ant operatio

ment program

ne whether a

credits, and a

xogenou

n Table 2 iden

and detail of

le.  Neverthe

GURE 3: COLOR

put 

ncial reports

the system's

bilities. Data

which units sh

is, when to b

ns, the incre

ms. GTMax a

n investmen

ancillary serv

us data 

ntifies some

data require

eless, certain

FIN

RADO RIVER ST

s, which help

s resources. T

a from GTMa

hould be disp

buy and sell p

mental value

lso gives fina

t is financiall

vices.

and con

of the key da

ed by produc

data depend

NAL REPORT

TORAGE PROJ

dispatchers

The model a

x are presen

patched, how

power on the

e of water, a

ancial market

ly viable, give

nstraints

ata provided

tion cost mo

dencies meri

T OF THE GT

JECT TOPOLOG

identify ope

lso allows us

nted in tables

w much pow

e spot marke

nd the value

t clearing pri

en the preva

s

d to the mode

odels makes a

it discussion.

TMax MODEL

GY IN GTMAX 

erational stra

sers to define

s and graphs,

er should be

et, the cost o

e of demand‐

ces, which ca

iling market

el for a simu

a complete d

.

L REVIEW PA

Pag

tegies to

e regions for

, providing

e generated a

f alternative

side

an be used to

rules for bid

lation.  The

description

ANEL 

ge 17

and

o

ding,
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 18

TABLE 2: GTMAX DATA AND MODELING CONSTRAINTS 

Key Input Data and Constraints 

Electric Requirements including losses (loads) 

Cogeneration thermal demand 

Irrigation requirements 

RiverWare forebay and stream inlet flows for alternate hydro 

conditions 

Fuel prices, O&M, spot and forward prices for power market 

purchases and sales 

Thermal start-up and minimum generation costs 

Optionally, contract data for Western customer studies 

Generating unit characteristics (e.g., heat rates, capabilities) 

Constraints on unit ramp rates, stream and reservoir levels, 

stream and reservoir level rates of change, power transmission 

flows 

Western relies on inputs from Reclamation’s RiverWare model for hydrologic modeling.  The

general structure of RiverWare is described in Zagona, et al (2001). As we understand it,

Reclamation uses this model to provide Western with forecasts of monthly water quantities

available for discharge through the Glen Canyon Dam. Reclamation may revise these forecasts over

the year as hydrologic conditions unfold. With a monthly water allocation as input GTMax then

simulates operation of the Glen Canyon Dam and associated facilities according to various economic

objectives and operational constraints. The economic factors that might affect the monthly

allocation of water across the year are opaque. Casual observation of various GTMax results

suggests that RiverWare has some notion that water is more valuable for power generation during

the summer and winter months than during the spring and fall.  

Western has also conducted studies that explicitly address hydrologic uncertainty. It is our

understanding that in these studies, the complete hydrologic record is viewed as a probability

sample from which representative elements are chosen to represent average, wet, and dry

conditions. One example of this approach is discussed in Loftin (2011). This procedure ignores the

possibility of climate change. Under a climate change hypothesis, some sample selection method

would need to be used to determine a going forward relevant range of hydrologic variation that

would be different, arguably drier, than what would be reflected using the complete hydrologic

record. We discuss this point further below.
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 19

Typical Production Cost Model Features Not in GTMax 

Table 3 lists important features found in many production cost models that are not contained in the

version of GTMax reviewed at the workshop. These features are important in that they are

necessary to conduct some of the types of analyses that Western is considering.  

TABLE 3: PRODUCTION COST MODEL FEATURES NOT CONTAINED IN GTMAX 

Capacity expansion algorithm None-staged after WASP for certain studies 

Reserves commitment 

algorithm 

None-capacity de-rated for energy effects 

Ancillary services algorithm None-capacity de-rated for energy effects 

Unit reliability algorithm None-capacity de-rated for energy effects 

Long-term risk algorithm None-Users typically perform scenario 

analysis on hydro conditions, power prices 

Capacity Expansion Algorithm 

Many, but by no means all, production simulation models have the ability to add generation units

within the study period, using an internal calculation.  There are several reasons why this is

valuable. Sometimes the question of generation capacity expansion is the central question.  Utilities

or marketers may want to know what kind of generation they need to add to their system in the

future, and how much capacity that new generation should provide.  Capacity expansion algorithms

typically choose from a portfolio of resources and endeavor to find the optimal timing and sizing for

each unit.

There are other reasons for having capacity expansion capability in a simulation model.  Without

such capability, simulated market prices for wholesale firm energy would reach unsustainable levels

as energy requirements grew or fuel prices changed.  These would become unsustainable because

substitution effects (new generation, demand management, or economic curtailment, etc.) would

prevail.  Valuation of power plants without economic capacity expansion in this case would be

invalid.

The proper valuation of capacity from generation is related to this issue. There are many different

types of capacity and many different ways of valuing capacity.  However, any kind of capacity will

have values that vary with supply and demand over time.  Estimating the capacity value without a
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 20

well‐established retail market is difficult and to some degree arbitrary.4 This is illustrated below in

our discussion of one Western/ANL study.

A significant advantage of a capacity expansion algorithm, however, is that endogenous capacity

expansion will capture the changing value of capacity over time.  This is a feature that many of the

rules‐of‐thumb for capacity value, such as the construction cost of the simple cycle turbine, do not

recognize.

Finally, while capacity expansion algorithms have significant advantages, most share one significant

disadvantage.  To minimize total costs over an entire study period, they must have perfect foresight

of future commodity prices and loads.  We return to the subject when we discuss long‐term risk

algorithms, below.

Reserves commitment and ancillary services algorithms 

In the discussion of objective functions and state variables, we point out that GTMax fails to make

commitments for security reserves, even at the hourly time step.  Instead, GTMax de‐rates

hydroelectric generation capability to capture the cost to provide for these reserves.

The increasing importance of variable generation resources, such as wind and solar, have made

ancillary services more valuable than ever.  While it may have sufficed in the past to merely reflect

the cost of producing these services for the native generation and load, it is becoming important to

also estimate the value of these services for possible resale.

As previously noted, GTMax cannot simulate operational periods of less than one hour duration.  

There are other models that are capable of sub‐hourly simulation.  These models can value many

kinds of ancillary services because they can simulate operation down to five‐minute intervals.  (Even

these simulation models, however, cannot deal with regulation requirements that vary from second

the second.)  These models are often used to create better operating heuristics.

Depending on the specifics of turbine design, hydroelectric generation can have distinct advantages

in providing ancillary services.  It often can operate in a range where generation efficiency does not

degrade.  While almost all generation technologies will incur more stress and wear adapting to

rapid changes in output, hydroelectric generation often incurs less operation and maintenance

costs per unit of capacity than thermal generation.  Often, the most significant cost associated with

hydroelectric generation providing operating reserves is an opportunity cost.  Incremental capacity

for ancillary services requires units to reduce firm energy generation.   Because incremental

operating reserves are often most valuable during on‐peak hours, on peak firm energy is effectively

shifted to off‐peak hours, when it is less valuable.

 

4 Capacity valuation cannot truly be made in the absence of some estimate of consumers’ “value of lost 

load.” 
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 21

With Western and the WECC entering into a period of increasing generation from renewables and

therefore also possibly the need for ancillary services, the absence of ancillary service valuation

capability is conspicuous.  While GTMax would not necessarily have to be modified to provide

Western and its constituents with this value, some valuation mechanism is certainly necessary to

studies Western will be performing.

Unit reliability algorithm 

Many production cost models provide detailed simulation of unit forced outages. The NERC

Generation Availability Data System (GADS) database provides operating history for units in the

United States and abroad.  The history contains important reliability data, such as mean time to

failure and mean time before recovery for classes of generation technologies. Simulations of energy

generation and cost are more meaningful if they recognize unit forced outages.  Such simulations,

for example, reflect the consequences of relying on a single, large power plant versus an ensemble

of smaller, independent plants.

An alternative to the detailed simulation approach is capacity deration.  This is the approach

adopted by GTMax.  The capacity deration method assumes that there are a sufficient number of

small contributions in an ensemble of generating units that the system as a whole is quite reliable.  

That is, the distribution of energy produced by the ensemble is narrow and has a mean value close

to the capacity at ensemble, discounted by the average amount of energy that would become

unavailable due to forced outages.

The capacity deration algorithm is quite helpful in situations such as the modeling of market prices

at trading hubs in the WECC.  None of the units in the WECC, except perhaps the nuclear units,

impose significant individual risk.  Since one of the assumptions is that all of the forced outages are

independent, there should be small forced outages occurring almost continuously with an average

effect that closely approximates a de‐rating of capacity.

For studies of the revenue impacts of Glen Canyon and Hoover dams on Western, however, we

need to reconsider this assumption.  These two facilities have considerable generation capability

relative to Western’s requirement.  Units associated with the dams have considerable common risk.  

For example, the failure of a substation transformer or transmission line at one of the dams takes

out the entire project.  We therefore question the application of capacity de‐rating for modeling

forced outages in representing Western’s system.

Long-term risk algorithm 

Utility planners often deal with risk by using two separate processes.  For short‐term variation in

commodity prices, an anticipated level of unit forced outages, and the impact of weather variation

on loads, planners use what they would call stochastic modeling. They often describe distributions

using historical data.  Typically, they tie their mean reverting stochastic processes to a single,
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 22

underlying benchmark forecast. For long‐term risk, to which they invariably refuse to assign

probabilities, planners use scenarios.5

GTMax has little short‐term risk simulation capability and virtually no long‐term risk simulation

capability. Nevertheless, Western and ANL have introduced some long‐term risk analysis by the use

of hydro‐electric generation condition scenarios and power price futures.  These, however, are

mean‐reverting processes that tend to understate risk.

Another limiting factor is the perfect foresight and perfect information usually implicit in

commitment and dispatch.  Operators never misjudge the weather or fail to anticipate fuel and

power price excursions in such models. This deficiency in GTMax is shared by almost all production

cost simulation models.  Models often need perfect foresight to arrive at solutions for capacity

expansion and dispatch.  Perfect foresight precludes any consideration of significant risk.  We return

to a discussion of the perfect foresight assumption later in this report.

OBSERVATIONS AND FINDINGS 

This section of our report details particular issues considered important to the future and ongoing

use of GTMax. In some cases, they lead to recommendations that are elaborated in the concluding

section.

Energy Market Prices 

For historical studies based on “economic modeling,” ANL and Western rely on actual daily prices at

the Palo Verde hub. These prices reflect bilateral trades, rather than organized markets, and are not

typically available on an hourly basis, which is the time step for GTMax dispatch simulation.

Therefore, some processing of the available data is required. This is discussed in the ANL report on

the Post‐ROD Analysis in Sections 3.4.1, 3.4.2 and 3.5.2 (Veselka et al., 2010).   

For “financial modeling” studies, ANL and Western rely on the actual prices paid by Western. Actual

and historical daily prices differ because Western hedges the large majority of its load with

contracts, so as not to be exposed to the volatility of daily prices. There can be substantial

differences between the actual Western contract prices and the daily prices. This is illustrated

below in Figure 4 (Veselka, 2011, Slide16).  

The “spiky” pattern in Figure 4 reflects daily price volatility. The contract price path exhibits “step‐

wise” discrete changes, reflecting the more stable pricing of contracts, whose price duration

 

5

 The form of scenario analysis popularized by Pierre Wack, Royal Dutch Shell’s London headquarters, in 

the 1970s also ignored probabilities. The purpose of their exercises, however, was to raise 

consciousness about potential, large-scale changes in their industry. Like a good military planner, Wack 

encouraged management to develop various contingency plans to implement in the event that one of 

these futures would come to pass.
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 23

exhibits varying length, and changes in level reflecting newly purchased blocks of contract power

and the expiration of previous contracts. Just after the California energy crisis of 2000‐2001 there

was a huge mismatch caused by Western’s hedges that were purchased just before the pricing

distortions in the daily market disappeared. Other market participants, such as the California

Department of Water Resources, experienced similar if not worse problems of this kind at this time.

There are several issues associated with using Palo Verde prices.  First is the question of whether

these prices are invariant to the policy and dispatch changes that Western is examining in its

historical studies.  All of the economic studies assume the same prices regardless of operational

changes at the Glen Canyon Dam.  The underlying assumption is that whatever changes might

possibly occur, they are too small to matter given the large size of the market. This assumption may

probably be true, but it is an assumption that should be examined. Second, for forward‐looking

studies the historical data may not be useful.

Western and ANL are proposing that an enhanced GTMax topology, based on the entire WECC

region be used to address both of these issues. Given the computational complexities associated

with a “full” representation of WECC, the proposed configuration would use 23 “zones” to model

the regional market. ANL has used such a representation in studies conducted on behalf of the US

Department of Energy. Other simplifications to the modeling would also be necessary.6 The

underlying assumption in this proposal is that simulation modeling is the preferred way to study

pricing.

 

6 Without going into too much detail, one important issue involves how the enhanced presentation would 

handle the commitment of thermal generators in the region. GTMax can represent thermal unit 

commitment in “small” systems by doing an exact calculation. For large systems the exact solution is 

unwieldy. Proprietary commercial production simulation models use various approximations and 

heuristics. The effect of this issue on the estimation of market clearing prices can be significant. There is 

an extensive discussion of these topics in Kahn (1995). 
An altern

is based o

statistica

independ

Barmack

Verde an

nuclear p

the obser

examined

introduce

authors f

leads to a

Columbia

variables

focusing o

Econome

imprecise

prices wit

Western.

ative approa

on economet

l representat

dent drivers o

et al. (2008)

d the Pacific

plant availabi

rved electrici

d closely, and

ed. The cong

ind is that at

about a 2% d

a prices, since

are used in t

on the Pacifi

etric models a

e estimates. T

thin the mod

The relevan

FIGURE 4: SO

ach to unders

tric methods

tions of mark

of an econom

model day‐a

Northwest M

lity, and regi

ity price. In t

d measures o

estion measu

t the Palo Ve

rop in price.

e hydropowe

the Palo Verd

c Northwest

are not a “m

Their advant

del. This cont

t question is

FIN

UTHWEST WEC

standing pric

s.  When app

ket prices ba

mic price form

ahead on‐pe

Mid‐Columbi

onal loads as

he Palo Verd

of transmissio

ures improve

rde hub, the

This paper u

er is so impo

de models.  W

.

agic bullet” i

tage is that th

trasts with th

s whether the

NAL REPORT

CC AND WESTE

ces at Palo Ve

lied to marke

sed typically

mation proce

ak prices for

a hubs. They

s the indepe

de models, th

on congestio

e the perform

availability o

uses hydrolog

rtant in the P

Woo et al. (2

n that they c

hey incorpor

he “point‐est

ey do a bette

T OF THE GT

ERN PURCHASE

erde, or any

et processes

y on a linear r

ess and the r

r the 16‐hour

y use the dai

ndent explan

he role of the

on between C

mance of the

of an additio

gic variables

Pacific North

2007) is anoth

can have sign

rate the stati

timate” appro

er or worse jo

TMax MODEL

E PRICES 

other electri

s econometri

relationship

esulting obse

r daily peak p

ly spot natur

natory variab

e California m

California and

e model. One

nal 1000 MW

in the mode

west.  No hy

her paper of

nificant issue

stical distrib

oach describ

ob at estima

L REVIEW PA

Pag

icity market

c models are

between

erved price.

period at Palo

ral gas price,

bles that prod

market is

d Arizona are

e result the

W nuclear un

els of the Mid

ydrologic

f this type

es and result

ution of obse

bed above fo

ting prices th

ANEL 

ge 24

hub,

e

o

duce

e also

nit

d‐

in

erved

r

han
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 25

production simulation models. Arnedillo (2011), in a context assessing market power in electricity,

argues that the perfect information assumption of production simulation models and other

simplifications make them poor predictors of electricity market prices. We will return to this topic

when we make recommendations.

Transmission topology and the WECC 

Transmission plays a vital role in power systems. Transmission congestion is said to occur when the

demand to move power from one part of the grid to another exceeds system capacity.  In electricity

markets transmission congestion means price separation between the low price export region and

the higher price import region. The Glen Canyon Dam is located in a region that primarily exports

power to the west. So if congestion occurs east to west, then the price of power from the export

zone declines compared to the price in the import zone.   

GTMax has the capability to represent the network topology of the CRSP region. Figure 3 above

illustrates this representation. Most studies by western and ANL using GTMax do not use this

topology. Instead they assume that there is never any transmission congestion, so one price is good

for all power produced in a given hour. Absent this assumption, then congestion would produce

different prices for Glen Canyon Dam output and power at Palo Verde, or other locations in the

CRSP region. The only study described at the workshop using the network topology of Figure 3 was

a “financial” analysis for all Western customers, and it used contract prices, which are presumably

constant whether congestion occurs or not.  

Ancillary Services 

Ancillary services refer to specific electricity products traded in formal electricity markets to

maintain system reliability in the face of potential disturbances. Some of these services provide very

short‐term energy balancing to maintain system frequency in its prescribed bounds. This is usually

called “regulation.” Other products are reserve capacity that can be available on very short notice

to meet unexpected conditions that may arise. Spinning and non‐spinning reserve are two such

products. All of these functions are self‐provided by utilities engaged in traditional scheduling. In

formal markets these services are unbundled, procured by the ISO/RTO, and priced through auction

mechanisms. Each service has different costs, typically opportunity costs that must be weighed

against their value as reflected in the market price.  

We have previously asserted both that hydro‐electric generation is particularly well‐suited to

provide ancillary services and that the demand for these services is likely to increase as the power

system relies more on variable sources of generation, particularly wind generation. Both points are

well illustrated by the recent evolution of the electricity industry in Spain. The Spanish power

system has been operating as a formal market since 1998. Policy initiatives to encourage wind

generation in Spain have been quite successful. By 2003, about 5% of electricity generation came

from wind. By 2010 the share of wind generation had risen to 16%. Between 2003 and 2010

demand increased by a bit more than 20%. The demand for ancillary services doubled during this
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 26

period. The increase in wind generation is the main cause of the increased demand for ancillary

services. Hydroelectric generation capacity is currently about 17% of the capacity in Spain. For some

of the reserve services, hydroelectric generation capacity provides more than half of the demand.7

For most of these services, hydroelectric generation has a share much greater than 17%. There is

nothing so unique about the Spanish market that would prevent the same factors from playing out

in the WECC region.

Capacity value—lack of conceptual clarity 

Among the more confusing topics in electricity economics is the notion of capacity and its value.

Electric energy is a straightforward and familiar notion. Capacity, on the other hand, is the

instantaneous ability to produce power, but not the power itself. In most of the ISO/RTOs there are

“capacity” products of different types. Some markets include payments for “Installed Capacity,”

adjusted typically for its reliability. Most markets also make payments for ancillary services capacity.

These payments are typically based on the results of an auction process. In the case of ancillary

services, what counts for capacity is the ability of generators to adjust output and respond in fairly

rapid (i.e. sub‐hourly) timescales.  These are different, indeed much lower numbers, than what

would count for installed capacity. Furthermore, different generating technologies are very

different in how much and how quickly they can change output levels. For reasons such as these,

some analysts find that using the same word “capacity” for these different functions isn’t helpful.  

The Pacific Northwest Utilities Coordinating Council (PNUCC), for example, has a recent white paper

on this topic where they propose to rely on the term “capability” as an alternative (PNUCC, 2011).

In economic studies, it is often necessary to take account of “capacity” effects. In most cases, the

context of the study will provide some indication of what is meant by the term, but not necessarily

complete clarity.    

One example of this ambiguity arose in the workshop discussion of the Post‐ROD analysis. There

was a numerically important effect of a high value proxy chosen for capacity cost in that study. ANL

presented a sensitivity analysis on 1 September that showed capacity costs dominating energy costs

after the California Energy Crisis effects on energy prices were removed (Slide 14 vs. Slide 12)8

.  

Table 4 shows a menu of choices for capacity values in one ANL spreadsheet file.

 

7 The factual basis of this discussion is documented in the various reports of the system operator in 

Spain, Red Electrica de Espana (REE). Their Annual Report provides data on installed capacity, energy 

production and ancillary services among other things. These reports are available at: 

http://www.ree.es/sistema_electrico/informeSEE.asp. For 2010 data on ancillary services, see REE 

(2011). 8

 Veselka (2011) slides 12 and 14. Table 4 has been reproduced (copied) from the slide version. 
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 27

TABLE 4: WESTERN CAPACITY REPLACEMENT SHEET 

It is our understanding that ANL used the natural gas combined cycle (NGCC) option to value the

capacity changes associated with the post‐ROD scenario. Clearly they recognize that other values

could have been used. Choosing the “replacement” value of $74 per kW would have changed the

conclusions of the Post‐ROD study substantially. It appears from  

Table 4: Western Capacity Replacement sheet that the low capacity value was used by Western in

some of the studies discussed in Loftin (2011), where the MLFF Scenario, for example, was

modeled. At a minimum, some discussion of the reason for choosing particular values for one study

versus another would be desirable.

GCD EIS 

MLFF Scenario Replacement Coal NGCC Technology 

74 1,511 633 Capital Cost ($/kW) 

1991 2004 2004 Capital Cost Base Year 

1 0 0 Unit Capacity (MW) 

1.25 10 10 Discount Rate (%) 

1 40 30 

Technology Lifetime 

(yrs) 

2000 2000 2000 Year Constructed 

0 25.07 10.65 Fixed O&M cost ($/kW) 

2009 2009 2009 New Base Year 

Capacity 

Replacement (DH) Gas Turbine Coal NGCC Technology 

(use for woROD 

scenario) 424 1,511 633 Capital Cost ($/kW) 

2004 2004 2004 Capital Cost Base Year 

-524 277 0 Unit Capacity (MW) 

10 10 10 Discount Rate (%) 

25 40 30 

Technology Lifetime 

(yrs) 

2000 2000 2000 Year Constructed 

9.59 25.07 10.65 Fixed O&M cost ($/kW) 

2009 2009 2009 New Base Year 
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 28

RiverWare 

Water flow is the critical input to the ability of the CRSP facilities to generate enough electricity to

both meet native load and have sufficient energy to participate in developing WECC markets.  The

use of RiverWare then is, or should be, subject to scrutiny along with the GTMax model.  RiverWare

is itself a combination simulation and optimization model with many user‐specified options as to

modeling approaches and algorithms that may be employed to model the water flow in a particular

basin.  Physical processes in question are modeled according to specific algorithms or methods the

user may select based on time step size, data availability, desired resolution, or institutional need.

For example, each of the power generation, tail water calculation, and reach routing processes may

be modeled using up to six different methods. For the RiverWare output used as input to the ANL

modeling the combination of methods actually used is unknown.  Furthermore, there may be

disagreement over specific methods applied to particular basins. Beyond hydrologic questions

about the appropriate modeling of river basins lie the questions of long‐term climate change that

may have dramatic effects on water flow and on competing uses for water. Models populated using

historical data may be inadequate to address the full range of these potential effects. Indeed,

recent experience indicates that competition for water is intensifying in the desert southwest and

that its supply has been below historical levels; this competition must be considered in

development of long‐term plans for water use.  

Large-Scale and Long-Term Risk 

The CRSP faces significant strategic risk in the form of potential climate change, diversions from the

river, increased summer temperature and penetration of air‐conditioning, new legislation, changes

in market structure, the regulation of fuel sources, carbon emission penalties, and technological

innovation.  These would directly or indirectly affect river and stream flows, customer

requirements, fuel availability and prices, market prices for wholesale energy and ancillary services,

availability and value of particular resources.

The GTMax model and Western’s analytical framework is designed primarily to evaluate short‐term

operations.  The authors of GTMax excluded long‐term planning features, like long‐term capacity

expansion capability.  One of the key input data sources for GTMax is the hydro‐electric generation

provided by the RiverWare model.  As mentioned in the previous section, Western takes these data

from Reclamation and therefore is not situated to modify hydro‐electric generation for climate

change or river diversion studies.  The design does not facilitate convenient or efficient evaluation

of alternative assumptions and futures.  The model does not have the geographic scope to study

the possible consequences of policy changes in other parts of the WECC.

The GTMax model performs well the jobs for which it was designed, i.e., assessments of short‐term

operational trade‐offs.  It may well be impractical or even counterproductive to design a model that

handles both short‐term operation and long‐term risk assessment.  Nevertheless, long‐term risks

assessment and management are important missing pieces of the planning Western should be

performing.
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 29

Economic and Financial Studies 

The sections Objective function, solution algorithms, fundamental period, and Energy Market Prices

above, describe the distinctions that Western draws between “Financial Studies” and “Economic

Studies.”  These differences are primarily in the treatment of loads and prices for purchased power,

both market and contract power.

Western appears to be conflating its evaluation of market exposure risk with financial reporting,

especially in its treatment of load.  The load following that Western illustrates in Figure 1 for its

Financial Studies and the prices that they adopt for these studies are characterized as required for

rate analysis and reporting purposes.  These conventions are outside our experience with other

power industry agencies and companies.  We do not understand them and suspect that they may

be reaction to events that proceeded from some market exposure during the energy crisis.

More standard language would help Western be more effective in communicating analysis and

results with stakeholders and customers.

The User Community and Process Issues 

Western uses GTMax for daily scheduling and for special purpose studies that examine various

policy choices. For policy analysis it is important that stakeholders understand the modeling process

and the choices made in performing particular studies. If the modeling process is perceived to be a

mysterious “black box” then acceptance of results is impeded. One purpose of the workshop was to

improve transparency so that stakeholders can contribute in a meaningful manner to future policy

discussions.  

It is an open question as to exactly what level of transparency is appropriate.  At one extreme, one

might imagine that particular stakeholders would run the model themselves to examine cases of

interest to them. This is probably exaggerated. The costs of such an alternative would be high in

both time and resources (stakeholders might need to use consultants to achieve the goal of

independent modeling).  One intermediate option would be to find a way to make modeling results

more understandable to stakeholders than is presently the case. It is often the case that one or two

critical assumptions determine the qualitative features of studies on a given topic. When the choice

of such critical inputs is subject to ambiguity, it can be useful to focus stakeholder discussion on

that choice. One example illustrated in our review, and discussed above, is the choice of capacity

value.  

It can be challenging to determine ex ante what the critical input assumptions are in a given case.

One way to aid the process of knowledge generation by stakeholders is to make diagnostic reports

available. Then anomalous or otherwise troubling results can be traced back to particular

assumptions or calculations. We had no discussion at the workshop of the extent to which GTMax

can generate diagnostic reports. Most commercial models have this capability. It should not be too
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 30

difficult to add this capability if it does not already exist. Similarly, written documentation of the

GTMax model would be useful. This topic was discussed at the workshop, but we have not seen

anything beyond what was presented and provided by the speakers for the workshop. 

Non-power values: reducing these to hydro constraints is 

the best that it can do. 

Finally, it should be clear by now that GTMax functions in a limited domain of the analysis space

that may be of interest to the GCDAMP. Many questions of interest can be examined through the

GTMax lens if they are formulated in a way that the model can handle. In practice this will amount

to running sensitivity cases where policies of interest are reduced to constraints on the operation of

the GCD. The problem formulation and specification will occur outside of the model, guided by the

knowledge and perspective of stakeholder groups.  

CONCLUSIONS AND RECOMMENDATIONS 

Anticipated operations and modeling implications 

Western may want to consider developing the conceptual framework and quantitative tools to

better prepare for its participation in ancillary services markets.  It is fundamental to have a clear

understanding of what these services are and from where they derive their value.  Western may or

may not already have this understanding.  What is evident, however, is that there is no consensus

on these concepts or their valuation among stakeholders and observers.  The lack of clarity and lack

of consensus on capacity value is one example.

Having built consensus understanding, Western should evaluate their participation in ancillary

service markets.  Even if participation in formal markets is not an objective that stakeholders share,

Western nevertheless needs to value the services.  Western stakeholders and customers need to

understand the tradeoffs that they are making in electing to participate or to refrain from

participation in these markets.

It is also right and proper for Western to investigate opportunities presented by its geography, that

is, the capability to bid into CAISO markets and enter into bilateral contracts with other entities.  For

example, Western should investigate energy arbitrage between day‐ahead and real‐time markets.  

Western, its customers, and its stakeholders may conclude that pursuit of such opportunities is

imprudent.  Nevertheless, they again need to know the tradeoffs these opportunities present in

terms of rewards and risks.  Consequently, they need appropriate tools to perform such studies.

This leads to our recommendations about transmission topology and modeling the interface with

the rest of the WECC.  This is clearly an area that needs enhancement.  Western needs to be able to

assess both the short‐term and long‐term effect of actions by Western and the rest of the WECC on

each other.   We believe that there are adaptations that would not require Western to abandon

GTMax.  It may be possible to perform special studies using other, more extensive models to
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 31

develop rules of thumb and qualitative understanding of the impact of particular actions.  As

explained in the next section, it may be possible to develop econometric models that capture some

of this interaction.

Econometric model of Palo Verde Prices 

Palo Verde energy prices play a critical role in the use of GTMax.  One strength of the current

modeling framework used by Western and ANL is that it avoids relying on the model to predict

prices. For various future purposes, Western and ANL have proposed enlarging the geographic

scope of GTMax to all of WECC. This approach would inevitably result in relying on the model to

predict price. Such a step should be viewed cautiously. Models typically calculate price as the

marginal cost of production. While price is typically related to the marginal cost of production, it is

not necessarily identical to it. This is relevant to our discussion of cost‐based versus value‐based

compensation earlier in the report.   Exploring an econometric approach to Palo Verde pricing is a

potential alternative that deserves serious consideration. For many business purposes, Western

needs to understand price formation at Palo Verde. Econometrics starts with the actual price data,

seeking to explain the variations in observed prices. Econometrics is an imperfect tool, like all other

tools. Expanding the toolkit, however, is often worthwhile.

Long-term, large-scale risk and modeling implications 

In our considered opinion, GTMax is not the model, nor is Western the sole organization to

investigate long‐term, large‐scale risks.  The uncertainties include climate change, technology

innovation, or market and regulatory changes that could affect wholesale power prices and the

price and availability of fuels.  If GTMax is to play a role in a more comprehensive framework for

study – an outcome we would not dismiss – GTMax needs to be augmented.  Either the RiverWare

model or other models to create hydroelectric generation input data must be brought in under the

framework.  Tools for evaluating capacity expansion are necessary.  The framework needs to make

efficient and thorough assessments of large‐scale, irreversible changes in future conditions.

Perhaps the most effective and efficient piece to the solution is a more open process around the

kinds of scenarios that Western examines.  We believe it is necessary to give more complete

consideration to contingency plans and to scenarios where circumstances do not play out according

to assumptions.  Studies need to reflect that circumstances can change at any time and typically

cannot be anticipated.  When Western learns of a new development, others will have as well, and

measures to mitigate the outcome will already be priced in the market.  Buying homeowner’s

insurance when the house is on fire is difficult and expensive.  Contingency planning is therefore

central to risk mitigation.

Western and ANL staff and stakeholder participants are important sources for the experience,

knowledge, and creative energy to address risk.  However, Western’s purpose, marketing power, is

less than a perfect match for a long‐term stewardship role implicit in long‐term risk management.  
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 32

We do not have a recommendation about which group or agency should assume this role, but it

does bring us to our last recommendation.

The User Community and Process Issues 

At several points in this report, we have pointed out the important role the stakeholders play in the

credibility and value of Western studies.  The section The User Community and Process Issues

recommends particular modeling enhancements that could make studies more accessible by

participants and observers.

We need to emphasize that we believe there are situations in which outside observers and experts

can bring more value to policy analysis than modelers.  Treatment of strategic risk, for example, is

often a question of analytical vision and discipline rather than computational horsepower.  A

forward‐looking analysis of potential climate change effects would be much more credible, for

example, if the hydro scenarios were structured by climate experts rather than simply chosen by

modelers.

In any case, consensus on language and methods, and access to the models and their results is key

to the credibility of any study.  By consensus and access, here, we refer not only to consensus

within Western and ANL and physical access to reports, but practices that make the studies and

their recommendations transparent to all stakeholders and observers.

REFERENCES 

Arnedillo, O., “What does the evidence really say about the Residual Supply Index?” The Electricity

Journal, v. 24, no. 1 (2011) 57‐ 64.

Barmack, M., E. Kahn, S. Tierney, C. Goldman, “Econometric models of power prices: An approach

to market monitoring in the Western US,” Utilities Policy, v.16 (2008) 307‐320

Energy and Environmental Economics (E3), WECC Energy Imbalance Market Benefits Study, March

24, 2011, available at:

www.wecc.biz/committees/BOD/EDTSC/EDTTRS/EDTTRS032411/Lists/Presentations/1/E3_EDT_Ph

ase1_2011‐03‐24_EDTTRS‐FINAL.pdf

Foley, A. , B. Ó Gallachóir, J. Hur,  R. Baldick , E. McKeogh, “A strategic review of electricity systems

models,” Energy 35 (2010) 4522‐ 4530.

Kahn, E. “Regulation by Simulation: The Use of Production Costing Models in Electricity Planning

and Pricing,” Operations Research v. 43, no. 3 (1995) 388‐398.

Loose, V. Quantifying the Value of Hydropower in the Electric Grid: Role of Hydropower in Existing
FINAL REPORT OF THE GTMax MODEL REVIEW PANEL 

Page 33

Markets, Sandia Report SAND2011‐1009, January 2011.

Loftin, S. GTMax Modeling Examples, GCMRC Workshop Presentation, September 1, 2011.

PNUCC System Planning Committee, Capabilities of Electric Power Resources, March, 2011 available

at http://www.pnucc.org/)

Red Electrica de Espana (REE),  “Operacion del Sistema,” Informe 2010, 2011, available at

http://www.ree.es/sistema_electrico/pdf/infosis/Inf_Sis_Elec_REE_2010_SistemaPeninsular05.pdf

Veselka, T., Methods for Representing CRSP Resources, GCMRC Workshop Presentation, September

1, 2011.

Veselka, T., L. Poch, C. Palmer, S. Lotfin and B. Osiek, Ex Post Power Economic Analysis of Record of

Decision Operational Restrictions at Glen Canyon Dam, ANL/DIS‐10‐6, July, 2010 available at

www.wapa.gov/crsp/newscrsp/Post‐ROD_Final_Rev27‐19‐10.docx

Veselka, T., L. Poch, C. Palmer, S. Lotfin and B. Osiek, Financial Analysis of Experimental Releases

Conducted at Glen Canyon Dam during Water Years 1997 through 2005, ANL/DIS‐10‐7, April, 2010

available at http://www.ipd.anl.gov/anlpubs/2010/04/66772.pdf

Woo, C.K., I. Horowitz, N. Toyama, A. Olson, A. Lai, and R. Wan (2007) “Fundamental Drivers of

Electricity Prices in the Pacific Northwest,” Advances in Quantitative Analysis of Finance and

Accounting, v.5, 299‐323.  

Zagona, E., T. Fulp, R. Shane, T. Magee and H. Goranflo, “RiverWare: A Generalized Tool for

Complex River Basin Modeling,”  Journal of the American Water Resources Association  v. 37, no. 4

(2001) 913‐929 available at:

http://cadswes.colorado.edu/PDF/RiverWare/Zagona_JWARA_2001.pdf
APPENDIX A

Professional Background of the Review Panel Members
Edward Kahn, Ph. D, Special Advisor, Analysis Group, Inc. Dr. Kahn specializes in 

electricity competition and regulation. He has consulted to numerous domestic and international 

private and public sector clients including The Federal Energy Regulatory Commission, the 

World Bank and the U.S. Congress. His expert testimony, presented in both regulatory 

proceedings and civil litigation, has covered issues ranging from competitive market pricing to 

market power mitigation to electricity industry restructuring. He has lectured widely to academic, 

industry, and professional groups. Dr. Kahn is co-author of the books Electric Utility 

Planning and Regulation and International Comparisons of Electricity Regulation, several book 

chapters, and nearly one hundred articles and reports. Before joining Analysis Group, he was a 

Vice President with National Economic Research Associates. Dr. Kahn was previously Senior 

Staff Scientist at the Lawrence Berkeley National Laboratory, Co-director of the Program on 

Workable Energy Regulation at the University of California Energy Institute, and Lecturer in the 

Energy and Resources Program at the University of California, Berkeley.

Verne W. Loose, Ph. D, Principal, Verne W. Loose Associates, LLC. Dr. Loose has an 

extensive career in economics, finance, and management spanning close to four decades. He is 

currently a member of the Energy Storage and Transmission Analysis Group at Sandia National 

Laboratories in Albuquerque, NM. For the bulk of his career Dr. Loose’s research focus has been 

on the energy and natural resource industries, markets, and evaluation with emphasis on electric 

power, coal, and oil and gas. Dr. Loose is a member of a study group investigating the potential 

for western hydroelectric projects to balance the variable renewable energy (wind and solar) 

projects being added to the western electricity grid. He has an extensive bibliography of refereed 

publications and conference papers and presentations in these fields. He has managed a 

professional staff of more than 20 Ph.D. and Master’s level economists during his tenure at Los 

Alamos National Laboratory, one of the nation’s premier research institutions. He has managed 

technical projects exceeding $5 million annually (1985 dollars) involving multiple federal 

government agency oversight and delivered in foreign countries. He is a member of the American 

Economic Association. 

Michael Schilmoeller, Ph.D, Senior Power Analyst, NW Power Conservation and Planning 

Council. Michael Schilmoeller joined the Council in 2002 to perform quantitative analysis and 

computer simulations of power systems. Prior to joining Council staff, he served in Portland 

General Electric's Financial Analysis, Corporate Planning, and Environmental Science 

departments, in PGE's analytical laboratory, and in Enron's research and development group. 

Much of his experience is in quantitative analysis of power resources and systems, especially 

developing and using computer software. Michael spent seven years with the Salt River Project, a 

power and water utility serving metropolitan Phoenix, Arizona, where he performed least-cost 

and strategic planning studies. Michael holds a bachelor's degree in mathematics from Stanford 

University, a masters in business administration from Arizona State University, and a bachelor's 

degree in physics, and master's and Ph.D degrees in mathematics from Portland State University. 

As a contractor to PGE in 1996, Michael co-authored and extended an hourly power price 

forecasting model built on the fundamental economic principles of a competitive power 

market. He enhanced the model by designing a capacity expansion algorithm that reproduced the 

economic and financial requirements of independent power producers. The entrance of such 

producers determines long-term prices in a deregulated wholesale energy market. At the Council, 

Michael built the resource portfolio strategy model that guided acquisition choices for the 

Council's Fifth and Sixth Power Plans. The model disregards forecasts and avoids resource 

additions that rely of perfect foresight. The strategy instead identifies resource options which 

future decision makers can pursue according to their circumstances. The model also informs 

energy efficiency acquisition policies and highlights other resource opportunities that minimize 

economic risk.

Part 2
Comments by S. Clayton Palmer

Department of Energy, Western Area Power Administration

on the Final Report Of The GTMax Model Review Panel:

Report of a Workshop held August31and September 1, 2011

in Flagstaff, Arizona, dated September 4, 2012
1

Comments by S. Clayton Palmer

on the

FINAL REPORT OF THE GTMAX MODEL REVIEW PANEL: 

Report	of	a	Workshop	held	August	31	and	September	1,	2011	in	Flagstaff,	Arizona

Draft	of	Tuesday,	September	04,	2012

Background Information

In	recent	years,	the	Glen	Canyon	Adaptive	Management	program	committees,	specifically	the	Glen	

Canyon	Adaptive	Management	Work	Group	(AMWG)	and	the	Technical	Work	Group	(TWG)	have	been	

interested	in	an	expanded	and	more	transparent	program	of	gathering	and analyzing	economic	data	

related	to	the	operation	of	Glen	Canyon	Dam	(GCD).	The	purpose	of	these	committees	is	to	make	

recommendations	to	the	Secretary	of	the	Department	of	Interior	(DOI)	regarding	the	operation	of	this	

dam.	Economic	analysis	is	key	to	making	informed	recommendations.	

In	order	to	begin	the	development	of	this	expanded	economic	program,	the	Grand	Canyon	Monitoring	and	

Research	Center (GCMRC) convened	a	workshop	on	behalf	of	the	TWG	that	included	a	panel	of	

economists	This	workshop	occurred in	December,	2009.	The economic	panel	prepared	a	report	with	a	set	

of	recommendations.	Subsequently,	the	TWG	established	a	subcommittee	to	review	the	

recommendations	of	this	panel	and	make	its	own	recommendations	regarding	implementing	the	panel’s	

report:	the	Socioeconomic	Ad	Hoc	Group	(SEAHG).	

Western	Area	Power	Administration	(Western)	sponsored	the	development	of	a	simulation	model	– the	

GT	Max	– as	a	representation	of	the	production	of	hydropower	in	the	Upper	Colorado	River	System.	The	

model was	developed	by	Argonne	National	Laboratory.	The	model	had	two	purposes: to	assist	Western	in	

making		its	operation	of	the	CRSP	powerplants	more	efficient,	and	to	estimate	the	electrical	power	

production	impacts	of	different	operational	regimes.	For	the	latter	purpose,	the	U.S.	Bureau	of	

Reclamation	has	also	made	use	of	the	GT	Max	model	– specifically	for	estimating	the	impact	of	proposed	

changes	to	the	operation	of	CRSP	powerplants	in	environmental	planning	documents.	

The	purpose	for	my	recitation	of	this	history	is	to	explain	why	the	GCD	adaptive	management	program	has	

taken	an	interest	in	this	model.	At	one	of	its	recent	meetings,	the	AMWG	recommended	that	Western	use	

the	GT	Max	model	for	evaluating	the	electrical	power	system	impacts	of	proposed	changes	in	GCD	

operations.	The	GCMRC,	the	AMWG	and	TWG	became	interested	in	considering	the	suitability	of	the	GT	

Max	model:		is	it	a	suitable	model	– properly	specified	and	sufficiently	robust	– to	address	the	information	

needs	of	the	stakeholders?		

The	purpose	of	the	GCD	AMP’s	interest	in	considering	the	GT	Max	model	for	estimating	the	economic	

impacts	on	the	electrical	power	system	should	be	made	clear.	The	GCD	AMP	program	requires	

information	on	the	economic	impact	on	the	electrical	power	system,	along	with	other	economic	data	and	

analysis,	in	order	to	make	recommendation	on	the	operation	of	the	GCD.	The	SEAHG,	following	up	on	its	

task	to	consider	the	economic	panel’s	report,	prepared	its	recommendations.	These	were	adopted	by	the	

TWG	and	AMWG1.	The	information	required	is	economic	valuation	of	proposed	changes	to	GCD	operation	

– one	component	of	which	is	in	regard	to	the	use	values	related	to	electrical	power.	

																																																												 1 RECOMMENDED INFORMATION NEEDS AND PROGRAM ELEMENTS FOR A PROPOSED AMP 

SOCIOECONOMIC PROGRAM, APPROVED BY AMWG, FEBRUARY 23, 2012
2

The	GT	Max	Panel	Report

As	reflected	by	the	subject	report,	the	GT	Max	review	panel	appears	to have	a	set	of	objectives	that	differ	

from	the	information	needs	of	the	GCD	AMP.	In	addition	to	a	review	of	the	GT	Max	model,	the	panelists	

provide	a	set	of	recommendations	related	to	improving	Western’s	position	in	the	electrical	utility	industry	

[page	30]:

It	is	also	right	and	proper	for	Western	to	investigate	opportunities	presented	by	

its	geography,	that is,	the	capability	to	bid	into	CAISO	markets	and	enter	into	

bilateral	contracts	with	other	entities.	For example,	Western	should	investigate	

energy	arbitrage	between	day-ahead	and	real-time	markets.

In	addition	to	advocating	that	Western	modify	its	position	vis-à-vis	real-time	markets,	the	panelists	

propose	that	Western	participate	further	in	ancillary	service	markets	[page	29]:	

Western	should	evaluate	their	participation	in	ancillary service	markets.

Moreover,	in	various	places	throughout	the	report,	the	authors	suggest	that	Western should	seek	to	

improve	its	profit position,	increasing net	revenues	by bidding its	generation	into	California’s	ancillary	

services	market.

Not	only	do	these	recommendations	seem	to	have	little	or	no	relationship	to	the	goals	of	the	GCD	AMP,	it	

crosses	from	positive	analysis	to	normative	analysis.		An	advocacy	of	a	modified	set	of	operating	policies	

and legal	framework	seems	foreign	from	an	advocacy	of	unfettered	and	objective	scientific	inquiry.	It	

further	indicates	a	panel	that	is	unfamiliar	with	the	legal	framework	in	which	GCD	electrical	power	is	

marketed.	

Another	important	mistake	of	the	panel’s	report	on	the	GT	Max	model	is	this:	the	GT	Max	model	is	

routinely	used	by	Western	to	estimate	the	impact	of	short-term	experiments,	to	provide	an	indication	of	

direction	and	magnitude	of	a	proposed	GCD	operational	change	where	an	impact is	required	quickly	or	to	

evaluate existing	scheduling	practices.	The	presentations	made	by	the	GT	Max	model	creators	and	users	

during	the	workshop	were	geared	to	explaining	these	routine	uses.	As	a	presenter,	I	could	have	explained	

how	the	GT	Max	model	might	be	reconfigured	to	address	the	issues	in	the	panel	report.	After	all,	the	GT	

Max	model	has	been	used	to	model	substantially	larger	electrical	systems,	both	in	geography	and	in	type,	

than	the	CRSP	power	system.	2 Questions	related	to	how	the	GT	Max	model	– a	production	cost	model	–

might	fit	into	a	process	of	a	more	geographically	or	temporally	robust	framework	was	not	discussed	during	

the	workshop.	In	this	response	to	the	panel	report,	I	will	attempt	to	illustrate	an	example	of	how	this	has	

been	done.	

The	GT	Max	Model	and	the	GCD	AMP	Relevant	Questions	

The	relevant	questions	for	the	GCD	AMP,	its	committees,	stakeholders	and	the	GCMRC	is	this:	

• For	developing	economic	analyses	of	the	electrical	power	system,	can	the	GT	Max	be	a	useful	tool	

and/or	how should	the	GT	Max	be	configured	and	calibrated,	what	data	should	be	used	as	inputs	and	

																																																												 2 I	will	go	into	more	detail	about	this	subject	in	subsequent	pages.
3

how	should	the	GTMax	model	outputs	be	evaluated	to	develop	scientifically	adequate	analysis	to	meet	

the	information	needs	of	the	GCD	AMP?			

Economic	use	Values	for	the	Electrical	Power	System

The	determination	of	economic	use	values	for	the	electrical	power	system	for	changed	operations	at	the	

GCD	requires	the	following:	

• The	identification	of	the	relevant	market.	

This	issue appears	to	be	elusive	and	has	been	considered	(or	ignored)	over	several	years.	In	order	to	

determine	the	economic	value	of	a	good	or	service,	the	market	within	which	the	good	or	service	is	

produced	and	consumed	needs	to	be	identified.	In	the	current	context: is	electrical	production	at	GCD	

exchanged	and	consumed	within	a	North	American	market,	within	the	“western	grid”	or	is	production,	

consumption	and	exchange	confined	within	the	CRSP	system	and	the	electrical	systems	to	which	it	is	

directly	tied?	

“Everything	affects	everything	else”	is	an	oft	used	phrase.		Never-the-less,	everything	doesn’t	affect	

everything	else	in	equal	proportions	and	some	effects	are	irrelevant	or	inconsequential.	In	the	December	

2009, economic	panel	report,	the	economists	that	served	on	that	panel	were	also	interested	in	

establishing	economic	values	for	use	in	evaluating	changes	in	operations	at	Glen	Canyon	Dam.	They	were	

well	aware	that an	electrical	“grid”	exists	in	the	Western	United	States.	As	they	noted3:	

[the]	GCD	and	the	CRSP	system	are	embedded	in	the	larger	western	power	grid	(the	WECC).	

Similarly,	the	utilities	to	which	CRSP	sells	power	are	embedded	in	the	WECC.	Therefore,	in	principle,	

the	market	by	reference	to	which	the	economic	value	of	GCD	power	is	determined	is	not the	CRSP	

system	but	the	WECC.	At	any	point	in	time,	it	is	the	marginal	price	of	electricity	in	the	WECC	that	

determines	the	economic	value	of	power	generated	at	GCD.	

However,	the	panelists	also	suspected	that	the	electrical	power	output	of	GCD	is	small relative	to	the	total	

electrical	capacity	in	this	“grid”.		The	panelists	suggested	a	study	to	determine	the	extent	to	which	a	

change	in	the	operation	of	GCD	“spill	over” into	the	WECC	“grid”.	 The	proposal	from	the	panelists	is as	

follows:

Given	the	alternatives,	existing	models	used	by	WAPA	to	optimize	the	operation	of	the	

integrated	system	of	generation	resources	should	be	used	to	determine	if	all	

consequences	of	changed	operations	can	be	managed	within	the	WAPA	marketing	area,	

or	if	electrical	(and	thus	economic)	“spill-over”	effects	will	alter	generation	patterns,	

market	prices	or	transmission	bottlenecks	elsewhere	in	the	WECC	system.	If	the	effects	of	

changed	operations	at	Glen	Canyon	can	be	managed	by	WAPA	without	economically	

significant	changes	in	the	rest	of	the	western	U.S.,	then	the	economic	consequences	of	

such	operations	will	be	limited	to	WAPA’s	customers,	and	the	modeling	effort	limited.

The	logic	is	this;	if	the	electrical	production	at	Glen	Canyon	Dam	is	“small”	relative	to	other	resources	

in	the	western	electrical	grid,	a	change	in	its	operation	will	not	affect	exchange	rates	(prices)	in	the	

																																																												 3 Final	Report	of	the	GCMRC	Socioeconomic	Research	Review	Panel	Report	of	a	Workshop	held	December	2	&	3,	2009	

Phoenix,	Arizona,	Hamilton,	J,	Hanemann,	M.,	Loomis,	J.,	&	Peters, L.,	Grand	Canyon	Monitoring	and	Research	

Center,	February	26,	2010
4

WECC	system.	If	this	is	the	case,	then	the	so	called	“WAPA	models”	that	are	configured	for	the	CRSP	

electrical	system	and	its	contractors	is	sufficient	to	establish	economic	values.

If	GCD,	the	CRSP	electrical	system	and	its	use	in	a	contractor’s	resource	stack	doesn’t	affect	market	

prices,	it	therefore	follows	that	the	electrical	production	at	the	GCD	is	“small”	in	the	economic	sense	

and	WECC market	price	are	exogenously	determined4.	

The	resultant	estimates would	be	the	appropriate	economic	values	provided	that	a	change	in	the	

operation	of	that	resulted	in	a	loss	of	the	electrical	power	resource	at	GCD	would	give	rise	to	the	

need	for	additional	resource	to	come	into	being	to	replace	the	loss.	As	the	panelists	noted:	

.	.	.the	marginal	cost	of	this	extra	capacity	would	count	as	a	real	economic	cost.	It	

would	not	necessarily	be	the	cost	of	additional	capacity	in	CRSP	– it	would	be	the	cost	

of	additional	capacity	anywhere	in	the	WECC	system	to	which	WAPA	and/or	WAPA	

contractors	have	access.

• Dealing	with	Institutional	Constraints:

A	second	aspect	of	establishing	economic	values	is	to	determine	whether	institutional	constraints	distort	

economic	values.	More	specifically,	if	market	prices	are	to	be	used	as	a	representation	of	economic	cost,	

are	there	institutional	constraints	that	cause	market	prices	to	significantly	differ	from	economic	costs?	If	

this	is	the	case,	do	these	“distortions”	cause	a	difference	that	is	so	great	as	to	cause	differences	in	the	

magnitude	of	economic	impact	of	ordering	of	the	impact	of	various	alternative	GCD	operational	regimes?		

If	so,	what	modifications	to	the	GT	Max	model,	to	the	input	data	or	to	the	analysis	of	model	output	should	

occur?	

For	clarity,	institutional	constraints	differ	from	limitations	on	infrastructure.	The	legal	mandate	that	

requires	that	Western	market	the	GCD	electrical	output	to	the	“preference”	customers	rather	than	to	“the	

market”	is	independent	of	whether	unencumbered	transmission	systems	exist	such	that	Western	could	

sell	electricity	to	buyers	in	Northern	California.	

There	are	two	reasons	why	institutional	constraints	may	not	fatally	encumber	the	economic	evaluation	

that	would	be	needed	the	GCD	AMP	for	appropriate	decision	making.	In	the	EIS	prepared	by	the	DOI	and	

Reclamation	on	the	operation	of	the	GCD	in	19955,	two	estimates	are	made	on	the	economic	impact	to	

electrical	power	for	each	alternative.	One	approach	(CROD)	assumes	the	continuation	of	existing	federal	

contracts	for	electricity.	The	other	approach	(Hydro)	assumes	that	GCD	power	is	allowed	to	be	sold	to	the	

WECC	market	and	is	dispatch	for	“peak	shaving”.		In	essence,	the	Hydro	approach	disregards	existing	legal	

requirements	for	the	sale	of	electrical	power.	The	results	showed	that,	while	the	magnitude	of	the	

economic	impact	differed	(the	Hydro	approach	showed	a	lesser	impact),	the	differences	between	the	

																																																												 4 “Everything	affects	everything	else”.	So,	a	determination	that	a	change	in	the	operation	at	GCD	doesn’t	affect	

market	prices	in	the	WECC	would	have	to	be	a	determination	made	based	on	statistical	significance.	There	is	also	the	

possibility	that	WECC	market	prices	could	be	statistically	significantly	different,	given	a	change	in	GCD	operations,	but	

not	consequential	and/or	that	the	added	precision	is	not	worth	the	added	expense.	I	will	have	more	to	say	on	this	in	

due	course.	

5 Operation	of	Glen	Canyon	Dam,	Final	Environmental	Impact	Statement,	U.S.	Bureau	of	Reclamation	and	

Department	of	Interior,	November,	1995
5

CROD	approach	and	the	Hydro	approach	were	proportional	and	maintained	the	relative	ranking	of	the	

alternatives	in	terms	of	impact.	

If	this	example	is	illustrative	of	the	effect	of	institutional	constraints	on	economic	impact	analysis,	it	may	

be	that	market	prices,	even	in	light	of	institutional	and	legal	constrains,	within	the context	of	GCD	AMP	

decision	making,	may	serve	to	estimate	the	economic	value	of	proposed	changes	to	GCD	operation.	

Another	important	consideration	regarding	institutional	and	legal	constraints	is	this;	the	GCD	AMP	stake	

holders	are	often	interested	in	the	distribution of	economic	impacts.	For	example,	the	SEAHG	report	(cited	

above)	describes	an	information	need	related	to	the	economic	impact	of	modified	GCD	operations	on	

Native	American	Tribes.	

For	distributional	impacts	to	be	considered,	it	is	important	to	identify	and	have	a	proper	specification	of	

institutional	constraints	and	of	the	legal	framework	in	which	federal	power	from	the	GCD	is	marketed.	For	

example,	federal	power	from	the	GCD	power	plant	is	required,	under	the	law,	to	be	sold	as	firm	electrical	

power	and	energy,	preferentially,	to	certain	utilities,	state	and	federal	facilities,	irrigation	districts	and	

Native	American	tribes.	

The	GT	Max	model	in	consort	with	capacity	expansion	and other	models

If	a	change	in	operation	at	the	GCD	gives	rise to a	consequential	change	in	electrical	exchange	and/or	

prices	within	the	WECC	area,	that	would	indicate	that	the	relevant	market	for	GCD	electrical	power	is	at	

least	the	“western	grid”.	A different	modeling	process	would	be	needed	to	estimate	economic	values, one	

that	includes	the	electrical	resources	in	the	WECC	region.	This,	more	robust	modeling	would	be	necessary	

to	estimate	economic	values.			

Figure	1	below,	is	an	illustration	of	a	modeling process	for	a	more	robust	and	geographically	more	

complete	modeling	process	6.		This	is	a	flow	chart	that	illustrates	a	modeling	process	undertaken	by	

Western	for	an	environmental	impact	statement	on	one	of	its	marketing	programs7.	This	process	includes	

the	several	models:	one	that	determines	hydrological	conditions	and	GCD	release	volumes	on	a	monthly	

basis	(Hydro	Forecast/Condition)8,	a	production	model	(Hydro	Dam	Operational	Restrictions),	a	capacity	

expansion	model	(Utility	Supply-Side	Expansion) and	a dispatch	model	(Utility	Dispatch).		This	flow	chart	

illustrates	how	several	models	– including	a	models	that	deal	with	capacity	expansion	and	production	cost,	

have	worked	together	to	achieve	an	appropriate	estimate	of	economic	values.	

When	the	GT	Max	model	– a	production	cost	model	– is	included	in	a	modeling	effort	that	includes	a	

capacity	expansion	model,	the	lack	of	a	capacity	expansion	algorithm	described	in	the	panel	report	(page	

19) becomes	a	mute	point.	The	panel	report	states:

The	GTMax	model	and	Western’s	analytical	framework	is	designed	primarily	to	

evaluate	short-term operations.	The	authors	of	GTMax	excluded	long-term	planning	

features,	like	long-term	capacity expansion	capability.	

																																																												 6 Salt	Lake	City	Area	Integrated	Projects	Electric	Power	Marketing	Environmental	Impact	Statement,	Volume	3:	

Appendix	A,	U.S.	Department	of	Energy,	Western	Area	Power	Administration,	February,	1994

7 The	EIS	was	prepared	by	Argonne	National	Laboratory	for	Western.	It	was	the	Post	1989	Marketing	and	Allocation	

Program.

8 This	model	(and	others	in	this	flow	chart)	were	replace	with	the	GT	Max	model.	Therefore,	in	describing	this	flow	

chart,	I	use	the	term	production	cost	model.	
6

This	statement	indicates	a	lack	of	understanding.	The	GT	Max	model	is	production	cost	model	which	can	

be,	and	is	often,	configured	for	uses	beyond	estimating	short-term	financial	costs	in	a	closed	electrical	

system.	It	can	be	sufficiently	detailed	so	that	it	can	estimate	power	production	on	an	hourly	basis,	

dynamically,	from	a	set	of	three	conjoined	dams	in	Colorado	(the	Aspinall	Units)	and,	when	combined	with	

other	models		can	serve	to	estimate	long-run	marginal	cost	within	a	geographically	large	and	complex	set	

of	electrical	systems9.	

A	production	cost	model	(as	shown	in	this	flow	chart) is	also	useful	for	another	reason.	In	addition	to	

economic	values,	this	model,	in	addition	to	other	considerations,	such	as	Western	contracts	and	

commitment	levels,	allows for	estimates	of	how	economic	impacts	are	distributed	to	various	contractors.	

The	distribution	of	economic	effects	is	often	of	interest	to	stakeholders	and	decision	makers	and	would	be	

less	precisely	determined	if	a	capacity	expansion	model	or	WECC-wide	model	worked	alone.	

Estimates	of	economic	values	related	to	the	change	in	operation	at	the	GCD	was	prepared	for	the	

Bureau	of	Reclamation	(Reclamation)	for	the	EIS	on	the	operation	of	GCD	published	in	199510.		At	the	

time	the	analysis	was	done	for	this	EIS,	electrical	capacity	in	the	WECC	region	was	in	surplus. In	

addition,	the	EIS	required	a	20-year	evaluation	of	the	affected	resources.	The	consulting	firm	of	Stone	

and	Webster,	in	consultation	with	the	cooperating	agencies,	prepared	economic	impact	analysis	for	

the	EIS	alternatives	– all	of	which	were	changes	in	the	operation	of	GCD.	

																																																												 9 In	the	SLCA/IP	Post	89	Marketing	EIS,	a	forerunner	of	the	GT	Max	model	was	used	within	a	set	of	interactive	models	

to	analyze	15	electrical	systems	in	the	Rocky	Mountain	West	along	with	the	CRSP	power	system.	

10 Operation	of	Glen	Canyon	Dam,	Final	Environmental	Impact	Statement,	U.S.	Department	of	the	Interior, Bureau	of	

Reclamation,	March,	1995.
7
Comments by Sam Loftin

Department of Energy, Western Area Power Administration

on the Final Report Of The GTMax Model Review Panel:

Report of a Workshop held August31and September 1, 2011

in Flagstaff, Arizona, dated September 4, 2012
1

Comments	by	Sam	Loftin

on	the

FINAL	REPORT	OF	THE	GTMAX	MODEL	REVIEW	PANEL:	

Report	of	a	Workshop	held	August	31	and	September	1,	2011	in	Flagstaff,	Arizona

My	comments	are	based	on	my	use	of	the	GT-Max	and	predecessor	models	for	the	last	decade	or	so,	

and	on	my	33	years	of	experience	as	an	electrical	engineer	at	Western	Area	Power	Administration’s	

CRSP	office.

Executive	Summary,	Pages	1-3

The	 purpose	of	the Report	by	the	reviewers was	to	assess	Western’s	and	Argonne’s current	GT-Max	

model	usage,		usage in	the	LTEMP	EIS,	and	in	large	geographic	and	time-scale	power	system	studies.		In	

this	section,	the reviewers summarize some	limitations	of the	current	model	in its present configuration

– some	of	which	 I	agree	with.		Addressing	some	if	not	most of	these limitations	to the	current	GT-Max

model	is	the	purpose	of	the	current	Argonne-led	DOE	lab	effort	to	create	a	new	hydro	modeling	system.

As	is	typical	of	such	models,	it	simplifies	what	is	in	reality	a	very complex	system	of	inter-related	

electrical	generation	facilities	and	decision	factors,	and	therefore,	like	all	such	models,	it	is	an	

imperfect	representation	of	reality	and	has	limitations	in	how	it	can	be	applied.

I	agree	with	the	above	statement	from	the	Report,	and	especially	the	underlined	portion (my	

emphasis).		We	need	to	keep	in	mind	that	computer	models	are	meant	to	help	us	inform	decisions,	

but	are	limited in	applicability,	scope, and	accuracy.		The	statement	applies	not	only	to	GT-Max	but	

to	all	computer	models,	including	river	system	models like	Riverware,	sediment	models,	beach	

building	models,	and	climate	change	models.

In	summary,	the	GTMax	model	and	Western’s	analytical	framework	are	designed	primarily	to

evaluate	short-term	operations.	The	authors	of	the	GTMax	model	did	not	include	long-term

planning	features,	such	as	long-term	capacity	expansion	algorithms,	which	limits	its	utility	for

forecasting	economic	implications	of	long-term	operations.	The	model	is	not	capable	of	modeling

operations	of	less	than	one-hour	duration,	which	means	that	the	potentially	significant	economic

value	of	ancillary	services	cannot	be	adequately	represented	in	the	modeling	results.

I	think	that	this	statement	is	accurate.		In	the	body	of	the	report	the	reviewers	discuss	how	GT-Max

was	designed	primarily	to	inform	short-term	hydroelectric	operational	needs,	and	only	secondarily	

for	longer-term financial	and	economic	studies.	Western	and	Argonne	designed	GT-Max	to	do	some	

things	very	well,	and	that	is	what	GT-Max does	best.		That	doesn’t	mean	that	GT-Max can’t be	used	

for	other	modeling	problems,	only	that	it	may	be more	complex or	time	consuming	to	use	it	for	

longer	time	duration	or	larger	geographical	area	studies. Or	that	GT-Max might	need	model	input	

from	other	model	outputs.

The	model	relies	on	inputs	from	the	RiverWare	model,	which	are	provided	by	Reclamation;	this	

constrains	Western’s	ability	to	modify	hydrogeneration	scenarios	for	assessing	impacts	of	climate	
2

change	or	river	diversions.	Thus,	the	model	does	not	facilitate	convenient	or	efficient	evaluation	of	

alternative	assumptions	that	may	be	substantially	different	in	the	future.

The	reviewers	discussion	of	Riverware	here	and	in	the	body	of	the	Report	leads me	to	think	 that	

they	don’t	really	understand	how	Western	uses	Riverware	output	in GT-Max	studies.		Riverware	

output	(more	specifically	the	multi-trace [ISM] version	of	CRSS)	is	used	in some	GT-Max	studies	

Western	has	performed	or	participated	in,	particularly	the	longer-term	studies like	the	Shortage

Criteria	EIS,	the	Colorado	River	Basin	Water	Supply	and	Demand	Study,	or	in	projecting	purchase	

power	expenses	for	Western’s SLIP	Power	Repayment	Studies.		Western	uses	Reclamation’s	24-

month	study	output	more	often than	Riverware,	particularly	in	the	monthly	prechedule	studies	that	

I	do.		There	is	no	requirement	to	use	any	Reclamation	model	output	in	GT-Max	– we	could	easily	

use	other	sources	of	water	release	data	as	long	as	those data	are in	a	format that	GT-Max	can	use

(for	instance,	monthly	reservoir	releases	in	acre	feet	and	reservoir	elevation	in	feet).

Furthermore,	the	model	does	not	have	the	geographic	scope	or	an	adequate	representation	of	

transmission	to	study	possible	consequences	of	policy	changes	in	other	parts	of	the	WECC.	The	

strength	of	the	model	as	currently	formulated	lies	in	its	ability	to	examine	the	consequences	of	

following	specified	management	regimes	over	short	periods	of	time	when	water	conditions,	

electricity	prices,	and	other	variables	are	reasonably	stable.

Again,	true,	but	that	wasn’t	what	the	GT-Max model was	designed	to	do.		GT-Max	is	unrivalled	in	

my	experience in	modeling	the	complex	operational	and	environmental	constraints	that	we	

encounter	at	CRSP	power	plants.		That	is	where	its	true	usefulness	lies,	and	that	is	where	other	

power	system	models	that	I	have	encountered	are	lacking.

Three	Business	Models,	pages	7-9

The	reviewers	present	an	overview	of	the	business	models	found	in	North	American	power	

markets;		ISOs	like	the	California	ISO,	traditional	bilateral	markets,	and	a	mixture	of	the	two	like	is	

found	in	the	Western	Interconnection.	They	then	discuss	Western	business	model.

With	available	excess	capacity,	Western	could	bid	into	California markets	when	either	energy	or	

ancillary	services	prices	make	it	financially	attractive.

Reading	these	sections	suggests that	the	reviewers	prefer	the	ISO	model	and	would	like	Western	to	

move	its	business	model	in	that	direction.	They	advocate	for	models	that	can	better	characterize	

sub-hourly	hydroelectric	operations	for	ancillary	services. This	advocacy	continues	in	other	sections	

of	the	report	and	in	the	Conclusions	and	Recommendations.		This	business	model	doesn’t	really	fit	

with	the	current	model	that	Western	operates	under.		Our	statutory	requirements	and	limitations	

make	Western	significantly	different	from	a	typical	Investor-Owned	or	Publicly-Owned	Utility and	

limits	our	ability	to	do	as	the	reviewers	propose.

Typical	Application	and	Unique	Features,	Page	12

Most	commercially	available	production	cost	models	have	very	simple	representations	of	the
3

hydroelectric	generation	system.	The	standard	approach	is	to	divide	hydro	resources	into	“base	

load” and	“load-following”	segments	for	each	simulation	period.	The	base	load	segment	is

dispatched	in	all	hours	of	the	period.	The	load	following	energy	is	used	to	“peak	shave”	the

anticipated	load	profile	up	to	the	limit	of	hydroelectric	capacity.	The	result	of	this	is	a	flattened	load

shape	that	is	used	to	simulate	the	operation	of	thermal	power	plants.

The	reviewers	compare	GT-Max	with	other	commercially	available	production	cost	models.		They	

correctly	emphasize	the	superiority	of	GT-Max	for	modeling	complex	hydro	operations.		This	is	a	

topic	that	the WECC	Hydro	Modeling	Task	Force	(HMTF),	that	I	participate	in,	spends	a	great	deal	of	

its	time	on. WECC	is	using	commercial	products such	as	Promod	and	Gridview to	simulate	WECCwide	operations	while	simulating	widely	varying	hydro	plants,	each	with	its	own	set	of	constraints.		

The	hydro	operations	of	individual	members’ systems	are	much	more	complex	than	can	be	

characterized	in	such	models,	so the	HMTF	continues	to	experiment	with, develop,	and	implement

methods	to	better	characterize	hydro	operations.

Objective	function,	solution	algorithms,	fundamental	period,	pages	12-15

In	any	case,	all	objectives	have	to	be	reduced	to	a	single	number	for a	computer	model	or	analyst	to	

compare	and	rank	alternative	plans.

In	my	modeling work	at	Western,	I	never	look	at	the	“single	number”.		The	useful	output	from	the	

model	is	the	modeled hourly	operations in	megawatts	at	each	power	plant,	total	generation	over	

the	month, and	the	overall	purchase/sales dollar information.

Key	Exogenous	Data	and	Constraints,	pages	17, 18

Western	relies	on	inputs	from	Reclamation’s	RiverWare model	for	hydrologic	modeling.	The

general	structure	of	RiverWare	is	described	in	Zagona,	et	al	(2001).	As	we	understand	it,

Reclamation	uses	this	model	to	provide	Western	with	forecasts	of	monthly	water	quantities

available for	discharge	through	the	Glen	Canyon	Dam.	Reclamation	may	revise	these	forecasts	over

the	year	as	hydrologic	conditions	unfold.	With	a	monthly	water	allocation	as	input	GTMax	then

simulates	operation	of	the	Glen	Canyon	Dam	and	associated	facilities	according	to	various	economic

objectives	and	operational	constraints.	The	economic	factors	that	might	affect	the	monthly

allocation	of	water	across	the	year	are	opaque.	Casual	observation	of	various	GTMax	results

suggests	that	RiverWare	has	some	notion	that	water	is	more	valuable	for	power	generation	during

the	summer	and	winter	months	than	during	the	spring	and	fall.

See	my	comments	on	Riverware	in	the	Executive	Summary section.	

The	reviewers	appear to	have	the	relationship	between	Riverware	and	water	release	decisions	

reversed.		Reclamation,	at	Western’s	request,	programs	additional	water	into	the	summer	and	

winter	months	because it	is	more	valuable	to	Western then.		Riverware	then	takes	those	release	

preferences	and	incorporates	them	into	future	release	patterns.	In	this	section,	the	reviewers	imply

that	water	is	not	more	valuable	for	generation	in	the	summer	and	winter than	the	spring	and	fall.		

Western’s	power	allocations	to	its	customers	peak	in	the	winter	months	(primarily	December,	

January,	and	February)	and	summer	months	(primarily	July	and	August),	so	that	is	why	we	ask	
4

Reclamation	to	program	higher	water	releases	in	those	months. Additionally,	prices	for	firming	

power	purchases	also	peak	in	those	months so	we	try	and	avoid	making	purchases	then	when	

possible.

This	procedure	ignores	the	possibility	of	climate	change.	Under	a	climate	change	hypothesis,	some	

sample	selection	method	would	need	to	be	used	to	determine	a	going	forward	relevant	range	of	

hydrologic	variation	that	would	be	different,	arguably drier,	than	what	would	be	reflected	using	the	

complete	hydrologic	record.	We	discuss	this	point	further	below.

Reclamation	has	looked	at	potential	climate	change	effects	on	Colorado	River	water	resources	in	

the	recently	completed	Colorado	River	Basin	Water	Supply	and	Demand	Studyi,	done	in	conjunction	

with	the	seven Colorado	Basin	States.		DOE	has	also	recently	completed its	report	on	the	effect	of	

potential	climate	change	on	Federal	Power	Marketing	Administrations,	the	Section	9505	Report ii.		

Both	of	these	reports	specifically	address	potential	changes	to	Colorado	River	runoff	and	its	effect	

on	hydropower	operations.		Reclamation	can	incorporate	information	from	those	studies	into	

Riverware,	and	that	information	can	be	incorporated	into	GT-Max	modeling.

It	is	interesting	to	note	in	particular	that	the	Section	9505	Report	looked	at	projections	of	climate	

change	in	the	next	several	decades.		The	projections were	prepared	by	Oak	Ridge	Laboratory	on		

behalf	of	the	Department	of	Energy	and	showed	that	the	potential	changes	in	runoff	are	well	within	

the	historical	climate	variability	that	Western	has	experienced	over	the	last	20	to 30	years.		The	

projections	actually	showed	an	increase	in	Western	generation	in	several	of	the	river	basin	regions

of	our	service	area,	contradicting	the	drying	trend	in	the	western	states	that	is	generally	assumed	to	

be	the	result	of	global	warming,	and	is	predicted	in	the	Colorado	River	Basin	Water	Supply	and	

Demand	Study.	This	result	led	Western	to	conclude	in	the	report	that	our	existing	authorities	and	

operations	and	marketing	programs	were	adequate	to	accommodate	any	potential	changes due	to	

climate.

Capacity	Expansion	Algorithm,	pages	19,	20

This	section	notes	the	lack	of	capacity	expansion	planning	in	GT-Max.		Since	Western	has	no	legal	

authority	to	add	capacity	or	serve	load	growth,	it	is	understandable	why	GT-Max	would	not	include	

this	feature.		It	might	be	easier	to	just	access	capacity	expansion	data and	model	output and	import	

it	into	GT-Max	when	a	longer	term	study	is required,	rather	than	adding	the	bulk	and	complexity	to	

the	existing	model.

Reserves	commitment	and	ancillary	services	algorithms,	pages	20	and	21

See	my	comments	to	the	Three	Business	Models	Section.	The	reviewers	again	advocate	for	using	

Western’s	hydro	system	for	providing	ancillary	services	to	renewable,	non-dispatchable	generation.		

This	advocacy	seems	misplaced	in	a	review	of	the	GT-Max	model.	 Perhaps	the	reviewers	saw	this	

report	as	their	only	opportunity to	influence	Western’s	policy	direction.

Unit	reliability	algorithm,	page	21
5

The	reviewers	are	correct	to	note	that	GT-Max	treatment	of	unit	outages	is	less	that	optimal.		GTMax	simulates	power	plants	as	a	lumped	total,	rather	than	at	a	unit	level,	and	any	plant	deratings	

are	in	monthly	increments.		In	my	experience	using	GT-Max,	accurately	modeling	unit	outages	is	

one	of	the	more	challenging	and	time	consuming	aspects	of	the	job.		It	is	my understanding	that	the	

new	Hydro	model	Argonne	is	creating	will	significantly	improve	on	this	aspect,	being	able	to	model	

individual	units	on	time	scales	of	less	than	a	month.

Long-term	risk	algorithm,	pages	21	and	22

I	disagree	with	the	reviewers thoughts	that	GT-Max	should	include	long-term	risk	algorithms.		I	

think	that	computer	models	that	become	too	large	and	complex	become	unwieldy	to	use	and	more	

prone	to	giving	inaccurate	results. If	you	want	a	good	laugh,	go	back	and	read	some	of	the	reports	

that	were	produced	20	years	ago	using	those	expansion	models	and	see	what	they	were	projecting.		

I	agree	that	long-term	risk	is	a	worthy	subject	to	study,	but	think	that	it	should	be	done	separately	

from	GT-Max.

Energy	Market	Prices,	pages	22	and	23

All	of	the	economic	studies	assume	the	same	prices	regardless	of	operational changes	at	the	Glen	

Canyon	Dam.	The	underlying	assumption	is	that	whatever	changes	might	possibly	occur,	they	are	

too	small	to	matter	given	the	large	size	of	the	market.

I	suspect that	generation	changes	at	Glen	Canyon and	CRSP	in	general	could,	in	the	most	extreme	

circumstances	(High	Flow	Experiments	or	very	low	flow	drought	scenarios), affect	local power	prices	

by	flooding	the	market with	surplus	generation,	or	by	buying	most	available	generation	surpluses.		I	

don’t	have	any	evidence,	however,	it	is	just	a	hunch.

I	agree	with	reviewers	that	the	ability	to	incorporate	a	probability	distribution	of	power	prices	

rather	than	a	single	price	would	be	an	improvement	to	GT-Max.		That	would	require	multiple	runs	

of	the	model	to	look	at	the	entire	range	of	prices,	which	would	add	considerably	to	the	complexity	

and	model run	time.

Ancillary	Services,	pages	25	and	26

The	reviewers	again	advocate that Western	enter		the ancillary	services	market in	the	Western	

Interconnection. See	my	comments	above in	the	Reserves	commitment	and	ancillary	services	

algorithms section	and	the	Three	Business	Models section.

Capacity	value—lack	of	conceptual	clarity,	pages	26	and	27

I	agree	with	the	reviewers	that	Western’s	treatment	of	capacity	and	its	value	is	inconsistent.		Most	of	

the	monthly	GT-Max	operational	modeling that	 I	perform doesn’t	take	capacity	value	into	account	at	

all,	since	we	firm	our hydro	capacity	almost	completely	with	firm	energy	purchases rather	than	capacity	

purchases.		In	other studies Western	has	done, capacity	value is	included,	but	the	values	vary	 based	on	

what	 information	we	have	available	at	the	time.		I	think	the	lack	of	good	sources of	knowledge	of	what	
6

capacity	sells	for	in	the	Western	Interconnection	is	a	big	impediment – more	so	than	any	deficiency	in	

GT-Max.

Riverware,	page	28

See	my	comments	on	Riverware	in	the	Executive	Summary section.	

Large-Scale	and	Long-Term	Risk,	page	28

See	my comments	in	the	Key	Exogenous	Data	and	Constraints section	above.		The	reviewers	

misunderstand that	Western,	as	a	wholesale	supplier	of	a	defined	resource	base, has	no	responsibility	

for	load	growth	or	changes	in	retail	loads.		Western	can’t	just	decide	to	change	its	business	model	

without	change	to	the	legislation	that	governs	our	operations.

Conclusions	and	Recommendations,	pages	30	through	32

I	think	all	these	subjects	have	been	covered	in	my	comments	above.

																																																												 i

http://www.usbr.gov/lc/region/programs/crbstudy/finalreport/index.html

ii Department	of	Energy	(DOE)	Report	to	Congress	on	Climate	Change	and	Federal	Hydropower

(Section	9505	Report)	http://nhaap.ornl.gov/content/climate-change-impacts
Comments by Tom Veselka and Les Poch

Department of Energy, Argonne National Laboratory

on the Final Report Of The GTMax Model Review Panel:

Report of a Workshop held August 31and September 1, 2011

in Flagstaff, Arizona, dated September 4, 2012
Comments by Tom Veselka and Les Poch

on the

FINAL REPORT OF THE GTMAX MODEL REVIEW PANEL: 

Report of a Workshop held August 31 and September 1, 2011 in Flagstaff, Arizona

Draft of Tuesday, September 04, 2012

Argonne National Laboratory (Argonne) staff would like to thank the Grand Canyon Monitoring and 

Research Center (GCRMC) for hosting the “GTMax Model Review and Knowledge Assessment for 

Hydropower” workshop conducted in Flagstaff, Arizona on August 31 and September 1, 2011.

Participants included staff from GCRMC, the Western Area Power Administration (Western), Argonne, 

members of the review panel, and other interested individuals. 

The comments below are made by Les Poch and Tom Veselka from Argonne who presented GTMax 

materials at the workshop. Comments are mainly in response to information contained in the document 

entitled “Final Report of the GTMax Model Review Panel” that Western management received for 

comment on October 10, 2012. Note the review document was distributed to presenters more than 13 

months after the conclusion of the workshop. Therefore, instead of relying on our recollection of what 

transpired during the workshop, the comments below are primarily based on the review document and 

PowerPoint presentations given at the workshop.

Reviewer comments and suggestions address GTMax general modeling frameworks and methodologies 

as well as more specific aspects of hydropower modeling. Below we remark on several facets of 

reviewer comments, add background information about modeling, and provide justification for the 

GTMax structure, scope, and methodology.

Production Cost Modeling and Mixed Integer Linear Problem Formulation

In the 1st paragraph of the executive summary the review document states “GTMax belongs to a class 

of models known as production simulation models. The model’s primary objective is to simulate the most 

efficient operation of the Colorado River Storage Project (CRSP) hydroelectric power facilities at least 

cost. As is typical of such models, it simplifies what is in reality a very complex system of interrelated 

electrical generation facilities and decision factors, and therefore, like all such models, it is an imperfect 

representation of reality and has limitations in how it can be applied.” 

In general, we agree with the above reviewer comments as individual isolated statements. As modelers 

with a combined working experience of almost 70 years in power system modeling and analysis, we

recognize that no model is perfect. Furthermore, we acknowledge that proper model application and 

interpretation of results are of utmost importance. However, we would like to clarify that GTMax is 

more accurately classified as a production cost optimization model as opposed to a simulation model.

Also the objective of GTMax as it is applied to the CRSP system is to maximize the economic value of 

federal power resources using market price signals, not to minimize hydropower production costs as 

stated by the review panel. In the main body of the final report, reviewers state that “If purchases are 

less expensive than dispatching a Western unit, the model will displace the unit.” This is an inaccurate 

portrayal of GTMax and the dispatch of hydropower resources in general. CRSP marginal production 

costs are miniscule – there are no fuel costs and variable operating and maintenance (O&M) costs are 

for all practical purposes either zero or very small. Therefore, CRSP hydropower production costs are 
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

2

almost always less than market prices. Instead of using a production cost minimization objective, GTMax 

schedules limited water resources to maximize hydropower water resources. 

Readers of the external review document should also be aware that, although not perfect, production 

costs models are extremely useful. Readers should also realize that the optimization techniques that 

GTMax utilizes have been extensively used by other models in the past and continue to be widely 

applied by utility companies and academia throughout the world. 

The GTMax model is comprised of a large set of equations that are formulated as a network problem 

which describe the operation of one or more interdependent systems. In addition, it includes equations 

that represent physical processes, institutional limitations, and time sensitive scheduling problems. 

Although the GTMax model possesses several aspects of a network flow optimization problem, there are 

some fundamental differences from a pure network problem. Unlike traditional network flow models, 

GTMax contains more complex features. Properties of a resource may be altered as it flows through a 

network and/or over time and there are numerous temporal constraints, which are more common in 

sequencing and scheduling problems. 

Core mathematical relationships in GTMax require that equations are linear functions. In addition some 

variables are declared as “integers”; that is, solutions for these variables are required to be 

discontinuous integer number values. Solutions for other variables, referred to as “real variables” in the 

mathematical problem, are allowed to be continuous real number values. These two requirements place 

GTMax core functions into a classification that is commonly referred to as a “mixed-integer linear 

program (MILP)” problem. The solver that GTMax uses has been shown to solve real-life problems such 

as the ones required to accurately optimize Colorado River Storage Project (CRSP) operations with tens 

of thousands of variables and constraints. 

Water and hydropower systems are inherently nonlinear and discontinuous, seriously challenging the 

limitations of leading optimization software packages and the computational capabilities of our most 

advanced computer systems. While a Mixed Integer Non-Linear Program (MINLP) can most accurately 

represent the problems modeled by GTMax, the current state of MINLP technology limits its use to tiny 

models only. GTMax problems typically require thousands of variables and constraints. Considering the 

complex structure of GTMax and the large problem sizes it solves, we use a general-purpose MILP solver 

to find the solution.

Linear Programming (LP) techniques in electricity systems modeling have been widely used by 

academics and electric utility companies since the 1950’s. One study stated that the MILP approach 

shows the greatest potential for addressing the unit commitment problem for power systems that have 

combined-cycle generating units (Kinetrics, Inc. 2005). LP and MILP problem formulations have been 

used to determine Security Constrained Unit Commitment (SCUC) and Security Constrained Economic 

Dispatch (SCED) for the largest power grids in the world. Some electricity market simulation models 

using LP and/or MILP include AURORAxmp, PLEXOS, GridView, PROMOD, GE MAPS, and GenTrader.

AURORAxmp was developed by EPIS, Inc., in 1997 to model competitive wholesale electricity markets. It 

has been licensed to over 60 users such as electric utilities, independent power producers (IPPs), 

government agencies and regulators, electricity traders, hedge funds, and energy industry consultants in 

North America, Europe and Asia.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

3

PLEXOS was developed by Energy Exemplar to model electricity markets. By the beginning of 2012 

PLEXOS was installed at over 135 sites worldwide. It is currently being used by National Renewable 

Energy Laboratory (NREL) to study the impact of a proposed energy imbalance market (EIM) in the

Western Interconnect (WI).

GridView was developed by ABB and is a security-constrained unit commitment and economic dispatch 

model. It has been used by the California Independent System Operator (ISO), Southwest Power Pool, 

the New York ISO, NREL, and others. It was used to perform a cost/benefit analysis of a proposed EIM 

for WECC in 2011.

PROMOD was developed by Venytx and has been used by energy companies for 30 years. It is currently 

being used by WECC to perform integrated energy and capacity analyses.

The Multi-Area Production Simulation Software (MAPS) is a chronological simulation model developed 

by GE. It has been used by numerous transmission companies, PJM, the New York ISO, the New England 

ISO, and the Midwest ISO to perform reliability studies, and transmission and generator expansion 

studies.

GenTrader was developed by PCI in 1999. It has been used by utilities such as South Carolina Electric & 

Gas, Mirant, Kansas City Power and Light, and utilities in the deregulated Texas market such as Reliant 

Energy, TXU, and Calpine Corp.

Last, but not least, GTMax and its predecessor, the Hydro LP model, have been used for approximately 

20 years. When it was first developed no production cost model was available to adequately address 

hydropower issues at a level of granularity that Western needed to conduct analyses in support of a 

Power Marketing Environmental Impact Statement (EIS). Western funded Argonne to create a model 

that mimics its core business function – the dispatch of the Colorado River Storage Project (CRSP). The 

basic design however is very flexible and can also be used to optimize operations of other systems. 

Consequently GTMax projects have been sponsored by the Bureau of Reclamation (Reclamation),

International Atomic Energy Agency (IAEA), USAID, World Bank, Fichtner GmbH & Co KG (Germany), 

Adica Consulting, ENRON, and Chubu Electric Power Company (Japan). These studies have been applied 

to hydropower-centric problems in the United States, Asia, Africa, and several regions in Europe.

The review panel concluded that among production cost models GTMax is exceptional in its 

representation of hydroelectric systems. We are pleased that reviewers gave GTMax high marks for the 

purpose it was designed; that is, optimizing the economic value of hydropower plants under a large 

range of operating criteria and constraints. Determining economic value is a core task that is performed

under EISs that examine operational changes in electric power systems. 

GTMax has a hydropower-centric design and is tailored to perform specific tasks. It focuses on what is 

important for problems that are of interest to Western and CRSP stakeholders. This design helps it 

overcome some of the shortcomings of dispatch models that are designed for more general and broader 

applications such as imperfect modeling of “inter-related electricity faculties” and “decision factors” that 

review committees members listed in the 1st paragraph of the executive summary.

Not only does GTMax better dispatch hydropower plants than the vast majority of production cost 

models, it also better reflects actual “decision factors” regarding water operating criteria as it respects 

dispatch guidelines and goals that are specific to the CRSP system. It also addresses “inter-related 
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

4

electricity facilities.” The GTMax CRSP topology links CRSP power facilities together as an interconnected 

system and explicitly simulates the connectivity of water resources, the flow of water throughout the 

systems, and water related impacts on power resources. In addition to water resources, GTMax

simultaneously schedules Western’s federal hydropower resources and engages in short-term firm 

bilateral power market transactions and spot market activities to meet its obligations to serve firm 

loads. 

The review panel recognized that GTMax is able to account for and simulate the effects of these 

relationships among CRSP facilities. However, reviewers stated “Glen Canyon Dam is just one component 

of the CRSP system, and decisions made at one facility affect all the others, so there are significant 

challenges associated with extracting effects of Glen Canyon Dam operations from the rest of the system 

using this model.” Reviewers failed to mention that Western and Argonne staff devised an elegant and 

effective methodology for isolating and assigning these effects to Glen Canyon Dam. The technique was 

described in the workshop presentation titled, Methods for Representing CRSP Resources.

Reducing Operating Objectives and Constraints to a Single Number

GTMax determines values for “decision variables” that maximize a given function, called the “objectivefunction,” while satisfying given inequalities of other functions called “constraint-functions.” The 

executive summary of the review panel document discusses the “single number” that the model 

produces. This number is essentially the GTMax objective function value. For CRSP applications, the 

objective function maximizes the economic value of CRSP energy resources over a user-specified time 

period. This “bottom-line” economic value is comprised of a multitude of additive components. Readers 

not knowledgeable in LP and MILP modeling should realize that the objective function in large problems

such as those posed by the CRSP system is dependent on numerous variables. 

The GTMax model determines values of decision variables for all chronological hourly time steps during 

the model timeline. This timeline typically spans a one-week period. This time chronology feature of 

GTMax allows it to simulate a host of operational aspects, such as generator ramp-rate limitations, unit 

commitment schedules, thermal unit starts and stops, and multi-time-period constraints, including 

maximum and minimum water releases from a reservoir over the simulated period. When optimizing, 

the model recognizes that the solved state of the system at any point in time affects operations at all 

other points during the optimized time period. Simulated time is also critical for solving network 

interdependency problems, such as cascaded reservoirs, because GTMax recognizes reservoir 

connectivity and the time it takes water to flow through a system.

Therefore, in addition to producing a value for the objective function, GTMax also solves for and 

produces literally thousands of other results in a typical CRSP optimization run. Some key model outputs 

include hourly generation levels for each hydropower plant, power and non-power reservoir water 

releases, reservoir storage volumes and associated forebay elevations, downstream water flow rates, 

river stage, and marginal values for both water and energy throughout the interconnected network. 

Therefore, GTMax should not be viewed as a “black-box” model that produces a single inexplicable

number. Instead each component of the objective function value is transparent, explainable, and 

traceable through an analysis of model outputs, constraints, scalars, and equation constants and 

coefficients. 

Workshop Purpose and Proposed Markets

One section of the review panel report describes the purpose of the workshop. We agree with the 

review panel’s first sentence that states “The purpose of this workshop was to review the capabilities 
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

5

and uses of the GTMax model toward the investigation of possible changes in the operating regime of 

the Glen Canyon Dam.” However, in the text that follows we find a disconnect between what we were 

asked to present by GCRMC organizers and several of the statements made by the review panel. The 

review panel report discusses “proposed and future uses of GTMax and assess the applicability of the 

model to anticipated future economic and financial issues.” Reviewers then briefly describe how the 

power sector is evolving and different types of power sector business models. Then in the sections that 

follow reviewers are critical of GTMax in its present form since it may be ill-equipped to model potential 

changes in the Western Interconnect (WI) such as evolving market structures and future supply resource 

mix – in particular, an increase in variable resources. 

Regarding the applicability of GTMax for future studies we would like to point out that GTMax is not a 

static model. Instead, it has been extensively modified and enhanced to meet the evolving needs of 

Western to reflect changes in reservoir operating criteria and alterations in Western’s marketing goals 

and objectives. In our opinion it is premature for Western to expend some of its limited resources to 

either upgrade GTMax or develop new models to better represent potential market structures and 

business models that are not yet applicable to the CRSP system. Reviewers specifically refer to the 

proposed WI Energy Imbalance Market (EIM). At the time the workshop was conducted EIM market 

rules and structures were under development. Although more than16 months have passed, EIM rules 

have not yet been fully defined and agreed upon. Basic economic analyses continue to be conducted 

and it is not certain if the envisioned structure of the WI-wide EIM will materialize in the foreseeable 

future. Western has generously funded GTMax upgrades on an as-needed basis to meet specific needs 

or in support of well-defined projects. It is anticipated that future upgrades will be made as deemed 

necessary by Western or in support of GTMax projects funded by other organizations.

Argonne and Western staff are fully aware of changes in the WI markets and are engaged in EIM related 

activities via participation in webinars, workshops, internal meetings, and through extensive reviews of 

EIM reports and presentations. In addition, at the request of Western, Argonne has performed an 

extensive analysis of EIM models, methods, and results to help Western make more informed EIM 

decisions (Veselka et al. 2012); the report is available to the public at the Department of Energy (DOE) 

Information Bridge web site at http://www.osti.gov/bridge/.

In addition, several years ago Argonne developed the Electricity Market Complex Adaptive System 

(EMCAS) to model the behavior of participants in electricity markets under a well-defined set of market 

rules (DIS 2007). The model has been applied to studies in both the U.S. and Europe. Through this work 

and our extensive study of both domestic and international markets we disagree with reviewer 

statements that suggest that “the fundamental economic behavior driven by cost minimization provides 

a common foundation that underlies the behavior of market participants in both market and non-market 

segments of the industry.” In contrast we find that in a competitive marketplace, market players acting 

as autonomous agents, are driven by corporate objectives such as profit maximization and/or to 

increase market share. This is in stark contrast to traditional markets in which the cost-minimization

objective is the primary force behind utility behavior. In a perfectly competitive market in which there 

are no barriers for new market entrance, the end result in theory may be cost minimization. However, 

no such market exists – especially in the utility sector in which market players under some conditions

can influence market clearing prices. Later in the review document it states “Formal electricity markets 

sharpen the profit incentive of all participants ....” The statement appears to conflict with prior 

statements since there are stark differences between cost minimization and profit maximization 

objectives.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

6

If we had been informed that the workshop would focus more on how Western may model future 

markets and conditions, the content and focus of workshop presentations would have been different. 

We would have discussed market issues and ways in which EMCAS algorithms could potentially be 

leveraged to better represent anticipated changes in the WI and Western’s response to these changes.

However, it is premature to develop a model for a voluntary EIM that is not yet fully defined and may 

never be implemented. 

Review panelists also suggest that Western engage in marketplace activities that overstep its statutory 

boundaries and in some instances are physically impossible. Some of the statements include 

“opportunities for western generation asset owners to increase returns on operation of their plants”,

“the potential to expand the opportunities for profitable operation”, “bid into California markets when 

either energy or ancillary services prices make it financially attractive”, and engage in day-ahead and 

real-time market “arbitrage.” Western is a U.S. government agency that schedulers federal resources; it 

is not a publicly-traded company that is obligated to maximize shareholder value. Instead its primary

mission is to maximize the economic value of federal resources. Ultimately these federal facilities belong 

to the citizens of the U.S. Western must abide by the Colorado River Storage Project Act, its statutory 

and delivery obligations, and WAPA’s prevailing contractual delivery obligations to existing customers.

CRSP resources are already fully dedicated to existing contractual obligations, regardless of specific 

Colorado River Basin hydrological condition. This includes Western’s statutory obligation to market 

federal project hydropower at the lowest possible rates to consumers consistent with sound business 

practices. In our view, Western has neither the resource flexibility nor the discretion to pursue reviewer 

suggestions.

Ancillary Services and Higher Penetration of Variable Resources

Regarding comments concerning the model’s treatment of ancillary services we find that reviewers were 

not adequately informed about or did not fully grasp the model’s capabilities. The reviewers state that

“GTMax de-rates hydroelectric generation capability to capture the cost to provide for these services.” 

This statement is correct as it applies to the treatment of ancillary services in the CRSP system. We 

utilized this methodology for CRSP because it adequately captures the impacts of providing ancillary 

services on hydropower plants operations and system economics. However, reviewers also state that 

“GTMax fails to make commitments for security reserves, even at the hourly time step” and later in the 

text “the absence of ancillary service valuation capacity is conspicuous.” These statements are incorrect.

Not only can GTMax explicitly model ancillary services, it also contains options to represent spinning 

reserves on an hourly basis as spare generating capacity. Units that provide spinning reserve services 

must be synchronized to the grid (i.e., are in generation mode in GTMax). The regulation service is used 

to compensate for (or react to) very short-term (in the range of seconds) changes in the grid through the 

use of unit automatic generation controls (AGCs). A powerplant providing this service must also be in 

generation mode in GTMax. 

In GTMax units that supply ancillary services affect the range and flexibility of operation to serve load 

and/or for energy sales. Regulation up and spinning reserves reduces maximum energy schedules. That 

is, the sum of those two ancillary services plus scheduled energy production at all times must be less 

than or equal to the maximum operational capacity of the units. For hydropower operations GTMax also 

ensures that hydropower plants release enough water from reservoirs to accommodate the sum of the 

minimum flow rate requirement plus regulation services down. The GTMax user sets ancillary service 

limits by unit/plant and the model then optimizes ancillary service assignments to meet total system 

and/or subsystem requirements. We find that other production cost models use a similar approach.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

7

Although the current GTMax treatment of ancillary services is adequate under current conditions, the 

situation may change in the future. The panel review states “With Western and the WECC entering into 

a period of increasing generation from renewables and therefore also possibly the need for ancillary 

services....” We agree that there is a trend toward more renewables (more specifically variable 

resources) in the WI. However, it should be noted that Western’s current ancillary service obligations 

are based on its participation in a reserve sharing group and it will sell additional services through 

bilateral agreements. Also, Western will not construct any variable resources in the foreseeable future 

and at this point in time there are no plans for forming ancillary services markets in the larger WI

footprint. Ancillary services markets are outside the realm of the proposed WI EIM. It is prudent for 

Western to wait and gain more information about how the system will evolve and about future 

institutional arrangements before making modifications to either GTMax or adopting a new model. 

Regarding ancillary services, reviewers also state “GTMax cannot simulate operational periods of less 

than one hour.” We acknowledge that it cannot solve for movements at sub-hourly time intervals such 

as instantaneous variations in load, continuous changes in generation as a unit ramps between set 

points, and second-by-second fluctuations in production as units respond to AGC signals. Improvements 

can be made in this area. To increase and more effectively utilize hydropower in the U.S. the 

Department of Energy (through Western) entered into a memorandum of understanding with the 

Department of Interior (through Reclamation) and the Department of the Army (through the Army Corp 

of Engineers). This partnership is assisting a team of specialists from 4 DOE laboratories to demonstrate 

a newly developed Water Use Optimization Toolset (WUOT) at the CRSP system. One component of 

these toolset models is a very detailed, high fidelity hydropower day-ahead scheduling and real-time 

dispatch tool that operates at user-specified time steps. At present it is designed to model operations at 

time steps as short as 5 minutes. Another tool deals with inflow water forecasts, longer-term reservoir

water routing, and the environment.

The WUOT is also being demonstrated at the Oroville-Thermalito Complex which is in the California ISO 

and at the Conowingo/Muddy Run pumped storage facility located in the PJM footprint. Western’s 

participation in the WUOT project requires minimal expense and will help prepare for its next 

generation modeling needs. Application of the day-ahead scheduling and real-time dispatch tool to the 

CRSP system is ongoing. Preliminary results have been documented and sent to DOE for review.

Modeling Scope and Fidelity

Reviewers suggested that the scope of GTMax be expanded in time and space with a more refined level 

of granularity and fidelity. To some degree we appreciate this broader perspective and the notion that in 

principle, everything in this universe and beyond is connected. Sir Arthur Eddington wrote, “You cannot 

disturb the tiniest petal of a flower without the troubling of a distant star.” Unfortunately given our 

present state of technology and knowledge, there are practical limits to achieving this goal. No single

model can do everything and answer every question perfectly with a high degree of fidelity. Therefore, a 

crucial part of any modeling task is to determine what is important and what is not to answer a specific 

question within the scope and budget allocated to a project. We chose to concentrate on the central 

problem when applying GTMax. This strategy has been very successful and useful. For example, 

schedulers in the CRSP Energy Management and Marketing Office located in Montrose, Colorado review 

GTMax model results each month and use model outputs to aid in their decision making. This is a strong 

testimony that the model accurately represents of the dispatch of the CRSP system and provides both 

realistic and meaningful results.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

8

Our approach has been and continues to be to improve the water management and power dispatch 

functions in GTMax, rather than to expand its scope. As a model grows in scope, the problem size often 

becomes overwhelming and forces modelers to make compromises in terms of spatial and temporal 

fidelity. This tactic could potentially compromise the current integrity of the model in terms of its ability 

to produce realistic CRSP dispatch results. Furthermore, Western has no legal authority to build new 

capacity, such as wind and solar, or to serve load growth. 

To investigate some of the broader issues we leverage a set of specialized tools to feed pertinent 

information into GTMax. Some of the toolset models (and/or model outputs) that we have used in 

conjunction with GTMax in the past include: (1) WASP - capacity expansion results provide GTMax with 

new thermal and hydropower plant additions; (2) Aurora model - results supply GTMax with estimates 

of future locational marginal prices (LMPs); (3) Riverware - provides GTMax with estimates of future 

monthly water releases and reservoir storage levels; and, (4) VALORAGUA - provides GTMax with 

hydrological information in a multi-country region. This approach allows GTMax to focus solely on 

weekly CRSP operations based on a set of operating limits, goals, and guidelines to evaluate operations 

in terms of (1) the economics/financial value of energy; and, (2) plant-level “operating” capacity while 

leveraging results and insights produced by other models to help guide the hourly dispatch from a 

longer term and broader perspective. 

In our opinion, using the toolset approach is a practical and reasonable methodology for conducting 

studies within the confines of project resources. However, the review panel report states “The reviewers 

had concerns, however, with the model’s application for other purposes for which it was not originally 

designed and is not well-suited. Specifically, the model is not well-suited for forecasting economic 

implications of long-term operational scenarios.” We disagree with the reviewers on this point. Without 

challenge from numerous affected parties over the last 20 years, GTMax and GTMax-light have been 

used for this exact purpose. This encompasses numerous studies in support of EISs, and both economic 

and financial analyzes of CRSP facilities. GTMax performs an economic dispatch under a very specific set 

of conditions. This set of conditions may be for today, next week, a year from now, or 50 years into the 

future. Basic dispatch principles are not expected to change; however, model inputs as guided by 

specialized long-term tools will vary during the course of a long study time horizon. For example, for 

long-range studies, GTMax relies on results from the Riverware model for future estimates of reservoir 

water releases and reservoir storage volumes. This allows GTMax to alter its dispatch in response to 

anticipated changes in hydrological conditions over time. To the extent that the Riverware run has 

incorporated the effects of global warming into its projections, these will also be reflected in GTMax 

model results. In this context, GTMax is one piece a model toolset; it is not the sole analytical tool.

Reviewers are concerned about the applicability of GTMax for long-term studies because it does not 

expand system capacity. The review document states “There are other reasons for having capacity 

expansion capability in a simulation model. Without such capability, simulated market prices for 

wholesale firm energy would reach unsustainable levels ....” We agree that new capacity will be required 

in the future to maintain reasonable prices. However, embedding MILP equations in a production cost 

model to expand capacity is not the only approach or always the best method of estimating future WI 

prices. Reviewers point to research that “argues that the perfect information assumption of production

simulation models and other simplifications make them a poor predictor of electricity market prices.” For 

CRSP applications, we typically project prices based on futures market indices. As required we can 

extend forecasts using Energy Information Administration electricity prices projected by the National 

Energy Modeling System (NEMS) through 2040. This model has a very broad national perspective that 
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

9

goes beyond the WI and the electric sector. We have also used projections made by other models such 

as Aurora.

We acknowledge that using a fixed price vector has a conceptual weakness since market prices do not 

react to actions taken at Glen Canyon Dam. One approach that we used to model hydropower systems 

in the Iberian Peninsula is to use a market price curve instead of a static value because alternative 

actions often resulted in significant changes in system marginal production costs. However, changes in 

Glen Canyon Dam operating criteria affects only 200 MW to 400 MW of capacity; that is only 0.1 % to 

0.2% of the total WI capacity. Therefore, in past model applications we assumed that capacity swings of 

this magnitude would have only miniscule impacts on market clearing prices. We based this assumption 

not only on the small change in generation swings, but the fact that the WI marginal fuel is 

predominately natural gas. Nevertheless, Argonne is conducting a simplified WI analysis to further study 

the validity of this assumption. 

It should be noted that GTMax has the capability to compute LMPs instead of using exogenous market 

prices. This modeling option has been exercised in GTMax applications in Europe, Asia and Africa. For 

long-term applications, we always incorporated exogenous capacity expansion schedules into GTMax 

topologies. Also, in each of these studies there were adequate data available and sufficient support from 

utility system experts to perform realistic price calculations.

From comments in the review panel report, it appeared reviewers did not fully understand and

appreciate the approximation methods that were used for evaluating capacity costs when time and 

money do not allow for a more thorough analysis. One of these approximation methods was used for 

two Reclamation EISs and approved by Reclamation, Western, and Western customer representatives. 

During the Power Marketing EIS, which was led by Western, and the Glen Canyon Dam EIS, which was 

led by Reclamation, a more rigorous approach was implemented. Modelers and analysts prefer a more 

detailed approach, but it is also very expensive and time consuming. In these studies capacity expansion 

models were run for each of Western’s largest customers for each EIS alternative. Experts from each of 

the utilities modeled provided advice and supplied detailed information about systems operations, 

objectives, and the slate of candidate expansion technologies. 

It is our impression that reviewers would like to employ an even larger and broader perspective by 

estimating the capacity expansion impacts on the Western Interconnect (WI). Such a model may 

generate numbers, but it is important to recognize when model “answers” are meaningful. We contend 

that capacity expansion for the entire WI is based on judgments made by a multitude of independent 

decision makers. Each has unique objectives, operating conditions, and financial status. Coupled with 

federal and state laws, renewable portfolio standards (RPS), wind and solar integrations, uncertainty, 

unique system-level hydropower/water conditions and regulations, national policies etc. makes 

modeling realistic capacity expansions on a WI level extremely difficult.

Connection issues in the WI are extremely complex and interwoven which makes some of the broader 

issues beyond the mission and capacity of Western. For example, reviewers suggested that GTMax 

should quantity the grid benefits of ancillary services. This one task by itself is an enormous problem. 

Over the last several years, DOE and other organizations such as EPRI have sunk many millions of dollars 

investigating the grid value of hydropower. Argonne staff members have participated in several 

hydropower grid services meetings and have reviewed project documents. More recently DOE has 

funded Argonne to lead a team comprised of national laboratory experts, prominent hydropower 

consultants, software developers, and utility experts to perform further modeling and analysis on this 
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

10

topic. This project is in addition to the WUOT mentioned earlier. Similar massive studies, years in the 

making, investigate WI wind and solar integration, energy markets, the system-wide level 

representation of hydropower, global warming effects, etc. These investigations are ongoing and will 

continue into the foreseeable future. To the extent possible, it may be possible to leverage information 

and/or products generated by these projects. 

These WI power issues and very long-term trends such as climate change are of great interest, but in our 

opinion the marginal value of incorporating these factors into CRSP studies for evaluating the power 

economics of Glen Canyon Dam under alternative operating criteria are very small, exorbitantly 

expensive, and the confidence level of the model results would be low. We base this option on the 

following rationale. 

(1) Alternative operating criteria imposed at Glen Canyon Dam has very little or no impact on the 

annual amount of power it produces. In an average hydrological year this accounts for less than 

0.6% of WI generation and will progressively decrease in percentage terms over time.

(2) Alternative operating criteria typically increases or decreases Glen Canyon Dam operating 

capacity by only 200 to 400 MW. This represents only 0.1% to 0.2% of WI generating capacity 

and it will progressively decrease in percentage terms over time.

(3) The marginal fuel in WI interconnect is predominately natural gas which is abundant and 

projected to continue to be inexpensive. Therefore, in theory any change in production cost as a 

result of Glen Canyon operations is primarily a function of changes in heat rate making the cost

curve relatively flat. This is the primary reason that pumped storage power plants are rarely 

used for arbitrage in current WI markets.

(4) Since economic analyses focus on differences among alternatives and not absolute values, the 

effects of long-term large scale issues such as climate change on the decision-making processes 

are significantly diminished compared to changes in absolute values.

(5) WI operations are influenced by a multitude of autonomous decision makers in a massive 

system that is not only exceptionally diverse and complex, but interwoven with water, 

economic, and political/legal considerations that transcend even the most sophisticated models. 

Although models may produce “numbers” based on economic theory, this result is very difficult 

to defend given the simplifying assumptions that are required and the quality and granularity of 

model input data. Especially when operational changes at Glen Canyon Dam comprise a small 

fraction of the total grid resource.

Therefore, it is our opinion that it would be a mistake for GTMax to shift focus away from the “flower 

petal” and on to the “distant star.” At the CRSP level, Western has adequate information and data along 

with the expertise to produce high quality results. In addition, CRSP relationships are well defined and 

understandable with a high degree of fidelity. To the extent that changes in long-term large scale grid 

operations can be measured with an acceptable level of accuracy we rely on other models to provide

GTMax with direction. 

Concluding Remarks

In the introduction the review states that the objectives of the report are “to review the GT Max model 

and its uses for economic analysis of issues relevant to the Glen Canyon Dam Adaptive Management 

Program.” It appears the focus of the report significantly strayed from this objective throughout much of 

the text. Instead, it discusses market issues that are not yet applicable to Western and futures that may 

or may not come into fruition. We are pleased that reviewers found GTMax exceptional in its ability to 

perform hydroelectric analyses. Unfortunately, most of the review discussed issues that are beyond the
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

11

current design and scope of the model; that is, it mainly discussed what GTMax does not do. It focused 

on the inability of GTMax to adequately represent potential WI-wide changes that, from their viewpoint,

may be important in the future. It also suggests the model include actions that are outside the 

boundaries of Western’s statutory obligations.

The reader of the review should recognize that GTMax is not a stagnant model, but it evolves to reflect 

changing conditions. As instructed by workshop organizers, our presentations were geared toward 

describing the model and past applications. If informed prior to the workshop that the focus was on 

issues such as global warming, different utility business models, capacity expansion, and ancillary 

services, we would have structured our presentations accordingly to describe how GTMax can be 

integrated with other tools to address these issues. 

While we have an appreciation for longer-term and broader perspectives discussed by the reviewers, it 

should be recognized that no model can do everything. Therefore given limited resources we have 

developed GTMax to addresses critical operational issues as it pertains to the CRSP system. The 

narrower scope of GTMax should be viewed as a strength, not as a weakness. It allows GTMax to 

produce realistic results that are both useful and used with acceptable run times. A testimony to the 

validity is that it is used on a monthly basis by the CRSP Energy Management and Marketing Office to 

make CRSP-related decisions. By broadening the scope of GTMax this strength may become diluted. 

Although there are some very important and interesting topics raised by the reviewers, it is our opinion 

that some modeling suggestions are extremely costly to implement and of small marginal benefit in 

terms of addressing pertinent questions related to the economic costs of altering Glen Canyon Dam 

operations.

As with any model, it is important to make decisions regarding what issues are of importance to the 

problem it addresses. Reviewers however, discuss broad WI issues that span multiple decades, yet 

advocate a modeling time step of 5 minutes or less over a vast and complex system. It is our opinion 

that these issues are beyond the direct scope of issues related to the Glen Canyon Dam Adaptive 

Management Program and best addressed in other forums.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

12

References

DIS (Decision and Information Sciences Division), 2007, Electricity Market Complex Adaptive System 

(EMCAS) User’s Manual – Version 2.0, Argonne National Laboratory Report, December.

Kinetrics, Inc., 2005, Commitment Techniques for Combined-Cycle Generating Units, prepared for CEA 

Technologies, Inc. (CEATI), CEATI Report No. T053700-3103, Dec. 

[http://www.nyiso.com/public/webdocs/markets_operations/committees/bic_mswg/meeting_material

s/2006-02-15/3103_Commitment_Techniques_for_CCGUs.pdf], accessed January 3, 2013.

Veselka, T.D., L. Poch, and A. Botterud, 2012, Review of the WECC EDT Phase 2 EIM

Benefits Analysis and Results Report, ANL/DIS-12-2, Argonne National Laboratory, April.

Bibliography

Abhishek, K., S. Leyffer, and J.T. Linderoth, 2010, “FilMINT: An Outer Approximation-Based Solver for 

Convex Mixed-Integer Nonlinear Programs,” INFORMS Journal on Computing, vol. 22, pp. 555–567.

Achterberg, T. 2009, “SCIP: Solving Constraint Integer Programs,” Mathematical Programming 

Computation, vol. 1, no. 1, p. 1–41.

Ahuja, R.K., T.L. Magnanti, and J.B. Orlin, 1988, Network Flows: Theory, Algorithms, And Applications, 

Prentice Hall, NJ.

Belotti, P., 2009, COUENNE: A User’s Manual, Department of Mathematical Sciences, Clemson 

University, Clemson, SC, available at http://www.coin-or.org/Couenne/couenne-user-manual.pdf, 

accessed April 23, 2012.

Bonami, P., L.T. Biegler, A.R. Conn, G. Cornuejols, I.E. Grossmann, C.D. Laird, J. Lee, A. Lodi, F. Margot, N. 

Sawaya, and A. Waechter, 2008, “An Algorithmic Framework for Convex Mixed Integer Nonlinear 

Programs,” Discrete Optimization, vol. 5, no. 2, pp. 186–204.

FICO, 2012, “Test Drive the Xpress-Optimization Suite,” available at 

http://www.fico.com/en/Products/DMTools/xpress-overview/Pages/Xpress-Optimizer.aspx, accessed 

April 23, 2012.

Forrest, J.J., 2006, “Coin Branch-and-Cut Solver,” July 18, available at 

http://dimacs.rutgers.edu/Workshops/COIN/slides/forest2.pdf, accessed April 23, 2012.

Forrest, J.J., and E.M.L. Beale, 1976, “Global Optimization Using Special Ordered Sets,” Mathematical 

Programming, vol. 10, pp. 52–69.

Fourer, R., D.M. Gay, and B.W. Kernighan, 2002, AMPL: A Modeling Language for Mathematical 

Programming, Brooks/Cole Publishing Company.

GUROBI Optimization, undated, “An Easier Way to Better Decisions,” available at 

http://www.gurobi.com, accessed April 23, 2012.
Tom V. & Les P. Comments on GTMax Review Panel Report

January 2013

13

IBM, undated, “IBM ILOG CPLEX Optimizer: High-Performance Mathematical Programming Solver for 

Linear Programming, Mixed Integer Programming, and Quadratic Programming,” available at 

http://www-01.ibm.com/software/integration/optimization/

cplex-optimizer/, accessed April 23, 2012.

Mahajan, A., S. Leyffer, J.T. Linderoth, and J. Luedtke, 2012, “MINOTAUR wiki page,” Argonne National 

Laboratory, April 12, available at http://wiki.mcs.anl.gov/minotaur/

index.php/Main_Page, accessed April 23, 2012.

Papadimitriou, C. H., 1981, “On the Complexity of Integer Programming,” Journal of the Association for 

Computing Machinery, vol. 28, pp. 765–768.

Ralphs, T., M. Guzelsoy, and A. Mahajan, 2011, SYMPHONY Version 5.3 User’s Manual, COR@L 

Laboratory, Industrial and Systems Engineering Department, Lehigh University, Bethlehem, PA. 

Sahinidis, N.V., 1996, “BARON: A General Purpose Global Optimization Software Package,” Journal of 

Global Optimization, vol. 8, pp. 201–205.

Tomlin, J.A., 1981, “A Suggested Extension of Special Ordered Sets to Non-Separable Non-Convex 

Programming Problems,” in Annals of Discrete Mathematics (11) Studies on Graphs and Discrete 

Programming, P. Hanson (Ed.), North-Holland Publishing Company, Amsterdam, NY.

Wang, J., X. Wang and Y. Wu, 2005, Operating Reserve Model in the Power Market, IEEE Transactions on 

Power Systems, Vol. 20, No. 1, February.
Letter from Leslie James, Executive Director, 

Colorado Energy Distributors Association 

to David Lytle, Director, USGS Southwest Biological Center and 

Jack Schmidt, Chief, Grand Canyon Monitoring & Research 

Center, commenting on the Final Report Of The GTMax Model 

Review 
 

ARIZONA

Arizona Municipal Power Users Association

Arizona Power Authority

Arizona Power Pooling Association

Irrigation and Electrical Districts 

Association

Navajo Tribal Utility Authority

(also New Mexico, Utah)

Salt River Project

COLORADO

Colorado Springs Utilities

Intermountain Rural Electric Association

Platte River Power Authority

Tri-State Generation & Transmission 

Association, Inc.

(also Nebraska, Wyoming, New Mexico)

Yampa Valley Electric

Association, Inc.

NEVADA

Colorado River Commission

of Nevada

Silver State Energy Association

NEW MEXICO

Farmington Electric Utility System

Los Alamos County

City of Truth or Consequences

UTAH

City of Provo

City of St. George

South Utah Valley Electric Service District

Utah Associated Municipal Power Systems

Utah Municipal Power Agency

WYOMING

Wyoming Municipal Power Agency

Leslie James

Executive Director

CREDA

10429 S. 51st St., Suite 230

Phoenix, Arizona 85044

Phone: 480-477-8646

Fax: 480-477-8647

Cellular: 602-469-4046

Email: creda@qwest.net

Website: www.creda.org

CREDA

Colorado River Energy Distributors Association

January 7, 2013 VIA EMAIL

David Lytle - USGS

Jack Schmidt - GCMRC

RE: Final Report of the GT Max Model Review Panel (September 2012)

Dear David and Jack:

CREDA members appreciated the opportunity to participate in the August 

31-September 1, 2011 workshop held at GCRMC’s offices, and during day two of the 

workshop, presented information reflecting work undertaken by a CREDA planning, 

operational and analysis committee over the past couple of years. As a member of 

the committee, Dave Slick presented a utility perspective and there was discussion 

of the information amongst the panelists and participants. As some of the 

information contained in the September 4, 2012 Final Report reflects the utility 

perspective discussion, CREDA is offering the following observations on the report. 

As you read the following observations, please keep in mind that the 

stakeholder participants in the workshop have not been provided the specific charge 

or assignment given to the panel participants, so the following comments are based 

on our assumptions derived from reading the final report. We would welcome an 

opportunity to discuss the report with you or your staff to have a better 

understanding as to the task(s)/question(s) the panelists were asked to consider 

during the workshop and in preparation of the report. We are offering the 

following technical comments on the report, as well as offering a perspective on 

how the report may affect collaboration within the Glen Canyon Dam Adaptive 

Management Program (AMP).

1. Stated Objective Versus Resultant Work Product

In the last paragraph of the introduction on page 6, the author(s) state that their 

role was “to review the GT Max model and its uses for economic analysis of issues 

relevant to the Glen Canyon Dam Adaptive Management Program.” We are unable 

to understand how the report addresses that objective; rather, the report 

emphasizes a formal, centralized electric market advocacy position. 

Rather than being a constructive, useful tool to assess applicability of GT Max for 

AMP applications, the tone and content of the report are quite different than the 

actual workshop discussions among the participants. The report appears to focus 

on what the model doesn’t do, and what WAPA doesn’t do in fulfilling its statutorily 

mandated mission, as opposed to assessing the model’s applicability to AMP 

applications.

2. Concept of Peer Review 

It was our understanding that a purpose of the workshop was to provide a peer 

review of the GT Max model. We think that a peer is a person who has equal 

standing in some respect. We believe that the object of the federal government’s 

peer review process is, generally, to obtain an objective assessment of a body of 
work by a person who has a similar expert understanding about the subject, or body of work, as the 

person or persons who perform the work that is being peer reviewed. 

As stated in the report, GT Max is a production cost simulation model. Employees of WAPA and 

Argonne who presented information at the workshop and use the model in the conduct of daily 

business responsibilities are production cost simulation experts, with very specific, special knowledge 

and experience in dealing with the complicated nuances associated with CRSP hydro system 

operation. 

Based on many statements/conclusions/recommendations contained in the report, it appears to us 

that the author(s) may not be production cost simulation experts. Consequently, we are not 

convinced that an actual peer review has occurred. As pointed out by the author(s), there are a 

number of production cost models being used in the United States, and a large body of literature 

about the subject. 

We may be misinformed as to the purpose and scope of the workshop and role/charge to the 

panelists, but possibly that could be resolved if such information is made available.

3. WAPA’s Statutory and Contractual Obligations

In various places throughout the report, the author(s) suggest that WAPA should seek to improve its 

profit position, increase net revenues, bid its generation into California’s ancillary services market, 

maximize revenues, and other similar ideas. 

These suggestions signal a lack of understanding about the Colorado River Storage Project Act, 

WAPA‘s statutory obligation, delivery obligation concepts such as contract rate of delivery and 

sustainable hydropower, and WAPA’s contractual delivery obligations to existing customers.

In short, WAPA’s CRSP resources are already fully dedicated to existing contractual obligations, 

regardless of any year’s specific Colorado River basin hydrology (wet or dry). And WAPA’s statutory 

obligation is to market federal project hydropower at the lowest possible rates to consumers 

consistent with sound business practices. Those rates are structured to recover 100% of the 

construction and O&M costs allocated to power AND to repay a significant amount of the investment 

in irrigation projects in the upper Colorado River basin. That irrigation repayment obligation alone sets 

WAPA apart from any other power marketing administration.

In our view, WAPA has neither the resource flexibility nor the discretion needed to pursue the 

author(s)’ suggestions in a way that would have a material impact on either WAPA or western region 

electric business economics.

It is our opinion that these comments/recommendations are out of scope (as we understand it) and 

unfortunately could lead to a serious misunderstanding of these issues by AMP stakeholders. If there 

is a basic misunderstanding of these issues amongst the AMP stakeholders, it makes collaboration and 

consensus building, already a challenge, much more elusive.

4. Centralized Market Effectiveness in the Western U.S.

As reflected in the report, a viewpoint of at least one panelist during the workshop was that formal, 

centralized markets are the panacea solution for all electric business problems, and that WAPA can 

materially improve its economic welfare through increasingly robust participation in such markets.

An assessment of actual historic experience in the Western U.S. suggests that retail customers in 

California have not benefited from the presence of the centralized market in that state. 
The analysis included with this email transmittal (attachments 1-3) compares the change in prices for 

the California Independent System Operator (CAISO) participants between 1995 (the year prior to 

CAISO launch) and 2010 (the latest year for which information was available at the time the analysis 

was done) versus the change in prices in other Western States where no centralized markets exist. 

The essential message of all three attachments is that the organized wholesale market in California has 

not served to provide bottom line retail cost benefits to consumers relative to electric business 

management in other Western states during this 15 year period.

While conceding that near-term costs for consumers will go up, a March 16, 2012 DOE Secretary of 

Energy Chu memorandum expressed the belief that in the long run, consumer costs would go down as 

a result of centralized market formation. From the perspective of many of WAPA’s customers, 

centralized markets may not be “the” answer.

As with item 3 above, we believe that this portion of the report is out of scope and could lead to a 

flawed understanding by AMP stakeholders and readers of the report.

5. Revenue Versus Cost

In addition to numerous references to maximizing revenues and improving profits, the author(s) state 

that “minimizing costs may not be the most important objective”, and proceed to suggest that 

maximizing revenues would constitute a mathematically equivalent approach.

From a WAPA customer perspective, we believe that there is a significant policy-level difference 

between a statutory obligation that requires WAPA to market federal project hydropower at the lowest 

possible rates to consumers consistent with sound business practices, and a revenue maximization goal.

On one hand, the author(s) correctly observe that the objective function of the GT Max model is to 

minimize cost. In this regard, we believe that WAPA and Argonne have correctly oriented the tool so as 

to be reflective of WAPA’s legal obligation. 

On the other hand, the author(s) describe the business area where 100% of WAPA’s customers reside 

as “the regulated IOU business model”, which suggests to us that the author(s) may not understand 

the public power characteristics of WAPA’s customers.

The foregoing observations notwithstanding, the real damage in the revenue versus cost discussion is 

the promulgation of the misleading suggestion that utility business entities (WAPA, WAPA’s customers, 

other public power entities, IOUs, etc.) are exclusively focused on maximizing profits. This is 

particularly harmful in the AMP arena, where the majority of participants are not utility business 

professionals, and a perception that “profits trump natural resources” could be reinforced. 

As in items 3-4 above, the promotion of false perceptions such as this serves only to establish or 

strengthen barriers among AMP stakeholders who might otherwise be able to collaborate more 

effectively. 

6. Capacity Expansion Algorithms

The report’s author(s) suggest that the absence of a capacity expansion algorithm is a weakness of GT 

Max for the “types of analyses that Western is considering”.

In contrast, our understanding of WAPA’s scope of federal responsibility and mission is such that we do 

not understand why such modeling capability would ever be needed. WAPA has no retail obligation to 

serve, and no load growth responsibility. In analyses of alternative Glen Canyon and other CRSP hydro 
system flow regimes, the prospect of federal construction of new generating capability to offset capacity 

lost due to flow regime changes has, to our knowledge, never been contemplated.

Institutions that are in the business of using production cost models to forecast and study projected 

future generating system operation and costs (because they have the responsibility to plan for the 

future electric service requirements of their retail customers) do not use capacity expansion algorithms, 

and consider doing so to be an academic exercise that is not particularly helpful to the conduct of real 

world decision-making.

Finally, we think the suggestion that a capacity expansion algorithm is necessary to the production of 

superior electric price forecast information in future years for the valuation of capacity is not valid.

7. Other

There are other technical errors in the report. For example, the statement on page 21 that “the failure 

of a substation transformer or transmission line at one of the dams” (referring to Hoover and Glen 

Canyon) “takes out the entire project” is incorrect. The CRSP (Glen Canyon) and Boulder Canyon 

Project (Hoover) are separately authorized and marketed projects. Further, the Boulder Canyon 

Project does not have associated transmission, and the transmission system(s) used to market its 

output are not automatically “taken out” by a transformer failure or transmission line in the CRSP 

project (and vice versa). 

We are available to discuss these observations with you or your staff, the panelists, and/or the 

author(s) of the report. Please don’t hesitate to contact me at 480-477-8646, or Dave Slick at 602-236-

2082.

Sincerely,

/s/ Leslie James

Leslie James

Executive Director

Cc: CREDA Board

 Darren Buck – WAPA

Att: 4 attachments to email transmittal
Western States Retail Electric Price Information 

Western States Retail Electric Price Information Fig 1 May 2012.docx

Figure 1: Cent / KWh Increase Between 1995 and 2010

Description

Figure 1 displays the retail electric price increase in each state between 1995 and 2010 as measured 

in cents per kilowatt hour (Cents/KWh). For example, in Idaho the retail electric price in 2010 was 

2.45 cents/KWh higher than the retail electric price in 1995. 

Message

The retail electric price increase in California between 1995 and 2010 was significantly higher than 

every other Western state. The organized wholesale market in California (CAISO) did not serve to 

provide bottom line retail cost benefits to consumers relative to electric business management in other 

Western states during this 15 year period. 

Details

The Energy Information Administration (EIA) was the source for all data. California prices represent 

the average of those California entities that are CAISO transmission owners and serve retail 

customers (PG&E, SCE, SDG&E, Anaheim, Azusa, Banning, Pasadena, Riverside and Vernon). The 

investor owned utilities (PG&E, SCE, SDG&E) represent 96% of retail sales made by these entities.

2.56

2.89 2.45

3.63

4.33

3.23

1.88

1.64

2.07

3.03

1.63
Western States Retail Electric Price Information 

Western States Retail Electric Price Information Fig 2 May 2012.docx

Figure 2: Cent/KWh Differences Between California Electricity Prices and Other States

Description

Figure 2 displays retail electric price differences between California and other Western states during 

1995 and 2010 as measured in cents per kilowatt hour (Cents/KWh). For example, in 1995 the retail 

electric price in Utah was 5.10 cents/KWh lower than California, and in 2010 the retail electric price in 

Utah was 7.79 cents/KWh lower than California. 

Message

In 1995 retail electric prices in other Western states were significantly lower than retail electric prices

in California, and by 2010 the gap between retail electric prices in other Western states and California 

had widened. The organized wholesale market in California (CAISO) did not serve to provide bottom 

line retail cost benefits to consumers relative to electric business management in other Western states 

during this 15 year period. 

Details

The Energy Information Administration (EIA) was the source for all data. California prices represent 

the average of those California entities that are CAISO transmission owners and serve retail 

customers (PG&E, SCE, SDG&E, Anaheim, Azusa, Banning, Pasadena, Riverside and Vernon). The 

investor owned utilities (PG&E, SCE, SDG&E) represent 96% of retail sales made by these entities.

Cents / KWh

1995: 10.40 

2010: 14.73

1995: 6.30 

2010: 8.07

1995: 5.10

2010: 7.79

1995: 5.73

2010: 7.17

1995: 5.75

2010: 6.85

1995: 6.08

2010: 8.53

1995: 6.31

2010: 8.19

1995: 4.30

2010: 5.00 1995: 4.28

2010: 5.58

1995: 2.78

2010: 5.04

1995: 3.63

2010: 6.33
Western States Retail Electric Price Information 

Western States Retail Electric Price Information Fig 3 May 2012.docx

Figure 3: Annual Savings For A Family of Four Relative to California Electricity Prices

Description

Figure 3 displays the dollars that a family of four in other Western states saved relative to a family of 

four in California that consumed the same amount of electricity in 1995 and 2010. For example, in

Colorado a family of four saved $1,883 in 1995 and $2,678 in 2010 relative to a family of four in 

California that consumed the exact same amount of electricity in those years.

Message

For the same amount of electricity, the cost for a family of four in 1995 in all other Western states was 

lower than that for a family of four in California, and by 2010 the gap between California and other 

Western states had widened. The organized wholesale market in California (CAISO) did not serve to 

provide bottom line retail cost benefits to consumers relative to electric business management in other 

Western states during this 15 year period. 

Details

The Energy Information Administration (EIA) was the source for all price data. California prices 

represent the average of those California entities that are CAISO transmission owners and serve retail 

customers (PG&E, SCE, SDG&E, Anaheim, Azusa, Banning, Pasadena, Riverside and Vernon). The 

investor owned utilities (PG&E, SCE, SDG&E) represent 96% of retail sales made by these entities.

1995: $2,772 

2010: $3,874

1995: $2,244

2010: $3,739

1995: $2,521

2010: $3,442

1995: $2,530

2010: $3,288

1995: $2,675

2010: $4,094

1995: $2,776

2010: $3,931

1995: $1,892

2010: $2,400 1995: $1,883

2010: $2,678

1995: $1,223

2010: $2,419

1995: $1,597

2010: $3,038