Resource Adequacy Requirements: 

Reliability and Economic Implications

 

 September 2013 

Johannes P. Pfeifenberger 

Kathleen Spees 

The Brattle Group 

Kevin Carden 

Nick Wintermantel 

Astrape Consulting

Prepared for 
Acknowledgements and Disclaimer 

The authors would like to thank the FERC staff for their cooperation and support. In particular, we 

would like to acknowledge the contributions of Robert Hellrich-Dawson, J. Arnold Quinn, Michael 

P. McLaughlin, Jordan Kwok, Abdur Masood, Carl Pechman, and William Murrell of the FERC 

staff and Samuel A. Newell of The Brattle Group. Opinions expressed in this report, as well as 

any errors or omissions, are the authors’ alone. The examples, facts, and requirements 

summarized in this report represent our interpretations. Nothing herein is intended to provide a 

legal opinion. 

Copyright © 2013 The Brattle Group, Inc. and Astrape Consulting. This material may be cited subject to inclusion of 

this copyright notice. Reproduction or modification of materials is prohibited without written permission from the 

authors. 
i 

TABLE OF CONTENTS 

Executive Summary ..................................................................................................................... iii

I. Introduction and Background .............................................................................................. 1

A. Study Purpose and Approach .............................................................................................1

B. Approaches to Setting Resource Adequacy Standards ......................................................2

1. Interpreting 1-in-10 and Other Reliability Criteria .......................................................2

2. Determining System Reliability as a Function of Planning Reserve Margin ...............3

3. Determining Economically “Optimal” Reserve Margins .............................................5

C. Survey of Resource Adequacy Criteria in North America .................................................7

1. Differing Resource Adequacy Standards ......................................................................8

2. Differing Conventions for Calculating Reserve Margins .............................................8

3. Differing Approaches to Conducting Reliability Modeling .......................................14

II. Approach to Modeling the Economics of Resource Adequacy ........................................ 17

A. Strategic Energy Risk Valuation Model Overview ..........................................................17

B. System Topology and Transmission Assumptions ..........................................................19

C. Resource Assumptions .....................................................................................................20

1. Resource Mix ..............................................................................................................20

2. Energy Market Supply Curve .....................................................................................22

3. Generation Outage and Availability Modeling ...........................................................23

4. Economic and Emergency Demand Response Resources ..........................................24

D. Load Modeling .................................................................................................................28

1. Weather Uncertainty ...................................................................................................28

2. Economic Load Growth Forecasting Uncertainty ......................................................29

E. Scarcity Conditions ..........................................................................................................31

F. Marginal Resource Cost and Performance .......................................................................35

G. Base Case and Alternative Simulation Cases ...................................................................36

III. Simulated Reliability Levels and System Cost Results .................................................... 38

A. Base Case Economic and Reliability Results ...................................................................38

1. Reserve Margins Needed to Achieve Physical Reliability Standards ........................38

2. Minimizing Cost from a Risk-Neutral, Cost-of-Service Perspective .........................41

3. Minimizing Cost from a Risk-Neutral, Societal Perspective ......................................47

4. Risk Mitigation Benefit of Increasing Reserve Margins ............................................50

5. Sensitivity to Forecast Error and Forward Planning Period .......................................52

B. Impact of System Characteristics .....................................................................................55

1. Impact of System Size ................................................................................................55

2. Intertie Size and Neighbor Assistance ........................................................................57

3. Combined Cycle Plants as the Marginal Technology .................................................60

4. Intermittent Renewable Resource Penetration ............................................................63

C. Displacing Generation with Demand Response ...............................................................65

1. Reliability Value of Emergency Demand Response ..................................................65

2. Cost-Minimizing Level of Emergency Demand Response Penetration .....................68

3. Cost-Minimizing Level of Economic Demand Response Penetration .......................69

D. Comparison of Reserve Margin Targets ..........................................................................70
ii 

IV. Market Design Implications ................................................................................................ 73

A. Energy-Only Markets .......................................................................................................73

1. Economic Equilibrium at the Cost of New Entry .......................................................74

2. Conditions for Achieving Socially Optimal Investment Levels .................................75

3. Volatility in Supplier Energy Margins .......................................................................78

4. Missing Money and the Impact of Price Caps ............................................................83

5. Implications of Varying System Conditions and Study Assumptions ........................84

B. Capacity Markets .............................................................................................................86

1. Capacity Payments Required to Achieve Reliability Targets .....................................86

2. Impact of Lower Energy Market Price Caps on the Capacity Market .......................88

3. Cost-Minimizing Capacity Demand Curves ...............................................................89

4. Current RTO Demand Curves Compared to Cost-Minimizing Curves ......................93

C. Implications of Increased Demand Response Penetration ...............................................95

1. Energy Market Impacts ...............................................................................................95

2. Impact on Generator Energy Margins ........................................................................97

3. Impact on Capacity Market ........................................................................................99

D. Comparison of Capacity and Energy-Only Market Designs ..........................................101

1. Supplier Net Revenues .............................................................................................101

2. Total Customer Costs ...............................................................................................103

V. Conclusions ......................................................................................................................... 107

List of Acronyms ....................................................................................................................... 109

Bibliography .............................................................................................................................. 112

Appendices ................................................................................................................................. 119

A. Detail on Survey of North American Resource Adequacy Criteria .............................. A-1

1. Resource Adequacy Standards Used Across North America .................................. A-1

2. Illustration of Differences in Resource Adequacy Modeling Assumptions ............ A-3

B. Detail on Simulation Modeling and Assumptions .........................................................B-1

1. Interregional Tie Line Availability ...........................................................................B-1

2. Thermal Resource Forced and Maintenance Outage Assumptions ..........................B-2

3. Hydro Modeling ........................................................................................................B-4

4. Intermittent Resource Modeling ...............................................................................B-5

5. Treatment of Weather Uncertainty in Load Modeling .............................................B-6
iii 

EXECUTIVE SUMMARY 

This report, prepared for the Federal Energy Regulatory Commission (FERC), assesses the 

economic and reliability implications of different resource adequacy standards. We examine the 

widely-used one-day-in-ten-years (1-in-10) loss of load standard, compare it to alternative 

approaches to defining resource adequacy, and evaluate the implications of different resource 

adequacy standards from a customer cost, societal cost, risk mitigation, market structure, and 

market design perspective. 

The 1-in-10 Resource Adequacy Standard 

Utilities, system operators, and regulators across North America have relied on variations of the 

1-in-10 standard for many decades, and typically enforce the standard without evaluating its 

economic implications. In most U.S. power systems, this standard is interpreted to mean that 

planning reserve margins need to be high enough that involuntary load shedding due to 

inadequate supply would occur only once in ten years. One event in ten years translates to 0.1 

loss of load events (LOLE) per year, regardless of the magnitude or duration of the anticipated 

individual involuntary load shed events. Alternatively, one day in ten years translates to 2.4 loss 

of load hours (LOLH) per year, regardless of the magnitude or number of such outages. As we 

show, the difference between these interpretations of the 1-in-10 standard translates to 

differences in planning reserve margins that may exceed five percentage points, with planning 

reserve margins of possibly less than 10% based on the 2.4 LOLH standard and more than 15% 

based on the 0.1 LOLE standard. 

We survey the resource adequacy standards used in North American power markets. This survey 

shows that most North American systems set planning reserve margins using some variation of 

the 1-in-10 standard, although some set planning reserve margins based on economic 

considerations. Of the regions relying on the 1-in-10 standard, the majority interpret it as 0.1 

LOLE. However, despite attempts to create a more uniform approach to interpreting and 

estimating LOLE, large differences remain in how the standard is interpreted across systems. In 

calculating LOLE, some systems define a reliability event as the involuntary curtailment of firm 

load. Other systems also consider other, less severe reliability events as contributing to LOLE, 

including voltage reductions or operating reserve depletions. We show that changing event 

definitions alone can translate to a four percentage point range in planning reserve margins. Due 

to these differences and its other limitations, the 0.1 LOLE standard does not represent a uniform 

level of reliability but instead can represent very different expected customer outage levels. 

Additional differences exist in how planning reserve margins are calculated. For example, some 

system operators calculate reserve margins using the nameplate capacity of intermittent 

generation such as wind and solar, while others use a derated capacity value. Other differences 

exist in how voltage reductions or demand response are considered in reserve margin 

calculations. As we show, this means that reported planning reserve margins can differ by five 

percentage points based solely on supply and demand accounting conventions. Overall, these 

differences make it very challenging to meaningfully compare reported reserve margins across 

regions. 
iv 

Simulating the Reliability and Economic Implications of Planning Reserve Margins 

To examine the economic implications of the 1-in-10 resource adequacy standard, we conduct a 

series of reliability simulations using the Strategic Energy and Risk Valuation Model (SERVM) 

developed by Astrape Consulting. We simulate a hypothetical Study Regional Transmission 

Organization (RTO) system with a peak load of 50,000 MW that is interconnected through 

11,000 MW of transmission interties to three neighboring regions with a combined peak load of 

130,000 MW and planning reserve margins of 15%. The hypothetical Study RTO and 

neighboring regions do not exactly reflect any particular real-world system, but have realistic 

characteristics based on actual systems’ hourly load shapes, load diversity, resource mix, 

generation performance statistics, weather conditions, demand response penetrations, and other 

characteristics derived from actual U.S. power system data. 

We probabilistically evaluate resource adequacy conditions by simulating hourly generation 

availability, load profiles, load uncertainty, transmission availability, and other factors to 

estimate standard reliability metrics including LOLE and LOLH, as well as the economic 

implications of different planning reserve margins. We use 9,600 annual simulations for each 

case and Study RTO planning reserve margin level to evaluate: (1) reliability outcomes 

considering hourly generation and transmission intertie outages, uncertainties in weather, hydro, 

wind, and solar conditions, and economic load growth uncertainty; and (2) economic outcomes 

including hourly and annual production costs, customer costs, market prices, net import costs, 

load shed costs, and generator energy margins. 

Through these probabilistic multi-area simulations, we evaluate the physical reliability, 

economic, and risk mitigation implications of different planning reserve margins under varying 

study assumptions related to: RTO size, intertie size, renewable penetration, emergency and 

demand response (DR) penetration, the cost of new generating plants, multi-year forward 

planning periods and the size of load forecast error, varying energy market price caps, and 

energy-only or capacity market designs. We did not evaluate some factors such as fuel 

availability or common mode failures of generators, pipelines, and transmission components. 

While SERVM is capable of modeling the economic and reliability impacts of each of these 

drivers, the scope of this study is limited to variables most commonly analyzed in other 

reliability studies. 

 Planning Reserve Margin Results Based on Economics and Physical Reliability 

In our Base Case simulation, the Study RTO requires a planning reserve margin of 15.2% to 

meet the 0.1 LOLE resource adequacy standard, or a planning reserve margin of 8.2% to meet 

the 2.4 LOLH standard. Figure ES-1 summarizes the economic implications of the planning 

reserve margin, showing the tradeoff between reliability event costs (that decrease with higher 

planning reserve margins) and system capital costs (that increase with planning reserve margins). 

A planning reserve margin of 10.3% yields the lowest total annual system cost from a riskneutral, cost-of-service perspective. From a societal cost perspective, the risk-neutral 

economically-optimal planning reserve margin would be only 7.9% above the Study RTO’s noncoincident peak load, if each Neighbor continues to maintain a 15% reserve margin above their 

own non-coincident peak load. A 7.9% Study RTO reserve margin combined with 15% 

Neighbor reserve margins is equivalent to an overall four-system reserve margin of: (a) 13.0% if 
v 

measured against the sum of the RTOs’ non-coincident peak loads, and (b) 16.5% if measured 

against the coincident peak load of the combined system.1

 

Figure ES-1 also shows that the total average annual cost curve is relatively flat near the 

minimum cost point, indicating that expected total costs do not vary substantially between 

reserve margins of 8% and 14%. However, the lower end of that reserve margin range is 

associated with substantially more uncertainty in realized annual reliability costs and a much 

larger number of severe, high-cost reliability events. 

Figure ES-1 

Study RTO Reliability Costs as a Function of Study RTO Reserve Margin 

(Risk-Neutral, Cost-of-Service Perspective) 

Table ES-1 summarizes the reliability-based and economically-based reserve margin targets for 

the Base Case simulation and for each of the alternative simulation cases we examined. These 

results show that varying assumptions regarding system size, topology, price cap, and other 

factors can substantially impact planning reserve margin targets. For example, the 0.1 LOLE 

reliability standard could require a reserve margin anywhere from 13.0% to 18.5% depending on 

the assumed system size, intertie level, and other characteristics. Similarly, economicallyoptimal reserve margins could vary anywhere from less than 6% to 16.5% from either a cost-ofservice or societal perspective. The most significant factor impacting a region’s planning reserve 

margin is the size of transmission interties, which resulted in Study RTO planning reserve 

margins ranging from 15.2% to 18.5% to maintain 0.1 LOLE or from 7.9% to 16.5% in terms of 

 

1

 See Table 7 in Section III.A.3 for a more detailed explanation of this coincident vs. non-coincident reserve 

margin accounting convention. 
vi 

economically-optimal reserve margins. Despite these differences, system characteristics have 

similar directional impacts on reserve margin targets for all resource adequacy criteria, both in 

terms of physical reliability and economics. 

In addition to the simulation cases summarized in Table ES-1, we also analyze the implications 

of different levels of “emergency” and “economic” DR and different levels of intermittent 

renewable generation. With respect to dispatch-limited emergency demand response, we find 

that as emergency DR penetration increases, total costs initially decrease because these resources 

have lower capital costs than the conventional generation resources they replace. Even though 

emergency DR resources have a much higher dispatch cost than the displaced generating plants, 

the number of dispatch hours is low enough that it is advantageous to continue adding more DR 

initially. At higher penetration levels, however, limits on annual dispatch hours create a 

constraint after which it is no longer cost-effective to add emergency DR. Increasing the 

penetration of economic DR resources with unlimited call hours is more valuable in reducing 

system costs, with economically-optimal penetrations ranging from 8% to 14% depending on the 

fixed and variable costs of the assumed DR resources. 

Table ES-1 

Reliability-Based and Economically-Based Planning Reserve Margin Targets 

Note: 

Normalized Expected Unserved Energy (EUE) represents the percent of total energy not delivered (see Seciton I.B.1). 

Market Design Implications 

We also evaluate the implications of our simulation results within restructured wholesale power 

markets in which investments can only be attracted if prices are high enough on average for 

suppliers to earn the necessary return on their investments. Within energy-only markets, which 

do not rely on mandated planning reserve margins, investments must be attracted by net revenues 

earned from the energy market alone. Because supplier energy margins decline with increasing 

reserve margins, energy-only markets achieve an equilibrium reserve margin where net revenues 

Simulation Reliability-Based Risk-Neutral, Cost-Minimizing 

0.1 LOLE 2.4 LOLH 0.001% 

Normalized EUE

Cost-of-Service 

Perspective

Societal 

Perspective

Base Case 15.2% 8.2% 9.6% 10.3% 7.9%

Lower Price Caps

$1,000 Price Cap Case 15.2% 8.2% 9.6% 8.7% 7.9%

$3,000 Price Cap Case 15.2% 8.2% 9.6% 9.5% 7.9%

Smaller System Size

40% Size Case 14.8% <6% 7.5% <6% <6%

40% Size and Transmission 15.1% 6.9% 8.1% <6% <6%

Neighbor Assistance

Long Neighbors Case 13.0% <6% 7.0% 8.0% <6%

50% Transmission Case 15.8% 9.8% 10.0% 12.3% 10.5%

Island Case 18.5% 16.5% 15.8% 16.5% 16.5%

Marginal CC Case 15.3% 8.3% 9.8% 10.1% 7.7%
vii 

are equal to investment costs. Our simulations show that energy-only markets can attract the 

optimal level of generation investments from a societal-cost perspective. However, we also 

demonstrate that if energy prices are suppressed below true marginal system costs during load 

shedding or other scarcity events, an energy-only market design will not be able to maintain an 

optimal level of investments. This highlights the importance of efficient price formation during 

scarcity events through well-designed scarcity pricing mechanisms that reflect the marginal 

system costs of emergency interventions and deploying demand-side resources. 

Because the reserve margin in an energy-only market is the result of economic incentives, there 

is no guarantee that such a market will achieve any particular reserve margin or reliability level. 

Energy-only markets can suffer from a “missing money problem” with insufficient economic 

incentives to invest in new resources. This missing money problem may result from two 

different types of challenges: (1) missing money arising from energy prices that are artificially 

suppressed, for example, by low price caps or inefficiently low market prices during scarcity 

conditions; and (2) missing money relative to what is needed to achieve the reserve margin that 

policy makers desire, which may be higher than what an economically efficient energy-only 

market would support. 

Resource adequacy requirements can be imposed to address the missing money problem and 

attain desired planning reserve margin targets. Imposing such a requirement will make capacity 

a valuable product in addition to the energy and ancillary services that are compensated in an 

energy-only market. Maintaining a higher reserve margin requirement will yield higher capacity 

prices to compensate for declining energy margins. If energy and ancillary service market prices 

are suppressed by price caps or other factors, this will also translate into higher capacity prices. 

To date, capacity markets have been designed to achieve planning reserve margins based on 

physical reliability standards rather than economic objectives such as minimizing system costs. 

Our simulations demonstrate that a capacity market could be designed to achieve economicallyoptimal reserve margins through either: (a) imposing economically-based rather than physical 

reliability-based planning reserve margin requirements; or (b) implementing a demand curve for 

capacity that is designed to procure the cost-minimizing quantity of capacity. We show how 

such a cost-minimizing demand curve could be determined and compare the result to current 

RTO capacity demand curves as illustrated in Figure ES-2. The two red curves shown in the 

figures represent economically-optimal demand curves from a risk-neutral, societal perspective, 

assuming neighboring regions will maintain a 15% planning reserve margin. The risk-neutral 

cost-minimizing demand curves are plotted against: (1) the Study-RTO-internal reserve margin 

above Study-RTO non-coincident peak load (solid lines), as well as (2) the combined foursystem reserve margin above the combined systems’ coincident peak load (dotted lines). 

We also examine the energy and capacity market outcomes at increased levels of DR 

penetration, assuming that DR will displace traditional generation. We show that increasing DR 

levels will result in increasing average energy prices, increasing energy price volatility, but 

decreasing capacity prices. Generally, economic DR that participates in the energy market has 

more favorable impacts on the market than dispatch-limited emergency DR. Currently, most DR 

assets have little incentive to participate in energy markets because energy market prices remain 

below their curtailment costs except under rare circumstances. However, at higher price caps 
viii 

and higher DR penetration levels, we anticipate that more DR resources will migrate from 

emergency to economic programs. 

Overall, we find that the choice of an energy-only or energy-plus-capacity market design does 

not by itself have great implications for average supplier net revenues or average customer cost. 

Because supplier’s net revenues must on average be sufficient to recover investment costs in 

equilibrium, changing market design or reserve margins does not change the annual average of 

anticipated net revenues. Rather, it only changes the proportion of revenues that on average 

would be earned from the energy market versus the capacity market. For customers, average 

annual costs do increase as reserve margins rise above the risk-neutral cost-minimizing level, but 

this impact represents only a very small portion of customers’ bills. In fact, increasing the 

planning reserve margin from the risk-neutral, societal optimum of 7.9% to the 15.2% reserve 

margin needed to maintain 0.1 LOLE increases average annual customer costs by only 1.5%, but 

substantially reduces the variance of annual cost outcomes, including the potentially very high 

costs of extreme events. Other features of the regional power market, such as the amount of 

intertie capacity to neighboring regions, have a much larger impact on total customer costs. 

Figure ES-2 

Cost-Minimizing Capacity Demand Curve vs. Current RTO Demand Curves 

Notes: 

 All curves converted from source units into ICAP terms for both reserve margin and price. 

 Cost-minimizing demand curves plotted with x-axis as: (1) the Study RTO reserve margin against StudyRTO non-coincident peak load (solid lines); and (2) against the combined four-system reserve margin 

against the combined coincident peak load (dotted lines). See Table 7 in Section III.A.3 for a more 

detailed explanation of this coincident vs. non-coincident reserve margin accounting.

 
1 

I. INTRODUCTION AND BACKGROUND 

The Brattle Group and Astrape Consulting have been asked by the Federal Energy Regulatory 

Commission (FERC) to assess the economic implications of resource adequacy standards. In 

particular, we examine the widely-used one-day-in-ten-years (1-in-10) loss of load standard and 

compare it to alternative approaches to defining resource adequacy. In this Section we: 

(1) describe study scope and approach; (2) explain how 1-in-10 and alternative resource 

adequacy standards are determined; and (3) provide a summary that compares how various 

standards are implemented in practice. 

A. STUDY PURPOSE AND APPROACH 

The 1-in-10 loss of load event (LOLE) standard requires that an electric system maintain 

sufficient generation and demand response resources such that system peak load is likely to 

exceed available supply only once in any ten-year period. Utilities, system operators, and 

regulators across North America have relied on variations of the 1-in-10 standard for many 

decades, and typically enforce the standard without an evaluation of its economic implications.2 

However, because the economic implications of different resource adequacy levels can be quite 

large, we and other industry observers have highlighted the need to examine the 1-in-10 standard 

from an economic perspective. To the extent that the economic value of resource adequacy 

standards is discussed in the industry today, there is no uniform approach to measuring the value 

of resource adequacy, such as the ability of incremental resources to reduce the frequency of 

customer outages and other emergency events, moderate energy price spikes, and increase 

wholesale competition. This lack of clarity makes it difficult for policymakers and industry 

participants to evaluate fundamental questions such as: (a) whether 1-in-10 criterion is still the 

right standard; (b) how higher planning reserve margins benefit customers; and (c) at what price 

capacity should be compensated.3 

With this study, we seek to contribute to the understanding of these questions by developing a 

framework for assessing the economic value of resource adequacy and examining the 

implications of the 1-in-10 standard. We provide a brief summary of resource adequacy 

concepts in Section I.B, followed by a survey of how 1-in-10 and alternative resource adequacy 

standards are defined and implemented across North America in Section I.C. 

In the remainder of the report we evaluate the economics of resource adequacy for a hypothetical 

interconnected power system that resembles existing regional transmission organization (RTO) 

and independent system operator (ISO) footprints. We use probabilistic simulation modeling to 

examine how reliability metrics, customer costs, and system costs vary with reserve margin. We 

also test the sensitivity of our results to input assumptions and underlying system conditions and 

compare the economics of the 1-in-10 standard to alternative reliability-based standards and 

standards based on economic objectives such as cost reduction and risk mitigation. We describe 

 

2

 Exceptions exist, particularly in the Southeastern U.S., where the utility and regulators explicitly consider 

the economic implications of planning reserve margins when setting the reserve margin target as 

summarized in Section I.C.1. 

3

 For example, see Wilson (2010a-b). 
2 

our simulation approach and assumptions in Section II, report economic and reliability results in 

Section III, and summarize the implications for wholesale electricity market design in Section 

IV. 

B. APPROACHES TO SETTING RESOURCE ADEQUACY STANDARDS 

Resource adequacy can be assessed either in terms of physical reliability criteria or economic 

objectives. This section of our report defines the range of physical reliability criteria used in 

setting resource adequacy standards and explains how these standards are translated into target 

planning reserve margins. We then describe economic approaches to determining target 

planning reserve margins. 

1. Interpreting 1-in-10 and Other Reliability Criteria 

Although the 1-in-10 standard is widely used across North America, substantial variations in 

how it is implemented mean that it does not represent a uniform level of reliability. As 

recognized in recent work by the North American Electric Reliability Corporation (NERC) and 

ReliabilityFirst Corporation (RFC), the 1-in-10 standard may be interpreted as either one event

in ten years or one day in ten years. One event in ten years translates to 0.1 loss of load events 

(LOLE) per year, regardless of the magnitude or duration of the anticipated individual 

involuntary load shed events. One day in ten years translates to 2.4 loss of load hours (LOLH) 

per year, regardless of the magnitude or number of such outages.4

 These two interpretations can 

represent very different levels of physical reliability, although there is no direct translation 

between them unless one assumes a typical outage duration. For example, if one assumes a 

4-hour outage, then the events-based definition represents a reliability standard with outage 

durations that are one-sixth the outage duration achieved by the LOLH-based definition.5

 

The 1-in-10 standard can also represent different levels of reliability for different systems. This 

is the case because neither the LOLE nor the LOLH metric considers the MW size of the outage 

endured. For example, a one-hour, 100 MWh outage event and a one-hour, 10,000 MWh outage 

event would be counted identically under the LOLE and LOLH metrics, even though the load 

shed amount under the second outage event is one hundred times greater in magnitude. Further, 

the 1-in-10 standard represents a higher level of reliability in a large system than in a small 

system because neither the LOLE nor the LOLH metric is normalized to system size.6

 This 

means that a 100 MWh, one-hour outage will have the same LOLE and LOLH values in a 

10,000 MW and 100,000 MW power system, even though individual customers in the smaller 

system are ten times more likely to endure an outage. We further illustrate the differences 

among these various interpretations in Section I.C for actual power systems and in Section III for 

a hypothetical system with realistic assumed characteristics. 

 

4

 We also clarify that while we use these definitions of LOLE and LOLH throughout this report, others may 

use somewhat different definitions as we explain further in Section I.C. 

5

 In other words, 0.1 LOLE times a 4-hour outage results in 0.4 LOLH per year, or only one sixth of the 2.4 

LOLH assumed in the hours-based interpretation. However, the comparison depends entirely upon the 

assumed outage duration. 

6

 This is true as long as the one event represents a smaller proportion of total load in a large system than in a 

small system. 
3 

Recognizing these limitations of the 1-in-10 standard, a recent NERC task force recommended 

using Normalized Expected Unserved Energy (EUE) as a new physical reliability metric.7

 

Normalized EUE is the total expected firm load shed due to supply shortages, divided by the 

total system net energy for load, and therefore represents an overall percentage of system load 

that cannot be served. We agree with that report’s conclusion that Normalized EUE is the most 

meaningful reliability metric that can be compared across systems of many sizes, load shapes, 

and other uncertainty factors. In fact, Normalized EUE is already used in some international 

markets to set minimum reliability thresholds or trigger administrative interventions, although 

the metric may be referred to as Loss of Load Probability (LOLP) or Unserved Energy (USE). 

Examples of metrics equivalent to Normalized EUE that are used in international markets 

include: (a) a 0.001% LOLP standard in Scandinavia; and (b) a 0.002% USE standard in 

Australia’s National Energy Market (NEM) and South West Interconnected System (SWIS).8

Unfortunately, these and similar terms such as the Probability of Lost Load (POLL) are also not 

used and defined uniformly. For our purposes and for the remainder of this report we will refer 

only to: (1) LOLE measured in events per year; (2) LOLH measured in hours per year; and (3) 

Normalized EUE measured in percent of system-wide load unserved. While there is not a direct 

translation among these three physical reliability metrics (unless one assumes an average outage 

size and duration), they can be estimated and compared for an individual system that has a 

particular distribution of expected load and other uncertainty factors. Section III presents such 

comparisons. 

Also note that, historically, the 1-in-10 standard as applied by system operators in North America 

translates to expected annual outages for end-use customers that average less than a few minutes 

per year on a system-wide basis. This compares to total customer outages averaging several 

hundred minutes per year on a system-wide basis, most of which are caused by distribution 

system outages.9

 Nevertheless, despite the fact resource adequacy-related reliability events 

account for only a very small fraction customer outages, such “rolling blackouts” attract 

disproportionate media and political attention whenever they happen.10

2. Determining System Reliability as a Function of Planning Reserve Margin 

The 1-in-10 resource adequacy standard is not a readily observable metric. For example, one 

cannot simply review a region’s integrated resource plan (IRP) or reserve margin outlook to 

determine whether the 1-in10 standard will be achieved. This is because such supply-related 

 

7

 See the NERC (2010). 

8

 See Nordel (2009), p. 5; AEMC (2007), pp. 29-30, (2010), p. viii. 

9

 See Newell, et al. (2012), pp. 101-2. 

10 For example, during the summer of 2012, the Alberta Electric System Operator (AESO) instituted load 

shedding due to heatwave-related peak loads and unexpected generation outages. This required rolling 30-

minute curtailments of 200 MW of firm system load over a two-hour period. Despite attracting substantial 

media and political attention (see, for example, Sellin (2012) and Henton (2012)), this 200 MW 

curtailment affected only approximately 2% of almost 10,000 MW of total system load, which means an 

average customer outage duration of only approximately 2.4 minutes (2% of 120 minutes) on a systemwide basis. Only 3 such events occurred in Alberta in the last decade, one on July 2, 2013 (but triggered 

by a transformer outage, not resource adequacy constraints). 
4 

reliability outcomes depend on a host of complex factors such as anticipated resource mix and 

weather uncertainty. For this reason, most system operators use reliability modeling to translate 

the 1-in-10 standard into a planning reserve margin.11 Such reliability modeling is used regularly 

even in markets without explicit resource adequacy requirements, including Texas, Alberta, 

Australia’s NEM, and Scandinavia, to monitor reliability levels and determine whether 

administrative interventions are warranted to maintain adequate system resources.12

For markets with explicit planning reserve margin requirements, the requirement is typically 

calculated based on probabilistic reliability modeling that simulates LOLE as a function of the 

level of installed generating capacity. Such a simulation exercise requires statistically 

characterizing the uncertainty in forecasted load, the reliability of the generation fleet including 

intermittent resources, the likely availability of imports, and a number of other factors as 

explained further in Section II. The simulations are used to estimate reliability levels over a 

range of reserve margins, showing that the number or duration of expected load shed events 

decreases as capacity is added and the planning reserve margin increases. 

Figure 1 illustrates how LOLE may decrease with reserve margin, showing an example where a 

planning reserve margin of 15% would be needed to achieve the 0.1 LOLE standard. We 

implement this type of reliability modeling for a realistic but hypothetical RTO in this study, as 

discussed in Sections II through IV. 

To understand the economic value of different resource adequacy levels, one must also estimate 

the value of reliability to customers. One component of this reliability value is the ability to 

avoid costly load shedding events, the value of which can be estimated by: (1) determining the 

approximate value of lost load (VOLL), which is the cost of an outage to customers or the price 

that an average customer would be willing to pay to avoid an involuntary interruption of their 

electricity supply; and (2) multiplying VOLL by the amount of load shed. This estimate of the 

 

11 In addition to planning reserve margins based on installed generating capacity and anticipated (50/50) 

peak load during normal weather conditions, power markets also require various types of “operating 

reserves” such as regulating reserves, spinning reserves, non-spinning reserves, and replacement reserves. 

Operating reserves are needed to balance load and supply reliably on a moment-by-moment basis and 

enable the system operator to quickly and reliably react to contingency events. In contrast to operating 

reserve requirements, which apply during real-time system operations irrespective of load levels or plant 

availability, planning reserve margins are meant to ensure that sufficient supply is available to address 

reliability challenges due to differences between: (1) near-term or medium-term peak load forecasts 

(which reflect average weather) and typical generation outages; and (2) actual peak loads (which can 

exceed reflected weather-related spikes in load) and spikes in generation outages. Because of the 

differences between forecast and actual peak loads, the year-to-year realized reserve margin will fluctuate 

considerably around the average. In this section, as well as throughout this report, we refer to “planning 

reserve margin,” “target reserve margin,” or simply “reserve margin” to refer to this forward-looking 

estimate of installed capacity (ICAP) relative to the forecast normalized peak load. We refer to “operating 

reserves requirements” or “operating reserves” when referring to the quantity of regulating and 

contingency reserves that are needed to maintain real-time grid stability in every hour. 

12 AESO developed its long-term adequacy (LTA) metrics to determine a forecast level of reliability at 

which point AESO may need to intervene in the market, see AESO (2008), Section 1. For additional 

discussion on similar regulatory methods in other countries, see Pfeifenberger, Spees, and Schumacher 

(2009), pp. 28-29. 
5 

economic cost of load shed events will be a decreasing function of the reserve margin with a 

shape similar to that in Figure 1.13 However, as we discuss further below, the system-wide 

expected VOLL is only one component of total reliability-related system and customer costs that 

must be considered when evaluating the economic implications of different resource adequacy 

levels. 

Figure 1 

Reliability vs. Reserve Margin 

Notes: 

Illustrative figure does not represent any actual or simulated system. 

 Similar charts for real systems are available in various reliability 

studies, for example see Electric Reliability Council of Texas 

(ERCOT) results in ECCO (2013), pp. 8-9. 

3. Determining Economically “Optimal” Reserve Margins 

To understand economic tradeoffs over a range of planning reserve margins, one must also 

estimate the incremental value and costs associated with additional capacity resources. In this 

study we implement such economic reliability simulations to assess these tradeoffs, building 

upon previous studies conducted for regulated entities and in academia.14 In particular, Astrape 

Consulting has conducted a number of such studies using the Strategic Energy Risk Valuation 

Model (SERVM) reliability simulation tool, which we also use to analyze the case studies 

 

13 See Wilson (2010a) and Centolella (2010). 

14 For example, a 1992 study for Pacific Gas and Electric compared the traditional 1-in-10 standard against a 

value-of-service approach, showing that the traditional standard required a reserve margin of 22.5% while 

the value-of-service approach indicated an efficient reserve margin of 16.2%. See Keane, et al. (1992), 

pp. 824, 826. For similar analyses, see also Poland (1988), p.21; Munasinghe (1988), pp. 5-7, 12-13; and 

Carden, Pfeifenberger, and Wintermantel (2011). 
6 

presented in this report. The objective of such an economic reliability study is to determine an 

“optimal” reserve margin that minimizes total system or customer costs, including the capital 

cost of adding resources, production costs, off-system and emergency purchase costs, and the 

cost of load shed events. 

The objective of minimizing total system costs is demonstrated schematically in Figure 2. The 

figure shows that increasing the planning reserve margin will increase capital costs as more 

resources are added to the system. However, increasing the reserve margin also reduces outage 

costs by reducing the frequency, duration, and magnitude of load shed events, may reduce 

production costs by reducing the need to dispatch higher-cost peaking resources, and may reduce 

power purchase costs by reducing the need to call on higher-cost imports during reliability 

events.15 The “optimal” reserve margin is that which minimizes expected system or customer 

costs by weighing the tradeoff between the incremental costs and the benefits of increasing the 

reserve margin. 

However, this cost-minimizing reserve margin only represents the point at which costs are 

minimized on an expected value basis, and therefore represents the “optimum” reserve margin 

from a risk-neutral perspective. As we discuss further in Sections III.A.2–4 below, the point of 

the lowest system cost represents the probability-weighted average cost over many possible 

system conditions. This probability-weighted average cost is influenced greatly by lowprobability, very-high-cost shortage events, such as an extreme heat wave that happens to 

coincide with substantial generation outages. Because customers and policy makers are risk 

averse, avoiding such high-impact events will be valuable. Customers and policy makers may 

therefore prefer to pay more to maintain a somewhat higher reserve margin that can mitigate the 

risks and impacts of extreme shortage events. 

 

15 The magnitude of avoided load shed costs will not typically depend on what type of capacity resource is 

added because many types of resources have similar likelihoods of being available for dispatch during load 

shed events. However, the magnitude of net reductions in production costs, avoided import costs, and 

increases in capital costs are highly dependent on the type of capacity added. For example, adding 

combustion turbines (CTs) will incur lower capital costs than adding combined cycles (CCs), but will also 

provide fewer benefits from avoided production and import costs. For this reason, the “optimal” reserve 

margin depends on the type of marginal technology assumed when varying the reserve margin as 

discussed further in Section III.B.3. 
7 

Figure 2 

“Optimal” Reserve Margin at Lowest System Cost 

Notes: 

Illustrative figure does not represent any actual or simulated system. 

 See Sections III.A.2–4 for a more thorough discussion and estimate of 

the economic optimum within a hypothetical simulated system. 

C. SURVEY OF RESOURCE ADEQUACY CRITERIA IN NORTH AMERICA 

This section documents variations in how the 1-in-10 and alternative resource adequacy 

standards are used and interpreted in existing power markets across the U.S. and Canada. This 

discussion is based on our interpretation of RTO and utility planning and reserve margin studies 

as summarized in Appendix A. However, we caution that, to completely understand the nuances 

and complexities of these studies, one would need to discuss implementation details with the 

individuals responsible for conducting the studies. The public documentation of these studies is 

often insufficiently detailed or can easily be misinterpreted. Because we have not conducted 

such interviews to be able to document the assumptions and the complexities of each study, our 

discussion should be interpreted as a preliminary summary of general industry practices, not a 

fully-verified documentation of any one region’s approach. 

As we explain, most of these regions rely on a 1-in-10 reliability standard, but substantial 

variations in interpretation, conventions used in calculating reserve margins, modeling 

approaches, and underlying system conditions make it very difficult to meaningfully compare 

different systems’ reliability levels and reported reserve margins.16 

 

16 The markets we survey here include PJM Interconnection (PJM), New York ISO (NYISO), ISO New 

England (ISO-NE), California ISO (CAISO), Northwest Power Pool (NWPP), Florida Reliability 

Coordinating Council (FRCC), Electric Reliability Council of Texas (ERCOT), Southwest Power Pool 

Continued on next page 

Expected System Cost

Reserve Margin (%) 

Lowest

System

Cost

Optimal Reserve 

Margin 

0% 10% 20%

Total Cost

VOLL, Emergency 

Procedure, Import, and 

Production Costs 

Capacity Capital Costs
8 

1. Differing Resource Adequacy Standards 

Appendix A.1 summarizes the resource adequacy criteria established by system operators and 

reliability coordinators across the U.S. and Canada. As shown, the majority, but not all, of these 

regions use the 1-in-10 resource adequacy standard. Most of these regions define 1-in-10 as 

1 event in 10 years, or 0.1 LOLE per year. However, SPP differs in that it interprets 1-in-10 as 

24 hours of curtailments in 10 years, or 2.4 LOLH per year. 

Several reliability operators in the Southeast establish resource adequacy targets based on 

economic analysis with the objectives of minimizing customer costs and mitigating the risk of 

high-cost events. Other entities use alternative physical reliability criteria tailored to their own 

needs, such as the BC Hydro system that must also consider the potential for energy-constrained 

low-hydro conditions leading to overall energy limitations. Finally, Texas and Alberta both have 

reliability targets but do not impose these targets as mandatory standards for resource adequacy. 

2. Differing Conventions for Calculating Reserve Margins 

One important reason that it is difficult to compare the level of resource adequacy across systems 

is that each region adheres to a different set of conventions when calculating their reserve 

margins. In other words, even if two regions had identical peak load and expected resource mix, 

they may calculate different reserve margins using different conventions. In fact, even within a 

single market, there may be different approaches used when calculating reserve margins for the 

purposes of publishing summer or long-term resource adequacy assessments, conducting 

transmission planning exercises, conducting a reliability studies, or implementing a capacity 

market. 

a. Accounting for Demand-Side Resources 

All regions have some demand response (DR), load curtailment, retail price response, or other 

load-side resources that contribute to resource adequacy. Aside from the fact that it is sometimes 

difficult to estimate the total resource adequacy value of a particular type of DR program, there 

is also no standard approach regarding how these resources are included in the reserve margin 

calculation. Some entities include demand response as a reduction in load while other regions 

include it as a supply-side resource. 

Example: DR counting as a supply-side resource versus a reduction in load changes 

calculated reserve margin (RM) by 5% from 20% to 25%.

Assume: 

 Load = 100,000 MW 

 Gen = 100,000 MW 

 DR = 20,000 MW 

Reserve margin if treating DR as supply resource: 

 

Continued from previous page 

(SPP), and various SERC Reliability Corporation (SERC) and Western Electricity Coordinating Council 

(WECC) entities. We draw upon previous NERC surveys as well as individual entities’ planning and 

reliability modeling documentation. For example, see NERC (2010a). 
9 

 RM = (Gen + DR) / Load – 100% 

 20% = (100,000 + 20,000) / 100,000 – 100% 

Reserve margin if treating DR as a reduction in peak load: 

 RM = Gen / (Load – DR) – 100% 

 25% = (100,000) / (100,000 – 20,000) – 100% 

Further, there are different conventions used for adjusting nominal DR capability by: (a) derating 

that capability by an expected availability rate; or (b) if treating DR on the supply side, possibly 

grossing up the nominal capacity value for avoided line losses and reserve margin requirements. 

These differences in DR accounting change the numeric value of the planning reserve margin, 

but not the system’s underlying resource adequacy situation. 

b. Accounting for Emergency Operations 

Similarly, while all regions implement various types of emergency procedures before shedding 

load, there is no standard approach as to how these procedures are accounted for when 

calculating reserve margins. For example, PJM will implement voltage reductions and may 

achieve approximately 2.5% reduction in peak load before firm load shedding is required. In the 

Southern Company (SoCo) region, approximately 2% of peak load is available from voltage 

reduction and other emergency procedures.17 

However, the regions differ in terms of how these procedures are accounted for in their reserve 

margins, with PJM not considering the capacity benefit of emergency procedures and SoCo 

treating emergency procedures as supply resources. Overall, as summarized in Table 1, the 

convention of whether to count emergency procedures as a supply resource, load reduction, or 

neither would result in calculated reserve margins ranging from 15.6% to 18.6% in PJM and 

from 13.0% to 15.3% in SoCo. 

Table 1 

Emergency Procedure Accounting Impact on PJM and SoCo Reserve Margins 

Sources and Notes: 

 As-reported reserve margin requirements from PJM (2012c), and Georgia Power (2010). 

 Alternative reserve margin calculations assume no change other than how voltage reductions are included or not. 

 

17 PJM voltage reduction value approximately calculated from PJM (2012a), p. 63. 

Load ICAP Reserve Margin

Reduction from 

Emergency 

Procedures

As 

Reported

Emergency 

Procedures 

Counted as 

Supply Resources

Emergency 

Procedures 

Counted as 

Load Reduction

Emergency 

Procedures 

Not Counted

(%) (%) (%) (%) (%)

PJM 2.5% 15.6% 18.1% 18.6% 15.6%

SoCo 2.0% 15.0% 15.0% 15.3% 13.0%
10 

c. Derating Capacity from Different Resource Types 

When calculating the reserve margin, some planning regions include all resources at their 

nameplate capacities, while others include only a derated capacity value for intermittent or 

dispatch-limited resources, such as wind, solar, hydro, or storage. These derated capacity values 

are calculated using a variety of different approaches.18 The decision of whether to derate certain 

resource types and, if so, how to calculate the derate magnitude, is partly a matter of convention 

and partly a reflection of the importance of various resource types in the overall resource 

adequacy picture in a particular region. For example, a region with few hydro resources may not 

place particular emphasis on developing a detailed assessment of those assets’ reliability value; 

while a hydro-dominated region such as British Columbia must conduct detailed evaluations of 

its hydro resource base and probabilistic availability under a number of weather and hydrology 

scenarios. 

To illustrate the importance of derate accounting conventions, we compare the conventions PJM 

and NYISO used in their reliability assessments. PJM adopted a 15.6% installed capacity 

(ICAP) planning reserve margin target in its most recent reliability study.19 Under PJM’s 

resource accounting methodology, demand response is derated to 95% of nameplate capacity, 

wind to 13%, and solar to 38% of nameplate capacity.20 In comparison, NYISO calculates an 

ICAP reserve margin requirement of 16.1%, but uses the full nameplate capacity of all its 

resources in that calculation.21 As explained in the NYISO study, this difference in accounting 

conventions has a substantial impact on the calculated ICAP reserve margin, particularly under 

very different levels of wind penetration. 

Taking NYISO’s resources and using PJM’s derate methodology, NYISO’s 16.1% ICAP reserve 

margin target would decrease to 10.7% as shown in Table 2 below. PJM’s target reserve margin 

using the NYISO’s resource accounting methodology would increase to 16.8% from 15.6%. 

 

18 For example, for wind, PJM uses a unit-specific availability calculation over a three-year history during 

summer peak hours, while the Midcontinent ISO (MISO) conducts a fleet-wide Effective Load Carrying 

Capability (ELCC) study each year. See PJM (2010), MISO (2012a). 

19 Note that PJM conducts a new reliability study each year, and in each study may recommend a different 

reliability requirement for each delivery year. In this subsection we refer to the year 2012 reliability study 

as effective for the 2016/17 delivery year, see PJM (2012c). 

20 To add an even further layer of complexity, the capacity market through which the reserve margin 

requirement is procured is conducted on an unforced capacity (UCAP) rather than ICAP basis as discussed 

further in the following subsection I.C.2.f. The deratings reported here apply to the calculation of the 

ICAP reserve margin. However, a different, additional adjustment is also applied to each resource type 

when calculating its UCAP value. For the purposes of calculating UCAP from ICAP, DR resources’ 

capacity value is grossed up to account for avoided reserve margin needs, most generators’ capacity value 

is reduced according to unit-specific outage rates, and wind and solar units’ derated capacity value is the 

same on either an ICAP or UCAP basis. See PJM (2012c, 2010, 2013a). 

21 For the 2012/13 delivery year, as estimated in the 2011 study. As in PJM, there is also a separate 

translation for determining the UCAP capacity value of each resource as would be used in the capacity 

market. See NYSRC (2011). 
11 

Table 2 

Illustration of Differences in PJM and NYISO Target Reserve Margin Calculations 

Sources and Notes: 

 See PJM (2012c), NYSRC (2011). 

 We do not attempt to document the entirety of differences in reserve margin accounting, but only illustrate the impact of these 

specific derating conventions applied over this reported quantity of resources. A number of other factors related to how these

resources are modeled will also influence the reserve margin calculation, which we do not attempt to discuss here. 

 NYISO DR value refers only to special case resources (SCRs), see NYSRC (2011), p. 51. 

This shows that, while NYISO’s reported ICAP reserve margin target exceeds that of PJM, it is 

lower than PJM’s when calculated on a comparable basis. But note that, because the technical 

details of resource adequacy studies and the regions themselves can vary considerably, even if 

PJM’s and NYISO’s reserve margins were calculated consistently with respect to the treatment 

of the resources’ nameplate capacity, a lower reported planning reserve margin would not 

necessarily mean lower reliability.

d. Uncommitted Resources, Planned Retirements, and Planned Additions 

Another difficulty in comparing reserve margins across regions or even within a particular region 

is that one must determine whether and how to account for resources that may or may not be 

available in a future delivery period. In particular, there are different approaches to accounting 

for: 

Uncommitted Resources that exist within the footprint and so are likely to provide resource 

adequacy value, but that do not have any firm commitment to be available through either a 

contractual agreement or a capacity market obligation. Without a firm commitment to 

provide internal capacity, these resources are free to retire or export their capacity and so 

may not be available when needed.22

Planned Retirements are sometimes known with substantial lead time and so can be properly 

accounted for. But in many cases plant owners do not have to report retirements to the 

system operator until shortly prior to the delivery period. This uncertainty is currently of 

substantial concern in MISO, where a large number of coal plants are at risk for retirement 

under the Mercury and Air Toxics Standard (MATS). However, the RTO has a limited 

 

22 This is one reason that some RTOs including PJM separately report committed and uncommitted “reserve 

margins” when conducting summer resource assessments, see PJM (2013e). 

Peak Load Nameplate Capacity Derated Value ICAP Reserve Margin

Summer 

2012

Wind Solar DR Wind Solar DR As 

Reported

Using 

Nameplate 

Capacity 

Using 

Derated 

Capacity 

(MW) (MW) (MW) (MW) (%) (%) (%) (%) (%) (%)

PJM 151,394 1,305 564 7,263 13% 38% 95% 15.6% 16.8% 15.6%

NYISO 33,335 1,648 39 2,192 11% 65% 85% 16.1% 16.1% 10.7%
12 

forward view regarding the quantity and location of these retirements, which introduces 

substantial uncertainty for both transmission and resource planning.23

Planned Additions are similarly uncertain on a forward basis. The majority of announced 

generation projects are never brought into commercial operation, but may be partly 

developed before being postponed or cancelled depending on market and site conditions. 

For this reason one must carefully evaluate the commercial probability of various proposed 

resources when calculating a future reserve margin, for example, by assigning different 

probabilities that each resource will come online depending on how far along it is in the 

development process.24

Again, the conventions used to account for these resources change the reported reserve margins 

despite the fact that they do not change the underlying resource adequacy position. 

Unfortunately, attempting to standardize the treatment of these uncertainties may be counterproductive, since the most appropriate convention to use is highly dependent on the purpose for 

which the reserve margin is calculated. For example, a transmission planning study that looks 

five to twenty years into the future must include some reasonable assumptions about speculative 

additions and retirements, while a near-term summer assessment may most appropriately 

consider only committed resources that can definitely be relied upon. 

The large range in different reserve margins that can be calculated using different conventions is 

illustrated each year in the annual NERC Long Term Reliability Assessment (LTRA) that 

documents the reserve margin outlook for all regions. To help with some standardization across 

regions, NERC provides guidance regarding how to account for uncertain resource types for a 

range of optimistic or pessimistic outlooks. This improves the ability to compare outlooks across 

regions, but does not solve the problem entirely due to differences in interpretation, even on a 

very near-term basis where uncertainty is lowest. For example, the 2012 LTRA outlook reports 

the summer 2013 anticipated reserve margins for 26 different regions, of which: (a) 12 reported 

no difference between the most pessimistic and optimistic reserve margin outlooks for the 

summer of 2013; (b) another 8 reported up to 1% uncertainty range; (c) another 3 reported up to 

10% uncertainty range; and (d) another 3 reported higher uncertainty ranges of up to 18% 

difference between the most optimistic and pessimistic cases.25 The ranges diverge even further 

when looking further into the future. 

e. Accounting for Tie Benefits 

Being interconnected to neighboring regions can improve resource adequacy in two ways. First, 

by enabling a region to procure firm commitments from external resources to supply and deliver 

power during emergency events. And second, by creating tie benefits that enable the region to 

request neighbor assistance, if available, through non-firm imports during emergency events. 

The latter type of assistance is neither guaranteed nor contractually obligated to materialize, but 

 

23 For example, see a discussion of these uncertainties and MISO’s concerns about these uncertainties in 

MISO (2013b). 

24 PJM uses this approach for a number of purposes, see PJM (2012c). 

25 See NERC (2012), p. 68. 
13 

the likely availability of these tie benefits can be probabilistically evaluated as discussed in 

Section I.C.3.d below. 

It is standard practice to include firm imports on the supply side of a reserve margin calculation, 

but there is no standard practice for how to include tie benefits. For example: (a) the most 

common approach, such as used in PJM and MISO, is to exclude tie benefits from the calculated 

reserve margin, which means the target reserve margin is lower than it would be if tie benefits 

were included as a resource; (b) some regions, such as ERCOT, treat all or part of their interties 

as a supply resource that increases the calculated reserve margin; (c) other regions, such as 

Alberta, sidestep the problem of evaluating the reliability value of the interties by simply 

reporting the reserve margin with the interties contributing at zero or 100% of their path rating; 

(d) some regions, such as SPP, do not account for tie benefits at all; and (e) yet other regions 

have special circumstances, such as ISO-NE’s separate treatment of two portions of the tie 

benefits, one part that is specifically allocated to entities that own the rights to an intertie with 

Québec, and another part that is allocated equally to all loads.26

f. Accounting for Installed and Unforced Capacity 

A final source of discrepancy is the differing conventions for expressing resource adequacy 

targets either in terms of: (1) “installed” capacity ratings that account for thermal generators at 

their maximum output capability; or (2) “unforced” capacity ratings that discount resources’ 

capacity rating by an expected forced outage rate (EFORd). All regions use ICAP reserve 

margins for most reporting purposes, including summer and long-term resource assessments. 

However, even under ICAP accounting there are varying degrees of uncertainty. For example, it 

is relatively easy to find the same plant listed at different ICAP values since the RTO may be 

reporting a nameplate, summer, winter, emergency maximum, committed, or an annual capacity 

test value. It is not unusual for an individual thermal generation resource to have reported 

capacity values that vary by many percentage points. 

The consequences of such accounting discrepancies can have high financial implications in 

regions with bilateral or centralized capacity markets, with individual suppliers’ capacity 

payments potentially materially impacted by such differences. For this reason, MISO, PJM, and 

NYISO have all opted to conduct their capacity auctions on a UCAP basis, which more 

systematically accounts for individual resources’ summer availability after accounting for unitspecific historical availability ratings.27 Under UCAP accounting, all resource types are derated 

to their expected “unforced” availability during summer peak conditions so that resources have a 

more similar contribution to resource adequacy. While the approach to calculating UCAP is 

different for each resource type and also varies by region, the UCAP-based reliability 

requirement is more meaningful and comparable under different resource mixes and across 

regions. Despite the advantages of UCAP accounting, however, RTOs generally continue their 

 

26 See PJM (2012c); MISO (2012c); ERCOT (2013b); SPP (2012b), p. 20; ISO-NE (2012); and AESO 

(2013). 

27 However, both California’s Resource Adequacy Requirements and ISO-NE’s Forward Capacity Market 

are implemented using ICAP accounting. 
14 

historical approach of reporting ICAP-based planning reserve margins for nearly all purposes 

other than capacity auctions. 

3. Differing Approaches to Conducting Reliability Modeling 

Even though most of the planning entities surveyed define the 1-in-10 standard as 0.1 LOLE, the 

resource adequacy levels of these regions nevertheless may not be comparable. Each system 

operator conducts its own LOLE study using a range of different modeling assumptions. As a 

consequence 0.1 LOLE does not represent the same level of reliability in each system. These 

differences result from using different reliability modeling tools and a wide range of different 

study assumptions. 

In this section, we highlight a number of important factors that create these discrepancies. In 

Appendix A.2 we further illustrate the wide range of approaches by summarizing those used for 

ISO-NE, NYISO, PJM, MISO, SPP, and SoCo.28 To fully understand reliability differences 

across different systems, all studies and reserve margin calculations would need to be undertaken 

using consistent assumptions, methodologies, and definitions. A recent initiative by NERC 

resulted in recommended modeling assumptions that attempt to reconcile at least some of these 

existing discrepancies.29 We also note that this list of differences among modeling approaches 

represents only a small subset of all potential differences, and that we have not attempted to 

document other important factors such as hydro resource modeling, wind resource modeling, 

approaches to characterizing correlations among load, wind, hydro, and generator outages, or 

approaches to calculating sub-regional resource adequacy requirements. 

a. Different Reliability Modeling Packages 

Different system operators use different models to conduct their reliability studies, including GEMARS, SERVM, ABB Gridview, Genesys, and other non-commercially available models such 

as PJM’s Prism Model and Florida Reliability Coordinating Council’s (FRCC’s) TIGER 

Model.30 The models differ considerably, including in the range of uncertainties considered and 

how these uncertainties are captured in simulations. These differences can translate to very 

different study results. 

b. Definition of Reliability “Events” 

Regions that use the 0.1 LOLE standard hope to achieve a resource adequacy level of only one 

“event” in ten years. However, as noted previously, there is no standard approach for 

determining what constitutes a reliability “event.” Some LOLE studies and system operators 

define a reliability event as any event requiring emergency operating procedures, such as calling 

 

28 Tennessee Valley Authority (TVA), Duke, Progress Energy, Louisville Gas and Electric Company 

(LG&E), and Kentucky Utilities Company (KU) use assumptions similar (though not identical) to SoCo. 

For example, TVA does not include DR as a resource. Duke, Progress, LG&E, KU, and TVA define a 

reliability event as an event that requires the depletion of spinning reserves, whereas SoCo defines a 

reliability event as an event that requires depletion of any type of operating reserve. 

29 NERC (2010b). 

30 See Appendix A.1 for a summary of the reliability modeling packages used in various systems. 
15 

demand response, implementing voltage control, or depleting operating reserves. In other 

systems, the studies consider reliability events as contributing to the tabulated LOLE only if 

operating reserves are depleted and firm load must be shed. Finally, many reliability studies do 

not explicitly account for the nature, order, and impact of various emergency intervention events. 

These factors can have a substantial impact on study results as illustrated in Section III.A.1 

below. 

c. Treatment of Load Uncertainty 

One of the most important factors driving resource adequacy is uncertainty in peak load, which is 

driven by both weather uncertainty and economic forecast uncertainty.31 For example, the 

combination of the number of weather years studied, the process used to derive peak and hourly 

loads under normal weather conditions, and the probabilities assigned to extreme weather 

patterns can change planning reserve margin targets by several percentage points. For example, 

a recent LOLE study by ERCOT found differences in weighting the 2011 weather year, which 

some refer to as a 1-in-100 year heat wave. Depending on how likely such extreme weather is to 

recur, the resulting target reserve margins would range from a low of 13.7% (with zero 

probability) to a high of 18.9% (with a 5% probability).32 Even differences in how the projected 

load growth is applied to a base-year profile yielded target reserve margins ranging from 15.3% 

to 16.1%.33 As another example, some regions such as PJM consider uncertainty due to 

economic load forecast error over the planning horizon when calculating the reserve margin 

needed to meet 0.1 LOLE, while other regions such as ISO-NE exclude this portion of load 

forecast uncertainty and instead consider only weather-related load uncertainty.34 

d. Estimating Tie Benefits 

As explained in subsection I.C.2.e above, regions differ in how they account for tie benefits in 

their reserve margin calculations. In this subsection, we further explain how regions differ in 

their approaches to estimating tie benefits. Some studies assume a maximum import capability 

or tie benefit, simply counting that quantity as a firm resource. Other studies explicitly model 

surrounding regions’ load and resources to probabilistically estimate the availability of imports 

during reliability events. Among the regions that do probabilistically assess tie benefits, 

differences in modeling approaches include assumptions regarding: 

Maximum Intertie Ratings – Determining the total quantity of intertie capability to use in 

resource adequacy studies is not as simple as calculating the line ratings for every import 

point in a region. Instead, this is a complex engineering estimate that must consider 

simultaneous import capability from multiple directions under the expected peak season 

power flows. Further, the maximum intertie portion used for the purposes of calculating 

non-firm tie benefits is the total path rating minus any portion of the interties already used 

to support firm import commitments as discussed further in Section III.B.2 below. Most 

 

31 As discussed further in Sections II.D and III.A.5 below. 

32 ERCOT (2013a), pp. 24-26. 

33 ERCOT (2013b), Section 2.2.3 (p. 4) and pp. 7-8. 

34 See PJM (2012c) and ISO-NE (2012). 
16 

systems use simplified assumptions about maximum intertie ratings that may not be 

updated regularly to reflect changing system conditions.35 

Probabilistically Available Intertie Capability – Many reliability studies assume a fixed path 

rating and assume that the transmission will be entirely available at all times. However, 

this is not strictly true since every path may have variable availability, for example due to 

simultaneity, line outages, or temperature deratings. Some studies probabilistically account 

for these uncertainties in path ratings, as we do in the hypothetical RTO simulations 

presented in this report.36 

Load Diversity – One of the important drivers of tie benefits is the extent of load diversity 

across regions. For example, if one region experiences a peak load emergency while the 

neighboring region does not, then assistance should be available over the interties. 

However, the scale and variability of load diversity can be characterized in many different 

ways. For example, NYISO’s planning models simulate neighboring regions, but assume 

that all neighboring regions have the same highest peak-load hours, all based on a single 

historical year. As explained in their study, this is a conservative assumption that reduces 

the amount of capacity that would be available from neighboring regions during NYISO 

reliability events.37 In contrast, PJM’s planning study assumes different levels of peak load 

diversity with each of its neighbors, meaning that PJM assumes other regions’ peak loads 

are a certain percentage below their respective non-coincident peak loads when PJM is 

peaking.38 This will yield a higher resource adequacy value of interties with neighboring 

regions, thereby reducing the likelihood of load-shed events and the corresponding target 

planning reserve margins. Other studies, including the simulations presented below, rely 

on a large number of historical weather years for the study region and neighboring regions 

to more accurately capture the benefits of tielines and load diversity. For example, the 

simulations in this report are based on 32 different synthetic annual load shapes for the 

Study RTO and neighboring regions based on 32 years of actual historical load data in each 

region.39

Supply in External Regions – To calculate tie benefits, it is also necessary to assume the level 

of supply in each neighboring region and the extent to which such resources will be made 

available for neighbor assistance during reliability events. This requires assuming reserve 

margins for each neighboring region, including whether to model the external market at 

their planning reserve margin target or at their most likely actual level of installed capacity. 

Further, studies differ in the extent to which each region is assumed to dispatch demand 

response or other energy-limited resources to provide assistance to its neighbors. 

 

35 For example, PJM uses a total assumed simultaneous intertie capability or Capacity Benefit Margin 

(CBM) of 3,500 MW to be used for the purposes of calculating tie benefits. This rating has not been 

updated since at least 2007, even though the RTO’s footprint has expanded several times since then and 

intertie topology and capability would necessarily have changed at least somewhat. See PJM (2007, 

2012c). 

36 See Section II.B. 

37 See NYSRC (2011), p. 47. 

38 See PJM (2012c), p. 75. 

39 See Section II.D and Appendix B.5 for additional discussion. 
17 

Overall, these differences in study assumptions can substantially affect the planning reserve 

margins necessary to achieve the chosen reliability targets, as we illustrate further in Section 

III.B.2 below. 

II. APPROACH TO MODELING THE ECONOMICS OF RESOURCE ADEQUACY 

To examine the economic implications of the 1-in-10 resource adequacy standard, we conduct a 

series of reliability simulations using the SERVM model developed by Astrape Consulting.40 In 

this section, we provide an overview of our simulation approach, including a description of the 

SERVM model, the characteristics of the hypothetical system we examine, and our key modeling 

assumptions. 

We simulate a hypothetical RTO system that is interconnected to three neighboring systems. 

This hypothetical RTO is not intended to represent any actual system, but has characteristics 

similar to those of many U.S. markets. For example, we assume that the Study RTO’s resource 

mix is similar to that of NYISO, while the weather and wind profiles are similar to those in the 

TVA region. The resource mix, weather, and wind profiles of Neighbors 1, 2, and 3 are similar 

to those of MISO, PJM, and SoCo. We model both reliability and economic outcomes for the 

Study RTO in 2016 as the assumed delivery year, as forecasted from a 2013 resource planning 

year. We describe our modeling approach and assumptions in this section and provide additional 

detail in Appendix B. 

A. STRATEGIC ENERGY RISK VALUATION MODEL OVERVIEW

We use the Strategic Energy Risk Valuation Model (SERVM) to study the reliability and 

economic implications of the 1-in-10 standard and alternative resource adequacy standards. Like 

other reliability modeling tools, SERVM probabilistically evaluates resource adequacy 

conditions by simulating generation availability, load profiles, load uncertainty, transmission 

availability, and other factors. Based on these reliability simulations, SERVM estimates 

standard reliability metrics including LOLE, LOLH, and EUE. Unlike other reliability modeling 

packages, however, SERVM also simulates economic outcomes including hourly generation 

dispatch, import-export dynamics, ancillary services, emergency procedures, and investment 

costs. SERVM estimates hourly and annual production costs, customer costs, market prices, net 

import costs, load shed costs, and generator energy margins as a function of planning reserve 

 

40 The SERVM package was originally developed by SoCo in the 1980s to support resource adequacy 

decisions and significant resources have been devoted to develop the model since then in a number of 

applications by various companies. Today, SERVM is managed by Astrape Consulting, who provides 

consulting services and licenses of SERVM to inform resource adequacy decisions to a number of clients, 

including LG&E, KU, SoCo, TVA, CleCo, SCANA, Duke Energy, Progress Energy, and Terna (the 

Italian system operator). 
18 

margins. These results allow us to compare these variable costs against the incremental capital 

costs required to achieve the higher planning reserve margins.41

The multi-area economic and reliability simulations in SERVM include an hourly chronological 

economic dispatch that is subject to inter-regional transmission constraints. Each generation unit 

is modeled individually, characterized by its own economic and physical characteristics. 

Planned outages are scheduled in off-peak seasons to minimize the impact on reliability, while 

unplanned outages and derates occur stochastically using historical distributions of time between 

failures and time to repair. Load, hydro, wind, and solar conditions are modeled based on 

profiles consistent with individual historical weather years. Dispatch limitations and limitations 

on annual energy output are also imposed on certain types of resources such as demand response, 

hydro generation, pumped storage, and environmentally-limited combustion turbines (CTs). 

The model implements a weekly commitment algorithm in each study region that considers the 

outlook for weather and planned generation outages. The model then conducts an hourly 

economic dispatch of baseload, intermediate, and peaking resources, including an optimization 

of transmission-constrained inter-regional power flows to minimize total cost. Pumped storage 

resources are dispatched economically to capture differences in pricing between peak and offpeak periods. During most hours, hourly prices in each region reflect marginal production costs, 

with importing regions realizing higher prices when import constraints are binding. During 

emergency and other peaking conditions, SERVM simulates scarcity prices that exceed 

generators’ marginal production costs as explained further below. 

To examine a full range of potential economic and reliability outcomes, SERVM conducts a 

Monte Carlo analysis over a large number of scenarios varying with both demand and supply 

conditions. Because reliability events occur only when system conditions that reflect unusually 

high loads or limited supply, these simulations must capture wide distributions of possible 

weather, load growth, and generation performance scenarios. In this study, we incorporate 32 

weather years, 6 economic load forecast error points, and 50 draws of generating unit 

performance for a total of 9,600 scenarios for each simulated reserve margin case, with each 

scenario simulating all 8,760 hours of the year.42 This large number of simulations is necessary 

for accurately characterizing the reliability and economic implications of different planning 

reserve margins because the vast majority of reliability-related costs are incurred within a small 

 

41 Note that SERVM as a modeling tool does not endogenously estimate capital costs, which are reflected as 

a fixed annual cost (in $/kW-year). Total capacity costs simply increase as a function of the planning 

reserve margin being evaluated. The question of whether a particular reserve margin is actually 

achievable or realistic under various market designs, including in regions with capacity markets, energyonly markets, or cost-of-service regulation, depends on whether those markets have been constructed such 

that investors are able to recover their fixed costs at that reserve margin. We examine these and other 

market design implications of our modeling exercise in Section IV below. 

42 Note that SERVM results incorporate this set of scenarios for each reserve margin level simulated for the 

base case and each alternative case we present in this report. Depending on data availability and study 

needs, SERVM can incorporate any number of weather, hydro, wind, solar, economic forecast, and other 

scenarios. For the purpose of this study and to facilitate exploration of a larger number of alternative 

cases, we streamlined the simulation effort by modeling only 50 draws of generation outage conditions. In 

other studies, Astrape typically models several hundred generation outage draws. 
19 

number of simulation runs (or, in real-world systems, in a small number of infrequent emergency 

events). Such reliability events are typically triggered by rare circumstances that reflect a 

combination of extreme weather-related loads, high load-growth forecast error, and unusual 

combinations of generation outages. 

To properly capture the magnitude and impact of reliability conditions during extreme events, a 

critical aspect of this modeling exercise the economic and operational characterization of 

emergency procedures. For this reason, SERVM simulates a range of detailed emergency 

procedures, accounting for energy and call-hour limitations, dispatch prices, voltage reductions, 

operating reserve depletion, dispatch of economic and emergency demand-response resources, 

and administrative scarcity pricing. 

B. SYSTEM TOPOLOGY AND TRANSMISSION ASSUMPTIONS 

We evaluate the economic reliability implications of resource adequacy by analyzing a 

hypothetical RTO interconnected to three adjacent systems. This hypothetical RTO has system 

characteristics similar to those of many U.S. markets, but does not exactly reflect any one region 

in particular. By constructing such a generic system as a Base Case and comparing those results 

to a series of Change Cases, we are able to examine our results under a variety of realistic 

resource mixes, load profiles, market structures, and economic assumptions. This allows us to 

draw conclusions that are relevant to a number of real-world systems without narrowing our 

focus to any individual real-world system. As summarized in Figure 3, the Study RTO is a 

50,000 MW system interconnected with three similarly-sized regions. 

The transmission capabilities shown in the figure represent the maximum import and export 

capabilities of the interties, which vary stochastically over a distribution between 60% and 100% 

of the maximum. As we explain further in Appendix B.1, the maximum simultaneous import 

capability to the study region is 11,000 MW but drops to below 8,000 MW in 50% of all hours.43 

We assume that no portion of the interties’ transfer capability is dedicated to the firm import of 

external resources committed towards resource adequacy of the Study RTO.44 

 

43 We statistically represent the variable availability of inter-regional transmission capability, but do not 

explicitly model the various reasons that interties may be partly unavailable at different times. However, 

we note that some of these factors include line outages, path ratings that may vary with temperature or 

operating procedures that sometimes limit imports to avoid great exposure to single large contingencies. 

44 In real-world systems, it is important to distinguish the portion of each intertie that is dedicated to firm 

capacity imports under firm Point-to-Point (PtP) transmission from the remaining portion of the interties 

that can assist internal resource adequacy on a non-firm basis as long as the neighboring system’s 

resources exceed the needs of their native load. For our purposes we treat all external resources as being 

available to the Study RTO on a non-firm basis driven by economics, but assume that in emergency events 

each system will have first call on their internal generating resources. We discuss these topics further in 

Section III.B.2 below. 
20 

Figure 3 

Transmission Topology and Non-Coincident Peak Loads 

Notes: 

Stated intertie ratings represent the path maximum. 

 See Appendix B.1 for additional detail on intertie availability. 

C. RESOURCE ASSUMPTIONS

To characterize the generation and demand response resources available to meet load the 

simulated Study RTO and the neighboring regions, we assume resource characteristics consistent 

with many real-world systems. We present here a description of our assumptions and approach 

to modeling system supply, including each region’s resource mix, supply curves, generation 

resource availability, and demand response resources. 

1. Resource Mix 

Figure 4 summarizes the supply resource mix in the Study RTO and the neighboring systems, 

with the Study RTO represented at the 15% reserve margin. We model each system with a 

reasonable proportion of baseload, intermediate, and peaking resources. Each region also has 

some energy-limited and intermittent resources such as hydro, pumped storage, wind, solar, and 

emergency demand response. Some characteristics of the Study RTO that materially impact 

study results include that: (1) the Study RTO depends more heavily on natural-gas-fired plants 

while the neighboring regions have more coal, meaning that the Study RTO will tend to be a net 

importer at the assumed natural gas and coal prices; (2) the 10% hydro penetration in the Study 

Region is higher than in neighboring regions, adding some additional uncertainty into the Study 

RTO’s resource availability; and (3) assumed DR penetration is 6-7% of peak load depending on 

Study RTO 

50,000 MW

Neighbor 3

30,000 MW

Neighbor 1

60,000 

Neighbor 2

40,000 

4,000 MW 4,000 MW

3,000 MW

3,000 MW
21 

the region, which is consistent with many current markets but below the penetration levels 

expected in coming years in some of the eastern RTOs, such as PJM and ISO-NE.45 

Figure 4 

Installed Capacity and Resource Mix 

Notes: 

All regions shown at 15% planning reserve margin, although simulated reserve margins 

vary by model scenario. 

 Wind and solar shown at capacity credits of 15% and 25% respectively, which is the same 

as their contribution to the reserve margin (other resources are represented at nameplate) 

We assume a planning reserve margin of 15% is maintained in all neighboring regions, similar to 

the current reserve margin targets in a number of actual North American markets. We calculate 

reserve margins based on nameplate capacity for all resource types except for wind and solar, to 

which we assign capacity value at 15% and 25% of nameplate, respectively. Economic and 

emergency demand response capacity are included on the supply side in the reserve margin 

calculation.46 The load impact of voltage reduction is not considered in the reserve margin 

 

45 For example an ISO/RTO Council Study estimated a 6.6% overall DR penetration level across all North 

American RTOs as of 2008, while PJM and ISO-NE’s capacity auctions for 2015/16 and 2016/17 cleared 

DR commitments equal to 9.9% and 9.3% of peak load respectively. See IRC (2009), p. 26; PJM (2013a); 

and ISO-NE (2013). 

46 Section II.C.4 explains our approach to modeling economic and emergency DR, including a description of 

how we distinguish these two resource types from a modeling perspective. 
22 

calculation, but we do implement voltage reductions to avoid shedding load in our simulations as 

discussed in Section II.E. 

2. Energy Market Supply Curve 

In modeling production costs for each supply resource, we assume plant costs and efficiencies 

that are typical for each resource type and that vary among individual units. The aggregate 

system supply curves for each region are shown in Figure 5 and account for fuel costs and 

variable operations and maintenance (VOM) costs. 47 We assume delivered fuel costs as 

projected for the year 2016 in the Annual Energy Outlook, with coal at $2.50/MMbtu, natural gas 

at $5/MMbtu, and fuel oil at $25/MMbtu.48 

As Figure 5 shows, the Study RTO’s supply stack is similar to, but not identical to, those of its 

neighbors. Over most of the relevant dispatch ranges from 50% to 100% of peak load, the Study 

Region’s available supply from natural gas combined cycles (CCs) is slightly more expensive 

than supply from the neighboring regions that have a higher share of lower-cost coal resources. 

At the very top of the supply stack shown in Figure 5, the Study RTO supply curve is also higher 

because it depends somewhat more heavily on high-dispatch-cost demand response resources 

and emergency hydro for super-peaking periods, while the neighboring regions have more 

peaking gas and oil units. The highest-price portion of the supply curve at dispatch costs above 

$500/MWh is not shown in this figure, but causes the Study RTO supply curve to rise sooner 

because it is shown as a percentage of peak load. We more fully describe the emergency 

procedures, economic costs, and energy pricing mechanisms assumed during high-price events in 

Section II.E below. 

 

47 While SERVM also has the capability to model a host of other unit parameters including startup and 

shutdown costs, minimum up and down times, emissions costs, and heatrate curves, we exclude these 

parameters from our study for simplicity. 

48 Specifically, we use average national prices as-delivered to the electric power sector from the High 

Economic Growth case in the 2012 Annual Energy Outlook. Fuel prices reported in nominal dollars, see 

EIA (2012). 
23 

Figure 5 

Energy Market Supply Curve by Region 

Notes: 

 Wind and solar capacity derated to 15% and 25% of nameplate. 

 Pumped storage represented at a $0/MWh, although dispatched only at daily peak. 

 $2,000/MWh emergency DR and $3,000/MWh emergency hydro not shown due to scale. 

3. Generation Outage and Availability Modeling 

A major component of reliability analyses is modeling the availability of supply resources after 

considering planned outages, forced outages, intermittency, and energy limitations. We use 

SERVM to simulate generation resource availability on a unit-specific basis, but characterize 

resource utilization and availability differently depending on resource type: 

Thermal Resource Outages – We assume planned and forced outage rates for thermal 

generation resources based on unit type and size, consistent with recent NERC data from 

the Generation Availability Data System (GADS).49 SERVM schedules planned outages 

during low-demand periods in the spring and fall, but models maintenance and forced 

outages stochastically. Partial and full forced outages occur stochastically for each 

individual unit based on probability distributions accounting for time-to-fail, time-torepair, and partial outage derate percentages. Similarly, SERVM models the need for 

maintenance outages stochastically, but implements these outages with some scheduling 

flexibility during off-peak hours to the extent feasible. This outage modeling approach 

allows SERVM to capture the potential for severe shortages caused by a number of 

coincident outages.50 Appendix B.2 contains additional detail describing forced and 

maintenance outage modeling. 

 

49 See NERC (2011b). 

50 Capturing the possibility of such low-probability, high-impact events is an advantage of the unit-specific 

Monte Carlo outage modeling used in SERVM. Other production cost and reliability models typically use 

Continued on next page 
24 

Hydroelectric Availability – We model 30 years of hydro resource availability using monthly 

hydro generation data for the respective regions and historical years used in our load 

modeling described in Section II.D.1 below.51 These hydro resources have varying levels 

of flexibility as either run-of-river plants with a fixed schedule, peak shaving plants that 

are dispatched based on the difference between peak and off peak prices, or emergency 

hydro capacity dispatchable at $3,000/MWh. See Appendix B.3 for additional detail on 

hydro resource modeling. 

Pumped Storage – We characterize pumped storage resources according to reservoir size, 

pumping efficiency, and capacity and dispatch these resources to maximize their 

economic value assuming imperfect foresight of hourly energy prices.52 The units pump 

during off-peak hours and generate during on-peak hours, but only when the anticipated 

price differential is large enough to overcome pumping efficiency losses. 

Intermittent Resources – We model intermittent wind and photovoltaic solar resources 

assuming non-dispatchable hourly generation profiles for the Study RTO’s modest 

penetration levels of 3.8% and 0.09% for wind and solar respectively in the Base Case.53 

For wind generation, the hourly profiles reflect three years of data from the Eastern Wind 

Integration Transmission Study (EWITS); for solar they reflect one year of data from 

PV Watts.

54 In both cases, the hourly profiles are consistent with the weather data for the 

four regions used in our load modeling. To capture intermittent resource availability as 

well as inter-regional weather correlations, SERVM randomly draws a particular day 

within the month being simulated and uses hourly profiles from that same date for each 

modeled region. See Appendix B.4 for additional detail on our intermittent resource 

assumptions. 

After SERVM accounts for all of these factors, the model determines the aggregate quantity of 

resources available for economic dispatch as described in Section II.A above. 

4. Economic and Emergency Demand Response Resources 

Most regions have a variety of DR programs that can be dispatched whenever a certain price or 

reliability event occurs.55 While the nuances of each variety of DR program could be modeled in 

SERVM consistent with a region’s actual DR portfolio, we focus on simulating two broad 

categories of DR programs: 

 

Continued from previous page 

a simpler convolution method, resulting in a distribution of outages that may under-estimate the potential 

for extreme events, especially in small systems. 

51 Monthly hydro data for each region obtained from EIA (2013a). 

52 Specifically, to reflect the scheduling activity of a pumped hydro resource with slightly imperfect foresight 

of actual market prices in each hour, we implement this hourly scheduling using a price profile from the 

same date from a different run of SERVM with the same load profile but with different outages. 

53 These penetration levels are calculated as nameplate as percent of summer peak load. 

54 ETWITS and PV Watts are both developed by the National Renewable Energy Laboratory, see NREL 

(2013a-b). 

55 For a survey of various types of demand response programs, see the summary and referenced supporting 

documentation in IRC (2009), Section IV and IRC (2010). 
25 

Emergency DR – The majority of existing RTO-dispatchable DR programs would be most 

accurately described as “emergency” or “capacity-only” resources. Emergency DR 

programs contribute to resource adequacy and increase planning reserve margins because 

they can be dispatched during reliability events. These resources will not dispatched on 

an economic basis, but will be triggered during emergency conditions according to the 

system operator’s emergency procedures as explained in Section II.E below. We assume 

that these DR resources have a curtailment cost of $2,000/MWh and a 100 hour-per-year 

maximum dispatch limit.56 

Economic DR – A minority of existing RTO-dispatchable DR programs are what we classify 

as “economic” DR resources. These resources bid into the energy market at a resourcespecific strike price that reflects the customer’s willingness to pay for energy. Whenever 

wholesale energy market prices exceed this strike price, the resources would be 

dispatched just like a generator that offered into the market at the same offer price. 

Economic DR resources also contribute to planning reserve margins, but we assume they 

do not have any limit on the annual call hours. We simulate a supply curve of economic 

DR resources with strike prices ranging from $100 to $1,000/MWh in the Base Case and 

over a wider range of strike prices up to the price cap of $7,500/MWh in a high economic 

DR penetration case as explained further below.57 

Our Base Case simulations assume DR penetration levels of 6-7% of peak load across the 

different regions as shown in Table 3, consistent with the penetrations in many actual systems 

today.58 We further assume for all regions that 80% of the DR resources are emergency DR 

while 20% are economic DR programs.59 

 

56 It is standard practice among many systems to have “emergency” or “capacity” DR programs that are 

triggered by emergency conditions rather than based on energy market prices. Many of these programs 

also have call-hour or other types of restrictions that limit their total annual use, but there is no standard 

call-hour limitation and so our 100 hour limit is somewhat arbitrary. For example, PJM’s limited DR 

product can be called only in summer peak periods for up to 10 calls and up to 6 hours per call. The percall cost of these programs is also relatively arbitrary, and so we assume an interruption cost that is 

consistent with the price cap applicable in some markets that implement these programs as well as the 

order in which these emergency resources are called as discussed further in Section II.E. See IRC (2010). 

57 Economic DR suppliers may offer at any strike price of their own choosing, but at a price that reflects the 

underlying customer’s willingness to pay for energy. We assume an offer price range of $100-

$1000/MWh in the Base Case, reflecting the assumption that economic DR would typically bid only at 

prices above the average retail price for energy. We also evaluate an alternative scenario with 

substantially more economic DR, as explained further below and illustrated in Figure 6. 

58 For example, see IRC (2009), p. 26. 

59 This ratio of emergency DR to economic DR is based on the current proportion in PJM, see PJM (2013c), 

p. 5. 
26 

Table 3 

Base Case Demand Response Penetration 

Notes: 

 Base Case DR penetration levels are consistent with levels currently achieved in 

many real-world RTOs, see IRC (2009), p. 26, 

We simulate a series of alternative DR cases to evaluate the implications of growing levels of 

DR penetration. The consequences of relying more heavily on DR for resource adequacy 

purposes will be particularly important over the coming years in PJM, which has attracted large 

quantities of DR commitments through its forward capacity market, amounting to 9.9% of 

normalized peak load by 2015/16.60 

At PJM’s high historical reserve margins, increasing quantities of DR would not be expected to 

substantially impact PJM’s energy market pricing or reliability outcomes. This is because the 

“generation-only” reserve margin was high enough to meet the reliability target even without any 

DR commitments. This will change in the coming years, however, because the large increase in 

DR penetration is also enabling the retirement of a large quantity of uneconomic generation 

resources.61 Displacing generation with DR resources has substantial implications, because the 

“generation-only” reserve margin will soon be far below the reliability target. This means that 

DR resources will have to be dispatched more frequently. It appears likely that this combination 

of factors will: (a) cause dispatch-limited DR to have a somewhat lower reliability value, a 

concern that PJM has already addressed by introducing multiple DR products; (b) require DR 

suppliers to reevaluate their willingness to sell capacity at a particular price; (c) result in higher 

energy market prices during scarcity events; and (d) motivate more DR resources to participate 

as “economic” rather than “emergency” resources.62 

We evaluate the consequences of high levels of DR penetration in two alternative scenarios. We 

begin with a no-DR case in which all DR is removed from the Study RTO, and with total 

generation resources sufficient to maintain a 15% reserve margin. We then evaluate the 

consequences of displacing generation with DR by removing gas CTs and replacing them with 

 

60 See PJM (2013a). 

61 For additional documentation of these displacement quantities, see PJM (2013a); Newell and Spees 

(2013); and Spees, et al. (2013). 

62 See additional discussion of these topics in Sections III.C and IV.C below as well as Newell and Spees 

(2013) and Spees, et al. (2013). For an explanation and discussion of PJM’s multiple DR products, see 

Pfeifenberger, et al. (2011a), Section VI.C.1. 

Region Demand Response Capacity Total DR

Emergency Economic Penetration

(MW) (MW) (% of Peak Load)

Study RTO 2,778 694 6.9%

Neighbor 1 3,380 845 7.0%

Neighbor 2 2,132 533 6.7%

Neighbor 3 1,424 356 5.9%
27 

either: (1) Emergency DR in one scenario; or (2) Economic DR in a second scenario. In both the 

Emergency and Economic DR cases, we model DR penetration levels of 0%, 5%, 10%, and 20% 

of peak load while maintaining a 15% reserve margin across all runs. 

We assume that Emergency DR has the same characteristics as in the Base Case, with a 

$2,000/MWh dispatch cost and a 100 hour dispatch limit (although we also test the impact of 

varying the dispatch limit). We model Economic DR differently, however, with Economic DR 

resources offering over a wide range of prices up to the price cap of $7,500/MWh, as shown in 

Figure 6. This Economic DR supply curve is based on available evidence of how responsive 

customers could be to high energy market prices, but required us to make additional assumptions 

about how customers would behave at penetration levels that are far above historical 

experience.63 To create this economic DR supply curve, we assume that the aggregate customer 

base has a constant elasticity of demand, and adjust this elasticity to achieve the 5% to 20% total 

economic DR quantity up to the energy market maximum price of $7,500/MWh.64 The shape of 

this economic DR supply curve is generally consistent with the results of various real-time and 

critical-peak-period pricing experiments, although those experiments are more often centered on 

estimating peak load reductions or on-peak versus off-peak elasticities of substitution.65

 

63 For example, in PJM, economic DR offers in the energy market are only approximately 1.4% of summer 

peak load, see PJM (2013c), p. 5. 

64 More specifically, we assume that customers are represented by the formula Q = a · PE

, where: Q is 

weather-normalized peak load; a is a constant; and P is the customer-realized electricity price including 

energy (at $57/MWh on average at the 15% reserve margin as calculated in the Base Case), capacity (at 

$9/MWh at the 15% reserve margin as calculated in the Base Case), and transmission and distribution 

(T&D) charges (at $34/MWh), consistent with the national average. We calculate the parameters a and E, 

assuming that peak load would be at the normalized (50/50) forecast (consistent with the annual average 

energy price), reduced by the total DR penetration percentage at the maximum energy market price of 

$7,500/MWh. Average T&D charges are based on 2011 annual average charges of delivery-only electric 

providers averaged across all customer classes, see EIA (2013b). 

65 These economic DR supply curves are consistent with constant demand elasticities 

of -0.012, -0.024, -0.038, and -0.052 for the simulated 5%, 10%, 15%, and 20% DR penetration cases. 

These elasticities are generally consistent with estimates from various pilot studies of customer priceresponsiveness to real-time and critical peak pricing programs, although those studies often estimate 

elasticities of substitution or simple peak reductions as a function of price or price ratio. See, for example, 

Faruqui and Sergici (2010, 2011, 2013). 
28 

Figure 6 

Energy Market Supply Curve for Economic DR Penetration Case 

 

Notes: 

 Calculated assuming constant demand elasticities of -0.012 to -0.052, see also Footnotes 64 and 65. 

D. LOAD MODELING

Load uncertainty is a key factor driving reliability events. We separately account for the two 

primary drivers of load uncertainty: (1) weather, which varies from year to year over a wide 

range of hourly profiles; and (2) economic load forecasting uncertainty, which characterizes the 

uncertainty in load growth independently of weather. 

1. Weather Uncertainty 

To characterize load uncertainty caused by weather, we use 32 years of historical hourly 

temperature and rainfall data and five years of historical hourly load data for each study region. 

Weather and load profiles in the Study RTO are based on data for TVA, while Neighbors 1 

through 3 are based on central MISO, western PJM, and SoCo respectively.66 We use the most 

recent five years of hourly weather and loads to develop a statistical relationship between load 

and weather, accounting for a number of variables including month of year, time of day, and day 

of week. We use this relationship to create 32 synthetic load profiles that are consistent with 

each of the 32 historical weather years. Figure 7 shows that the resulting peak loads range from -

 

66 Hourly Historical Load Data for the Study RTO (based on TVA) and Neighbor 3 (based on SoCo) are 

derived from FERC 714 Forms. Load data for Neighbor 1 (based on central MISO) and Neighbor 2 

(based on western PJM) are derived from RTO data. All hourly weather data are from the National 

Oceanic and Atmospheric Administration (NOAA). See FERC (2013); PJM (2013b); MISO (2013a); and 

NOAA (2013). 
29 

7.84% below to 8.85% above the weather-normalized peak load in the Study RTO. Each 

weather year also has a different overall load duration curve, with some years having sustained 

heat waves that cause a much larger number of high-demand hours. 

By using weather profiles from the same historical years for each of the modeled regions, we are 

able to capture the inter-regional correlation in loads and level of load diversity on high-demand 

days. Since weather patterns can differ substantially across the four regions on summer days, the 

timing of each region’s highest peak loads will be different as well. If one region is at its system 

peak load but a neighboring region is not, the load diversity between the two systems will reduce 

their combined resource needs. Overall, the load diversity across the four regions yields a 

coincident peak load of 174,668 MW compared to the sum of regional non-coincident peaks of 

180,000 MW, or a 3% reduction due to load diversity. Appendix B.5 contains additional detail 

on weather uncertainty in load modeling. 

Figure 7 

Study RTO Summer Peak Load under Different Weather Profiles 

Notes: 

 Range in peak load shown is due to weather uncertainty alone, without economic load growth forecasting error. 

2. Economic Load Growth Forecasting Uncertainty 

Load uncertainty is driven not only by year-to-year volatility in weather patterns, but also by an 

underlying load growth forecasting error over the forward study period. This is the level of 

uncertainty in 2016 peak load as would be forecast in 2013, even if the modeler were certain that 

2016 would be a “weather normal” year. An important driver of that uncertainty for resource 

planning purposes is the uncertain outlook for economic growth. Unanticipated economic 

growth or downturns can result in peak loads that are substantially higher or lower than the 

forecast. Further, economic uncertainty increases with the forward planning period; this is 

unlike weather-based uncertainty, which is constant across all forward periods. 
30 

Figure 8 shows how the assumed level of the economic load growth forecasting error increases 

with the forward planning period. This assumed economic forecast error is based on the last 20 

years of Congressional Budget Office (CBO) four-year forward forecasts of U.S. national real 

gross domestic product (GDP) growth compared to actual GDP growth.67 We translate this 

economic growth uncertainty into load forecast uncertainty by assuming that electricity load will 

grow at 40% of the GDP growth rate, reflecting the approximate relationship between electric 

energy consumption and GDP growth over the period 1980 to 2010.68 

Note that Figure 8 shows only the peak load forecasting error related to underlying economic 

uncertainties and excludes the portion of load uncertainty related to weather. The chart shows 

that, although peak load forecast error is zero on average, the spread in possible outcomes 

increases over time so that the 80% confidence interval for economic load growth forecasting 

errors increases from a spread of only 2.2% at one year forward, to 5.4% three years forward, 

and to 17% ten years forward. Note that increasing the forward planning period in a reliability 

analysis such as ours will translate into a higher likelihood of reliability events at any given 

reserve margin, and will therefore result in setting a higher reserve margin requirement. 

Figure 8 

Economic Load Forecast Error vs. Forward Planning Period 

 

Notes: 

 Load forecast error is normally distributed around zero with a 0.8% standard deviation at 

one year forward, with standard deviation increasing 0.6% for each additional year. 

In our Base Case simulation, we assume an economic load forecast error consistent with a fouryear forward planning exercise. This assumption is consistent with many system operators’ 

planning studies and is approximately consistent with the lead time for developing a new 

 

67 See CBO (2012). 

68 During this 30-year period, electric energy consumption rose by 2% per year on average while GDP grew 

by approximately 5% per year, suggesting the relationship is approximately 0.4% load growth for every 

1% of nominal GDP growth. See CRS (2012); Data360 (2013). 
31 

combustion turbine plant.69 The choice of this forward period for our simulations implicitly 

assumes that all resource planning decisions are locked in on a four-year forward basis with no 

shorter-term flexibility from uprates, DR procurement, delayed retirements, or off-system power 

purchase agreements (PPAs). However, if there is some flexibility to procure a portion of supply 

on a shorter-term basis, then total system costs could be reduced by: (1) maintaining a lower 

reserve margin while meeting the same reliability requirement; and (2) capturing the option 

value of waiting to procure a portion of resources until after some of the economic forecast 

uncertainty has resolved. We more fully explain and examine the economic and reliability 

implications of the forward planning period in Section III.A.5 below. 

We represent this four-year forward distribution using six discrete levels of economic load 

forecast error with each assigned a specific probability level as summarized in Figure 9. We 

apply this economic forecast error equally to the Study RTO and the four neighboring regions, 

assuming economic load growth uncertainty is perfectly correlated across the study regions.70

Figure 9 

Probability of Various Levels of Economic Load Forecast Error 

Notes: 

 Economic forecast error consistent with a four-year forward period. 

E. SCARCITY CONDITIONS

To calculate production costs and market prices during most hours, SERVM uses hourly supply 

curves of available resources in each region. The higher the generation outages and load in a 

particular hour, the higher the marginal dispatch cost and market price as explained in 

 

69 Note that total lead times including development, siting, and permitting may be substantially longer, but 

these phases of development represent a relatively small component of total plant costs. 

70 While there will likely be at least some diversity of load forecasting error across the five regions, the use 

of U.S.-wide GDP growth forecast already captures at least some such diversity because forecast error for 

economic growth within a particular region generally exceeds that of the nation as a whole. 
32 

Section II.A above. However during scarcity conditions, when system demand plus operating 

reserves requirements approach or exceed available supply, system operators must implement a 

series of emergency procedures to maintain system stability. One critical aspect of our modeling 

exercise is simulating: (a) what emergency procedures are implemented at each level of scarcity 

including the most extreme cases when load must be shed as well as somewhat less severe 

scarcity events; (b) what system costs are imposed under each type of emergency event; and (c) 

what market prices would prevail during these events. 

For the purposes of our simulations, we assume that prices will always be set at marginal system 

cost, including during scarcity events when emergency procedures must be implemented. We 

also assume that non-emergency dispatch and emergency procedures will be implemented in 

order of ascending marginal system costs, as described in Table 4. This series of emergency 

procedures is generally consistent with the emergency actions implemented by many system 

operators, but is not intended to exactly reflect any particular system.71 

Generally, the marginal system costs and resulting market prices represented in Table 4 increase 

with the severity of a shortage event up to the maximum energy price at the average customer 

value of lost load.72 In our Base Case, we assume that the value of lost load is $7,500/MWh, 

representing the approximate value that the average customer places on service interruptions 

during load shed events. Customers’ VOLL is a highly uncertain parameter that has been 

estimated over a wide range. Most academic and industry studies estimate residential customers’ 

VOLL at below $5,000/MWh, while industrial and commercial customers’ VOLL can exceed 

$15,000/MWh on average. To demonstrate the impact of this assumption, we analyze the 

sensitivity of our modeling results to this parameter in Section III.D. 

We assume that each system is also willing to implement some (but not all) of these procedures 

to benefit neighbors when those systems are experiencing higher-cost emergency events. Calling 

Emergency DR is the most costly emergency procedure that each system will implement to 

maintain exports, but exports will be curtailed if any costlier emergency procedures would be 

required. In other words, we assume that system operators will not be willing to dispatch 

emergency hydro, endure operating reserves shortages, implement voltage control, or shed 

internal load for the benefit of neighboring systems. 

 

71 For example see AESO (2012) or PJM (2013d). 

72 A rich literature exists demonstrating that VOLL is the efficient energy market price to impose in extreme 

shortage events during which load must be shed. For additional discussion, see Section IV.A.2 below. 
33 

Table 4 

Marginal Costs and Prices by Resource Type and Emergency Procedure 

Marginal Resource or 

Emergency Procedure

Description 

Generation 

$0 to $460/MWh 

During most hours, marginal system cost is the marginal dispatch cost 

of generation, ranging from $0/MWh (for wind and solar) to 

approximately $460 (for inefficient oil peakers), see Section II.C.2. 

Economic DR 

$100 to $1,000/MWh 

Economic DR has a marginal cost of $100-$1,000/MWh in the Base 

Case, or up to the price cap in a high DR case, see Section II.C.4. 

Emergency Generation 

$500/MWh 

We assume that all resources have an “economic maximum” capacity 

rating used under normal conditions, and an “emergency maximum” 

rating that is 1% higher.73 Achieving this higher emergency output 

level imposes substantial maintenance costs, assumed at $500/MWh. 

Emergency DR 

$2,000/MWh 

Emergency DR is available for dispatch through the supply stack 

similarly to generation resources at a $2,000/MWh price, although 

these resources are limited to a total of 100 call hours per year as 

explained in Section II.C.4. 

Emergency Hydro 

$3,000/MWh 

Emergency hydro is peaking resource that is costly and use-limited, 

with the costs being driven by the opportunity cost of not saving these 

resources for another time as well as environmental and societal costs 

imposed if reservoir levels are depleted. 

Operating Reserves Shortages 

$3,000 to $7,000/MWh 

Depleting operating reserves imposes marginal system costs by 

exposing the market to greater contingency risks. We model marginal 

system cost consistent with the administrative scarcity pricing curve 

shown in Figure 10 below. 

Voltage Reduction 

$7,000/MWh 

We assume that load can be reduced by 1% if implementing a systemwide voltage reduction at a marginal cost of $7,000/MWh.74 

Firm Load Shedding 

$7,500/MWh 

The maximum price and marginal system cost is at the assumed VOLL 

of $7,500/MWh, based on a mid-range estimate of customer-specific 

VOLL, assuming that load-shed events are more focused on locations 

with non-critical loads and more residential (rather than commercialindustrial) areas of the system . 

 

73 Distinguishing between generators’ economic and emergency maximum ratings is common, with the 

emergency maximum output being dispatched only during scarcity conditions. For example see AESO 

(2012), p. 6; PJM (2013d), p. 23. 

74 The costs imposed by this type of voltage curtailment have not been widely studied, although the costs of 

different types of voltage sag or power quality events have been evaluated in some studies. We assume a 

cost slightly below the VOLL, consistent with the economics implied by the fact that voltage reductions, if 

implemented at all, are typically invoked only after all other procedures as a last resort to avoiding firm 

load shed, for example see PJM (2013d), p. 14; Sullivan, et al. (2009); CEER (2010). 
34 

As explained in Table 4, most resources are functionally modeled as part of the supply stack, 

including resources such as economic DR, emergency generation, emergency DR, and 

emergency hydro. These resources are always available for dispatch, but they are available only 

at high strike prices and so are dispatched only under scarcity conditions. However, for the most 

extreme scarcity events when other emergency procedures must be implemented, prices are set 

according to the administrative scarcity pricing function described in Figure 10. The figure 

shows that the administrative price would be set as a function of the hourly supply cushion, equal 

to the total quantity of non-emergency and emergency resources available in the supply stack 

minus the hourly load. 

Any time the hourly supply cushion (hourly supply stack minus load) drops below a 5% 

operating reserves target, the system operator will have to implement an emergency measure to 

maintain supply-demand balance. First, the system will endure an operating reserves shortage 

relative to the 5% target maintained under normal system conditions. Simulated prices increase 

as the size of the operating reserves shortage increases, according to the specified administrative 

scarcity function. However, under no circumstances can operating reserves be allowed to drop 

below the 2% minimum level that must be maintained for system security even during load shed 

events. This downward-sloping portion of the administrative scarcity pricing function operates 

similarly to the “operating reserves demand curves” and equivalent mechanisms implemented by 

several RTOs over recent years.75 We assume that this administrative scarcity pricing curve 

reflects the marginal system cost imposed by operating reserve shortages. These marginal 

system costs can be quite high because depleting operating reserves increases the risk of load 

shed events and even cascading outages that could be triggered by generation or transmission 

contingencies or sudden changes in load or wind generation.76 

Once operating reserves are at the minimum level of 2%, the system operator will implement: (a) 

a system-wide voltage reduction that is assumed to achieve a 1% reduction in load at a marginal 

system cost of $7,000/MWh; and finally, as a last resort, (b) curtailments of firm load at a 

marginal system cost of $7,500/MWh.77 

 

75 Examples of RTOs who implemented administrative scarcity pricing include MISO, PJM, and CAISO. 

76 For a more comprehensive discussion of how a system operator could estimate these marginal system 

costs for setting an operating reserves demand curve, see Newell, et al. (2012). Section V.A.3. 

77 Note that load must be shed when the supply cushion drops below 1% above hourly load. This is because: 

(a) actual load (after voltage reduction) will be hourly load minus 1%; and (b) the 2% operating reserves 

requirement must still be maintained. 
35 

Figure 10 

Administrative Scarcity Pricing Function 

Notes: 

 Horizontal axis of the administrative pricing curve is calculated as a percent of peak load 

and does not vary with system conditions. 

F. MARGINAL RESOURCE COST AND PERFORMANCE

To examine economic and reliability outcomes over a range of reserve margins, we must 

increase or decrease installed capacity of a certain resource type. In our Base Case, we assume 

that the marginal technology is a natural-gas-fired combustion turbine (CT). We also examine 

an alternative case in which the marginal technology is a gas combined-cycle (CC) plant.78 The 

incremental capital and fixed costs required to build and maintain an additional resource is 

measured as the levelized cost of new entry (CONE), and includes both: (a) annual fixed 

operations and maintenance (FOM) costs; and (b) the annual levelized capital costs of 

constructing a new plant. Table 5 summarizes our assumed CONE and heatrates for CT and CC 

plants, which are approximately consistent with current PJM parameters.79 We examine the 

sensitivity of our results to our assumed CONE value in Section IV.B.3. 

 

78 See Section III.B.3. 

79 Actual plant costs could be anywhere over a substantial range. These reported costs are within the range 

of CONE values we estimated for different locations within PJM on a level-nominal basis for CT and CC 

plants with a 2015 online date. For a more precise report of construction costs and plant performance for a 

specific technology, see Spees, et al. (2011), pp. 2–3, 18. 
36 

Table 5 

Assumed Cost and Performance of New Gas CT and CC Plants 

CONE 

($/kW-yr) 

Heatrate 

(btu/kWh) 

Combustion Turbine $120 10,500 

Combined Cycle $150 7,000 

Notes: 

Approximately consistent with current PJM parameters, see 

Spees, et al. (2011), pp. 2-3, 18. 

G. BASE CASE AND ALTERNATIVE SIMULATION CASES

To explore the reliability and economic implications of varying reserve margins, system 

configuration, and other study assumptions, we simulate the interconnected illustrative power 

system for a Base Case and test the sensitivity of our results to various assumptions. We also 

implement a number of Change Case simulations designed to evaluate the impact of system 

characteristics as summarized in Table 6 below. As shown, the alternative simulations test the 

impacts of: (1) lowering the energy market price cap; (2) assuming a smaller system size (with or 

without decreasing intertie capability); (3) increasing or decreasing the availability of neighbor 

assistance; (4) assuming that a CC rather than a CT is the marginal technology; (5) increasing 

wind penetration; and (6) increasing DR penetration. 
37 

Table 6 

Summary of Simulation Cases

Simulation Case Simulation Description 

Base Case - System assumptions as defined in Section II. 

- Vary Study RTO margin from 7% to 17% by adding gas CTs 

Lower Price Caps

 $1,000 Price Cap Case - Impose $1,000/MWh energy price cap in Study RTO and Neighbors 

$3,000 Price Cap Case - Impose $3,000/MWh energy price cap in Study RTO and Neighbors 

Smaller System Size

 40% Size Case - Study RTO size 40% of Base Case (20,000 MW peak load) 

 40% Size and Transmission - Reduce Study RTO size to 40% of Base Case (20,000 MW peak load) 

- Reduce interties to 40% of Base Case (4,000 MW) 

Neighbor Assistance

Long Neighbors Case - Increase Neighbors’ reserve margins from 15% to 20% 

 50% Transmission Case - Reduce Study RTO interties to 50% of Base Case (5,500 MW) 

 Island Case - Eliminate Study RTO interconnections with neighbors 

Marginal CC Case - Add gas CCs to increase the Study RTO reserve margin from 7% to 17%

Wind Penetration Case - Increase wind penetration up from 5% of peak load (at a 12% Study RTO 

reserve margin) up to 30% of peak load 

DR Penetration 

 Emergency DR Penetration - Fix Study RTO reserve margin at 15% 

- Vary DR penetration from 0% to 20% of peak load by displacing CTs 

with Emergency DR (100 hours, $2,000/MWh strike price) 

 Economic DR Penetration - Fix Study RTO reserve margin at 15% 

- Vary DR penetration from 0% to 20% of peak load by displacing CTs 

with Economic DR (unlimited call hours, strike prices from $0 to cap) 

 
38 

III. SIMULATED RELIABILITY LEVELS AND SYSTEM COST RESULTS 

In this section, we report the results of a series of economic and reliability simulations for a 

hypothetical RTO with realistic attributes. We do not attempt to exactly replicate any particular 

real-world power system, but instead evaluate base and sensitivity case simulations that are 

generally illustrative of the economics of resource adequacy. 

We first estimate the planning reserve margin needed to achieve different physical reliability 

standards such as 0.1 LOLE, 2.4 LOLH, and 0.001% Normalized EUE. We then compare those 

results to alternative standards based on economic criteria such as cost minimization and risk 

exposure. Finally, we evaluate the reliability and economic impacts of making different 

assumptions about system conditions, resource attributes, and transmission interconnection 

levels. 

A. BASE CASE ECONOMIC AND RELIABILITY RESULTS

We present here the primary reliability and economic results from our Base Case analysis by: (a) 

comparing the reserve margins needed to achieve different reliability-based resource adequacy 

standards; (b) estimating the planning reserve margins that would minimize total system costs on 

a risk-neutral basis, defined from either a cost-of-service or a societal perspective; (c) discussing 

the risk mitigation benefits achieved by increasing the target planning reserve margin; and (d) 

evaluating the sensitivity of these economic results to study assumptions. 

1. Reserve Margins Needed to Achieve Physical Reliability Standards 

As explained in Section I.B.2 above, resource adequacy-related system reliability increases with 

reserve margin.80 As a result, the frequency of outage events decreases at higher planning 

reserve margins. Figure 11 shows this on an installed capacity (ICAP) basis for our Study 

RTO.81 The chart documents that, for our study RTO, the 1-in-10 standard translates to a 15.2% 

planning reserve margin target when defined as 0.1 LOLE, using a probability-weighted average 

of outage events across all scenario runs.82 The figure also shows that even under identical 

 

80 Recall that resource adequacy (i.e., the extent to which physical supply exceeds anticipated load) accounts 

for only a small portion of end-use customer load-shed events, the majority of which are caused by smaller 

failures on the distribution system and to a lesser extent on the transmission system. 

81 In this figure and the entirety of this study, we calculate the reserve margin on an ICAP basis, assuming a 

capacity value 100% of nameplate capacity for all resources, including both economic and reliability DR, 

with the exception of renewable generation, which we assign capacity value of 15% of nameplate for wind 

and 25% for solar. We do not assign any capacity value to voltage reductions when calculating the reserve 

margin (to be consistent with most others’ definitions), although we do not ignore the availability of 

voltage reductions for avoiding load shedding. 

82 Note that not all of the 9,600 runs that we simulate under each reserve margin are equally likely. We 

assume that any one of the 32 weather years and 50 draws on unit performance are equally likely, but the 6 

different economic forecast error points are not equally likely with the more extreme over- or underforecasts being less likely as explained in Section II.D.2 above. We account for these different likelihoods 

Continued on next page 
39 

modeling and system assumptions, the target planning reserve margin ranges from 15.2% to 

19.2%, depending on how “reliability events” are defined. In the remainder of this report, we 

treat events as contributing to LOLE only if firm load is shed, although some system operators 

define reliability “events” more broadly as discussed in Section I.C above. For example, the 

reserve margin requirement for 0.1 LOLE would be 16.2% if voltage reductions were considered 

reliability events or 19.2% if any depletion of operating reserves constituted an event. 

Figure 11 

Planning Reserve Margins for 0.1 LOLE with Different “Event” Definitions 

Notes: 

Study RTO in Base Case simulation results. 

The target reserve margin also depends on which type of physical reliability standard we impose, 

even if one consistently defines reliability events as load-shed events. Figure 12 shows (from 

left to right) a planning reserve margin requirement of: (a) 15.2% when the 1-in-10 standard is 

defined as 0.1 LOLE (as it is in most U.S. regions); (b) 8.2% if the 1-in-10 standard is defined as 

2.4 LOLH; or (c) 9.6% under the 0.001% Normalized EUE standard that is used in some 

international markets.83 

This analysis shows that a 0.1 LOLE standard is a much more stringent reliability standard than 

2.4 LOLH for the Study RTO, despite the fact that these two standards are both commonly 

referred to as “1 day in 10 years.” The magnitude of the difference between the two standards 

 

Continued from previous page 

when reporting average and percentile outcomes, although for simplicity we usually refer only to 

“average” results rather than “probability-weighted average” results. 

83 See additional discussion of these alternative reliability standards in Sections I.B.1and I.C above. 
40 

will not be the same in every system because it is driven by the average duration of the firm load 

shed event. For the Study RTO, the average duration of a single firm load shed event is typically 

only a small number of hours, which means accumulating 24 hours over ten years requires a 

number of individual outage events. 

Figure 12 

Planning Reserve Margins Required to Meet Different Physical Reliability Standards 

Notes: 

Study RTO Base Case simulation results. 

The smooth and well-behaved appearance of these physical reliability metrics masks the 

substantial sensitivity to the underlying simulation assumptions and particularly to assumptions 

affecting extreme events. Although most reliability simulations consider thousands of scenarios, 

the value of the reliability metrics is usually driven by only a small number of extreme outage, 

weather, and load forecasting error scenarios, as illustrated in Figure 13. The chart shows the 

EUE and LOLH observed at a 10% reserve margin by month of year (on the left), and 

cumulatively for all 9,600 simulated years (on the right). As in all summer-peaking systems, our 

hypothetical RTO realizes nearly all outage events during the summer peak months. The 

importance of extreme events is highlighted even more by the right-hand chart, which shows that 

45% of all simulated years have no outages, while the single worst simulation shows 68 load 

shed hours in the year. While only 10% of years exceed the 2.4 LOLH threshold, the 

probability-weighted average over all cases is 1.4 LOLH. 
41 

Figure 13 

Distribution of Loss of Load Hours at 12% Planning Reserve Margin 

Across Months (Left) and Across Simulation Years (Right) 

Notes: 

Distribution of realized EUE and LOLH in the Study RTO Base Case at 10% planning reserve margin. 

2. Minimizing Cost from a Risk-Neutral, Cost-of-Service Perspective 

As discussed in Section I.B.3 above, utilities and regulators in some regions use economic rather 

than reliability criteria to determine target reserve margins. Broadly speaking, these economic 

analyses aim to minimize total system costs by determining the “economically optimal” reserve 

margin. Such an economic analysis weighs the tradeoff between: (a) increasing capital and fixed 

costs associated with adding capacity to the system; and (b) decreasing reliability-related costs 

incurred during scarcity events, such as interrupting load, dispatching high-cost resources, 

procuring high-cost imports, and so on. 

Accurately modeling or even defining some of these reliability-related costs is a challenging 

exercise. The exercise is further complicated by the fact that the cost-minimizing reserve margin 

is different depending upon the perspective from which costs are to be minimized. Some of the 

important questions to answer are: 

 Is the objective to minimize societal costs, including all fixed and variable costs, 

regardless of whether those costs will be passed along to customers? Or, alternately, to 

minimize customer costs after evaluating how various costs will ultimately impact 

customer in their bills and realized outage rates? 

 Should one consider only costs within the study region or also the costs imposed on 

external regions by relying on imports during scarcity events? 
42 

 If minimizing customer costs is the objective, is it from the perspective of a cost-ofservice-regulated utility or the cost of serving customers in a competitive retail market?84 

Further, how do the costs of imports and revenues from exports affect customer costs 

within the Study RTO? 

 Are costs to be minimized on a risk-neutral basis, which assumes that customers and 

regulators only care about the lowest average costs but not the proportion of costs 

introduced by extreme events? Or is it appropriate to assume some risk aversion and 

therefore ascribe an insurance value to avoiding high-cost events? 

Depending on the perspective applied to a particular economic reliability study, choices implicit 

in each of these questions would yield a different planning reserve margin. For the purpose of 

our study, we first evaluate the economics of resource adequacy from a risk-neutral cost-ofservice perspective. In this context, the objective is to minimize the expected cost of service 

within the Study RTO while ignoring any economic or reliability costs imposed on neighboring 

regions. We then examine the same reliability and economic questions from a societal cost 

perspective in the following Section III.A.3. Finally, Section III.A.4 explores how risk 

mitigation objectives could affect the planning reserve margin target, because higher reserve 

margins will substantially reduce the magnitude and likelihood of extreme events. 

To estimate the risk-neutral, “economically optimal” reserve margin, we simulate realized 

reliability costs over a range of reserve margins as summarized in Figure 14. We vary the Study 

RTO planning reserve margins by adding or subtracting CTs as the marginal resource.85 The 

Study RTO’s total reliability-related costs for the Base Case, defined from this cost-of-service 

perspective, include: 

Marginal Unit Carrying Costs – Fixed costs and annualized capital costs incurred by adding 

incremental capacity, assuming that the marginal resource is a CT with a CONE of 

$120/kW-year.86 We do not report fixed costs associated with the existing generation 

fleet because these costs do not vary with reserve margin. 

Marginal Unit Production Costs – Production costs associated with dispatching the added 

new CTs. These are the incremental production costs incurred to displace higher-cost 

existing resources, higher-cost imports, and higher-cost emergency procedures. 

 

84 In a competitive retail market environment, adopting the objective of minimizing customer costs may be 

inappropriate in some circumstances, particularly where doing so would tacitly condone suppression of 

market prices. Note, however, that minimizing customer costs and minimizing societal costs are 

equivalent under the idealized case in which all internal and external markets are perfectly competitive, 

efficiently designed, and in a long-term equilibrium. 

85 We make alternative marginal resource assumptions in Section III.B.3 below, where we examine the 

impacts of adding or subtracting CCs instead of CTs. 

86 As explained in Section II.F above, this capital cost is within the range of level-nominal CONE values 

estimated for different regions in PJM for a CT with an online date in 2015, see Spees, et al. (2011), pp. 

2-3 and 18. Section IV.B.3 below shows the sensitivity of the economic reserve margin to a range of 

CONE assumptions. 
43 

Net Import Costs – Total costs of purchasing imports net of any revenues from exporting 

power.87 This treatment of imports and exports is consistent with how cost-of-service 

regulated utilities will typically evaluate net import costs.88 It is also consistent with the 

Adjusted Production Costs (APC) metric commonly used within RTO systems for 

production cost analyses and transmission benefit-cost studies.89 Note that this approach 

to tabulating import costs results in incorporating price suppression as a benefit, because 

displacing imports through new CTs during scarcity conditions will not only reduce the 

quantity of imports but will also reduce the price paid for all other imports. The pricesuppression component of reduced import costs would not be considered a benefit if 

evaluated from a broader “societal” perspective as discussed further below.

Peaker Production Costs – Production costs associated with dispatching peaking resources 

that have higher operating costs than the marginal CT plants added to the system. Note 

that we would arrive at the same study results if we considered total system production 

costs, but the much larger total costs would make visual representation of the results 

more difficult.90

Emergency Generation – Emergency generation is the incremental cost associated with 

dispatching generating resources at very high output levels that exceed their typical 

economic operating range. Pushing units to generate at such high levels can impose 

additional wear and tear on units, and consequently impose incremental maintenance 

costs. We assume that all generating units can produce incremental output at their 

“emergency maximum” rating, which is assumed to exceed their “economic maximum” 

rating by 1% at a marginal cost of $500/MWh, as explained in Section II.E above. 

 

87 The realized price of purchasing imports and selling exports also depends on who owns the transmission 

rights needed to import and export power. Whoever owns the rights to send power across a particular 

transmission path will likely be able to capture the value of that transmission, buy power in the lower-price 

market, schedule the power to flow across that path, and sell into the higher-price market. For example, if 

exporters generally own the transmission rights, then they obtains the market price at the buyer’s location 

(i.e., the higher-cost market). Alternately, if importers generally own the transmission rights, then the 

importer can buy the power at the seller’s location (i.e., the lower-cost market). For the purpose of our 

simulations, we assume that importers and exporters each own 50% of transmission rights needed for 

import and export transactions. This means that imports and exports are transacted by the Study RTO 

utility at market price that reflects the average of the market prices at the source and delivery point. 

However, as we note further below, changing this assumption can have a substantial impact on total 

customer reliability costs and, thus, a significant effect on economic planning reserve margins. 

88 This approach implies a number of assumptions, including that the net export revenues will be credited 

against the Study RTO cost of service and, thus, flow back to customers. This would not necessarily be 

the case if, for example, some export revenues were earned by independent power producers (IPPs) rather 

than a regulated utility. 

89 For a summary of the APC metric and its limitations see SPP (2012a), Section 5. 

90 In other words, adding CTs has a negligible impact on the dispatch profile of baseload units and so 

baseload-related production costs will remain unchanged across different reserve margin levels. However, 

baseload production costs are orders of magnitude greater than the reliability-related costs we report, 

making it difficult to report these results graphically. We do summarize total costs, including baseloadrelated costs, when comparing total customer costs in Section IV.D.2 below. 
44 

Emergency Hydro – Reliability costs associated with emergency hydro dispatch. The 

marginal system cost of dispatching these resources is difficult to quantify because it 

depends on the opportunity cost of not saving the resources for later emergencies as well 

as any societal costs associated with reduced reservoir levels. We assume emergency 

hydro has a marginal system cost and dispatch price of $3,000/MWh.91

Economic and Emergency DR – Load curtailment costs associated with two types of 

demand response resources: (1) Economic DR, which offers into the energy market in all 

hours at costs ranging from $100 to $1,000/MWh and which may be dispatched an 

unlimited number of times; and (2) Emergency DR, which does not offer into the energy 

market but can be dispatched a limited number of times during emergencies at a cost 

above $2,000/MWh. We assume that these dispatch costs reflect the payments made to 

these DR resources and assume that it also reflects the net cost that service interruptions 

impose on DR suppliers, as explained in Section II.C.4 above. 

Operating Reserves Shortages – Reliability costs imposed by temporarily operating the 

system with depleted operating reserves during scarcity events. We do not attempt to 

calculate these costs explicitly because doing so is a complex exercise in evaluating the 

incremental contingency risks endured at each reserve level. Instead, we calculate these 

costs implicitly using the administrative “operating reserves demand curve” covering a 

$3,000-$7,000/MWh range of prices as discussed in Section II.E above. This 

administrative pricing curve is similar to the administrative scarcity pricing mechanisms 

recently implemented by several U.S. RTOs.92 If such an operating reserves demand 

curve is designed efficiently, it will reflect the marginal system costs incurred when 

depleting operating reserves. 

Voltage Reductions – System costs imposed by implementing a system-wide voltage 

reduction as the last resort before shedding load. These costs could include both 

equipment damage caused by the voltage reduction and lost economic value to end users 

(e.g., due to motors running at a lower speed). We assume voltage reductions incur a 

system cost of $7,000/MWh, or just below the assumed VOLL as explained in Section 

II.E above. 

Load Shedding – Service interruption costs imposed at a VOLL of $7,500/MWh for the 

average customer typically curtailed as explained in Section II.E above. 

Figure 14 summarizes the annual averages of total Study RTO-wide reliability costs over a range 

of planning reserve margins. For each reserve margin level, we show the weighted-average 

reliability costs across all 9,600 annual simulations. At the lowest reserve margin shown in the 

chart, average annual reliability costs are high and are primarily driven by the high cost of 

imports during scarcity conditions. Other factors such as the value of lost load, DR interruption 

costs, and peaker dispatch costs impose relatively modest reliability costs even at low reserve 

margins. As planning reserve margins are increased, total reliability costs drop more quickly 

than the increases in capital and production cost associated with adding CTs. As a result, total 

 

91 See additional discussion in Section II.E above. 

92 For additional discussion of how administrative demand curves for operating reserves are used to set 

prices and reflect marginal system costs during scarcity events, see Newell, et al. (2012), Section V.A.2. 
45 

costs drop as the reserve margin increases until the (risk-neutral) “optimal” quantity of capacity 

has been added at a planning reserve margin of 10.3%. After crossing this minimum cost point, 

the capital costs of adding more capacity exceed the benefits from reduced reliability costs, and 

so total costs increase. 

Note that at the 10.3% reserve margin, the Study RTO has a planning reserve margin that is 

substantially below the 15% reserve margin we assumed in all of the neighboring regions. This 

means that the neighbors are providing substantial assistance to the Study RTO, allowing it to 

reduce both costs and reliability events. This also means that the Study RTO is relying on its 

neighbors in a way that increases those regions’ reliability-related costs. In other words, the 

Study RTO is able to minimize its costs at the 10.3% reserve margin only as long as the 

neighbors are indirectly “subsidizing” resource adequacy for the Study RTO. If the neighbors 

were to reduce their own reserve margins below 15%, then the Study RTO’s cost-minimizing 

reserve margin would increase.93

Another prominent feature of the chart is that net import costs represent a large portion of total 

reliability-related costs and these costs drop more quickly than other costs. This is because 

adding capacity within the Study RTO not only allows one to displace higher-cost imports with 

lower-cost internal supply, but also reduces the market price at which all imports must be 

purchased. In other words, we incorporate price suppression as a “benefit” from this cost-ofservice perspective, even though it is not a benefit from a broader societal perspective as 

discussed further in the following section. Further, this benefit is tied to the assumption that 

external regions would maintain a 15% planning reserve margin. The benefit consequently 

would not be sustained if the external regions’ generation investment decreased.94

Additionally, the size of net import costs is greatly influenced by our assumption about who 

owns the transmission rights. We assume that customers in the Study RTO own the rights to half 

of all import and export capability on each of its interties, meaning that they will earn the price 

differential between the importing and exporting markets in all hours. However, as a general 

rule any importer, exporter, power marketer, or transmission owner may own some portion of 

these transmission rights. If we make alternative assumptions about transmission ownership, the 

optimal cost-of-service reserve margin changes from 10.3% in the base case, down to 8.3% with 

an importer-owns assumption and up to 12.3% with an exporter-owns assumption.95

 

93 See additional discussion of these impacts in Sections III.A.3 and III.B.2. 

94 Note that increasing the Study RTO reserve margin will reduce prices and reliability events in neighboring 

regions, both of which may result in the external region reducing its own reserve margin. 

95 Note that as a net importer, the Study RTO achieves greater “price suppression” benefits from building 

capacity under the exporter-owns assumption than under the importer-owns assumption (because in the 

importer-owns case the price suppression “benefit” is offset by a similarly-sized offsetting price 

suppression “cost” imposed by the reduced value of its transmission rights). This is why the importerowns assumption results in a lower optimal reserve margin while the exporter-owns assumption results in 

a higher optimal reserve margin. 
46 

The total cost curve shown in Figure 14 has a shape similar to what we have observed in many 

different systems in our own studies as well as in other value of service studies.96 The curve is 

relatively flat near the minimum average cost point, indicating that expected total costs do not 

vary substantially between reserve margins of 8% to 14%. However, as we discuss further 

below, the lower end of that minimum cost range is associated with much more uncertainty in 

realized annual reliability costs and a much larger number of severe, high-cost reliability events. 

While the weighted average of annual costs for the Study RTO is very similar over the 8% to 

14% planning reserve margin range, the higher reserve margin would avoid a majority of the 

high-impact, high-cost events that would be experienced at the 8% planning reserve margin. In 

other words, at the 14% reserve margin, a greater proportion of total annual costs are associated 

with added CTs (which have little uncertainty), and a smaller proportion of the average annual 

costs are from low-probability but high-cost reliability events.

Figure 14 

Study RTO Reliability Costs as a Function of Study RTO Reserve Margin 

(Risk-Neutral, Cost-of-Service Perspective) 

Notes: 

Study RTO reliability-related costs for Base Case simulations. 

 Expected annual costs are minimized from a cost-of-service perspective at a planning reserve margin of 10.3%. 

 

96 For example, see Keane, et al. (1992) pp. 824, 826; Poland (1988), p.21; Munasinghe (1988), pp. 5-7 and 

12-13; and Carden, Pfeifenberger, and Wintermantel (2011). 
47 

3. Minimizing Cost from a Risk-Neutral, Societal Perspective 

In this section, we estimate the risk-neutral, cost-minimizing reserve margin from an overall 

societal rather than a cost-of-service perspective. Under the cost-of-service perspective, we took 

a narrow geographic view and evaluated end-user costs for only the Study RTO without 

considering any reliability-related costs imposed on neighboring regions. In this section, we 

evaluate total reliability-related costs incurred across all four regions, regardless of whether those 

costs are incurred within the Study RTO or one of the neighboring regions. While any one 

individual utility may rationally apply the objective of minimizing customer costs from a singlesystem perspective, the broader societal perspective is more applicable in many contexts. For 

example, this broader geographic view of societal costs is more relevant for a large RTO or 

jurisdictional authority whose responsibility may span many utility service areas or several 

states. 

We simulate reliability costs over the same range of reserve margins examined in the previous 

section, but estimate total system costs across all four study regions. The following cost 

components are identical from both the societal and cost-of-service perspectives as discussed 

above: 

 Marginal Unit Carrying Costs 

 Marginal Unit Production Costs 

 Peaker Production Costs 

 Emergency Generation Costs 

 Emergency Hydro Dispatch Costs 

 Economic and Emergency DR Interruption Costs 

 Operating Reserves Shortage Costs 

 Voltage Reduction Costs 

 Load Shed Costs 

However, the remaining components differ from a societal perspective. While the cost-ofservice perspective assesses the costs of all imports (exports) at the purchase (or sales) price, this 

accounting does not apply to a multi-regional societal cost perspective. We thus removed the 

Net Import Cost category and added two cost categories to account for reliability costs incurred 

in neighboring regions. Note, however, that in this study marginal generation capital and 

production costs are only incurred in the Study RTO; no generation is being added in the 

neighboring regions. The reliability costs incurred in neighboring regions are calculated in the 

same way that they are calculated for the Study RTO, but are reported separately as: 

Neighbors’ Emergency Event Costs – Neighbors’ emergency event costs include most of the 

cost categories evaluated, including: emergency generation, emergency hydro, economic 

DR, emergency DR, operating reserves shortages, voltage reductions, and load shedding. 

We assume that the neighboring regions have the same reliability procedures and 

emergency costs as the Study RTO. Note that adding additional capacity within the 

Study RTO will reduce the frequency of each of these types of emergency events not 

only internally, but also externally. 
48 

Neighbors’ CT and Peaker Production Costs – Neighbors’ CT and peaker production costs 

mirror the Peaker Production Cost category of the Study RTO.97 Adding capacity in the 

Study RTO will reduce these costs in neighboring regions by allowing the Neighbor to 

import more during some occasions when high-cost peakers would otherwise need to be 

dispatched.

Figure 15 summarizes these total system reliability costs from a societal perspective over a range 

of reserve margins based on all 9,600 annual simulations for each reserve margin level, with 

reserve margins reported for both the Study RTO as well as the combined four-system region. 

As in the previous section, these costs represent the weighted average across all simulated 

scenarios. The total represents the combined total of the four-system societal costs on a riskneutral basis. All of the colored slices are identical in size and meaning as those reported in the 

previous cost-of-service section, except for the two blue slices representing cost changes in the 

neighboring regions. In other words, we have eliminated net import costs (light blue slice) from 

Figure 14 above, but added the neighboring regions’ reliability costs (light and medium blue 

slices) to Figure 15 below. In addition, we report cost results along the x-axis not only as a 

function of the planning reserve margin in the Study RTO, but also as a function of the reserve 

margin for the combined four-system region.98 

As shown in Figure 15, from a total societal cost perspective, the optimal target reserve margin 

in the Study RTO (assuming the neighboring regions maintain a 15% reserve margin) is reduced 

from 10.3% under the cost-of-service perspective to 7.9% under a societal perspective. The 

primary reason for this difference is that, while customers within the Study RTO can benefit 

from import price reductions by increasing their own internal reserve margin, such price 

reductions are not a benefit from a societal-cost perspective. From a study RTO cost-of-service 

perspective, increasing the Study-RTO reserve margin to 10.3% is cost-effective because it 

reduces the net cost of imported power, both by reducing the amount of energy imported and by 

reducing the price paid for imported power. However, additional capacity above the 7.9% 

reserve margin is not beneficial from a societal-cost perspective because the Study-RTO benefit 

is outweighed by the combined impact on neighboring RTOs and the additional capital 

expenditure.99 

 

97 With the exception that this category for the external regions also includes costs from units that happen to 

have the exact same price as the marginal CT. Sometimes the marginal unit will displace imports from 

higher-cost resources, but sometimes it will also displace imports from equally-priced CTs, which is why 

we include these costs as well for the neighboring regions. 

98 Note that we add capacity only inside the Study RTO and not in any of the Outside Regions. The total 

reserve margin for the four systems combined changes only because capacity is being added in the Study 

RTO. The installed capacity in the neighboring regions is held constant at the original 15% reserve 

margin. 

99 In other words, “optimizing” the system from a Study RTO cost-of-service perspective aims to provide 

maximum benefit to customers (excluding suppliers impacts) in the Study RTO (excluding impacts on 

neighboring regions), which would generally introduce some deadweight loss relative to the “optimum” 

solution that one would achieve if planning across multiple systems from a societal perspective without 

regard to which supplier, customer, or region benefits most. 
49 

Figure 15 and Table 7 also show that the 7.9% Study RTO reserve margin, combined with 15% 

Neighbor reserve margins, is equivalent to an overall four-system reserve margin of: (a) 13.0% if 

measured against the non-coincident peak load, and (b) 16.5% if measured against the coincident 

peak load as reported in the figure. This 16.5% reserve margin is higher than the individual 

systems’ reserve margins because the neighbors’ 15% reserve margin and the Study RTO’s 7.9% 

target reserve margin are all reported relative to their individual systems’ non-coincident peak 

loads, the sum of which is higher than the combined four-system coincident peak load. 

Because the Study RTO’s optimal internal reserve margin is much lower than the neighbors’ 

assumed 15% reserve margins, this means that the Study RTO is relying more heavily on the 

neighboring regions during scarcity conditions, and that the neighboring regions are “supporting” 

resource adequacy in the Study RTO through the combination of their 15% reserve margin and 

intertie capabilities. 

Figure 15 

Reliability Costs in all Regions as a Function of Study RTO Reserve Margin 

(Risk-Neutral, Societal Perspective) 

Notes: 

Reliability-related costs for all four regions in the Base Case scenario. 

 Societal costs are minimized at reserve margins of 7.9% in the Study RTO (measured against Study RTO noncoincident peak load) or 16.5% across all four regions (measured against four-system coincident peak). 
50 

Table 7 

Coincident and Non-Coincident Reserve Margin Accounting 

At the Societally Optimal Reserve Margin 

A planning effort designed to optimize the location and size of resource additions across all four 

interconnected regions would likely result in decreasing the neighbors’ reserve margins while 

increasing the Study RTO reserve margin. However, the target reserve margins for the four 

regions that result in the lowest cost for the entire system will depend on a wide variety of 

factors. For example, some of the factors determining the cost effectiveness of increasing the 

reserve margin in a particular region would include: (a) whether a region is an importconstrained area like the Los Angeles Basin, New York City, or Eastern PJM; (b) whether the 

sub-region is highly dependent on limited-dispatch DR during peak periods, which would make 

adding CTs more valuable; (c) whether two sub-regions are well-interconnected with large load 

diversity, allowing both sub-regions to gain substantial resource adequacy benefits from the 

same capacity resource; and (d) whether adding new CTs in one region would allow for the 

retirement of aging resources with high fixed costs. 

4. Risk Mitigation Benefit of Increasing Reserve Margins 

The economic results shown in the previous sections assume risk neutrality with respect to the 

uncertainty and volatility of reliability-related costs. This allows us to compare total costs at 

different reserve margins in Figure 14 and Figure 15 simply as the probability-weighted average 

of annual reliability costs for all 9,600 simulated scenarios. However, there is substantial 

volatility around the average level of possible reliability cost outcomes. Most simulated years 

will have only very modest reliability costs, while a small number of years have very high 

reliability costs. These high-cost outcomes account for the majority of the weighted average 

annual costs shown as the individual bars in Figure 14 and Figure 15. 

Figure 16 summarizes this risk exposure by comparing the weighted average costs for different 

reserve margins (shown as the individual bars in Figure 14, from a cost-of-service perspective) to 

annual costs under the most costly possible outcomes, represented by the 85th, 90th, and 95th

percentile of annual reliability costs across all 9,600 simulated scenarios. The figure shows that 

the majority of all reliability-related costs are concentrated in the most expensive 15% of all 

simulation runs for each planning reserve margin. In other words, for 15% of possible annual 

outcomes, the annual reliability costs are at or above the 85th percentile line shown in the chart. 

This means that, while total average costs change by a relatively modest amount over a range of 

planning reserve margins, differences in planning reserve margins have a much larger impact on 

Generation Peak Load Reserve Margin

Non-Coincident Coincident Non-Coincident Coincident

(MW) (MW) (MW) (%) (%)

Study RTO 53,950 50,000 48,605 7.9% n/a

Neighbor 1 69,000 60,000 58,686 15% n/a

Neighbor 2 46,000 40,000 38,604 15% n/a

Neighbor 3 34,500 30,000 28,773 15% n/a

Total 203,450 180,000 174,668 13.0% 16.5%
51 

the uncertainty in reliability costs and the likelihood of high-cost outcomes than can be 

encountered in any particular year. 

Considering the much higher cost uncertainty exposure at lower reserve margins, many planners 

and policy makers may wish to set planning reserve margins above the risk-neutral economic 

optimum. In our sample system, even a several percentage point increase in the target reserve 

margin would only slightly increase the average annual costs, but substantially reduce the 

likelihood of experiencing very high-cost events. However, without evaluating the magnitude of 

the increase in average costs and decrease in extreme event costs, policy makers would not be in 

a position to determine whether a higher reserve margin is justified by its risk mitigation 

benefits. 

For example, our simulations show that the risk-neutral optimal planning reserve margin is 

10.3% for the Study RTO from a cost-of-service perspective. In comparison, the planning 

reserve margin would need to be 15.2% to achieve the 0.1 LOLE standard. As Figure 14 and 

Figure 16 show, the increase in average annual costs required to achieve the 15.2% planning 

reserve margin is relatively modest at approximately $90 million per year. However Figure 16 

also shows that this would reduce the annual costs incurred once in a decade (i.e., costs above the 

90th percentile) by at least $270 million per year, and the costs incurred once in 20 years (i.e., 

costs above the 95th percentile) by at least $630 million per year. In other words: (a) a riskneutral policy maker would not increase reserve margins above the 10.3% risk-neutral optimum 

because, by definition, the expected costs would exceed expected benefits; (b) a somewhat riskaverse policy maker might increase reserve margins slightly but possibly not enough to meet 0.1 

LOLE at 15.2% reserve margin where the costs exceed the benefits by a ratio of approximately 

three-to-one; and (c) a highly risk-averse policy maker might wish to meet or even exceed the 

15.2% reserve margin needed to meet 0.1 LOLE. 

In evaluating these cost and risk tradeoffs, policy makers should also consider that suppliers and 

customers will be able to hedge a significant portion of these risks through forward contracting. 

As we discuss further in Section IV, a large portion of the risk of encountering the high-cost 

outcomes illustrated in Figure 16 is related to weather uncertainty. Some of that risk exposure 

can likely be hedged cost effectively, including through seasonal forward contracts. For 

example, many suppliers and load serving entities (LSEs) in ERCOT hedge most of their 

summer price spike exposure through forward contracts on a seasonal basis.100 

 

100 See additional discussion and documentation in Section IV.A.3 below. 
52 

Figure 16 

Uncertainty Distribution of Annual Reliability-Related Costs 

 (Cost-of-Service Perspective) 

Notes: 

Study RTO reliability-related costs for Base Case from a cost-of-service perspective. 

5. Sensitivity to Forecast Error and Forward Planning Period 

As explained in Section II.D.2 above, uncertainty in economic growth and the consequential 

economic load forecasting uncertainty are key planning considerations. Uncertainty caused by 

economic forecasting error increases with the forward period. This is unlike annual weatherrelated uncertainty, which is generally assumed to be constant over time. Adopting a longer 

forward planning period therefore will result in higher load forecast uncertainty and, if resource 

additions cannot be modified on a shorter-term basis, a higher likelihood of reliability and 

scarcity events. This means that increasing the forward period in resource adequacy studies will 

also increase the reserve margin necessary to achieve any given reliability standard. 

In our Base Case analysis, we adopt a four-year forward planning period based on a conservative 

estimate of the approximate lead time needed to develop new resources.101 The lead time for 

 

101 Note that although the entire development timeline for most new plants is greater than four years, much of 

that development work, including siting and permitting, can be done without making major irreversible 

financial commitments. This means that the time needed for actual plant construction is most relevant 

when the resource investment decision is truly locked in. Developing and constructing a natural gas CC or 

CT takes approximately 3.25 and 2.8 years respectively, see Spees, et al. (2011), Appendices A.3 and B.3. 
53 

developing new resources is also the reason that PJM and ISO-NE’s capacity markets rely on a 

three-year forward period. In non-RTO markets and the integrated planning processes subject to 

state regulation, at least a portion of all planning decisions are often made on a schedule that 

looks more than three years forward.102 Note, however, that most systems also have substantial 

flexibility to adjust their total resource portfolio on a shorter-term basis, including even on a oneyear forward basis. For example, there is substantial capability to adjust DR commitments, 

revise import-export balance, adjust retirement or retrofit decisions, delay or accelerate plant 

development efforts, and invest in plant upgrades on a shorter-term basis. 

We examine the implications of revising the forward planning period on the reserve margin 

standard and total system costs by adopting alternative economic forecast error assumptions 

consistent with different forward periods as documented in Section II.D.2 above. Increasing the 

forward period increases the economic load forecast error, which consequently increases the 

planning reserve margin whether the reserve margin is reliability-based as shown in Figure 17, or 

economically-based as shown in Figure 18. 

As Figure 17 shows, reducing the forward planning period from four years to one year would 

reduce the 0.1 LOLE-based planning reserve margin by 1.7%, from 15.2% to 13.5%. This 

would, of course, require that sufficient short-term resources exist that could be mobilized on a 

12-month basis should economic growth prove greater than anticipated.103 As noted, such shortterm resources might include DR resources (assuming the market is not fully saturated), 

upgrades to existing plants (or new plants under construction), reactivations of mothballed 

plants, an increase in net import commitments, or accelerating in-service dates of new plants 

under development. 

 

102 For example, see the discussion of the Long-Term Resource Planning processes implemented by 

California’s investor owned utilities under the oversight of the state commission in Pfeifenberger, et al.

(2012). 

103 For example, assume that most resource investment decisions (e.g., for existing generation) were made on 

a longer term basis but the final 5% of resources were not procured until one year prior to delivery. 

However, the actual amount of incremental resources needed on a one-year forward basis may only be 3% 

or as high as 7% if economic growth were higher or lower than expected in prior planning exercises. 
54 

Figure 17 

Reliability-Based Resource Adequacy Standard vs. Forward Planning Period 

Notes: 

Study RTO reliability-based reserve margin vs. forward period. 

 Load forecast error varies with planning period, see Section II.D.2. 

Figure 18 similarly shows that increasing the forward period will increase the economically 

optimal planning reserve margin. The chart shows total reliability-related costs from a riskneutral cost-of-service perspective (as in Section III.A.2 above) under different forward periods. 

The Base Case assumption of a four-year forward period is shown as a blue line while the gray 

lines show costs ranging from a one-year to a ten-year forward planning period. Total costs 

increase substantially with a higher forward period because there is a much greater risk of highimpact events associated with under-forecasting economic load growth (and greater capital costs 

required to build sufficient capacity to avoid such events). However, if sufficient short-term 

resources exist to allow for a reduction of the forward planning period from four years to one 

year, then the risk-neutral optimal reserve margin would decrease by 1.3%, from 10.3% to 9.0%. 

This suggests that if a region has a reasonably high amount of additional short lead-time DR 

resources, it may be beneficial to adopt a forward procurement approach similar to that used in 

PJM, where most resources are procured three years forward but the remaining 2.5% of 

resources are procured on 2-year, 1-year, and short-term basis. This could reduce total 

procurement costs by reducing the procured reserve margin more often than not. However, it 

also increases the risk of under-procurement in a high load growth situation. If substantially 

more than 2.5% of resources are needed on a short-term basis, there may not be enough supply 

available to meet the need; or the only available resources may be high-cost DR that is not costcompetitive under ordinary circumstances. We further examine the implications of high DR 

penetration in Sections III.C and IV.C below. 
55 

Figure 18 

“Optimal” Reserve Margin as a Function of Forward-Planning Period 

(Risk-neutral, Cost-of-Service Perspective) 

Notes: 

Study RTO reliability-related costs for Base Case from a cost-of-service perspective. 

 Load forecast error varies with planning period as described in Section II.D.2. 

B. IMPACT OF SYSTEM CHARACTERISTICS

This section of our report summarizes the extent to which target reserve margins are affected by 

system characteristics. The specific sensitivities we explore are: (1) system size (40% Size Case 

and 40% Size and Transmission Case); (2) intertie size and neighbor assistance (Long Neighbors 

Case, 50% Transmission Case, and Island Case); (3) using CCs rather than CTs as the marginal 

capacity resource (Marginal CC Case); and (4) higher penetration levels of intermittent 

renewable resources (High Wind Penetration Case). 

1. Impact of System Size 

We first examine the implications of system size under two alternative cases, in which: (1) we 

reduce the RTO Study Region to 40% of its Base Case size (from a peak load of 50,000 MW to 

20,000 MW); and (2) we reduce the system to 40% of its Base Case size and also reduce all 

interconnections with neighboring regions to 40% of their Base Case size. In both of these cases 

we maintain the same resource mix, unit sizes, and external region characteristics as in the Base 

Case. 

Table 8 summarizes the target reserve margin results for these two change cases compared to the 

Base Case. The 40% Size Case shows that both the 1-in-10-based and the economic target 

reserve margins decrease for the Study RTO. The 0.1 LOLE target drops only slightly from 
56 

15.2% in the Base Case to 14.8% in the 40% Size Case, while the other reliability-based targets 

drop more substantially from 8.2% to less than 6% under the 2.4 LOLH standard and from 9.6% 

to 7.5% under the 0.001% Normalized EUE standard. The risk-neutral, cost-minimizing reserve 

margin decreases even further, from 10.3% (cost-of-service perspective) and 7.9% (societal 

perspective) to less than 6% under both perspectives. The reason for the lower planning reserve 

margin targets is that the smaller Study RTO is able to rely more heavily on neighbor assistance. 

This is the case because the smaller RTO requires a fewer imports during reliability events. In 

other words, the neighbors are able to provide proportionally greater assistance to the smaller 

Study RTO. 

If we also proportionally reduce the size of the interconnections with neighboring regions, then 

some of these neighbor-assistance benefits are lost to the smaller Study RTO as also shown in 

Table 8. The target reserve margin based on 0.1 LOLE is reduced from 15.2% to only 15.1% 

(compared to 14.8% when maintaining the Base Case size of the interconnections) while the 

0.001% Normalized EUE and 2.4 LOLH standards only increase to 8.1% and 6.9% respectively. 

The economic target reserve margins still remain at less 6% (outside our study range). The 

combination of these alternative simulation cases shows that: (a) a small system can rely more 

readily on neighbor assistance to reduce its planning reserve margin; and (b) the value of 

neighbor assistance is substantially greater for a well-interconnected small system. 

Table 8 

Study RTO Reserve Margin Targets with a Smaller System Size 

These lower target reserve margins for the smaller system may initially appear counter-intuitive 

or inconsistent with general industry experience. Historically, many small utilities maintained 

reserve margins in excess of 20% and were able to reduce these margins over time by merging, 

interconnecting with neighbors, developing reserve-sharing agreements, or joining larger RTO 

systems. This apparent disconnect with our results is primarily driven by the fact that traditional 

utilities were much smaller and more “islanded” than even the “small” Study RTO (20,000 MW) 

in our simulations. Traditionally, many utilities may have relied less on interties for reliability 

because many: (a) were not as well-interconnected with their neighbors; (b) would have faced 

more difficulty in obtaining neighbor assistance during peak events, particularly prior to the 

implementation of reserve sharing agreements and liquid wholesale power markets; and (c) may 

have been conservative in assessing the reliability value of their interties in the absence of firm 

power purchase commitments. 

In addition, the high reserve margin targets of many small utilities are driven by a single-largestcontingency analysis. For example, a small utility’s reliability standard may stipulate that the 

utility is able to meet peak load even if the single largest generator or transmission line were to 

Simulation Reliability-Based Risk-Neutral, Cost-Minimizing 

0.1 LOLE 2.4 LOLH 0.001% 

Normalized EUE

Cost-of-Service 

Perspective

Societal 

Perspective

Base Case (50,000 MW) 15.2% 8.2% 9.6% 10.3% 7.9%

40% RTO Size (20,000 MW) 14.8% <6% 7.5% <6% <6%

40% Size w/ 40% Intertie 15.1% 6.9% 8.1% <6% <6%
57 

fail. In a large system, such a contingency criterion typically does not affect the planning reserve 

margin. However, for a small utility serving only a few thousand MW of load, contingency 

considerations can substantially increase planning reserve requirements. We did not conduct any 

contingency analysis in this study, but even in the “small” 20,000 MW Study RTO Case, such 

contingencies are unlikely to affect planning reserve margins. Nevertheless, if we had treated the 

small system as a substantially more islanded region, then it would better reflect the historical 

experience of smaller utilities and would result a higher reserve margin requirement. 

2. Intertie Size and Neighbor Assistance 

Strong interties with neighboring regions provide both economic and physical reliability value 

during peaking conditions. Load and generation diversity mean that the most extreme scarcity 

conditions are unlikely to occur at the same time in neighboring markets. This inter-regional 

diversity means that the Study RTO will often be able to rely on emergency imports to avoid 

load shedding and improve system reliability. Interties create economic benefits as well, for 

example, by allowing the Study RTO to purchase imports from external CTs rather than 

dispatching higher-cost internal DR resources. 

We examine the value of interties and neighbor assistance for a number of cases, with the 

planning reserve margin results summarized in Table 9. The results are shown in order of 

increasing availability of neighbor assistance, including: (1) a “Long Neighbors Case” where the 

neighbors’ reserve margins are increased to 20% compared to 15% in the Base Case (and intertie 

capability equal to the Base Case); (2) the Base Case, where the neighbors have 15% reserve 

margins and the Study RTO has 11,000 MW of intertie capacity); (3) a “50% Transmission 

Case,” with interties at 50% relative to the Base Case (and neighbors’ reserve margins at 15%); 

and (4) an “Island Case” with no interties. 

The table shows that the interties offer substantial benefits from both a physical reliability and 

economic perspective. Maintaining the 0.1 LOLE requires increasing the internal reserve margin 

from 13.0%% in the Long Neighbors Case, to 15.2% in the Base Case, 15.8% in the 50% 

Transmission Case, and 18.5% in the Island Case. In terms of the economic benefits of neighbor 

assistance and interties, these cases show that the cost-minimizing reserve margin (from a riskneutral cost-of-service perspective) increases from 8% in the Long Neighbors Case, to 10.3% in 

the Base Case, 12.3% in the 50% Transmission Case, and 16% in the Island Case. 

Table 9 

Study RTO Reserve Margin Targets with Varying Availability of Neighbor Assistance 

Simulation Reliability-Based Risk-Neutral, Cost-Minimizing 

0.1 LOLE 2.4 LOLH 0.001% 

Normalized EUE

Cost-of-Service 

Perspective

Societal 

Perspective

Long Neighbors Case 13.0% <6% 7.0% 8.0% <6%

Base Case 15.2% 8.2% 9.6% 10.3% 7.9%

50% Transmission Case 15.8% 9.8% 10.0% 12.3% 10.5%

Island Case 18.5% 16.5% 15.8% 16.5% 16.5%
58 

That interties provide substantial economic value is also shown in Figure 19, which displays total 

system production and reliability-related costs from a risk-neutral, cost-of-service perspective for 

different planning reserve margins and interconnection assumption. Not counting the costs of 

the interconnections themselves, the figure shows that system costs increase as interconnections 

decrease for a given level of reserve margin. For example, at a 14% planning reserve margin for 

the Study RTO, reducing interties by 50% increases average annual costs by $21 million per year 

relative to the Base Case. Totally eliminating the interties increases average annual costs by 

$262 million per year relative to the Base Case. However, as shown in Figure 19, these 

economic costs can also be reduced substantially by increasing the planning reserve margin. 

Reducing interties at a particular reserve margin increases system costs because it reduces the 

ability of the Study RTO to import lower-cost resources. This generally reduces costs at any 

time when the system is importing, which is particularly valuable during scarcity and emergency 

events. The Long Neighbor Case also illustrates that assumptions about reserve margins 

maintained in external systems has a substantial impact on the Study RTO’s target reserve 

margin. 

Maintaining a 0.1 LOLE also imposes greater costs in more isolated systems, because the Study 

Region not only loses the opportunity of lower-cost imports, but it also incurs the large capital 

costs associated with having to maintain a higher reserve margin. Maintaining a 0.1 LOLE in the 

50% Transmission Case incurs an additional $43 million of average annual costs compared to the 

Base Case, while the Islanded system incurs $219 million in additional average annual costs. As 

the differences in reserve margin and total costs for the economic and LOLE-based reserve 

margin targets shown in Figure 19 demonstrate, the cost savings from increased interconnections 

and neighbor assistance are more substantial if the reserve margin is determined economically 

rather than with LOLE targets. A full study of the economic value of a particular intertie 

upgrade would also consider the cost of the expanded interties relative to these economic 

benefits, in addition to other reliability and operational benefits that the intertie might provide. 

Figure 19 also shows that the value of interties increases when neighboring regions have excess 

capacity, because the Study RTO can rely more heavily on the neighboring regions for low-cost 

imports under normal as well as scarcity conditions. For example, increasing external reserve 

margins from 15% to 20% allows the Study RTO to decrease its internal reserve margin from 

15.2% to 13.0% while maintaining 0.1 LOLE. However, it is important to note that adding 

external capacity is not as valuable as adding internal capacity, from either a cost or reliability 

perspective. In fact, increasing external capacity by 6,500 MW across the three external regions 

reduces the Study RTO’s internal reliability requirement by only 1,100 MW. This is because 

external resources can only be imported if: (1) interties are sufficiently sized; (2) the interties are 

not derated; and (3) external regions are not simultaneously experiencing scarcity or emergency 

peak load conditions. The capacity value of external resources would increase if they were 

committed to the Study RTO on a firm basis (with firm import rights), which would mean that 

the Study RTO would have first rights to call them. 
59 

Figure 19 

Total System Costs vs. Reserve Margin with Varying Intertie Assumptions 

Note: 

The vertical scale is much greater than in Figure 14 and Figure 18 above, which plot the same cost 

data in a different way. The magnitude of the change in system costs in the Base Case is identical 

for all charts, but this figure adds a baseline of total Study RTO production costs to allow the 

comparison of total system costs across the different cases. 

The reduction in Study RTO reserve margins enabled by the interties relative to the Island case, 

is sometimes referred to as the “tie benefit” or “capacity benefit of interties.”104 The magnitude 

of these intertie benefits is estimated probabilistically as the likely quantity of energy that will be 

available for non-firm import from neighbors during scarcity or emergency conditions. Such 

non-firm imports that contribute to intertie benefits are different from, and in addition to, firm 

imports that are committed to the Study RTO. 

The most obvious approach for modeling resources in neighboring regions might be to include 

all projected capacity resources, including surplus capacity. The surplus capacity would increase 

the estimated tie benefits and reduce the Study RTO’s target reserve margin. The problem with 

this approach is that it will not represent a long-term stable resource adequacy target because 

Study RTO target reserve margins would fluctuate with changes in the external regions’ reserve 

margins (e.g., as a result of plant retirements that reduce surplus capacity). Further, the study 

RTO may have insufficient information to accurately project the likely quantity of resources 

available in the external system. Because of these challenges, ISO-NE recently abandoned this 

approach, opting instead to model external markets “at criterion,” regardless of any actual 

 

104 For example, see PJM (2011). 
60 

excess.105 This more conservative “at criterion” assumption is standard among most RTOs, 

primarily because the Study RTO can be fairly certain that external regions will meet their own 

planning reserve margin targets but cannot be totally certain that an external system’s existing 

surplus would be maintained. The “at criterion” approach also results in a more stable internal 

reserve margin target. While the Study RTO may still benefit from the external regions’ surplus 

capacity through lower-cost imports and improved reliability, changing amounts of surplus 

would not affect the Study RTO’s planning reserve margin. 

A final, more complex question relates to how much of an RTO’s intertie capability should be 

left uncommitted or even set aside as the capacity benefit margin.106 This is one of the key 

questions currently being addressed at the MISO-PJM seam in the FERC cross-border capacity 

deliverability proceeding.107 Under the Point-to-Point transmission rights model used across 

most of North America, intertie capability is broken into two components: (1) transmission 

capability available to support long-term firm transmission rights that are needed for a generator 

to sell firm capacity across market interties (e.g., from MISO to PJM); and (2) transmission set 

aside as CBM and awarded only on a short-term basis for non-firm energy sales. Only the 

uncommitted portion of the interties, including any portion that is specifically set aside as CBM, 

would contribute to probabilistically estimated tie benefits.108 Firm import commitments are 

generally treated similarly to internal resources for determining target reserve margins.109

There are also a number of factors to consider in determining what portion of intertie capability 

to set aside for CBM. As explained by MISO and its market monitor, reducing CBM would 

increase the transmission that could be awarded for firm exports.110 This would likely reduce 

PJM’s capacity costs by allowing more MISO resources to sell capacity into PJM. On the other 

hand, as PJM explains, reducing CBM would reduce the realized intertie benefits and thereby 

increase PJM’s planning reserve margin. Ultimately, determining the optimal quantity of 

uncommitted transmission (e.g., by setting it aside as CBM) is an economic question that 

requires balancing the tradeoff between these factors. 

3. Combined Cycle Plants as the Marginal Technology 

Another study assumption is the type of marginal resource that is added in our simulations to 

increase the Study RTO planning reserve margin. In the Base Case, we add CTs as the marginal 

resource. While this is consistent with the use of CTs as the standard peaking resource, a mix of 

peaking, intermediate, and baseload resources is typically added under real-world conditions and 

in regulated resource planning efforts. The most economic type of resource to build depends on 

the composition of the existing fleet, the forecast load profile, projected fuel prices, and other 

 

105 For a more comprehensive discussion of tie benefits questions within ISO-NE, see FERC (2012), p. 2. 

106 For a detailed description of NERC standards relating to CBM, see NERC (2008). 

107 Under FERC Docket AD12-16-000. 

108 For example, see PJM’s calculation of the capacity benefit of ties based on the CBM portion of its 

interties, PJM (2011). 

109 Note that in our modeling we effectively treat all intertie capability as CBM. 

110 See MISO (2012b). 
61 

factors.111 We test the sensitivity of our results to the assumed marginal technology under a case 

in which we add CCs rather than CTs to increase reserve margins.112 

As shown in Table 10, the physical reliability implications of adding CCs and CTs are essentially 

identical because these technologies have similar outage characteristics. Using CCs as the 

marginal resource results in 15.2% and 15.3% reserve margin targets under the 0.1 LOLE 

standard. The small differences in CT and CC outage characteristics are not large enough to 

make a substantial impact on physical reliability metrics. This would generally be the expected 

result from adding different types of thermal generation resources that have similar outage 

characteristics.113 

Table 10 

Reserve Margin Targets with CC and CTs as the Marginal Resource 

Coincidentally, the economic target reserve margins between the CC and CT cases are also quite 

similar, but slightly lower in the CC case. This result will not hold true under all circumstances 

since the economic consequences of adding different types of technologies may be vary 

considerably. The fact that it is true in our simulations consequently cannot be generalized. 

Importantly, comparing the economic reserve margin targets does not help to determine whether 

it would be more beneficial to add CCs or CTs. Comparing the net system benefits achieved by 

adding two different types of resources requires comparing the total system costs under each 

case. This is illustrated in Figure 20 below. This figure shows total production and reliability 

 

111 See a more comprehensive discussion of optimal resource mix conflicts in Pfeifenberger, et al. (2009), 

Section III.A. 

112 More generally, a thorough system planning effort could evaluate many types of resources to determine 

which is the most economic. Alternately, in a deregulated market-based environment, merchant suppliers 

would determine which type of resource to build by forecasting the profitability of each type of potential 

investment. Under certain, idealized conditions the system planning and market-based approaches would 

both result in the same type of resource being added as discussed further in Section IV.A.1. 

113 We assume that CCs have a very slightly higher forced outage rate causing the reliability-based reserve 

margin targets to increase by a small amount of approximately 0.1% in the Marginal CC case. Conducting 

a modeling exercise at a higher level of granularity would be required to evaluate additional nuances 

regarding the reliability impacts of different types of thermal generation resources. For example, even 

though resources may have the same overall forced outage rates, there may be systems in which it would 

be relevant to examine the reliability implications of: (a) a large number of natural gas plants that are 

interconnected to the same pipeline that could expose the region to shortages caused by pipeline outages or 

other fuel supply constraints; (b) resources that do or do not have backup diesel fuel capability that would 

protect against many fuel-related outages; and (c) systems with high wind penetration or other contingency 

risks where fast-response CTs would provide additional reliability and operational value. 

Simulation Reliability-Based Risk-Neutral, Cost-Minimizing 

0.1 LOLE 2.4 LOLH 0.001% 

Normalized EUE

Cost-of-Service 

Perspective

Societal 

Perspective

Base Case (Marginal CT) 15.2% 8.2% 9.6% 10.3% 7.9%

Marginal CC Case 15.3% 8.3% 9.8% 10.1% 7.7%
62 

costs in the Study RTO from a risk-neutral, cost-of-service perspective for both CCs and CTs as 

the marginal resource. The figure shows that all-in fixed and variable costs drop faster when 

adding CCs than when adding CTs if starting with the same resource mix at the 7% reserve 

margin. This is because the greater production cost savings associated with adding CCs 

outweighs that technology’s greater capital costs. In other words, even though the economic 

reserve margin targets are similar, it is more beneficial to add CCs than CTs because it will 

achieve greater system costs savings. The vertical difference between the red dots in the chart 

shows that the lowest average annual cost for CCs is $21 million per year below that of CTs. 

However, this incremental cost advantage of CCs declines as more CCs are added, which is why 

the two cost curves are more parallel at higher reserve margins. At some point, the capital and 

operating cost tradeoff between the two technologies will equalize, at which point it will be 

optimal to add a mix of the two resource types.114 ` 

Figure 20 

Study RTO Total Production and Reliability-Related Costs with Marginal CC or CT 

(Risk-Neutral, Cost of Service Perspective) 

Note: 

 The vertical scale is much greater than in Figure 14 and Figure 18 above, which plot the same 

cost data in a different way. The magnitude of the change in system costs in the Base Case is 

identical for all charts, but this figure adds a baseline of total Study RTO production costs to 

allow the comparison of total system costs across the different cases. 

 

114 We have not attempted to identify the optimal quantity of additional CCs before the two types of resources 

become equally valuable. Identifying this tradeoff point would require a slightly different analysis in 

which we would: (1) test the incremental cost savings after adding one CC vs. one CT; and then (2) if the 

CC provides more value, we would keep that asset in the mix and then test the value of adding one more 

CC vs. one more CT. 
63 

4. Intermittent Renewable Resource Penetration 

Intermittent renewable resources such as wind and solar have a resource adequacy value 

significantly below their nameplate capacity because they typically cannot generate at their full 

capacity during peak load conditions. Because high load conditions also often correspond with 

low wind speeds, wind turbines’ resource adequacy value generally is even below the resource’s 

average annual capacity factor. On the other hand, a solar resource’s capacity value is usually 

higher than its average annual capacity factor because solar output typically is more correlated 

with peak-load conditions. Recognizing these factors, most RTOs assign intermittent resources a 

capacity value based on average output during peak load hours.115 When we calculate planning 

reserve margins throughout this study, we assign a nominal 15% capacity value to wind 

generators and 25% to solar resources. 

The actual resource adequacy value of intermittent resources will depend on a number of factors 

and increase with: (1) the degree of correlation between load and the intermittent resource’s 

output profile; and (2) the study region’s penetration of dispatch-limited resources and storage 

because renewable generation during near-peak conditions will allow storage and dispatchlimited resources to generate more during peak conditions. 

A more precise estimate of the intermittent resources’ resource adequacy value can be calculated 

based on Effective Load Carrying Capability (ELCC) as currently implemented in MISO.116 

Broadly, the objective of using ELCC is to assign a capacity value to each resource type that 

makes them interchangeable from a resource adequacy and planning reserve margin perspective.

117 While there are various approaches to calculating ELCC, our approach is to: (a) add a small 

quantity of gas CTs and determine the resulting reduction in LOLE; and (b) remove the gas CTs 

to restore the system to its starting configuration, then add intermittent wind resources until the 

system LOLE is reduced by the same amount as was achieved by adding the CT. Note that this 

LOLE-based approach is most appropriate for a system with an LOLE-based resource adequacy 

 

115 For example, PJM assigns intermittent resources a capacity value equal to the capacity factor during 

summer peak periods over the previous three years, or a generic value of 13% or 38% for new wind and 

solar resources respectively. See PJM (2010). For a wider survey of RTO practices for assigning wind 

resources’ capacity value, see Milligan and Porter (2005). 

116 MISO conducts an ELCC study each year to determine the capacity value that will be assigned to all wind 

resources, currently set to 13.3% of nameplate, see MISO (2012a). For additional discussion of 

approaches to calculating ELCC, see Milligan (1997); Wiser and Bolinger (2005); Kahn (2004). 

117 To date, ELCC calculations only consider the interchangeability from a physical reliability perspective in 

terms of reducing LOLE, LOLH, or EUE. It does not, and we do not here, attempt to ensure 

interchangeability from an economic perspective, which is inherently difficult because much of the 

economic value of a zero-marginal cost resource (like wind generation) is its energy value during off-peak 

conditions that do not materially relate to resource adequacy. Developing an economic comparison 

between wind and thermal resources would require one to either: (a) arbitrarily determine which 

component of energy value would be compared (i.e., above some peaking cost threshold) to calculate 

ELCC; or else, (b) would simply compare the total economic value from energy and resource adequacy 

contributions, which is most relevant comparison for IRP purposes, but is not conceptually similar to 

ELCC. 
64 

standard. Slightly different approaches would be needed in systems with LOLH or EUE-based 

standards or those that account for resources in UCAP rather than ICAP terms. 

We use this approach to calculate the ELCC of wind generation at varying wind penetration 

levels as shown in Figure 21 for the Study RTO. The figure shows that ELCC drops as wind 

penetration increases because the system is then more vulnerable to correlated low-wind 

conditions during peak load events. The figure shows that wind ELCC is only 12% at the 5% 

penetration level and drops all the way to 8% at the 30% penetration level. This indicates that 

the 15% capacity value that we assign to wind is high relative to its reliability contribution.118 

Note, however, that our relatively low ELCC results are driven by relatively poor wind profiles 

that correspond to the weather and load profiles of the study region. The average annual 

capacity factor for wind generators in the Study RTO is approximately 31%, but the output 

during peak load conditions is much lower. 

Figure 21 

Wind Generator’s Effective Load Carrying Capability vs. Penetration 

Notes: 

Wind ELCC calculated as for the Base Case simulation, with 0.1% solar penetration. 

 Wind penetration expressed as wind nameplate capacity as percent of summer peak load. 

 Study RTO reserve margin is 12% at 5% wind penetration and increases as wind increases. 

 

118 Note that the Base Case and all the other simulation cases incorporate 3.9% renewable penetration, 

expressed as a percent of peak load (3.8% wind, 0.1% solar). 
65 

C. DISPLACING GENERATION WITH DEMAND RESPONSE

As explained in Section II.C.4 above, an important set of questions for resource adequacy in the 

coming years relates to the reliability and economic implications of increasing quantities of 

demand response. Evaluating the implications of increased DR penetration requires an approach 

that differs from that of using CTs or CCs as a marginal resource. When evaluating high demand 

response penetration, the most important questions are less related to the implications of 

choosing a different reserve margin but, rather, related to the implications of relying more 

heavily on DR as a substitute for traditional generation at a given planning reserve margin. 

For this reason, we evaluate the consequences of higher demand response penetration at a fixed 

15% planning reserve margin, but substitute CTs with DR resources at higher penetration levels. 

We evaluate this in two series of scenarios: (a) substituting CTs with emergency DR that is calllimited, at DR penetration levels of 0%, 5%, 10%, 15%, and 20%; and (b) substituting CTs with 

economic DR that is not call-limited but rather dispatched based on bid price at the same range 

of DR penetration levels. For emergency DR, we also test the impacts of varying the dispatch 

limit. In the following sections we discuss: (1) the reliability implications of relying more 

heavily on emergency DR; (2) the economic implications of higher emergency DR penetration; 

and (3) the economic implications of higher economic DR penetration. 

1. Reliability Value of Emergency Demand Response 

In this section we evaluate how the assumed 100-hour dispatch limit reduces the resource 

adequacy value of Emergency DR resources, particularly at higher penetration rates. The impact 

of a particular call-hour limit can be understood by considering the load duration curve for the 

top 10% to 15% of hours as shown in Figure 22. The chart shows the 400 highest-load hours in 

the weather-normalized year, a mild-weather year, and a high-load year. The horizontal lines 

show the maximum load that can be supported solely with traditional generation resources at 

various levels of DR penetration. As the DR level increases from 0% up to 20% of peak load, 

the quantity of installed generation decreases and the maximum load that can be supported 

without DR calls decreases. 
66 

Figure 22 

Approximate Emergency DR Dispatch Hours at Varying DR Penetration Levels 

Notes: 

The chart is indicative of the expected quantity of DR calls but is imprecise because it does not account for the 

(generally offsetting) effects of generation outages, import availability, or load forecast error. See also 

Figure 23, which accounts for all of these factors. 

The chart shows that, even without considering the impact of economic uncertainty, at a 10% DR 

penetration level the 100-hour dispatch limit of Emergency DR resources would be exceeded in 

extreme weather years. As the red dots on the chart show at 10% penetration, Emergency DR 

resources would be need to be dispatched approximately 5, 50, or 122 hours under mild, normal, 

or extreme weather respectively. At 15% DR penetration, the 100-hour dispatch limit would be 

exceeded even under normal weather, and with 20% DR penetration the 100-hour limit would be 

exceeded even with mild weather. In other words, at DR penetration levels of more than 5%, the 

resource adequacy value of adding more dispatch-limited DR will deteriorate rapidly unless the 

call limits are increased substantially. 

Most regions have historically been able to rely on Emergency DR with similar or lower callhour limitations because: (a) DR penetration levels have not exceeded approximately 5–7% in 

most regions; and (b) many regions across North America have enjoyed surplus generation 

conditions with reserve margins greater than 20%, or well above their reserve margin targets.119 

We simulate regions at their target planning reserve margins and with higher DR penetration 

 

119 See IRC (2009), p. 26, and NERC (2011a). 
67 

levels, which consequently requires DR to be dispatched more frequently than has been the 

industry experience in recent years. 

Figure 23 shows the effective load carrying capability (ELCC) of Emergency DR resources. The 

ELCC of Emergency DR increases with the dispatch price and decreases at higher DR 

penetration levels. We calculate ELCC as in Section III.B.4 above by determining the quantity 

of Emergency DR that would have to be added to achieve the same reduction in LOLE as adding 

a fully-dispatchable traditional CT. The chart shows that even at a 0% penetration level, DR 

with a dispatch limit of only 10 hours per year has a capacity value only approximately 30% of 

an equally-sized CT. This is because, even with very low penetration levels, the 10-hour 

dispatch limit would be exceeded significantly under outcomes with high economic growth or 

very hot weather. 

Under the Base Case assumption with a 100-hour call limit, Emergency DR has an ELCC of 

approximately 100% at the 0% penetration level, 18% at 10% penetration, and only 8% at 20% 

penetration. The figure also shows that to maintain 100% ELCC, the call limit would have to be 

increased to 190 hours at 10% penetration and 250 hours at 20% penetration. 

These results also show that, as long as DR penetration levels are low, a system operator does 

not necessarily need to consider the risk of exceeding dispatch hour limits. However, at higher 

penetration levels, reliability can only be maintained at criterion if: (1) Emergency DR resources 

are awarded a lower capacity value for the purposes of calculating their contribution to the 

reserve margin; or (2) Emergency DR resources are required to increase their call-hour limit. 

Some regions may implement a combination of these options as DR penetration increases in 

order to accommodate different types of DR that have more or less callability, as has been done 

in PJM. PJM now allows DR resources to participate either as dispatch-limited DR that may 

earn lower capacity payments or unlimited DR that can earn full capacity payments.120

 

120 For a discussion of these multiple DR products, see Pfeifenberger, et al. (2011a), Section VI.C.1. 
68 

Figure 23 

Emergency DR’s Effective Load Carrying Capability 

(Varying DR Penetration and Call Hours) 

Notes: 

Study RTO reserve margin fixed at 15% with CTs displaced by increasing DR. 

 ELCC of 25% indicates that 4 MW of DR would be needed to achieve the same 

reduction in LOLE as adding 1 MW of CTs. 

2. Cost-Minimizing Level of Emergency Demand Response Penetration 

At higher penetration levels, the incremental value of adding Emergency DR drops from a cost 

perspective, as well as from a reliability perspective. We illustrate this in Figure 24, which 

shows Study RTO production and reliability-related costs as the Emergency DR penetration level 

increases (and installed generation levels consequently decrease). The chart shows total costs 

under: (a) varying fixed-cost assumptions for DR ranging from $0/kW-year to $60/kW-year (on 

the left); and (b) DR dispatch-cost assumptions ranging from $500/MWh to $3,000/MWh (on the 

right). 

As shown in Figure 24, beginning at 0% DR penetration total costs initially decrease when 

adding more DR because these resources have lower capital costs than the CTs that they replace. 

Even though Emergency DR has a much higher dispatch cost than the displaced CTs, the number 

of dispatch hours is low enough that it is advantageous to continue adding more DR. However, 

at higher penetration levels the number of dispatch hours becomes a limiting factor and it is no 

longer cost-effective to continue replacing CTs with more DR. Overall, and depending on the 

assumed capital costs and dispatch hours, the cost-minimizing level of Emergency DR ranges 

from approximately 8% to 12% of peak load as indicated by the red dots. 
69 

Figure 24 

Production and Reliability Costs vs. Emergency DR Penetration 

(Risk-Neutral, Cost of Service) 

Notes: 

Study RTO reserve margin fixed at 15% with increasing DR offset by reductions in CTs. 

 Left chart assumes DR dispatch cost of $2,000/MWh; right chart assumes DR fixed costs of $15/kW-yr. 

Note that the optimal level of DR penetration is also dependent on the planning reserve margin, 

which we assumed to be 15%. If the target reserve margin were higher, then higher DR 

penetrations could be achieved without incurring as many dispatch hours. As a result, the 

optimal penetration level would be higher. Higher dispatch limits would also increase the 

optimal penetration levels because Emergency DR, while still a high-dispatch-cost resource, 

would be able to displace even higher-cost emergency procedures during more hours. 

The simulation results highlight the complex role of demand response in reliability planning. 

DR penetration levels, the design of the products (including dispatch limits and price), and the 

system load shape all affect the economic and reliability value provided by these resources. The 

results also show that DR products have to be defined carefully to provide maximum system 

reliability and economic benefit as customers will only participate in these programs if they are 

adequately compensated for the fixed and dispatch costs they incur. Ultimately, whether a 

reasonably efficient portfolio of DR resources will be attracted will depend on: (a) the level of 

flexibility provided to DR suppliers in defining their own dispatch prices and call-hour limits; 

and (b) the efficiency of the combined energy and capacity market incentives provided as 

discussed further in Section IV.C below. 

3. Cost-Minimizing Level of Economic Demand Response Penetration 

Figure 25 shows the Study RTO production and reliability-related costs as a function of the 

Economic DR penetration level. The chart is very similar to that shown in the previous section 

for Emergency DR. As with Emergency DR, adding Economic DR reduces total system costs by 
70 

displacing CTs with higher capital costs. However, Economic DR is more valuable than 

Emergency DR in our simulations because: (a) a substantial portion of the Economic DR is 

assumed to have lower dispatch costs than Emergency DR; and (b) Economic DR is not calllimited. For these reasons, the optimal level of Economic DR penetration ranges from 

approximately 10% to 14%, depending on the assumed annual fixed costs. However, this level 

of Economic DR penetration can only be achieved if a sufficient quantity of potential DR 

resources are willing to participate in the energy markets at the price levels assumed in our 

simulations (as described in Section II.C.4 above). We further explore the energy and capacity 

market design implications of high DR penetration levels in Section IV.C below. 

Figure 25 

Production and Reliability Costs vs. Economic DR Penetration 

(Risk-Neutral, Cost of Service) 

Notes: 

Study RTO reserve margin fixed at 15% with increasing DR offset by reductions in CTs. 

 Economic DR dispatch costs are set according to the DR energy supply curve from Section II.C.4 above. 

D. COMPARISON OF RESERVE MARGIN TARGETS 

The preceding sections evaluated the reliability and economic implications of a wide range of 

study assumptions and system conditions and explained how different reliability and economic 

objectives would lead to different planning reserve margins. Table 11 provides a brief summary 

of these results, including the sensitivity of the “optimal” (risk-neutral, cost-of-service) reserve 

margin to various economic assumptions. Table 12 summarizes the reliability-based and 

economically-based reserve margins under various system characteristics. 

As Table 11 shows, the risk-neutral, economically-optimal planning reserve margins could vary 

by several percentage points under a range of reasonable cost and other economic study 
71 

assumptions. For example, different reasonable assumptions for emergency event costs 

correspond to planning reserve margins ranging from 9.2% to 12.1%. Although this range is 

substantial, it may be surprisingly small given that the actual cost of emergency events is poorly 

understood and quite uncertain. Note, however, that the impact of the assumed VOLL, 

representing the costliest emergency intervention of curtailing firm load, has only a modest 

impact. Varying VOLL over a wide range from $3,750 to $15,000/MWh results in an economic 

reserve margin ranging only from 10.0% to 11.6%. 

Other important economic assumptions that also affect the optimal reserve margin include: 

(1) the forward planning period, resulting in planning reserve margins ranging from 9.4% to 

11.0% for forward planning periods from two to six years; (2) the CT CONE, resulting in 

planning reserve margins ranging from 9.5% to 11.0% for CONE values from $100 to $140/kWyear; and (3) the ownership of transmission rights, resulting in planning reserve margins ranging 

from 8.3% to 12.3% depending on whether the importer owns rights (allowing it to buy power at 

the lower external price) or the exporter owns rights (allowing it to sell power at the higher 

external price).121

Table 11 

Sensitivity of Economically Optimal Reserve Margin to Economic Study Assumptions 

(Risk Neutral, Cost-of-Service Perspective) 

Notes: 

Study RTO “optimal” reserve margin from the risk-neutral, cost-of-service perspective. 

 Impact of economically-determined planning reserve margin on various economic sensitivities. 

Table 12 summarizes the reliability-based and economically-based reserve margin targets under 

the Base Case scenario as well as under each of the alternative simulation cases we examined in 

 

121 See Sections III.A.5 and III.A.2 for additional discussion of forward period and transmission rights 

assumptions. 

Reserve Margin Range Base Case Low/High Sensitivity

(% ICAP)

Base Case 10.30% n/a n/a

Emergency Event Costs

Emergency Generation 10.2% - 10.5% $500/MWh $250 - $1000/MWh

Emergency DR 9.9% - 10.9% $2000/MWh $1000 - $3000/MWh

Emergency Hydro 10.2% - 10.5% $3,000/MWh $1,500 - $6,000/MWh

Voltage Reduction 10.2% - 10.4% $7,000/MWh $3,500 - $14,000/MWh

VOLL 10.0% - 11.6% $7,500/MWh $3,750 - $15,000/MWh

All Emergency Event Costs 9.2% - 12.1% Base 50% or 200% Base

Other Assumptions

Load Forecast Error 9.4% - 11.0% 4 Years Forward 2 Years - 6 Years

CONE 9.5% - 11.3% $120/kW-y $100 - $140/kW-y

Transmission Ownership 8.3% - 12.3% 50/50 Ownership Importer/Exporter Owns
72 

the preceding sections. We also show the results from the simulation cases with $3,000/MWh 

and $1,000/MWh price caps, which we discuss in the context of market design in Section IV of 

this report. Generally, these results show that differing system size, topology, and other 

conditions can substantially impact planning reserve margin targets. For example, maintaining a 

0.001% Normalized EUE reliability standard could require a reserve margin anywhere from 

7.5% up to 15.8% depending on the assumed system topology, size, and other characteristics. 

Similarly, economically-optimal, risk-neutral reserve margins could vary anywhere from less 

than 6% to 16.5% for both cost-of-service and societal perspectives. 

Nevertheless, changing system characteristics has similar directional impacts on reserve margin 

targets for all resource adequacy criteria, both in terms of physical reliability and economics. 

For example, the Island Case always requires higher planning reserve margins because the 

system is not able to rely on neighbors for any resource adequacy support. Similarly, the Long 

Neighbors Case always allows for lower planning reserve margins in the Study RTO due to the 

greater availability of neighbor support. 

Table 12 

Reliability-Based and Economically-Based Reserve Margin Targets 

(Across Base and Sensitivity Case Simulations) 

Simulation Reliability-Based Risk-Neutral, Cost-Minimizing

0.1 LOLE 2.4 LOLH 0.001% 

Normalized EUE

Cost-of-Service 

Perspective

Societal 

Perspective

Base Case 15.2% 8.2% 9.6% 10.3% 7.9%

Lower Price Caps

$1,000 Price Cap Case 15.2% 8.2% 9.6% 8.7% 7.9%

$3,000 Price Cap Case 15.2% 8.2% 9.6% 9.5% 7.9%

Smaller System Size

40% Size Case 14.8% <6% 7.5% <6% <6%

40% Size and Transmission 15.1% 6.9% 8.1% <6% <6%

Neighbor Assistance

Long Neighbors Case 13.0% <6% 7.0% 8.0% <6%

50% Transmission Case 15.8% 9.8% 10.0% 12.3% 10.5%

Island Case 18.5% 16.5% 15.8% 16.5% 16.5%

Marginal CC Case 15.3% 8.3% 9.8% 10.1% 7.7%
73 

IV. MARKET DESIGN IMPLICATIONS 

Up to this point, we evaluated only the reliability and economic implications of different 

planning reserve margins, system conditions, and resource penetration levels with respect to 

annual system costs. Much of that analysis is relevant irrespective of market design and whether 

generation resources are cost-of-service regulated or restructured. However, the ability to 

achieve a particular planning reserve margin, as well as the economic implications for customer 

costs, supplier returns, and market prices, cannot be evaluated without considering the market 

design of the study region. In cost-of-service regulated environments, resource adequacy is 

achieved through integrated resource planning and supported by rate recovery for all approved 

investments in generation. In market-based environments, investments can only be attracted if 

prices are high enough on average for suppliers to earn a sufficient return on their investments. 

In this section, we evaluate the implications for both customers and suppliers with respect to: 

(a) energy-only markets that do not have a reliability requirement and whose reserve margin is 

determined by the quantity of resource investments that can be attracted by revenues from the 

region’s energy and ancillary services (A/S) markets; (b) capacity markets that do have a 

planning reserve margin standard, with supply investments attracted by a combination of energy, 

ancillary service, and capacity market revenues; (c) the implications of integrating high levels of 

demand response into energy-only and capacity markets; and (d) a comparison of customer and 

supplier impacts under various energy-only and capacity market designs.122

A. ENERGY-ONLY MARKETS

Restructured markets, including both energy-only markets and regions that rely on capacity 

markets, can only attract investments in needed resources if prices are high enough on average 

that suppliers can expect a sufficient return on their investments. In “energy-only” markets, 

there is no mandatory minimum reserve margin. In these markets, including Texas, Alberta, 

Australia’s NEM, and Scandinavia, the reserve margin achieved depends on the market prices 

for energy and ancillary services, with new generation being built only when suppliers believe 

that their investments can earn their cost of capital. In other words, there is no guarantee that 

enough generation will be built to maintain a certain reserve margin or resource adequacy level 

on average.123 

 

122 In the following sections, we briefly explain the major features and differences between energy-only and 

capacity markets, but stop short of a full theoretical treatment. For a more comprehensive survey of 

market designs for resource adequacy, we refer the reader to Pfeifenberger, Spees, and Schumacher 

(2009); and Spees, Newell, and Pfeifenberger (2013). 

123 Most so-called “energy-only” markets also rely on some reliability backstops to prevent unacceptably low 

reliability events. For example, Scandinavian markets maintain “strategic reserves” of usually high-cost, 

older capacity that is only dispatched under emergency conditions at very high prices (i.e., with market 

prices approximately reflecting a world in which those strategic reserves did not exist at all). Similarly, 

Texas regularly procures demand response supplies that are to be deployed only during emergency 

conditions. Many markets, including Texas and Alberta for example, also have mechanisms for 

administrative intervention into the markets if reliability is expected to drop to very low levels. See 

Pfeifenberger, Spees, and Schumacher (2009), pp. 28-29; AESO (2008); European Commission (2012), 

p. 8. 
74 

In this section, we evaluate the economic and reliability implications of resource adequacy 

within energy-only markets, including: (1) how the equilibrium reserve margin is determined 

within energy-only markets; (2) the preconditions that are necessary in order for the equilibrium 

reserve margin to reflect the socially optimal level; (3) the volatility in energy market returns that 

suppliers would have to expect in an energy-only market; (4) the impact of different system 

conditions and modeling assumptions; and (5) the “missing money” problem that can cause 

energy-only markets to fail to attract investments sufficient to meet system operators’ or policy 

makers’ reliability objectives. 

1. Economic Equilibrium at the Cost of New Entry 

In energy-only markets, system operators are not able to mandate a minimum planning reserve 

margin. Instead, the reserve margin achieved within an energy-only market depends on market 

prices and on suppliers’ expected returns. As shown in Figure 26 for the Base Case simulations, 

the energy margins that a new gas CT would expect to earn on average in the Study RTO are a 

downward-sloping function of reserve margin. We show energy margins relative to the CONE 

value of a CT that reflect the average annual net revenues required to recover investment and 

fixed costs over the economic life of the plant.124

The figure shows that average annual energy margins are high at very low reserve margins 

because tight supply conditions create high energy prices and more frequent price spikes. Under 

such conditions, an energy-only market would attract additional investments because suppliers 

would expect to earn on average more than enough in the energy market alone to recover their 

fixed costs. As the market attracts more investments and reserve margins increase, energy prices 

and supplier returns decline. The energy-only market will continue to attract incremental 

investments until energy margins drop to CONE, which represents an equilibrium point for 

achievable reserve margin. No additional investment would be attracted beyond this point 

because investors would not expect to recover their costs of new entry. 

In our Base Case simulations, the Study RTO would achieve an economic equilibrium reserve 

margin of 7.9% in the Study RTO (which corresponds to a 16.5% reserve margin over the 

combined four-system area, as explained in Table 7 in Section III above). This economic 

equilibrium is indicated by the red dot in Figure 26. Note, however, that because the achieved 

reserve margin is determined entirely by market forces, there is no guarantee of achieving any 

specific resource adequacy level in an energy-only market. In our hypothetical system, the 

economic equilibrium reserve margin is lower than the reserve margin that would be required to 

maintain a typical reserve margin of 15.2% under the 0.1 LOLE standard, 8.2% under the 2.4 

LOLH standard, or 9.6% under a 0.001% Normalized EUE standard. 

Finally, we note that to illustrate this equilibrium reserve margin result, we assumed that a 

natural-gas-fired CT is the marginal technology over the entire range of reserve margins. A 

more precise determination of equilibrium conditions would evaluate not only the equilibrium 

reserve margin but also the equilibrium resource mix. A system in disequilibrium would 

 

124 We refer to “net revenues” as the revenues earned in energy and ancillary service markets net of the 

plant’s variable operating and maintenance costs. Our simulation results show only net energy revenues as 

they do not separately consider ancillary service markets and associated revenues. 
75 

experience entry (or exit) of the most profitable (most uneconomic) types of assets from among 

many asset classes, and would reach true equilibrium at a particular reserve margin with a 

particular resource mix. We do not evaluate equilibrium resource mix in our study, but note that 

actual power markets will always experience at least some disequilibrium due to changing 

system conditions, regulations, and supplier expectations over time. 

Figure 26 

CT Energy Margins vs. Study RTO Reserve Margin 

Notes: 

CT energy margins in the Study RTO are from the Base Case simulation. 

2. Conditions for Achieving Socially Optimal Investment Levels 

The most important benefit of energy-only markets, compared to other market designs for 

resource adequacy, is that energy-only markets theoretically can attract the optimal, welfaremaximizing level of investments from a societal perspective. This is because prices in a 

perfectly-designed energy-only market will always reflect the marginal system cost defined by 

the intersection of supply and demand. This means that customers are able to choose how much 

reliability for which they are willing to pay. For example, some customers place a relatively low 

value on reliability, and so would rather reduce consumption than pay the $3,000/MWh price 

realized when emergency hydro resources need to be dispatched in our simulations. Other 

customers place a high value on service and so will be willing to pay the $7,000/MWh price 

incurred when implementing voltage reductions. In such a perfectly-designed energy-only 

market, the decisions of suppliers, customers, and the system operator will all be made relative to 

the marginal societal costs and benefits of their actions. 

We show CT energy margins relative to CONE for our Study RTO Base Case in Figure 27. The 

figure, which assumes risk neutrality for all market participants, shows CT energy margins 
76 

relative to CONE (left axis) and total annual societal reliability-related costs (right axis).125 As 

explained in the previous section, the energy-only market will achieve an equilibrium reserve 

margin of 7.9%. This is also the reserve margin that will minimize total societal costs as 

tabulated in Section III.A.3 above.126 

Figure 27 

CT Energy Margins vs. Study RTO Reserve Margin and Societal Costs 

Notes: 

CT energy margins vs. societal reliability costs in Study RTO from Base Case simulation. 

 See Section III.A.3 for the derivation of risk-neutral, societal reliability-related costs. 

The reason that an energy-only market can achieve this outcome is that a perfectly-designed 

energy market will produce hourly energy prices equal to true marginal system costs.127 During 

 

125 The reader may note that we now shift from a discussion of “societal welfare-maximizing” reserve 

margins to a discussion of “societal cost-minimizing” reserve margins. We make this change as a matter 

of simplicity and convenience, since the objectives of “maximizing welfare” and “minimizing cost” are 

identical in a system with inelastic demand like our Study RTO. We do not ignore the possibility of 

elastic demand however, but instead account for net impacts on customer welfare indirectly by tabulating 

the “costs” imposed by dispatching economic DR, emergency DR, and involuntary load shedding. An 

alternative approach in which we calculate total societal welfare directly (consumer surplus plus producer 

surplus) would be more conceptually pure, but would unnecessarily complicate our discussion. 

126 Note that this reserve margin does not have any pre-determined relationship to the reserve margin that 

minimizes costs from a utility cost-of-service perspective as evaluated in Section III.A.2 above. 

127 The perfectly-designed market will also attract the optimal proportion of baseload, intermediate, and 

peaking units for the same reason that it would achieve the optimal reserve margin. In each case, for 

baseload as well as peakers, the net revenues available from the market will be equal to the net societal 

value created by displacing higher-cost resources or emergency actions. This means that baseload units 

will have sufficient incentives to enter the market only if the net societal value they create is equal to or 

Continued on next page 
77 

most hours, marginal system costs are equal to the production costs of the highest-cost 

generation or demand response asset dispatched to meet load. During peaking and scarcity 

events however, marginal system costs rise to higher levels driven by the marginal cost of 

emergency procedures such as calling emergency DR, emergency generation, emergency hydro, 

operating reserves shortages, voltage reductions, or load curtailments. This means that the profit 

that a new generating plant earns in a particular hour is exactly equal to the marginal reduction in 

the societal costs that it provides.128 If the reduction in total annual system costs is greater than 

the marginal unit’s total annual capital and fixed costs at CONE, then total societal costs will be 

reduced by that investment and suppliers have an incentive to enter the market. 

As promising as these theoretical benefits sound, we stress that actual power markets cannot be 

expected to achieve the optimal reserve margin in all cases. We are able to demonstrate the 

result in our simulations only because we have constructed a carefully-designed scarcity pricing 

mechanism to ensure that our simulated energy prices are always exactly equal to marginal 

system costs. Achieving such perfect energy prices that exactly equal marginal system costs is 

difficult if not impossible under real-world conditions. If those imperfections cause energy 

prices to be too high on average, the market will attract more than the optimal level of 

investments and will achieve a higher reserve margin. Of greater concern to most regulators and 

suppliers is the possibility of the opposite outcome in which energy prices and the resulting 

reserve margins are too low, a topic that we discuss further in Section IV.A.4 below. 

There are three types of factors that make it difficult to set prices perfectly in wholesale electric 

markets. First, there is insufficient integration of price responsive demand in wholesale electric 

markets, because most customers pay only a fixed price for energy and so have no incentive to 

respond to wholesale market conditions. Even among customers that do participate in wholesale 

DR programs or retail pricing that encourages price response, their willingness to pay for energy 

may not be fully incorporated into wholesale energy market prices due to the DR program 

design, technical challenges in price formation, or lack of enabling technology. Second, in 

circumstances when operator interventions are needed to address emergency conditions, prices 

can be artificially suppressed by the “additional supply” that the intervention creates (e.g., by 

dispatching emergency DR or implementing a voltage reduction). Several RTOs are working on 

 

Continued from previous page 

greater than their investment costs. There is also a displacement effect among resources in that adding 

1 MW of baseload would reduce net revenues to CTs, and adding 1 MW of CTs would reduce net 

revenues to baseload (although by a lesser amount). Overall, one would expect an investor to add the most 

profitable resource type at any given time in a way that moves the system toward an optimal equilibrium 

reserve margin and resource mix. Again, any deviations from perfect pricing that reflects marginal system 

costs will not only cause the equilibrium reserve margin deviate from the optimal reserve margin, but it 

will also cause the resource mix to deviate from the optimal resource mix. 

128 For example, consider the impact of adding a very small 1 MW CT with production costs of $62.50/MWh 

as in our Base Case. In an hour when load shedding would be required in the absence of the CT, the 

reduction in societal costs is equal to: (a) the reduction in unserved energy, times the VOLL, or 1 MWh 

times $7,500/MWh; minus (b) the increase in total production costs incurred by dispatching the CT, or 1 

MWh times $62.50/MWh. This reduction in total societal costs is also equal to the hourly energy margins 

earned by the CT which is the hourly price minus production cost (i.e., price at $7,500/MWh minus cost at 

$62.50/MWh). 
78 

efforts to prevent such price suppression, but these efforts are still ongoing. Finally, if an RTO 

opts to set administratively-determined scarcity prices to prevent price suppression during 

emergency events, it is a remaining challenge to determine the correct price for these events, 

such that the administrative price is reflective of the marginal cost of the reliability interventions. 

These are challenges that energy markets across North America are currently addressing as they 

attempt to improve their pricing mechanisms, particularly during scarcity hours.129 

3. Volatility in Supplier Energy Margins 

Figure 28 summarizes annual energy margins earned on average for a new CT (on the left) and a 

new CC (on the right) at different reserve margins. The figure also shows the range of annual 

energy margins that would be realized in spot energy markets due to uncertainties such as 

weather, load growth, hydro conditions, or outages. The annual average of energy margins 

reflects the expected value across all 9,600 simulations for each reserve margin.130 

To demonstrate the year-to-year volatility in supplier returns for an unhedged supplier, we also 

show the spot energy margins that a new unit would earn in the median year, along with the 

middle 50% of all years (the range between the 25th and 75th percentiles) and the middle 80% of 

all years (the range between the 10th and 90th percentiles). The median energy margin is 

representative of a “typical year” in the sense that there is a 50% chance that the actual energy 

margins will be above or below that value. The 90th percentile (top gray line) is particularly 

interesting, as it approximately represents a “once in ten years” event in which suppliers could 

earn more than these energy margins due to unusual tight supply or high load conditions. 

 

129 For a more comprehensive discussion of scarcity pricing mechanisms, including how the energy-only 

markets of Alberta and Texas are addressing these and other challenges, see Newell, et al.(2012),Section 

V.A.2; Pfeifenberger, et al. (2009), Section IV; and Pfeifenberger, et al. (2011a, 2013). 

130 As discussed in Section II.A, each of the 9,600 simulations for the Base Case and Change Cases estimates 

hourly reliability and economic system results for a full calendar year, with each of the 9,600 simulations 

representing different load shapes, economic forecast uncertainties, hydro conditions, unit outages, etc.
79 

Figure 28 

Volatility in CT (left) and CC (right) Energy Margins vs. Reserve Margin 

Notes: 

CT and CC energy margins the Study RTO are from the Base Case simulation. 

 Gray lines represent the 10th, 25th, 50th, 75th, and 90th percentile estimates of spot energy margins (i.e., net 

revenues earned in spot energy market without forward contracting or other risk mitigation). 

These results for CTs’ and CCs’ annual energy margins show that year-to-year volatility in 

supplier returns is quite large. For example, assuming no forward contracting, at the 7.9% 

equilibrium reserve margin at which a new CT will earn CONE on average across all years, we 

estimate that the plant would earn only approximately $30/kW-year (30% of CONE) in the 

“typical” year. However, the CT would earn more than $260/kW-year (more than twice CONE) 

approximately once in a decade. Similarly, the figure shows that at the equilibrium reserve 

margins where a new CC earns its CONE on average across all years, the plant would earn only 

approximately $55/kW-year (38% of CONE) in a typical year, although the plant would earn 

more than $290/kW-year (more than almost twice CONE) once in a decade. 

Figure 29 further illustrates spot energy market volatility by comparing three annual price 

duration curves at the equilibrium, the 7.9% reserve margin level. The first price duration curve 

is based on the weighted average across all 9,600 annual simulations (dark blue line) while the 

lighter blue lines illustrate a worst weather scenario (medium blue) and a worst weather 

combined with highest load growth scenario (light blue). On average over the years, spot energy 

prices would rise to high levels above $1,000/MWh only approximately 35 hours per year, while 

prices would rise above $1,000/MWh in approximately 140 hours under the worst weather year 

(and the weighted average of economic growth forecasting errors) and in approximately 325 

hours in the worst case scenario that combines the worst weather year with the highest 

unanticipated load growth. Note that one would not expect prices this high in many markets 

even under such a worst case scenario, since many markets maintain reserve margins that are 
80 

much higher than the 7.9% shown in the figure and also have lower price caps than the 

$7,500/MWh utilized in our Base Case simulations. 

Figure 29 

Price Duration Curve at the 7.9% Equilibrium Reserve Margin 

Notes: 

Study RTO Base Case Price duration curve on average and in extreme weather and load growth cases. 

The reason that, in the absence of forward contracting and hedging, supplier energy margins can 

rise so high relative to their median or even average annual levels is that market prices in most 

years will be set by the marginal costs of the generation fleet in nearly all hours. However, both 

our Study RTO and actual energy-only markets will also experience periodic severe price spikes 

that are driven by scarcity conditions reflecting extreme heat waves or unanticipated high load 

growth.131 Under these conditions, market prices can rise to very high levels as the need for 

high-cost emergency operating procedures increases. Realized prices in our Study RTO rise with 

the severity of the shortage event according to the scarcity pricing function described in Section 

II.E above. Prices may reach as high as the VOLL-based price cap of $7,500/MWh in the 

extreme circumstances when load must be shed. As explained in the previous section, these 

periodic, severe price spikes are necessary if the energy-only market is to attract the sociallyoptimal level of investments. 

The VOLL-based $7,500/MWh price cap and administrative scarcity pricing function also mean 

there is great uncertainty around the spot energy margins that power plants would earn and 

 

131 For a comparison of the frequency and severity of scarcity pricing events in various real-world energyonly markets, see Newell, et al. (2012), Section I.B. 
81 

customers would need to pay in any particular year in the absence of forward contracting or 

hedging. Unhedged suppliers would face substantial investment risk under a market 

environment in which returns in the median year are less than half the returns that would be 

expected on average over the years. This also means that the energy margins that a plant could 

have earned in the spot market in a recent year (or even several recent years) are not a good 

indicator of the average energy margins that plants will likely earn over their economic life. For 

example, at the equilibrium reserve margin, a CT selling solely into the spot market would earn 

less than 30% of CONE in half of all years. Further, though not shown at the scale of this chart, 

in the most extreme 5% of simulation cases (representing a 1-in-20 event that a unit may 

experience once over its useful economic life) annual CT margins earned in spot markets rise 5.5 

times above CONE. 

Overall, these results mean that if suppliers relied solely on spot market sales, a substantial 

portion of total investor returns would be earned in only a small number of extreme years. To 

mitigate this risk, asset owners typically use a portfolio of hedges or contracts. But the scale of 

the volatility nevertheless has a number of implications for lenders, equity investors, and policy 

makers. In particular, extreme price events (even if mostly hedged) have important political 

consequences that may trigger administrative or legislative interventions, including interventions 

that could undermine investment signals needed for resource adequacy. The regulatory risk of 

such potential interventions would require investors to discount the expected value of energy 

margins earned in such extreme cases, with greater perceived regulatory risk translating to lower 

realized investment levels in energy-only markets. We examine these and other considerations 

related to the risks to total supplier returns more fully in other publications.132 

Note, however, that neither customers nor suppliers are fully exposed to this level of spot price 

uncertainty. This is because all prudent suppliers and load-serving entities will mitigate a 

significant portion of these risks through hedging tools, such as forward contracts. Forward 

contracting will allow entities to lock in anticipated average prices for the future, meaning that 

while the contract price will exceed the realized spot price most of the time, it will be far below 

realized spot prices during extreme outcomes. The farther forward a supplier or load-serving 

entity is able to hedge,133 the less spot energy market risk exposure the supplier will have. For 

example, long-term contracts will mitigate nearly all price risks, including risk factors such as 

economic growth, weather, and outages. However, hedging even a few months forward will 

mitigate most weather and outage-related risks. Because weather and outage risks are shown to 

account for the majority of the uncertainty range in our market simulations, even short-term 

forward contracts (such as seasonal contracts) will be able to hedge a large portion of the 

uncertainty range shown above.134 In either case, the weighted average of energy margins will 

 

132 We address a number of related questions more extensively in Pfeifenberger and Newell (2011c); and 

Newell, et al. (2012), Section II. 

133 Note that to mitigate uncertainty in spot energy margins, suppliers will need to hedge both their energy 

market revenues and their fuel costs. 

134 For example, our Base Case simulations show that for the 7.9% equilibrium reserve margin, the 

uncertainty range between the 10th and 90th percentiles of CT energy margins is $320/kW-year. For 

normal weather and the weighted average of outage risks (i.e., with exposure only to multi-year economic 

load growth uncertainty), the uncertainty range between the 10th and 90th percentiles is reduced to 

$100/kW-year. 
82 

not be significantly affected by such hedging activity unless forward contracts sell at a premium 

above expected average outcomes because buyers are more risk averse than suppliers (or, 

alternately, at a discount if suppliers are much more risk averse).135 

The substantial risk mitigation that can be achieved even through seasonal forward contracts is 

illustrated by the example of ERCOT in 2011 and 2012. The summer weather for 2011 was an 

example of extreme conditions, while 2012 was milder than normal.136 Based on stakeholder 

interviews we conducted for ERCOT in a 2012 study prompted by these extreme events, we 

learned that both generators and competitive retail suppliers engage in a substantial amount of 

forward hedging at periods of up to three to five years. On a seasonal forward basis of several 

months, some retailers even buy hedges sufficient to cover their entire anticipated summer 

electricity needs. As a consequence, market participants have ample opportunities to mitigate 

extreme spot prices and are likely to pay or earn prices much closer to the expected average in 

many years. 

Table 13 illustrates the magnitude of risk mitigation through seasonal forward contracting. The 

table compares seasonal futures prices against realized hourly spot market prices in ERCOT for 

the Summers of 2011 and 2012. More specifically, the table compares for these two years: 

(1) the prices for June through September on-peak futures contracts as of April of each year; with 

(2) the realized on-peak spot prices for the same hours and months.137 As shown, for the peak 

months of August 2011 and August 2012, futures contract prices for these two months differed 

by less than $15/MWh or about 20%. In contrast, August on-peak spot prices for 2011 and 2012 

differed by $175/MWh, or approximately 280% of the average August futures price. This shows 

that reasonably effective market-based hedging tools are available to market participants who 

wish to mitigate spot price exposure. How much additional risk mitigation should be achieved 

by increasing planning reserve margins above the risk-neutral economic optimum is 

consequently a difficult question that will need to be addressed through additional research. 

 

135 If buyers are more risk averse they will be willing to pay a premium above expected spot prices to secure a 

higher level of forward certainty which would then translate into slightly higher supplier returns and 

realized reserve margins. 

136 See an extensive discussion of the Summer 2011 realized prices, scarcity events, and weather events, as 

well as our evaluation of impact on investors and buyers in Newell, et al. (2012). 

137 Data from Ventyx (2013). 
83 

Table 13

Illustration of Risk Mitigation Achieved by Seasonal Forward Contracting 

Sources and Notes: 

Futures and spot prices are for ERCOT North On-Peak prices. 

Futures prices are averaged across all trade dates in April of 2011 or 

2012, two to six months prior to delivery, from Ventyx (2013). 

4. Missing Money and the Impact of Price Caps 

The Base Case assumes that energy prices can rise up to the VOLL-based energy price cap of 

$7,500/MWh. As explained above, this VOLL-based price cap, combined with an efficient 

scarcity pricing mechanism tied to marginal system cost creates efficient real-time energy market 

price signals that accurately reflect the marginal value of supply at all times. In some cases, 

particularly during extreme scarcity events, these efficient prices can be very high relative to 

average prices. Imposing a price cap that is far below VOLL or otherwise suppressing prices 

during scarcity events will have the inefficient effect of suppressing prices and supplier returns 

below what is needed to achieve the risk-neutral societally optimal investment levels. This 

creates a so-called “missing money” problem in the energy market. 

Figure 30 demonstrates the concept of missing money for the Study RTO in the Base Case 

Simulation and in two alternative scenarios in which we introduce price caps of $3,000/MWh 

and $1,000/MWh. At the 7.9% risk-neutral, societal optimum reserve margin, suppliers earn 

high enough energy margins to sustain investment. However, imposing price caps of 

$3,000/MWh or $1,000/MWh eliminates the high prices of some events and causes supplier net 

revenues to drop to only 84% or 49% of CONE respectively. In other words, the price-capped 

market would have a “missing money” problem of $20 and $62/kW-year respectively, relative to 

the energy margins needed to sustain the socially optimal reserve margin. 

An even greater “missing money” problem exists if policy makers wish to maintain the 15.2% 

reserve margin needed to meet the 0.1 LOLE reliability standard. In this case, there are two 

sources of missing money: (1) energy prices that are suppressed by the price caps; and (2) even if 

price formation is efficient as in the Base Case, the energy-only market does not achieve an 

economic equilibrium reserve margin at the planning reserve margin (e.g., to achieve 0.1 LOLE) 

Futures Traded

 in April

Realized Real-Time 

Spot Prices 

Delta

($/MWh) ($/MWh) ($/MWh)

Summer 2011

June $45.94 $51.66 $5.72

July $55.71 $58.67 $2.96

August $55.71 $209.99 $154.27

September $42.79 $39.68 ($3.11)

Summer 2012

June $38.11 $38.22 $0.11

July $70.07 $31.50 ($38.56)

August $70.07 $34.98 ($35.09)

September $38.62 $29.02 ($9.59)
84 

that regulators may wish to maintain. This topic is the subject of current market reform efforts in 

Texas.138

Figure 30 

Energy Margins and “Missing Money” at Different Price Caps 

Notes: 

Average CT energy margins with price caps of $7,500/MWh (Base Case), $3,000/MWh, and $1,000/MWh. 

5. Implications of Varying System Conditions and Study Assumptions 

Figure 31 shows average CT energy margins for the Base Case and each of the alternative 

simulation scenarios that we have examined. Compared to the Base Case, large declines in 

energy margins would be realized under the lower price cap cases (as discussed above), or in any 

of the scenarios where the Study RTO is able to rely more heavily on imports from the 

neighboring regions with higher reserve margins. In these cases, under an energy-only market, 

the Study RTO would realize an economic reserve margin below minimum the 7% level that we 

have examined. However, note that in cases where energy margins are driven down by increased 

neighbor assistance, these lower realized internal reserve margins may not necessarily cause any 

additional reliability concerns since the Study RTO is also in a better position to rely on physical 

neighbor assistance during these events. 

 

138 See Newell, et al. (2012), Sections I.D-F. 
85 

The largest increases in CT energy margins are caused by reduced transmission interconnections 

with neighboring regions. In fact, the 50% Transmission and Island Cases would increase 

supplier returns sufficiently to increase the Study RTO’s equilibrium reserve margin to 12.3% 

and 16.5% respectively. 

Figure 31

Summary of Average CT Energy Margins for All Simulated Cases 

Notes: 

Average CT energy margins in the Base Case and all change case scenarios. 

Overall, comparison of these results shows that energy margins are heavily influenced by 

conditions experienced in neighboring regions, particularly when those neighbors are strongly 

interconnected. When the Study RTO is experiencing a severe shortage condition, it can 

partially alleviate the shortage and reduce energy prices by importing from neighboring systems. 

The price-moderating effect of such imports will be greater in more interconnected systems and 

neighboring systems with higher load diversity (so that scarcity does not happen simultaneously 

in all regions). However, the reverse can also be true. When the neighboring regions experience 

an extreme price spike, this will also attract imports from the Study RTO that will introduce 

scarcity-level prices in the Study RTO as well. 

The overall impact of neighboring regions on Study RTO prices and energy margins therefore 

depends on the characteristics of the outside system, the level of load diversity among the 

systems, and the strength of interconnections. For example, because the systems we examine are 

quite similar and have significant load diversity, increasing interconnections will generally tend 

to moderate prices. In select circumstances however, expanded interconnections could also 
86 

increase price volatility in the Study RTO if a neighboring region has a greater potential for 

severe price spikes and shortage conditions. This would generally be the case for neighboring 

regions that are hydro-dominated with the potential for extreme shortages during low-water 

conditions that occur only once every five or ten years.139

B. CAPACITY MARKETS

Bilateral and centralized capacity markets were first implemented in the U.S. in the late 1990s as 

a means of achieving the prevailing 1-in-10 resource adequacy standards within restructured 

markets. Restructured markets rely on voluntary supplier investments to sustain resource 

adequacy, with incremental investments attracted only when suppliers can anticipate operating 

margins sufficiently high on average to cover the plants’ fixed costs, recover investments, and 

earn a return equal to their cost of capital.140

The primary advantage of restructured markets that include a capacity market relative to energyonly markets is that the system operator is able to ensure that the administratively-determined 

planning reserve margins will be achieved. Depending on the policy objectives, these reserve 

margins may be set to achieve either: (a) the reliability objective such as 1-in-10, as is the case in 

the capacity markets of PJM, ISO-NE, and NYISO; or (b) an economic objective such as 

minimizing societal or customer costs, as in the proposed Italian capacity market.141 

1. Capacity Payments Required to Achieve Reliability Targets 

As discussed in Section IV.A above, energy-only markets by themselves will maintain some 

equilibrium level of investments, but will not necessarily achieve any particular administrativelydetermined reserve margin target. As explained in Section IV.A.4, the additional supplier 

returns required to achieve a particular target reserve margin is often referred to as “missing 

money”.142

 

139 This type of situation is one that we have previously examined in Alberta that despite little hydro resource 

of its own is exposed to the risk of low-hydro conditions via its interconnection with BC Hydro. See 

Pfeifenberger and Spees (2011a), Section III.F. 

140 Note that even restructured markets do not rely entirely on merchant entry for all new supply. There are 

usually at least a small proportion of new assets within each footprint that are built on a regulated basis. 

An example is public power entities that build for self-supply. However, we classify a market as 

“restructured” if it will require at least some merchant entry to maintain resource adequacy in the region. 

This is the case, for example, in PJM where most new entry must come from merchant investors, although 

some new entry will be built on a regulated basis such as by Dominion, by a number of municipalities and 

cooperatives, or under the direction of state entities as in a selection of contracts in New Jersey and 

Maryland. 

141 We note that these markets actually express the resource adequacy standard as a “demand curve” for 

capacity that covers a range of reserve margins, although the range is developed around the underlying 

reliability or economic objective as discussed further in Section IV.B.4 below. 

142 For a discussion of investment shortfalls and the “missing money” problem in energy-only markets, see 

Cramton and Stoft (2006), Joskow (2008), and Pfeifenberger, et al. (2009). Note, however, that not all 

energy-only markets suffer from a missing money problem, and the problem can be addressed through 

means other than capacity markets. See, for example, Pfeifenberger and Spees (2013), documenting 

Continued on next page 
87 

The missing money problem can be solved in deregulated markets by imposing resource 

adequacy standards on all LSEs. Such a resource adequacy standard requires LSEs to procure 

either sufficient generation or DR capacity to serve their own customers’ coincident peak load 

plus the specified mandatory reserve margin. Such an LSE-based resource adequacy standard is 

imposed in all capacity markets that cover primarily deregulated regions, such as in NYISO, 

ISO-NE, and PJM. It is also used in California and regions with primarily (but not exclusively) 

regulated utilities such as MISO.143 

Imposing a resource adequacy standard on LSEs automatically creates a bilateral market for 

capacity because it creates demand for installed capacity that is separate from the demand for 

energy and ancillary services. As a result, LSEs will have to pay for capacity to secure the 

needed obligations.144 Within centralized capacity markets—as is the case in PJM, ISO-NE, and 

NYISO—the RTO procures the needed capacity resources on behalf of all customers and then 

assigns procurement costs to individual LSEs in proportion to peak load during the delivery 

period. However, regardless of whether the capacity market is centralized or bilateral, the 

existence of an enforceable resource adequacy standard means that the system operator will 

always be able to achieve the specified planning reserve margin. 

The size of the capacity payments required to maintain the resource adequacy standard is a 

function of the target reserve margin as shown in Figure 32. Suppliers must anticipate being able 

to earn total operating margins equal to or greater than CONE before they will invest in the new 

resources. As explained in Section IV.A.1 above, energy margins exceed CONE at Study RTO 

reserve margins up to 7.9% (which corresponds to a 16.5% reserve margin in the combined foursystem area, see Table 7 above). This means that only if the anticipated Study RTO reserve 

margin dropped below 7.9% would suppliers invest without the prospect of earning additional 

capacity payments. This also means that the capacity price would be zero if the Study RTO 

imposed a minimum reserve margin requirement at or below 7.9%. 

For reserve margin requirements greater than 7.9% however, suppliers will not be willing to 

invest based on energy margins alone. At these higher reserve margins, suppliers must expect 

capacity payments equal to their Net CONE, which is the difference between their CONE and 

their anticipated energy margins. As shown in Figure 32 for our Base Case simulations, the 

equilibrium capacity price (red line) increases with the reserve margin requirement. This is 

because energy prices and suppliers’ energy margins decrease causing Net CONE to increase. 

 

Continued from previous page 

market prices that support sufficient investment in the Alberta energy-only market; and Newell et al.

(2012) discussing a range of options, with and without a capacity market, to address a missing money 

problem in the Texas energy-only market. 

143 For additional discussion of these capacity market designs as well as various alternative market designs for 

resource adequacy, see Spees, et al. (2013); and Pfeifenberger, et al (2009). 

144 In regions with primarily cost-of-service regulated utilities, most of these capacity commitments are 

determined through integrated resource planning (IRP) and then secured through self-supply or long-term 

PPAs for bundled energy and capacity. However, there are often still at least some additional transactions 

for capacity as a distinct product. For example, see an analysis of California’s short-term bilateral market 

for capacity in CPUC (2011), particularly in Section 4.3. See also Pfeifenberger, et al. (2012). 
88 

At the 15.2% reserve margin needed to maintain 0.1 LOLE, for example, an investor in a new 

CT would expect to earn approximately $40/kW-year in energy margins on average, while the 

total annualized investment and fixed cost (i.e., CONE) is $120/kW-year. That supplier would 

not voluntarily invest unless capacity prices were expected to be at least $80/kW-year on average 

over the economic life of the plant.145 By comparison, a capacity price of less than $30/kW-year 

would be needed to sustain the lower 10.3% reserve margin that is consistent with the riskneutral cost-of-service optimum as discussed above. 

Figure 32 

Equilibrium Capacity Price vs. Reserve Margin Requirement 

Notes: 

Equilibrium capacity price equal to CT Net CONE of Base Case simulation. 

2. Impact of Lower Energy Market Price Caps on the Capacity Market 

Because capacity prices must be equal to Net CONE over the long-run to sustain a particular 

reserve margin, any factors that cause anticipated energy margins to drop will also cause 

capacity prices to increase if the reserve margin is to be maintained. Figure 33 illustrates this 

effect for our Study RTO, by showing the equilibrium capacity price level for simulation cases 

where the energy price cap is: (a) set at the VOLL of $7,500/MWh as in the Base Case; 

(b) reduced to $3,000/MWh; and (c) reduced to $1,000/MWh. 

As explained in Section IV.A.4 above, reducing the price cap below VOLL or otherwise 

suppressing energy prices below marginal system costs will result in under-investment in an 

energy-only market design. This is not the case in regions with reserve margin requirements, 

 

145 In other words, Net CONE ($80/kW-year) = CONE ($120/kW-year) – Energy Margins ($40/kW-year). 
89 

because any reduction in energy margins will be offset by an increase in supplier compensation 

through increases in capacity prices. As Figure 33 shows, at any level of reserve margin, 

lowering the cap on energy prices significantly increases capacity prices. The impact of price 

caps is greater at lower reserve margins because scarcity conditions and associated price spikes 

are encountered more frequently at the lower margins. 

This interaction between energy margins and capacity market prices also means that regions with 

resource adequacy requirements and associated (bilateral or centralized) capacity markets will be 

able to achieve their reliability targets even if energy prices are artificially suppressed. This does 

not mean, however, that capped or otherwise suppressed energy prices have no consequences on 

market efficiency. Rather, overall market efficiency will be lower because the capped or 

otherwise lower energy prices will lead to inefficient investment decisions, inefficient hedging 

and forward contracting incentives, inefficient demand response, inefficient real-time 

performance incentives, and inefficient dispatch of resources. 

Figure 33 

Equilibrium Capacity Prices with Different Energy Market Price Caps 

Notes: 

Equilibrium capacity price is at CT Net CONE. 

 Calculated at price caps of $7,500/MWh (Base Case), $3,000/MWh, and $1,000/MWh. 

3. Cost-Minimizing Capacity Demand Curves 

The optimal, cost-minimizing reserve margin also depends on the levelized fixed costs of adding 

more capacity resources to the system. In estimating the cost-minimizing reserve margin in 

Sections III.A.2 and III.A.3 above, we assume levelized fixed costs equal to a CT CONE of 

$120/kW-year. In reality there is some uncertainty in CONE from an administrative perspective, 
90 

due to variability in construction costs, financing costs, and even technology specifications 

among potential investors. 

Figure 34 shows how total reliability-related costs change with varying CONE levels and reserve 

margins, and highlights the minimum average cost point with a red dot in each case. The figure 

shows total reliability-related costs as calculated in Section III.A.2 from a risk-neutral, cost-ofservice perspective. The figure also shows that changes in the assumed level of CONE shift the 

economically-optimal reserve margin by several percentage points. Varying CONE between 

$70/kW-year to $170/kW-year corresponds to cost-minimizing reserve margins that range from 

8.4% to 13.1%. 

Figure 34 

Cost-Minimizing Reserve Margin with Varying CT CONE 

(Risk-Neutral, Cost of Service Perspective) 

Notes: 

 Base Case reliability-related costs from a risk-neutral, cost-of-service perspective, see Section III.A.2. 

The fact that the economically-optimal reserve margin depends on CONE suggests that 

administratively-determined reserve margin requirements should also vary with CONE. In other 

words, it would be more efficient to maintain a higher reserve margin if additional capacity can 

be added at relatively low cost. Similarly, mandated reserve margins should be lower if new 

capacity is expensive. This economically-efficient approach to procuring capacity could be 

achieved in either cost-of-service regulated power markets or deregulated regions within a 

capacity market. It would require: (a) estimating the incremental value of capacity using 

probabilistic economic and reliability simulations (as we have done for the hypothetical Study 

RTO); and (b) comparing that value against the total cost of procuring capacity, which may be 

unknown in advance. 
91 

Figure 35 shows the value of incremental capacity as a function of reserve margins. We 

calculate the incremental capacity value as the reduction in total system costs (excluding capital 

costs) that would be achieved by adding incremental capacity. As the figure shows, this 

incremental value of capacity also depends on whether one applies a cost-of-service or societal 

perspective for calculating total system costs. We thus show capacity values for: (a) a riskneutral, cost-of-service perspective in a system with a price cap at the $7,500/MWh VOLL, as 

well as at the lower price caps of $3,000/MWh and $1,000/MWh; and (b) a risk-neutral societal 

perspective. The value of capacity is a function of price caps only under a cost-of-service 

perspective, because the lower price caps (which are applied in both Study RTO and neighboring 

regions) will reduce the cost of procuring emergency imports. From a societal perspective, 

however, the value of capacity does not depend on energy prices. It depends only on internal 

and external production and reliability-related costs, not on the market prices themselves. 

Figure 35 

Reduction in Expected System Costs Achieved by Adding Capacity 

(Excluding the Carrying Costs Required to Build the Incremental Capacity) 

Notes: 

 Capacity value is the reduction in system costs (excluding capital costs) achieved per unit of 

capacity added. 

 Cost-of-Service costs are calculated for the Base Case as well as the $3,000/MWh and $1,000/MWh 

price cap cases from a risk-neutral cost-of-service perspective as in Section III.A above. Societal 

costs are calculated for the Base Case on a risk-neutral basis as in Section III.A.3 and do not vary 

with price cap levels. 

 Plotted with x-axis based on the Study RTO internal reserve margin. Note that the 7.9% “socially 

optimum” reserve margin in the Study RTO is equivalent to a 16.5% reserve margin across all 

systems’ coincident peak loads, as explained in Table 7 in Section III.A.3. 
92 

Any time the incremental value of capacity (shown as downward-sloping lines), exceeds the 

incremental cost of new capacity, it would be efficient to procure more capacity. This 

comparison must consider the total procurement costs, including both capacity payments and net 

energy payments.146 Figure 35 compares these value-of-capacity curves for our Study RTO to 

the Base Case CONE value of $120/kW-year. The value of capacity is equal to CONE at the 

risk-neutral, economically-optimal reserve margin as calculated previously and indicated with 

red dots in the chart. These optimum reserve margins would increase if CONE were lower, as 

discussed above. 

Within a capacity market, one can translate this value-of-capacity calculation into a costminimizing demand curve for capacity by subtracting out the portion of total costs that suppliers 

would be able to cover through energy margins as shown in Figure 36. This demand curve for 

capacity would procure the quantity of capacity needed to minimize total system production, 

reliability, and capital costs, as calculated on a risk-neutral basis. These demand curves would 

need to be shifted to the right if policy makers or RTOs wished to incorporate some risk aversion 

into their procurement levels as discussed in Section III.A.4 above. Note, however, that the risk 

mitigation preferences of suppliers and customers might alternately be resolved through marketbased hedging and forward contracting, at least in efficient and competitive markets with liquid 

futures markets and forward contracting opportunities. 

The cost-minimizing demand curve from a societal perspective (left chart) is zero at all reserve 

margins in our Base Case where the price cap is set at VOLL. This is because, under the special 

circumstances of having perfectly efficient energy market prices, the energy-only market will be 

sufficient to attract the efficient quantity of capacity investments from a societal perspective as 

explained in Section IV.A.2. However, if the price cap is below VOLL or energy prices are 

otherwise suppressed, it would be efficient to procure additional capacity through a capacity 

market. The lower the price cap and the greater the price suppression, the higher the capacity 

demand curve would need to be to procure the socially-optimal quantity. 

From a risk-neutral cost-of-service perspective (right chart), the cost-minimizing demand curve 

is also a downward-sloping function with a very similar shape. As the chart shows, however, the 

height of the demand curve is not as directly related to the price cap as is the case for the 

socially-optimal demand curves. This is because the value of capacity is partly tied to the 

suppression of energy prices and associated reduction of net import costs. With a lower price 

cap, the value of higher reserve margins on reducing import costs is less (lowering the demand 

curve) but the effect is offset because suppliers’ energy margins would also be reduced 

(increasing the demand curve). These offsetting factors result in a demand curve that is not as 

heavily impacted by the price cap. 

 

146 In the context of a region with a capacity market, the total payments would be the sum of energy margins 

and capacity payments based on the market prices for energy and capacity. In a regulated utility setting 

with PPAs, the total procurement cost would be net energy payments plus capacity payments under the 

PPA, which may be broken down in various ways depending on contract terms. 
93 

Figure 36 

Risk-Neutral, Cost-Minimizing Demand Curve for Capacity 

Notes: 

 Calculated as the value of capacity from Figure 35 minus the energy margins from Figure 26. 

4. Current RTO Demand Curves Compared to Cost-Minimizing Curves 

Figure 37 compares the cost-minimizing demand curves calculated in the previous section to 

demand curves that are currently being used in the capacity markets in NYISO, ISO-NE, PJM, 

and MISO. We compare the demand curves that would be calculated under the Base and 50% 

Transmission Cases from the risk-neutral societal perspective under a $1,000/MWh price cap. 

We selected these cases to compare because we believe that these two simulation cases may 

most closely resemble these various real-world RTOs, but stress that the value-based demand 

curve would be quite different if estimated for any particular region. 

We plot the curves in two different ways: (1) with the x-axis defined as the Study RTO’s reserve 

margin compared against the Study-RTO non-coincident peak load (solid lines); and (2) with the 

x-axis defined as the combined four-system reserve margin against the combined systems’ 

coincident peak load (dotted lines). Further, we note that these risk-neutral demand curves 

would have to be shifted to the right if policy makers or RTOs wished to incorporate risk 

mitigation benefits into the demand curves. 

Not surprisingly, the cost-minimizing demand curves from a risk-neutral societal perspective 

have lower prices than the actual RTO demand curves when compared against the Study RTO 

reserve margin. This is primarily because these actual RTO demand curves were developed with 

the objective of achieving the 0.1 LOLE standard, while a risk-neutral societal approach would 

aim to procure a lower reserve margin and reliability level under our study assumptions. Actual 

RTO demand curves also reflect a number of other objectives and real-world considerations such 

as: (a) the objective of mitigating capacity price volatility; (b) the practical difficulties of 

administratively estimating demand curve parameters, including the approximate Net CONE at 
94 

the target reserve margin; and (c) the difficulty of building consensus on a capacity demand 

curve’s shape, given the multitude of stakeholders with disparate and competing interests that are 

involved in such market development efforts. 

While U.S. RTOs with capacity markets and their regulators have not yet demonstrated 

substantial interest in considering such a value-based approach to estimating demand curves 

(whether risk-neutral, or risk-adjusted), we note that Italy will soon implement a capacity market 

with a value-based demand curve that is at least conceptually similar to the approach that we 

have outlined in the previous section.147

Figure 37 

Cost-Minimizing Capacity Demand Curve vs. Current RTO Demand Curves 

Sources and Notes: 

 All curves converted from source units into ICAP terms for both reserve margin and price. 

 Cost-minimizing demand curves plotted with x-axis as: (1) the Study RTO reserve margin against Study-RTO 

non-coincident peak load (solid lines); and (2) against the combined four-system reserve margin against 

the combined coincident peak load (dotted lines). See Table 7 in Section III.A.3 for a more detailed 

explanation of this coincident vs. non-coincident reserve margin accounting convention. 

 See ISO-NE (2013); MISO (2012c, 2012d); NYISO (2013); PJM (2013a). 

 

147 Italy’s system operator Terna has submitted its proposal for calculating this value-based demand curve, as 

briefly outlined in Terna (2012), Sections 2.3-2.4. 
95 

C. IMPLICATIONS OF INCREASED DEMAND RESPONSE PENETRATION

Section IV.C examines the system cost and reliability implications of increasing levels of 

demand response penetration, assuming that increasing DR resources would displace traditional 

generation. Increasing DR penetration also has a number of important implications for energy 

and capacity market outcomes. Namely, as DR penetration increases and displaces generation 

resources, it will: (1) cause the energy supply curve to shift upward because lower-dispatch-cost 

CTs are being replaced by DR with higher dispatch cost, resulting in higher energy prices during 

peak load and scarcity conditions; (2) increase average generator margins due to the higher peak 

energy prices; and (3) reduce capacity market prices by increasing generators’ energy margins. 

We discuss these and related implications for market design in the following sections, and 

evaluate the relevance of these results to PJM’s market, which is the U.S. market that will be 

relying most heavily on DR resources over the coming years. 

1. Energy Market Impacts 

As DR penetration increases, it will displace traditional generation resources if the reserve 

margin requirement remains fixed. This will increase the supply curve and realized prices in the 

energy market, because lower-dispatch-cost generation resources will be displaced by higherdispatch-cost DR. If that DR is primarily “emergency” rather than “economic” DR it will have 

an even greater impact on the energy market because: (1) emergency DR is available only during 

emergency conditions (at a high strike price of $2,000/MWh assumed in our simulations), while 

economic DR is available over a wide range of dispatch prices, including a portion that is 

available at relatively low dispatch prices (a few hundred dollars in our simulations); and 

(2) emergency DR is a dispatch-limited resource (in our simulations it can only be dispatched up 

to 100 hours per year), resulting in even higher energy prices during any years when the dispatch 

limit is exceeded. 

Figure 38 illustrates these impacts by showing the price duration curve for the Base Case 

simulations and a 15% planning reserve margin and varying market penetration levels for 

emergency DR (left chart) and economic DR (right chart). For comparison, we also show the 

price duration curve from the Base Case simulations at the same 15% reserve margin.148 The 

distinction between emergency and economic DR is not particularly important at relatively low 

penetration levels of up to 5% (similar to historical experience), because the frequency with 

which the DR resource will be dispatched is quite low at these penetration levels. As explained 

in Section III.C.1 above, at the 5% penetration level, one would expect few or no DR calls in 

years with typical weather. However, with higher levels of DR penetration (darker blue lines), 

the entire price duration curve is shifted up, with the increase being substantially larger if the DR 

resources consist only of dispatch-limited emergency DR rather than economic DR. 

 

148 Recall that the Base Case simulation has a 6.9% DR penetration level, with DR resources consisting of 

80% emergency DR and 20% economic DR. 
96 

Figure 38 

Energy Price Duration Curve with Increasing DR Penetration 

(Average Year Results at Fixed 15% Reserve Margin) 

Notes: 

The average price duration curve at 15% reserve margin and varying levels of Emergency (left) or Economic 

(right) DR penetration. 

In our simulations, the high reliance on Emergency DR also has the effect of creating a “shelf” in 

the price duration curve at the assumed emergency DR strike price of $2,000/MWh.149 The shelf 

in energy market prices is an artifact of the high-dispatch-cost nature of Emergency DR. 

Emergency DR implicitly assumes that all DR resources are identical and that the underlying DR 

resources all have the same strike price. In reality, however, every individual DR asset will have 

its own strike price that could be above or below the $2,000/MWh assumed in our simulations. 

Indiscriminately dispatching Emergency DR assets as if they had a uniform strike price will have 

the problematic effect of calling on DR resources with high strike prices of several thousand 

dollars, when lower-cost economic DR and non-DR resources would likely be available. 

Additionally, DR resources with a strike price of only a few hundred dollars would prefer to be 

interrupted much more frequently by customers trying to avoid high energy prices. 

The Economic DR penetration simulation cases demonstrate generally preferable outcomes. 

Integrating high-levels of Economic DR results in a smoother, better-behaved price duration 

curve with no artificial “shelf” caused by an administratively-determined Emergency DR strike 

price level. This illustrates the efficiency gains from integrating economic DR resources into 

wholesale markets, whether they are directly dispatchable by the system operator or only 

 

149 Note that the shelf is somewhat smoothed by the fact that the plots show the average price duration curve 

over many simulation runs. In any individual run or delivery year the shelf would be an even more 

prominently “blocky” feature of the emergency DR curve. 
97 

responding to market prices. The market is more efficient overall because each DR resource has 

self-selected its own level and price of curtailments. For example, end users place relatively 

little value on service for at least a portion of their total load may respond to energy market 

prices and curtail demand at a relatively low strike price of $500/MWh. In our simulations, this 

DR resource would be curtailed approximately 40 times per year (or less than one hour per 

week) in the 20% penetration case, an interruption frequency that many end users may find 

acceptable for non-essential uses.150

To date, the consequences of relying heavily on Emergency DR have not yet been a major 

concern because Emergency DR is rarely, if ever, dispatched in most systems. This is because: 

(a) many U.S. systems have had modest DR penetration levels of only 6% to 7%; (b) many U.S. 

RTOs have an surplus generation capacity, which means Emergency DR would rarely be called 

even if the system had a high DR penetration level; and (c) energy prices are capped and rarely 

exceed a few hundred dollars in most systems, meaning that only DR resources with the very 

lowest strike prices have had any reason to participate in the energy market. 

This will change over the next years however, particularly in PJM, where high levels of DR have 

displaced generation for the next several delivery years. This will cause a substantial increase in 

DR dispatch over the coming years.151 Historically, and consistent with our Base Case 

assumptions, 80% of PJM’s DR assets have participated as emergency-type DR, while only 20% 

participated in the energy market. It seems very likely that, as energy prices and DR dispatch 

hours increase over the coming years, more DR will begin participating in the energy market 

because there will be increasingly more hours during which the “true” strike price of lower-cost 

DR resources will be exceeded.152 Resources with very high strike prices will not opt to 

participate directly in PJM’s energy market, however, because: (a) their strike prices may exceed 

the energy price cap, in which case they would never wish to be interrupted; or (b) their number 

of call hours is too low, so that the benefits of participating in the energy market would not 

justify the infrastructure or overhead costs of direct market participation. 

2. Impact on Generator Energy Margins 

Consistent with the increase in energy prices caused by displacing generation with DR resources, 

generator energy margins would increase as shown in Figure 39. The figure shows the average 

annual energy margins (red) as a function of DR penetration, along with the annual margins 

expected in the median year (blue), middle 50% of years (dark gray), and middle 90% of years 

 

150 However, even very high interruption rates may be tolerable if managed within a portfolio of resources by 

a DR aggregator, because an aggregator could limit the interruption rate of any individual DR asset by 

maintaining a high quantity of DR assets that exceeds its curtailment obligation under any one event. 

151 For an analysis of how DR displacing generation may increase DR call hours in PJM over the coming 

years, see Newell and Spees (2013). 

152 DR may participate directly by offering hourly bids into the energy market or, possibly more commonly 

for resources that wish to avoid the substantial effort required to submit hourly energy bids into the nodal 

market, these assets may submit a single energy strike price that is applicable all year and that would be 

used with slightly less locational dispatch precision. For an overview of PJM’s new scarcity pricing 

mechanism, including its approach to incorporating emergency and economic DR into price-setting, see 

PJM (2012b). 
98 

(light gray). Consistent with the observations about energy price impacts, relying more heavily 

on Emergency DR will increase generator energy margins more quickly than relying more 

heavily on Economic DR. Again, this is because Economic DR will offer into the energy market 

at a large range of prices, including at some relatively low prices of only a few hundred dollars. 

The smoother and better-behaved price formation associated with Economic DR also translates 

into substantially less volatility in year-to-year spot energy margins earned by suppliers. With 

Emergency DR, the 100 hour call limit and energy price “shelf” at the assumed administrativelyset level of $2,000/MWh result in a much larger difference in energy margins between moderate 

years (with few or no DR calls) and extreme years (with many DR calls all at the same high 

price). At the 15% Emergency DR penetration level, our simulations result in spot-market-based 

energy margins of $48/kW-year (41% of CONE) for the typical year but up to $334/kW-year 

(2.8 times CONE) once per decade. In comparison, CTs would experience lower (but much less 

volatile) energy margins in a market with 15% Economic DR penetration. For that case, our 

simulations yield spot energy margins of $18/kW-year (15% of CONE) for the typical year and 

as high as $152/kW-year (1.3 times CONE) once per decade. 

Figure 39 

Volatility in Energy Margins vs. Emergency DR (left) or Economic DR (right) Penetration 

(Fixed 15% Reserve Margin)

Notes: 

CT energy margins in the Study RTO when displacing CTs with increasing levels of DR penetration. 

 Gray lines represent the 10th, 25th, 50th, 75th, and 90th percentile estimates of energy margins. 
99 

3. Impact on Capacity Market 

The increase in energy margins with higher DR penetration will also have the long-run effect of 

decreasing equilibrium capacity prices, because total generator net energy and capacity revenues 

will have to equal CONE on average as explained in Section IV.B.1 above. This consequence is 

demonstrated in Figure 40 that shows the equilibrium capacity price at a 15% reserve margin 

with increasing Emergency DR penetration, increasing Economic DR penetration, and under our 

Base Case simulation. 

Capacity prices are the same with Emergency and Economic DR at the 0% penetration level, and 

decrease more quickly in the Emergency DR Case because energy market prices increase more 

quickly.153 In fact, at approximately 14% Emergency DR penetration, the capacity market price 

would be driven all the way to zero, with energy prices so high that traditional generators would 

be willing to build even if they had no capacity payments. Obviously, this outcome is 

economically infeasible because Emergency DR providers would need a capacity payment that is 

at least high enough to pay for their fixed annual costs and expected net interruption costs.154 In 

other words, if only Emergency DR existed, then the capacity market would converge to an 

equilibrium DR penetration level somewhere below 14%.155

If only Economic DR existed and at the quantities that we assume can actually be realized, then 

capacity prices would drop with increasing penetration levels but would not drop to zero over the 

penetration levels we examine. In that case, DR penetration would level off at the fixed costs of 

maintaining DR minus the net value of participating in the energy market.156 We do not attempt 

to quantify that equilibrium penetration level for our purposes, but note that the fixed costs of 

adding more DR will necessarily increase at higher penetration levels as the lowest-cost DR 

 

153 Note that in a real-world capacity market the difference between the Emergency and Economic DR Cases 

might be somewhat smaller because the RTO is likely to reduce the capacity credit awarded to Emergency 

DR at high penetration levels as it becomes obvious that the call-limited Emergency DR is providing 

substantially lower reliability value. This would mean that the Emergency DR would displace somewhat 

less generation capacity than the Economic DR, which would somewhat reduce the impacts on energy 

market prices, CT energy margins, and capacity margins. 

154 Note that DR suppliers have an annual “net interruption cost” that would be calculated very similarly to 

generator energy margins but can actually be positive or negative in total value. The value of the “net 

interruption cost” is the energy price when called minus the true strike price. If the energy price when 

called exceeds the DR asset’s true strike price then the interruption provides positive value to the DR 

supplier, who avoids paying energy prices in excess of their private value (note that for Economic DR the 

energy price always exceeds the true strike price when called and so participating in the energy market 

would always create positive value). In the alternate case when the energy price is below the DR asset’s 

true strike price, the interruption will impose a net cost on the DR supplier. For example, if a DR asset’s 

true strike price is $5,000/MWh, but the energy market price cap is $1,000/MWh (or the administrativelydetermined Emergency DR strike price is $2,000/MWh), then every interruption would impose a net cost. 

155 Penetration levels of Emergency DR would also be limited by the fact that call-limited DR has lower 

capacity value and would probably therefore be assigned lower capacity payments at these high 

penetration levels. PJM has already implemented such a mechanism for differentiating capacity payments 

among call-limited and unlimited DR products. For an explanation and discussion of PJM’s multiple DR 

products, see Pfeifenberger, et al. (2011a), Section VI.C.1. 

156 See footnote 154 for an explanation of the net value that a DR asset would earn in the energy market. 
100 

resources are already committed and only higher-cost DR options remain available for 

incremental procurement. 

Figure 40 

Equilibrium Capacity Prices with Increasing Levels of DR Penetration 

(Fixed 15% Reserve Margin) 

Notes: 

Equilibrium capacity price is at CT Net CONE. 

In real markets, most DR is currently Emergency DR, suggesting capacity and energy market 

impacts more similar to the Emergency DR case at low penetration levels. For example, in PJM 

approximately 80% is Emergency DR while approximately 20% is Economic.157 However, as 

total DR penetration levels increase, we believe it is likely that a greater proportion of DR will 

begin to participate in the energy market, driven by increasing energy prices as explained in 

Section IV.C.1. However, some Emergency DR will never migrate into the energy market 

because energy market prices would so infrequently exceed their true strike price (or may never 

exceed their true strike price if the energy market cap is far below VOLL).158 In other words, it is 

still plausible that even with a price cap that is somewhat below the likely VOLL, the 

 

157 See PJM (2013c), p. 5. 

158 Emergency DR would only migrate into the energy market if the net value of participating in the energy 

market exceeds the increase in fixed infrastructure and administrative costs incurred. 
101 

combination of PJM’s energy and capacity markets may be able to attract a relatively efficient 

portfolio of Economic and Emergency DR.159

D. COMPARISON OF CAPACITY AND ENERGY-ONLY MARKET DESIGNS

This section of our report provides a high-level comparison of realized energy and capacity 

prices, supplier net revenues, and customer costs under alternative energy-only and capacity 

market designs. We also compare simulation results with varying price caps and target reserve 

margin levels. These results reflect realized prices and costs under a restructured energy-only or 

capacity market in which suppliers earn market-based revenues to recovery their fixed costs and 

customers pay market prices for energy and capacity. 

1. Supplier Net Revenues 

Figure 41 shows supplier net revenues (i.e., total market-based revenues less operating costs) for 

a new CT from energy margins and capacity payments under the Base Case simulation over a 

range of reserve margins.160 As explained previously, in a restructured wholesale energy-only or 

capacity market, supplier net revenues must be equal to the fixed costs at CONE to sustain the 

investment needed for a particular equilibrium reserve margin. Under our simulations, we 

estimate 7.9% equilibrium reserve margin for the energy-only market design. At this 

equilibrium reserve margin, suppliers would earn net revenues that fully recover their investment 

and earn sufficient returns in the energy market. Policy makers would have the option to require 

a higher planning reserve margin, but doing so would necessitate a resource adequacy 

requirement that creates additional compensation through bilateral or centralized capacity 

markets. In the following discussion we focus on energy-only and centralized capacity market 

designs.161 

As planning reserve margin requirements increase, energy margins will decrease. This means 

that the proportion of supplier net revenues from capacity payments—needed to avoid underinvestment due to “missing money”—will increase at higher reserve margins. Capacity 

payments will, for example, need to cover more than 70% of CONE at the 17% reserve margin. 

For intermediate and baseload units, capacity revenues would be the same but represent a lower 

fraction of net revenues because those resource types would earn higher energy margins and 

have higher fixed costs. 

In other words, in equilibrium, supplier net revenues are the same in energy-only and capacity 

market designs and cover the long-run marginal cost of supply. However, the proportion of net 

 

159 With the primary inefficiency stemming from the lack of an efficient call order among resources whose 

strike price is near to or exceeds the energy market cap. PJM’s current cap on the energy component of 

the LMP is $1,500/MWh,as of October 2012 and will rise to $2,700/MWh by June 2015, see PJM 

(2012b), p. 162. 

160 Average energy margins and capacity prices are calculated as explained in Sections IV.A.1and IV.B.1 

above respectively. 

161 While other approaches to achieving higher planning reserve margins exist, we focus here only on energyonly and capacity market approaches for simplicity. For a broader discussion of alternative approaches, 

see Pfeifenberger, et al. (2009); and Spees, et al. (2013). 
102 

revenues from the energy and capacity markets is quite dependent on market design, reserve 

margin requirement, and other fundamentals that affect energy market prices. The proportion of 

supplier returns from the energy and capacity markets can also have substantial implications if 

one or the other of these markets is particularly exposed to regulatory risks or inefficiencies as 

discussed above. Further, any inefficiency in the capacity or energy only market designs could 

result in attracting or maintaining an inefficient mix of baseload, intermediate, and peaking 

resources. 

Figure 41 

Supplier Net Revenues as a Function of Planning Reserve Margin Requirements 

Notes: 

Study RTO Base Case results, see Sections IV.A.1and IV.B.1 for calculation details. 

Figure 42 similarly shows supplier net revenues from energy and capacity markets under 

different market designs, price caps, reserve margin targets, and DR penetration levels. Again, 

note that, to be able to attract investment, supplier net revenues are always equal to CONE in 

equilibrium, although the proportion from capacity versus energy margins varies with design.162 

The figure shows that a capacity market with a planning reserve margin requirement set at the 

risk-neutral, cost-of-service optimum would require that suppliers earn 27% to 56% of their net 

revenues from the capacity market (depending on the price cap), while the capacity market with 

a reserve margin requirement based on the 0.1 LOLE standard would require that 66% to 80% of 

 

162 Also note that the case with energy margins exceeding CONE indicates a disequilibrium that would attract 

more supply until reserve margins increased enough to decrease supplier net revenues until they equal 

CONE. 
103 

net revenues are earned in the capacity market. With higher levels of DR penetration, generation 

suppliers would earn a greater portion of their net revenues in the energy market. 

Figure 42 

Supplier Net Revenues Under Different Market Designs and System Conditions 

Notes: 

Study RTO results for Base, Price Cap, and DR Penetration Cases. 

2. Total Customer Costs 

Figure 43 shows the annual average of total customer costs as a function of planning reserve 

margin in the Base Case simulation, including transmission and distribution, energy, and 

capacity costs.163 Note that T&D costs do not change with the reserve margin, while energy 

costs decrease and capacity costs increase, consistent with the trends in market prices discussed 

above. The directional change in the proportion of customer costs incurred for energy and 

capacity are similar to the directional changes in supplier returns for the marginal CT, but the 

magnitudes are different because: (a) customer costs reflect total energy costs not just the abovecost portion represented by energy margins; and (b) customer costs reflect total energy and 

capacity payments to all types of resources other than just the marginal CT including 

 

163 T&D costs are based on a national average, from EIA (2013b).
104 

intermediate and baseload resources whose total returns are more focused on energy than 

capacity.164

Total customer costs are minimized at the risk-neutral, “societally-optimal” reserve margin of 

7.9%, consistent with the discussion in Section IV.A.2 above. Unlike individual suppliers’ 

equilibrium net revenues, customer costs vary with the reserve margin even in equilibrium 

conditions. This is because the total system cost increases with planning reserve margins and the 

total generation base that must earn sufficient net revenues increases. However, the increase in 

total customer costs with reserve margin is quite small as a percent of total costs, making the 

increase difficult to see when total costs are shown to scale. In fact, increasing the reserve 

margin above the risk-neutral, societal optimum of 7.9% to the 15.2% reserve margin needed 

under the 0.1 LOLE standard increases average customer costs by $1.63/MWh or only 1.5%. 

Importantly, we note that the incremental customer cost imposed by implementing a capacity 

market relative to an energy-only market cannot be calculated simply by estimating the final 

capacity price and multiplying by the procured quantity. This is because increasing the reserve 

margin with a capacity market not only introduces capacity costs but also reduces energy-related 

costs, a factor which is not considered in many discussions. 

Figure 43 is also useful for answering the deceptively simple question of whether energy-only or 

capacity markets are more expensive for customers. The real answer is that total customer costs 

have less to do with energy-only versus capacity market design, but more to do with the 

equilibrium reserve margin. Whether through energy-only or capacity market design, total 

capacity costs must be sufficient to cover the capital and operating costs of the fleet in 

expectation. The exact reserve margin achieved depends on market design details, with: (a) the 

capacity market reserve margin determined by the planning reserve margin target, and (b) the 

energy-only market reserve margin determined by the level of market prices (which may be 

higher or lower than the “optimum” depending on whether energy prices are more often above or 

below the perfectly efficient level). 

 

164 However, in a true, perfect equilibrium, the Net CONE of all resource types would be equal even though 

energy margins will be different. See Section IV.A.2 for additional discussion. 
105 

Figure 43 

Total Customer Costs and Spot Market Exposure for Different Planning Reserve Margins 

Notes: 

Study RTO Base Case results, with customers paying market prices for energy and capacity. 

 Sensitivity bars shows annual customer cost averaged for 10% of the most expensive years, assuming 

full exposure to spot energy market uncertainty (i.e., no LSE risk mitigation through seasonal or 

longer-term forward contracting). 

Average T&D costs are a 2011 U.S. national average, see EIA (2013b). 

This modest increase in average annual customer costs also achieves a risk mitigation benefit. 

As indicated by Figure 43, at the 7.9% planning reserve margin annual customer costs are likely 

to rise as high as $177/MWh once in ten years if those customers are fully exposed to spot 

market uncertainty.165 This once-per-decade cost exposure is 68% higher than the average 

expected cost. At the higher 15.2% planning reserve margin, the once-per-decade spot market 

cost exposure is only $140/MWh, or 31% higher than the average. In other words, increasing 

planning reserve margins from 7.9% to 15.2% increases the average annual costs by $1.63/MWh 

but decreases the once-per-decade spot market cost exposure by $38/MWh or by approximately 

36% of the total bill. As discussed earlier, however, this comparison will overstate the risk 

mitigation benefit of higher planning reserve margins because a large portion of all spot market 

uncertainties can be hedged readily by LSEs even on a relatively short-term basis, such as

through forward contracting for an upcoming summer season. 

 

165 This expected cost is calculated as the average expected cost of the top 10% of the most expensive energy 

price years, i.e., the expected cost of the most extreme year in any particular decade. All percentages 

reported are calculated as a percent of the total customer bill in the average year at the 7.9% reserve 

margin. 
106 

Figure 44 similarly shows average customer costs and once-per-decade spot market exposure 

under different energy-only and capacity market designs, planning reserve margins, price caps, 

and DR penetration levels. Again, the directional changes in energy and capacity costs mirror 

the changes observed for supplier net revenues in the previous section. As noted above, the 

annual average of total customer costs can change by 1% to 2% under different planning reserve 

margins and price caps. However, an unhedged customer’s exposure to high-cost years would be 

much higher at lower reserve margins, with higher energy market price caps, and with higher DR 

penetration levels (particularly if that DR is Emergency DR rather than Economic DR). Also 

note that average annual customer costs in our simulations are as much as 5% lower at a high 

20% economic DR penetration level, due to the reduction in overall generating capacity needs in 

combination with lower capacity prices that would result if such a substantial resource base of 

relatively low-cost DR were available. 

Figure 44 

Average Annual Customer Costs for Different Market Designs and System Conditions 

Notes:

Study RTO Base, Price Cap, and DR Penetration Cases results, with customers paying market prices. 

 Sensitivity bar shows annual customer cost averaged for 10% of the most expensive years, assuming full 

exposure to spot energy market uncertainty (i.e., no LSE risk mitigation through seasonal or longerterm forward contracting). 

 Average T&D costs are a 2011 U.S. national average, see EIA (2013b). 
107 

V. CONCLUSIONS 

As we more fully explain in the Executive Summary, this study documents a wide range of 

accounting and methodological conventions that make reserve margins and resource adequacy 

assessments very difficult to compare across regions. Regions that appear to adopt the same 

1-in-10 resource adequacy standard may in fact apply very different standards. For example, we 

demonstrate that in a hypothetical system under identical methodological assumptions, the 1-in10 standard may require a planning reserve margin anywhere from 8.2% to 19.2% depending on: 

(a) whether 1-in-10 is interpreted as 0.1 LOLE or 2.4 LOLH; and (b) whether only load-shedding 

constitutes a reliability event or whether voltage reductions and operating reserves shortages may 

also be classified as “events.” 

Relatively few North American regions currently evaluate the economic implications of their 

resource adequacy standards, with the exceptions being several entities in SERC. Evaluating the 

economics of resource adequacy standards is a difficult modeling challenge and involves a 

complex set of questions and tradeoffs among total cost, reliability, and risk of extreme events. 

However, unless these questions are explicitly evaluated, policy makers will not be in a position 

to make informed choices among their competing economic and reliability objectives. 

In our illustrations for a hypothetical system, we show that, depending on how a regulator’s 

economic objectives are defined, the “optimal” reserve margin may be 7.9% from a societal 

perspective, 10.3% from a cost-of-service perspective, or possibly higher if regulators 

incorporate risk mitigation benefits into their decision-making. In this and other studies, we 

demonstrate that the likelihood of extreme reliability and wholesale price spike events can be 

substantially mitigated by increasing reserve margins with only a modest increase in customer 

costs. For example, increasing the reserve margin from the 7.9% “socially optimal” level to the 

15.2% required to maintain 0.1 LOLE increases customer costs by approximately 1.5%. 

In implementing their resource adequacy objectives, policy makers may use a number of 

different planning or market-based approaches. We specifically evaluate the economics of 

resource adequacy under energy-only and capacity market designs. As has been theoretically 

discussed elsewhere, we demonstrate through our modeling results that a perfectly efficient 

energy-only market will achieve the socially optimal reserve margin in expectation. However, 

we caution that no market design is perfect and so energy-only markets may be susceptible to 

either: (a) under-investment, if wholesale prices are artificially suppressed by low price caps or 

insufficient administrative scarcity pricing measures; or (b) over-investment, if wholesale prices 

are kept artificially high by factors such as sustained exercise of market power. Further, even if 

an energy-only market achieves a socially optimal level of investments, it may or may not 

achieve other policy objectives such as maintaining resource adequacy as defined by the 1-in-10 

standard. 

If regulators of restructured power markets wish to maintain a reserve margin higher than what 

an energy-only market will likely support, they may implement a resource adequacy standard 

within a competitive capacity market. Under a capacity market design, system operators can 

achieve their resource adequacy target and suppliers will earn sufficient revenues from the 

capacity market to attract the desired level of new entry. This is true even if wholesale energy 

market prices may be artificially suppressed, for example during scarcity conditions. 
108 

Overall, average customer costs are not substantially impacted by market design in the long term, 

except to the extent that different market designs may sustain different reserve margin levels 

over the long term. Customer costs are much more heavily impacted by fundamental factors 

such as fuel prices, load shape, and the cost and mix of supply resources. In particular, 

incorporating substantial quantities of DR resources into wholesale capacity and energy markets 

can increase the efficiency of those markets and also reduce customer costs to the extent that 

substantial quantities of low-cost DR opportunities exist. 
109 

LIST OF ACRONYMS 

1-in-10 One-Day-In-Ten-Years 

AESO Alberta Electric System Operator 

APC Adjusted Production Costs 

APS Arizona Power Service 

APSC Arizona Public Service Company 

BPA Bonneville Power Administration 

CAISO California ISO 

CBM Capacity Benefit Margin 

CBO Congressional Budget Office 

CC Combined Cycle 

CEC California Energy Commission 

CEER Council of European Energy Regulators 

CONE Cost of New Entry 

CPUC California Public Utilities Commission 

CRS Congressional Research Service 

CT Combustion Turbine 

CVaR Conditional Value at Risk 

DR Demand Response 

EC European Commission 

EFORd Expected Forced Outage Rate Data 

EIA Energy Information Administration 

ELCC Effective Load Carrying Capability 

ENS Energy-Not-Served 

EOP Emergency Operating Procedure 

ERCOT Electric Reliability Council of Texas 

ETIP Energy Technology Innovation Policy 

EWITS Eastern Wind Integration Transmission Study 

EUE Expected Unserved Energy 

FERC Federal Energy Regulatory Commission 

FOM Fixed Operations and Maintenance 

FPSC Florida Public Service Commission 

FRCC Florida Reliability Coordinating Council 

GADS Generation Availability Data System 

GDP Gross Domestic Product 

GE-MARS General Electric – Multi-Area Reliability Simulation 
110 

GEN Generation 

GTRPMTF Generation and Transmission Reliability Planning Models Task Force 

HR Hour 

ICAP Installed Capacity 

IESO Independent Electricity System Operator 

IRM Integrated Resource Management 

IRP Integrated Resource Plan 

ISO Independent System Operator 

ISO-NE ISO New England 

kW Kilowatt 

kW-y Kilowatt Year 

kW-yr Kilowatt Year 

kWh Kilowatt Hour 

KU Kentucky Utilities Company 

LFG Landfill Gas 

LGE Louisville Gas and Electric Company 

LOLE Loss of Load Event 

LOLEWG Loss of Load Expectation Working Group 

LOLH Loss of Load Hours 

LOLP Loss of Load Probability 

LTA Long-Term Adequacy 

LTRA Long-Term Reliability Assessment 

MAPP Mid-Continent Area Power Pool 

MARS Multi-Area Reliability Simulation 

MATS Mercury and Air Toxics Standard 

MIS Mission: Integrated Systems (MIS Energy Management) 

MMbtu Million Metric British Thermal Units 

MISO Midcontinent, formerly Midwest, ISO 

MW Megawatt 

MWh Megawatt Hour 

NEM National Energy Market 

NERC North American Electric Reliability Corporation 

NOAA National Oceanic and Atmospheric Administration 

NPC Nevada Power Company 

NPCC Northeast Power Coordinating Council 

NREL National Renewable Energy Laboratory 

NRRI Natural Resource Research Institute 
111 

NWPP Northwest Power Pool 

NYISO New York ISO 

NYSRC New York State Reliability Council 

PJM PJM Interconnection 

PNM Public Service Company of New Mexico 

PNW Pinnacle West Capital Corporation 

POLL Probability of Lost Load 

PPA Power Purchase Agreement 

PRISM Parameter-Elevation Regressions on Independent Slopes Model 

PRM Planning Reserve Margin 

PtP Point-to-Point 

PV Photovoltaic 

RES Regulatory and Economic Studies 

RFC ReliabilityFirst Corporation 

RM Reserve Margin 

RPM Reliability Pricing MODEL 

RTO Regional Transmission Organization 

RWG Resource Working Group 

SCE&G South Carolina Electric & Gas Company 

SCR Special Case Resource 

SERC SERC Reliability Corporation 

SERVM Strategic Energy Risk Valuation Model 

SoCo Southern Company 

SPP Southwest Power Pool 

SWIS South West Interconnected System 

T&D Transmission and Distribution 

TVA Tennessee Valley Authority 

UCAP Unforced Capacity 

USE Unserved Energy 

VOLL Value of Lost Load 

VOM Variable Operations and Maintenance 

WECC Western Electricity Coordinating Council 
112 

BIBLIOGRAPHY 

Alberta Electric System Operator (2008). Long Term Adequacy Metrics, Threshold and Threshold Actions 

Recommendation Paper. February 7, 2008. Retrieved from 

http://www.aeso.ca/downloads/Long_Term_Adequacy_Feb_7.pdf 

Alberta Electric System Operator (2012). “Short Term Adequacy and Supply Shortfall,” ID # 2012-006 (R). 

Retrieved from http://www.aeso.ca/downloads/ID_2012-

006R_Short_Term_Adequacy_and_Supply_Shortfall.pdf 

Alberta Electric System Operator. (2013) “Long Term Adequacy Metrics,” May 2013. Retrieved from 

http://www.aeso.ca/downloads/2013_05_LTA.pdf 

Energy Information Administration (2012). Annual Energy Outlook 2012.June 25, 2012. Retrieved from 

http://www.eia.gov/forecasts/archive/aeo12/ 

Arizona Public Service Company (2012). Integrated Resource Plan. March 2012. filed in Compliance with 

A.A.C. $R14-2-703. Retrieved from http://www.aps.com/_files/various/

ResourceAlt/2012ResourcePlan.pdf 

Australia Energy Market Commission (2007). Comprehensive Reliability Review: Final Report. December 

2007. Retrieved from http://www.aemc.gov.au/market-reviews/completed/comprehensive-reliabilityreview.html 

Australia Energy Market Commission (2010). Reliability Standard and Reliability Settings Review. April 30, 

2010. Retrieved from http://www.aemc.gov.au/Media/docs/Final%20Report%20-

%20Reliability%20Standard%20and%20Settings%20Review-c3994458-70b3-4d66-badd-dc7e8b809f68-

1.PDF 

Bonneville Power Administration (2011). Pacific Northwest Loads and Resources Study. DOE/BP-4354, May 

2011. Retrieved from http://www.bpa.gov/power/pgp/whitebook/2011/

WhiteBook2011_SummaryDocument_Final.pdf 

California Public Utilities Commission and the California Energy Commission (2005). Resource Adequacy 

Phase 2 Workshop Report. June 10, 2005. Retrieved from 

http://docs.cpuc.ca.gov/published//REPORT/46914.PDF 

California Public Utility Commission (2011a). 2010 Resource Adequacy Report. April 22, 2011. Retrieved 

from http://www.cpuc.ca.gov/PUC/energy/Procurement/RA/ 

Carden, Kevin, Nick Wintermantel, and Johannes Pfeifenberger (2011). The Economics of Resource 

Adequacy Planning: Why Reserve Margins Are Not Just About Keeping the Lights On. April 2011. 

Retrieved from http://www.nrri.org/pubs/electricity/NRRI_resource_adequacy_planning_april11-09.pdf 

Centolella, Paul A. (2010). “An Essential Attribute: Facilitating A Transition to Efficient Markets.” PJM Long 

Term Capacity Issues Symposium. January 26, 2010. Retrieved from 

http://www.pjm.com/~/media/committees-groups/stakeholder-meetings/ltci/20100126/20100126-panel-1-

centolella-puc-ohio.ashx 

Congressional Budget Office (2012). The Budget and Economic Outlook: Fiscal Years 2012 to 2022. January 

2012. Retrieved from http://www.cbo.gov/sites/default/files/cbofiles/attachments/01-31-2012_Outlook.pdf 

Congressional Research Service (2012). U.S. Energy: Overview and Key Statistics. April 11, 2012. Retrieved 

from http://www.fas.org/sgp/crs/misc/R40187.pdf 

Council of European Energy Regulators (2010). Guidelines of Good Practice on Estimation of Costs due to 

Electricity Interruption ad Voltage Disturbances. December 7, 2010. Retrieved from http://www.energyregulators.eu/portal/page/portal/EER_HOME/EER_PUBLICATIONS/CEER_PAPERS/Electricity/2010/C

10-EQS-41-03_GGP%20interuptions%20and%20voltage_7-Dec-2010.pdf 
113 

Cramton, P., and Stoft, S. (2006). “The Convergence of Market Designs for Adequate Generating Capacity 

with Special Attention to the CAISO’s Resource Adequacy Problem.” Retrieved from 

http://web.mit.edu/ceepr/www/publications/workingpapers/2006-007.pdf. 

Data 360 (2013). “GDP United States.” Accessed May 2013 at 

http://www.data360.org/dsg.aspx?Data_Set_Group_Id=230 

Duke Energy Carolinas (2012). The Duke Energy Carolinas Integrated Resource Plan: Annual Report. Filed 

September 4, 2012 with the North Carolina Utility Commission. Docket No. E-100, Sub 137. Retrieved 

from http://ncuc.commerce.state.nc.us/cgi-bin/webview/senddoc.pgm? 

dispfmt=&itype=Q&authorization=&parm2=KBAAAA05221B&parm3=000139578 

ECCO International Inc. (2013). 2012 ERCOT Loss of Load Study: Study Results. March 11, 2013. Retrieved 

from 

http://www.ercot.com/content/meetings/gatf/keydocs/2013/0322/07_pres_ercot_loss_of_load_study_repor

t_part_2.pdf 

Electric Reliability Council of Texas (2012). “Special WMS Meeting: LOLE Study and ETIP.” October 25, 

2012. Retrieved from http://www.ercot.com/calendar/2010/10/20101025-WMS 

Electric Reliability Council of Texas (2013a). “Results of the 2012 Loss of Load Study,” Generation 

Adequacy Task Force, January 18, 2013. Retrieved from 

http://www.ercot.com/content/meetings/gatf/keydocs/2013/0118/Loss_of_Load_Study_2013_GATF.ppt 

Electric Reliability Council of Texas (2013b). “Capacity Demand and Reserves Report,” May 2013. Retrieved 

from http://www.ercot.com/news/presentations/ 

Energy Information Administration (2012). Annual Energy Outlook 2012. June 2012. Retrieved from 

http://www.eia.gov/forecasts/archive/aeo12/ 

Energy Information Administration (2013a). “Form EIA-923: Power Plant Operations Report,” Retrieved from 

http://www.eia.gov/electricity/data/eia923/ 

Energy Information Administration (2013b). “Table 2.4 Average Retail Price of Electricity to Ultimate 

Customers by End-Use Sectors 2011 Through 2011.” Retrieved from 

http://www.eia.gov/electricity/data.cfm#sales 

European Commission (2012). Consultation Paper on Generation Adequacy, Capacity Mechanisms, and the 

Internal Market in Electricity. Retrieved from 

http://ec.europa.eu/energy/gas_electricity/consultations/doc/20130207_generation_adequacy_consultation

_document.pdf 

Faruqui, Ahmad, and Sanem Sergici (2010). “Household Response to dynamic Pricing of Electricity: A Survey 

of 15 Experiments,” Journal of Regulatory Economics. Vol. 38, No. 2. October 2010. 

Faruqui, Ahmad, and Sanem Sergici (2011). “Dynamic Pricing of Electricity in the Mid-Atlantic Region: 

Econometric Results from the Baltimore Gas and Electric Company Experiment.” Journal of Regulatory 

Economics. August 2011. 

Faruqui, Ahmad, Sanem Sergici, and Lamine Akaba (2013). “Dynamic Pricing of Electricity for Residential 

Customers: The Evidence from Michigan.” Energy Efficiency. February 1, 2013. 

Federal Energy Regulatory Commission (2012). “Order Denying Rehearing and Accepting Compliance 

Filing,” with respect to the calculation of Tie Benefits in ISO-NE. Issued March 15, 2012. Docket Nos. 

ER11-2580-001, -002. 

Federal Energy Regulatory Commission (2013). “Form 714: Annual Electric Control and Planning Area 

Report.” Data retrieved from http://www.ferc.gov/docs-filing/forms.asp 

Florida Reliability Coordinating Council, Inc. (2012). FRCC 2012 Load & Resource Reliability Assessment 

Report. FRCC-RE-PL-002-r0. July 10, 2012. Retrieved from 

https://www.frcc.com/Reliability/Shared%20Documents/FRCC%20Reliability%20Assessments/
114 

FRCC%202012%20Load%20and%20Resource%20Reliability%20Assessment%20Report%20RE%20PC

%20Approved%20071012.pdf 

Georgia Power Company (2010). 2010 Integrated Resource Plan. Docket No. 31081 Georgia Public Service 

Commission, Document Filing #125981. January 29, 2010. Retrieved from 

http://www.psc.state.ga.us/factsv2/Document.aspx?documentNumber=125981 

Henton, Darcy (2012). “Beyond the Blackouts.” Calgary Herald, August 11, 2012. Retrieved from: 

http://www2.canada.com/calgaryherald/news/story.html?id=4f44ada3-7dc4-4881-959d-526367515708. 

Hydro-Québec (2011). Québec Balancing Authority Area Comprehensive Review of Resource Adequacy: Final 

Report. October 27, 2011, approved by NPCC RCC on November 29, 2011. Retrieved from 

https://www.npcc.org/Library/Resource%20Adequacy/Qu%C3%A9bec%20Comprehensive%20Review%

202011.pdf 

Independent Electricity System Operator (2012). “Ontario Reserve Margin Requirements 2012–2016.” Issue 

1.0. IESO_REP_0707. February 23, 2012. Retrieved from 

http://www.ieso.ca/imoweb/pubs/marketReports/Ontario-Reserve-Margin-Requirements-2012-2016.pdf 

ISO New England (2011). New England 2011 Comprehensive Review of Resource Adequacy: Final Report. 

November 29, 2011. Retrieved from https://www.npcc.org/Library/Resource%20Adequacy/NE_2011_

Comprehensive_Review_of_Resource_Adequacy%20-%20RCC%20Approval%20-%2020111129.pdf 

ISO New England (2012), Installed Capacity Requirement, Local Sourcing Requirements, and Maximum 

Capacity Limit for the 2015/16 Capability Year. February 2012. Retrieved from http://www.isone.com/genrtion_resrcs/reports/nepool_oc_review/2012/icr_2015_2016_report_final.pdf 

ISO New England (2013). “Forward Capacity Market (FCM).” Accessed May 2013 from http://www.isone.com/markets/othrmkts_data/fcm/index.html 

ISO/RTO Council (2009). 2009 State of the Markets Report. Retrieved from 

http://www.isorto.org/atf/cf/%7B5B4E85C6-7EAC-40A0-8DC3-

003829518EBD%7D/2009%20IRC%20State%20of%20Markets%20Report.pdf 

ISO/RTO Council (2010). “2010 North American Wholesale Electricity Demand Response 2010 

Comparison.” May 17, 2010. Retrieved from 

http://www.isorto.org/site/apps/nlnet/content2.aspx?c=jhKQIZPBImE&b=2708737&ct=8400541 

Joskow, Paul L (2008). “Capacity Payments in Imperfect Electricity Markets: Need and Design.” Utilities 

Policy, 16:1590170, 2008. 

Kahn, Edward P. (2004). “Effective Load Carrying Capability of Wind Generation: Initial Results with Public 

Data,” The Electricity Journal. Volume 17, Issue 10. December 2004. Pp. 85-89. 

LG&E and KU Energy LLC. (2011). Generation Planning & Analysis: Analysis of Supply-Side Technology 

Alternatives. March 2011, with Recommendations in PSC Staff Report on Last IRP, Case No. 2008-

00148. Filed with the Kentucky Public Service Commission on April 21, 2011. Retrieved from 

http://psc.ky.gov/pscscf/2011%20cases/2011-00140/20110421_LG&E-KU_IRP_Volume%20III.pdf 

Manitoba Hydro (2010). 2010/11 Power Resource Plan. September 24, 2010. Retrieved from 

http://www.hydro.mb.ca/regulatory_affairs/electric/gra_2010_2012/Appendix_84.pdf 

Midwest Independent Transmission System Operator (2011). Planning Year 2012 LOLE Study Report. 

November 2011. Retrieved from 

https://www.midwestiso.org/Library/Repository/Study/LOLE/2012%20LOLE%20Study%20Report.pdf 

Midwest Independent Transmission System Operator (2012a). Planning Year 2013-2014 Wind Capacity 

Credit. December 2012. Retrieved from 

https://www.midwestiso.org/Library/Repository/Study/LOLE/2013%20Wind%20Capacity%20Report.pdf 

Midwest Independent Transmission System Operator (2012b). Comments on “Capacity Deliverability Across 

the Midwest Independent Transmission System Operator, Inc/PJM Interconnection, L.L.C. Seam” 
115 

Submitted August 10, 2012 before the Federal Energy Regulatory Commission. Docket No. AD12-16-

000. 

Midwest Independent Transmission System Operator. (2012c) Planning Year 2013 LOLE Study Report. 

November 1, 2012. Retrieved from 

https://www.midwestiso.org/Library/Repository/Study/LOLE/2013%20LOLE%20Study%20Report.pdf 

Midwest Independent Transmission System Operator. (2012d) Business Practice Manual: Resource 

Adequacy: Planning Years 2013 and Beyond. Manual No. 011. Effective October 1, 2012. Retrieved from 

https://www.midwestiso.org/Library/BusinessPracticesManuals/Pages/BusinessPracticesManuals.aspx 

Midwest Independent Transmission System Operator (2013a). “Market Reports.” Retrieved from 

https://www.misoenergy.org/Library/MarketReports/Pages/MarketReports.aspx 

Midwest Independent Transmission System Operator (2013b). “The Next Generation of MISO Resource 

Adequacy Enhancements: Increasing Forward Transparency for Better Assuring Regional Adequacy and 

Enabling Efficient Planning,” Markets Committee Meeting. April 24, 2013. Retrieved from 

https://www.midwestiso.org/Library/Repository/Meeting%20Material/Stakeholder/BOD/Markets%20Co

mmittee/2013/20130424/20130424%20Markets%20Committee%20of%20the%20BOD%20Item%2005%

20Resource%20Adequacy.pdf 

Milligan, Michael (1997). “Wind Plant Capacity Credit Variations: A Comparison of Results Using Multiyear 

Actual and Simulated Wind-Speed Data,” Presented at Windpower ’97. June 15-18, 1997. Retrieved from 

http://wind.ucdavis.edu/rpsintegration/library/NREL-CP-440-23096%20Jun97%20Milligan.pdf 

Milligan, M. and K. Porter (2005). Determining the Capacity Vale of Wind: A Survey of Methods and 

Implementation. May 15, 2005. Retrieved from 

http://www.nerc.com/docs/pc/ivgtf/milligan_porter_capacity_paper_2005.pdf 

Munasinghe, M. and A. Sanghvi (1988). “Reliability of Supply, Outage Costs and Value of Service: An 

Overview,” The Energy Journal. Volume 9, Special Issue on Electricity Reliability. 

National Oceanic and Atmospheric Administration (2013). “Climate Data Online: Text & Map Search.” 

Retrieved from http://www.ncdc.noaa.gov/cdo-web/ 

National Renewable Energy Laboratory (2013a). “Eastern Wind Integration Transmission Study.” Data 

Retrieved from http://www.nrel.gov/electricity/transmission/eastern_wind_dataset.html 

National Renewable Energy Laboratory (2013b). “PVWatts,” Retrieved from http://rredc.nrel.gov/

solar/calculators/pvwatts/version1/ 

Nevada Power Company (2012). “Volume 4 of 24, IRP Testimony,” Pre-Filed Direct Testimonies before the 

Public Utilities Commission of Nevada, on June 8, 2012. Retrieved from 

https://www.nvenergy.com/company/rates/filings/IRP/NPC_IRP/images/Vol_4.pdf 

New Brunswick System Operator, Nova Scotia Power Incorporated, Maritime Electric Company Limited, and 

Northern Main ISA, Inc. (2011). NPCC 2011 Maritimes Area Interim Review of Resource Adequacy. 

September 2011, approved at November 29, 2011 RCC Meeting. Retrieved from 

https://www.npcc.org/Library/Resource%20Adequacy/RCC%20Approved%202011%20Maritimes%20Ar

ea%20Interim%20Resource%20Adequacy%20Review%20for%20TFCP.pdf 

Newell, Samuel A., Kathleen Spees, Johannes P. Pfeifenberger, Robert S. Mudge, Michael DeLucia, Robert 

Carlton (2012). ERCOT Investment Incentives and Resource Adequacy. June 1, 2012. Retrieved from 

http://www.brattle.com/_documents/UploadLibrary/Upload1047.pdf 

Newell, Samuel A., and Kathleen Spees (2013). “Get Ready for Much Spikier Energy Prices: The UnderAppreciated Market Impacts of Displacing Generation with Demand Response,” February 7, 2013. 

Retrieved from http://www.brattle.com/_documents/UploadLibrary/Upload1117.pdf 
116 

New York State Reliability Council, LLC. (2011). New York Control Areas Installed Capacity Requirements 

for the Period May 2012–April 2013: Technical Study Report. December 2, 2011. Retrieved 

fromhttp://www.nysrc.org/pdf/Reports/2012%20IRM%20Final%20Report.pdf 

New York Independent System Operator. (2013) “ICAP Data and Information,” Accessed March 2013 from 

http://www.nyiso.com/public/markets_operations/market_data/icap/index.jsp 

Nordel (2009). Guidelines for the Implementation of Transitional Peak Load Arrangements: Proposal of 

Nordel. March 2009. Retrieved from 

http://www.nordpoolspot.com/Global/Download%20Center/TSO/entsoe_nordic-guidelines-forimplementation-of-transitional-peak-load-arrangements.pdf 

North American Electric Reliability Corporation (2008). “Standard MOD-004-1: Capacity Benefit Margin.” 

Adopted November 13, 2008. Retrieved from http://www.nerc.com/files/MOD-004-1.pdf 

North American Electric Reliability Corporation (2010a). Final Report on Methodologies and Metrics - 

September and December, 2010 with Approvals and Revisions. Retrieved from 

http://www.pjm.com/~/media/committees-groups/subcommittees/raas/20100907/20100907-item-06bgtrpmtf-final-report.ashx 

North American Electric Reliability Corporation (2010b). G&T Reliability Planning Models Task Force 

(GTRPMTF): Methodology and Metrics, Revised October 15, 2010. Retrieved from 

http://www.wecc.biz/committees/StandingCommittees/PCC/LRS/01062011/Lists/Minutes/1/GTRPMTF%

20Final%20PC-approved%20Methodology%20and%20Metrics%20redline%2029Oct10%20mods.pdf 

North American Electric Reliability Corporation (2011a). 2011 Long-Term Reliability Assessment. November 

2011. Retrieved from http://www.nerc.com/files/2011LTRA_Final.pdf 

North American Electric Reliability Corporation (2011b). “2006-2010 Generating Unit Statistical Brochure – 

Units Reporting Events,” August 16, 2011. Retrieved from http://www.nerc.com/page.php?cid=4|43|47 

North American Electric Reliability Corporation. (2012) 2012 Long-Term Reliability Assessment. November 

2012. Retrieved from 

http://www.nerc.com/pa/RAPA/ra/Reliability%20Assessments%20DL/2012_LTRA_FINAL.pdf 

Pfeifenberger, Johannes, Kathleen Spees, and Adam Schumacher (2009). A Comparison of PJM’s RPM with 

Alternative Energy and Capacity Market Designs. Prepared for PJM Interconnection, Inc. September 

2009. Retrieved from http://www.brattle.com/_documents/UploadLibrary/Upload807.pdf 

Pfeifenberger, Johannes and Kathleen Spees (2011a). “Evaluation of Market Fundamentals and Challenges to 

Long-Term System Adequacy in Alberta’s Electricity Market.” April 2011. Retrieved from 

http://www.brattle.com/_documents/UploadLibrary/Upload943.pdf 

Pfeifenberger, Johannes, Samuel A. Newell, Kathleen Spees, Attila Hajos, and Kamen Madjarov (2011b). 

“Second Performance Assessment of PJM's Reliability Pricing Model” August 26, 2011. Retrieved from 

http://www.brattle.com/_documents/UploadLibrary/Upload972.pdf 

Pfeifenberger, Johannes P. And Samuel A. Newell (2011c). “Trusting Capacity Markets: Does the Lack of 

Long-Term Pricing Undermine the Financing of New Power Plants?” Public Utilities Fortnightly.

December 2011. Retrieved from http://www.fortnightly.com/fortnightly/2011/12/trusting-capacity-markets 

Pfeifenberger, Johannes P., Kathleen Spees, and Samuel A. Newell (2012). Resource Adequacy in California: 

Options for Improving Efficiency and Effectiveness. October 2012. Retrieved from 

http://www.brattle.com/_documents/UploadLibrary/Upload1088.pdf 

Pfeifenberger, Johannes P., Kathleen Spees, and Michael DeLucia (2013). Evaluation of Market Fundamentals 

and Challenges to Long-Term System Adequacy in Alberta’s Electricity Market: 2013 Update. March 

2013. Retrieved from http://www.brattle.com/_documents/UploadLibrary/Upload1123.pdf 

PJM Interconnection, LLC. (2007). PJM Reserve Requirement Study. August 15, 2007. Retrieved from 

http://www.pjm.com/~/media/documents/reports/2007-pjm-reserve-requirement-study.ashx 
117 

PJM Interconnection, LLC. (2010). Manual 21:Rules and Procedures for Determination of Generating 

Capability. Revision 9. Effective May 1, 2010. Retrieved from 

http://www.pjm.com/~/media/documents/manuals/m21.ashx 

PJM Interconnection, LLC. (2011). 2011 PJM Reserve Requirement Study: 11-year Planning Horizon: June 

1st 2011–May 31st 2022. September 29, 2011. Retrieved from http://www.pjm.com/~/media/committeesgroups/subcommittees/raas/20110929/20110929-2011-pjm-reserve-requirement-study.ashx 

PJM Interconnection, LLC. (2012a). Capacity Shortages, PJM State and Member Training Dept., October 9, 

2012. Retrieved from: http://www.pjm.com/~/media/training/core-curriculum/ip-ops-101/ops-101-

capacity-shortages.ashx 

PJM Interconnection, LLC. (2012b). “Shortage Pricing,” September 20, 2012. Retrieved from 

http://www.pjm.com/~/media/markets-ops/energy/shortage-pricing/shortage-pricing-training.ashx 

PJM Interconnection, LLC. (2012c). 2012 PJM Reserve Requirement Study. October 5, 2012. Retrieved from 

http://www.pjm.com/~/media/planning/res-adeq/2012-pjm-reserve-requirement-study.ashx 

PJM Interconnection, LLC. (2013a). “RPM Auction User Information.” Accessed May 2013 from 

http://www.pjm.com/markets-and-operations/rpm/rpm-auction-user-info.aspx 

PJM Interconnection, LLC. (2013b). “Historical Metered Load Data.” Retrieved from 

http://www.pjm.com/markets-and-operations/ops-analysis/historical-load-data.aspx 

PJM Interconnection, LLC. (2013c). “2013 Load Response Activity Report,” May 2013. Retrieved from 

http://www.pjm.com/~/media/markets-ops/dsr/2013-dsr-activity-report-20130510.ashx 

PJM Interconnection, LLC. (2013d). Manual 13: Emergency Procedures. Revision 52. February 1, 2013. 

Retrieved from http://www.pjm.com/~/media/documents/manuals/m13.ashx 

PJM Interconnection, LLC. (2013e). “Summer Outlook,” Retrieved from 

http://www.pjm.com/~/media/planning/rtep-dev/2013-summer-outlook.ashx 

Poland, William B. (1988). “The Importance of Including Uncertainties in Economic Generation Reliability 

Planning,” The Energy Journal. Volume 9, Special Issue on Electricity Reliability. 

Progress Energy. (2012). Progress Energy Carolinas Integrated Resource Plan. Filed September 4, 2012 

before the North Carolina Utility Commission. Docket No. E-100, Sub 137. Retrieved from 

http://ncuc.commerce.state.nc.us/cgi-bin/webview/senddoc.pgm?dispfmt=&itype=Q&authorization=

&parm2=BAAAAA45221B&parm3=000139578 

Public Service Company of New Mexico (2011). Integrated Resource Planning Department, Electric 

Integrated Resource Plan 2011–2030. July 2011. Retrieved from http://www.pnm.com/regulatory/

pdf_electricity/irp_2011-2030.pdf 

Sellin, Reece (2012). “Rolling Blackouts Spawn Storm of Debate over Electricity Industry,” Over Easy, 

Please, July 13, 2012. Retrieved from: http://www.overeasyplease.com/oep/2012/07/13/rolling-blackoutsspawn-storm-of-debate-over-electricity-industry/ 

South Carolina Electric & Gas Company (2011). 2011 Integrated Resource Plan. Filed before the Public 

Service Commission of South Carolina. Retrieved from 

http://www.energy.sc.gov/publications/SCEG2011IntegratedResourcePlan.pdf 

Southwest Power Pool (2010). 2010 Loss of Load Expectation Report. October 8, 2010. Retrieved from 

http://www.spp.org/publications/LOLE%20Report_5%20Draft_cc.pdf 

Southwest Power Pool (2012a). Benefits for the 2013 Regional Cost Allocation Review. July 5, 2012. SPP 

Metrics Task Force. Retrieved from 

http://www.spp.org/publications/20120717%20Metrics%20Task%20Force%20Report.doc 

Southwest Power Pool (2012b). 2011 Probabilistic Assessment. January 11, 2012. Retrieved from 

http://www.spp.org/publications/SPP%202011%20Probabilistic%20Assessment%20Final.pdf 
118 

Spees, Kathleen, Samuel A. Newell, Robert Carlton, Bin Zhou, and Johannes P. Pfeifenberger (2011). Cost of 

New Entry Estimates for Combustion-Turbine and Combined-Cycle Plants in PJM. August 24, 2011. 

Retrieved from http://www.brattle.com/_documents/UploadLibrary/Upload971.pdf 

Spees, Kathleen, Samuel A. Newell, and Johannes P. Pfeifenberger (2013). “Capacity Markets: Lessons 

Learned from the First Decade,” Economics of Energy and Environmental Policy. Fall 2013 Issue 

(forthcoming). 

Sullivan, Michael J., Matthew Mercurio, and Josh Schellenberg (2009). Estimated Value of Service Reliability 

for Electric Utility Customers in the United States. June 2009. Retrieved from http://certs.lbl.gov/pdf/lbnl2132e.pdf 

Tennessee Valley Authority (2011). Integrated Resource Plan: TVA’s Environmental & Energy Future. March 

2011. Retrieved from http://www.tva.com/environment/reports/irp/pdf/Final_IRP_complete.pdf 

Terna. (2012) Schema di Proposta di Disciplina del Sistema di Remunerazione della Disponibilitá di Capacitá 

Produttiva di Energia Elettrica. Submitted before the Autoritá per L’energia Elettrica e il Gas. 

Ventyx. (2012) Energy Velocity Suite. Data downloaded over January – April 2013. 

Western Electricity Coordinating Council (2011). 2011 Power Supply Assessment. November 17, 2011. 

Retrieved from http://www.wecc.biz/library/WECC%20Documents/Publications/

Power%20Supply%20Assessments/2011%20Power%20Suppy%20Assessment.pdf 

Wilson, James F. (2010a). “Reconsidering Resource Adequacy, Part 1: Has the one-day-in-10-years criterion 

outlived its usefulness?” Public Utilities Fortnightly, April 2010. 

Wilson, James F. (2010b). “Reconsidering Resource Adequacy, Part 2: Capacity planning for the smart grid,” 

Public Utilities Fortnightly, May 2010. 

Wiser, Ryan, and Mark Bolinger (2005). Balancing Cost and Risk: The Treatment of Renewable Energy in 

Western Utility Resource Plans. August 2005. Retrieved from 

http://emp.lbl.gov/sites/all/files/JOURNAL%20ARTICLE%20lbnl%20-%2058450.pdf 
APPENDICES 

In these appendices, we provide supplemental information underlying the general discussion 

regarding: (a) our survey of resource adequacy standards in place across North America, as 

summarized in Section I.C above; and (b) our simulation assumptions for analyzing the 

reliability and economic implications of resource adequacy, as summarized in Section II above. 

These details are provided in Appendices A and B respectively. 

A. DETAIL ON SURVEY OF NORTH AMERICAN RESOURCE ADEQUACY CRITERIA 

B. DETAIL ON SIMULATION MODELING AND ASSUMPTIONS
A-1 

A. DETAIL ON SURVEY OF NORTH AMERICAN RESOURCE ADEQUACY CRITERIA 

In this Appendix, we report additional detail from our survey of resource adequacy criteria used 

in U.S. and Canadian power systems, as summarized in Section I.C above. However, as noted 

above, to completely understand the nuances and complexities of these studies, one would need 

to discuss implementation details with the individuals responsible for implementing the studies. 

The public documentation of these studies is often insufficiently detailed or can easily be 

misinterpreted. Because we have not conducted such interviews to be able to document reliably 

the assumptions and the complexities of each study, our discussion should be interpreted as a 

summary of general industry practices, not a fully-verified documentation of any one region’s 

approach. 

1. Resource Adequacy Standards Used Across North America 

Table 14 is a summary of resource adequacy standards across U.S. and Canadian power systems, 

as discussed in Section I.C.1 above. 

Table 14 

Survey of Resource Adequacy Criteria Across U.S. and Canadian Power Systems 

Region Standard Model Notes 

PJM(a) 0.1 LOLE PRISM and 

GE-MARS 

The LOLE based target reserve margin and various other calculations provide key inputs into 

the PJM capacity market.

MISO(b) 0.1 LOLE GE-MARS Performed Annually by the ISO. Regional reserve margin of 16.7% but after diversity allows 

its load serving entities to carry an 11.3% reserve margin. 

NYISO(c) 0.1 LOLE GE-MARS Resulted in a reserve margin of 16.1% for the period May 2012 to April 2013. Reserve 

Margin calculation includes nameplate of all resources including wind. Results are adapted 

to derated UCAP for implementation in the NYISO capacity market. 

ISO-NE(d) 0.1 LOLE GE-MARS 2012 ICR report calculates the requirement needed to meet its 1 day in 10 year standard, load 

uncertainty considers weather but not economic forecast error. Results used capacity market. 

SPP(e) 2.4 LOLH ABB Grid 

View Capacity margin criterion of 12% for RTO members that are steam based and 9% for hydro 

based; results in capacity margin criterion above the 1 day in 10 year definition. 

Maritimes (f) 20% RM and 0.1 

LOLE 

NPCC uses 

MARS 

Maritimes uses a 20% reserve margin criterion for planning purposes but at the same time 

adheres to the NPCC requirement of not shedding firm load more than 1 day in 10 years. 

Quebec(g) 0.1 LOLE NPCC uses 

MARS 

Based on an LOLE of 0.1, Quebec requires a 10% reserve margin for the 2012/2013 winter 

peak. By the 2015/2016 winter peak, Quebec requires a 12.2% reserve margin. Because of 

its dependence on hydro generation, Quebec also imposes an energy requirement to withstand

2 consecutive years of low water inflows. 

IESO(h) 0.1 LOLE NPCC uses 

MARS 

The target for 2013 to meet the one day in 10 year target is 19.7% in which the region meets 

easily with an anticipated reserve margin of 40.1%. 

Saskatchewan(i) EUE Standard Sask Power uses a 13% RM based on probabilistic analysis of Expected Unserved Energy. 

Manitoba(j) Both RM and 

energy standards 

due to hydro 

dependence 

 The energy criterion requires adequate energy resources to supply firm energy demand in the 

event that the lowest recorded coincident river flow conditions are repeated. The capacity 

reserve margin is at least 12%. 

MAPP(i) 1 day in 10 years 

(LOLE of 0.1) 

 Some MAPP members self-impose a planning reserve margin of 15% based on the results of 

an LOLE study performed in 2009. 

SERC/ 

General 

No mandatory 

requirement 

 RA targets set by individual load serving members subject to regulatory review. With this 

approach, the criteria and final reserve margins vary across the region. 

SERC/SoCo(k) Economics SERVM The target is based on minimizing customer costs. 
A-2 

Region Standard Model Notes 

SERC/Duke 

Energy 

Carolinas(l) 

0.1 LOLE and 

Economic 

Assessment 

SERVM Set minimum RM based on LOLE values but base target RM on an economic assessment, 

which is slightly higher than the LOLE target. 

SERC/ 

Progress 

Energy 

Carolinas(m) 

1 day in 10 years 

(LOLE of 0.1) 

and Economic 

Assessment 

SERVM Set minimum RM based on LOLE values but base target RM on an economic assessment, 

which is slightly higher than the LOLE target. 

SERC/TVA(n) Economics SERVM The target is based on minimizing customer costs. 

SERC/Santee 

Cooper(o) 

Economics SERVM The target is based on minimizing customer costs. 

SERC/ 

LGE&KU(p) 

Economics SERVM The target is based on minimizing customer costs. 

SERC/ 

Entergy(i)

1 day in 10 years 

(LOLE of 0.1) 

ERAILS 

SERC/ 

SCE&G(q) 

12–18% RM 

FRCC(r) 0.1 LOLE Tiger “The FRCC has a resource criterion of a 15% minimum Regional Reserve Margin based on 

firm load. The FRCC assesses the upcoming ten-year summer and winter peak hours on an 

annual basis to ensure that the Regional Reserve Margin requirement is satisfied. Since the 

summer of 2004, the three Investor Owned Utilities (Florida Power & Light 

Company, Progress Energy Florida, and Tampa Electric Company) are currently maintaining 

a 20% minimum Reserve Margin planning criterion, consistent with a voluntary stipulation 

agreed to by the FPSC. Other utilities employ a 15% to 18% minimum Reserve Margin 

planning criterion.” 

ERCOT(s) 0.1 LOLE target 

(not mandatory) 

Internal 

Model 

ERCOT operates as an energy-only market and so does not mandate a RM; but performs one 

day in 10 year standard assessment to inform ERCOT and 

WECC/ 

General(t) 

No mandatory 

requirement 

 Individual balancing areas within WECC determine their own resource adequacy 

requirements in various ways and are subject to review by state regulators 

CAISO(u) 15% RM In January 2004, the CPUC established a long-term Resource Adequacy framework (D.04-01-

050). This decision adopted a 15% to 17% planning reserve margin (PRM) and directed that 

each LSE is responsible for acquiring sufficient reserves to meet its own customer 

load. CAISO has since performed LOLE studies but the studies have not impacted the 

decision made in 2004 to maintain at a minimum 15% reserve margin 

Northwest/ 

BPA(v) 

Loss-of-Load 

Probability 

(LOLP) of 5%; 

and conditional 

value at risk 

(CVaR) to 

evaluate energy 

not served (ENS) 

events 

Genesys 

Model 

A completely different method from 1 day in 10 years. Method was developed in cooperation 

with the Northwest Council to take into account the predominantly hydro resource mix of the 

Northwest. For this use, LOLP is not defined as hours per year. It is instead a percentage of 

iterations that contain any EUE. The target allows no more than 5% of all iterations to 

contain EUE. 

Southwest/ 

APS(w) 

0.1 LOLE APS 2012 IRP states that at 15% planning reserve margin criterion, LOLE is less than 1 day 

in 10 years. 

Southwest/ 

PNW(x) 

NM State 

Commission set 

target at 13% 

 Notes that reserve margin would likely increase if a one day in 10 year standard were used. 

Southwest/ 

NV Energy(y) 

1-in-10 Definition of 1 day in 10 years is not reported. 

Alberta No RA 

requirement 

 Intervention possible if expected EUE over a two-year outlook increases above 1,600 MWh. 

Sources: 

 From regional resource adequacy studies: (a) PJM (2011); (b) MISO (2011); (c) NYSRC (2011); (d) ISO-NE (2011); (e) SPP (2010); (f) 

NBSO (2011); (g) Hydro-Québec (2011); (h) IESO (2012); (i) NERC (2011a); (j) Manitoba Hydro (2010); (k) Georgia Power (2010); (l) 

Duke (2012); (m) Progress (2012); (n) TVA (2011); (o) 2012 IRP, forthcoming; (p) LG&E and KU (2011); (q) SCE&G (2011); (r) FRCC

(2012); (s) ERCOT (2012); (t) WECC (2011); (u) CPUC and CEC (2005); (v) BPA (2011); (w) APS (2012); (x) PNM (2011); and (y) 

Nevada Power (2012). 
A-3 

2. Illustration of Differences in Resource Adequacy Modeling Assumptions 

Table 15 shows an illustrative comparison of select model assumptions as used in various 

regions’ reliability planning studies, as discussed in Section I.C.3 above. 

Table 15 

Summary Comparison of Select Regions’ Reliability and Resource Adequacy Studies 

ISO-NE PJM NYISO MISO SPP SoCo 

Reliability 

Event 

Definition 

Maintain 200 MW 

of operating 

reserves when 

shedding firm 

load; voltage 

reduction and 

demand resources 

are deployed 

before shedding 

firm load. 

Firm load shed 

after deploying 

interruptible and 

depletion of 30 

minute reserves, 

but before voltage 

reduction or 

depletion of 10 

minute reserves. 

Firm load shed 

after depletion of 

30 and 10 minute 

reserves, and 

calling voltage 

reduction. 

Emergency 

Operating 

Procedures also 

anticipate some 

benefit from 

voluntary 

curtailment and 

public appeals. 

Firm load after all 

operating reserves 

are depleted and 

system is just able 

to sustain 

frequency with no 

regulation. No 

mention of voltage 

control. DR is 

removed from 

load before 

simulation and is 

therefore treated 

as deployed. 

Not explicitly 

defined. 

Before firm load 

shed, all voltage 

reduction options 

are exhausted. 

Firm load will be 

shed to maintain 

the full operating 

reserve 

requirement 

including 

spinning and 

non-spinning 

reserves. 

Reserve 

Margin 

Accounting 

Reliability 

requirement 

reported both 

including and 

excluding portion 

of tie benefits; 

intermittent 

resource values 

derated. 

Nameplate 

capacity for 

thermal resources; 

derate wind and 

solar according to 

expected 

reliability benefit. 

Does not include 

voltage reduction. 

Nameplate 

capacity for all 

resources; does 

not include 

voltage 

reduction. 

Nameplate 

capacity for 

thermal resources 

but derate wind. 

Nameplate 

capacity for 

thermal 

resources; derate 

wind and solar 

according to 

expected 

reliability 

benefit. Does not 

include voltage 

reduction. 

Voltage 

reduction treated 

as a supply 

resource. Derate 

interruptible and 

intermittent 

resources 

according to their 

expected 

reliability 

benefit. 

Neighbor 

Assistance 

Uses a tie benefit 

of 1,676 MW that 

is modeled as a 

resource; different 

treatment of firm 

and non-firm 

imports. 

Model external 

areas with 

diversity of 

weather and unit 

performance, up to 

a maximum 

assumed non-firm 

import capability 

of 3,500 MW. 

Model external 

areas with same 

load shape 

during peak 

events. 

Model 7 external 

regions at their 

planning reserve 

margin levels. 

Neighbors provide 

assistance until all 

operating reserves 

are depleted. 

Assumes 9,000 

MW of import 

capability. 

Assume 0 MW 

from outside 

areas. 

Model 

neighbors’ load 

and resource 

diversity. 

Model GE-MARS GE-MARS and 

PRISM 

GE-MARS GE-MARS ABB Grid View SERVM 

Sources: 

ISO-NE (2012); PJM (2011); NYSRC (2011); MISO (2012c); SPP (2010); and Georgia Power (2010). 
B-1 

B. DETAIL ON SIMULATION MODELING AND ASSUMPTIONS 

The most important modeling assumptions implemented in our study as well as a general 

description of how these assumptions are implemented within SERVM are documented in 

Section II within the body of this report. We supplement that more general discussion within 

this Appendix by providing additional supporting detail on our assumptions regarding: (1) 

intertie availability; (2) forced and maintenance outage modeling for thermal resources; (3) 

hydro resource modeling; (4) intermittent resource modeling; and (5) weather uncertainty and 

load modeling. 

1. Interregional Tie Line Availability 

As explained in Sections II.A-B, SERVM uses a transmission-constrained economic dispatch to 

schedule power flows among regions. We simulate the availability of each intertie over a 

probability distribution between 60% and 100% of the path rating. For simplicity, we assume 

that the interties are simultaneously feasible and that the availability of any one intertie is 

independent of the others’ availability. Figure 45 shows the non-simultaneous import capability 

of each path, while Figure 46 shows the simultaneous import capability of the Study RTO from 

all directions. The maximum simultaneous import capability to the Study RTO Region is 11,000 

MW, although it is less than 8,000 MW in 50% of all hours. 

Figure 45 

Non-Simultaneous Tie-Line Availability Distributions 
B-2 

Figure 46 

Simultaneous Study RTO Import Capability 

2. Thermal Resource Forced and Maintenance Outage Assumptions 

We provide here additional detail regarding how SERVM captures the probability of forced and 

maintenance outages for thermal resources. Average resource availability is consistent with 

NERC Effective Forced Outage Rate (EFORd) data from 2006–2010 by unit type and size.166 

Unlike most production cost and reliability models, SERVM does not use an average outage rate 

for each unit to approximately capture unit availability. Instead, SERVM simulates unit outages 

using random event draws from representative probability distributions. We developed these 

distributions based on previous modeling work, and scaled these distributions to match the 

national average outage rates. We also vary the outage rates seasonally to capture different 

operating profiles in summer, shoulder, and winter months. 

 

166 From NERC (2011b). 
B-3 

Specifically, outage events are characterized by the probability distributions around the 

following variables for full, partial, and maintenance outages, with the fleet-average values of 

key parameters as summarized in Table 16: 

Full Outages are characterized by distributions of: (1) Time-to-Repair Hours; and (2) Timeto-Fail Hours.

Partial Outages are characterized by distributions of: (1) Partial Outage Time-to-Repair 

Hours; (2) Partial Outage Derate Percentage; and (3) Partial Outage Time-to-Fail Hours.

Maintenance Outages are identified based on the percentage of full outages that have some 

scheduling flexibility. These outages are scheduled in off-peak periods to the extent 

feasible. 

To further illustrate the outage modeling approach in SERVM, consider a unit with assumed 

distributions for Time-to-Repair after an event, Time-to-Fail between events, and the other 

variables listed above. If the unit is online in hour 1, SERVM will randomly draw Time-to-Fail 

values from the distributions for both full outages and partial outages. The unit will run for that 

amount of time before failing. A partial outage will be triggered first if the selected Time-to-Fail 

value is less than the selected full outage Time-to-Fail value. Next, the SERVM will draw a 

Time-to-Repair value from the distribution and be on forced outage for that number of hours. 

When the repair is complete, it will draw new partial-outage and full-outage Time-to-Fail values. 

The process repeats until the end of the simulated year, when it will begin again for the 

subsequent iteration. This detailed multi-state stochastic modeling captures the tails of the 

distribution over potential low-probability, high-impact reliability outcomes that a simple 

convolution method would not capture. 

Table 16 

Fleet-Average Outage Characteristics by Unit Type 

Sources and Notes: 

Represents fleet average values, weighted by units’ nameplate capacity. 

 Mean time to repair and mean time to fail values exclude partial outages. 

Forced outage data from the Generation Availability Data System (GADS), NERC (2011b). 

Unit Type Forced Outage 

Rate

Mean Time to 

Repair

Mean Time to 

Fail

(%) (hours) (hours)

Nuclear 3.0% 146 6,454

Coal 6.8% 95 2,087

Biomass/LFG 8.4% 120 2,295

Gas Combined Cycle 5.0% 68 2,018

Gas Steam Turbines 7.0% 104 2,223

Steam Oil 6.2% 108 2,691

Gas Combustion Turbines 7.8% 47 551

Oil Combustion Turbines 11.8% 60 451

Overall Fleet 6.3% 92 2,443
B-4 

3. Hydro Modeling 

We model the hydro portfolio in each region based on segments treated as non-schedulable run 

of river, schedulable peak shaving hydro, and dispatchable emergency capacity. The run-of-river 

segment is dispatched as baseload capacity providing its designated capacity every hour of the 

year. The schedulable hydro is used for shaving the daily peak load, subject to minimum daily 

flow requirements. The emergency capacity is used only to prevent firm load shed, meaning that 

SERVM allows the emergency mode to “borrow” up to a few hours’ worth of energy from future 

dispatch of the scheduled hydro output. Typically, hydro resources cannot dispatch at their 

nameplate capacity during peak hours due to water supply limitations or river flow requirements. 

By modeling hydro resources in these three segments, SERVM captures the likelihood that 

incremental hydro resources will be available during peak and emergency conditions. 

Using FERC data on historical monthly hydro generation, we prepared resource availability 

profiles for the same regions and historical weather years used in our load modeling.167 In years 

with little historical hydro generation, SERVM models correspondingly little generation and 

capacity available during peak conditions. In years with substantial historical generation, 

SERVM models a substantial portion of the hydro fleet in run-of-river status with additional 

dispatch flexibility during peak conditions. Extreme shortage events in which high load and low 

hydro conditions coincide are modeled in SERVM only to the extent that such conditions have 

been observed over the historical weather and hydro years that we examine. 

Figure 47 shows the distribution of monthly hydro generation for each modeled region across the 

range of weather scenarios modeled. The figure also shows that not all modeled regions have the 

same level of reliability in their hydro resource, with Neighbor 1 having the most consistent 

hydro resource and Neighbor 3 realizing the most frequent low-hydro conditions. 

 

167 Data from EIA (2013). 
B-5 

Figure 47 

Hydro Energy Variation by Month and Region 

Sources and Notes: 

Data adapted from actual systems’ monthly hydro data as reported in EIA (2013). 

4. Intermittent Resource Modeling 

As explained in Section II.C.3 above, wind and solar resources are modeled as non-dispatchable 

intermittent resources. For wind resources, we use hourly EWITS data from years 2004–2006.168 

We aggregate these data by state, consistent with the weather stations we used in our load 

modeling to develop three years’ worth of hourly wind profiles. SERVM randomly draws a date 

from within the month being simulated, and uses the daily wind profile from each region 

consistent with that date in order to maintain the appropriate inter-regional correlations in wind 

output. For example, if SERVM is currently simulating the month of July, it may randomly 

select the date July 5, 2006 as the basis for the wind profiles from all four regions. 

As demonstrated in Figure 48 for Regions 1 and 2 (based on EWITS wind profiles for central 

MISO and western PJM), there is some correlation in hourly wind output among neighboring 

regions, but this correlation is small compared to the total volatility in wind output. Consistent 

with the geography of weather and hydro conditions, we used aggregate EWITS wind profiles 

for Tennessee (with an annual capacity factor 31%) for the Study RTO. 

We assume that solar capacity has a low penetration level in all regions that we model and so this 

resource has minimal impact on study results. We use PV Watts data to develop a typical hourly 

 

168 See NREL (2013a). 
B-6 

annual profile for each region.169 Similar to wind, SERVM randomly draws a daily profile by 

month for the aggregated solar generation in each region. 

Figure 48 

Correlation in Hourly Wind Output in Neighbors 1 and 2 During Summer Peak Hours 

Sources and Notes: 

Data adapted from EWITS data for central MISO and western PJM, see NREL (2013a). 

5. Treatment of Weather Uncertainty in Load Modeling 

This section provides additional detail on how weather uncertainty is modeled, as summarized in 

Section II.D.1 above. Table 17 lists the weather stations used to represent the Study RTO and 

three neighboring systems. 

 

169 From NREL (2013b). 
B-7 

Table 17 

Weather Stations Used to Represent Each Modeled System 

Modeled System Weather Stations

Study RTO Chattanooga, TN 

Nashville, TN 

Knoxville, TN 

Huntsville, AL 

Neighbor 1 Indianapolis, IN 

St. Louis, MO 

Springfield, IL

Neighbor 2 Columbus, OH 

Baltimore, MD 

Charleston, WV 

Richmond, VA

Neighbor 3 Atlanta, GA 

Birmingham, AL 

Mobile, AL 

Montgomery, AL 

Macon, GA

As explained in Section II.D.1, we estimate econometric relationships between recent historical 

loads and weather to develop 32 synthetic annul load profiles that are consistent with 32 years of 

historical weather data. The following three figures show the range of load profiles we model, 

with summer peak load in Figure 49, winter peak load in Figure 50, and total annual energy in 

Figure 51. In each case, the figures show the percentage by which the 99th percentile, 95th

percentile, and 90th percentile extreme loads exceed the weather-normalized load. 

As shown in Figure 28, extreme weather can cause peak load to substantially exceed the 

expected normal (i.e., 50/50 peak load) condition. In fact, each region can expect actual peak 

load to exceed weather-normal peak load by more than 5% once every ten years solely due to 

weather conditions, with extremes of up to 9% in rarer cases. As shown in Figure 50, winter 

peak variance is highest for the Study RTO Region and Outside Region 3, although the duration 

of these peaks is shorter and the average winter peak is lower than the average summer peak. 

This means that most reliability events are still most likely to occur during the summer. Outside 

Regions 1 and 2 are only summer-peaking, and so cold weather events do not impact these 

regions substantially. Finally, as shown in Figure 51, annual energy does not vary as 

substantially as summer and winter peak loads because it takes into account all hours of the year, 

and many off-peak hours have more stable load profiles that are not as heavily influenced by 

weather. 
B-8 

Figure 49 

Summer Peak Load in Extreme Years as Percent above Weather-Normal Peak 

Figure 50 

Winter Peak Load in Extreme Years as Percent above Weather-Normal Peak 
B-9 

Figure 51 

Annual Energy in Extreme Years as Percent above Weather-Normal Year 

Reliability outcomes are also heavily influenced by the duration of severe temperatures, 

particularly because some summers may have extremely high temperatures for only a few days 

while other years’ experience sustained high-cost events with high temperatures for the duration 

of the entire summer. For example, Figure 52 shows the load duration curve for the Study RTO 

Region relative to weather-normalized peak load for two years, one that has high relatively high 

system peak load and one relatively low peak load. Note that 2004 weather year produced a 

peak load of only 95% of the weather-normal peak load, while the 1980 weather year 

experienced a peak of 10% above normal, as well as a much larger number of high-load hours, 

driven by a sustained summer heat wave. 
B-10 

Figure 52 

Duration of Extreme Hours 

Finally, each region can expect some level of neighbor assistance during peaking conditions, 

since the systems will peak at slightly different times. Table 18 summarizes shows the load 

diversity between the Study RTO and the three neighbors in the weather-normal year, comparing 

non-coincident peak load to peak load during: (a) the multi-regional coincident peak; and (b) the 

Study RTO peak. Note that diversity between the Study RTO and Neighbor 3 is relatively low 

while the Study RTO has more diversity with Neighbors 1 and 2. This is important in our 

reliability simulations in that most neighbor assistance during the Study RTO’s system peak will 

be available from these two regions. On a total system basis, the weather diversity reduces the 

individual regions’ non-coincident peak load of 180,000 MW by 3% to a combined multi-region 

coincident peak of 174,668 MW. 

Table 18 

Load Diversity for Study RTO and its Neighbors 

(During Multi-System Coincident Peak and Study RTO Peak) 

Peak Load Load Diversity

NonCoincident 

Peak

During 

Multi-Region 

Coincident 

Peak

During 

Study RTO 

Peak

Relative to 

Multi-Region 

Coincident 

Peak

Relative to 

Study RTO 

Peak

(MW) (MW) (MW) (%) (%)

Study RTO 50,000 48,605 50,000 2.8% 0.0%

Neighbor 1 60,000 58,686 54,399 2.2% 9.3%

Neighbor 2 40,000 38,604 35,242 3.5% 11.8%

Neighbor 3 30,000 28,773 28,887 4.1% 3.7%

Total 180,000 174,668 168,528 3.0% 6.4%