# Presentation: 877647

## STEO Electricity Modeling:Data Analysis and Model Calibration

**Phillip Tseng, SMG, Dave Costello, EMEU **
**Energy Information Administration**
**April 28, 2005**

**Notes:**
Good morning, members of ASA committee, Mr. Caruso, EIA colleagues, ladies and gentlemen.

Today I will talk about the regional electricity model for the new regional short term model.  Before I start, I would like to thank the EMEU modeling team for letting me give this talk.  Many people contributed to this challenging task.

## Outline

**Purpose of the model**
**Model structure**
**Model calibration **
**Data Sources and Data Issues**
**Performance of the model**
**Questions for the Committee**

**Notes:**
I will briefly describe the purpose of the model and model structure. The primary focus of my talk today is model calibration and data issues.  I will also report the performance of the model. Our questions for the Committee are related to model calibration and performance of the model.

## Purpose of the model

- 	Provide a link to the new integrated regional energy system in answering region specific questions:
**Summer electricity market**
**Natural gas market **
**Winter heating fuel market **

## Model Structure -1of 2

**The regional electricity model has 13 regions, which are grouped into three trade zones:**
** ****Eastern Interconnection**
** ****Western Interconnection**
** ****Texas**
**Each region has its own demand and supply representations**
**A 14****th**** region (Hawaii + Alaska) is treated separately (and simply) for completeness in deriving U.S. aggregate demand and supply**

## RSTEM 13 Electricity Demand and Supply Regions

## Model Structure -2 of 2

**For each region, an average daily 24-hour load curve is derived from monthly demand, historical load data and temperature.**
**Supply curves are created from the EIA860 generator data file, which reports capacities and heat rates**
**The model searches for a minimum-cost supply which meets hourly demand.**

## Model Calibration  

**For a single region**
**Demand + Transmission&Distribution(T&D) Losses - Net Imports = Generation**
**For a trade zone such as the Western**
**Interconnection**
**Σ****i****(Demand + T&D Losses - Net Imports) =**
**Σ****i****(Generation)**
**	****where i=number of regions in a trade zone**
**The challenge is to make the model replicate**
**historical generation patterns and fuel shares**

**Notes:**
We use 2002 data to calibrate the model

## Texas Monthly Generation of Electric Power in 2002

- (Thousand Megwatthours)

|  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Month | RF13 | DS13 | NG13 | COAL13 | NUKE13 | HYDRO13 | CHP13 | TOTAL13 |
| 1 | 0 | 8 | 10856 | 11894 | 3555 | 89 | 3570 | 26946 |
| 2 | 35 | 38 | 9799 | 10545 | 3211 | 77 | 3238 | 24101 |
| 3 | 20 | 70 | 11194 | 10160 | 3500 | 90 | 3436 | 25441 |
| 4 | 0 | 15 | 12340 | 10106 | 2596 | 122 | 3287 | 25655 |
| 5 | 0 | 7 | 13098 | 12104 | 3321 | 101 | 3368 | 29147 |
| 6 | 1 | 9 | 16410 | 12542 | 3206 | 90 | 3426 | 32844 |
| 7 | 0 | 6 | 18702 | 13001 | 3432 | 187 | 3669 | 35917 |
| 8 | 0 | 12 | 19880 | 12995 | 3500 | 118 | 3589 | 37154 |
| 9 | 0 | 11 | 15489 | 11990 | 3324 | 78 | 3350 | 31317 |
| 10 | 0 | 15 | 13281 | 10777 | 1800 | 45 | 3106 | 26295 |
| 11 | 0 | 15 | 11209 | 10589 | 1604 | 44 | 3298 | 23847 |
| 12 | 0 | 24 | 10838 | 12579 | 2567 | 83 | 3095 | 26561 |

**Notes:**
For each region we need to calibrate the model so that fuel shares of fossil plants match historical data.

## Slide 9

| Comparison of Western Interconnection Estimated Demand and 2002 Monthly Generation (000 mwh)  |
| --- |

|  | Estimated Demand  |  |  |  | Generation |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Month | Region 8 | Region 9 | Region 10 |   | Region 8 | Region 9 | Region 10 |
| 1 | 21310 | 11813 | 21999 |   | 26462 | 14086 | 14602 |
| 2 | 19221 | 10522 | 18723 |   | 23909 | 11918 | 12665 |
| 3 | 20663 | 11089 | 21528 |   | 25475 | 12031 | 15797 |
| 4 | 19541 | 9955 | 21137 |   | 24001 | 12651 | 14006 |
| 5 | 20887 | 9316 | 21939 |   | 26631 | 13332 | 13946 |
| 6 | 24165 | 9378 | 24823 |   | 27027 | 15005 | 16355 |
| 7 | 26185 | 10249 | 26673 |   | 29942 | 13547 | 19645 |
| 8 | 24251 | 9886 | 25482 |   | 28916 | 11951 | 18778 |
| 9 | 21443 | 9033 | 24070 |   | 27208 | 10361 | 16999 |
| 10 | 20031 | 9360 | 21572 |   | 25623 | 11016 | 14348 |
| 11 | 19742 | 10093 | 20106 |   | 25606 | 11702 | 12646 |
| 12 | 20999 | 10863 | 22425 |   | 27501 | 12264 | 14547 |

**Notes:**
The estimated demand and generation numbers show that region 10 is clearly an importing region.  However, both region 8 and region 9 are exporting regions.  
The calibration process involves identifying and estimating key factors that determine regional generation patterns.

## Model Calibration  1 0f 5

**Steps in Model calibration**
**Develop and verify that regional demand and generation data are comparable **
**Analyze historical generation data at the generator level to identify outliers **
**Estimate key model parameters**
**Check for data inconsistency and iterate the process again**

**Notes:**
We use the 2002 data to calibrate the model
The model structure is generic and its solution relies on a very simple economic principle, i.e. least cost dispatching.  The reality, however, is not as simple.  Power plants serving a specific local load center which faces transmission constraints may have higher variable than neighboring regions.  Yet, operators of these power plants can still compete because of the constraints.    The model calibration process uses available historical data to shed lights on operators’ dispatching behavior and modify the dispatching order of power plants accordingly.

## Model Calibration  2 0f 5

**1. Historical demand and generation data **
**Regional demand is derived from EIA826 which reports monthly sales that are based on ****billing cycles**** and in most instances are not calendar month data.**
**Regional generation is from EIA906 which is calendar month data.**
**For each region, demand is converted to calendar month.  Demand, supply, and imports, must satisfy an identity.**

**Notes:**
At our last ASA meeting, two questions were raised.
The first is about the measurement and then implementation of the  transmission constraints. 
The second is about interaction between prices and demand when the monthly demand is transformed into daily load curves. An additional question in this area is about model validation when baseline data is an issue. Part of the problem is that some of the data necessary to make really good assessments is proprietary data that's not immediately available and the question was raised whether or not if given the potential usefulness of this model, whether or not it might be worth expenditure of EIA resources to go out and actually collect that data directly.
Response to ASA Comments:
EIA modeling team has performed extensive search of transmission data and found that the North American Electricity Reliability Council (NERC) publishes report on NERC regional transmission capacity.  However, NERC regions do not have all the needed detailed information on power flows.  In response to the committee comments and as part of model calibration effort, EIA will document the method used to estimate transmission constraints at the April 2005 ASA Energy Committee meeting.
To address the second question regarding the quality of hourly load curve,  EIA compared daily 24-hour load data from published sources such as the Midwest Independent System Operator (MISO), the New York ISO, and California ISO.  The proprietary data provided by the consulting firm compares well with available data.  EIA plans to discuss the transformation of monthly data to hourly load and the effects of prices on demand at the April ASA Energy Committee meeting as well.  The presentation on model calibration will address data issues and the assessment of model performance. 

## Model Calibration  3 0f 5

**2. Analyze generation data **
**Use the net generation and fuel use data to estimate heat rate **
**Use heat rate and fuel prices to compute variable costs**
**Use data on generation capacity and net generation to compute utilization rates**
**Run regression analysis between utilization rate and variable costs to identify outliers and assign dispatching order**

**Notes:**
Some factors affecting dispatching can not be observed easily.  For example, unadjusted and unconstrained, the model likes more coal for the Western Interconnection because coal is cheaper in the Mountain than the Pacific region.  The 2002 and 2003 EIA906, however, reports consistent coal use in the OR/WA region.  This pattern may reflect transmission limits or certain economic condition that are not available in the regional data.
The regression analysis uses data for each trading block, i.e. Eastern Interconnection or Western Interconnection.  Regression results show that utilization is a negative function of variable costs.  In other words, lower cost units report high utilization rate.  There are outliers. We observe that using one standard deviation as a cut off point, there are high cost high utilization generators as well as low cost low utilization generators.  
We created four  dispatching groups: 1. 75% and higher, 2. 50-74%, and 25-50%, and 0-25%. 

## A Scatter Plot of  California Fossil Fuel Power Plants for July 2002

**Notes:**
Based on the computed utilization rate, all fossil fuel power plants are assigned a dispatching priority code.
Plant type code = 1 if utilization rate >=75%
Plant type code = 2 if utilization rate is between 50 and 75%
Plant type code = 3 if utilization rate is between 25 and 50 %
Plant type code  = 4 if utilization rate is less than 25%
We also assign a plant type code of zero for nuclear and hydro power, based on zero fuel costs. 

The outliers, defined as points more than one standard deviation away from the fitted trend line, receive different treatment in the dispatching order depending upon the nature of the deviation.  For plants with high cost and high utilization generators, the dispatching rankings are moved higher. 

## Model Calibration  4 0f 5

**3. Estimate Model Parameters**
**Import constraints**
**Designation of generators to one of the four load groups**
**Numerical criteria for fuel switching**
**Share of oil and gas in coal power plants**
**Share of hydro power allocated to base load and the dispatching profile for none base load power**

**Notes:**
I will use western interconnection as an example to illustrate the process.  This region is simpler because California is the only import region.

Run Western interconnection and compare monthly generation results with historical data.
Adjust the level of import constraint until the difference between model results and historical data is less than 4% or less.
Import constraints could be either transmission capacity or economic factors not easily observed.  North America Electricity Reliability Council (NERC) publishes flow potential between NERC regions.  Import limits for California, New York, and Florida, however, are not available.

## Model Calibration  5 0f 5

**4. Check for data inconsistency**
**In each trade zone, we first solve the model and then use import constraints to calibrate total generation for import regions. **
**We analyze 906 generation data to adjust dispatching pattern of outliers.**
**We move some power plants to base load dispatching group to change dispatching order.**

**Notes:**
The calibration is an iteration process.  

## Data Sources

**EIA-906: monthly data on generation and fuel use at the generator or plant level by states.  **
**EIA-826: monthly electric utility sales and revenue report with state distributions**
**EIA-423 and FERC-423: monthly cost and quality of fuels for electric plants **
**EIA-860: Capacity of generators by technology and fuel at plant level**
**Office of DOE Fossil Energy (FE):  Annual Report of International Electric Import/Export data**

**Notes:**
The challenge of this project, at the outset, is developing a model capable of linking all  EIA electricity data sources and making it operational.

## Data Issues

**Sales and generation data are subject to:**
**Reporting errors**
**Estimation errors**
**Regional aggregation may introduce biases to the cost based dispatching**
**Oil and gas consumed in coal plants are hard to predict**
**Timing of hydro power dispatching could affect peak load dispatching decisions**
**Ex ante or even ex post events on power plants can not be easily captured **

**Notes:**
In the mountain region, we have only one price for each coal, oil, and gas.  Price variations and distance to load centers can definitely affect dispatching decisions.

There are cases where reported generation do not correspond correctly to the 860 capacity

## Examples of Data 

**The Western Interconnection is used to illustrate some of the data issues**

## Slide 19

**Notes:**
The outlier in this chart alerted our data analyst.  We had to dig deeper to find out what really happened. The problem was data processing in matching the 906 generation to the 860 capacity.  Sometime records are not aligned, consequently, generators are dropped out.  We had to talk to CNEAF and resolve the issues and it is a bit time consuming but rewarding experience.

## Slide 20

## Slide 21

## Performance of the Model

**Comparison of model results with **
**2002 monthly data**

## Slide 23

| Percent Differences:
Simulated Values vs. 2002 Monthly Data |
| --- |

|  | R8 |  | R9 |  | R10 |  |
| --- | --- | --- | --- | --- | --- | --- |
| Month | total | gas | total | gas | total | gas |
| 1 | 0.18% | 16.72% | -0.05% | -4.77% | -0.28% | -0.73% |
| 2 | 0.41% | 4.71% | 0.75% | -7.33% | -1.47% | -3.48% |
| 3 | 1.16% | -5.13% | 0.22% | -5.61% | -2.04% | -4.91% |
| 4 | -3.88% | -0.57% | 10.42% | 134.46% | -2.76% | -8.98% |
| 5 | -6.15% | -10.87% | 11.11% | 221.82% | 1.11% | 3.01% |
| 6 | -6.02% | -34.89% | 11.58% | 112.56% | -0.67% | -1.91% |
| 7 | -4.32% | -19.71% | 11.70% | 169.14% | -1.49% | -3.74% |
| 8 | 0.29% | -7.62% | 3.39% | 24.83% | -2.60% | -6.07% |
| 9 | 0.40% | -2.40% | 0.08% | -2.76% | -0.68% | -1.84% |
| 10 | 0.09% | -12.90% | 0.65% | 3.77% | -0.67% | -1.33% |
| 11 | -0.07% | -4.86% | 0.97% | 5.75% | -0.77% | -2.17% |
| 12 | 0.98% | 3.40% | -0.89% | -9.41% | -1.10% | -2.62% |

## Differences between Simulated Values and 2002 Annual Total (000 mwh)

| Region | gas | coal |
| --- | --- | --- |
| R8 |     (3,239) |     (1,353) |
|  |   |  |
| R9 |      2,811  |      3,927  |
|  |   |  |
| R10 |     (2,242) |         176  |
|   |   |   |
| Total |     (2,670) |      2,750  |

## Slide 25

| Total Generation from Coal  plants in 2002 (000 mwh) |
| --- |

| Region | Coal | Gas share  |
| --- | --- | --- |
| R8 |   208,157  |        1,041  |
|  |   |  |
| R9 |     12,395  |           620  |
|  |   |  |
| R10 |      1,832  |               9 |
|   |   |   |
| Total |   222,383  |         1,670 |

## Questions for the Committee

- Is the regression technique we use acceptable for changing the dispatching order of some power plants?
- Given the data quality, what is an acceptable level of margin for error?

## 

Thanks

- Thanks