AMERICAN STATISTICAL ASSOCIATION

COMMITTEE ON ENERGY STATISTICS

\+ + + + +

PUBLIC MEETING

\+ + + + +

THURSDAY

NOVEMBER 13, 1997

\+ + + + +

WASHINGTON, D.C.

The Committee met in the Clark Room of the Capitol Holiday Inn, 550 C
Street, S.W., at 9:00 a.m., G. Campbell Watkins, Chair, presiding.

PRESENT:

G. CAMPBELL WATKINS Chair

DANIEL A. RELLES Vice Chair

DAVID R. BELLHOUSE

R. SAMPRIT CHATTERJEE

BRENDA G. COX

CAROL A. GOTWAY CRAWFORD

PHILIP HANSEN

CALVIN KENT

GRETA M. LJUNG

ROY WHITMORE

INVITED GUESTS:

SEYMOUR SUDMAN

RICHARD TABORS

EIA STAFF:

JAY HAKES EIA Administrator

BILL WEINIG

LYNDA CARLSON

ROBERT SCHNAPP

BETSY O\'BRIEN

LAWRENCE J. PRETE

PETER WHITMAN

JACOB BOURNAZIAN

THANH H. LUONG

DOUGLAS R. HALE

ROBERT L. MANICKE

BILL LIGGETT

JAMES KNAUB

EUGENE BURNS

ROBERT LATTA

SWIGHT FRENCH

JOHN WOOD

[C O N T E N T S]{.underline}

[PAGE]{.underline}

Introduction by the Chair 6

Remarks by EIA Administrator 11

Electricity Today: A Briefing

Robert Schnapp 32

Betsy O\'Brien 35

Lawrence J. Prete 53

Committee Questions 85

Electricity Prices in a Competitive

Environment: Next Steps in the Analysis

Peter Whitman 107

Calvin A. Kent 117

Richard Tabors 125

Petroleum Marketing Disclosure

Avoidance Techniques

Jacob Bournazian 144

Carol Gotway Crawford 157

LUNCHEON RECESS

A Prototype Network Model of the

New England Power Pool

Douglas R. Hale 167

Thanh H. Luong 174

Philip Q. Hanser 183

Daniel Relles 196

Committee Questions 203

Electricity: \"From Regulation to

Open Transmission Access\"

Robert L. Manicke 214

Committee Questions 229

Use of Covariates to Improve

Efficiency of Estimation

David R. Bellhouse 247

James Knaub 256

Eugene Burns 271

Committee Questions 281

Public Comments 291

[P R O C E E D I N G S]{.underline}

9:10 a.m.

CHAIR WATKINS: Good morning. My name is Campbell Watkins and I\'d like
to commence this meeting. The meeting is being held under the provisions
of the Federal Advisory Committee Act and it is an American Statistical
Association Committee, not an Energy Information Administration
Committee, and this committee provides periodic advice to the EIA.

The meeting as you can see, is open to the public; public comments are
welcome, and time will be set aside for comments at the end of each
session \-- not only from the committee itself but from the floor as
well.

In commenting I would ask please, each member of the public to stand and
state their name and affiliation, speak into the microphone. I\'m sure
our recorder will be happy to have them do that.

I want to just express regrets for Charles Bischoff who is unable to
join us, unfortunately under doctor\'s orders; he\'s not well. Also John
Grace and Bradley Skarpness who could not attend because of unforeseen
business commitments.

We\'ve lost one committee member, Michelle Foss, whose ability to attend
the meetings on a regular basis was going to be jeopardized by other
commitments and felt she \-- she felt compelled to resign.

Brenda Cox has had to step down as Vice Chair. I\'m pleased to say,
however, she is going to continue as a committee member. I shall miss
her help and advice, but I\'m pleased to announce also that Dan Relles
has taken over as Vice Chair and will assume the Chair at the next
meeting.

I\'d like to mention two members who will be joining the committee next
year. First is Dr. Seymour Sudman who is Walter Stellner distinguished
professor of Marketing at the University of Illinois. He\'s deputy
director of Survey Research Laboratory at the University of Illinois as
well, and as a sort of initiation rite he\'s agreed to come to this
meeting here. But I\'ll have the committee members introduce themselves
in a moment.

Another committee member who will be joining the committee next year was
invited to this meeting, but unfortunately not able to make it, is Dr.
Polly Phipps who is research director at the State Government of
Washington.

We also have with us as a special invitation today, Dr. Richard Tabors
who is senior lecturer at MIT and also president of Tabors Caramanis
Consulting Firm.

Now what I would like to do is have the attendees here just introduce
themselves. I think we\'ll go around \-- the speakers will \-- you\'ll
introduce yourselves later. I\'d like to start with Samprit and then go
around the table.

DR. CHATTERJEE: Samprit Chatterjee from New York University.

DR. CRAWFORD: I\'m Carol Gotway Crawford from the Centers for Disease
Control.

DR. HANSER: I\'m Phil Hanser from The Battle Group.

DR. LJUNG: I\'m Greta Ljung, Institute for Business and Home Safety,
Boston.

VICE CHAIR RELLES: I\'m Dan Relles from the Rand Corporation.

DR. WHITMORE: I\'m Roy Whitmore from Research Triangle Institute.

DR. BELLHOUSE: David Bellhouse from the University of Western Ontario.

DR. KENT: I\'m Cal Kent from Marshall University.

DR. COX: Brenda Cox from Mathematica Policy Research.

MS. CARLSON: Lynda Carlson from Statistics and Methods Group, BIA.

MR. WEINIG: Bill Weinig, Statistics and Methods Group, BIA.

DR. HAKES: Jay Hakes, BIA.

CHAIR WATKINS: Thank you. Now a couple of other announcements. There
will be a luncheon for the committee and invited guests at 12:30 today
in the Lewis Room. I do want to also say one or two things about
arrangements for tomorrow.

It\'s a really early morning breakfast for the committee in the Lewis
Room at 7:15, and as you\'ll see from the program we\'ll be going over
to the Forrestal Building and we have to leave here \-- Bill Weinig has
emphasized to me strongly on many occasions \-- we have to leave here at
8 p.m. sharp \-- 8 a.m. sharp.

The purpose of this trip is to look at Mathematica\'s Power Web and for
members of the public who wish to go there, they are asked to meet Bill
at 8 o\'clock at the Forrestal Building, 1000 Independence Avenue. We
have all sorts of security procedures to go through and I\'m told that
that\'s quite time-consuming.

For the committee we have a dinner this evening, 7 o\'clock at the 710
Restaurant. We use that location for the past two times, having dinner
there. I think we do need a head count and maybe you could let Tracey
Dowden know whether you\'re coming or not, say by lunchtime.

I think that\'s all I have to say. Bill, have I missed out anything?

MR. WEINIG: No.

CHAIR WATKINS: I\'d like to hand over to Jay Hakes.

DR. HAKES: I\'d like to do two things: recognize some individuals for
their contribution to the committee, and then also make some general
remarks about what EIA\'s currently working on and what we foresee for
the future.

Because we have term limitations here we do have a turnover on the
committee and lose people that we\'re very reluctant to lose, who have
made a great contribution. Dr. John Grace is not here today but he has
been a valuable member of the committee.

And I\'d like to especially recognize Campbell Watkins who has been the
chair for the last two years and is completing six years on the advisory
committee. Several things to note about his service is, I believe he has
not missed a single meeting during the six years \-- which I think in
and of itself, is a great achievement.

But I think probably more important than that, as someone who\'s
attended a good number of these meetings, is the thoughtfulness of his
comments whenever he\'s been a discussant. The depth and the preparation
that\'s evident in these discussions has been extremely valuable kind of
almost the ideal role model of what we hope to give back in terms of
penetrating ways of looking at things differently or more carefully.

And we want to recognize him with the maximum recognition that we\'re
allowed to give under Federal Law and within our budget, which is an EIA
T-shirt and coffee mug.

There\'s a letter here \-- this is from me and there will be a
subsequent one from the Secretary of Energy. But just to read in part it
says, \"We appreciate your guidance and sound judgment on many difficult
matters, particularly in the matters of energy supply and demand
modeling and forecasting techniques, industrial energy demand and
productivity, and regulatory economics.

Your thoroughness in providing both written comments and oral
discussions on a wide range of energy and energy-related subjects has
been particularly useful and stimulating. Additionally, you have
contributed greatly to the quality of the Energy Information
Administration\'s data, models and analysis. We hope that we may
continue to call on you from time to time for advice and assistance\".

CHAIR WATKINS: Let me just say this is a very generous acknowledgement
and I do appreciate working with yourself and also Calvin Kent who was
the initial administrator when I first joined the committee. Thank you
very much, indeed. It\'s been an honor to serve.

DR. HAKES: I must say, I look forward to working with Dan. Campbell has
big shoes to fill but Dan has, in his time on the committee, has made a
great contribution and we\'re grateful for his willingness to serve.

I\'d also like to join in welcoming Dr. Sudman today. We appreciate his
coming early and know that he will be a very valuable addition to the
committee and he will receive a fuller introduction when he officially
begins in the spring.

I\'d also like to recognize today, Dr. Richard Tabors who has been good
enough to come down and join in, in our discussion of electric
restructuring. He\'s with Tabors Caramanis and Associates in Cambridge,
Massachusetts, and has 20 years of domestic and international experience
in energy planning and pricing.

Dr. Tabors is also a senior research engineer at the Laboratory for
Electromagnetic and Electronic Systems and senior lecturer in Technology
Management and Policy and Intellectual Engineering at MIT. We just have
added five new Presidential appointees at the department in the last few
weeks and two of them are from MIT, so the school is doing well. We also
have a mug and a T-shirt for you as well.

I think I pointed out before, you have to be careful when you wear
things that say EIA on them because that also stands for the
Environmental Industries Association, and Exercise-Induced Asthma.

The final recognition is to Renee Miller who\'s served as liaison to the
advisory committee since the spring meeting in April of 1986. Back in
those days Noel Bartlett was chair; he was with SOHIO.

Other members of the committee included: Jack Schuenemeyer of the
University of Delaware; Gordon Kauffman at MIT; Jim Mahoney from Detroit
Edison; Morris Hamburg, University of Pennsylvania; Tim Coburn of
Marathon Oil; Bill Eddy of Carnegie-mellon; Tom Witt, West Virginia
University; Gordon Sande, Statistics Canada; and Irving Roshwalb of
Audits and Surveys.

Back in those days, Helmut Merkleim\'s predecessor was an administrator
of EIA and the meetings started at 1:30 in the afternoon rather than
nine in the morning.

Renee has managed over 20 semi-annual meetings since then and she\'s, I
think, both been able to look at things from an EIA point of view and a
committee point of view. I know I\'ve heard Campbell and others comment
in very glowing terms about her work, and we of course, have been
impressed by everything Renee does, but also her work with this
committee. So we\'d like to recognize her at this time.

Renee will be receiving the Exceptional Service Award which is a pretty
nice plaque and a gold medal for her extraordinary stewardship of the
American Statistical Association Advisory Committee to the Energy
Information Administration. Ms. Miller\'s work with the committee has
substantially enhanced the quality and usefulness of EIA statistical and
analytic products.

I\'d like to make a few general comments as I often do, about what\'s
going on at EIA. Let me first talk about resources which are basically
people and money, in that sort of order of importance.

A few years ago we started on a period of decline in terms of resources
where we were given instructions to reduce substantially the number of
employees at EIA and to reduce our budget. We\'ve gone a number of
people from a Federal staff of about 480 to about 400 people.
Proportionately we\'ve had bigger cuts in our contract workforce.

During that time we\'ve gone from a budget from \$85 million to about
\$67 million for the next fiscal year. These have been difficult times,
although I would say that I think EIA has been very adroit and skillful
of how it has done these adjustments. Although we\'ve had to cut back in
certain areas we\'ve also been able to enhance our service in other
areas, particularly the World Wide Web.

What was more alarming than the current figures were projections that
were contained in various government documents that had us ending up in
the low-\$50 million range, which I think would have brought about
consequences that really were not tolerable for the energy statistical
system.

However, I\'m glad to report that now the lines into the future are for
basically stable funding at the \$68 million, and in fact, there is some
indication recently that the administration may be requesting more money
than that in the coming year, given the key role that EIA has been
playing in electric restructuring and climate change work.

So I think \-- and we also are expecting a revision of our Federal
employee targets in the next few months and if things go well, we may
even start hiring some people beginning next year. We have not hired
anybody for several years.

So I think there is some evidence that things are turning. Part of it
has been a recognition I think, of the valuable role that EIA plays, and
we\'re hoping that this will continue to grow.

I mentioned the Web site. This is a service that I think has had huge
impacts in the ability of people who perhaps have not in the past,
availed themselves of EIA information, can now get it quite readily and
use it on a regular basis. And we find now on our busy days, that over
3,000 people come onto the Web site \-- that\'s not hits but that\'s
actual people who come onto the site. And that\'s I think, a quite heavy
usage.

It\'s interesting to note too, we\'ve had a phrase that we use at EIA
saying that we want to be the first place to go for the last word in
energy information. And if you go to the Lycos search engine which has a
feature called \"top five percent sites\" which they do for all
different subject areas, and you type in \"energy\", EIA comes up as the
number one site. So I think that\'s some rough evidence that we\'re
achieving our goal.

The reviewers at Lycos like our site for its good design and great
content. The reviewer obviously didn\'t understand quadrillion Btus and
felt that one had to have a Ph.D. sometimes to use our site. But I think
that it is being used. It\'s had a big impact around the country and
around the world, and we think that\'s important.

Another development which I think many of you may be aware of, but if
you\'re not I would point out, is that in working with the other Federal
statistical agencies we now have a site called FedStats. It\'s
www.fedstats.gov and it contains all the Federal statistical agencies
\-- including Census, Bureau of Labor Statistics \-- and there\'s some
ability to search by subject rather than organization within that site.

The goal being that a person shouldn\'t have to know what agency or what
part of an agency collects certain data to be able to find it. So I
think the findability of data now has become easier for the average
citizen. We like to say, in energy at least, that a high school student
with a modem in Boise, Idaho has better access to energy information
than the top experts in the government had five years ago. Some find
that a scary thought, but I think it\'s literally true.

Another area that I would bring to your attention is, we have I think,
derived some benefits in the last year or two from a new group called
the Analysis Review Board, which tries to look at EIA in a global sense
rather than just an office-by-office sense, and try to see whether our
analytic work is focused in the areas of greatest interest.

To spur this development the Board has sponsored meetings with major
energy opinion-makers around town, including representatives from the
Federal agencies, from the Congress, and from various think tanks and
different groups, to try and find out what were the kinds of analysis
that were going to be most useful to policymakers and to the general
public.

One of the results of that has been a focus on electric restructuring as
an issue. I don\'t think it was a great secret that this was an issue
that had huge economic impacts and one that would be facing the country,
both at a national and state level.

And we have put out a number of publications in this area. An analytic
publication trying to assess what the price structure might be in a new
and regulated environment \-- had well over 700 accesses its first day
on the Internet and got quite a bit of attention.

I won\'t say too much about electric restructuring today because it\'s
kind of the theme of the two days and I think those of you who are in
the room have a really valuable opportunity to get a lot of information
in a very convenient form.

The other area of course, that is getting a lot of attention and one in
which EIA needs to play a major role, is the area of greenhouse gas
emissions. The Energy Policy Act of 1992 made EIA the major reporter of
U.S. greenhouse gas emissions.

That was relatively easy for us to handle in carbon emissions which are
so closely related to energy, but it got us into a lot of other gases
that we hadn\'t really dealt with before. And I think we\'ve done a
great job in that area and in the area of collecting reporting data
about voluntary programs that industry has to reduce emissions.

The other area where we have been active is, in our annual process of
setting energy baselines for the next 15 or 20 years, we do estimates of
energy consumption and estimates of carbon emissions.

And one of the things that I think has been encouraging about the debate
about carbon emissions is that amidst all the TV ads and the arguments
coming from the environmental community, the business community, people
who have very divergent views on the issue, virtually everyone that I\'m
aware of has used our baseline as the baseline for carbon emissions.

In other words, if carbon emissions are going to be 34 percent higher in
2010, in the EIA projections people will say, well therefore we need to
reduce carbon emissions 34 percent below what they are likely to be.

It\'s been so widely accepted that it almost could pass notice, but I
think what it is, is a recognition that EIA is seen as a neutral player
that is not pushing a particular policy but can be trusted to do a fair
job in making those kinds of assessments.

And so I want to use just my few remaining minutes here to talk a little
bit about this carbon situation since we had a fairly major press
conference on it yesterday. There were articles about it today in [The
Wall Street Journal]{.underline} and [USA Today]{.underline}. It was
mentioned in [The Washington Post]{.underline} and [The New York
Times]{.underline} was there and I\'m pretty sure they will be doing a
major study.

I have tiny charts today so I\'m going to hold them up so they\'re not
too far away from you. But this basically is economic growth projected
to the year 2020. And basically what we\'ve shown is economic growth,
which is the red line on top, and this year\'s estimate is greater than
it was last year.

In other words, last year we said economic growth to 2015 would be at
the rate of about 1.9 percent a year; now to the year 2015 where you can
compare the two lines, it\'s growing to 2.1. And as the economists in
the room can tell you, there\'s a big difference if you take an annual
rate over a long period of time, that difference will produce some real
changes in the economy.

Now, the other thing I would say here, is you see we\'re going to the
year 2020 this year for the first time, which is also I think, of some
interest.

I\'m giving you just a few highlights, but here\'s electricity prices.
Last year, the green line going down somewhat in constant dollars, but
going down much more in this year\'s analysis. This is not trying to
anticipate all the impacts of restructuring but to include the impacts
in the areas that have already passed plans, and it\'s also based on
things that the industry is doing to prepare for competition.

And in constant costs, electricity prices now across all sectors
including industrial, 6.9 percent, gets down to \-- excuse me, 6.9 cents
\-- and it gets down to 5.5 cents at 2020. Now obviously, if normal
cause and effect relationships are at work, that\'s going to raise
energy consumption because lower prices decreases the incentive for
efficient equipment and provides less incentives to conserve.

And economic growth increases discretionary customer spending, which
will lead to things like more power for automobiles and other things
that often are associated with higher energy use.

So if you look at total carbon emissions, we were saying last year that
between 1990 which is the year that some policymakers are trying to
stabilize to, they were going to grow by 28 percent. Now we\'re saying
they\'re going to grow by 34 percent.

Now some people said this was a plot by the White House because they
wanted the numbers to be higher to create a greater sense of emergency.
Other people said, the White House is going to be really upset because
this shows current plans aren\'t working and this is going to be a tough
thing to do.

I don\'t know which of those things is true; all\'s I know is that this
is a result of an independent, analytic process that I think is pretty
soundly based. But the bar for carbon emissions mitigation has been
raised substantially in the last day or two and I think it\'s a
challenge for people to deal with.

This is the carbon emissions by sector. As you can see that half of the
increase is in transportation. Where in the last \-- if you take the
same type model car over the last ten years, horsepower\'s increased by
56 percent. We don\'t anticipate quite that level of increase in the
future but we anticipate horsepower will grow, and that doesn\'t even
account for people switching from cars to vans and recreational
vehicles.

And then just one final breakdown. If you look at residential energy
consumption, the growth is in \"other\". Right now \"other\", which is
things other than space heating and cooling, water heating,
refrigeration and lighting - in other words, things like stereos, home
theaters, computers \-- its estimated growth from 29 percent residential
consumption to 39 percent of residential consumption.

Space heating is growing very gradually because of efficient equipment;
same with space cooling; same with water heating; same with lighting.
But refrigeration you see something different. Refrigeration actually
will be very much substantially lower than it is now, in the future,
despite the fact of growing population and the growing economy.

The reason for that is two-fold. One is that the equipment needed to
make refrigerators more efficient is relatively inexpensive compared to
the equipment that\'s needed to make, for instance, an air conditioner
more efficient. And secondly, the government has recently passed a new
rule which sets new efficiency standards for the year 2001.

Most appliances are governed by standards that went into effect in 1990,
and have become fairly dated in today\'s market. But you can see that if
somehow what had happened in refrigeration happened in other major areas
of equipment, there would not be any difficulty at all meeting the
stabilization target in the year 2010, despite the additional barriers
we\'ve put up in the last few hours.

I just show you that as an example of where I think EIA work has great
value to people who are working in these areas, and I think help advance
the analysis.

So at this point maybe, if you have any questions you have in general
that maybe aren\'t going to come up later in the sessions that you think
maybe I can answer, I would be glad to \-- I see Cal has his card up.

DR. KENT: Well, the first thing is \-- this is a snotty question but
I\'ll ask it anyway \-- in light of what you have just told us, how come
you\'re eliminating the transportation survey and going only four years
on your other consumption surveys?

DR. HAKES: Well, the reason is that we just had a cut of \$17 million
and we had to cut somewhere. The consumption surveys \-- that decision
was made two years ago; whether that would be made today I don\'t know.
But we\'ve cut back there. We\'ve cut back on the sampling for the oil
and gas reserves, which I think the committee has discussed before. If
people have other areas that they would suggest for cuts in lieu of
these, we would be glad to consider those.

DR. KENT: Well no, the only reason I raise it and that\'s \-- I knew
what the answer was of course \-- but I think that your latest \-- the
presentation you just gave us, the press conference yesterday and
everything, really does indicate how valuable and crucial these surveys
are in doing \-- or, doing this more often is if you\'re going to have
the type of information that people seem to value so highly.

And I just wanted to put it on the record that these should be funded if
the government does expect to continue to have the statistics that it
needs for making public policy, and then I\'ll get off this horse.

DR. HAKES: I think on the consumption surveys we could also offset some
of the disadvantages of doing them every four years than every three
years by getting the data out quicker to the point of its collection. I
think there\'s a lot of opportunities in that area for improved timing.
Because we\'ve had other areas where we\'re experiencing like, 25
percent improvements in the timeliness of getting the data out.

And so I think some of those efficiencies can at least partially offset
for some of the cutbacks.

DR. KENT: Yes, I think it will \-- not to completely disagree with that
point \-- but I think it will get the information out but you still
won\'t be able to measure your deltas, and that\'s really what you want
to know for public policy.

What you want to know is what the changes are: are you having any
impacts, what the changes are in consumption patterns, what are people
doing? And particularly in this area of emissions, the statistics you
just showed us about transportation and the role that it plays in
emission \-- not to even have that survey \-- just seems to me is going
to leave a phenomenal void.

DR. HAKES: Okay. Any other comments or questions? Okay, I\'ll be around
most of the day. There\'s a few other presentations I\'ve heard before
so I\'m going to slip out from time-to-time, but I\'ll be here most of
the day. Thank you.

CHAIR WATKINS: Thank you, Jay. I did want to, perhaps on behalf of the
committee, add my own comments about Renee Miller and to thank her for
her help. I think I would put in a way that was completely without any
kind of flamboyance at all. She, in a very quiet, and efficient way,
managed the affairs of the committee and was always very responsive to
any requests. So I do congratulate you again; thank you for all your
help.

Now, there\'s one thing I did forget to do in the introductions. That
is, I am obliged to ask the members in our audience to identify
themselves \-- people in the back of the room \-- for the reporter. So
could you please do that and maybe use the microphone. We\'ll start with
you, Doug.

MR. HALE: Doug Hale, EIA.

MR. WOOD: John Wood, EIA.

MR. JOHN KELLY: John Kelly, The American Public Power Association.

MR. BROENE: Tom Broene, EIA.

MS. MARTIN: Phyllis Martin, EIA.

MR. SKELLY: Dan Skelly, EIA.

MS. DUCCO: Ann Ducco, EIA.

MS. GORDON: Janet Gordon, EIA.

MR. LUONG: Thanh Luong, EIA.

MR. DAMAIS: Rodney Damais, AIC.

MR. ANGELIO: Frank Angelio, BLS.

MR. MANICKE: Bob Manicke, EIA.

MR. JOE KELLY: Joe Kelly, BLS.

MS. BASS: Gail C. Bass, EIA.

MS. HORNING: Pat Horning, Bureau of the Census.

MS. BUCCI: Susan Bucci, Bureau of the Census.

MS. BODEN: Deborah Boden, EIA.

MS. JACOBS: Cynthia Jacobs, SAIC.

MR. ENGLE: Louis Engle, EIA.

MS. MILLER: Renee Miller, EIA.

CHAIR WATKINS: Thank you. I\'d now like to start the next session.
Larry, should I give it over to you or to Bob?

MR. SCHNAPP: Good morning. Thank you for inviting us to attend today. My
name is Bob Schnapp. I\'m the director of the Analysis and Systems
Division in the Office of Coal, Nuclear, Electric, and Alternative
Fuels.

Over the past two years EIA as Dr. Hakes said, has established electric
power restructuring as its number-1 analytical theme. To successfully
undertake future analyses as well as to provide the public the energy
industries and Congress with the basic tools to understand, to conduct
these analyses, EIA is spending a great deal of time assessing our
current data collection efforts.

We\'ve previously briefed you on some of those activities; however,
before we go much further in asking your advice on specific issues, we
believe that it\'s important to provide you with a basic understanding
of the industry.

EIA has put out a report entitled, \"The Changing Structure of the
Electric Power Industry and Update\", a copy of which is at your desks
now. Our presentation more or less summarizes that work. Today our
office will be doing something different from what is normally presented
by EIA or expected of the ASA committee.

The purpose of this briefing is to educate you rather than to ask for
your advice. As many of you are not intimately familiar with the
electric power industry, the purpose of the briefing is to put into
perspective where the electric power industry has come from, why and how
it evolved the way it did, and just briefly touch on some of the issues
facing the industry today in light of the initiatives to restructure the
industry.

We\'re making these presentations to assist you in understanding the
context and issues which our agency will be facing over the next several
years so you can aid us in changing our data collection instruments.

The briefing you\'re about to receive from Betsy O\'Brien and Larry
Prete has been given to more than 50 Congressional staffers, one
Senator, and several coal and electric power conferences. It has
received wide acclaim from those who will be helping to shape the
nation\'s policies over the next few years.

I personally have heard this at least a dozen times and have thoroughly
enjoyed it each time. Betsy will talk about the structure and ownership
of the industry and then the financial aspects and regulatory
constraints. Larry will follow and discuss the physical operations of
the industry, generation by fuel type, and the transmission system and
how it works.

The briefing will take about an hour; however, please ask questions as
we go along to help you understand. Let me first then, introduce Betsy
O\'Brien, the chief of the Coal and Electric Analysis Branch. Betsy.

MS. O\'BRIEN: Good morning. I\'m happy to be with you today to give a
background on the changing structure of the electric power industry. And
this first viewgraph that I\'ve put up is a product that we have on our
Web site and it shows \-- we update it monthly \-- and it shows the
status of the state actions on deregulating the industry.

Currently, there are eight states that have passed legislation to change
their electric industry into competitive industry with retail choice of
an electric supplier, and five states have issued regulatory orders.
There are 21 other states that are in the process of either passing
legislation or issuing orders.

This is really significant because only just a little over three years
ago the topics at conferences were really debates about whether or not
to open the electric industry to competition, and if we did, what would
be the consequences. In three years that debate has now changed to more
questions of, when will it be accomplished and the specifics of how it
will be accomplished.

And as you can see, the states that are marked in red are the ones that
have already proceeded with legislation and will be opening their
markets. California is the first state that will start in January of
next year.

In the next slide it shows the wide range in the average revenues
collected for electricity from customers by state. And this is one of
the reasons that has encouraged the industry to move toward competition,
especially industrial customers. The states in the Northeast and in
California were the highest priced states in the country, and those
states have been some of the first ones that have moved towards opening
up their electric markets.

But having a high price isn\'t the only reason. For example, the state
of Oklahoma, which is one of our lower priced states, has also actually
passed legislation and will be having retail competition in their
market.

Now I\'d like to go on and describe a little bit about the industry as
it exists today and how we expect it to exist in the future. This first
viewgraph is showing the structure that exists today. There are really
two types of electric generating companies: non-utilities and utilities.
And they\'re very similar except the main difference is in their
markets.

Today, a non-utility can only sell power into the wholesale market or it
can generate power for its own use; while the utilities, in addition to
selling to power in the wholesale market, are the suppliers of retail
electricity to the residential, commercial, and industrial customers.

In addition, most of the utilities were what we call vertically
integrated, and this is a unique feature of the electric utility
industry. It\'s different from the gas industry in that one company owns
several stages of the process of production and sales. They typically
own the generation facilities, the transmission lines, and the
distribution lines.

And this was an issue to non-utility companies. They were concerned
about having fair and equal access to those transmission lines. So a
year ago the Federal Energy Regulatory Commission issued special orders
to assure that there would be open and fair access to the transmission
lines to encourage expanded wholesale competition.

In addition, utilities in the past have been considered regulated
monopolies. There are three features of a monopoly: that they had a
designated service area and any customer within that area had to use the
utility that was designated for their service.

In response, the utility had an obligation to serve all those customers.
They couldn\'t choose to serve only the most economically beneficial
customers, but they had to serve all customers within the area. And to
be considered a monopoly the service is also necessary, widely-used, and
that there\'s no good substitute.

We\'re moving away from that idea of monopoly today into competition.
And the next slide shows an idea of a structure that \-- as the industry
may exist in the future \-- that the companies will no longer be
vertically integrated; that there will be power-producing companies;
there will be a transmission company; and there may be a separate
distribution company.

The transmission would be operated by an independent system operator to
assure fair and equal access to those transmission lines, and the power
producers may sell their powers into a power pool or they may sell
individually to a customer on a specific, bilateral contract.

The ownership also still may be somewhat vertically integrated but they
are required to have the operation of the transmission separate, even
though a company may still own it that\'s also a generating company.
This is just one of the ideas of how the industry may look in the
future.

Now I\'d like to talk about the industry as it exists today. There were
almost 3200 electric utility companies and over 2000 non-utility
companies that produced electricity in 1995. On the left you see the
investor-owned utilities. There are four classes of ownership of utility
companies.

And investor-owned companies, although small in number \-- there are
only 244 of those companies \-- they\'re really the most important in
terms of generation. Because you can see on the graph to the right that
they produce 66 percent of the generation. They also own most of the
capacity, the transmission lines, and the distribution lines, and so
they play an important role in financial markets too, as they raise
money to build all those facilities.

The second bar on each chart represents publicly-owned utilities. And
these are companies that are owned by municipal, state, or other
government organizations. And although they\'re large in number \--
there are over 2,000 of them \-- they\'ve produced only ten percent of
the electricity.

And they\'re usually small companies and some of them don\'t even own
generation facilities, they just have distribution lines, and they
purchase the power for resale to retail customers. In the past, their
facilities have often built through tax-free financing, and they also
are exempt from certain taxes and dividends.

Although this isn\'t necessarily the cause, publicly-owned utilities
tend to have lower prices than investor-owned utilities. And one issue,
as the industry moves to competition, is whether or not those facilities
that were built with public financing should be allowed to sell power
outside the geographic area that financed them initially. This is one of
the issues that\'s being debated in Federal legislation today.

The third bar on each graph represents Federally-owned utilities. There
are ten Federally-owned utilities. They\'re either generating companies
or they\'re marketing companies. You\'re all familiar I\'m sure, with
the Tennessee Valley Authority which is one of the generating companies,
or Bonneville and Western Marketing. Those are examples of Federal
utilities. They were built really, in the 1930s and frequently they were
built for flood control and irrigation purposes, so almost all the power
is produced with hydroelectric generation.

And they sell power at cost to priority customers which are
publicly-owned utilities primarily. And one of the issues related to
Federal utilities in a competitive market is whether or not that power
should still be sold just at cost or if it should be allowed to have a
competitive price like other generating companies.

And also whether or not their transmission lines should be required to
be opened up to open access. They\'re not currently under FERC ruling.
So those are again, issues that are being debated in proposed Federal
legislation today.

The last type of electric utility is considered a cooperative utility,
and these came about also in the \'30s with the Rural Electrification
Act. They were established because of the cost of building transmission
lines to remote areas and very distributed customers in rural areas.

So the Federal Government provided financing for cooperatives to build
the transmission and distribution lines to those customers, and it
accomplished what it was set out to do in electrifying our rural areas.
In the \'30s, only about ten percent of rural farms and areas had
electricity, and today it\'s over 99 percent.

The next bar represents non-utilities and although there are a large
number of companies \-- over 2000 \-- today they generate about 11
percent of the electricity.

This graph just shows on Federal utilities, the service areas of those
power marketing areas, and it shows that all parts of the country except
for the Midwest and the Northeast, benefit from Federal power. The
Northwest especially, benefits from Federal power in being a region that
has some of the lowest electricity prices today.

This shows the non-utility generating companies. They\'ve been around
really, from the beginning. It started with some industrial or
manufacturing companies that found it economic to use a co-generation
technology where they would produce electricity for their own plant use
as well as this heat for their industrial process.

But they really didn\'t have a market beyond their own facility and in
1978 when the Public Utility Regulatory Policy Act was passed, that Act
opened up markets to non-utility generators that met certain types of
conditions. They had to either be a co-generator or use renewables as a
resource.

And what the Act did was require electric utilities to buy power from
those qualifying facilities at their avoided cost. And avoided cost was
defined as the price it would have cost the utility to generate the
power or purchase it from another supplier.

So since the passage of that Act, the growth and generation of
non-utilities was quite extensive. And today, being a qualifying
facility and having that right of having the utility being required to
purchase your power is still significant, because over 80 percent of the
electricity generated by non-utilities is generated by the ones that are
qualifying facilities and have that right of forcing the utility to buy
their power.

The non-utilities have clustered really, in just a few states, and over
two-thirds of non-utility capacity exists in nine states: California,
Texas, New York, Florida, New Jersey, Michigan, Louisiana, Virginia, and
Pennsylvania. A lot of those states is because of the industry that is
contained in them and co-generation is attractive there. And then in
other states like California, it also have renewable resources that have
been attractive.

With competition the Federal bills that have been introduced are
proposing to abolish this mandatory purchase part of the Public Utility
Regulatory Act, because the non-utilities would be on a competitive
basis with utility generators in the future.

This next graph just shows how electricity is sold; that we really have
almost equal shares of electricity being sold to the residential,
commercial, and industrial sectors. And the pie chart on the right shows
that the revenues that are generated from those sales do not match the
percentage of sales; that residential customers pay 42 percent of the
revenues while they purchase only 34 percent of the sales.

And although this may seem inequitable, it\'s actually not because it\'s
based on the cost of serving that type of customer. And there are more
metering and billing expenses that would go to a residential customer,
and they also are charged based on a percent of electricity used during
peak hours of the day.

And homeowners contribute more to that peak in the morning or the
evening when they\'re getting ready for work or cooking dinner and all.
So that\'s reflected in the rates and shows why the percentage of
revenues are different than the percentage of sales.

The next viewgraph is just to talk a little bit about how utilities are
regulated. Because they were a monopoly, regulation was the way to
obtain economic benefits from the public and also have the efficiency of
the operation as a monopoly. And it was the substitute for the economic
kind of controls that would come about with competition.

There are two steps to regulation of electricity, and one is in the
wholesale market and the other is for the retail market. The regulation
really began with the beginning of the industry. It started with, when
utility was just at the municipal level the municipal government
regulated the rates.

But as utility service areas expanded it was necessary to set up State
utility commissions to regulate the rates for their state, and as
wholesale markets expanded and utilities found it advantageous to trade
and trade across states, the Federal regulation came about.

So this graph just shows that when the electricity is generated, if
it\'s going to be sold in the wholesale market, it first has to go
through hearings at the Federal Energy Regulatory Commission to set the
rate at which that wholesale power can be sold. And then when it\'s
being sold to the retail customer it goes through hearings at the State
Public Utility Commission to set the rates for the retail customers.

And the rates are based on what\'s called cost of service and the next
graph shows there are four different categories of revenues that are
required to be raised to cover cost of service. The first one is the
operating expenses: the cost of operating the plants and the
transmission and distribution that works in purchasing the power, and
other kinds of customer service.

They\'re also allowed to depreciate the cost of building their power
plants and they are allowed to recover taxes they pay to state and local
governments, and they\'re also given a return on their asset base. Those
are the components that make up revenue requirements.

In this graph it\'s showing \-- I split out transmission and
distribution so that you can see the different percentages that are
related to power generation versus transmission and distribution, and
the fixed charges for the plants.

In a competitive market, rather than cost of service, the price of
electricity will be based on the competitive market \-- the price that
the competitive market produces. And it\'s treated as the marginal cost
of the last supplier that\'s selected. The idea is that power generators
will bid into a market and the cost of the last generator that\'s
selected to produce power will set the marginal cost.

And all suppliers at that time will receive that same marginal cost. It
will vary by times of day, from \-- during peak hours the cost will be
higher than during the base low times of the day. And these graphs are
just trying to show, as percentage of time, what the price would be. So
that at less than ten percent of the time the price will be in a peak
price and there will be a long, flat time when the price is more stable.

So competitors will be looking at prices like that to bid in and then
receive the revenues that come from that competitive price.
Unfortunately, the revenues that are generated in this method may not
cover all the costs that a utility supplier has today.

On this next graph is showing some estimates of what are called stranded
costs. The estimates of stranded costs can vary widely by different
people that estimate it. We\'ve had ranges from ten million to 500
billion. And this particular viewgraph is from a company called DRI.
Later today I think you\'ll be given estimates that EIA has made in
their analysis.

I think the point is that the numbers themselves are probably not
accurate but they do give an indication of the magnitude by different
regions, and that stranded costs will be more of a problem in some
regions than others.

Stranded costs are composed of the capital part of a plant, the asset
base that has not been written down and recovered by the utility
to-date, and in the competitive market they can\'t recover enough to pay
back that asset cost.

It also may include some purchases from non-utilities where they were
paying prices at avoided cost, but in a competitive market they may not
be able to sell that power at that same cost. Or it could be some
contracts with fuel suppliers.

And a third type of stranded cost that exists are something that\'s
called regulatory assets. In the past, the State Utility Commission
allowed a utility to put something in their books called regulatory
assets which they said that they would be able to recover later.

It may have been part of a nuclear plant or some environmental costs,
but as we move to competition there\'s no way to really recover those
regulatory assets. So all those things are being lumped into what are
called stranded costs, and the states are deciding how much and how
those costs can be recovered in the future.

To better position themselves for competition, the utilities have
already started trying to lower their costs. And the biggest areas in
lowering their operating and maintenance costs \-- this graph shows that
the major part has actually been in declining fuel prices.

But there also has been some decline in the other operating and
maintenance expense, and they\'ve been positioning themselves to have
that be even lower in the future by lowering the number of employees
that run the power plants. Right now I think we haven\'t seen that in
the actual operating expenses reported because they pay severance costs
and other things for those lowered number of employees, but I would
expect those numbers to decline more in the future.

Another area where utilities are trying to position themselves for
competition is through mergers. By merging with another company they may
be able to lower their total cost, they can reduce their administrative
overhead.

And we\'ve seen a tremendous growth in applications for mergers that
have been sent to the Federal Energy Regulatory Commission in just the
last few years. And we\'re showing that the number of companies has
dropped from over 280 in 1986 to 244 in 1995. And that trend is
continuing in the future.

The last way that utilities are trying to position themselves for
competition is through investing in different types of subsidiary
companies. It may be that a holding company develops a subsidiary
that\'s a non-utility or they may be investing in oil or gas exploration
\-- they\'re production companies \-- and many of them are looking for
investments in foreign countries.

And as utilities have seen that the future is for competition, they\'re
working right now to get themselves positioned to be better to survive
in the new market.

And at this I\'ll turn it over to Larry who will now talk about the more
physical aspects of generation of transmission.

MR. PRETE: Good morning. I\'m going to try and give you a real broad
overview of the physical operations of the electric utility industry. By
understanding maybe you\'ll have a better feel when you understand the
changes that are taking place \-- either the ones that have taken place
or that are being proposed \-- how this will impact the industry.

In fact, I have tried to change my slides and I was working on it the
other night. My wife happened to see me at the kitchen table and she
said, what are you doing? I said well, I\'ve been asked to give a talk
on electric power restructuring at the ASA and I\'m working on a couple
of slides. And she says, oh no, are they going through an electric power
restructuring? My bills are going to go up.

And I said no, you don\'t understand. I said, the whole idea of
competition \-- they\'ve had competition in the airline industry, the
gas industry, telecommunications. I said that regulation is really a
substitute for competition, but a lot of people say it\'s been very
inefficient, and by having competition there\'s all kinds of numbers
being thrown out how rates have come down between 10 to 50 percent.

Well, my wife pays most of the bills so she went and grabbed all her
utility bills, took them out and started showing them to me and said,
look at my gas bill, look at my cable bill, look at our phone bill. She
said, you\'re not going to ever get me to believe rates are going to go
down. And it put me on the spot and I said, explain to me our phone
bill, and she took out six pieces of paper \-- and if you count the
advertising, seven pieces of paper.

What I have here \-- we have LCI which we just switched to \-- now AT&T
calls us every night about switching back. But we have the LCI bill,
state and local surcharges, the Federal tax. I\'ve got a 911 fee and
then a state and local tax on top of that. I also have a universal
service fee, I\'ve got a Federal and state tax on my universal service
fee.

I\'ve got an automatic savings charge. I\'m on savings plan; I\'m being
charged for a savings plan. I also have service not regulated by a
public service commission. I also have a Federal subscriber line charge.
And then on top of all this, I even have a miscellaneous charge.

Well, I tried to explain to my wife that this is all because they\'re
unbundling service. Well, after we went through this and I tried to
explain to her about what an unbundling service is, she gave me a new
definition of unbundling services believe me, you\'re not going to read
in any of the literature you\'ve been taking a look at.

(Laughter.)

This is just to give you a little background about what the current
energy sources are about how the industry generates right now. You can
see that most of it comes from coal, gas, oil, nuclear and so forth.
There\'s a reason for this. You have to remember that most of this
generation has been in the rate base for a number of years and been
built over time.

Most of this are your large, fossil, baseload steam units, and at one
time back in the \'70s when the industry was expanding, the industry was
taking advantage of economies of scale, where the larger and bigger
units you could build the more efficient it would be.

Actually, they ran out of the technology. I think the largest unit built
was about 1300 megawatts. They ran into metal fatigue so they probably
would have had to stop building larger units anyway, but they really ran
out of economies to scale.

The economy went down, the load that was doubling at about a compound
rate of seven percent which is doubling every ten years, that slowed
down. So they ran out of the economies to scale. But this is the time
period this has been built so currently, last year, this is how we
generated the electricity in this country.

This is to try and show though, over the last ten years, in the blue,
what was built over the last ten years; and then in the purple, what\'s
being projected for the next ten years. As we said, that the thinking
was historically, was to build \-- and these are large, baseload units.

And we could see \-- well, we all know what happened with nuclear; that
at one time nuclear fuel \-- in fact, there was the famous, back then
the AEC chairman that said, with the advent of nuclear power it
wouldn\'t be worth metering electric it would be so cheap.

Well, we all know what happened after Three Mile Island. Except for a
couple of power plants that were in the pipeline that went into service,
there have been no new planned nuclear units. But we can see how the
industry is changing from coal to gas, and if we were to show what the
non-utility industry is using for generation, we could see it\'s very
similar to the mix that what the electric utility industry is going into
the future.

But remember, as Betsy said that we had PURPA, which really prompted
this non-utility generation \-- that was passed in \'78 but it got hung
up in the courts whether it was legal or not \-- so it was really in the
mid-\'80s when the non-utilities really started building this capacity.

And it\'s really the fact that when they deregulated gas we do now have
a gas bubble; the price of gas went down. There is also a huge,
technological advancement in the construction of gas turbine technology
which is much smaller units. A gas turbine \-- think of a jet engine
like a gas turbine, but instead of the steam pushing the airplane
forward or the exhaust, it runs a turbine to produce electricity.

They\'ve much improved that technology \-- so much so that people are
able to build a gas turbine plant to use gas and to actually use that
technology and to generate electricity cheaper than many of the power
plants that are currently in operation. That\'s where we get into what
the stranded asset is.

One definition of a stranded asset is, is to look at what its revenue
stream would be without competition or under regulation, and then see
what its revenue stream would be in the future under competition.

And this is what your avoided cost is. And the fact that people can
enter the market and build this technology and compete, is what is
giving you \-- I\'m sorry, advice of the Board \-- of course I mean
stranded assets \-- is what\'s giving you stranded assets.

What I wanted to do is, before I really get into how the industry
operates in the transmission system and how the power gets from the
generating plant to your home, I just want to give a very simplistic
example of how the power flows. There is a huge difference \-- you and I
have heard people argue this \-- between electricity and let\'s say,
telephone lines or the gas.

Electricity is not a switching circuit; it doesn\'t have valves; and you
cannot store electricity. Electricity flows by what\'s called
Kirchoff\'s Law. It\'s going to flow in reverse proportion to the
resistance on a line. So I\'ve tried to give you a very simplistic
example, that if you had lines that had equal resistance but this line
is limited in only being able to carry 600 megawatts \-- for thermal
limitations or whatever the reason \-- that depending on which power
plant you dispatch makes a huge difference on how much power can get to
this load.

If I try and dispatch this generating unit, when I get up to 900
megawatts \-- since all these lines have equal resistance \-- I would
have twice the resistance going in this direction than I have in this
one. So when I\'m up to 900 megawatts, 600 flows this way, 300 flows
this way.

The moment I put one more megawatt on this line I blow this breaker and
trip this line out of service because it can only hold 600. The
electricity can\'t say to itself, well wait a minute, I\'ve got a lot of
excess capacity in this line; I\'ll flow this direction. It\'s going to
flow by the physical laws of physics.

But if I was to dispatch this generating unit you could see what would
happen. I could actually go up to 1800 megawatts because 600 would flow
in this direction being there\'s twice the resistance, 1200 would flow
in this direction, and I could get up to 1800 megawatts. So this is just
\-- I could also dispatch from both generators and it would fall then,
anywhere between 900 and 1800 depending on the loading on each of those
generators.

Now, even though in the future they\'re talking about having phase
shifters and better ways to shift the circuit, under today\'s technology
this is basically \--

MR. WOOD: Could I ask a question?

MR. PRETE: Sure.

MR. WOOD: Since you encouraged it. And it\'s related to something Betsy
said and something you may touch on. There are several, almost critical
things. One, as this restructuring takes place who is really responsible
for guaranteeing the reliability of that system? Who is really
responsible for guaranteeing we don\'t \--

MR. PRETE: I will touch on that. Rather than jump to it now \-- I could
\--

MR. WOOD: I\'d like to know that you\'ll get there \--

MR. PRETE: Yes.

MR. WOOD: \-- and there\'s several related questions. Two, how is the,
and at what level is access to that transmission line for electric power
going to be guaranteed? You know, can it go directly from a generator of
electricity to the individual consumer in his home?

And a third related question, even though my personal power company in
Texas is doing all the things that are mandated to help me conserve
energy, on the billing side they do everything in their power to make me
use more electricity and hide from me the costs of their providing the
service.

And is it feasible to have my billing reflect that cost of service \--
i.e., if it\'s 6 p.m. in the evening my rates are the highest \-- do I
get billed at 6 p.m. in the evening a higher rate to discourage my use
so I can control my power consumption? Rather than as my company likes
to do, they like to bill me the same amount ever month for the power I
use all year and they take care of regulating my access to that power.

That\'s a lot of questions but they\'re all related, and that lightbulb
can go out, and has.

MR. PRETE: I will discuss reliability. I think I will answer most of
your questions in the talk. And even though we are encouraging
questions, I hope \-- you know, if it\'s related to understanding this I
would cover it now. It you don\'t mind \-- since I think you\'ll have a
better understanding even later if I wanted to answer it, if you saw the
rest of the presentation \-- I\'d like to run through it and then go
back and answer your questions if I haven\'t, if you don\'t mind.

MR. WOOD: That would be fine.

MR. PRETE: Again, this is a very simplistic approach, but you know, it
will be needed when we talk later about how power is dispatched that
remember they\'re changing of the loading on the machines.

This could also be used \-- getting into what you\'re maybe alluding at
\-- that later when they \-- there is different theories about how to
price transmission and have congestion pricing, and whether the
congestion pricing should be based on a capacity charge or an energy
charge or the capacity charge and only cover the additional cost due to
congestion.

This would also help \-- and I\'m not really going to get into that \--
but this simplistic diagram would help explain that; where there\'s also
related to \-- there\'s a theory about nodal pricing or zone pricing,
where you put the nodes together in a zone. This simplistic diagram
would help in that.

Basically the United States is broken up into three major
interconnections: the Western interconnection, the Eastern
interconnection, and Texas. And by a major interconnection what I mean
is that all the utilities and all the generators in this area are
synchronized \-- and in fact it is connected up in Canada and down in
Baja with New Mexico \-- but by that I mean that all the generators, if
you were to look at the rotor, they all have to be in the same position,
being it\'s AC.

I may touch on something just \-- the reason we have AC for those of you
that don\'t know it \-- you could put AC through a transformer and step
the voltage up and down very simply. Transformer is 99-point-something
percent efficient; there\'s no moving parts; there little losses in
heat. But DC you can\'t do that. AC will make a North and a South pole
and when you change the North and the South being it\'s AC and it then
becomes the South and the North, there\'s electromagnetic lines of flux,
cutting of the winding and induce a higher voltage.

We could put that higher voltage on a transmission line and based on our
laws of physics what happens is, since voltage loss is I X R, if I could
lower the current in the line I lower my losses on the line. And being I
cannot make or create energy, when I talk about this transformer and
I\'m raising the voltage way up, as the voltage goes up the current goes
down. That keeps my power constant because power is my voltage times my
current.

So I step the voltage way up, put it on a transmission line, I can send
that power across the country with very minimal losses. If I tried to
send 120 volts across a transmission line, by the time you got to the
other end you\'d have 50 volts. So you\'d raise it up; it minimizes the
losses. That\'s the reason we use AC even though AC has some real
problems that you wouldn\'t have if you used DC.

But we have these three interconnected networks and technically they\'re
operating separately. You can almost consider them being isolated. All
the generators here are working together and then in Texas. The reason
for this is that with the Rocky Mountains and the distance between load
centers, they tried to build AC lines but due to the distance and the
amount of load, every time there\'s a disturbance and they got a reverse
flow, they would trip the breakers.

It\'s really for economic reasons. They technically could do it but it
would cost so much it\'s not worth it for the economics. In Texas it\'s
slightly different. As Betsy mentioned, there are two types of
legislation. There\'s Federal which is anything that goes across
interstate commerce, and then there\'s state legislation or regulation,
which is responsible for intrastate transactions like retail bills and
the siting of a power plant or the siting of transmission lines.

Texas, to try and avoid certain Federal regulations, has isolated
themselves within the state, even though in both these areas there are
some DC lines which are called asynchronous lines, so there is some
power that technically can flow between these regions. But for all, you
know, reality, these are three isolated regions.

But the real backbone to the system are trying to dispatch the power \--
remember, we talked about having 3200 utilities, even though only about
1,000 of them operate power plants \-- but they own and operate about
3500 power plants, over 10,000 generating units, and the non-utilities
have another 3,000 generating units.

The backbone of dispatching this power through the system is done
through what are called control areas. There\'s approximately 150
control areas in the country. These are not transmission lines; these
are communication lines between the control areas. The transmission grid
would go up totally different than that.

But all the control areas it\'s really \-- it\'s a huge computer system.
And what it\'s basically responsible for is maintaining a balance
between generation and the load within the control area. And so when
everything\'s normal that\'s basically what it\'s doing. It\'s
telemetering all the lines.

Now, it also has to maintain the trade now, between the control areas,
so it meters the tie line flow. And what happens is, when you\'re going
to actually send power between control areas, usually they do this on
the hour. They would schedule that so much power would flow from one
control area to another, and like I said, it\'s just a huge computer
center.

They have what\'s called the security constrained dispatch program which
is run like, every five minutes, and what it does is, it looks at what
the load has been in the past, what the load should be over the next
five minutes, and then sends out its signal to dispatch that power from
the generating units.

Remember we said that\'s the way that triangle \-- we said that\'s the
way you control the flow on the lines. It dispatches a signal to the
power plants to have the right amount of flow to meet the load. Though
every six seconds or five seconds, there\'s what\'s called an automatic,
generation control equipment. That\'s what\'s looking to see, do you
really have the right amount of power flowing between the control areas
and also is the frequency right?

Because remember, when somebody throws in a light switch technically
what happens is, the frequency comes down because that extra load is
picked up by the inertia in the machine. It\'s so slight you really
can\'t see it. But then the control equipment come out, steps the
generators back up, and puts everything back in sync.

So usually like every few seconds there\'s this AGC program that
actually goes through to look to see if the flows are right, where the
frequency is, and to adjust the generators. If you\'re doing economic
dispatch it\'s really more of a free-flow with the tie line, because
what you\'re trying to do is then tell the system, you go and dispatch
the most economic point or let\'s utilize the transmission lines in the
most efficient way.

As I showed you in that triangle, if you dispatch the wrong generator
you\'re very limited to what you can send to the load. But if that\'s
the power plant that\'s bid in it\'s not clear in the future, how will
we dispatch power?

If that\'s the lowest cost point and this is the way you could gain the
system, if you go and dispatch that as the lowest cost point and tie up
the lines, you could then be causing constraint on this system to make
additional profits from another generator. It\'s not clear how this will
work in the future.

Getting to your question on reliability, back in 1965 there was the
infamous power failure of the whole Northeast seaboard where we lost all
power. In fact, they even made a movie, \"Where Were You When the Lights
Went Out?\". That was where a line trip coming down from Canada caused a
cascading effect.

Based on that outage the government probably would have stepped in and
starting regulating the industry. Rather than do that \-- it was a
couple of years later, two or three years later in \'67 or \'68 \-- the
industry, basically the investor-owned utilities, got together and
created what\'s known as the North American Electric Reliability
Council.

They voluntarily formed this organization and it\'s this organization
which is really responsible for reliability in this country. They set
certain types of standards and operating procedures that we talked about
\-- these 150 control areas \-- that they would follow. So it\'s really
this organization that right now is maintaining reliability.

Under regulation, the way you would really, let\'s say maintain
reliability and generation, would be \-- they just do a very simple \--
one way to do it, anyway, is a loss of load program where you have a
whole lot of generators and if I know what the probability is of losing
any one of them, and if I know what my peak load is, I know how many of
these generators I have to have in any one time to meet the load.

So if I say that I never want to lose load except for one day in ten
years, I could then do just a simple, statistical study to see how many
of those \-- what\'s the probability of losing enough of them at one
time that my available capacity goes below my load? So basically, under
regulation, that\'s what they would do. They would go to the State
Commission, they would say look, we need to build more power plants,
they would get approval from the State to build it, and once it was
built and operational it goes into rate base.

It\'s not clear how the decision will be made in the future. It\'s going
to be based on price. People are going to look on this transmission
system, when you bid into the grid, what you\'re bidding at and is it
going to be profitable at that location to build a power plant to recoup
your costs?

When we talk about reliability of a transmission system we\'re really
talking about \-- it\'s a double part: it\'s really adequacy and
security. Adequacy is, do you have enough transmission, and the security
is, how well, the way this system is designed, does it tolerate a
disturbance on the system and then to be able to go back to normal
operation.

One of the things I didn\'t mention when I was talking about the control
areas, that in this program they have they\'re actually doing what\'s
called first contingency planning; meaning that they would never operate
the system in such a way that if you lost one generator or one piece of
equipment or line, that you would then cause a loading to change on
another machine to cause the next one to go down.

Because that\'s what gives you a cascading effect and that\'s what
happened back in 1965 where you got into a cascading outage and then you
lose load. Generally, the biggest reason for losing load is at the
distribution level. That\'s because you have radial distribution lines
\-- and let\'s face it, you\'ve got thousands of them going to all the
individual consumers \-- so over 90 percent of outages are due to
distribution.

Other outages, the higher percentage is probably due to transmission
where you lose part of your bulk power network, and that\'s what
happened last summer out West. In fact, one of the main causes they said
was tree-trimming, where the trees touched the line shorting \-- there
was bad communication between Bonneville and some of the generators up
on top.

They didn\'t realize how serious the situation was and before they knew
it, they did go into a cascading outage, even though there are certain
rules and we all know, I guess, rules are meant to be broken.

And then the system did what the bottom line is. You go to what\'s
called Iwinding. You have little pockets of generationing load that
you\'re able to match up until you can bring everything back together.

And then the least amount of outages would be at the generator level.
Now, when I say the least outages I\'m talking about where you lose
loads. Many times generators go down but as we said, we have this
reserve study with this excess capacity, so if we lose one generator,
and since they actually operate this system in such a way that if you
lost one piece of equipment you\'re not going to lose load, many times
generators go down but you never know it.

So being you have backup on that in the transmission, few are outages
due to generation transmission. The biggest one I can think of that was
really generator-related was the one we had here in 1994 \-- I think it
was the winter of \'94 \-- when the whole PJM system \-- that\'s a power
pool and we\'re part of that pool \-- where the weather was so cold and
it covered so much of the country, that everybody was running short of
capacity.

They had trouble starting some remote plants and coal piles were frozen,
there were a whole lot of incidents that took place, and they actually
were low on capacity. But that\'s one of the few I know of that was
really capacity-related. Most of them like I said, will be more at the
transmission line level, and of course the majority at the distribution
level.

I\'m going to just say something to try and answer your question. What
happened after that \'94 outage and in fact, even after the one out
West, we did work with another office of NG preparedness \-- I forget
the title of the office \-- helping them put together a staff report.

And what we did is, we actually met with NERC, the people responsible.
In fact, they put out a separate report and then EIA or DOE, Department
of Energy, went and put out an independent study. And what we did is, we
analyzed: why did this happen, what was the impact, and how do you
prevent this from happening again?

So basically that\'s how reliability is handled. After the outage out
West, Secretary \-- it was Hazel O\'Leary at the time \-- the Secretary
of Energy did ask her advisory committee to hold hearings on
reliability.

In fact, just this past Thursday they had the sixth meeting of this
reliability group \-- and I think it\'s being headed up by Congressman
Sharp \-- and they are looking into this whole issue of how are we going
to maintain reliability in a competitive environment.

Here I just wanted to say that there\'s a huge difference between,
let\'s say contract flow when you buy and sell, and physical flow. Even
though we have marketers and a lot of new people entering, and you\'re
going to have a lot more sales taking place, it\'s not clear whether
you\'ve changed the physical flows on the transmission system or not.

In fact, the way most billing is done right now it\'s really based on a
path, on a \-- I can\'t think of the word \-- on a \-- you look at lines
and you say, okay I\'m going to assume this is the path that the
generation went on. And we talk, when we get into a competitive
environment, of actually doing load flows, seeing how the power really
flows, and then you\'ll get into parallel flows and loop flows and bill
people for actual flows rather than just looking at the way you\'re
assuming it would flow through the lines.

But it\'s not clear, just because you get more buying and selling, if
the actual \-- if there\'s been a change. In fact, I was working with
the Environmental Protection Agency and some other offices trying to
answer this question; that in competition, are you going to have a
change of generation in the Midwest and will we increase the pollutants
of CO~2~ or NOX and SOX and so forth, because you\'ll get more
generation here and sent the power over to the Northeast. So that is one
of the issues: how will generation change in a competitive environment?

One of the things I just wanted to touch on \-- because I guess
economists would say, it\'s a failure of the market \-- how you cover
what\'s called public interest programs. Under regulation, the
regulators would approve the utilities to spend money to implement
programs like renewables, the environment, demand-side management,
low-income programs, and research and development.

But the problem is that in many of these programs it\'s not clear if
they\'re cost effective. And I don\'t want to argue the issues because
there are both sides to whether renewables are cost-effective, which DSM
programs are cost-effective. It\'s not clear if you spent money on
research and development would you get to recoup those costs; what would
you find; would you be able to use it?

But I just want to make clear that \-- and this is one of the major
issues as we go to competition \-- how will these programs be funded?
And there\'s been a number of proposals in some of the legislative bills
that have been proposed, handling some of these differences.

Some ways to do it is, you can have a non-bypassable fee on the
transmission system that everybody uses; that would help do it. Or you
could match State funds. Or of course, renewables, there\'s talk about
going to green pricing; that maybe people would be willing to pay a
premium for that.

In fact, I was at a meeting where they want to label power plants, and
if you think that telephone bill was complicated, wait till somebody
tells you that if you pick different marketers that this one has so many
thousand tons of SOX, NOX, CO~2~, and VOX, and this guy has a different
amount.

Now, you\'re going to be stuck choosing, do I want to go with this
supplier and maybe he\'s got a lot of particulate matter and I\'ll die
of lung disease; rather than picking the guy that\'s got SOX and NOX and
I\'m going to have acid rain and destroy all my plants and my car paint;
or do I want to go with the guy who\'s got CO~2~ and we\'re going to die
of the greenhouse effect? So you may have the choice of making this
decision.

When I told them it was so complicated I didn\'t see how it would work,
I got booed out of the meeting. And I really felt bad that night. I was
sitting home with my wife and I said, boy I said, you know, everybody
disagreed with me but it\'s so complicated I\'m not sure how to make it
work.

And I said something to her about she picked out the refrigerator we
bought. I said, I know that, you know, they do have minimum requirements
for now, refrigerators \-- you know, minimum efficiency requirements \--
in fact, DOE has that \-- and I said, did you get a low, medium, or high
efficiency refrigerator?

And she looked at me and said, what are you talking about? And I said,
you know, they have the labels. I said, which refrigerator did you buy;
what efficiency was it? And she kind of hesitated for a second and she
looked at me and she said, well, you know, I don\'t hear it go on and
off at night like the old one used to wake me up, so I\'m assuming we
got the high efficiency one.

I just can\'t wait to go back to another one of these meetings with the
labeling program and let them know about my wife buying a refrigerator
and what she looked at.

What this all boils down to, I happen to be in the data group of
electric energy, and what I\'m really looking at is \-- and there\'s
some crazy paper we wrote that\'s about 50 pages long that basically
summarizes a lot of what I said here \-- and the bottom line is, we\'re
looking to understand how the industry is changing and how does that
change our data requirements?

In other words, we\'ll get away from cost-based data. Maybe price will
be the key in the future. Do we want to be collecting data about what
power points they\'re going to build in the future because maybe that\'s
not the issue? Maybe there are other ways to look at reliability.

So one is, we\'re trying to look at the data requirements, but what came
out of it is, seven objectives that we felt we ought to try and be \--
but that should be implemented while we\'re reviewing our data
requirements. This would be equitability of data collection \-- meaning
there are utilities and non-utilities.

Right now we collect a lot more from utilities than non-utilities but it
doesn\'t mean everybody reports the same. It could be if a guy\'s large
or has more revenue he would report more data, but maybe there ought to
be more equitability.

A huge issue of confidentiality. In fact, we have a lawsuit that\'s in
the courts right now in D.C. where we\'re being sued for non-utility
data which is confidential, and the people feel that it ought to be made
public because they\'re stuck paying high avoided costs.

In the meantime, we have an electric utility that\'s in the PJM system
that\'s saying wait a minute. We compete now. They just went into an ISO
power exchange with interim approval by FERC. They said we\'re
competing; our data ought to be confidential.

So this whole issue of confidentiality, \-- we\'re determining that we
ought to look at our policy, revise the policy, and also have
equitability now as we go into a competitive environment, about
confidentiality, to try and do more electronic processing.

In fact, it\'s funny that we had \-- that the paper was written a few
months ago. FERC just had a meeting on Friday where they\'re talking
about something very similar \-- how to have electronic filing, how you
should file, how it should be disseminated \-- so at least there\'s
something in common that people could use.

And this is standardization. If data were filed using similar formats,
using similar codes \-- like if you used the data file by EPA in their
constant emission monitoring system which gives all the emissants from
power plants, they use the same codes that EIA uses in our file, so you
could merge those files. It would be nice if this was done across
Federal agencies.

Streamlining, as the administrator mentioned, Jay, that our budget has
been cut. We ought to look to see where we can streamline; what data are
we now collecting that we shouldn\'t collect in the future?
Coordination: trying to get together as I mentioned, with other Federal
agencies that uses the data \-- maybe even the States; and EIA has a
role somewhere in trying to coordinate the data that is put out on the
Internet.

And here is, as the industry changes we\'re going to try to take
advantage of new data sources. There\'s a future market, the NYMEX
futures market. There\'s going to be the OASIS system where maybe we can
tap \-- well, right now I think there\'s 22 nodes \-- whether we should
tap those nodes; with the ISOs, where they have a lot of information.
Maybe we should be going to them to collect information.

Because there\'s going to be so many players in the future it\'s not
clear how to go through all these players and collect data; so that we
should evaluate, if there are new data sources out there as the industry
goes to competition.

And then in the analysis side, here are for the next year, four of the
reports which we feel are important or going to be impacting the
industry, and these are the reports which in fact, Betsy is responsible
for, for producing these reports.

I do apologize if I went over my time. The only thing I can tell you is
that last week I had to give one for a Chinese delegation and none of
the other speakers showed up and they asked if I\'d be able to cover the
three hours \-- which I did \-- so I hope you\'ll look at this from a
positive point of view and say, hey look, he cut over two hours off his
talk.

(Laughter.)

CHAIR WATKINS: I\'m sure your wife is grateful as well. But we have
about 20 minutes for questions and we\'ve got kind of unbundled
structure here in that we usually have a designated discussant, but we
don\'t here. It\'s open to the committee first and then I would like to
open it up to the floor.

And the usual procedure, if anybody wants to ask a question or make a
comment, would they put their name tag vertically?

MR. WOOD: (Not on mike) The reliability part \-- if Betsy would like to
comment on, on how people actually can access the transmission line down
to the lowest level \-- i.e., the individual consumer \-- and how you
get into it on the other side. And then, is it expected or something
that the public can demand as they did \--. \-- if they could actually
control their usage and control their \--

CHAIR WATKINS: Can you use the microphone please, John? Our recorder is
objecting.

MR. WOOD: (not on mike) It\'s a question for \--.

MS. O\'BRIEN: First, on the reliability question itself. The Secretary
of Energy established a Reliability Board to look at how to assure
reliability in a competitive industry. And some of the findings that
they\'ve come out with so far said, the way we\'ve done it in the past
probably is not suitable in the future; that reliability was controlled
by the industry itself. Those North American Electric Reliability
Councils were industry groups and there was no government oversight.

The Board is recommending that it may move more into a format like the
SEC where you might have an industry group that looks at reliability in
a government oversight group, or they may recommend that the Federal
Energy Regulatory Commission takes more of a role in reliability.

So it\'s not decided yet, but they do have this panel of experts that\'s
been meeting to make recommendations, and so they are thinking about the
issue. On the DOE Web site the findings of that committee are there, but
it\'s difficult \-- you have to go through organization and people. If
you want to contact us we can tell you how to get to it.

MR. PRETE: But there\'s an interim report that will explain the findings
so far.

MS. O\'BRIEN: The other thing is on, I think a question about metering.
There is the ability to provide that kind of information of time-of-day
rates to a consumer. Some utilities in the past have offered time-of-day
rates to large industrial customers, and in doing that they had to
install the metering equipment so the customer would know what the
charge was at different times of the day.

I think the question in the future is the expense of that equipment and
how far down \-- would it go to residential customers or not. I\'m not
sure. I think eventually that\'s what will be necessary.

And the last part about getting the power from \-- if you choose one
supplier, getting the power from that supplier to you. I don\'t know.
We\'re actually going to visit PJM to learn a little bit more about how
they\'re going to do that.

Really, once the power gets in the system you don\'t know whether it
gets to you or not. So it may really be more a balancing; that the
independent system operator makes sure that your supplier puts enough
into the system so that when you as the consumer take power, your
supplier has provided enough.

But the details, like I said, we\'re going to go visit the PJM power
pool; that it\'s starting to implement those kinds of things shortly to
really hear how, in a practical sense, they\'re going to accomplish
that.

MR. MANICKE: Betsy. I will \--

CHAIR WATKINS: Can you hold on one moment?

MR. MANICKE: Excuse me.

CHAIR WATKINS: Do you want the \-- is it a short question?

MR. MANICKE: No, I was going to answer his question also. If Larry and
Betsy would allow me, I will also discuss this a little bit in my talk
this afternoon, about the independent system operator role and
reliability.

CHAIR WATKINS: Phil, you had a question.

DR. HANSER: Yes, just a couple of things. The first thing is I guess,
I\'m a little bit concerned about the numbers that are being used in
these presentations, just because they look to me to be delivered costs
of electricity. And I\'m a little concerned that they represent the
bundled dollars, including T&D and generation.

And I think you\'d be better off just putting out numbers which
represent the dollar amounts associated with the generation costs or the
price that\'s being \-- that customers pay for. Just because the fixed
cost of the electric distribution and transmission system are largely
not going to vary as you change competition.

And furthermore, since the argument that you could make about the
pricing of electricity is such that you\'d only expect that the variable
portion of the bill would affect the level of demand through the usual
sort of substitution effect. Whereas the fixed portion of the bill has
an impact only through the income effect and since that\'s relatively
small compared to people\'s income, you won\'t expect that in the long
run, to have a large impact.

So, the point is that in terms of putting out the statistics in terms of
the dollar amounts per megawatt hour or per kilowatt hour the customers
face, I think you would be better to break them up and separate them out
into different pieces.

I also was a little concerned about the New Jersey status. My
understanding is that the Board of Public Utilities or whatever the
current name of that agency is, that it essentially has created a
deregulated state and your map at this point in time shows it sort of
being in process. In fact, there are hearings right now on the size of
stranded costs and the nature of the process by which they will
collected in New Jersey.

And as far as PJM is concerned, it\'s treating New Jersey as if it was a
deregulated state for the purpose of setting capacity requirements in
PJM. So you may want to just check that.

I think a point on the cost side in talking about declining costs is
probably most useful to talk about in your diagram about, what I would
call fixed O&M and capital additions, since in essence, if you\'re going
to talk about how costs are changing, those are really the numbers that,
as a company, that would be concerned about future costs of production.

Those are the ones that are really important. And you may want to try
and see what those statistics look like since that in fact, is going to
be determinative of what the power \-- the cost it will clear.

On the mergers, I think frankly, that you\'re going to continue to see
mergers but frankly, FERC\'s decision on Primergy \-- as well as on a
couple of others \-- is probably going to have a chilling effect on
mergers in the future. That is to say, any merger which will tend to
increase market power or market concentration.

It\'s simply not going to be permitted as far as we can see, by FERC,
and so it\'s pretty clear that that downward trend in the number of
companies isn\'t probably going to continue. But you know, you probably
should think about the guidelines for mergers that FERC put out which
are essentially what come out of the Department of Justice relative to
the normal sort of mergers, and that will probably constrain mergers in
the future.

Rate reductions due to regulations. I think you know, we\'ve drawn a
rather odd picture of deregulations, particularly with regard to
residential customers. I got a bill from Boston Edison, my local
company, and my total bill is about \$90, and of that, about \$12 was in
generation. Now, if somebody comes to me and says, have I got a deal for
you, I\'m going to lower your rates by ten percent \-- your generation
costs by ten percent \-- it\'s \$1.20 a month.

The question I really sort of have to ask myself is, is this all worth
it, you know, at that point? Now, part of that bill right now includes
stranded asset cost collection, and that will end in six or seven years,
and so as a proportion of my bill, generation will go up.

But I think that relative to residential customers, frankly they\'re not
going to see a lot of impact on their bills right-off for several years.
And besides, as you\'ve already pointed out, a large proportion of their
bill is in fixed cost in the T&D system and you wouldn\'t expect that
deregulating the generation market would have a large impact on it.

The first contingency analysis that was mentioned \-- you know, I really
have a major problem with first contingency analysis, and it goes like
this. Almost every blackout that\'s occurred in the last 10 or 20 years
hasn\'t been an N-1 \-- i.e., one failure of the system \-- it\'s been
two and three and four, and in the case in the West, it was eight, I
think is what I remember as the count being.

I don\'t think that\'s going to change very much, frankly, over time.
It\'s an engineering criteria. I\'ve always sort of wondered why
somebody didn\'t do a more probabilistic analysis \-- much like you do
on generations but do it on the transmission system \-- to get to the
same kind of reliability requirements as opposed to the engineering
kinds of analyses. And I\'ve always sort of wondered why EIA sort of
didn\'t try doing something like that given the quality of the
statisticians it has.

I liked the comment about energy efficiency. We did a survey when I was
at EPRI, on refrigerators and trying to understand what the criteria
were that purchasers made of choosing various refrigerators. And on the
list of ten things, energy efficiency was 11th.

The number one rating for purchases of a refrigerator was whether or not
the surface of the refrigerator was textured, so as to avoid the showing
of children\'s fingerprints. So I think labeling \-- information
programs have been pushed by utilities for years and years. They have
very mixed results. You know, my bet is, I\'m more a believer in prices.

One last comment on reliability. I have a real concern about reliability
but I\'m more concerned about the regulation of it than I am about its
level. I\'m more concerned that what\'s going to happen is that benefits
from competition are going to be lost largely by the concerns about
reliability.

In particular, what I\'m concerned about is, is that our desire for
reliability is going to set some form of an engineering criteria and
which will create what I would call, a permanent situation of excess
capacity relative to the marketplace. In which case what will arise is,
is basically a retreat back to the system that we currently have in the
sense of, the price of capacity will end up being a regulated price and
that will deter the competitive market from really achieving the price
reductions that it could.

The problem \-- we\'ve had reserve margins set at 15 percent and
whatever else, but we\'ve also had a regulated market whereby the costs
of recovering the capital costs associated with reserve margins like
that have been done through regulation.

In a competitive market the cost of capacity is basically achieved
through plants being able to sell their power at less than \-- being
able to produce power at less than the market clearing price. And that
difference, that operating margin that they receive, goes to covering
the capital costs of their equipment. And that\'s the way competitive
markets operate.

So if you go to a system in which you set a reserve requirement and you
tell everybody, you know, this is what you have to have in terms of
capacity, you basically eliminate that as a possibility. And it\'s
fairly easy to see that that yield can bring you to a situation where
the permanent excess capacity and in the end, the cost of capacity, ends
up being a regulated price in order to be financially viable.

And I\'m a little concerned that nobody\'s sort of taken a look at that
or realizes that is a possibility. But I do know that the power
producers out there certainly are. End of comment.

CHAIR WATKINS: Richard.

DR. TABORS: I was just going to add sort of one comment to Betsy\'s and
that is that, we\'ve had time and use rates for a long time for large
consumers. I think the difference that we\'re seeing \-- and I\'ll make
some comments about it later on as well \-- is that the rates that
we\'re beginning to look at are going to something that\'s really a
real-time rate, or you need another piece of vocabulary to describe it.

Time of use rates are nothing more than sort of time blocks that get set
two years in advance if you\'re lucky, or a year in advance, and
there\'s been some long discussion in the industry as to whether they
have much of a response value.

To answer I think, the question that was asked, what we\'re looking at
now are something where the price of electricity is going to somehow or
other be reflective of the actual cost of delivery of that electricity
in real time, or forecast it a day ahead.

And that means, you know, if you use the UK system you\'re going to end
up looking at 48 spot prices for the next 24 hours, and that\'s very
different from a time of use rate. So my only comment would be just in
terms of vocabulary that that gives the wrong impression to anybody who
knows, you know, what\'s been going on in the industry and makes you
think, oh well, this is just kind of the same thing all over again.

I could comment on Phil\'s comments but I think that would be unfair.
General, which I agree with 99.99 percent.

CHAIR WATKINS: It\'s never restricted us before, Richard.

DR. TABORS: Well, I\'ll be polite because I\'m a guest.

CHAIR WATKINS: Calvin.

DR. KENT: I think everybody would expect me to ask this question. How
are we doing on standardization? I mean, you mentioned it as one of the
problems, we had a lengthy discussion about it last time around. I\'m
still very much concerned about whether or not we get the data, whether
or not we get the data in the standardized from so we can make
comparisons.

And one of the things I\'ll say later \-- maybe I\'ll say it now so I
won\'t have to say it later \-- that bothers me. When we see these sorts
of reports, is how good is the underlying data on which these reports
are based, and as we go more and more to deregulation we\'re going to
have less and less standardization of data. So I just want to know how
we\'re doing on that.

MR. PRETE: Like I mentioned, the EPA \-- Environmental Protection agency
\-- does use EIA codes as far as their constant emission monitoring
systems data, so they \-- in fact, they now are thinking about
collecting some additional data and they are more than willing to work
with EIA to have some sort of standardization.

What we\'ve been trying to really do is work more with the Federal
Energy Regulatory Commission. The thing is that they historically, have
been more interested in doing specific rate cases, or specific studies
where somebody comes in and let\'s say, wants to add a transmission line
or an interconnection. So they haven\'t historically been as concerned
about aggregating data or being able to computerize data.

They have recently gone through a reorganization where they now have a
data group \-- in fact, it\'s headed up by Bill Booth \-- and I think as
somebody mentioned, that they also will be looking more into market
power rather than be looking at the historic rate of return of cost of
service.

So looking at market power you need to understand the total industry. I
think there will be more inclination on their part to want to work with
EIA. As I mentioned earlier, this past Friday they had a meeting with
their looking into electronic filing and in fact, are also working with
Lynda Carlson\'s office in trying to get together with FERC and several
other offices in Standardization.

Now, there\'s several ways to standardize. One is with electronic
filing, and we have recently gone to electronic form with our Form 861
which goes to all 3200 electric utilities. Now there we give them a lot
of options on how to file \-- I mean, it could be faxed, it could be on
diskette, it could be through E-mail \-- because you\'re dealing with a
lot of very small utilities so we give them a lot of flexibility.

But on several of our other surveys we\'ve also gone to electronic
filing: it\'s in Windows 95 and we\'re trying to go more to E-mail. We
have like I said, met with the Federal Energy Regulatory Commission
several months back when they were going through this reorganization, to
try and talk about the benefits of standardizing and to get electronic
filing.

And like I said this meeting, they\'re going right along that stream \--
of changing their Home Page to a Web site, and where there will be a
much better exchange of data. At the last NARUC meeting \-- the National
Association of Regulatory Utility Commissioners \-- we did talk to NARUC
about also trying to maybe standardize the Web site between the states
and the EIA for an exchange of information; that we think there would be
benefits in that.

So we are working in several areas simultaneously to make this work.

CHAIR WATKINS: Thank you. We\'ll take one more question from Brenda,
because otherwise we\'re going to get congestion on this system if we
don\'t \--

DR. COX: Yes. Some of the themes you were discussing we\'ve heard before
in similar talks \-- the stranded assets, the idea of figuring out how
the system is going to work and how this affects EIA.

But there was one new aspect that I don\'t recall hearing before and I
wanted to bring it up just because I\'m hoping it doesn\'t get short
shrift in all the other things you\'re doing. And that is the
environmental aspect.

The effects in other words, high emission type generators \-- use of
them because they\'re low cost. You also talked about non-utility
generators and that\'s the first time I\'ve heard that issue discussed.
And so I would just like to say, I hope the environmental aspect
doesn\'t get lost in all the other things that you have to do.

CHAIR WATKINS: Do you have a quick comment on that before I close this
session? Dan?

VICE CHAIR RELLES: I\'m sorry. Was there a comment?

MR. PRETE: Oh, just as I mentioned, I have actually worked with EPA on
this issue about, with deregulation if there will be a change. As I
tried to mention, there\'s a huge difference between actual generation
and flows on the system, than just the fact that people are buying and
selling.

It may be the same power just being, you know, bought by a middleman and
resold, so I have worked with EPA trying to see if there will be a
change in environmental impact. And of course, EPA does have certain
rules and regs as far as emissions of SOX and they\'re relooking at
their NOX rules and so forth.

And as the administrator mentioned, we are looking into the greenhouse
program. So hopefully \-- I agree with you \-- environmental is a big
issue and that we won\'t lose sight of it.

CHAIR WATKINS: Thank you. Dan.

VICE CHAIR RELLES: Yes. Stranded costs have been referred to several
times, and I wonder if we could look a little bit further into it. It
seems to me that stranded cost decisions are going to dramatically
affect who can compete and who cannot compete in this new environment.

If I have a nuclear generator and you\'re allowing me to pass the cost
on to the State taxpayers, I\'m probably going to be a very effective
competitor. But if I\'ve got to sort of take it out of my revenues, I\'m
not going to compete very effectively.

So I guess the question is whether it\'s recognized how deeply stranded
costs can affect competitiveness and what\'s being done about it?

MR. PRETE: FERC has, in their order, said that they believe stranded
costs, 100 should be recovered. What\'s happened is as I mentioned, with
stranded costs, it\'s really a difference between your revenue stream
under regulation as it would be under competition. So that really means
that you have to estimate it.

And what\'s happened like \-- I\'ll give you one example \-- such as in
the state of California, what they\'ve had the three major utilities in
California do \-- because they will be going to full competition as of
January 1st \-- said that you have to cap rates to residential and
small, commercial customers, and then you have to lower it by a certain
date by ten percent, by another date by 20 percent, and that as of a
certain date they were capping the rates to the large, commercial and
industrial customers.

And that if you now could make yourself more efficient to be able to
gain the excess profits above that cap, that you would be able to use
that revenue as an opportunity to pay off your stranded costs. So while
the state of California is given the opportunity to recoup 100 percent,
it\'s not clear whether they will. But they are given the opportunity.

And what I\'m saying is, that\'s just one way of trying to recoup this
\-- I\'m sorry, stranded costs. And then what they did is, there\'s not
only stranded assets but there\'s stranded costs which are long-term
contracts \-- either purchased power contracts or fuel purchased. And
California has said that for the life of that contract you can continue
to recoup those costs, so you\'re not cut off. You can continue to try
and get those, but you have to make yourself efficient to gain these
excess profits.

Like I said, that\'s just one way, but there\'s a number of ways. And as
Betsy mentioned, that there are all kinds of numbers floating around
about what this stranded asset is, or stranded cost will be.

MS. O\'BRIEN: I think too, this is a very controversial issue and on the
one hand people say that the plants were built during regulated monopoly
system and there was a compact that they would be able to recover those
costs. So people that feel they should be recovered point to that issue
as why they should be recovered.

Other \-- in New Hampshire they\'ve taken the most limited view of
recovery of stranded costs, and they said unless it was a decision by
the PUC, and essentially for purchased contracts through PURPA, that
they don\'t feel that there was a regulatory compact for the utilities
in that state to recover their stranded costs.

But what\'s happened is, New Hampshire is being locked up in the courts.
So it seems that most states are \-- have to make a decision some \-- if
they want to move to competition quickly they feel that a lot of
recovery of stranded costs is just one of the conditions they\'re going
to have to assure the utilities to not end up in courts and delay the
whole process. But it\'s a very controversial issue and it\'s still
being debated.

CHAIR WATKINS: Thank you. We\'re on a very tight schedule here so we\'ll
close this session and start the next one at 11:15.

(Whereupon, the foregoing matter went off

the record at 11:13 a.m. and went back on

the record at 11:27 a.m.)

CHAIR WATKINS: We\'re now going to return to a more specific topic,
Electricity Pricing. Speaker.

MR. WHITMAN: I titled this session Electricity Prices in a Competitive
Environment: Next Steps. Through our preliminary analysis we published
in August the paper, \"Electricity Prices in a Competitive
Environment\".

Since it\'s already been out for three months I\'m not going to go
through very much or very little on the actual results that were
incorporated in that paper or most of the methodology. I want to focus
on just three issues for the committee to consider of work that needs to
be done in order to further the analysis.

Some of the key analysis issues that were in the paper. Our competitive
prices were based on marginal costs. This was a strictly marginal cost
of the last unit \-- just the most expensive unit dispatched in numeric
order. That consisted of the short-run marginal costs and in order to
reach an economic equilibrium we added additional factors such as the
reliability component and O&M and other adders to get a full price in
transmission and distribution.

One of the assumptions that goes along with that is that there was no
market power; that is, none of the producers could raise their costs
above the marginal costs of production. Also in the model we had reserve
margins were endogenously created balancing the marginal cost of
capacity with the marginal outage cost, where we defined a marginal
outage cost as marginal expected unserved energy times a value to the
consumer of unserved energy. And in the paper, transmission and
distribution remained regulated and they were taken as average costs.

The model that was used in that study was an offline model that worked
in parallel with the National Energy Modeling System; that is, NEMS.
What we did for AEO98 was, we integrated that model, called VALCAP, to
create a single, integrated energy model with marginal cost pricing.

The things that we added for instance, were end-use service prices and
consumption \-- that is, residential heating, cooling, that kind of
thing. We added into our electricity module endogenous capacity planning
based on the marginal capacity \-- cost of capacity.

And in the financing and pricing we used the marginal cost of the last
\-- or the cost of the last unit, most expensive unit dispatched to get
marginal energy costs, and recalculated the reliability component which
represented the capital component of the highest or most expensive unit
dispatched.

However, several \-- I want to focus on three issues that we need to
work on for the \-- continuing for the next AEO. First is sectoral
pricing. In our August paper we only calculated a spot price for a
single, each time segment. We didn\'t attempt to calculate prices for
the customer classes.

And secondly, the capacity additions were not done possibly as well as
they could have been, and primarily what we need to focus on now is the
way one calculates capacity. In an economic equilibrium we look at
market entry and market exit. In other words, those units would only be
built if the prices that resulted would support their total cost, and
those units that are too expensive to run \-- that is, the prices are
too high or not high enough to cover all of their costs, not just their
marginal running costs \-- should be retired.

The final issue that we need to look at is the transmission constraints
and the effect of transmission in congestion on new pricing at the NERC
region level or census region level.

Our customer class prices, in NEMS we look at residential, commercial,
industrial, and transportation. Typically, in rate of return regulation
customer class prices are based on allocating costs. There\'s a total
pool of costs from the total cost of generation transmission and
distribution over the course of the entire year.

And those costs are allocated to the customer classes in some manner and
from those allocations, those costs divided by the sales of the customer
class, give the rates for that particular customer class.

Typically, what one has done is, one looks at for the absolute peak of
the year, for instance \-- that\'s called at the coincident peak \-- and
looks at the contribution of each of the customer classes to the
creation of that peak. And from that one can allocate costs of fraction
of the cost to each of the customer classes and from there get the
rates.

A similar algorithm looks at say, the monthly peaks and allocates \--
takes a weighted average of the monthly peaks to get the contribution of
each of the customer classes to those peaks, and from there can get
rates.

Clearly, in the competitive environment one cannot allocate costs to
customer classes, so part of our next piece of work is to look at the
determinants of the customer class pricing. In particular, we believe
three of the issues that will cause customer class prices that will
drive customer class prices, are the varying demand elasticities between
the customer classes \-- industrial, commercial, residential.

The varying distribution costs, and in particular possibly the
cross-subsidization that presently goes on between customer classes may
be more difficult when the generation portion of the prices split off
from the regulated distribution.

And finally, in market power, the ability of suppliers to price
discriminate among their customer classes and whether that\'s going to
be a significant factor.

What we hope to do is start, of course, with the literature view of the
standard methods that we already have for allocation of the costs under
regulations, and look at the international experience of the customer
classes. We\'ve already seen some information from what happened in the
UK under the restructuring, the pilot programs in the United States, and
experience of customer class prices in the natural gas industry.

From that we hope to get a simple model incorporating the previous
factors, demand elasticities and market power, to get a simple model put
into National Energy Modeling System. And with that, develop enough
modifications to get customer class pricing for the next AEO.

Capacity additions are very important, if not the most important factor
in getting overall prices under a competitive environment when the
capacity comes online. In the present AEO we used the simplified
algorithm in what\'s called the capacity planning portion. We equated
marginal capacity costs with the marginal outage costs and used that to
determine what amount of capacity should be added so that those two
things were equal.

Returns were determined exogenously in the model. In true economic
equilibrium market entry and market exit should be as the proper measure
of capacity additions. What we should show is that new capacities should
have enough money \-- receive enough revenues so that it would meet all
of its long-run costs and the old capacity should retire if it does not
receive enough revenue to cover its fixed costs.

So what we intend to do for AEO99 is add economic retirement so as the
model runs, those units that cannot recover their costs will be retired,
rather than making an exogenous.

We will look at the marginal cost of capacity. In our equations we
looked at equating marginal cost of capacity with marginal outage costs.
One of the issues is that we traditionally treated marginal cost of
capacity as a fixed value, as the cost of a cheap turbine.

There certainly could be other ways of interpreting the marginal costs
of capacity. For instance, whether certain units not be retired, and
that might set the upper bound for marginal cost of additional capacity.

Finally, one of the things that we discovered, as the plant maintenance
scheduling becomes far more important in a competitive environment where
very small changes of capacity could have a very large influence in
price. Therefore, the plant maintenance scheduling is going to have to
be revisited such that it\'s optimal under a competitive, rather just a
rate of return regulation.

Finally, another significant issue in this model is that of the case of
transmission constraints in congestion pricing. Right now, the way the
National Energy Modeling System operates, essentially at each NERC
region level is considered a single pool. It\'s kind of like an ISO in
which merit order dispatch is used for all of the \-- all of the
capacity is put into the merit order.

Clearly, as you\'ve got constraints between the various parts of the
subregions, that wouldn\'t \-- it over-optimizes what that marginal cost
should be. So as we\'ll discuss later, we have another project that\'s
looking at the New England power pool and investigating congestion in
transmission. We need to start with that effort.

And one issue is, how much out of merit order and dispatch is going to
change our marginal cost? We have some marginal or out of merit order
dispatch \-- in other words, more expensive units are dispatched before
cheaper units already in the model \-- but we don\'t know, we have not
incorporated the effects of a true competitive environment with that.

Also, we don\'t have really a good knowledge of what congestion pricing,
congestion might do for the \-- influence the merit order dispatch. This
will all directly affect the marginal price.

Third, we have no capacity planning for transmission as it stands in the
model; right now it\'s a simple ladder to the generation costs. We need
to look at whether we should have capacity planning and how one would
put capacity planning in and whether that would have a major impact.

Finally, we need to investigate whether our present level of regionality
which consists of 13 regions, is sufficient to encompass the whole \--
the richness of different prices across the country and whether we\'re
going to need to go to a more detailed regionality and incorporate
different constraints between trade, between those regions, and
different merit orders in each one of those regions, to allow us to get
a good number, a good price, for each one of our census or NERC region
areas.

Our plans for this coming year then \-- in particular, the sectoral
pricing is a very \-- it\'s a very interesting and rich topic at this
point. We need to look at the basis of the sectoral \-- why the
residential, commercial, and industrial prices are going to vary, what
the effects of market power are going to be, and the effect of the
elasticities \-- the different elasticities amongst the customer
classes.

Secondly, we\'re going to need to add and will add, true economic
capacity additions represented by market entry and market exit. Finally,
we\'ll continue to investigate whether we need to change the regionality
of the model and whether our inter-regional trade has got to be modified
to more closely reflect a truly competitive environment.

Thank you.

CHAIR WATKINS: Thank you, Peter. Now we have two formal discussants from
the committee; first will be Calvin Kent and then Richard Tabors.

DR. KENT: I want to begin by passing out a compliment to the authors and
to the division of EIA which produced this report. It is very excellent
and the first time that I read it and recognized that I was going to
need to critique it, I thought maybe I just wouldn\'t show up and Bill
could take over for me and cover in my absence.

But realizing that that probably would not be acceptable and I also
would not get to see my grandson who lives up in this area, I decided to
come anyway and to make these comments. But I certainly do wish to
indicate that whatever comments I make should be taken in the context of
a compliment to the overall report and what was done here.

In order to save time I think I will dispense with some of the
introductory remarks that I wanted to make about traditional microtheory
and begin just by talking about some of the overall problems that I see
in using the short-run, competitive model as an appropriate way for
determining what the path of price is going to be in the future for
electricity. And this is more of a criticism of the overall approach
perhaps, than dealing specifically with some of the issues that have
been raised here.

But let me begin by saying that I am not at all convinced that the
short-run is the actual, relevant time period to be used for forecasting
purposes, and this bring us back to an issue that we\'ve already
discussed before and I think it bears even continued discussion and that
is, the impact of stranded costs and what they\'re going to be.

Basically, the competitive model just does not deal with the issue of
stranded costs because it assumes that they are largely irrelevant. And
yet I think that we can honestly say that stranded costs are at least to
a large degree, going to have to be covered even in the short run.

The stranded costs of the returns that are needed to cover the past
investment \-- and the fact that if past investment is not going to be
covered, that\'s going to have great implications for new investment \--
the stranded costs EIA estimates will be between 70 and 170 billion.

Now, the model assumes that the increased efficiency which is created by
competition, will generate some cost savings. Yet I feel that these cost
savings \-- and the EIA report points this out \-- will only be
sufficient to cover a very small fraction of the stranded costs.

In addition to this, EIA feels that savings on income taxes at the
Federal and the state level, will to a degree, offset these losses to
stranded cost and therefore, the impact of the stranded costs may not be
as high.

I should point out that the assumption that all state taxes are based on
net income is not a correct assumption. Most state and local taxes are
based either on gross receipts or property evaluations and therefore the
projected savings in the EIA model are not likely to be there.

I think the result is that utility shareholders are going to lose
sufficient amounts of money. The EIA estimate is that that at least 30
percent is going to be wiped away of shareholder value. I think that
that figure is at best, conservative.

I think that there\'s likely to be bankruptcies, although EIA again, in
their analysis does not feel that there\'s likely to be a large number
of these. But I still think that what you are going to see is that there
is going to be more problems in the short run with stranded costs than
what this analysis has concluded.

The EIA analysis assumes that over time the high cost stranded assets
will disappear and that ultimately generators will be able to cover all
of their fixed costs and add new capacity and that the expanding demand
that they see out there can be met.

The problem here as I\'m indicating, is with the transition and in
forecasting that transition. I think that the forecasting should not
assume that the long run is nothing more than a series of short runs. If
generators cannot recover all of their costs in the first short run when
these substantial losses will occur, then I think there is no realistic
hope of recovering them in the next short run period.

As a result of this, financial capital will flee the industry and the
next short run period will see significant increases in the cost of
capital, and there will also be a significant increase in uncertainly
and there will be a decrease in reliability.

An additional problem is also created as to whether or not these
financially-strapped firms who cannot recover their fixed costs will be
able to do the research and development that is embedded in the EIA
long-term forecast. And as a result of this, I\'m not sure the
efficiency is going to be there that EIA thinks will be there.

None of these results that I have just mentioned are consistent with the
falling costs of electricity which the EIA model produces. Let me give
you just one example out of their report and there are others that we
could list.

EIA correctly notes that there will have to be a reliability premium
added to marginal costs as consumers are going to be willing to pay for
some excess capacity to make sure that unanticipated peaks in demands
can be met. This reliability premium will be considerably higher and
will increase in ensuing short run periods as generators are unable to
cover their fixed costs.

This is why California and other states which are moving towards
deregulation as well as FERC, are leaning towards at least partial
recovery of these stranded costs in the short run. And I think the same
conclusion can be reached from taking a look at the British experience.

In fact, the increased reliability premium which I think will be
demanded by consumers, could actually push deregulated prices near or
possibly above the prices that prevailed under regulations.

Then to summarize the first point, forecasting prices based on short run
competitive equilibriums can be questioned on both theoretical grounds
and on actual, early experience with deregulation.

Second point: under deregulation, consumers or consumption will not take
place under competitive conditions. This has already been alluded to.
But the point here is that electricity pricing has always been
characterized by a single factor and that is, to the protection of
households.

On the demand side, pure competition simply is not going to exist.
Households simply do not possess the market power of large, industrial
and commercial consumers, and as a result of that there is not going to
be pure competition on the demand side of the market.

These large consumers will have the greater capacity to continuously
monitor the electricity market, purchasing electricity often just as
they need it. Households will not have this capacity or this expertise.

There therefore, is not going to be a single price or a competitive
market since households do not possess the market power, but they do
possess considerable political power, and they will be effective in
resisting any significant increases in their prices relative to the
prices that are paid by the larger consumers.

The third point that I want to make that the competition is not likely
to exist on the supply side of the electric market either, over many
short run periods. The reason for this is \-- and here perhaps, Phil and
I will have a disagreement \-- is that consolidation is going to take
place as more and more of the generating companies are forced into
mergers, particularly if they cannot cover their fixed costs.

To summarize then, what I am saying here is the question poised is, what
are the next steps that are needed in the analysis of prices under
electricity deregulation? I think that the current EIA approach have
probably overestimated the price reductions over the forecast period for
the reasons that I have stated before.

The purely competitive model is simply not going to be a description. We
have deregulation but we do not have the establishment of competition on
either the supply or the demand side of the market. The improved model
must recognize that pure competition is not likely to be approached over
the forecast period.

There should be an increased recognition of the reduced reliability of
electric supply under competition and its effect on prices. The impact
of stranded costs is likely to be more immediate, severe, and persistent
than the current model foresees.

Markets are likely under deregulation, to retain large, non-competitive
elements and conditions on both the supply and demand side, which will
cause some forms of regulation to continue, and the price of electricity
to remain above those which were forecast by a purely competitive model.

CHAIR WATKINS: Thank you, Calvin. We\'re grateful that your grandson is
here. Richard Tabors is next.

DR. TABORS: Thank you for the invitation, and let me see if I can be as
succinct and as well-stated as Cal was. First of all, let me compliment
the people who wrote the document. I think it\'s a spectacular effort.
What I\'m going to say is going to sound critical so take it as that it
was a spectacular effort and I\'m going to try and add some pieces along
the line.

I was asked really to comment on the three major issues, or open issues
that the author saw. The first one \-- I\'m going to do them slightly
out of order \-- the first one is capacity, and I\'m going to agree with
Cal but not quite agree \-- maybe that\'s the way to say it.

The capacity market over the next few years has every indication to me
of being totally irrational from an economic perspective. And I think
for the perspective of what EIA is trying to do, that irrationality
somehow has to end up inside the forecasting process in forms of
evaluation of uncertainty.

Let me give you only two quick examples and that is, that if you look
both internationally and in the United States at what is being paid for
capacity relative to what the so-called stranded costs or the book value
for that capacity is, as far as I can tell there is no relationship
between those two values.

Either that, or the people who are doing the buying are putting
incredible premia on the value of those assets. And this has a
tremendous impact, both in terms of how the stranded asset is going to
be washed through the structure \-- i.e., you\'re not going to have to
pay for it under these circumstances on the front end \-- or also in
terms of how we just think about stranded assets.

I work in New England. For anyone who watched the sale of New England
Electric what you saw was the sale to a subsidiary of Pacific Gas and
Electric of all of the non-nuclear assets, for \$1.6 billion. The book
value was \$1 billion or less. That\'s a 60 percent premium on book.
That\'s high; let\'s just leave it at that.

I\'ve worked internationally \-- extensively on sales of assets
internationally, and in the international world, whether you go to
Argentina or you go to Australia or take your pick \-- or to the United
Kingdom where I spent three years working \-- what\'s very clear is,
there\'s a feeding frenzy right now in the capacity end of it. So I
think that\'s an issue that we have to worry about.

The second issue is, I think there\'s an assumption in the entry and
exit \-- and I\'m only looking at the entry for the moment \-- that
somehow this is also economically-driven. I agree it\'s economically
driven but as this market starts, and remembering that the time domain
of the market is very long \-- 20, 30, 40 years \-- that we\'re seeing
tremendous quantities of new capacity lined up at the moment ready to
come online, in any of the zones where there\'s even a hint of a
tightness in the market.

And again, looking at New England, there\'s something on the order of
10,000 megawatts of capacity currently, partially licensed \-- some of
it fully licensed \-- to come online. That\'s a tremendous amount when
you look at the total capacity of the regions. So there\'s this
balancing question of how we model things that have to be described as
totally imperfect markets.

If I look at the exit question which I think is a tremendously
interesting one \-- let me just say there are two elements to this that
I think can be modeled and need to be modeled.

The first one is that if I\'m looking at elasticities of response with
regard to price, as opposed to time-of-use \-- so I\'m looking at an RTP
as opposed to a TOD rate \-- I think under those circumstances you\'re
going to underestimate the response of the demand side, and as a result,
overestimate the longevity of life of units that are out there.

Again, it\'s a question of the modeling structure at the present time
which I think is too gross to pick up these major issues. Now let me
flip that over and say, there\'s a countervailing force in terms of
retirements which has been alluded to, but let me make it as explicit as
possible.

What the Federal Energy Regulatory Commission allows for, or what the
state or regional commissions require in terms of reliability \--
particularly in terms of the long-term aspect of reliability \-- will
almost totally determine the longevity of the units that are going to
stay out there. And we\'re going to have to figure out how to pay for
those.

In New England there\'s an effort right now to have a market for
capacity. That market for capacity is above and beyond what economic
theory says we should have. And it\'s clearly a tax and that tax has got
to be taken into consideration in the modeling as we\'re trying to
understand how it\'s going to unwind.

The second question dealt with sectoral \-- sort of sectoral pricing
responses and I think Cal covered some of those. Let me just pick up on
a couple of more points in terms of the modeling capability.

Clearly, industrial, commercial, and residential have got to be broken
out as though they were markets, not as though they were allocative
functions. In addition to that, there\'s a tremendous difference when
you start playing with the SIC codes between big and small, and between
big within an SIC code, small within an SIC code, front edge and back
edge.

And I think these are challenges that are exceedingly difficult to get a
handle on. Some industries that I work with have electricity at 75 to 85
percent of their cost of final product; other industries have it at the
one to half-a-percent of final product.

The response to price in those circumstances is just dramatically
different. On the one case, clearly I have, you know, clients who are
just very interested in cutting their electric bill, but to go from, you
know, \$1.3 a month to a million a month is a big deal. To go \-- you
know, particularly when that\'s 75 percent of your bill. You can go from
\$1.3 million to a million a month and if it\'s one percent of your bill
\-- again, a little like Phil\'s example \-- I don\'t know that I care
at that point.

So the price responsiveness on these industries I think is critical. My
comment would be, in order to understand this we\'re going to have to do
the comparisons that were suggested earlier, but I think there\'s a
survey requirement out there in terms of U.S. industries just following
through and understanding what the capability of that response is.

And again, going to the modeling question, you can\'t do it without
getting into something that looks more like a real-time price-based
mentality. The hourly price, the spot price that we\'re seeing in the
market becomes critical because the way in which a large industry
responds and therefore what we\'re trying to model, is in fact, entirely
a function of that particular characteristic.

The hour-to-hour change is tremendously different from a 3-hour to
3-hour change. What I can do in an hour-to-hour is very different than a
3-hour to 3-hour. And if I look at an 8-hour block you can\'t \-- as an
industrial person you can\'t think in those terms; it\'s almost
meaningless.

Transmission was the third question. I think the zones that I\'ve looked
at in terms of the report are probably wrong. The NERC regions were put
together for a specific purpose. They\'ve held together for a long
period of time except Florida deciding it\'s going to kind of go off on
its own. They\'re either too big or too small; take your pick.

They\'re too small because in fact, if I look at the Northeast that\'s
truly one region \-- and electrically it has no meaning to chop it into
three \-- the fact that there\'s a boundary between PJM and the State of
New York is an historical accident and has absolutely nothing to do with
electrical flows in the slightest. New England is a case under itself.

When I move out West the zones begin to look like they make more sense,
but I think there\'s a need in the modeling to ask very seriously, what
is the objective and where are the transmission constraints and then put
the zonal boundaries \-- because what we\'re trying to do is get the
zones to represent markets \-- put those boundaries where they make
sense electrically today, as opposed to where they made sense
electrically in 1969, which is when we invented them, by-and-large.

Let me pick up just three other points that I think are important in the
modeling question as you move ahead. Short run marginal cost is what we
all use. I helped invent some of this stuff so I\'m guilty eight ways to
the middle.

But short run marginal costs, when you begin to ask the question of a
bidding environment thinking only in those terms, may not in fact, get
you the right number in terms of what the market clearing price is going
to look like. And the reason for that is as follows.

What we\'re learning in the games that are being held in what\'s called
now, experimental economics, is that people ask a question not \-- you
know, what is my short run marginal cost and therefore what\'s the
minimum that I can take out of this system \-- but they actually ask the
question, what\'s the maximum I can take out of the system without
losing my position in the loading order?

And what this means from a modeling perspective is quite simple
actually, but it\'s not clear how your model is. And that is that, I
will charge you in my bidding structure \-- not my short run marginal
costs but epsilon less than the guide just one unit above me in the
loading order. And that turns out to have a lot of money in it.

And so from the perspective of the industry as a whole, when we\'re
modeling things like stranded assets, we may well be modeling the wrong
number. And if we\'re modeling the wrong number and coming out with 170
billion or whatever that certainly large number is \-- as opposed to 70
billion which is probably also too high \-- there\'s a need in here to
sort of look at this and say, not what is theoretically correct \--
because I think we know what theory tell us, we\'re all good economists
\-- but in reality what we\'re now learning is that theory and real life
don\'t quite make it in the electric power industry.

And the reason they don\'t make it in the electric power industry is
because, unlike most other industries where it\'s a hunt-and-peck
operation to understand your competitors, in the electric industry we
really understand our competitors.

We understand it because I actually know who the next guy is in line
above me on the marginal cost curve, and if I don\'t understand it it
takes me only about three days of experimentation to find that point.
And I just play up and down and up and down and up and down and bing, I
know what his marginal cost is, and he plays up and down and finds it.

I\'m serious; there\'s a lot of money involved in that and I think
that\'s the key. It\'s a different way of looking at the problem. So
that\'s point number one on the modeling issue.

Point number two is, I think your modeling reserves incorrectly. And I
think it\'s an incorrectness that goes straight through the way we\'ve
all looked at it for a long time. What you\'re modeling in the model is
\-- what you\'re actually doing in the model, I think, is you\'re
modeling adequacy, and that has to do with both the time domain and the
approach that you\'re using.

Adequacy is interesting but it\'s like only one side of the coin. The
real side of the coin, and I think where most of the money is, is in the
operational or the stability end \-- or the security end it\'s called
\-- of adequacy \-- or the security end of the reserve.

Under those circumstances what you\'re shooting for is to understand how
to deal with the operational costs associated with \-- what I\'ll call
for the moment and lump a lot of things together \-- spinning reserves.
Spinning reserves are a cost of the system that unless we change the
rules, represent a first contingency planning criteria.

Which means that you\'ve got to have as excess capacity at any moment,
to be able to come on, enough capacity to cover the largest, single
outage function that can occur on the system.

On systems with nuclear power units or like we\'re hooked up to, Hydro
Quebec, those are 1,000 to 2,000 megawatt blocks that are sitting around
that have to be there by the rules that we live with, plus there\'s a
stochastic demand variability term in there that\'s critical as well.

My argument is, those things are not being handled at the moment and are
much more significant than the adequacy part that we\'re looking at at
the moment. The implication is there\'s just a lot there.

Third and fourth points, I\'m not sure how to handle non-fuel O&M, but
if you look at the 50/50 split that you currently have in the system, it
seems to me that second 50 percent is going to come back out at the
customer and it\'s going to get bid into the system because everybody
has the same problem; what you\'ve done is, you\'ve moved it over to the
other side.

And I think what I\'m going to do is stop there and say thank you for
the opportunity to comment.

CHAIR WATKINS: Thank you, Dick. Well, we\'ve got about five minutes left
or so. Phil, you look as it you\'ve got something significant to say.

DR. HANSER: Just a couple of comments. Do we have the time?

CHAIR WATKINS: Yes, but could I ask you please, just to get to the
microphones? The reporter tells me it\'s only got about a 2-foot radius.

DR. HANSER: I guess just a couple of things. One of them is a question
about whether you want to model reality as it exists right now or
whether you want to model what you think will be reality.

And in particular, you have examples right now, for example of PJM
already having a reserve margin requirement. Does it make sense to
impose an economically rational means of getting to capacity when in
fact, the Reliability Council has already decided that it\'s going to go
for reserve margin?

So you have to make the decision I think, in your modeling about which
way you want to go that way.

I\'m a little concerned about the fact that you\'ve got to kind of \--
for lack of a better word \-- jiggle the census regions to match the
reliability councils. I\'m not quite sure how you do that to do it quite
right.

I remember a long time ago when we did some other modeling, that that
was a similar problematic kind of thing. I don\'t know how much bias
arises in the model as a result of how you have to do that, because all
those allocation rules tend to be arbitrary and vary, you know, from
data imputation; that you can get whatever answer you want given how you
want to impute the data.

I think the issue about falling costs of power probably isn\'t a
function of R&D or anything else at this point, to be honest. The simple
fact is you can now buy a combined cycle gas turbine from a reliable
manufacturer, and the cost of that is about \$300 per kW for those
units.

If you look at the typical cost of almost all the other plants that are
sitting on the system, they are considerably higher than that. And so
long as gas prices remain low and gas remains available, I think you\'re
going to see falling prices on the generation side sort of no matter
what.

And lastly, I think there\'s a real question in my mind about the
generation mix and what you\'re thinking about in the model and how do
you get to that generation mix. That, at least in the modeling that
we\'ve done looking at regions, is very important. It changes the market
clearing price substantially; it also changes the stranded assets that
are sitting out there.

I\'m not sure what is the appropriate way to get to that; whether you
want to think of optimal capacity expansion or whatever else. To some
degree you know from stuff that Ralph Turvey did in 1961, I guess, that
the price generation curve is essentially fixed by whatever is the
marginal cost of the incremental generating units.

You can use that as a means of backing out to where you ought to be, and
that\'s something to sort of think about and you might want to take that
approach. But I do think the generation mix is fairly important.

One last thing. the environment I think, comes into this in a funny way
that nobody\'s really sort of thought about \-- at least that I haven\'t
seen. In particular, I\'m concerned about the NOX production
implications for coal units because as proposed regulations go through,
those kinds of things could have very large implications.

There is for example, very little available in the way of selective \--
in the way of SERs, like the catalytic reductions for coal units. That
could drive up the price fairly enormously. Also, if you simply went to
bubble systems for your SO~2~ transportation and the pricing of permits,
you could dramatically change the value of the permits and the cost of
power. Those are two very simple things that have been talked about on
the environmental side and that could have large implications.

And I don\'t even have to talk about greenhouse gases and CO~2~ taxes
and whatever else. They really throw a monkey wrench in all of this. So
those are just some things to sort of think about on the environmental
side in terms of the modeling.

CHAIR WATKINS: Thank you, Phil. Peter, I would like to give you an
opportunity for responding, quickly, before we move on to the next
session.

MR. WHITMAN: Let me say just a couple of things. On stranded costs, the
study specifically did not add stranded costs in. I would go with saying
that the \-- one can argue that the stranded costs should be added not
as a price as a marginal cost but in some other manner. And in that way
it would not affect the economics of the price reductions. That could be
added as a separate chunk.

On issues of the long-term versus short-term pricing, in general I
certainly agree; we should be hitting long-term pricing. My
interpretation of what we\'re trying to do is that the long-term price
in a large degree, is established in the capacity planning model which
is a 4- or 5-year extension of the prices and that the short-term prices
that are calculated to hourly or in live slice, are really just for
determination of the demand response in a particular year.

And the third issue relating to sectoral pricing, I think we would
totally agree that there\'s going to be a large number of political
factors that we have not accounted for, and it would be difficult to
account for all the different aspects of that.

CHAIR WATKINS: Thank you. Is it a short one, John?

MR. WOOD: Well, yes. In the drilling industry, in the boom in the late
\'70s and early \'80s, a capital expenditure of roughly the same
magnitude as a stranded cost was probably made. And then the prices
cratered and lots and lots of those companies went bankrupt. And you
went from \-- speaking of reserve capacity \-- you went from having 4500
rigs active to 600.

So what\'s wrong with letting all the utilities that can\'t make it in a
fully competitive environment declare bankruptcy, not pay their
creditors, and compete like airlines have done? Because the bankruptcy
laws protect them and if that takes down the banking system the way
America has dealt with it in the past is, you bail them out.

And just as a working assumption, that anyone that can declare
bankruptcy can compete.

CHAIR WATKINS: One phrase we haven\'t heard yet, and I was going to
bring it up later on, is the obligation to serve. And this is part of
the elements in that equation.

Anyway, I do have to close this session now to move on to the next one.
Could I have the next speaker, please?

MR. BOURNAZIAN: I\'d like everyone to first imagine themselves playing a
game of chess and you wish to move the queen on the board. And if
you\'re familiar with that game, the queen can move in three different
directions: vertically, horizontally along a row, and then even
diagonally. And so in making your selection you\'re going to first go
through each combination of move along the row or column or diagonal,
and then you\'ll finally make your selection and move that piece.

That\'s the same methodology we applied for petroleum marketing data at
the Energy Information Administration in protecting the confidentiality
of survey data that\'s reported to us. We use this methodology in a
couple of publication. One\'s an annual publication. It has basically
2-dimensional data tables. And then we use it again in a monthly
publication. It has about 50 tables in it; some of them go up to
4-dimensional.

It doesn\'t matter whether we\'re talking about crude oil or refined
products, monthly data or annual data \-- the methodology remains the
same.

And the few terms that I\'ll be using in the presentation today. I just
want to distinguish the grounds between primary and complementary
suppression. All data that\'s reported to us, we have to preserve the
confidentiality for these companies, so we were going to test data to
make sure that this publication cells are not so concentrated that
we\'re revealing an actual company\'s data.

We call that primary suppression, however as you can see in a simple
graph there, we call it a single number because it\'s sensitive to
disclosing company-level data. You\'re going to have to go back and
withhold other non-sensitive data because you can certainly identify
that through subtraction when we\'re compiling a total. We call that
complementary suppression and that\'s going to be the focus of our
presentation today.

Because when we\'re designing complementary suppression program we have
a couple of objectives for our monthly publication here. We first do get
a large amount of data, about 15,000 cells in these tables, and you add
four dimensions to the data and that would complicate or at least
something we had to tackle in having an automated system, especially
when you get into gasoline under the product category; that\'s been
expanded briefly in the last five years.

Today the way the program works is with 15,000 cells we end up under
primary suppression, suppressing about 1200/1300 cells, and about
another 500 to 600 cells with complementary suppression. But in the
process I\'ll be explaining today, just keep in mind that this program
does the entire pattern in roughly two minutes.

Another important point for a complementary suppression system is we
wanted to minimize the suppression of non-sensitive cells. We\'re in the
business of getting the data out to our customers and so you want to get
much of the non-sensitive data out there published.

We also wanted to avoid suppressing totals. In our monthly publication
especially, you want to be consistent every month. Again, our customers
are building databases, time series data, and so you want to minimize
the interruption by suppressing different cells on different rows and
different patterns.

And most importantly, most of these data in the tables is going to
relate to other parts. We have volume data which translates into
percentage marketshare tables, we have price tables that compare all the
volume data.

So it\'s possible, for the system to take into account was that you
could identify a suppressed cell by using a total from another table. So
we had to make sure the system looked at all the relationships.

I just want to touch briefly on progress suppression. All the data \--
and this is how the system starts off \-- we take all the data in the
publication file and we\'re going to subject it to a formula. This is an
objective test to determine whether these cells are sensitive when
exposing company-level data, and the \-- focus on the formula itself
other than the fact that I just want you to take back with you the fact
that the formula is focusing in on the largest seller and the second
largest seller in that publication cell.

It\'s going to generate a value for the dependent variable and it\'s
zero or positive, that cell is sensitive to disclosing company level
data and it will be suppressed. And that\'s what generates about 1200 to
1300 hits on the 15,000 cell publication file.

The next step though is, once the suppression code has been written to
the publication file and we know which cells have failed primary
suppression, it\'s going to load that file into two, 4-dimensional
arrays. And when it\'s doing that it\'s going to initialize these arrays
by the data characteristics of the cell.

So here the data publication file has all the numbers which are going to
go into the publication. Well, it\'s going to go through a crosswalk,
and so now it doesn\'t care where this cell is actually being published.
It wants to describe it as retail gasoline number from Mississippi.

And so it\'s initializing down product, down geographic, down sales
type, and it\'s describing that cell. So now it no longer cares where
that cell appears \-- if it appears once or 50 times \-- it just wants
to describe it.

The second 4-dimensional array is carrying corresponding volumes at
every cell in the SW array. And we\'re going to use this second array as
a tiebreaker when it\'s selecting canopies.

I\'ll give you a simple visual of what it\'s going to be looking like.
As the data comes in it\'s going to create a series of suppression flags
which will signal the system for where to look to find a complement.
Flags really move this system. And so here, where a cell has been
suppressed in primary suppression, flags will register along respective
dimensions on where additional suppression is needed.

And again, the volume \-- volume data is loaded here in this array \--
I\'m going to come back to it later because it\'s not going to be the
actual volume data which corresponds to the data in the publication
file. The data in that volume array is going to be adjusted data.

Just to understand the flags, then can only have three values; we keep
it simple. So when the computer is going through it will first identify
where no suppression is needed at all; where it\'s going to be looking
for candidates is number one; and then two, it will also want to keep
track of where it has picked candidates and where no more suppression is
needed.

In the area of product and sales dimensions, now this takes on more of a
complicated structure. For example, we have two types of sales: retail
and wholesale. In the retail category though, about four years ago we
added petrochemical sales to our survey forms and started collecting
that data. It didn\'t cause any real change in the programming for the
system because retail itself becomes a separate marginal total in which
the separate categories add up to it.

Four years ago we started breaking out wholesale categories into three
different categories, pricing them separately. We created a separate
flag for wholesale and now that becomes its own marginal within three
wholesale categories.

Likewise when the clean fuel programs began, gasoline we had just
originally published as unleaded gasoline. And then we had introduced
mid-grade in 1987, so we always had three grades of gasoline. But then
when the clean fuel programs came, we had to change our publication,
change the survey forms, and the suppression system really didn\'t have
to be modified much.

Because again, gasoline was broken out into its three formulations and
in each formulation you had your three grades. So then we just added a
marginal total for each one of these three.

The same thing happened for low sulfur diesel when that was introduced.
Didn\'t have to make any modification; you just created a new marginal
for diesel fuel and then we were able to collect and publish low and
high sulfur diesel.

And then finally geography, which is the first dimension it always
begins with. We always wanted \-- our big concern was you would lose a
total by accident when you\'re running a complementary suppression
program. So when the system is going through its procedure blocks it\'s
always going to be looking at data from the highest level of aggregation
down.

So it looks at the U.S. and the regions within the U.S. It then goes to
a single region and the states within a region, because again the
concern was, you never wanted a single state \-- Iowa or Rhode Island
\-- taking down a regional number just searching for a complement. And
we can guard against that by the way it applies when it comes from a
top-down approach, always looking at the finest level of disaggregation
last.

So to finish this example on how a simple selection would work, it goes
through and it\'s going to start looking for its candidates, and it will
go take each dimension but it will look through all dimensions first
before it makes a selection.

Here, whenever it\'s several candidates it has to break the tie so it
goes over to the volume array and it will pick the lowest volume, make
its selection. Here\'s where I want to talk a little bit more about that
volume array because it doesn\'t have the actual data that\'s being
published.

What we go in and do, is we adjust it. We take all that data from the
marginals and column or row totals, and then we\'re going to subtract .1
and overwrite all of the data in the array. Here, while we do that, a
simple example. Suppression pattern would probably come up here and
suppress those block of four cells up there because it sees a low
volume.

What you do after data adjustment though, it\'s going to go down and
select a different block of cells. What you\'re doing by doing the data
adjustment is you\'re preserving the rank order of the cell within a row
or column. But then again, you\'re ensuring that the more higher volume,
more important cells that are being published, stay published and
aren\'t interrupted. And it also gives us a much more stable suppression
pattern on a month to month basis.

So going back to the example, once it makes a selection it goes back and
it will re-initialize all the flags and all the dimensions, and it only
takes one cell at a time. Then it will go through, continually looking
for candidates, and every time it needs to break the tie it goes over to
that volume array.

Once again, every selection requires all the flags and all the
dimensions to be updated. And again, here in the example, you know, some
rows and columns are getting completed but then it triggers suppression
and it keeps on its analysis until finally, where it\'s trying to
minimize as if a cell can serve as a candidate in more than one
dimension it automatically picks that. And that\'s how we\'ve used the
system to minimize the number of complementary suppressed cells.

Obviously with four dimensions it will go all the way up for an
intersection where one cell could serve as a candidate in three or four
dimensions. Whenever it picks one that has an intersection of more than
one dimension, that will be the choice. It doesn\'t even look at the
volume data underneath it.

Then it will continue to review until all the flags tell it that it\'s
picked complementary cells to suppress and no more suppression is
needed. Again, in the 15,000 cell file it will do this entire step and
pick its 500 candidate cells within two minutes.

Then once its assigned the codes to who\'s been selected as a complement
in this file, the file has a code containing what\'s failed primary
suppression, what\'s been selected as a complement, and is
non-sensitive. It\'s going to go back and it has to write this
information back.

Now, it\'s going to go right back to its crosswalk because in these
arrays, all it has is data characteristics. It knows its retail, it
knows through the product, it knows the geography. It has no idea where
it\'s appearing in the pub, but it knows that it\'s picked some of these
cells to be suppressed. So it goes through its crosswalk, now takes the
data key which has its characteristics, and it matches up to where the
cell appears in the pub.

So again, it doesn\'t matter if this cell is published four times or
even just once; it will find that out when it goes through the crosswalk
and then the pattern appears in the publication file \-- the output of
the publication file \-- and then you generate your hard copy pubs.

So what I want you to really take back with you on this automated
suppression system is that, one, it\'s easily modifiable \-- and it\'s
been actually redone three times in the last five years \-- as new
gasolines have then come and the pubs have to be redesigned.

Second of all, it gives you the flexibility for not only modifying your
survey forms and publication files, but gives you a low cost for
continuing to look at minimizing your suppression and stabilizing your
pattern. And so, here it\'s worked for us but we\'ve always been looking
at news ways for improving it, opened other areas to look at \-- I mean,
ensuring the production of the confidentiality data \-- and then second
of all, we\'ve been looking at always using this on other publications
and continuing to expand it because it\'s applied well for annual and
both monthly data.

CHAIR WATKINS: Thank you. Carol.

DR. GOTWAY CRAWFORD: My first comment is that this is a very important
paper. And it\'s very important for a couple of reasons. The primary
reason is, I think that data confidentiality is becoming a huge concern
as more and more industries and agencies collect more and more data at
finer scales, and at greater detail.

This is particularly true, I think, in the environmental reason where
they\'re actually measuring things down to the township level and
conveying things graphically. It\'s also a huge issue because the public
is demanding more and more access to information via the Internet, and
ensuring the confidentiality of data is important in that realm also.

I thought the paper was very good. I think that \-- I\'m not sure that
in his presentation Jacob really conveyed to you, but it was conveyed in
the paper \-- how simple the premise is. I mean, you want to suppress
cells that could lead you to reproducing sensitive information.

On the other hand, I don\'t know that he really conveyed to you how
complicated that really is when you\'re dealing with the number of
tables that they have to deal with in a publication and the large
dimensionality of those tables. So I think the premise is pretty simple
but its implementation isn\'t all that easy.

One of the things that I really liked about the paper and the algorithm
that was presented in the paper, was that the objectives for the
development of the suppression code \-- what was it supposed to do and
how it met them \-- were very detailed. I mean, there was a list of them
that you couldn\'t miss and that was the objectives of the program.

And often, in terms of self-suppression, one of the criticisms of it is
people are not always sure what the programs are doing, and that\'s
important in understanding how the cells come to be suppressed.

I also looked to see that, it certainly did seem to be effective, at
least in the publications that Jacob mentioned, it seemed like it would
be flexible and easily modified for other publications, and it\'s also
consistent with EIA policy. All the Federal agencies have policies
regarding data confidentiality and I checked with the policies I could
find it by EIA and the program honors that, as he said in the beginning
of his presentation.

I have a few comments and questions. Some of them pertain to the
methodology that Jacob presented and some of them concern cell
suppression methodology in general.

The first comment I have is, Larry Cox in 1980 I think, is the primary
developer of the mathematics behind complementary cell suppression. And
I went back and I read his paper and then I looked to see other people
that were discussing this methodology.

And I went all the way up to the, I guess 1989 was the last paper that I
found that mentioned the mathematics behind this, and I found two papers
\-- well, Cox states in his paper that his methodology \-- he uses graph
theory to decide which cells should be suppressed and ensure that there
isn\'t a unique solution to the resultant table with suppressed cells.

And so my question \-- but he mentions it doesn\'t apply to higher
dimensional tables. It\'s only 2-dimensional tables for which this
applies. I also found two other papers that mentioned that the
mathematics for higher dimensional tables were intractable. And I put a
question mark here because like I said, I could only find references up
to 1989, and so perhaps some mathematical development concerning this
has been developed past 1989.

Even if it hasn\'t, it\'s not clear that there could be a glitch because
of it, but I just raise the question: is there mathematics behind
applying this theory to higher dimensional tables?

My other comment, my take to this paper and complementary
self-suppression in general, is how useful or meaningful are the
resulting tables? I mean, what information is it that you\'re trying to
convey? What do you want your audience to get from there? Because the
numbers have been adjusted; some of them have been suppressed.

When should you look towards maybe data aggregation as a way to get
around this and perhaps would that be a more meaningful display of data
then by breaking things down so far you have to suppress certain cells?
Just a question.

It\'s very interesting too, in opposite to this, Samprit and I were
talking and he said, well I have a solution. So if we go back to this
final slide where the Ws are, he filled in the numbers and he says, here
is my solution. Well, the thing about complementary cell suppression is
there are many solutions to those Ws, okay. So what they started out
with is one, but there are others. So there is no unique solution;
that\'s what the algorithm gives you.

But what if somebody didn\'t know that? What if somebody said, aha, I
guessed what the Ws are; I\'m pretty smart. And then they went out and
started operating as if that were the case. What if a lot of people
started to do this? I mean, what kind of information actually gets
transmitted by publishing tables like this?

My third comment about this was, I couldn\'t tell from the paper, how
thoroughly was the program tested? Because I think anybody that\'s done
\-- worked with large computer programs knows that it takes a long time
to test preconceivable combination of numbers that might throw a glitch.

So I didn\'t get that from the paper. I was just kind of wondering, how
well did it perform, were there any tables or any situations that it
might now work with, any kind of, I don\'t know, screwy table full of
numbers that might cause it to break down?

And my last two comments are more general. They don\'t necessarily apply
to this paper but more to data suppression in general. What other kinds
of data need to be protected? I\'ve seen a lot of graphs. I\'ve seen
maps of the United States with price summaries on them. I\'ve seen last
time, time series. What kind of suppression methodologies are there for
those and to what extent are we worrying about suppressing that kind of
information?

And my final comment, that I can\'t understand why I don\'t see this
discussed in the literature is, who owns the data? Who owns the data and
who are we suppressing the information from and under what circumstances
do those two entities coincide?

And I\'ll leave you with that thought and I guess it\'s about lunchtime.

CHAIR WATKINS: Thank you, Carol. I would \-- you raised some interesting
questions and issues. Could I ask you to respond quickly?

MR. BOURNAZIAN: Well, beginning with your last point on data ownership.
It\'s been a big topic with OMB but I think it\'s clear, we \-- that EIA
owned this data but in trust with the companies, so I think we have that
obligation to protect that confidentiality.

The best way to test a system is to run it through an audit with a
linear program formula which would have a minimization constraint. And
we did that when we first developed it in the late \'80s. We haven\'t
been able to run an audit in the last four or five years, but that\'s
probably the best way to test any system to see how close you\'re coming
to minimization.

And then as far as \-- you raised a point on \-- I guess that was
basically it on her two points.

DR. GOTWAY CRAWFORD: Cox\'s algorithm \--

MR. BOURNAZIAN: Oh, maybe I didn\'t complete that. The numbers then, the
adjusted numbers get thrown away once the complementary suppression
finishes and the publication files always carry the actual data. So that
was my fault for misleading the group. What\'s published is the actual
survey data. We just use adjusted data through the system.

But the final table which you were showing and saying how meaningful it
is when you have some suppressed data on there, is \-- yes, there\'s
always people trying to crack in on suppressed data and on databases,
but again, you\'re trying to get the actual survey data out there. You
really just want to report what is. And to that extent, that\'s sort of
the mission or job of the program office.

How the customer uses that data or what guesses they\'re making in an
extrapolation, you really can\'t control for.

CHAIR WATKINS: Before I close the session, could I just ask you one
quick question? Do you ever come back to the people who you\'ve
identified that their data may be displayed, and ask them do they care?

MR. BOURNAZIAN: Basically going back and asking for their consent to
publish \-- you know, in revealing that \-- no, we haven\'t.

CHAIR WATKINS: Yes, John?

MR. WOOD: In the pre-reserves dataset we do, because there are many
states where, if you suppressed, you\'ve suppressed the state. And we
got on the phone with their pre-reserve manager and asked them if it is
okay if we actually publish a cell that, you know, anyone that knows the
industry can figure out what their position was in that state.

And in fact, we\'ve never been refused. Now, we\'ve suppressed stuff
anyway, but in those times we have asked the industry has been
cooperative.

CHAIR WATKINS: Thank you. Well, we\'ve run out of time on this session.
We\'ll reconvene at 1:30.

(Whereupon, a brief luncheon recess was taken at 12:39 p.m.)

A-F-T-E-R-N-O-O-N S-E-S-S-I-O-N

1:39 p.m.

CHAIR WATKINS: We are now ready to start this afternoon\'s session, and
we heard this morning about NEPOOL; we\'re now going to look at the
prototype model of that as given by Doug Hale and Thanh Luong.

MR. HALE: Good afternoon. It\'s a pleasure to be here. Our chairman has
made it clear that he wants plenty of time for discussion so we will
proceed very quickly, I hope. The first thing I\'d like to say before we
begin my talk and the following one, is the work you\'re going to be
hearing in the next two talks represents joint work by the Office of
Integrated Analysis and Forecasting and the Statistical Methods Group.

What we\'re trying to do is to get a better handle on how we should
approach electricity restructuring, and we\'re doing research to find
where we can make investments in the future to get our models in the
best possible shape for dealing with forecasting and analysis in this
very changed world.

Currently there are two projects. One is the project to take a look at
transmission within the electrical network. The second is to investigate
pricing: non-competitive pricing, perhaps opportunistic pricing. But
there we\'re looking at the whole range, from local monopoly type
practices to problems with the independent system operator, how to get
appropriate pricing of electricity.

The talk you\'re going to hear now is called the Exploration of Network
Modeling. This is a project that Thanh Luong and I are presenting.
Helping me is Joseph Mulholland, an engineer from the Office of Energy
Efficiency. Others who worked on this project are James Hewlitt of EIA
and Thomas Overbye at the University of Illinois.

The purpose of this work is to investigate the merit of explicitly
including the electricity network, the grid, into EIA\'s analysis and
forecasting models. And if you\'ve never seen one before, on the screen
there\'s a grid. It\'s a network model. It\'s a model of NEPOOL, the New
England Power Pool.

An electrical network consists of all the generators, all the demand
centers, and the transmission lines connecting them. Currently in our
models we include all the generators and all the demand centers, but we
abstract from the issues of the transmission lines. We do not therefore,
incorporate so-called network effects \-- which are known as congestion
\-- and line losses.

And a fair enough question is look, EIA has been doing analysis in this
area for 20 years very successfully and we\'ve done it without networks.
Why are we worrying about networks now? The answer to that is, the world
is changing.

In the past, we could count on utilities to deliver a bundled product,
electricity at the outlet, they dispatched generators to meet demand,
they built transmission lines to limit congestion and losses. And they
took care of holding the entire system together. At the same time, the
public utility commissions required average cost pricing which tended to
subsidize the high cost users in the service area.

Now we\'re starting to rely more and more on competitors to provide
pieces of the electricity service that we all enjoy. But instead of
providing a final bundled product \-- some people providing generation,
some marketing, some ancillary services, some risk management. And these
firms are responsible only to their stockholders for their own profits
and losses and are unsurprisingly, unwilling to subsidize anyone at any
time.

The advantage of that approach may be that we get more efficiency, we
get innovation, and we get risk-taking, or risk-bearing on the part of
competitors. But we may, at the same time, have a problem in assuring
universal, reliable service that we enjoyed from the utility era.

FERC is trying to get a little of the best of both worlds, and their
policy is to open access to the network so you get competition and
generation and assure appropriate pricing of transmission so that the
prices consumers see reflect actual cost.

Now, the problem with putting a network into the picture compared to a
normal market for say, automobiles or coffee beans or whatever, is that
these network effects add real complexity to markets. The network
effects of congestion and losses can fault very large in the total cost
of producing and generating electricity. Sometimes under certain
circumstances they can be larger than generation costs.

Second, the network effects determine how large of a competitive area
you\'re dealing with. If you\'re in an area that is well served by
transmission lines that have few losses and few opportunities for
congestion, then a large number of generators can compete to provide you
electricity.

If you\'re in an area that\'s more isolated or harder to get to or more
expensive, then you may be subject to some predatory pricing on the part
of the local suppliers.

The other thing network effects can do is they can induce unique, local
impacts so that the effects of closing a plant or demand growth in an
area, when averaged out over an entire service area, may look pretty
small. And that\'s what you have in the utility-dominated world.

In a competitive world where everybody is paying their own freight,
where there\'s no cross-subsidization, you could find that local impacts
are much different than the impacts across an area and they don\'t
average out.

I think the next thing about network effects \-- and we\'re very
fortunate here to have him \-- is network effects must be included in
the electricity price if you\'re to get maximum efficiency out of the
system. In fact, in a perfectly competitive world, prices would include
the effects of congestion and incremental line losses.

And in that regard we\'re very fortunate to have Dick Tabors here with
us today. The book that he and Fred Schweppe and their colleagues wrote,
\"Spot Pricing of Electricity\", is generally considered the Bible
amongst economists, for electricity power restructuring and has provided
the intellectual framework for much of what has gone on in the last 14
years. So I\'m very happy that he\'s here to hear this.

The other place network effects are important is in the transmission.
It\'s a difference in the local effects that lead to difference in the
transmission prices. From our point of view, from the point of view of
EIA modeling, the smaller our network effects the better our models are
as they now stand in terms of representing reality.

The smaller the differences are between the local impacts of particular
areas then the more robust is FERC\'s policy or strategy of saying
transportation rates according to formula, and just letting the market
work.

So this work has taken on to get at some primary research questions.
First, can we detect network effects in the first place? Is there a
routine way of figuring out whether an area has serious network effects
or not? If it has network effects, are they large?

And the second \-- or third sort of question is, would a network
modeling capability allow us to conduct more informative analysis of
Federal issues? To do that we built model of NEPOOL, a very crude model,
and we subjected it to three policy cases, or alternatives, which I\'ll
describe in a minute.

The first is a base case which reflects summer of 1995 in New England;
the second case involves shutting down two of the nuclear units \-- the
Yankee units \-- and seeing if that has an effect upon the average
prices and their distribution. These plants in fact, are now shut down.
And then finally, shutting down the Millstone plants in addition to the
Yankees and seeing if that has an effect on the system that shows up
through the network. The Millstone is now closed.

Now, to show you how we went about this, Thanh Luong now is going to
demonstrate how simplified versions of these models operate.

MR. LUONG: Hi. My name is Thanh Luong and I\'m going to present the
power system network of the New England Power Pool, but before I do that
I\'m going to zoom in and let you know the first, the basic idea about
the power commission.

Right now I\'d like to zoom in to the area in Maine right here, and you
will see the power system coming up. In the basic of the power system
commission you will see that you have a generation, load, and
transmission.

And you see that this is a generation just like you have the
presentation this morning, it\'s at a low voltage level; it\'s
collective transformer to step up to the high voltage level to this bus,
and then this is the transmission line to connect from one bus to
another bus.

And this one, the circle here and with the blue thing here just tell you
the percentage loading of the line. And I will show you the data for
that. And on the data that we received here is from Form 715, and this
has about a 2000 bus system on the New England power pool in 1995 \--
peak load demand.

And we reduce it down to about 148 bus system. And I will show you the
summary of the network. But if you see that, right here we have on the
resistance, reactance, and the charging, and on the limit on the line
come from the FERC data. And this is a solution of it and this telling
you the percentage loading of the line on each end of the bus. So
that\'s the basic idea.

And the next thing is the generation. And the generation will receive
the data from FERC regarding the bus name, the connectivity \-- you
know, is off, is open, is on the system right now or is open \-- and the
maximum and the minimum megawatt output, and the maximum and the minimum
megawatt \-- that the part of the reactive of the transmission of the
generation.

And we also have the data \-- we construct the data for the cost curve
average generation. And this is just the load unit telling you the load
at that particular time.

So that\'s the basic idea about the power transmission. I\'m going to
zoom out a little bit so you can see a bigger picture of that. And right
now I\'m going to run \-- simulate the situation and let it run and so
you can see the flow of the power system.

And as you see that this is right now, if you really look at it, the
system, even though in 1995 peak load New England Power Pool
transmission can handle without any problem because of the \-- look at
the loading on the line. That\'s a normal situation right now.

MR. HALE: And this is summer, 1995?

MR. LUONG: Yes, and it\'s just a peak load for summer of 1995. And we
can run a scenario \-- let me show you the \--

MR. HALE: Thanh, you probably want to get right to the scenario.

MR. LUONG: Okay, sure. Yes. So I\'m going to run the scenario. We
disconnect the Maine Yankee generation and you look at the megawatt. Now
this unit is offline. And we also tried to disconnect another unit \--
is the Connecticut Yankee. This bus number, that\'s Connecticut Yankee.
And this \-- of 50 or 60 megawatt so we opened just one.

And we\'re going to replace the generation loss \-- about 1500 megawatt
that we lost from importing the power from Canada.

MR. HALE: When we say Canada, that\'s in quotes, what we\'ve done is
selected a position, Sandy Point, and essentially put a generator there
in terms of the model, to represent the power that could come from
Canada. A bus by the way, is where a lot of lines come together.
Disconnecting from a bus is like throwing the switch, opening the
circuit breaker. So that\'s what\'s going on there.

MR. LUONG: Yes, because at Sandy Point there\'s two DC lines connected
straight from Canada down to this \-- to the Boston area. And if you
look at the loading on this line right here, between Sandy Point and
Tewksbury, right now you can see that the percent that\'s loading is
only 50 percent. But now I\'m running the model and you will see that
the percentage will change on the loading of that line. And now it\'s
about 77 percent loading on that line right now. And in order to
simulate a situation that \--

MR. HALE: Thanh, before you do that, could you back out a little bit so
we can see how more heavily these lines are loading now? Some of them
are not loaded at all but a large number of \-- that location, some of
the others, the circuits are about full which means that they\'re more
\-- reasonably heavily loaded. But as you get farther north they\'re
not.

DR. CHATTERJEE: In real life, is this done on automatic feedback
control, or is it a human person doing this?

MR. HALE: Thanh has actually done it. Let him answer.

MR. LUONG: Yes, normally most of the utilities have an energy control
center and they have a real-time network analysis and I used to work in
this area. And so it retrieved the data that came from the ATU \-- every
few minutes, and they viewed the topology, when the breaker open or
close, and then they run the stage meter and then they run the
contingency analysis. So it\'s doing it in real-time every five minutes
to tell that the system is different.

And then this also interfaces with AGC, automatic generation control and
unit commitment, so it is just a whole energy control center. And this
part is the almost \-- in a utility they have this one as the \-- just
like in the study, in the blending, so they have a study power flow and
then the real-time. So the engineer can sit down and do the blending,
you know, disconnect certain buses, disconnect certain generation, and
see how will happen to that system.

Right now I\'m going to do something drastically by just disconnecting
the three nuclear power plant units in Millstone, and without importing
anything from outside, and see how this will work with the system.

MR. HALE: Now we\'ve closed out Millstone and both of the Yankee plants.
We\'ve got the demand that was actually experienced in the summer of
\'95 in New England, and the power system has got to try to meet the
demand. So now Thanh \-- it didn\'t make it. It\'s called a blackout.

So, Thanh has now gone through the physical aspects of the system. I
just want to make a couple of quick points on economics so we can finish
in a second to keep the chairman happier.

CHAIR WATKINS: And the committee.

MR. HALE: And the committee happier; I\'m sorry. The first chart, it
shows you the model performs about the way you would expect. You start
out with a base case; there\'s plenty of generation. Then you shut down
the Yankees; the price goes up a little bit. You shut down Millstone and
in the first instance it blacks out, but if you add just barely enough
capacity \-- in line capacity, not generation capacity so much \-- to
get the system working, price is up very, very high.

The interesting piece is the standard deviation. In the first two cases
there\'s almost no variation between locational prices; which means the
transportation costs are basically zero. It also means that a model like
NEMS is very successful in predicting what\'s going on in that area
because it doesn\'t have to worry about congestion particularly, and the
prices aren\'t much different than they would be otherwise.

In the case of Millstone however, when they shut them down, you get a
lot of variation and that shows up in the prices. So if you\'ll just put
that up. That\'s what the prices look like in these calculations when
the model just barely gets operating again. We\'ve got a very wide range
of prices and those prices depend upon where you are.

If you\'ll put up the next slide. Again in the base case, prices at all
locations \-- I picked Boston, Hartford, New Haven, and Maine \-- are
essentially all the same. When you shut down the Yankees the prices go
up a little but they\'re almost all the same, still.

When you in addition, shut down Millstone and get just enough power to
get the system working again, prices at Boston and Maine are in one
group \-- they\'re high but they\'re similar \-- but the prices into
Hartford and New Haven are much higher. And that\'s because there\'s
kind of a partitioning of the electrical system. That reflects the way
the system was designed in the first place.

If you look at the transmission prices \-- cost of transmission \-- what
you see is in the base case, there\'s practically no transmission
charges at all. The system is well designed by the utilities. In the
Yankee case they\'re still practically zero. It\'s only when the
configuration of the system differs greatly from what the engineers had
planned, that the prices really matter in this case.

The question is, will that continue under competition? In terms of the
results we\'ve had, we now believe we can estimate network effects for
just about any area using FERC data and commercial software. It\'s clear
that congestion and losses in NEPOOL as configured in \'95 are small,
and NEMS will do a great job as long as those areas are like that. We
don\'t have to worry about it. Wow, is that a relief.

However, you can get situations where if the configuration changes, even
in something like NEPOOL you could get significant transmission and
network-related price effects. The questions for the committee have to
do with our modeling strategy and questions about whether the high power
grid is where we should put our focus.

One for the statisticians is, in building this prototype it turned out
to be a bit of a nightmare for getting reasonable, marginal cost
estimates, and I would appreciate any suggestions you all have on that
issue. So that\'s my presentation.

CHAIR WATKINS: Thank you, Doug. Phil, are you going to go first?

DR. HANSER: Despite the fact that I have a typo \-- Freudian because I
live in Boston \-- we are talking about the same paper.

CHAIR WATKINS: That\'s very selfish.

DR. HANSER: Well, you know, I\'m completely an egoist. Let\'s see,
let\'s go to the first slide. I think the conclusions of the paper are
reasonable in the following sense. It doesn\'t surprise me that when you
look at the peak periods that you don\'t necessarily have large
congestion problems or losses. Frankly, the systems are largely built to
deal with peak issues.

We\'ve looked at other pools \-- I looked at PJM \-- and it turned out,
where congestion arose was in fact, during non-peak periods in the
colder seasons when units were out for scheduled maintenance and the
flows are not going back and forth through the system as they would
during peak periods when all the units are running.

Then you found yourself in a situation where there were differing prices
of power depending on which side of the constraint you are. That\'s
known as a multilambda situation. And so the lack of congestion during
peak doesn\'t surprise me at all; that seems perfectly reasonable.

I think I would argue that the tighter and the older pools \-- PJM,
NEPOOL, NYPOOL, particularly \-- probably have well-functioning energy
markets; that they\'re good wholesale markets, they\'ve been in
existence for a long time. My guess also is that on the transmission
side, even if you had perfectly infinite transmission, that you
wouldn\'t see large differences in the overall cost for the pools.

That is to say, if I went to a system in which we were perfectly
unconstrained, the chances are that average costs probably wouldn\'t
fall dramatically in those pools. I won\'t say anything about other
areas like the Midwest or someplace else. I don\'t know what \--
probably wouldn\'t surprise me if it\'s fairly similar.

If that\'s true then the competition benefits \-- which I\'ve heard
spoken about \-- in terms of the potential for collecting economic
rents, it seems to me are pretty minimal. I\'ve heard \-- pardon me for,
you know, for saying this \-- but I\'ve heard at least one EPA
administrator talk about collecting \-- wanting to make sure that the
environmental protection was somehow pulled out of the economic rents
which were going to be competition benefits and getting a piece of that
pie.

You know, as the graph indicated earlier about the projects about AEO98,
you know, prices look like they\'re going down. Well, if that\'s the
case then there aren\'t many economic rents to be collected in the
wholesale energy markets from anybody. In fact, you\'d expect with
competitive markets, rents ought to be declining, right? That\'s why you
have competition.

I\'m a little bit concerned about the behavior of prices in the upper
tail. You know, we looked at PJM this past summer and you could see
prices for power in PJM spiked up to \$375 a megawatt hour at one point.
And that\'s an energy-only price, not including capacity.

One of the questions that comes to mind for me is, what about the
political acceptability of having pool prices that go to \$375 a
megawatt hour? In fact, in the FERC filing by PJM they put a ceiling of
\$1000 a megawatt hour on the maximum price that could be bid into the
pool for power.

So I\'m sort of wondering about, what will that mean? If FERC sees
\$1000 a megawatt hour will suddenly somebody scream, or will somebody
say, well that\'s the way competitive markets operate? You get what
people want. So that\'s something to think about and I think that\'s
important relative to some of the modeling issues.

One of the concerns I have of course, is that as a Federal agency, my
bet is that EIA will be called on to do things that it had no thought of
doing when it was building these models. I don\'t know why I would think
that.

So to me the bottom line is, I think the network effects are important
but I think that the market structure will end up being much more
important. In particular, what I tried to think about are the things
that I believe would cause a relatively stable situation occurring where
prices exceeded marginal cost by some substantial amount, and here are
things that would cause that and that need to be thought about in terms
of doing this modeling on a regional basis.

Top of my list was just the degree of price transparency. You know, a
lack of transparency impedes competition. If you can\'t know what the
prices are in the market it\'s hard to know what you should be bidding
against or bidding for. That\'s pretty straightforward.

Is there going to be a single price market or is there going to be a
split capacity and energy market? If you\'re concerned about for
example, congestion on the transmission system and providing the
appropriate price signal for relieving that congestion, then a single
price system provides a stronger price mechanism for implying that then
a split capacity and energy price.

Because in fact, in a split capacity and energy price, the energy price
would be \-- split capacity energy price markets as compared to a single
price market, you would of course, have it reduced by the size of the
capacity pattern.

Is it going to be a same-day market that we\'re about or is it a day
ahead and a same-day market? Is there a market in which you make a
commitment for resources the day before, and is that then followed by a
market which then tries to resolve the system on real-time?

The UK experience for example, has been such that in that 2-stage market
so to speak, by withholding capacity appropriately, you can drive the
price up quite dramatically. And England for an extended period of time,
had prices that were significantly higher than marginal costs, one could
argue, as a result.

Is the ISO a forecaster of loads and so it\'s a single buyer, or is the
ISO an aggregated? Why that\'s important is that in fact, what you can
show is that if the ISO acts as a single buyer versus the ISO acts as an
aggregator, that in fact it will have a reduced capability of dealing
with market power issues.

If it is sort of saying, I\'m guessing what the level of demand is as
opposed to collecting demand schedules, then there\'s no mechanism for
demanders to say, I object to that price, and reducing their level of
demand.

Are we going to look at congestion-based transmission pricing or are we
going to look at postage stamp rates? For some areas, a postage stamp
rate could have significant impacts on stranded assets. In particular,
it could very well be the case in some regions that lots of power could
go floating across borders that people wouldn\'t want them to, or
companies wouldn\'t want.

How about the size of the reliability region? Is it the Easter
Interconnect, is it PJM, is it some combination? We now have a Midwest
ISO. I\'m not even sure what it is \-- although it\'s labeled as an ISO.
But what region are we talking about relative to the way this market is
going to operate?

Is it a single stage bidding system or is it iterative? What we have
been able to show is that the phenomena that Dick was mentioning about
bids that are epsilontically smaller than the next bid, can largely be
removed if you allow some kind of iterative bidding scheme, as has been
proposed in California.

How are you going to deal with market power, both vertical and
horizontal? Horizontal market power in terms of the shared generation
which is owned by any single generator, versus vertical market power \--
a generator having some ability to constrain the transmission system by
their operation.

What about ISO strength? If I\'m a generator I think I want to go build
my next unit of generation in such a way so that I aggravate the
constraints on the system because that in fact, drives up the price that
I\'m liable to receive if I have any congestion pricing.

Is the ISO going to be strong enough to prevent that kind of situation
from arising, or is the ISO not going to have the capability of sort of
saying where the siting of generation\'s going to be?

What about governance? I mean, we create this ISO, there\'s a Board of
Supervisors or Governors that produces rules for it. Are those rules
going to be changeable over time?

We sort of kind of glossed over this issue about the transmission
companies and the ISO, but the transmission companies are the owners of
the transmission system and the ISO is the operator. It\'s unclear to me
that they necessarily are going to have a common set of interests. In
fact, quite to the contrary; they have different sets of incentives.

Finally, nobody\'s talked about distribution company regulation. In
England what\'s happened is, the distribution companies said, we entered
the generation market and are now building small generators of 50
megawatts \-- 49.9 megawatts, I\'m not sure what it is \-- or less. Does
that skew the generation market and the final price that will clear in
the end?

Because now you\'ve got regulated entities who can cross-subsidize their
generation costs with other kinds of revenues and therefore skew the way
that the price clears the market.

Here\'s the bad news. I don\'t think you\'re going to avoid doing
network modeling. I think partly because I think somebody\'s going to
call upon you to say sort of, what\'s going on here and tell me what\'s
happening with constraints. I sort of think that\'s unavoidable for the
EIA at some point.

I don\'t see FERC doing it and I don\'t see \-- the Department of
Justice doesn\'t have anybody that seems to understand transmission, so
you know, my guess is it\'s going to get dumped in EIA\'s lap. Partly
because, you can\'t do the final electric market unless you define the
transmission system that it operates in.

The DOJ guidelines suggest that there are classes of products on this
electric market and you\'re going to have to look at those, and that
invariably means modeling the network system.

As far as what level to do it at, FERC has already sort of said what
level it\'s going to do transmission pricing. It was a very technical
definition in terms of transmissions being loop versus radial, and that
unfortunately, is going to be the level. I think you\'re going to end up
having to do the modeling act because in fact, as long as you\'re a
Federal agency and you\'re going to be concerned with Federal issues,
you may as well follow what FERC as a Federal agency is pursuing.

And I don\'t know any way to resolve your data problems, frankly. There
are some engineering things you can throw in. There is \-- for example,
PJM has a committee on marginal costs and if everybody was required to
submit their marginal costs for review I wouldn\'t believe a number in
there. I have to be careful about that. Some of those estimates are less
than credible, let\'s put it that way, and I don\'t know how to resolve
it.

You can do this \-- you know, there are tricks to play with engineering
estimates of what generators look like \-- anchoring them at the average
costs that come from FERC \-- you can try that. You can try some other
games like that to get at marginal costs, but it\'s really a pain any
way you do it.

Market behavior models. I think the transition to equilibrium dynamics
is crucially important, as Calvin\'s already pointed out. I don\'t know
whether we\'re going to look at a real estate market kind of phenomena
where it goes through booms and busts in terms of too much capacity
comes in followed by too little, etc., and therefore you get large
swings in the prices in the markets.

It\'s not clear whether you get corner or some other kinds of solutions
in here in terms of the final equilibrium; I think that\'s important. I
think strategic bidding behavior is important, as Dick\'s already
pointed out.

And I also think that demand specification is going to turn out to be
very important because the only reasonable antidote when you\'re talking
about markets in which there aren\'t necessarily large numbers of
players, there\'s going to be some kind of price responsiveness and
somehow moving around demand.

Now, there are lots of different ways to model that. You can do options
markets, puts and calls on demand, you can go to some form of
modification of demand curve. I\'m not quite sure what\'s the best way
to do it in the end.

I do know that I don\'t like markets in which it\'s assumed that no
matter what the level of demand is there\'s always enough capacity
around to meet it, period. I don\'t think that\'s a realistic market and
I know it\'s not one that has much economics about it.

So that\'s sort of the good and bad news. I apologize. I thought it was
a good paper. I pity anybody that has to work with network models. The
last time I worked with a network model at any reasonable level of
detail there was a 6-hour turnaround time between runs for a single
year. I\'ll tell you about that model at some point.

MR. HALE: (inaudible)

DR. HANSER: Yes, that\'s exactly right. But I think it\'s a necessary
exercise, unfortunately. I don\'t know that EIA is going to avoid it.
And in the meantime, it may be a case of having to deal with some very
poor data for quite a long time, unless I\'m afraid, you can get some
kind of Federal legislation in terms of FERC\'s empowerment that will
basically free that data up.

But until such time I\'m afraid, you\'re going to be dealing with some
fairly approximate numbers and I don\'t think there\'s much way to avoid
it. I thank you.

CHAIR WATKINS: Thank you, Phil. Dan, you\'re next. Do you want the light
on?

VICE CHAIR RELLES: No, that\'s okay. Presumably, that\'s what I\'m going
to share. I agree with Phil on many things he said, especially that the
network models are unfortunately, unavoidable. And I think it\'s really
important that you now have a model like that in place that you can
begin to do what-if kinds of computations. And it\'s in that spirit that
I offer my comments. I\'ll try to talk about what you can do in three
different areas: data \-- and insights you might try to get from this.

I must say, I read the paper and essentially found inconsistency with a
null hypothesis of no effect. I\'m delighted to find that you actually
have encountered the cases where things really matter.

On data, I guess I\'m awfully pessimistic about the quality of the data
you can get to run this thing, but I do have a suggestion. Certainly,
you get the minimum and the maximum; that\'s critical. And as far as the
marginal cost curves I gather you have a way of pulling them off of FERC
annual data.

I know a little bit about how they try to estimate cost curves in the
utilities companies. Engineers carefully control how much fuel is
consumed and they try to carefully measure what comes out. And Phil
expressed a lot of pessimism about the quality of those things, but I
will say this; that in the competitive environment, they care a lot more
about those curves than they ever cared before.

And I think you can find in many places, good cost curve data \-- albeit
a cubic polynomial which I\'m not, you know \-- rather difficult to work
with. Nevertheless, I would suggest one thing you do is take your
approximately cost curves that you\'ve gotten from FERC and see if you
can validate them on a sample of cases.

I would guess that you\'ll see some consistency within types of
generators, with common engineering specifications, and that you might
be able to develop a taxonomy where, within certain classes of
generators you can indeed perhaps, fudge the FERC numbers, or at least
satisfy yourself that you might be in the ballpark.

I had no way of knowing based on reading the paper, whether the cost
curves you had were at all reasonable, and I kind of interpreted that
initially as the reason why you didn\'t really see a network effect.

One other aspect of cost that I think you really need to consider which
you haven\'t, is startup and shutdown costs. I know your computation is
done at peak and therefore it\'s really a single hour and it\'s actually
assuming that all generators are turned on and therefore you have a
fairly large spinning reserve.

The cost of turning these things on, especially peaking generators, is
peaking high. And I think that could change your marginal costs curves
by a factor of 50 percent in some cases if you recognize the fact that
these things are on for only a couple of hours.

In fact, that also suggests that it\'s \-- I think it\'s kind of
important to use this model, not only on a single time point, but to try
to look over profiles of the day, because turning generators on and off
is a large component of the cost of any utility company.

So I guess on data I would simply put in a plea to try to deal with
startup and shutdown and to try to validate your marginal cost curves.

On metrics, I recognize that you\'re trying to compute costs and the
paper was concerned only with costs, but I think it\'s important to
start thinking about reliability too. And there was an absence of
discussion on that. Clearly, when you turn off Millstone your spinning
reserve is less so your reliability is less too.

But costs and reliability need to trade off against each other and I
just don\'t see how you can be satisfied with reporting costs alone
without saying something about reliability. A metric on costs that I
would have liked to have seen \-- again, there are masses of information
being summarized in this model and I think it would be useful to try to
get it down to a couple of numbers here or there.

The cost measure of the network that I\'d be interested in seeing is
sort of the total cost of running the network with its constraints
divided by the total cost of running the network without any constraint.
This sounds like perhaps in the base case you heard about, a 97 level
percent level of efficiency but I\'m not really sure. But I think that
would be a useful number to have.

And of course, so would a reliability number like we got a .001 chance
of this thing going down. Outlying nodes of the network would presumably
show up as high cost extenders of the network, but if their cost is sort
of coming at an increase in reliability that\'s something we should
know. We don\'t want to be condemning somebody just because he\'s
expensive and especially if he\'s contributing to the safety factor.

As far as how you use this to measure reliability, if the model indeed,
takes 20 minutes to run it\'s going to be pretty hard, but I would think
there are things you could do to try to get that down.

One obvious thing to try perhaps, is to just separate out the networks
\-- try to take the network and separate it out into a bunch of smaller,
independent networks, computer the optimal dispatching within each
network, and then do some sort of fudging to figure out what the
transmission between the nodes of the network are.

I don\'t know exactly what you would do, to tell you the truth, but I do
think, you know, with some attempt at simplification, you might be able
to get at that.

The reliability issue I think also, you could try to deal with by doing
some simulation. If you can get the costs down then varying the load and
also sort of, perhaps using some Poisson assumptions about failures of
generators and maybe even assuming that each generator has \-- that only
one generator can fail and it\'s going to fail with some probably, and
what effect will that have on reliability \-- I think those would be
worthwhile computations to perform, too.

And finally, as far as insights, I think it would be worthwhile to try
to design sort of a huge computational experiment to understand what
this network model is telling you. What is it telling you as far as
summer months, winter months? You know, you\'d have to face issues there
like how long a period are you running for? Is it a week, is it a day?

Anyway, to try to think of this as a complicated black box function that
you\'re trying to understand the shape of, I think can be dealt with to
some degree by parameterizing facts about the network, acquiring sample
points to describe the outcomes here and then trying to summarize those
in a graphic of some sort.

That\'s pretty much all I want to say.

CHAIR WATKINS: Comments or questions from the committee? Dick.

DR. TABORS: I sort of had four quick ones. I\'ll start with the least
obnoxious one first, how\'s that, and then we\'ll go from there. It
isn\'t clear to me that nodal pricing is really the paradigm you ought
to be using for this. And I\'m going to go for the moment as that one of
the guys who at least was around when it was developed \-- although the
genius is Schweppe; clearly not me.

I think nodal pricing is at an order of and a degree of sophistication
or accuracy and I can\'t believe EIA can pull off in a way that fulfills
their function. So I would challenge the fact that nodal is the right
way to go.

Now let me suggest an alternative, which is that if you go to something
that looks more zonal you end up with what would be a simplified network
that may have information in it that the nodal does \-- that may have
more generalizable and usable information then you could ever get out of
a nodal model.

Now having said that, I\'ve given my personal prejudice and clearly
I\'ve gone on record at the FERC by saying that, so take it for what\'s
it\'s worth that that\'s where I come from.

I think the next point is that this has somehow got to be consistent
with what Peter Whitman was talking about this morning, and at the
moment, the ordering of these things seems not to be. I mean, you\'re
looking at New England with a microscope and he\'s looking at the United
States from 30,000 feet, and I think those two have to somehow come
together.

And then the final point is, what does this say? And this is where I\'m
going to be really obnoxious for 30 seconds and that is, that I can\'t
tell what the question is that you\'re trying to answer. And if I were
truly being cynical I would say you\'re playing with a really nifty tool
but you really aren\'t aiming at answering a question that I think EIA
has an ability to get an answer to.

I think the data question \-- and I grant you I didn\'t help you on that
so I\'m going to apologize now \-- I\'m going to say, there\'s a history
on the data question here. But I think that\'s just a classic example of
the problem; is that my inability or unwillingness to give you a
proprietary data set for all of the units in New England should be an
example of the fact that I\'m easy to get along with relative to most of
my utility colleagues who just say, you know, you must be kidding; I\'m
not going to do it.

So I think there is a real issue as to what\'s the question that you\'re
trying to answer with the transmission part of this, and then is this an
important way of getting that answer, or is there a simpler method that
gets you actually an answer to a question that you wanted to get an
answer to.

CHAIR WATKINS: Doug, I want to give you a chance to reply. I did have
one or two questions myself but I\'ll hold those back and let you
answer.

MR. HALE: I will get them though?

CHAIR WATKINS: Yes.

MR. HALE: Okay, good. I\'d like to first thank the committee for the
comments and their attention. I want to start with Phil\'s basic \-- one
of his most basic \-- his most basic point in my opinion, and that is,
EIA is going to be called upon to do things we have no idea that we\'re
going to be asked to do.

The reason we\'re doing this particular piece of research and the piece
you\'re going to see next, is to try to get ourselves up to speed and
covered for the questions we know that are coming. How we\'ve gone about
it is to look at the literature and see where the major debates seem to
be and then try to get, you know, knowledgeable in those areas.

So that is to a large extent, what\'s motivating all of this. Let me
first reassure Phil that many of his points about issues that are quite
important in their own right but don\'t involve the network explicitly,
we are looking at them.

We\'re looking at them in the context of experimental methods that
you\'re going to be hearing a little bit about this afternoon and then
more the next day. And we\'re very concerned with the rules for
governance of the ISO; we\'re very concerned about bidding strategies.
They could be used even in an unconstrained network which would assure
inefficient results.

We\'re concerned about gaming the system as the English experienced.
We\'re looking at all those in a preliminary, exploratory way to try to
get ourselves in a position so that when the impossible questions come
up at least we can reformulate them in a reasonable way and answer a
reasonable version of them.

I really appreciate Dan\'s comments as well, about the data. You asked
if the cost curves were reasonable. The answer to that is, they\'re well
documented, and I can tell you how we got them. The truth is that we
decided not to invest a lot in effort in cost curves until we could get
a better handle on the entire network to see how things fit in.

And one of the things we had to get a handle on was the absolute
complexity of this monster. We have got a very simple system and one of
the most discouraging things I heard today was Phil\'s charge that we
really get at network model. And I think doing a full-blown version is,
I believe as Dick said, is far beyond what we need to do or want to do.

However, I do think there are parts of the network and part of the
network features that we just have to have a handle on for very special
cases, especially \-- we are no longer going to have the mother utility
making sure everything\'s going to work out in the long run in terms of
the transmission system, and if we\'re leaving it up to a free market
there are very funny conflicts of interest between what appear to be the
public good and the private gain, and they don\'t seem to come together
as they do in some other markets.

And we need to be aware of that and to modify models and our forecasts
accordingly. And I think your notion of going to a zonal approach may be
a very good one.

The notion about thinking about reliability that Dan mentioned \-- we
set it aside for right now. We, I believe along with you, can do
simulation work and get a notion of what the reliability consequences
are. In a system like New England it looks awfully reliable.

Once you start getting a price sensitivity of demand, people rolling
back demand in response to higher prices, it may get more reliable. I
mean, it may be more of a self-regulating system. That would be worth
doing.

As far as the insights, I think we have learned a lot from this modeling
exercise. The electricity model always surprises us because we see
prices we don\'t expect to see, including negative prices from time to
time. We see that we apply pressure in one location and it pops up
somewhere else in completely anti-intuitive ways unless you\'re an
engineer, I guess. But there have been those sorts of insights that have
come out of it.

Dick\'s suggestion to start getting consistent with Peter Whitman is
well-taken. I\'m a microeconomist so I do micro. We clearly have to do
macro to make this useful, you know, for the agency and make it more
explainable to people.

I can assure you Pete and I are spending lots of time trying to put in
the capacity considerations and longer term price considerations. We are
very, very concerned about the incentives for transmission and the sort
of incentive, perhaps not helpful incentives you might get out of some
of the various schemes that have been proposed for that. So we\'re
working on that, but slowly. This turns out to be hard stuff as we\'ve
discovered.

And I\'d like to get back to the issue of what question to answer. EIA
has, as I said, a real challenge to answer the questions of
policymakers, on the one hand. On the other hand we have very limited,
analytical resources and research sources.

The area where there\'s a lot of controversy, where EIA\'s models
currently don\'t capture very well, is transmission. And our purpose
here was to find out what\'s there. Is there a big, you know, ticking
bomb? And I think the answer, as Pete and you both said, for
well-established areas like New England, if there\'s a problem it\'s not
immediate, it\'s not current, it\'s going to be down the road.

And for those areas, if we can perhaps set them aside and then look at
some of the other areas like the Midwest where there are more \-- or
maybe not, I don\'t know. I mean, just find out.

So that\'s my response to it. All these comments are extraordinarily
helpful. I see a research plan that\'s going to extend well beyond my
retirement. Thank you very much.

CHAIR WATKINS: Thank you, Doug. I also was concerned about Dick\'s
question because I thought I understood what this paper was addressing
and then maybe I don\'t know. But I thought it was to \-- so much of the
focus thus far and analysis being on the generation sector and what\'s
going to happen there, to the neglect of transmission.

And the way I interpreted some of Doug\'s work and his colleagues was
that yes, indeed, if you knock out some of the generation and you want
to use a market solution with the transmission system, that you can get
some very disparate prices across the system which have a lot of
implications, if you\'re going to use the price system in effect, as a
rationing device.

Now, was I missing something on that?

DR. TABORS: The only point I was making on it I guess, that kind of
picks up on that is that if you are going to use the price system as a
rationing event, you have to make sure that you give the price system a
chance to work rather than sort of turning it on in incredible detail.

Because at that level of detail you\'re either too \-- again, you\'re
either at 30,000 feet in which case you\'re going to get one answer, or
you\'re slightly below the treetops in which case you can\'t see, you
know, what you\'re doing.

And it\'s a very hard tradeoff in the transmission business. A lot of
reasons for that that are critical to the modeling, particularly when
you\'re trying to do the modeling. Real quick. The capabilities listed
for transmission lines and transformers is generally understated by
between 15 and 20 percent. So that\'s one.

Two, even if it\'s not understated you can always run them for an hour
at a way higher rating then they\'re meant at without melting the lines.
So what happens is, when we model them, based on the rules that the
utility engineers give us, we get an incredibly conservative answer. And
then you go talk to the utility operators you say, do you run it this
way? And the answer is no.

So when we get into the modeling problem now, you\'re going to see
numbers that are equivalent to this, but the argument is, are they real?
And the answer generally comes back, for the conditions that are listed,
no.

Phil hit the big problem. The big problem isn\'t the peak day of the
year because we\'re thinking about it on that day. The big problem is,
sometime in the middle of the year when half the damn stuff\'s out on
maintenance and one more unit goes out, or you slow something else down,
or you slow something else down.

So you know, again, it\'s this question of, exactly what are we looking
at by way of data for policy analysis?

CHAIR WATKINS: Right. Okay, we\'ll close this session and start the next
one.

MR. MANICKE: This is sort of Part 2 of a talk that Doug just gave. I\'m
going to talk about a specific methodology that will allow us to
experiment with pricing of electricity in an open market \-- open
transmission market.

And to do that I\'m going to give a little background of the business
inertia that has led to the deregulation fervor, a little bit of how
that might \-- the deregulation of electricity pricing might affect our
modeling needs, exigencies, then describe the methodology itself and
look at some potential hypothesis that we might come up with, and then
give some examples of the use of this methodology.

I\'ll have to apologize right-off. Pete Whitman and I at lunchtime
probably didn\'t let Richard Tabors eat. Pete had his right ear and I
had my left ear. But it was a good case of cognitive overload for me.

These are all facts that everybody\'s familiar with. There\'s many, many
things going on right now that is leading to the states and utilities
deregulating their power markets. The futures market established about a
year-and-a-half ago is now selling about 1300 contracts a day, an
average, only in two nodes.

After about a year-and-a-half the futures on natural gas was selling
quite a bit less. Right now natural gas futures is second only to crude
oil. California on January 1st is going to allow retailing. They\'re
going to have a power exchange. New York is considering an ISO in which
Niagara Mohawk will stop selling generation and just be a buyer in which
generation will be offered to them and they will distribute it in a
least-cost manner throughout the New York system.

Last but not least, there\'s a tremendous growth in wholesale marketing
of electricity. The marketers have grown tremendously and they\'re a
minor part compared to what the utilities are marketing and wheeling.

The marketers have grown so fast that they have actually, inside their
own corporations \-- the big corporations \-- have started hedging
internal from division to division with financial instruments inside
their own divisions.

As we\'ve talked about, all this fervor for wholesaling and movement of
electricity, wheeling, has given the utilities quite a number of
technical headaches and economic headaches. Probably the most important
is appropriate and real-time pricing, and has been discussed several
times, the development of new services that these vertically-integrated
utilities have right now, or exist right now, in that structure.

Standards of quality, reliability, and security is going to be a big
concern if there is competitive bidding in an independent system
operator receiving bid and optimizing over many utilities and
transmission coordination lines among these utilities.

A bit more of the background. Other countries that have gone to open
transmission access \-- Chile, Argentina now, UK, Australia \-- have
progressed similarly. And this is not in chronological order I want to
note.

Predominantly the area of the bilateral contracting started to power
exchanges where people, or companies, marketers, utilities, bought and
sold electricity like a stock exchange. And the development of the
system operators to control the whole gamut of the sales and buying and
distribution of electricity, developed.

In the final analysis, the independent system operator must determine
how the electricity is redistributed, how it is to be bought, what price
it is, in the sense that if this independent system operator were not
operating, the system could become unstable or lines could melt down \--
all kinds of bad things could happen. So the final analysis, the
cornerstone of these processes for deregulation and selling of
electricity in a deregulated market has been the independent system
operator.

As Betsy referred to, in a deregulated market the power producers may
break into generating companies, distribution companies, transmission
companies, or combinations thereof \-- marketers \-- that use all of
these and buy and sell among various things, various commodities in
here.

It\'s possible that transmission could become a commodity. That\'s a big
controversy, needless to say. Obviously, power is a commodity. So in
order to forecast and analyze in these deregulated markets we need to
construct hypotheses about how they might work, how they might behave.

What we plan on doing with this methodology called PowerWeb \-- which
I\'ll get to just in a second \-- is of course, construct different
kinds of electricity potential markets and auction protocols, and
experiment in a lab on what kinds of economic behaviors are arrived at.

And if we do get competitive and non-competitive behavior in some way,
some opportunistic behavior, we may want to incorporate them in the
forecast models.

Just quickly, right now utilities do use forecast models for demand and
the energy necessary to complete that demand. This is a big player, the
variable cost of input \-- energy, natural gas, coal, whatever. This is
a variable player that, both of these are important in the short run,
variable costs, today.

For forecasting, the issues and uses of forecast like our NEMS, have all
of these characteristics that affect one another. In a deregulated
market this may be a very big player, and that\'s the purpose of
experimenting with PowerWeb, is to see how that might change.

A little about PowerWeb. PowerWeb allows us to mimic how these
deregulated power markets could operate. It\'s uses an auction structure
to buy and sell electricity. It is, as Phil talked about, PowerWeb is a
single buyer. There will be many sellers or auctioners or offerors to
this system. And it works just like the real markets are supposed to
work \-- rewarding the most efficient generators and operators with
larger market shares and better rates of return.

We\'re working in conjunction with Cornell in helping experiment with
this. They\'re developing it through the National Science Foundation and
we\'re using it on a server that we have up there. What PowerWeb does
is, it takes these many offers and goes through an optimal power flow.

This is just an extension of economic dispatch. It does a least-cost
algorithm whereby it also satisfies the physical network laws at the
same time as site conditions, across large utility grid structures.

And this has been talked about also. The generation schedule may change
as one goes through this because it might have to be shifted to reduce
power flows across circuits that otherwise would become overloaded.

Again, we\'re just starting with experimenting with this. It\'s been
very interesting. We\'ve found some very interesting results so far, and
we\'re looking for you to help give us hypotheses to understand the
potential of pricing in this kind of structure.

Let me just give you an example of one of the ways that PowerWeb finds
and distributes \-- or finds pricing and distributes generation to
offerors on this auction. There are many option protocols. It has many
of them in it. It has the typical uniform, the discriminative, the
English and the Dutch.

The uniform has been sort of the backbone of most ISOs. What the uniform
price structure does \-- or uniform pricing auction protocol does \-- is
it assumes that, as Phil had stated already, that the ISO is a
forecaster, that in some hour in the future that ISO knows what power it
needs and it\'s going to forecast that and say, I\'m going to now need
300 megawatts for some hour in the future. This quantity right here is
given.

Well, the uniform protocol says, take the first price that is rejected
as the reigning price, and give everybody that price. Now, there are
lots of behaviors that one could use in doing this.

So everybody up \-- for purposes of exposition \-- excuse me, I should
have said this \-- for purposes of exposition this is one generator with
this capacity at this price \-- at this marginal cost, excuse me; the
second generator at this marginal cost; a third generator with this
capacity at this marginal cost.

The first two will sell all of their capacity at this price, at the
reigning price, given by the auction protocol. The third generator will
only sell this much, of course. Now, I\'m just going to give you one
possible scenario of ways of behavior of some of the behavioral things
that can happen in this.

Since these are marginal cost curves, this area right here of course,
represents now \-- price times quantity \-- the profit for generator 1.
This of course, would be for generator 2 and this for generator 3 \--
this part right here. Now, this is the reigning price, given these small
number of offerors.

Well, suppose generator 1 and generator 3 were owned by the same firm
and they had quite a bit of knowledge of the system \-- which we will
look at tomorrow morning \-- and they just said well, nothing wrong with
that. Let\'s just withhold generator 3.

Now looks what happens to the reigning price. It goes up to this and the
profit that the firm makes is this; which as you\'ll notice is
considerably more than they would have made had they not withheld that
in this option structure.

DR. HANSER: But you\'ve got a second price auction, not a uniform price
auction.

MR. MANICKE: Okay, second price auction. Okay.

DR. HANSER: Because, I mean, that\'s important because the majority of
the models that are going out in terms of the pools are \-- I just think
it\'s important. That\'s a second price auction; it\'s a Vickery
auction.

MR. MANICKE: It\'s a Vickery auction in which everybody gets the uniform
price above, and that\'s what we\'re just calling \-- it\'s a
modification of Vickery auction but everybody gets the uniform price.
That\'s what we\'re calling it, yes.

DR. HANSER: I just think it\'s important because it\'s not a uniform \--
it\'s a varied and not a uniform price auction, and isn\'t necessarily
the one that\'s most commonly being talked about relative to the pool
pricing. The pool pricing has tended to be, you\'re one step above where
the typical pool pricing is. That is to say, the marginal unit \--
everybody gets paid the price of the marginal unit. Now it\'s a marginal
unit plus one.

MR. MANICKE: Correct.

DR. HANSER: So I think that\'s important because it makes the margin
unit inframarginal at that point, and that\'s an important distinction
because when you come to the actual clearing of the markets it suggests
that they\'re getting at least a portion above their variable operating
costs being paid into this market. And that\'s not, you know, what\'s
typically done in an energy market. They\'re simply paid their marginal
costs. And that\'s an important distinction to make.

MR. MANICKE: I agree. What \--

CHAIR WATKINS: Could you repeat that again? I didn\'t understand that.

DR. HANSER: In the typical pool, the price that clears up the market is
the price of the last unit \--

DR. TABORS: The last unit loaded.

DR. HANSER: The last unit loaded \--

DR. TABORS: Which in that case is Q3.

DR. HANSER: That would be Q3.

DR. TABORS: Or C \-- I\'m sorry C \-- my eyes aren\'t good enough but I
think it\'s C3. In a normal market, what Bob\'s pointing to is what
would be the market clearing price.

DR. HANSER: Right.

CHAIR WATKINS: All right, but we\'re talking now about the wedge between
the highest cost source and what the market price would have been?

DR. HANSER: Right.

MR. MANICKE: Yes.

CHAIR WATKINS: But the point is that the market price of course, would
be a uniform price?

DR. HANSER: Right. But it makes a very big difference from the following
standpoint. If the marginal unit, the last unit, is being paid an
inframarginal price and he\'s only being paid energy, then he\'s
receiving no payment for this capacity.

And much of the debate in the power pools is precisely around this
question of, what is that ultimate, that last step of the unit being
paid? Because the way the pools are clearing right now and as they\'ve
been proposed, they\'re receiving \-- that unit is receiving no
inframarginal payments to cover the fixed costs of his operating that
unit, of his being available.

And much of the debate around capacity versus single price or is this
two price auctions in these markets, is precisely around that issue and
the payment of that unit.

CHAIR WATKINS: So that\'s like a monopsony buyer? You just \--

DR. HANSER: No, no.

DR. TABORS: No, the picture Bob has up there is the hypothesis that I
stated this morning on Peter\'s model that said that as a bidder, I
would try and in fact, extract the difference between \-- now I\'m
having a real trouble reading \-- but I would try to extract as a fact
as a bidder, the price that he shows there as PR. I\'d be epsilon short
of PR if I were a bidder and it were a first price auction.

If it\'s a second price auction then I don\'t have to behave that way
and I get the benefits of it anyway. That\'s really, truly a Vickery
auction under those circumstances.

Very important point about Vickeries getting a Nobel prize among other
things, was the fact that he said, let\'s give the marginal value or
that marginal surplus, back to the highest bidder. And that\'s what
Vickery\'s auctioning structure does. And that\'s what Bob has and is
showing.

MR. MANICKE: This is not a Vickery, but \--

CHAIR WATKINS: That\'s exactly the point I was making; that if you
don\'t do that you\'re like a \-- buyer.

MR. MANICKE: At that level \-- at the top level.

CHAIR WATKINS: Yes.

DR. HANSER: Yes, exactly. I misunderstood, I\'m sorry.

MR. MANICKE: That\'s right. That\'s all right. That\'s very good.
Tomorrow, we\'re going to experiment with this system \-- not be outdone
by the color slides \-- in which this is essentially three utilities, 30
node total \-- one utility of course, two utilities, third utility. Each
one of these are generators. Each one of you will be a generator that
wants to participate.

You will be given a marginal cost curve. It will be user friendly
because we\'ll have people there who will say, reload, do this and that.
It takes a bit of finesse to figure it out but it won\'t be that
difficult because we\'ll have somebody saying that and helping and all
that sort of stuff.

And I will be the ISO and I will calculate how much you will be \-- how
much of your offer in terms of megawatts, will be given at what price.
And I will be doing the optimal power flow calculations. Each one of you
will be given a different kind of generator.

Somebody may be given a 45 megawatt, high efficiency turbine, somebody
may be given a 350 megawatt, old coal-fired burner. And you\'ll see some
of these things that just was discussed now, happen \-- actually happen.

Something that Dr. Tabors said at lunchtime was very intriguing to me.
He says, if their behavior that is sort of cost revealing but not
colluding \-- that would be interesting. We have done a few experiments
and had a very interesting time with it.

We\'ve done this as a team and Doug Hale and Sam Cohen and I \-- we\'ve
actually said okay, let\'s collude. We made profits of like, \$9000 an
hour; that was the best we did, I\'ll tell you. And we had a lot of fun
doing things. Then we try and do our best knowledge, and a lot of us
lost a lot of money. But it\'s an interesting experiment.

CHAIR WATKINS: Thank you. It\'s now open to committee discussion and
questions.

DR. HANSER: I\'ll be obnoxious.

DR. SUDMAN: I\'m going to ask a question which isn\'t quite economic. It
has to do with the fact that, assuming that these, you know, power
producers also have economists on their staffs and have been working in
this area, is there any value in asking them what their potential future
plans are?

I mean, it comes from the survey perspective of course. I mean, are they
going to be able to \-- can they tell you some useful information which
can make this process more powerful?

MR. MANICKE: I would have to say, there will be some statistics that
will come out of experimentation and which unquestionably, the answer
would be yes. At this point I don\'t know. There\'s not been a lot of
\-- believe it or not, there\'s not been a lot of experimentation with
auction outcomes. But I\'m sure there will be and I\'m sure that
generators and offerors that have the comments who can think like this
will certainly be better off than those that don\'t.

CHAIR WATKINS: Calvin.

DR. KENT: Actually, I had a question more for Phil and Richard than
anything else, and that is, when we were having that discussion just a
moment ago about Vickery pricing and everything like that, I was
intrigued this morning with Richard\'s comment because, why in a market
that becomes open, wouldn\'t a power pool go for the Vickery price?
Rather than the first price?

DR. HANSER: Oh, there\'s experimental evidence that suggests that. In
essence, although the Vickery option looks like it ought to be
revealing, in fact it isn\'t. And there are a number of papers \--
there\'s one that was done, God, ten years ago by, what\'s his name
who\'s now at Rutgers \-- who\'s talking about Vickery options relative
to IPPs and all.

DR. TABORS: Mike Crew?

DR. HANSER: Mike \-- no, not Mike Crew. The guy who was at UC Berkeley.

MR. MANICKE: Orens?

DR. HANSER: No. No, I can\'t think of his \-- I\'m sorry, I\'m bad on
names. I apologize. It turns out, from an experimental standpoint, the
Vickery option doesn\'t produce any different results and it also turns
out that there is actually something peculiar that goes on in a Vickery
auction that makes you less revealing \-- that at least occurs in an
experimental setting than a single price option. So that\'s at least the
basis for that.

DR. KENT: Is this then a major issue? Because \--

DR. HANSER: Yes, it is a major issue.

DR. KENT: \-- first I heard you say that this was a major issue, and
obviously it\'s a major issue in the recovery to fixed costs. So I\'m
not sure where you guys are taking this.

DR. HANSER: The major issue is the one that we talked about \-- I
mentioned before which is, do you split the capacity and energy markets
in a formal way by creating capacity requirements on things like
demanders of energy.

DR. KENT: Okay.

DR. HANSER: That they go out and contract separately for a certain
amount of capacity, and which in essence idles that capacity and
prevents it from being utilized from the normal bidding process, being
held in abeyance as a reserve. The PJM\'s FERC filing requires that, and
we basically retain the current system that is in place for PJM.

The question is, is that an economic level of reserves, and if it\'s
uneconomic, is there the chance that in fact, because it\'s uneconomic,
it will so skew the economics of the competitive marketplace that in
fact, it will lead to uneconomic decisions, both in the additions to the
current generating capacity as well as the operation of the current
capacity \-- the current generating capacity in the long run. And
that\'s really what the source of the question is.

As opposed to a single price system where the price includes in some
sense, this cost of capacity by including what it costs to demanders of
electricity to not have it, and allow the market to clear on the basis
of price.

Up to this point in time the electricity markets have not cleared on the
basis of price. They have cleared on the basis of willingness to supply
up to, essentially, any level that\'s being demanded. And that\'s the
big difference. Do you allow the electricity markets to clear on the
price basis and allow price to ration the use of electricity, much as it
rations anything else? And that\'s an unresolved question at this point.

DR. TABORS: Let me go on half a step further in terms of Calvin\'s
question. The rest of the story gets back into this issue about just the
structure of the market and how much faith the regulators really, and
State Legislatures \-- because that\'s a major piece of it \-- have in
the workings of a true, economic market.

And insomuch as they believe they work, then you find that you\'ll go to
a single price structure, in which case you know, the bidding process
\-- however it works; I\'m going to call it a bidding process \-- will
in fact clear the market. And as soon as you move away from that you
begin to enter into the realm of market imperfections and sort of, every
time you do it you screw with the economic part of that market just one
more step.

So you know, Phil\'s indicated some of them. I can tell you that a
reliability requirement of *X* percentage does one more part of it, and
you can literally just kind of march down this path to creating, if you
will, ways for the current electric utility industry to collect \-- I
mean, whoever the owner is at this point \-- to collect more and more
money for a given resource.

And you can argue with all kinds of ways as to whether this is good or
bad, but I think as an economist you come back and say, what I want is a
single price auction. Again, if you look at what happened in California,
you know, there was a big push and there\'s still a push in New York in
PJM to get paid a fixed amount of money for just simply turning your
unit on.

That\'s sort of a no-load cost. And then you get paid a start-up cost.
And then once you get past the no-load and the start-up then you get
paid the marginal cost which is what we would agree is the market
clearing price.

Well, you know, I could show you units for which, by the time you pay
the start-up costs and the no-loads, Jesus, you\'ve made more money
turning the unit on then you\'re ever going to make running the damn
thing. You know, you\'ve got some very strange things that happen in the
system at that point, and it\'s a major issue.

What I\'m interested in in this, is that this simply a way of learning
\-- or not simply \-- this is a very sophisticated way of learning in an
experimental, economic sense, how the market will or may function. And I
think that\'s the interesting question in terms of this particular
experiment, which is I think what it is at this point, right?

MR. MANICKE: Yes.

CHAIR WATKINS: Is the person you say, was it Paul Milgrom?

DR. TABORS: No, Paul\'s out in California.

CHAIR WATKINS: That\'s what I thought. Rob Porter? No?

DR. HANSER: No, I\'m sorry.

DR. TABORS: I\'m embarrassed as well because I know who it is and I\'m
\--

CHAIR WATKINS: Let me ask you a question, please. One thing we kind of
glossed over so far is the ISO, the independent system operator. The way
I interpret that is really the function of, whoever is in that role is
to do some of the things that an integrated company would have done as
part of that planning process.

Where I see a gap at the moment in the material I\'ve been reading is
how they\'re going to be set up in the sense of, what are the criteria?
What\'s going to guide their actions? What are going to be the
parameters? What framework are they going to operate under? Do you have
any comments on that?

MR. MANICKE: I can only say that that\'s probably a random variable to
be determined, but they would just be the single buyer of the offerors
and distributed in the optimal, least-cost way satisfying the loop laws
of physics. Will there be some financial arrangements involved?
Possibly. I don\'t know. I expect Dr. Tabors may have a better answer or
more complete answer on that.

CHAIR WATKINS: One thing \-- which costs?

MR. MANICKE: Excuse me?

CHAIR WATKINS: Which costs?

MR. MANICKE: Oh, the marginal costs.

CHAIR WATKINS: Of what? Generation plus transmission?

MR. MANICKE: Okay, good. In this particular model we have a large set of
papers that describes this system: where low level lines are, what will
be transmission congestion, that sort of thing. The ISO takes in
consideration that and puts on what\'s called transmission congestion
increments onto the cost itself for your locational \-- for wherever
you\'re located.

For example, I would suggest that this generator will probably have more
locational adjustment than this generator; that sort of thing. For a
load right here or something like that. So these are built-in at this
point. The transmission congestion costs are built into this system
implicitly.

CHAIR WATKINS: Does it reverberate back then, that your choice of
generators \--

MR. MANICKE: Yes.

CHAIR WATKINS: \-- then is not necessarily what might, if you exclude
location, be the least cost \--

MR. MANICKE: That would be true, yes.

CHAIR WATKINS: Before we go to the floor, can I just see if there\'s any
more committee comments?

DR. HANSER: Two questions. First, one analogy you may want to think
about relative to the ISO is, think about the ISO a little bit like the
Federal Reserve System regulating the money supply. It\'s probably not a
complete analogy but in a sense, you\'ve got the same sort of
organization.

It\'s going to be a mixture of, you know, it will basically receive its
revenues by collecting a tax in some sense, off the transmission system
and operating it \-- much like the Federal Reserve does through
collecting money through selling Treasury Bills under rediscounting.

And you have the same problem in some sense that you had, at least
initially with the Fed in trying to figure out, how would you like to
govern it and what would you like to use as a criteria by which it\'s to
be judged? Sometimes that\'s hasn\'t done always as well as it ought to.
Arguably, the severity of the \'73/\'74 oil crisis was aggravated by
what the Fed did, but nobody sort of booted them out as being the Fed,
however.

I guess I have a question which is sort of like, why is EIA doing \--
I\'m going to be obnoxious about this one \-- in the sense of \--

CHAIR WATKINS: You set the style, Richard.

DR. HANSER: Yes, I\'m a bad influence on it. I guess I \-- there are at
least two, very large-scale economic sciences labs in existence right
now. There\'s one at the University of Arizona, there\'s one at
Cal-Tech, and probably others somewhere else that I don\'t know about.
And there\'s a very large literature \-- you know, there\'s a [Journal
of Experimental Economics]{.underline}, and so on and so forth.

So I guess the question I would have to pose to EIA is, what\'s the
value-added which EIA provides by being involved in this that the
academics don\'t already have? And is it that the academics lack the
tools? And it seems to me that EIA ought to act in a facilitative
process as opposed to being a participant, perhaps, or doing it.

MR. MANICKE: Hopefully both.

DR. HANSER: Maybe both.

MR. MANICKE: Both. EIA is obligated to produce forecasts and if the
price of electricity is deregulated \-- our model is a totally regulated
model right now. It would be pretty hard to validly sell that unless we
had some sort of site cases in which we looked at competitive models or
competitive behavior like this. And also using the academic findings and
embed them in the same kind of thinking with our forecast.

DR. HANSER: Then I guess I would recommend that, what used to be done is
develop a much more focused research plan which sort of says, what holds
in the literature of strategic bidding and auction are there that are
not \-- that prevent you from doing your forecasting properly, and then
focus on those things. And it\'s not clear to me that going through a
PowerWeb per se, you know, necessarily is the best way to do that.

I guess that\'s my concern, because it\'s a neat tool but I don\'t know
that it\'s going to get you further along in terms of your being able to
forecast, you know, what\'s going to happen if the markets are not
perfectly competitive. That\'s I guess, my concern. So I just raise
that.

MR. MANICKE: That\'s my concern, too. We have to look into that
carefully.

CHAIR WATKINS: We have one or two questions from the floor. John.

MR. WOOD: We would like to think that, I guess if natural gas was
deregulated before the electric utility and the electricity, that there
may be some information to be gained. Can you run the process backwards?
Can you look at what the clearing price was and impute the kind of
behavior that was actually going on in the suppliers of electricity or
natural gas?

And I will \-- the data isn\'t quite complete yet, but in 1996 there was
a very rapid escalation in the price of natural gas at the wellhead. And
there were thousands of suppliers. As as far as we can tell there was
plenty of capacity; and demand wasn\'t exceptional. There was 10 or 20
percent extra capacity.

It\'s kind of hard to get 3,000 people together and it\'s against the
law to collude on prices, but is the kind of process that you\'re saying
here, I mean, are there real life examples going on? Because this is a
lot of money \-- the price change from a dollar-and-a-half to over \$3,
\$3.50 at the wellhead. You know, it\'s the most volatile commodity
almost, in the world right now.

And you know, can you say from the games that you\'re ready to play, you
know, why that happened?

MR. MANICKE: No.

MR. WOOD: Or lend any guidance at all?

MR. MANICKE: I don\'t think anybody can, but I can tell you some
differences. It\'s not the \-- natural gas is not the most volatile
commodity. The most volatile commodity is electricity right now. An
interesting difference also, is that natural gas can be stored. That
adds a whole new dimension into the pricing. They have their bid week in
which things become sort of irrational at times.

One of the interesting things that happened in Australia is that on a
certain summer day, spot price in Australia went up to over 6,000 a
kilowatt hour. That\'s hard to imagine. The load forecasters were
extremely wrong, the bidding went absolutely insane, all the futures
markets were sort of cleared very quickly.

That was probably because of some certain congestion and the fact that
the bilateral, the forward contracts, don\'t have a natural barrier of
storage in electricity. In natural gas, the process in the futures
market, of tango of the futures market going up and up and up and up,
has a natural barrier of stores. And people say, well I might have to
take delivery so I might have to store this.

Well, in electricity, that doesn\'t happen. So is there an analogy
between natural gas and \-- only in that things are sort of irrationally
timed, is about as good an answer as I can give. Dr. Tabors may be able
to give a better answer.

MR. WOOD: If anyone could comment \-- it\'s the process, obviously the
gas industry could have made very large profits at 30 percent less
price, and yet somehow the price kept going up. Is there a \--

CHAIR WATKINS: But then it came down. There was another question from
the floor, I think. Could you please identify yourself?

MR. LIGGETT: Yes, my name is Bill Liggett; I\'m with EIA. I wanted to
ask Bob or the floor \-- there\'s an energy market and there\'s a
capacity market. I was wondering, is there room for a \-- or is there a
possibility of a transmission congestion market to hedge against the \--
I know that\'s a \-- I read in the literature there\'s something called
a transmission congestion contract. And so is there a market possible
\-- will a market develop for a transmission congestion?

MR. MANICKE: Well, that\'s a very good question and I guess the answer
is, yes and no depending on who you believe. One fella by the name of
Hogan from Harvard seems to think that that\'s not too good; that you
can\'t really eliminate energy and the transmission congestion and make
transmission congestion a commodity. So you can do it in a passive sort
of way.

But then there\'s a debate with Orens at the University of California
that says, active transmission charges in the future, such as forward
contracts, will be established and be very strong. I don\'t know. I
would like some of the responses \-- Dr. Tabors and Dr. Hanser on this
one.

Dr. Orens also notes that you can take the nodal spot prices between two
nodes and write them as a short forward, a long forward, a fixed
annuity. And then we get into zonal \-- there\'s thousands of nodes but
can you do this for every node? Of course not; that\'s ridiculous.

And then you get into Dr. Tabors \-- where are these zones and how
should these zones be calculated and that sort of thing. That\'s a very
complicated issue. I would like anybody to respond to that.

CHAIR WATKINS: Well, I can make one quick comment. You have to have a
market or ability to acquire transmission lines.

MR. MANICKE: Correct.

CHAIR WATKINS: I\'m afraid we\'ve run out of time on this session and
Jay Hakes would like to say a word or so and then we\'ll reconvene at
3:30.

DR. HAKES: This morning we gave Campbell a letter signed by me and a
certificate signed by me expressing our appreciation. After lunch in an
urgent folder we received a letter of appreciation and a certificate
signed by Secretary of Energy, Federico Pena. And given that this, I
think, better reflects the stature of the committee and Campbell\'s
service, we wanted to present this to him.

You might ask why this didn\'t arrive earlier. We looked at it very
closely and concluded it was because of congestion in the Department\'s
transmission system. And we also concluded that my letter of this
morning may become a stranded asset.

(Laughter and applause.)

CHAIR WATKINS: I\'m very pleased to know there\'s some reserve capacity.
We\'ll take a break and reconvene at 3:30.

(Whereupon, the foregoing matter went off

the record at 3:26 p.m. and went back on

the record at 3:41 p.m.)

CHAIR WATKINS: Okay. The next paper we\'re going to have is by one of
the committee members, David Bellhouse. So I\'m going to give you the
floor.

DR. BELLHOUSE: Thank you. When I talked to Bill Weinig about this
presentation, he expressed interest in my trying to give a presentation
that would generate discussion about the use of covariates and the
reduction of sample size in EIA surveys.

So I see myself as a front-man for the other two speakers, Jim Knaub and
Eugene Burns. They\'re going to present from the EIA point of view about
the use of covariates. And I see myself as saying that the use of
covariates in these surveys in technologically feasible.

And there was a paper that was circulated, written by myself, Mike
Hidiroglou from Stat Canada and a colleague, Jamie Stafford in my
department. That paper is about the technological feasibility of some
computer software that\'s being developed, and so I saw that as
background information. And so I won\'t be presenting the details of
that paper but talking more about what\'s technologically feasible.

So the first thing I\'d like to say is, if we\'d just all go back to
basics of regression analysis. If you put in some covariates into a
regression analysis \-- if you had no covariates and then added
covariates \-- what happens is that you\'ve reduced the residual
variance. By reducing the residual variance you are able to make
confidence interval statements that are narrower.

So if you were to set a desired confidence level or a confidence
interval width, you could use covariates then to reduce the sample size
to get the required confidence interval. So basically, we\'re doing the
same thing in sampling with the introduction of covariates.

What we\'d like to do is to use covariates to reduce the variation so
that if you wanted to have a survey with an acceptable level of
variation you could use covariates in order to reduce the sample size to
achieve the same level of variation.

So the kinds of slides that I\'ll present, partly historical, but it
will illustrate the changing technology and how covariates can be used
in surveys. So the first one talks about, basically, the classical ratio
estimator. If you have a covariate *X* and you have a population mean on
that covariate, you can easily calculate an estimate which will improve
in variance, provided that *Y* and *X* are positively correlated.

Under the sampling design it has nice properties \-- that is, under
simple random sampling it\'s approximately design unbiased or unbiased
with respect to the way in which you choose the sampling design. It\'s
more efficient than the usual estimates if *Y* and *X* are positively
correlated. And this is one of the earliest estimators used when you had
covariates because of its simplicity.

In the days of punched cards you were able to get separate estimates of
*X* and *Y*; you don\'t have to match *X* and *Y* \-- at least to get an
estimate. So in the days of punched cards it\'s an easy estimate to
obtain.

The other estimator which appears in standard sampling books is the
regression estimator. For that you do need to match *Y* and *X* because
you need to estimate this regression coefficient here. Again, it\'s for
large samples under simple random sampling, approximately unbiased with
respect to the sampling design, more efficient than the usual estimator.

And in this case it doesn\'t matter if you have positive or negative
correlation; it works either way. So in that sense this estimate is
preferable to the ratio estimate because you could have positive or
negative correlation. And it\'s often more efficient than the ratio
estimator.

So that\'s basic results and sampling that are not new. Some of the
stuff that is new is the newer technology called GREG estimation, or
generalized regression estimation. Basically what we\'re doing is, again
getting a regression coefficient, getting an estimate on the *Y*s,
getting an estimate for the *X*s, and having the covariate mean on the
*X*s, so you\'re still dealing with a covariate.

You have the *X*s and *Y*s matched like you did in the regression
estimator, but this applies to more general sampling designs. The
previous one applied in simple random sampling. This applies more
generally.

This estimate or system of estimation is due to Carl Sarndal, and it\'s
motivated through a model so it\'s called model assisted. So you assume
that there\'s an underlying model of a linear regression model, you come
up with the estimating equations that you would get to estimate the
regression coefficients, and you fix them up according to the sampling
design.

So if you were to take this matrix and multiply it over here, you would
get the usual estimating equation for the regression coefficients. If
you take out the *W*s it would be the usual estimate in simple linear
regression. But now you take into account the sampling design by putting
in the sampling weights which, if there\'s no non-response and
everything else is okay, it\'s related to the inclusion probability.

So what you\'ve done is motivated an estimator through linear
regression, fixed it up or changed it slightly under the sampling design
so that it would be approximately unbiased for large samples.

So in order to use covariates for an increase in efficiency you want to
find something that has fairly strong, positive or negative correlation
\-- as long as it\'s a strong correlation. If you use a regression
estimator you would gain an efficiency with at least a strong
correlation \-- either positively or negatively. You need to have the
sample weights and the units all matched, so you need a file with
everything matched on a case basis.

You need some sort of population mean, or you need means for the
covariates in order to calculate these estimators, and you need some
kind of automated system. You don\'t want to reinvent the wheel every
time and rewrite a whole bunch of software, so you need some kind of
automated system which \-- the Canadian version is called the
Generalized Estimation System.

So when you have all of those ingredients then you can go ahead and do
the estimation \-- and we have a brief, commercial pause here for the
Statistics Canada version. There are several other versions of software
that does survey estimation, but this one is the one put out by
Statistics Canada, and I\'ve put it up for another reason which I\'ll
get to in a moment.

Again, with this version, as with the others, you would just have a
survey file with everything matched. I believe *X*0, 1, and 2 are the
covariates and *Y* might be \-- one of the *Y*s might be the variate of
interest \-- so that everything in the file is matched so you can go
ahead and do that regression estimation.

And the concluding remarks that I\'d say in terms of introduction is,
one of the reasons why I put up the Generalized Estimation System is
that I\'m working with people at Statistics Canada on new versions of
this system. In particular, my input to the system is in computer
algebra, which is related to the paper that was handed out. I\'m working
on a system in Mathematica 3.0 that does automatically, all the
algebraic calculations and sampling.

So if you want to find expectations of estimators, if you want to find
variances or even third-order moments, it can be done automatically in
Mathematica without having to put pencil to paper.

So the last thing I\'d want to say is, ideas on how EIA might use this
in terms of improving \-- not improving necessarily, the surveys, but in
reducing costs is \-- if you have a covariate that you can get from
another source \-- and I can only guess at what those other sources
might be \-- there might be something in current population survey or
covariates from the census where you can get population totals of some
sort.

Then you ask the covariate information on the survey as long as it\'s
positively or negatively correlated with variables in the survey, then
you can improve on the estimator by lessening the variance, which then
allows you, if you have an acceptable level of variance, to reduce the
sample size and thereby save money.

The other possibility is in surveys that are taken over time. If you are
doing regular surveys you could on one occasion, do a fairly large
sample and in the next occasion do a smaller sample using the earlier
occasion as the covariate.

So it would allow you do to periodic surveys of modestly large samples
on every second, or every third, or every fourth occasion, and in the
intervening occasions, taking smaller sample sizes and using the larger
survey as the covariate and the smaller surveys merely to get updated
estimates.

That again, would allow a reduction in sample size and some cost
savings. So with that I\'ll turn it over to Jim and Gene to talk about
use of covariates at EIA.

MR. KNAUB: That\'s wrong. That\'s supposed to say Dean Martin and Jerry
Lewis instead of Burns and Knaub.

Actually, Gene and I don\'t cover all of EIA of course, but we\'re from
two different parts of EIA so the idea was that we could cover a couple
of major parts of EIA. But there would be other things that we wouldn\'t
mention that we might not even know about from, for example, Owner Gas,
other places.

Covariates and electric power, this may be quite different from other
parts of EIA because electric power surveys are establishment surveys
that are very highly skewed and smaller than other cases, and being
extremely highly skewed surveys found that its covariates have been very
useful.

I\'ll get to that in minute, actually; I\'m getting ahead of myself,
here. I wanted to mention here that people in EIA have worked a number
of areas and that covariates are a consideration.

Okay, something that Dr. Bellhouse just mentioned was that one of the
best covariates you can have is previous data, and we\'ve been using
previous census data on occasion, for covariate data, and this has been
highly successful in electric power.

If you go back to 1989, we were at that time had already been using
design-based sampling using previous census data for auxiliary data, and
the last year that we used a design-based sample for a major case \--
there\'s this one major survey, I should say, that we do sampling other.

There are other censuses in EIA and smaller samples are being done now,
but the major case where we use the sampling is a case where we have
sales and revenue data that are collected in the census annually and on
a monthly sample.

Like I say, the last time we did design-based sampling there\'s a
write-up in 1989, Survey Research Methods Section of the [Proceedings of
the ASA]{.underline} that explains it. There we used Kiefed\'s method
for ratio estimation and double-ratio estimation for revenue for
kilowatt hour.

And given the restrictions in data collection on a monthly basis and the
fact that we really did need to concentrate on the largest utilities,
this worked out very well.

A couple of years later we switched completely to model-based estimation
with a cutoff sample. The reason being that with this particular survey
I just mentioned \-- the sales and revenue data that are collected in
the census annually and a monthly sample \-- we wound up with a very
large \-- even when we had design-based sample using those other data
\-- we had a very large census of very large utilities.

And that census stratum of very large utilities covered say, at least 60
percent in most cases, of sales and revenues in the different economic
sectors \-- residential, commercial, industrial. And so there was this
census stratum and then there were these stratified random sample of the
smaller utilities.

But if we eliminated that stratified random sample of the smaller
utilities and only used the previous census stratum of the largest
utilities in a model-based sample, we got basically the same results \--
dropping 30 or 40 percent of the respondents. And those are the ones we
really wanted to drop anyway because with the smallest respondents there
are many, many of them that didn\'t add up to very much and people would
call if one of the utilities would fail an edit.

My favorite story is that when somebody called and asked about a
particular set of numbers that didn\'t look right and the guy that
answered the phone said well, my sister-in-law does the books and she\'s
out shopping. Whereas, with the largest utilities you\'ve got maybe a
whole department doing all the government forms.

So if we could eliminate a lot of non-sampling error and a lot of time
spent and a lot of burden in these monthly samples by getting rid of the
smallest utilities, that turned out to be advantageous.

And we\'re then using the previous values for both sales and revenue as
these sales in the previous Census as a regressor for sales in the
current sample and revenue in the previous census as revenue in the
current sample. And this has worked very well.

I also have done this use of auxiliary data and regressive data \-- you
think of covariates as either auxiliary data if you do have a
design-based sample or regressor data if it\'s a model-based sample.

There was a case where we were looking at generation and operational
costs information on plants and I tried both methods \-- PPS sampling
and model-based sampling \-- and the problem was that we had some
amputation that had to be done. Usually \-- in electric power it\'s sort
of unusual \-- but we do have close to 100 percent response in most
cases. But like I say, they\'re smaller surveys.

And in this case with this generation and operational cost information
on these plants, we did have some non-respondents. And so no matter what
you do you\'re going to have to be taking some sort of a risk there. Of
course, with the model-based sampling it\'s easy to at least get some
indication of whether or not you\'re imputing too much with \-- based on
the variance you get to on that.

So we explored use of covariates there and if it weren\'t for that, the
data was so stunted there wasn\'t a whole lot we could have said without
it. That was more like \-- that was more like in an econometric\'s case
where you\'ve got certain data you\'re just sort of stuck with. And when
we tried to do PPS sampling, like I said, we couldn\'t get all of it.

Okay. Since then, I\'ve used multiple regression estimation for sales
for resale of non-utilities. We have a distinction between utilities and
non-utilities that still exist now that may disappear in the future, and
we don\'t know what kind of service we\'re going to have in the future,
but right now those are collected separately.

And non-utilities, we were collecting data on that. When I tried to use
a single regressor just the previous day from a census, it wasn\'t
enough. Let\'s see, I think Skinner and Silva have done some work
recently, but that\'s in design-based sampling using auxiliary data,
where they have checked to see, what is the optimal number of
covariates.

Because if you had enough covariates then it\'s going to run into a case
where you use an optimal number. Around here we\'re fairly lucky to have
a couple. Usually there are several different bits of data you could
collect, but the best one would be \-- the best regressor would be the
same data from the previous census.

For this survey, the non-utilities, here\'s what it looks like if you
look at sales for resale as a regressor using sales for resale from a
previous census for sales for resale on the current sample. And it
actually sometimes looks better than the graph shows because in some
cases we have a whole lot of points on top of each other on the line
there. And so the few points that are scattered out make it look less
dissolved than it actually is.

But I did find that, in this case, that it wasn\'t satisfactory just to
use the one regressor, so I had to find something else besides previous
census for sales for resale. So another regressor I found which looks
worse by itself, is nameplate capacity. The nameplate capacity has the
advantage of never being negative; you can always get your hands on it;
never zero.

Whereas, in sales to resale one of the biggest problems was that sales
for resale from non-utilities is often zero one carried and not zero the
next \-- often enough that it caused problems. But if you throw in
nameplate capacity as a second regressor, at least you could always
count on it.

And together, they actually work really well when you use the two
together.

DR. HANSER: Excuse me. What are the axes?

MR. KNAUB: Okay. Yes, well actually I took those off because I didn\'t
want to \-- there\'s a little note at the bottom you probably can\'t
read that says, \"Real data used here were altered in the graph only to
avoid disclosure of sensitive information and yet maintain the general
appearance of the graph\".

And on this graph the *X* axis is the nameplate capacity and the *Y*
axis is the sales to resale from the sample. In the previous one the *X*
axis was a sales to resale from the previous census and the \-- you have
sales to resales sample on the *Y* axis both times.

Okay, I\'m also \-- modeling can also give us a lot of help on
imputation in electric power data because like I said, we generally have
close to 100 percent response, but that\'s changing a little bit with
the restructuring of the industry and more problems with confidentiality
and all, and people just not wanting to disclose their data.

So sometimes it would be useful to see, have we got 2-inch variance if
we just treat one of these guys as having not been in the model sample
in the first place? How much variance would we have then? Is it worth
trying to collect this guy\'s data before we publish or what? Because
we\'re always under time constraints so time and cost and burden
constraints on the more frequent samples are of concern to us.

Anyway, this paper mentions there\'s a, when you\'re looking at what\'s
your best model, heteroscedasticity is a big concern with getting the
best estimate, and estimating your degree of heteroscedasticity in these
models sometimes will \-- there\'s always model failure because the
model\'s never exactly right. It just depends on whether it\'s not
enough of a failure to make any difference or not.

If you\'re estimating for a whole lot of little, small guys that only
add up to 20 percent of the total, then you can be off by a bit and not
make as much difference as it would be if you were in say, a household
survey.

But if you\'re trying to get the best model for these data then
estimating heteroscedasticity is important and sometimes this change in
variance as you get away from the origin, actually there may be an
acceleration rather than just the change and it may depend on what part
of the graph you\'re looking at as to what you might want the model to
look like as far the estimate of heteroscedasticity.

And if you\'re using a cutoff model sample then just \-- you\'re looking
at \-- the guys that are missing are all near the origin. So maybe you
want to put a little extra weight on the observations you\'ve got that
are closer to those failures as opposed to the values that are further
out. That might be one consideration.

However, if you\'re imputing for somebody who\'s a little bit larger,
that would be a different part of the graph. So there\'s just a little
study done on that.

Currently, I\'m looking at time series and some fuel costs and related
data and some special coal stocks. And covariates are key there, and I
find that I\'m searching for the best use of covariates in a situation
dealing with some special stocks that have to do with distribution and
producer\'s stocks.

So we found a number of uses of regressors and past auxiliary data so
far in electric power, and expect that that\'s going to continue,
especially with the change in the electric power industry and what we
might look at might equate different.

But one thing that occurred to me after talking to Dr. Bellhouse the
other day was that for example, sometimes we\'ll have, as I mentioned
earlier, an annual census and a monthly sample, or maybe we\'ll wind up
with maybe weekly samples in some cases. Who knows what will \-- I know
we do over in Petroleum they have one of those weekly samples. And we
went with that and with electric power.

But we don\'t know what the future\'s going to hold. But one thing that
might \-- instead of having like a previous census and a current sample,
there might be an extra step in there where there might be a census done
every several years maybe and in between a \-- perhaps use the GREG and
estimate there, and then maybe a model-based sample on the most frequent
sample \-- maybe a 3-step type thing.

But anyway, people have been finding more and more usefulness for
covariates lately and sometimes because of the fact there\'s so much
pressure on us to get things out right away, you know, before you even
get the data, that \-- we have to watch out for that. There\'s a
pressure to use engineering estimates sometimes and I prefer having
something where you can at least get a better idea of what the accuracy
is.

Let\'s see, the electric power industry is changing and so are their
data collection. Now as I mentioned earlier \-- or, meant to mentioned
earlier \-- we want to \-- it\'s best to have a lot of different options
at your disposal. Like I was recently looking into WesVar PC and they
have this new tool that Dr. Bellhouse has added to GES, he talked about
yesterday at a WSS seminar \-- would be an interesting thing for us to
look at. And one possibility for looking at that.

And then we\'ve got one set of data that might be test data to compare
generation, consumption, and stocks. But I think \-- currently, the way
things are right now, we might not have too many good places to use it
except for like testing it in a case like this. But I\'m thinking that
you never know what\'s going to happen in the future with the
restructuring of the industry.

Here are some things here that may already be sort of answered. Let\'s
see. I don\'t think anything came up about hardware or software
restrictions. It\'s worth going with just about anything we\'ve got,
wouldn\'t you think?

DR. BELLHOUSE: It works on a PC.

MR. KNAUB: Most people have pentiums now and a 486, anyway.

DR. BELLHOUSE: Yes. It works on a PC; it is fairly user-friendly in that
it\'s menu-driven. And the one advantage that it could have over WesVar
PC would be that it handles the generalized \-- or the GREG estimates,
and I\'m not sure that WesVAR does.

MR. KNAUB: I think when we\'re done here you might want to see if you
can help us with. I think I have the answer from the first one from
hearing yesterday\'s talk. It\'s on \-- this is strictly \-- your method
is strictly using Taylor series, not replication, right?

DR. BELLHOUSE: Right.

MR. KNAUB: Anyway, that\'s another option, anyway. And let\'s see, the
possible use of covariates is important \-- okay, everything you were
dealing with was really designed-based, right? Didn\'t you say something
about you were looking into model-based sampling in the future?

DR. BELLHOUSE: Well, the material that I\'m working on is design-based;
that is, getting the algebraic forms of the estimators. But GES is
model-assisted; that is, you specify a model and it gets you an estimate
under that model which is fixed up according to the sampling design so
that you benefit from the model and you protect yourself by using the
sampling design in the estimate.

MR. KNAUB: I think that \-- see, the only problem with electric power as
far as the design goes is that, you know, the best thing about model
sampling there is also the worst thing about model sampling, and that\'s
that the smallest guys don\'t have a chance of being selected.

Well, that\'s good if the small guys are going to be full of
non-sampling error and you haven\'t got much time to collect them, and I
think that \-- we\'ve got cases where the smallest ones, if we tried to
collect the data in a decent sample \-- maybe even in a census in the
period of time we have \-- we may not get as accurate a result as we
would have gotten from the model, especially considering the cost and
time considerations.

So with the model-based sample we have a chance to concentrate on the
largest utilities and use their resources more efficiently. But of
course that\'s also the worst thing about model-based sampling because
you can have model failure that you need to be aware of.

Let\'s see, on the last thing \-- area sampling. I was kind of wondering
about that because this would be, you know, something where you \-- for
example, you didn\'t have a list. Have you considered area sampling?

DR. BELLHOUSE: I\'m not sure I can answer that. I\'m fairly sure it does
as long as you have a file with the survey weights and another file that
has the population means on the covariates, you\'re okay. It covers an
area sample.

MR. KNAUB: So, like an area sample you don\'t know actually, who\'s in
there until you \--

DR. BELLHOUSE: Well, but in the survey file you would have everything
matched, case-by-case. You would have all of the variants in the survey
file.

MR. KNAUB: That\'s all I have. I\'ll have Eugene come up and talk about
another area of EIA. Thank you, Dr. Bellhouse, for answering the
questions here.

MR. BURNS: Okay. Good afternoon. I\'ve always thought of sampling and
estimation as being like two sides of the same coin of survey design,
but after reading, in particular, the paper that Dr. Bellhouse
circulated with Hidiroglou and \-- the other author \-- you know, when
you go through this and you get the suggestion that there\'s really a
third; that we really should pay serious attention to the third side of
the coin. And that\'s the sort of auxiliary data that we use when we\'re
doing sampling or when we\'re making estimates.

And auxiliary data isn\'t something that\'s new. It\'s been there
implicitly. If you\'re doing say, PPS sampling or even if you\'re just
stratifying your population, you\'re using some auxiliary data. But what
I want to do here is just review how we\'re using auxiliary data when
we\'re doing the energy consumption surveys \-- our surveys of the
energy consumers as opposed to the energy suppliers which Jim was
working on.

Now, the first of the three surveys is the manufacturing energy
consumption survey and it\'s conducted for EIA by the Census Bureau. We
use data from the census of manufacturers, updated as necessary by the
annual survey of manufacturers.

We use that data as the \-- well, first of all, we use it as a frame \--
and also it\'s used to stratify the manufacturing establishments by SIC
code. And then it\'s also stratified by the Census Division and sample
is done by PPS. So this is sort of a PPS stratified sample. So that\'s
the sort of use that\'s being made right now on the manufacturing
survey.

Second big energy consumption survey is the residential energy
consumption survey. It\'s a multistage area probability sample, just
following up on Jim\'s last question. At various stages of the sample
we\'re using population data from the U.S. Census.

And in addition, there\'s another bit of data that\'s brought in,
auxiliary data, from the current population survey. And these data are
used to from control totals on the number of households. The control
totals are formed by geographic area, by metropolitan, non-metropolitan
location, and also by type of household.

As implemented in 1993, which is the last RECS where this was used, the
household counts were adjusted by geographic area first, then by the
metropolitan on the type of area and the type of household, and then
there\'s a final adjustment done once again to the household counts by
geographic region. So it\'s not quite a raking estimate but it has some
of the elements of it.

Okay, now the third big energy end-use survey is the commercial
buildings energy consumption survey. It\'s a sister survey to the RECS
and it\'s also a multistage area probability sample.

At the first stage, the primary sample units are selected using
population information; the secondary SSUs are selected using a
different type of information \-- it\'s employment data from county
business patterns \-- which gets us down to the zipcode area; and then
finally at the building selection area we are actually using lister
information where we actually send people out to list all the commercial
buildings by size and by gross measure of size and the gross measure of
what type of activity is going on within them.

It\'s also a dual-frame survey because we supplement the area frame with
lists of large buildings and different kinds of specialized buildings.
As far as finding other covariates for it, we\'re the only national
survey of buildings, which means that actually, for better or worse,
we\'re the benchmark for everyone else who wants to deal with buildings.

There is some information on [Value of Construction Put in
Place]{.underline} that is published by the Census Bureau, the
Construction Statistics Division. We know from tracking our data that we
have more coverage than those estimates do. We also know that early on,
before there was a commercial building survey, the amount of floor space
in commercial buildings was being grossly underestimated. So that\'s
about the state of covariates for the commercial building survey.

Another issue that\'s sort of important is the issue of data
availability on confidentiality, particularly the confidentiality part
of it. If Dave Bellhouse wants to do a survey of Canadian households he
can match back to the Census data. We don\'t have that set of access to
the covariates here that you would have in Canada, or even that Jim
Knaub would have with the electric power data.

For one thing the MECS data are held under Census Bureau confidentiality
protection. So all we receive are tabulations of data after the
disclosure avoidance. For RECS and CBECS we receive microdata that are
suitable for doing statistical tabulations but which have been stripped
of anything that might identify individual cases. And the frames and the
sample identifiers are kept at our data collection contractors.

When I said this to Dave Bellhouse yesterday his reaction was
indignation. It\'s one of these, what? Don\'t you have a statistic\'s
act here that protects the privacy of respondents? And the short answer
is no, we don\'t have one. We\'ve tried but that sort of protection is
really at the Census Bureau and not at this particular agency.

So that puts some sort of limits on how we\'re able to get back to
covariates. It can be done but it\'s not as easy for us to experiment
with what those would be if we were in the same organization that
actually had possession of these sets of information.

So basically, we are making some use of covariates at different parts of
the survey. It is more \-- I guess you would say, more traditional uses
of covariates in estimation. Then when you\'re looking at the RECS, sort
of control total adjustment on post-stratification, it seems like this
might be the thing that\'s sort of closest to the sort of methods that
Dr. Bellhouse is recommending.

Maybe we\'d be using more of a generalize raking then a generalized
regression estimator. I\'m thinking of some articles that Deville and
Seindel had in [JASA]{.underline} maybe five years ago \-- things on
calibrated survey estimates, calibrated weights, things like that.

I\'m not sure if actually doing that would give us a really significant
improvement in variance, but that would be the whole point of doing the
test. And I also know that the 1997 RECS is currently being adjusted, so
one of those things, when I talked to the survey manager it was a case
that if I had something programmed they\'re ready right now; we might be
able to try it. But as it is, this would be more of a, you know, for
future research.

I\'m not sure what can be done with the CBECS at this point. When I
spoke with \-- one of the \-- I think a good suggestion was the one
about maybe using \-- taking a large sample and then maybe relating the
other samples to the larger sample later on. Another suggestion that Dr.
Bellhouse made yesterday was that we could maybe relate our estimates to
the lister information from the previous survey \-- that\'s on our
frame.

I\'m uncomfortable with that because I always felt you should be
calibrating to something that\'s somehow more precise or more certain or
more accurate than what you started off with. For instance, when we\'re
calibrating RECS, household totals to the current population survey,
it\'s also a survey but it\'s a higher level of decision than ours is
because it\'s a much larger sample.

Anyhow, one of the basic things is that it\'s harder to experiment with
these surveys just because a lot of our information isn\'t in-house.

Now, the future. Well, it may be more exciting than we\'d hoped for.
There are a lot of changes going on in the industry. In a way it\'s fun
trying to keep track of it. There\'s also changes going on with our
budget that aren\'t as much fun. But on the plus side, one of the things
I see is that there could be some really interesting statistical work to
be done using covariates in the future.

We may be able to use lot of auxiliary data, or use it more, to do some
sort of data stretching. The survey intervals between our consumption
surveys has increased from three to four years. We may start getting a
demand for interyear estimates.

We also may be \-- at an earlier session of the committee there was a
presentation on state-level estimates form the RECS. And John Grace was
the discussant on that one.

But anyhow, we may be using some of these techniques \-- some of the
auxiliary data, some of these techniques to actually improve some of our
own estimates at lower levels. So it\'s like a small area estimation
applied to some things that we would have not used small area estimation
for if we had a large enough sample.

Another point is that for different subpopulations, for example we may
have an opportunity for the 1999 commercial building survey to make some
estimates for new, small buildings. And that has something to do with
not having enough money to go out and redo the frame so that we can get
coverage of new construction that\'s good in the year of the small
building.

So I guess in summary, for both myself and for Jim, we need a full bag
of tricks. Earlier on, in one of our E-mail conversations, Dave
Bellhouse referred to himself as the token Canadian on this committee.

Well, Canada happens to be the home of a world class statistical agency
\-- another world class statistical agency \-- and we\'d like to thank
Dr. Bellhouse for one, keeping us in touch with some of the new
developments that are going on with Statistics Canada, and second, just
to point out the useful directions for us to be going and to be
exploring in the future.

CHAIR WATKINS: Thank you. Before I throw this open for committee
discussion I just wanted to ask David Bellhouse, did you want to comment
at all on, in light of what the two discussants have said? Opportunities
you may see for the GES current application on some of these surveys.

DR. BELLHOUSE: I don\'t think I have any further comments. I think that
the GES is merely a tool for analysis. There are other packaged programs
available that are competitors and can be obtained cheaper. I\'d go for
those too.

CHAIR WATKINS: Let me broaden the question a little bit. Forgetting the
GES, are there things that were said about the work being pursued by EIA
where you think a covariate approach could be used that perhaps hasn\'t
been considered yet?

DR. BELLHOUSE: Well, I think in the sampling on various occasions where
you can take a subsample in between, might be a useful thing to explore.
And again, coming from the Canadian scene where everything\'s centered
in Statistics Canada and you get whatever data is available, if there\'s
any opportunity for interagency cooperation so that covariates could be
made available for these kinds of surveys within some kind of data
protection, then it should be explored.

CHAIR WATKINS: Jay?

DR. HAKES: One thing I sometimes do in opening remarks is report on our
efforts to get confidentiality protection that would enable us to share
data more easily with other statistical agencies. This year we actually
are making some progress in that respect.

Congressman Horne who\'s head of the relevant subcommittee in the House,
has introduced a Bill that would explore the possibility of combining
the Census Bureau, the Bureau of Labor Statistics, and the Bureau of
Economic Analysis into one agency, and provide equivalent
confidentiality protection across all of the statistical agencies,
including EIA.

EIA\'s protection would, in the first instance, apply only to
consumption surveys, and then once that was in place we could explore
expanding that to the other areas. There is some difference between
introducing the Bill and getting it passed, but it does have bipartisan
support. The administration is supporting the Bill, so at least we\'re
\-- it\'s not something that we can just sort of complain about now. We
actually have some vehicles to try to make some progress.

And I testified before the House several months ago on some of the
opportunities that would provide for us if we could have more
interagency cooperation.

CHAIR WATKINS: Brenda.

DR. COX: I think there\'s some basic confusion here. Dave\'s talk
yesterday was about a computer formula that he had developed that would
allow you to derive estimators, expectations, variances, higher order
moments, for almost anything. He derives basically the formula which
then maybe, depending on what the formula is, could be estimated in GES
or some other system.

You might have, depending on the estimator you develop, you might have
to write a custom program to develop it. GES though, I think is
basically a Taylor series approach, so it\'s like a competitor to
something like SUDAN. The WesVar PC is a replication-based approach and
that\'s just an alternative method for deriving the variance.

Now, the model-based approach that Jim Knaub was describing, I don\'t
know \-- I wouldn\'t call that model-based sampling. Actually what it is
is, it\'s a cutoff census with a model-based estimation. In other words,
you are using \-- it\'s John Wood\'s problem over here. In the offyears
for the oil and gas survey in the sense that, in the offyears he has
data for the large producers and nothing for the small. And not even
sample data for the small.

Fortunately though, he has data in the onyears that can be used to
develop the model. And the question that really comes up for John and
other people is, how do you develop the best model, and a related
question is, how do you know it\'s the best model? Given that you don\'t
have data for those years to say what really happened.

And the techniques that Dave is talking about might be one of a set of
tools that you can consider.

CHAIR WATKINS: Just one quick comment on the question you raised just at
the end. For the onyears then you can replicate \-- you have your model
estimates and then you have the actual data, and that will tell you
something.

David, do you have anything you want to comment on in light of what
Brenda just said? So it\'s \-- actually before that, Seymour.

DR. SUDMAN: Well, basically and obviously, the use of covariates is
eminently sensible. The question I have to raise, however, looking at
the RECS thing, is whether the right covariates are used.

I mean, I\'m looking \-- you know, if it\'s an energy consumption survey
I\'m looking for covariates that have something do to with energy
consumption and it\'s not \-- maybe you\'ve tested all sorts of things,
but I would have clearly tested \-- for example, you have single male,
single female, and multiperson. Certainly the number of people in a
household, just that number seems to me to be related, so why wouldn\'t
one use just number of household member, rather than this break?

You know, if energy is related to say, climate, why wouldn\'t one look
at say, different climate rather than say, four largest states. I guess
I\'m wondering, you know, it seems to me that if it hasn\'t been done
there is certainly some room for testing of different covariates which
might be more powerful. Maybe that\'s all been done.

MR. BURNS: Okay, I can answer part of that and I think \-- well, Bob
Latta is hopping up. Bob Latta is the survey manager for the
Presidential Energy Consumption Survey, and I was talking with him about
this, about what they\'re thinking and what they\'re doing right now. So
if you\'d rather \-- so here\'s the horse\'s mouth.

MR. LATTA: I\'m Robert Latta. There\'s actually one stage of
stratification that wasn\'t presented here. When you do census division
and region, it\'s also taking into account climate. So you don\'t put
Florida \-- you don\'t put say, Georgia in with West Virginia. So there
is climate taken into account when you do the stratification.

Also, at the secondary sampling unit level for metropolitan areas they
stratified the secondary sampling units by variables related to energy
consumption. In other words, they would put a census track which was
heavily large, single family, detached homes, in one stratum, and they
would put a census track which was primarily small apartments in another
stratum. So there were like three stratums: a low consumption stratum, a
medium consumption stratum, and a high consumption stratum. And then
they selected the secondary sampling units from those stratums.

So that there were some additions. Beyond the year 2000 \-- say for the
2005 RECS when we were probably due to a major redesign \-- there\'s
opportunity to do much more. In particular, when you talk to customers,
people are not interested metropolitan versus non-metropolitan, as much
as state or more local level estimates.

So if we drop a lot of trying to maintain purity of the non-metropolitan
versus metropolitan, we can stratify more geographically.

There\'s also, by that time, hopefully, the Census Bureau will be doing
something called the American Community Survey, and there\'s huge
opportunities to take advantage of that. In particular, if we\'re
allowed to share data we might be able to use that resource to
significantly do the sample design.

So that if we pay them a significant amount of money they may come back
and say, these are selected households. We might also be able to pay
them some money to ask some questions in one of their monthly surveys
that are very much energy-related like, do you have central air
conditioning? Which they don\'t plan to do now, but which is a very
important question for energy-wise, and some other energy-related
questions.

So there are some other things we have thought of where the covariates
are much more energy-related, and there\'s some huge potential for
improvements depending on what happens with confidentiality and what the
Census Bureau does.

CHAIR WATKINS: Brenda.

DR. COX: Just as a side issue, Bob, Bob was also looking at using
household level projections and estimates from the Census Bureau, and
that\'s another example of use of covariates.

The results were a little mixed and probably it is suitable to go into
it now but, one of the things that we\'ve been needing for our surveys
\-- quite often we post-stratify to age/race/sex counts for instance,
that we get from the Census Bureau. And that\'s a good way of
accomplishing two things at once: removing some of the effects of
undercoverage and adjusting a little bit more for non-response.

But it also brings comparability across national statistics. If they\'re
all forced to a common set of totals they become more comparable. So
that it\'s kind of a standard to adjust to those.

Now, for years we\'ve lacked equivalent estimates at the household
level, and the Census Bureau has started preparing those. They don\'t
sound \-- from Bob\'s research it doesn\'t sound like they\'re as
current or to the point, necessarily, where we want to be using them.

But it is something that I think is a direction that, let\'s say, it is
a need for the future to have accurate project projections of households
that could be used as a benchmark for all of our national surveys \--
residential surveys.

CHAIR WATKINS: Thank you. Carol.

DR. GOTWAY CRAWFORD: I think it\'s important to make the distinction \--
and Brenda just started to allude to it, that \-- I really see like, if
you will, two types of covariates. She mentioned three of them \-- like
age, race, and sex \-- that people don\'t even think of as being
covariates really, but you always adjust for those in trying to get some
meaningful sense out of your data.

I think some of the covariates that Seymour mentioned are starting to
become borderline. I mean, there are covariates that we would like to
have in order to reduce your variance or for the same variance, reduce
your costs. Where other covariates that are absolutely necessarily if
you want to make correct inferences about the data that you\'ve
collected.

And I think sometimes the distinction between the two is rather blurred.
And so my suggestion is to \-- I just wanted to emphasize the need to
consider incorporating covariates in the analysis, not because they can
reduce costs \-- although that\'s a great reason \-- but because in
fact, some of the other ones that we don\'t normally think of, might
actually be necessary to make correct inference.

CHAIR WATKINS: Any other comments? I want to give just a final thing for
David Bellhouse if he has any concluding comments. Before I do that, are
there any comments from the floor? Yes sir.

MR. FRENCH: I\'m Dwight French. I\'m overseeing the consumption survey
area right now. A couple of other examples of covariate use in our
consumption surveys over the years.

First of all, in 1988 the manufacturing energy consumption survey sample
was \-- there was some concern about the deterioration of it because of
loss of the capability to update the annual survey of manufacturers over
time. And of course, the year following the economic censuses is the
year in which coverage is worse for the ASM relative to the universe of
manufacturing establishments.

And so in that year we used ratio estimation with economic and energy
variates from the ASM and the census of manufacturers to adjust, not
only for the concern we had about the potential undercoverage of ASM,
but also we used it to adjust for the two percent of manufacturing
shipments and employment, and thus we think about two percent of
manufacturing energy use that is not covered even by the mail portion of
the census of manufacturers. That\'s one example in the manufacturing
area.

Gene alluded to something which may well be occurring the next couple of
years in the 1999 CBECS and that is, to save some money on design costs
we may be doing to a much simpler sample design and only updating using
something like FW Dodge listings.

Now, we did this once before, and the problem with FW Dodge listings is
that they\'re good for larger level buildings; they\'re not so good for
smaller level buildings.

DR. HANSER: They\'re awful. I mean, I\'ve used them and unless they\'ve
had a vast improvement in the quality of the data \--

MR. FRENCH: In fact, we had people in yesterday from FW Dodge talking
about this, and I\'m not convinced that there\'s been any major
improvement in FW Dodge listings for smaller building. I don\'t think
\-- you\'re not badly off for buildings above a certain cutoff, and I\'m
not exactly sure where that is. It might be 25,000 square feet,
something like that.

DR. HANSER: Yes, I think you\'d be \-- I mean, you would essentially
miss almost every small, commercial establishment \-- all the Mom and
Pop grocery stores, everything that\'s high consumption at the lower end
of \--

MR. FRENCH: Well, that\'s not necessarily clear because for one thing,
Dodge insists that they cover a lot of the big chains very well, which
are smaller establishments \-- I mean, your McDonald\'s and your Burger
King\'s and things like that \-- which are small but highly intensive
establishments.

The other thing is, we may have to use some sort of model approach to
deal with the small, new buildings this time. Now, what extent is that
of total, commercial energy consumption? It\'s about two percent.

DR. HANSER: Well, that depends on the area. I mean, in California where
I lived at one point in time, new buildings probably represented a much
larger fraction. So if you \--

MR. FRENCH: Well, we\'re not talking about new buildings; we\'re talking
about small, new buildings.

DR. HANSER: Well, it doesn\'t \-- the point is is that, that varies
enormously with where you\'re living, the part of the country, the rate
of growth that\'s going on in establishments. I mean \--

MR. FRENCH: Sure.

DR. HANSER: \-- that\'s not something that\'s fixed for all time. And
Dodge did it. I mean, maybe things have improved enormously, but boy, it
was really rotten. It just missed so much that when you went back to use
it, it was really a problem and I \-- you know, we\'ve done this in
commercial stuff, particularly in the smaller end, was just \-- it was
really just a problem to use.

And I know the experience of utilities who have used the Dodge data
early-on was the same sort of thing. You know, if you were big, if you
were fortunately, you know a part of the big chain \-- so they grabbed
everybody in a chain it was fine. But you know, if you didn\'t fall into
any of those classifications you just weren\'t going to be included.

And so, you know, it\'s a case of, what do you want to do \-- you know,
are the small guys important to you or not? And I don\'t know. You know,
Linda, you went through this years ago.

MS. CARLSON: Oh, Duane went through it too. I mean, it was very painful
to \--

MR. FRENCH: Yes, \'83 was probably the \-- and that\'s the one where we
\-- before once where we used FW Dodge, period, for the update sample.
And we did have problems, and we had problems with the smaller
buildings. I mean, it looked like the new buildings \-- we got the
coverage, we got the number of buildings that you would expect for that
4-year period from the previous survey, which was in 1979. But you
didn\'t get it with a smaller building.

Now the question is, is there some model-based way that you can deal
with that if you can\'t deal with it depending upon the sample cases and
the reported data that you get? And we\'re going to at least make a
consideration for how to do that.

DR. COX: Can you match the Dodge frame to an existing sample you have
that includes small buildings? With good coverage?

MR. FRENCH: It\'s possible that we could for some previous years \--
like 1989 to 1995, something like that. They have available a microfile
that they were talking about with us yesterday. However of course, we
have sample observations; we don\'t have complete coverage. You know, it
would be like matching to our needles and their haystack in a certain
sense.

DR. COX: Yes, but you\'ve got a national estimate and so you could use
your national estimate weighted up to estimate the undercoverage of \--

MR. FRENCH: Oh, sure. The real question is whether you simply model for
small, new buildings using what you know from CBECS in previous surveys.
As was mentioned somewhere earlier here, using the former CBEC sample as
your \"large sample\" and just modeling up using the CBEC\'s data alone,
or whether you take the information that you get from what you can from
new, small buildings from Dodge.

And apply CBEC\'s totals or relationships to Dodge that you can derive
to try to model-base the part of the estimate for small, new buildings.

DR. HANSER: Did you do any resampling in the CBEC stuff? Did you do
resampling where you thought you had live coverage, go back and sort of
see? Because those statistics in the resampling, if you have some, could
be used to create a small model-based approach to the small \--

MR. FRENCH: We did not do that in 1983.

MS. CARLSON: In \'83, but you did use the same sample, and you have
those.

MR. FRENCH: But he\'s talking about an operation where you go back and
you take a sample and you see how many fall in or out of the sample that
you \--

MS. CARLSON: And I\'m saying \-- you had a longitudinal sampling. You
know how many dropped out.

MR. FRENCH: I think it\'s a different issue from what you were referring
to.

CHAIR WATKINS: You have another comment?

MR. LATTA: Yes. I\'m Robert Latta. I sometimes work on CBECS too. We
have maintained the same secondary sampling units, so we have times
where we go out one year and we\'ve updated the listing, so we have
information on how many new listings we get in an area.

Unfortunately, sometimes some of the new listings are older buildings
that we should have got last time. They\'re not necessarily just new
ones. And cleaning that out would be, you know, they could be something
to look at but it\'s not guaranteed to be a totally clean population.

And as for comparing Dodge to RECS, Dodge is really a list of
construction projects, and we get more or less what the final occupants
are, and it might not be an easy thing to match the two.

MR. FRENCH: Interestingly enough, when Dodge was here yesterday they
said beginning in 1998, they are now going to start following their
projects through the construction phase. They are not just stopping at
the permit phase. So we may be able to get some better information from
Dodge for the years 1998 and 1999.

DR. HANSER: Have they \-- at one point in time somebody did a study
which indicated the percentage of the permits that actually led to
eventual construction, and that was a large issue for us because
granting a permit did not guarantee any way, shape, or form, that there
was final construction.

MR. FRENCH: That is true.

DR. HANSER: Has Dodge done anything to update what those rates are over
time?

MR. FRENCH: they have not, to this point. They will know a lot more if
in fact they carry out their plan to follow further down the road than
the permit stage, beginning in 1998. But that isn\'t a coverage
question; that\'s a variance question.

I mean, if you sample Dodge permits and you know 50 percent of them
aren\'t going to lead to anything, it\'s a problem of cost and it\'s a
problem of variance. You know, if you take a Dodge sample.

DR. HANSER: Well, it may not be. I mean, if you knew the projects that
failed to go to complete construction happen to fall into a particular
class, as they might very well, then in fact it\'s not a case of
variance, it\'s a case of bias and that\'s not going to affect just the
variance in the estimate, it will affect the \-- estimates.

MR. FRENCH: Well, except if it happens to be two-thirds in one building
type and one-tenth not going to construction in another, as long as
you\'ve got a sample across the building types you\'re still covered. I
mean, you find that out.

The problem is the buildings that Dodge never covers; not the ones that
they cover but never get built.

CHAIR WATKINS: Any other comments before I turn it over to David
Bellhouse?

DR. BELLHOUSE: I have nothing further of substance to add. A lot has
been added and I certainly appreciate the animated discussion. I think
it\'s been very useful.

CHAIR WATKINS: Okay. Thank you. I have a few administrative matters to
just deal with. Those of us going for dinner this evening, we\'re to
assemble at 6:45 in the lobby here. Who is coming, by the way? Can we
have a show of hands?

The second thing is a reminder for breakfast at 7:15 in the Clark Room,
just next door. And that\'s for an 8 o\'clock departure. Are you meeting
us there or in the lobby?

DR. WEINIG: (inaudible)

CHAIR WATKINS: Okay, so it won\'t be you here; it will be your other
colleagues. And while we\'re on that topic, could I ask for a show of
hands for those of the public left here? Anybody who\'s going over to
the Forrestal for 8 o\'clock?

Anything else, Bill? No? Meeting is closed for the day.

(Whereupon, the Public Meeting of the Committee on Energy Statistics was
adjourned at 4:55 p.m.)