AMERICAN STATISTICAL ASSOCIATION

\- - -

COMMITTEE ON ENERGY STATISTICS

\- - -

MEETING

\- - -

Thursday, April 25, 1996

\- - -

The Committee convened in the Clark Room, Holiday Inn Capitol, 550 C
Street, S.W., Washington, D.C., at 9:00 a.m., Dr. Timothy D. Mount,
Chairman, presiding.

PRESENT:

TIMOTHY D. MOUNT, Chairman

SAMPRIT CHATTERJEE

BRENDA G. COX

JOHN D. GRACE

CALVIN KENT

GRETA M. LJUNG

RICHARD A. LOCKHART

DANIEL A. RELLES

PRESENT (Continued):

BRADLEY O. SKARPNESS

G. CAMPBELL WATKINS

ALSO PRESENT:

RENEE MILLER

YVONNE BISHOP

MARY HUTZLER

JAY HAKES

DOUGLAS HALE

ART HOLLAND

ARTHUR RYPINSKI

LOUISE GUEY-LEE

JOHN CYMBALSKY

ERIN BOEDECKER

JERRY COFFEY

INDER KUNDRA

[C O N T E N T S]{.underline}

[PAGE]{.underline}

Presentation by Jay Hakes 6

Presentation by Yvonne Bishop 34

Presentation by Art Rypinski 43

Presentation by Richard A. Lockhart 61

Presentation by Douglas Hale 84

Presentation by Art Holland 97

Presentation By Timothy D. Mount 112

Presentation by John Cymbalsky 148

Presentation by Erin Boedecker 159

Presentation by Campbell Watkins 168

Presentation by Jerry Coffey 194

Presentation by Inder Kundra 240

Presentation by Greta Ljung 245

[P R O C E E D I N G S]{.underline}

(9:07 a.m.)

CHAIRMAN MOUNT: Let\'s begin this meeting.

My name is Tim Mount from Cornell University.

This meeting is being held under the provision of the Federal Advisory
Committee Act. This is an American Statistical Association, not Energy
Information Administration Commission, which periodically provides
advice to the EIA.

The meeting is open to the public. Public comments are welcome. Time
will be set aside for comments at the end of each session. Written
comments are welcome and can be sent to either ASA or EIA.

Non-EIA attendees should sign the register if they wish to receive a
copy of the meeting highlights.

In commenting, each member of the public is asked to stand, state his or
her name, and speak into the microphone at the podium there. The
transcriber will appreciate it. Also, members at the table need to speak
loudly, Committee members.

We did change the schedule originally, but I think we can go back to the
standard beginning where we introduce members of the Committee, and
hopefully by the time we\'ve done that, our first presenter will be
here.

So let\'s start going around.

MR. HAKES: Jay Hakes, EIA.

MS. BISHOP: Yvonne Bishop, EIA.

MS. MILLER: Renee Miller, EIA.

MS. LJUNG: Greta Ljung, MIT.

MR. RELLES: Dan Relles, RAND.

MR. GRACE: John Grace, Earth Science Associates.

MR. LOCKHART: Richard Lockhart, Simon Fraser University.

MR. WATKINS: Campbell Watkins, LECG.

MS. COX: Brenda Cox, Mathematical Policy Research.

MR. SKARPNESS: Bradley Skarpness, Battelle.

MR. CHATTERJEE: Samprit Chatterjee, the New York University.

CHAIRMAN MOUNT: And members in the public there?

MR. GROSS: John Gross, EIA.

PARTICIPANT: (Inaudible) EIA.

MR. WEINIG: Bill Weinig, EIA.

MR. WOOD: John Wood, EIA.

MR. CRONE: Tom Crone, EIA.

CHAIRMAN MOUNT: Thank you.

What else have I got here? There will be a luncheon for the Committee
and invited guests at 12:15 in the Lewis Room, and breakfast tomorrow
will be in the Lewis Room, Committee members.

So I think that we now turn to an important address from the boss of
EIA, Jay Hakes.

MR. HAKES: Well, first I thought I\'d deal with a question that most of
you are probably most interest in, and that is why your gasoline price
is so high.

(Laughter.)

MR. HAKES: The reason is we have a very cold winter. That put a lot of
pressure on heating oil stocks, which were low anyway. There\'s a lot of
uncertainty in the world market because of Iraq. It\'s not that the Iraq
capacity is needed to create adequate supplies. It\'s that people are
reluctant to bring on supplies if they think Iraq is going to come out
and the price might go way down.

Also there are some refinery problems in California now which are really
having a big impact out there, and also demand is rising very rapidly to
record levels, partly because as people continue to drive more and more,
it\'s not being offset by vehicle efficiency like it had been in the
past.

So the combination of all those things, plus the usual spring run-up you
get every year anyway, has created a pretty volatile situation that will
probably continue for another month or two, but when you average out the
whole year, probably we\'ll still be fairly low by historic standards.

Fortunately I did not write a detailed speech out yesterday because if I
had, I would have had to change it last night with regard to our budget,
but let me start by talking about the electronic revolution, and this is
something we\'ve talked about before.

It is hard to remember that only two years ago this was sort of an idea
in a few people\'s minds. We did not have a real electronic presence out
there, and we now have a Web site that is not fully completed yet, but
is a very usable, user friendly vehicle, and it\'s interesting to watch
the usage trends.

When we started off last summer, we were getting about 100 people per
day coming out of the system. We don\'t measure it by hits, but we
basically measure the number of people, subtract our own employees, and
use that as our metric, and we\'re now up to 800 people a day using and
visiting the Web site. That\'s been a very steady increase month by
month, except for December when the holidays, the weather, government
shutdowns caused the numbers to go down.

But if that trend continues, we see, you know, tremendous usage.

Another metric that\'s a good one for us, I think, is the number of
files that are downloaded off of our system. Last summer that was
running about three to 4,000 files being downloaded a month. It\'s now
up to about 14,000 data files being downloaded a month.

Some of these are people who are converting from hard copy usage to
electronic, but many of them seem to be new customers. There\'s a very
good cross-section of people from industry, government, academia, the
general public. A lot of users still are not ready yet to do things
electronically, but I keep repeating this. A high school student in
Boulder, Colorado, who\'s got a modem has better access to energy
information than any individual in the Forrestal Building had two years
ago. That\'s a major revolution.

In fact, that student can be in a foreign country, as well, because we
do have a fair amount of usage.

Our latest product is a CD-ROM that has now gone through data testing,
and it\'s generally available to the public. It sort of duplicates what
is on the Internet, but the difference is it\'s just a lot faster and
has more features. It has a search feature on it where you can type in
any word or phase, and in 15 seconds it will take you to every page in
our current publications where that word or phrase is mentioned with
that word highlighted.

It\'s really a tremendous asset for a non-energy specialist. For
instance, if you\'re a staffer on the Hill and you deal with energy in
several other substantive areas and your boss asks you to write a speech
or find some information on a specialized energy topic, you can now do
that in a couple of minutes, and there\'s really not too much that you
couldn\'t do using that.

I\'ve often said, too, that this is tremendous. It saves us all of the
embarrassment because I don\'t care who you are. There\'s no one that
knows all the acronyms and the jargon in energy, but who wants to admit
it, you know? So now you\'ve got this CD-ROM, and you whip right through
there, and you can find the definition to anything.

So this electronic revolution has had a major impact. It is a tribute to
a lot of employees at EIA who spent a lot of time on this. It ultimately
has very little incremental cost because you can start to prepare your
documents digitally to begin with, and so having them available to
transfer into this electronic form is important.

And also particularly with the Web, for you economists, it\'s about the
only thing I can think of the government can give to people that
doesn\'t have a marginal cost. Once we provide it 100 people off of our
server, we can provide it to 100 million people and the cost to us is
the same.

That\'s a new way of thinking because everyone wants us to cost out
everything these days, and I\'m trying to convince people that
information is a free good, ought to be treated as such. After all, the
Census Bureau was in the Constitution. So I think the electronic
revolution will help us, and frankly, it\'s a good selling point to
members of Congress because it sort of shows if usage is getting that
high, it starts to show them that people in their district, in their
state will use it.

I can also take the CD-ROM when I\'m showing them and type in cities in
their congressional district, and, boom, they get all of the energy
information about that city if it\'s mentioned, which it usually is if
it\'s a large city.

The second item is quality management. Let me get one prop here. I
don\'t know if Renee put this in the package, but for those of you who
are interested, we can certainly get you one by request.

This is our application for the 1996 energy quality award, and we just
finished this document this week. So it\'s very fresh information, and
basically what it talks about is the things that we\'ve done in terms of
quality management, the development of performance measures, and the use
of cross-cutting teams, how we\'ve addressed the electronic revolution,
the process we\'ve gone through on business reengineering, and I think
that is information you will find useful.

Last year we did such an application. This was the first time the
department had had such an award. They brought in external reviewers,
and we were one of only two organizations in the headquarters of DOE to
receive a quality award. In fact, of the various awards that have been
given on various quality topics at DOE, we have certainly won more than
any other organization in headquarters.

So I think there is a lot of change going on. Part of it has helped
produce the electronic changes that we\'ve seen, and I think there will
be more.

I mean we are in the process of deciding what parts of the business
reengineering proposals will be implemented, and there are a lot of
changes that are coming at EIA. We expect lower resources in the future.
We\'re under mandates to reduce the number of employees, both contractor
and federal. So we\'re having to change the way we do business.

On the personnel front, we are just about meeting our personnel
reductions. We\'re very close. Any week here now we\'ll meet our
personnel reductions for this fiscal year. So all the reductions will be
voluntary this year, which is a much better way to do it.

Now, on the budget front, there are very recent developments that seem
to be positive, but let me see if I can go through the complexity of the
federal budget system.

We are now in fiscal year 1996, which started on October 1 of 1995, and
we still don\'t have an appropriation yet. We have been funded through
this fiscal year by a series of continuing resolutions that provide
temporary funding at lower levels than the appropriated amount.

We do believe, however, that within the next day or two we will receive
an appropriation of approximately \$72 million. This now seems somewhat
high to us because we\'ve been operating under a continuing resolution
at the level of \$65 million. So we will be back up to the \$72 million
level, and life will continue a little bit. People can travel. We\'ve
had some promotions frozen and other things.

But the \$72 million level, we must remember, is a \$13 million cut from
the \$85 million appropriation in 1995. So we have basically undergone a
15 percent cut in our resources.

Well, you say: what difference does that make? The Congress directed
that half of that cut come out of our forecasting operations. This was
done at the very last minute in a small conference committee with no
consultation with us or anyone else that I\'m aware of, and the argument
the committee gave was that they wanted to, quote, protect the data.

So this resulted in a 35 percent cut in the amount of resources
available in the forecasting area and sort of sent some alarm bells out
that we have a job to explain to the Congress what the role of
forecasting and modeling is.

The Congress itself is a heavy user of our models, but what the
difficulty is that the staffers in the Congress understand pretty well
how these models are used, how our data are used, but the actual members
by the time it\'s filtered to them may not realize where it comes from
or how these calculations are made.

And the appropriations process has really changed in the last year or
two. EIA has been very popular with the staff on the Hill, and they have
sort of looked after our budget, but in last year\'s budget cutting
frenzy, the members themselves got more involved in the small budgets,
which they had not done before, and we were competing against the
National Endowment for the Arts, Indian education, other items that
seemed to be more visible in some ways politically, and so we did not
fare well.

But we are also going to be cutting out some data series. We\'re going
to be collecting less detail on electric utilities. We\'ll be collecting
a little more information on independent power producers. Some of our
technical publications, things that are highly technical, appendices and
things like that will be available only electronically. They will not be
available in hard copy.

We are trying to make these cuts in as painless a way as possible, but
it\'s not an easy situation. We are having to cut. We, for instance, had
to cut out our support of the Stanford Energy Modeling Forum, which
we\'ve provided some support to for a number of years, and we will be
publishing probably in the next few weeks a [Federal
Register]{.underline} notice listing the cuts that have been made and
ones that are being contemplated to invite public comment, and we urge
full participation in that because we do want to consult with our users
to make sure that the cuts that we make are the least damaging to the
work that people need to do.

We are consulting closely with Capitol Hill on this budget, and that\'s
one of the reasons I\'m going to be slipping in and out during the next
couple of days, is I do have some meetings over there talking with
people about our 1997 request.

EIA\'s 1997 request is for \$71 million, which would be pretty close to
a continuation of the \$72 million that we apparently now are going to
get for fiscal year \'96. However, we don\'t know whether we\'re going
to actually get that amount appropriated. Sixty-six million of it is
listed as our budget; approximately five million of it is listed in the
efficiency and renewables budget. I think there was an attempt to show
as much support for energy efficiency and renewables as possible. It\'s
the same money. It comes directly passed through to us, so there\'s no
ability of EE to influence this in any way.

But that\'s where we stand. We could get a continuation budget if we\'re
successful. However, looking at the steps that will be required to
balance the federal budget over the next five years and the pressures on
the discretionary spending, in other words, the nonentitlement part of
the budget, there\'s virtually no way that that number can be held for
very long, and we do expect further cuts, and we\'re planning to be able
to deal with those cuts, while at the same time trying to persuade
people of the value of the programs to the economy and to try to
minimize those cuts.

Certainly we do have some opportunities for efficiency, but not enough
to offset the levels of cuts that are being discussed. We\'re basically
looking ahead to 1998 and \'99. On Monday and Tuesday we are spending
two full days in strategic planning basically to try to look ahead to
what kind of organization EIA will be, what should be the appropriate
analysis between data analysis and forecasting. All of those things are
important. Should the balance in the future be more or less what it is
now? Does it need to change in some way?

I think on the data side there are opportunities for automation there
where I think the efficiency can be increased, and even with cuts we can
maintain a lot of the data. Analysis is sort of a very human activity,
less susceptible to automation, and that\'s more difficult.

I do think the model needs to certainly be maintained in a strong way.
It will be a simpler model than we might have contemplated a year or two
ago. We\'ve had to pull back on some of the development work on the
model. We\'ve also pulled back on the PC version of the NEMS, but we are
doing some limited work in that area, but so far at least the model is
still a strong analytic tool and still more used than ever before.

I think the best thing at this point may be, Tim, if it\'s all right, to
take questions.

CHAIRMAN MOUNT: Sure.

MR. HAKES: And see what comments and questions people have.

CHAIRMAN MOUNT: Sure.

MR. GRACE: What do you need to protect the data? You made the comment
that part of the modeling was to protect the data or protect the figures
or something like that. Maybe I misunderstood.

MR. HAKES: Well, so that the cuts would not be as great in the data
area, the data collection area.

MR. GRACE: Oh, I see. Okay.

MR. HAKES: I think some of you may be aware the [Washington
Post]{.underline} actually wrote two editorials last summer about EIA
because they were concerned that the nation might lose some of its
ability to deal with shortages of supply of fuels, and those editorials
particularly emphasized data, and I think some of the people got a
little worried about that.

I would hope that the next editorial from the [Washington
Post]{.underline} might mention forecasting, as well. You know,
obviously there\'s a whole set of issues like the restructuring of the
electricity area, how you might deal with greenhouse gas policies, all
of which will rely very heavily on the NEMS model.

MR. WATKINS: I was interested in your comments on the gasoline price
issue, and if I understood that correctly, one of the problems is the
use of No. 2 fuel oil, and higher requirements for that reduced the
build-up of the inventory because you have all of the requirement slate
on the use of No. 2 fuel oils. Have I got that right?

MR. HAKES: Yes, and secondly, I think, as I understand it, the
refineries undergo some maintenance usually when they\'re switching over
from heating oil to the spring gasoline system, and that maintenance was
delayed a little bit this year because of the colder winter, and so
it\'s the slate within refinery and the maintenance schedule.

MR. WATKINS: The inventory would normally be \--

MR. HAKES: Right. This is a good topic for somebody to do some very good
research on, and we\'re trying to do some. We went through the whole
winter with the inventories for heating oil being below the low point of
the historical range. Now, that\'s an interesting question because a lot
of people in industry will tell you we\'ve gotten more efficient
handling inventories, and they certainly have.

So maybe it\'s healthy to have low inventories because it reduces your
cost.

They also feel they can move product around the world a lot easier. To
some extent they can, but, for instance, California, which is a fairly
isolated market, takes three weeks to go through the Panama Canal.

So what would happen if the whole world had a cold winter? You know,
it\'s not statistically probably that likely, but there\'s an issue
there of whether we\'re going to get back into a more volatile situation
because of the much lower inventories than we have seen historically.

Yes.

MR. CHATTERJEE: You spoke a little about using a simplified forecasting
model. That may not be a bad idea. Sometimes forecasting models do as
effectively as the more complex models.

MR. HAKES: Well, I guess we\'re a little bit conflicting because one
thing you do is you forecast, but you\'re also doing \"what if\"
scenarios for purposes of legislation and other purposes. So the ability
to do \"what if\" scenarios is somewhat diminished with less detail even
if the detail doesn\'t add to or maybe even detracts a little bit from
your accuracy.

We did go through a lot of machinations this year on prices of fuels. We
found over the years that a lot of the projections seemed to hold up
pretty well in terms of patterns of consumption and production, but
price has been a variable, and we, like many forecasters, have been on
the high side compared to what\'s happened in actuality, and we\'ve
lowered very substantially this year our projections for natural gas,
and I think that was a good thing to do.

Yes.

MR. GRACE: Along the same lines, with the increasing support you\'ve
received by use of your Web site, and I\'ve been one of the people
who\'s been on there \--

MR. HAKES: Good.

MR. GRACE: \-- is the decision to pull back on the PC NEMS perhaps kind
of contrary to that philosophy, that if you had greater accessibility to
the models via a platform that was more popularly available, that you
might be able to build the same sort of support on the modeling side
that you\'re now building on the data side through the electronic access
afforded through your Web site?

MR. HAKES: Yeah, I mean, that\'s a good point. I think, you know, when
you talk a 35 percent cut, you sort of get a little desperate, and you
sort of cut what you can.

You know, ultimately, too, to the extent that our biggest customer is
the Congress, they probably are many years away from having the ability
to operate even a PC NEMS. We\'ll have to run it for them. So in terms
of our key customer that\'s going to make the appropriations decision,
it probably wouldn\'t make a huge difference.

I hope we can get that back on track because I think it is part of the
vision we have of really sort of allowing people to get in there and use
the stuff themselves, you know. With the smaller number of people that
we have that\'s almost a necessity.

Now, you know, two years ago we were on a mainframe. We\'re now down to
RISC work stations, and the PC thing would be nice.

Yeah, Dan.

MR. RELLES: Fifteen percent is a lot, and presumably there should be a
lot more pain to describe than you\'ve described here, especially since
I think any staffer or any Congressman, if he doesn\'t hear that
there\'s a lot of difficulty this has caused, will say, \"Well, okay.
I\'ll do 15 percent next year until we hear about it.\"

(Laughter.)

MR. RELLES: Now, presumably the voluntary retirements are not going to
be able to absorb another 15 percent, but what about the users? Are many
of them complaining about the reduced volume or quality or availability
of data?

MR. HAKES: We don\'t play the Washington Monument game like many people
do in town, that the first thing we\'ll cut is the most valuable thing
we do, and I think we\'ve done a good job of becoming more efficient.

I think there has been a lot of pain internally, but I think we\'ve
shielded a lot of our customers from it. You know, in the government
shutdown, we did not shut down at any point because we had carryover
money.

The next set of things are going to be very tough. I mean I think things
like the voluntary reporting on greenhouse gas, which is one of Art\'s
programs, is going to be tough. We\'re going to do like oil reserves
every other year instead of each year in terms of the detailed analysis.
Our consumption studies are going to probably be in four-year cycles
rather than three-year cycles. That in and of itself saves sort of a bit
of money. Now, is that enough to get people jumping up and down and
screaming? Probably not at this point, but, you know, if you, say,
reduce your sample size so you can\'t do regional analysis, you know,
again, is the average person in the street going to jump up and down and
say, \"I want regional analysis\"? Probably not. People like you might
yell and scream, and that might be a good thing.

I guess my preference would be on like, say, consumption studies is to
do them less frequently to preserve some detail and some regionality,
you know. So that\'s one way to save.

In a sense, the next five million is almost tougher than the first 15
million because it\'s just the nature of things and how rapidly you have
to change, but it\'s certainly tough.

I mean, in the modeling area we had some projects going, say, on the
international natural gas model, refinery model that we had spent some
money on over the years that we had to stop in the middle of the work
and suspend the project, and that\'s very irrational because you\'re
wasting money you\'ve already spent.

But when the Congress sort of targets the cuts that way, you know, we
don\'t really have any choice at that point.

MR. RELLES: Yeah. Do you think electronic media are going to be a big
money saver for you in the future by not having it published in paper?
You\'ll be able to save a fair amount?

MR. HAKES: Well, we spend about \$1 million a year on paper. So there is
definitely some money to be saved there. In a budget our size, that\'s
significant, but I think what it allows us to do is talk about expanding
our customer base without increasing cost, and so it saves some money.

Now, of course, automation is not just dissemination. It may be
collection; it may be processing. There\'s a lot of things that can be
automated. So the savings of automation and computers, in general,
probably is much greater than just the cost of saving paper and
dissemination.

But, you know, to the extent we can do that, we can absorb some cuts.
What we\'re trying to do is set aside money each year, too, for
investing in this technology. You know, we\'re trying to think, well,
where do we need to be in three years. What kind of training do we need
to get to that point? What kind of technologies do we need?

We\'re going to try to standardize a lot more the software that\'s being
used so that our offices communicate more easily with each other.
Maintenance costs go down. We project that getting totally off the
mainframe will probably save us about \$2 million, net savings.

So, you know, there are a lot of things that can be done, and you know,
we plan to do those anyway. So if we can keep continuation for a year or
two to sort of catch up from last year, I think we can maintain a very
high level of service.

MR. GRACE: I do a lot of work with the U.S. Geological Survey, and in
the last few years they have started to go a direction \-- and I don\'t
even know what the acronym stands for \-- creating CRADAs. CRADAs are
essentially allowing the Survey to take private sector money as income
and provide something like proprietary services, but not exactly. It\'s
somewhere in between, but it\'s a new source of income for them.

And especially when I think about, for instance, the reduction in the
rate from one year to two of collecting data on oil and gas reserves, is
there a possibility within the legislative confines of the EIA to set up
partnerships with the private sector to augment your budget in a way
that would allow the provision of services that might have a specific
client base out there that\'s willing to pay?

MR. HAKES: Well, I think it\'s worth discussing. I mean we have CRADAs
at the Department of Energy in a lot of our technical development areas
where there are partnerships with industry. It\'s very interesting.
Those CRADAs are under great attack in the Congress right now.

MR. GRACE: Are they really?

MR. HAKES: They\'re viewed as, quote, a subsidy to business. You know,
the current rhetoric and ideology is very interesting and how things get
interpreted.

I must say that there\'s a side of me that\'s much more comfortable
making the public good argument and continuing to ask for taxpayer
dollars than to get into the issue of proprietary information. You know,
Statistics Canada, for instance, copyrights their information. That does
create a different ball game.

I personally would not like to do that because if you look at the
history of energy in this country, one of the big issues in the \'70s
was the public didn\'t trust industry data. It\'s been very beneficial
to the industry, as well as to the environmental community, consumers,
and the government to now have some data that\'s viewed as being
objective. I mean I don\'t think anybody really questions our
objectivity, and it\'s now so assumed that people don\'t talk about it
anymore.

But there is in this country a suspicion of the oil industry, for
instance, which, you know, whether it\'s justified or not, it\'s sort of
there. So if you start saying, \"Okay. Well, we\'ve got these
partnerships and they\'re giving us some money,\" and all of that, I\'m
not saying we couldn\'t handle it and I\'m not sort of taking it off the
table, but I feel more comfortable arguing for \-- you know, the
government has something. I think the government has the responsibility
to provide the base information, and it\'s very interesting because when
the World Bank goes into a lot of the developing nations, they are
urging them to set up and EIA type organization, and so we are visited
frequently by people from other governments around the world saying,
well, how do we do this.

The reason is, one, the investors will invest more in a high information
system. You know, low information equates to risk. Risk relates to
higher rates of return, more reluctance to invest.

The other thing the World Bank feels is that public officials will make
better decisions about energy if they have data to deal with. Those are
all public reasons to have this kind of data. It would be sort of ironic
if in spreading our data out, helping other countries set up this system
that we ourselves sort of lost it.

But that\'s a question I get a lot. The Paper Work Reduction Act, of
course, which is part of the framework under which all of the
statistical agencies operate, says that our basic system is to charge
only for the dissemination of the information. It costs so much to print
a document. That\'s what the cost of the publication is, not our
collection cost.

And so to move away from that would be a pretty major change, and I have
done some fairly superficial look at Canada, New Zealand, and Australia,
and some other countries that have tried to move more into the cost
recovery phase, and I\'m not sure that they really recover costs at the
level that it\'s perceived that they do. I think, you know, we have
several hundred thousand dollars a year that goes into the federal
treasury from the sale of our publications. So we at that point recover
some cost, but I think information because it has such low marginal
cost, it\'s very difficult to get it to pay for itself.

Well, we thank you all. This year is a strange year because I think we
have stability on the Committee. I don\'t think there are any new
members to announce, but it\'s good to see you all back again, and I\'m
looking forward to the exchanges on this.

These critiques that you have done have been very helpful to us, and we
really appreciate the help that you\'re giving us.

CHAIRMAN MOUNT: Thank you, Jay.

I certainly am glad to hear you saying that this is an important public
good that you provide. You know, I really do agree with that, and I
think it\'s truly impressive to see the number of people and the new
types of people that are being able to get access to data on energy now
through the Internet.

So I think the next item is Yvonne Bishop to tell us how the EIA
responded to the last meeting.

MS. BISHOP: Well, he said a lot of the things I was going to say.

(Laughter.)

MS. BISHOP: On the last meeting Perry Lindstrom spoke about the
renewables in the AEO. I had a great big package from him responding to
the comments. I think I\'ll give it to the persons who made the comments
rather than repeat them all here.

The difference in the forecasts for renewables in \'93, \'94, and \'95
was queried, and he points out that although the differences are large
relative to the specific market, they\'re small relative to the total
energy picture, and the reasons for the differences include not only a
dynamic industry, but changes in the model as well.

And ethanol and biomass are not really renewable. They are agreed that,
say, they would like to use this broad definition because these are
treated the same way in a policy perspective.

The side cases taken as a whole give a likely range of renewable
penetration rather than a one point estimate. They feel that what
they\'re really trying to do is give plausible ranges for what might
happen in the future.

Doman and Peabody spoke on the international energy outlook. They much
appreciated Campbell Watkins\' comments. They tried to include as many
of them as they can. They\'ve discontinued the use of sensitivity ranges
and replaced them with two economic group scenarios, and they have
described the method by which the scenarios were developed.

The 1996 outlook, which will be here any moment, includes comparisons
with the previous year\'s projections and has \-- which is what was
suggested \-- and has added a lot of new features. They now include
exports from the former Soviet Union, China\'s nuclear expansion,
improved estimates of energy intensities, and there\'s a better
electricity demand model, and they\'re planning on including India in
the whole story.

The other issues concerned largely data collection and dissemination in
a changing environment. As Jay mentioned, we will address some of the
issues raised in the strategic planning next week, and it was suggested
a new mission statement. We may get around to that.

We need to define what is the core data, and we need to consider the
issues of integrating service.

In terms of how much we learn from our customers, we\'ve again had a
small survey of telephone customers with much the same results as last
time. We\'ve mailed questionnaires and included cards in publications
for various groups of readers.

For the Weekly Petroleum Status Group report, of the 200 who responded,
70 percent have computer capability, and nearly all of that 70 percent
said they don\'t want paper anymore. So we took them off the mailing
list.

(Laughter.)

MS. BISHOP: We have had many favorable comments spontaneously on the Web
site, and we are currently discussing development of a questionnaire for
electronic users to get their opinions of the strengths and weaknesses
of the current offerings.

The Information Products and Services Committee has begun to analyze
Internet accession. I got a different number than you. I\'ve got 90,000
a month, but anyhow, it\'s growing very rapidly, and \--

MR. HAKES: That\'s probably kids.

MS. BISHOP: No, sessions, and now totals 400,000. Anyway, whichever
number is right, it\'s getting bigger.

They can determine by looking at these the type of user, which files
were accessed, the number and order of the files accessed, and the
number of return visits. So this will give some indication of what\'s
popular.

We were planning to present this analysis in Chicago, but have not been
able to make any firm travel plans.

(Laughter.)

MS. BISHOP: On business reengineering, the executive sponsors reviewed
the team\'s plans and implementation suggestions. As of Tuesday, they
came out with a summary of their decisions, and they proposed some pilot
efforts and some reallocation of resources. No doubt details will be
available later.

Meanwhile some of the business reengineering enthusiasts have been
thinking about the issues, and two of them will be talking about the
more statistical issues today. Nancy Leach will be talking on
performance statistics for data process, and Renee Miller on the
important, but unresolved, issue of how we measure quality.

The issues arose because we recognize that if you\'re trying to improve
the efficiency of data processing, you\'d better have some measures
about how efficient you are, and the question of quality arose because
we had goal of speeding up how quickly we got the information out, and
we were anxious that this would not be a cost of the quality of the
data.

In general, many staff recognized that reduced resources implies a need
for training for those who take on more or different responsibilities.
Howard Magnus has been around to all of the offices and collected the
perceived needs for training and is now trying to arrange seminars and
workshops on these topics.

I have a list of the topics, and I thought that if I handed out this
list of the topics, maybe the Committee would help by suggesting persons
who might help us by leading a seminar or workshop in these areas.

We\'ve been doing some of it in house. Doug Hale has been organizing a
training seminar on electricity pricing based on Schweppe\'s book on
electricity spot prices, and Ruey-Pyng Lu has been conducting
introductory statistics courses for those who felt they needed that.

And I think that\'s all that we have today in terms of follow-up. I\'ll
hand those things out now.

MR. RELLES: You mentioned telephone surveys and the Internet in the same
sentence, and it actually made me think of an experience that I had
recently, which I must say was rather annoying, but it might be of use
to discuss it. I was on the Internet trying to get some stock market
information from Scudder, and before it would let me get into some of
the more interesting stuff beyond its home page, it asked me a whole
bunch of questions, and I couldn\'t get any further without answering
some questions.

(Laughter.)

MR. RELLES: Now, I mean \--

MS. BISHOP: That\'s pretty annoying.

MR. RELLES: But actually \--

PARTICIPANT: Did a broker call?

MR. RELLES: But actually I didn\'t mind it all that much because I
really did want to get it, and I figured, well, that\'s the cost of my
getting information, and it was phrased in a neutral way. You need a
password to get in there. So, okay, so I\'m going to make up a password,
and then it asks you your name and where you live and a few things like
that, but that might be a device that the EIA could consider using to
try to survey some of its customers, especially looking for ways to cut
costs. Instead of making phone surveys all the time, you\'ve got 14,000
accesses. If I\'m asked to answer some questions once and that registers
me with you and you know a lot about me, that could very well be a
useful device for collecting that kind of data.

I know it goes against the grain that information should be free because
I now have to spend some of my time answering your questions, but on the
other hand, given the value of the information, I think a lot of people
would be willing to do it.

MS. BISHOP: The telephones \-- sorry.

MR. HAKES: I think I can respond specifically to that. On the Web site
there is an icon called \"feedback,\" where we encourage people to do
it, and we do get some data.

Now, the Department of Agriculture has a system that\'s similar to what
you\'re describing, that you basically are giving it, and it sort of
hits you up front when you\'re identified as a first-time user, and
we\'re going to take a look at that.

I do think that you have to have a very easy bypass for it because a
citizen does have the right to come into that anonymously, I believe,
and I think you would have legal problems if you required them to fill
out a form, but I think we could do a better job of capturing that
automatically and maybe purging people more than the current system
does.

MR. RELLES: Yeah, I actually saw your attempt to request information,
and I bypassed that very quickly.

(Laughter.)

MR. HAKES: We appreciate your concern about respondent\'s burden.

(Laughter.)

MR. HAKES: You of all people.

MR. RELLES: Hey, I\'m counted as one of your users.

MS. BISHOP: Still the telephone service is for the people who call in on
the telephone. I should have made that clear. We still get thousands of
phone calls asking for information, and at the time they apparently say,
\"Would you mind if we phone you back?\"

MR. HAKES: The problem is a lot of our phone users are playing the
commodities market, and you can tell when you\'re interviewing them that
their mind is elsewhere, and they don\'t want to be distracted for too
long.

CHAIRMAN MOUNT: So we move to the first presentation of the results of
the greenhouse gas voluntary reporting program by Art Rypinski, Office
of Integrated Analysis and Forecasting.

MR. RYPINSKI: Okay. Good. We have the opening slide.

The greenhouse gases voluntary reporting survey is regrettably not a
household word. So I thought that it would be useful to provide you with
some background so you can figure out what this curious animal is.

Can I have the next slide, please?

We made a presentation to the ASA a couple of years ago when we were
first developing this program that stimulated a very lively
conversation, including Mr. Coffey of the Office of Management and
Budget, who argued, if I recall correctly, that if this was not a
statistical survey, that OMB if presented with it was under, of course,
no obligation to approve it.

So I just thought I would come back now and talk a little bit about now
that we\'ve completed our first reporting cycle what this program is,
what we think we\'ve learned, and you know, where we\'re going from
here.

The first thing to note about this program is that it\'s a statutory
program. It\'s specifically required by Section 1605(b) of the Energy
Policy Act, which, no, I don\'t expect you to remember that particular
section of what is, after all, a telephone book size document, but what
Section 1605(b) does is it requires that the Department of Energy will,
in cooperation with the Environmental Protection Agency, provide a
mechanism by which entities \-- and that\'s the word in the law,
\"entities\" \-- may report, and this is a quote, \"annual reductions of
greenhouse gas emissions and carbon fixation achieved through any
measures, including fuel switching, forest management practices, tree
planting, use of renewable energy, manufacture or use of vehicles with
reduced greenhouse gas emissions, appliance efficiency, methane
recovery, cogeneration, chloroflurocarbon capture and replacement, and
power plant heat rate improvement,\" and so other things, too, \"and an
aggregate calculation of greenhouse gas emissions by each reporting
entity.\"

The Energy Information Administration is required to develop forms for
voluntary reporting under these guidelines developed by the Department
of Energy, make the forms available to anyone who wishes them, and put
the data received in a database. So that\'s what we\'ve been doing.

The law was passed in October of 1992. The Department of Energy issued
its guidelines in October of 1994. In November of 1994, we submitted
draft forms for public comment. We submitted final forms to the Office
of Management and Budget for review under the Paper Work Reduction Act
in February \'95. They were cleared in May, the end of May. We spent six
weeks with the printer and putzing around with necessary changes, and
released them to the public in July of 1995.

The first reporting cycle purportedly closed on October 31st, but in
practice, the actual reports drifted in mostly after the deadline.

The forms development process was quite lengthy, as you would expect
from something that involved all of these items that I listed, and
essentially the problem we faced was that the Department of Energy,
faced with the question of building a structure for this program,
answered the question: who can report? Anyone. And what can they report?
Anything.

So whenever the department came to a binary choice, \"we can do it
either this way or that way,\" they always said, \"Let\'s do it both
ways. Make it possible to do it both ways.\"

And given this range of activities, we wound up with a fairly
complicated form. We worked closely with potential reporters. We worked
with the Policy Office of DOE. We worked with the Environmental
Protection Agency. We went out, and we developed the forms. We
pre-tested them with a number of companies, including Niagara Mohawk,
Houston Light and Power, General Motors, New England Electric System,
and that was extremely helpful.

We wound up with two kinds of forms. There\'s a long form, which is on
the order of 40 pages. It\'s like a 1040. It\'s divided into sections.
So no one reporter has to report all of it. You report the pieces you
need to report to cover your particular problem.

It comes in four schedules. The first schedule asks you what you want,
who you are, what\'s your name, what\'s your address, what is your
quest.

The second schedule covers people who would like to report projects,
which are actions which we define following the law as actions causing
reductions of emissions of greenhouse gases, and we have ten project
types of capture specific information for each project type. Most people
don\'t use very many project types.

And then we have a Schedule 3, which is the aggregate emissions and
reductions of the reporting entity.

As we were developing the forms, and as the guidelines were being
developed, in April 1993, President Clinton announced that he was
committing the United States to reduce its emissions of greenhouse gases
to the 1990 level by the year 2000, and he prepared, \"he,\" the
administration, prepared a climate action plan, which describes the
mechanisms by which the administration expects to use to get to that
objective.

Many of those mechanisms are voluntary programs which encourage firms
and individuals in the private sector to do things that reduce their
emissions of greenhouse gases.

We have, since this program was already in train in the statutory
program, we\'ve reached arrangements with many of these voluntary
programs that they will use the 1605 program as their reporting
mechanism. So it becomes a means by which third parties can assess the
success of these voluntary reporting programs.

We also have two kinds of forms, a long form and a short form. The short
form we thought of as being for individuals and households. It\'s two
pages, front and back, and then we have the 40-page long form.

We developed an electronic form, which is a visual Basic application,
which was widely used.

Next slide, please.

This is a list of our 108 reporters.

MR. WATKINS: Including you?

MR. RYPINSKI: Including my name. I am a reporter, and I can assure you
that nothing was more educational about the form than having to fill it
out myself.

(Laughter.)

MR. HAKES: How much did you save?

MR. RYPINSKI: How much did I save? Five tons. I\'m sorry. I admitted
five tons. I saved about \-- no, I saved five tons. That\'s right.
Sorry.

Next slide. I\'ll characterize those. Yeah, good.

As I said, we had 108 entities. Most of them, 96 of them, were electric
utilities. Most utilities were participants in a DOE voluntary program
for reducing emissions of greenhouse gases called Climate Challenge.
Even the 96 utilities is kind of an understatement of our coverage in
the electric utility industry because many of the utility reporters
reported as holding companies. For example, Southern Company, Cinergy,
flipping back, all of those guys with the \"ergy\" names turn out to own
multiple regulated utilities.

We had many investor-owned utilities. We had municipal utilities. We had
G&T co-ops. I think that the evidence is that we got somewhere between a
half and a third of the emissions of the electric utility sector.

At the project level, we received information on 645 individual
projects. One submission from GPU, which is a big utility up in
Pennsylvania, was 500 pages. So when we say 108 reporters, that doesn\'t
really do justice to the volume of stuff we got.

Non-utility participation was rather modest. There were three
manufacturers, General Motors, IBM, and Johnson & Johnson; two coal
companies, including Peabody Holding, which is the largest coal company
in the United States; two aluminum companies that were reporting on
perfluorocarbon reductions, a landfill methane form; two forestry
groups, and two households, including as Dr. Watkins pointed out myself.

Next slide, please.

Okay. How did they report? Basically about two-thirds of them used the
long form. About one-third used the EZ form.

What we discovered is that we thought of the short form as being a
mechanism for households and voluntary groups. It turned out that it
sorted out that it actually was a mechanism for voluntary burden
reduction for people who wanted to report, but didn\'t want to put in a
lot of efforts.

So one of our short form reporters, for example, was Pacific Gas and
Electric, which is a large firm, and one of our long form reporters was
yours truly, who\'s a household.

(Laughter.)

MR. RYPINSKI: So among the long form reporters, most of them \-- it
actually divides up a little more than half \-- some half, two-thirds
reported on the missions of their entire corporation, and about
one-third reported just on project, which are individual actions.

Nearly everybody who filed the long or actually not nearly everybody;
about two-thirds or three-quarters of the people who filed the long form
used the electronic form. Most people who used the short form used
paper. It\'s like the burden associated with flipping through those
diskettes and firing up the software approximated the burden of filling
out the form itself.

Next slide, please.

As you would expect, for a process that turned out to be dominated by
utilities on the project side, most of the projects were \-- that\'s
interesting. That\'s a different graph than I had with my \-- never
mind. Don\'t worry about it. No, no, no, it\'s right. It\'s just earlier
data. That\'s embarrassing. The results kind of changed.

Most of the projects were energy end use and power generation and
transmission. We got a lot of forestry projects, carbon sequestration
projects, 76 on that slide, and then there was what for me were the most
interesting projects, though not the most frequent, is distribution of
other activities, including particularly a lot of landfill methane
capture projects, gas pipeline methane capture transport projects, and
halocarbon reduction projects.

We also had a tenth category in our forms for \"other,\" which was for
projects we hadn\'t thought of, and it turned out we got quite a few
\"other\" projects. The most frequent \"other\" project was coal ash
recycling by electric utilities, which was something I didn\'t know
about, and basically coal ash substitutes for limestone in cement
manufacture, but doesn\'t need to be calcite. So it reduces the CFC
emissions.

By gas, the number of projects and reductions reported were dominated by
carbon dioxide and carbon sequestration, but we got a fair number of
other gases.

Let\'s see. Next slide please.

MR. WATKINS: I have one question.

MR. RYPINSKI: Yes, sir.

MR. WATKINS: Gen. tran. is generation and transmission \--

MR. RYPINSKI: Yes.

MR. WATKINS: \-- of electricity?

MR. RYPINSKI: Of electricity, yes.

MR. WATKINS: Okay. But the transport is the transportation of natural
gas mainly or \--

MR. RYPINSKI: No, it\'s motor vehicles essentially. Motor vehicles,
yeah. The pipeline projects fall under coal gas CH~4~.

As you can see, I did not submit this slide to OSS for approval, and I
was regrettably constrained on space or ingenuity or some combination
thereof.

We had 40 entity reports, which is people reporting the emissions of
their entire firm. Those emissions covered about 1.2 billion tons of
carbon dioxide. That\'s about 20 percent of the U.S. total, and included
most of the largest emitting firms in the United States.

The biggest single emitter turned out to be General Motors. We will come
to this in a minute. They claimed responsibility for the emissions of
their entire fleet of vehicles on the road.

(Laughter.)

MR. RYPINSKI: Which came to some 350 million tons, and they noted that
because the average MPG of the vehicles they\'re putting out on the road
is steadily improving, that this number has come way down from 1990.

Interestingly, they didn\'t claim formally a reduction. They just wanted
to make this point, which they did.

Most of the other \--

MR. WATKINS: They didn\'t take people who drive?

MR. RYPINSKI: Well, General Motors claims that if you\'re driving a GM
car, even if you\'re driving more, you\'re emitting less. I shouldn\'t
comment on that.

Most of the entities who reported used what we called in 1605 speak a
modified reference case. It means that emissions were higher than they
were in 1990, but lower than they would have been had they not
undertaken a number of good deeds. The most common good deed, the big
number good deed was improving the availability of nuclear power plants.

Twelve companies reported reductions on what we call, once again in 1605
speak, a basic reference case, which says that emissions were lower than
1990, and these firms were actually concentrated in New England, Niagara
Mohawk, New England Electric System, Public Service Electricity and Gas,
which is in New Jersey, and Northeast Utilities.

Most projects were associated and most of the reports were associated
with these firms participating in various voluntary action plans
associated with the President\'s Climate Change Action Plan. I noticed
the largest ton reductions, and at the entity level I think we got about
60 million tons of carbon dioxide in aggregate claimed reduction, though
I would be most loath to add those numbers up because of the differences
in definition \-- yes, I see. I\'ll move along swiftly.

From TVA, Duke Power, and Niagara Mohawk improving availability of their
nukes.

Next slide, please. That\'s been there before.

We learned an awful lot. The biggest single accounting issue was how do
you treat wholesale electricity transactions. Electric utilities tended
to claim emissions. It\'s very interesting. Consumers of electricity
tended to feel that they were responsible for their electricity
consumption, and electric utilities tended to feel that they were
responsible for the emissions arising from providing electricity to
their customers.

But then the question of who\'s responsible for wholesale electricity
transactions. Different reporters came down all over the map on that
one. Some said, \"We\'re responsible only for our plants, period.\" Some
said, \"We\'re responsible for our plants, plus our purchases.\" Some
said, \"We\'re responsible for our plants, plus our purchases net of
sales,\" and there are all of these various arguments.

There was a lot of discussion during the design of this program about
would people gain in the system, and what we learned is that these
reports were so complicated that with a couple of exceptions, people
didn\'t gain the system. They worked hard. They had enough trouble doing
the first report without doing five to figure out which one made them
look best.

Also, reporters tended to be very conservative in their claims and in
their accounting. They tended to follow the view \"if in doubt, leave it
out.\" If we\'re not sure about our claim, we won\'t make it. So for the
most part, a lot of the gaining problems turned out to be nonexistent.
They turned out to be self-enforcing.

There was a lot of discussion during the design about fuel cycle
effects. Reporters that they didn\'t know about fuel cycle effects.

We discovered that reporters required substantial assistance with
calculating their emissions of greenhouse gases, even if they were very
sophisticated reporters. It turns out that operationally, when we were
designing the system, we had this notion that these reports would be
filed by electric utility planning offices which would have
sophisticated generation planning models to run. What we found out was
for the most part they were filed by environmental or environmental
compliance offices, and one of the problems internally was getting the
data out of their operating arms and understanding it before they could
report it to us.

The concept of emissions for individual projects turned out to be very
problematic in the database. If you have a power plant whose heat rate
you improve so that it emits less, in effect, it affects the emissions
of every plant on your system because the improved heat rate means that
it jumps up in the merit order, and other plants go down. People did all
sort of things to capture this concept of emissions, and they weren\'t
very comparable across companies or projects.

I have a couple more slides, but I think I\'ll just do one more. What we
were planning to do next were the dates here, which are already out of
date as we spiral out of control here.

Data validation took far longer than we anticipated in large part \--
partly because the reports straggled in, partly because the reports were
far more complicated than we had imagined in our worst nightmares, and
partly because, as I alluded to earlier, the people needed a lot more
help than we anticipated.

We\'re working on a database, including a public use database, so that
we can distribute this information to the public. This slide says April.
That was what I thought in March. Now I think May.

We\'re in the throes of writing, which is what I have to go back to when
I finish here, a report on the data. I think that was due to the
administrator a couple of days ago. We\'re still struggling with it, and
that will probably be out in June now.

We\'ll be releasing the 1995 forms in June, and then the second
reporting season will close in October.

I guess that\'s where I\'ll end, and let my discussant, whom I terribly
skipped in sending him materials in advance, for which I apologize, have
a shot.

Dr. Lockhart.

MR. LOCKHART: I\'m being introduced now, right?

I have only two transparencies and only a very few remarks. I have been
struggling since 1993, when Art Andersen made a presentation to this
Committee, to understand what this survey is for, and I was pretty
critical of the idea in 1993, and I\'m still critical of the basic
concept.

So I put up some questions and answers.

Is a voluntary survey a good idea? I don\'t think so, and I haven\'t
changed my mind, except that I was impressed on this slide. I put this
caveat at the bottom: \"except when the outcome is nearly a census.\" So
if everyone fills it out, then the fact that it\'s voluntary, the
non-response bias is presumably negligible, and I gather that we have a
very high rate of return from the utilities. So at least in some
sections we\'re getting a reasonable supply of information.

Is EIA doing a good job of analyzing the pitfalls of a voluntary survey?
I think the answer seems to be yes. That last question, there were two
households that reported. That\'s the other household.

MR. RYPINSKI: No, the other household was Carter Lewis.

MR. LOCKHART: Oh, was it?

MR. RYPINSKI: Yes, Moorhead P.S. is \-- I\'m sorry. Moorhead Public
Service is a little muni. utility.

MR. LOCKHART: All right. I got it wrong. I was looking to try to figure
out who the other household one was.

In 1993, we were warned in advance there would be baseline problems,
that is, they couldn\'t decide whether to do this modified reference
case or basic reference case kind of baseline, and I see both have been
done, and that does seem to be a problem.

Government budget cuts, which are relative to imaginary future budget
increases, are not entirely \-- not universally seen as credible, and
that may be ‑‑ I just don\'t know whether it will be seen as credible
here either.

We were warned about other problems with few graphic scope, that is,
people reporting emission reductions that didn\'t really occur inside
the country because they were forestry projects that were happening
outside the country.

We were warned about problems with forestry and land use baselines, and
we were warned about this problem of many people taking credit, and the
example of GM taking the blame but also the credit for all of the
improvement in their fleet mileage performance so that we\'d get from
individuals the same claim that they\'ve made reductions if they bought
more fuel efficient cars. You can\'t count that kind of thing twice.

My last two questions on the slide are connected with the \"is a
voluntary survey a good idea.\" I don\'t think that they\'re generally a
good idea because I don\'t see how to make use of the data in any
further activity. You don\'t want to add it up. So I\'m wondering who
the users of such a database will be.

I\'m concerned that the answer is politicians with an ax to grind and a
point to make. I\'m wondering what possible use can be made of the data.

That\'s really all I have to say, but I do want to congratulate you on
what I think is a very good job of looking at the pitfalls. I have the
idea of reading or hearing the speech and reading the documentation
that, in fact, the EIA\'s job here is to certify individuals as having,
yes, achieved a reduction rather than maintaining a database, which is
sort of what I\'m, as a statistician, used to commenting on.

CHAIRMAN MOUNT: Some comments from the Committee? Bradley.

MR. SKARPNESS: It would have been useful if we could have had one of
these long forms or short forms just to see exactly, you know.

MR. RYPINSKI: I\'m sorry. I didn\'t know to even bring them.

MR. SKARPNESS: Yeah, I know, but if I could have seen them.

MR. RYPINSKI: I can get you one by lunchtime.

MR. SKARPNESS: I may fill it out or look at it.

(Laughter.)

MR. SKARPNESS: The other thing is that I used to work for TVA, and I see
that they have this largest tonnage reduction claim when they get their
nuclear reactors running, and when I was there, there was the potential
for that, but of the four reactors, none of them were running most of
the time, and so I\'m wondering. There is a claim there, but what is
actually happening?

In other words, when you have one of these humongous reactors on line,
you can really shut down a lot of the coal facilities, but when they
aren\'t on line, they\'re running gung-ho, and you\'re not getting the
savings that you potentially could. So it\'s sort of like a disconnect
of what actually is the reductions compared to what the claims are. So
that\'s where I think there is maybe a little bit of a disconnect
that\'s related to, you know, what is actually the data telling us.

Thank you.

CHAIRMAN MOUNT: Greta.

MS. LJUNG: Yeah, I think that it\'s a good idea if you gave everybody on
the Committee a form and we all filled it out. It would increase your
response rate by 1,000 percent.

(Laughter.)

MS. LJUNG: You said you have two individuals reporting?

MR. RYPINSKI: Yes, that\'s right.

MS. LJUNG: That would increase it 1,000 percent.

CHAIRMAN MOUNT: Brenda.

MS. COX: Oh, in Arthur\'s defense, I have to say we have seen a form
before, but I don\'t remember what it was. I thought it was just last
year. Yeah, the program was presented before.

I really think that this issue was bagged last time, and it was bagged
this time, and I kind of understand why, but I really think you should
grapple with the difficult issue of why in the world is this study being
done. I mean it\'s not to accomplish a statistical objective. I hope it
isn\'t because as a statistical objective goes, you\'re going to have
trouble defending it, other than maybe just to gain some information
that might allow you to do a valid study later.

In other words, you are gaining information, et cetera. The last time
that we spoke, I just assumed that Congress was trying to encourage
widespread voluntary efforts to reduce emissions, and maybe that\'s the
purpose, but I think you really need to grapple with it because it
certainly in these days of budget cutting and not being able to do
things that we know are valuable and important, you need to say why is
this being done.

The data deficiencies are obvious, not just the voluntary ones. You
really don\'t have a target population definition, and you don\'t have
definitions for the units that report, and they\'re vastly different. So
as a statistical database, you just can\'t do anything with it other
than kind of discuss its attributes the way you did.

I\'m actually impressed, by the way. You did a good job of a difficult
task, but you come back to why is this being done and should it continue
being done.

CHAIRMAN MOUNT: Samprit.

MR. CHATTERJEE: I think one reason, I think, which Brenda said, is
probably right. It\'s a consciousness raising act, you know, across the
industries as a whole. I think the information gathered may not be just
applicable directly to get an estimate, but I think it gives you the
gross aggregate figure in which we might be able to see the direction
and trend in which this is going and also to get some kind of very broad
estimate. You know, we\'re not talking about standard errors or anything
of that, but at least a ball park figure.

I think that\'s one thing which this particular form might be useful.
Another thing is basically, as you say, making people more aware of
generating greenhouse gases and some attempts to reduce them.

MR. LOCKHART: I would like to ask Arthur explicitly whether you think
that the EIA\'s role is principally asserted by that reductions have
been made.

MR. RYPINSKI: I do not. In fact, I go back to the language of the
statute which mercifully I didn\'t quote in full, but it says persons
reporting under this subsection shall certify the accuracy of the
information reported. EIA does not certify the accuracy of the
information reported, and you\'ll notice that some of my apparently
casual language was, in fact, exquisitely phrased. I said \"claims,\"
that General Motors \"claims\" responsibility for most of the vehicles
in the United States.

Read my lips.

(Laughter.)

MR. LOCKHART: But let me turn to the your function, the function of the
agency in collecting the data. Let me try it a different way. Is the
establishment of guidelines which will make, provided they are followed,
the individual claims more credible, that is, your guidelines carefully
prepared enable the companies that report to assert that they have made
calculations in a sensible way so that they can claim more credibility
having received reductions?

MR. RYPINSKI: It is my hope that the design of the forms enhances, does
things that enhance the credibility or, at the minimum, the transparency
of the information reported. That was one of our objectives, to make it
clear, as clear as possible, what was reported and what people were
claiming.

We do not assert that these claims, as an agency, that these claims are
accurate, and with the various accounting definitional issues, I don\'t
think the problem is people are claiming things that are not true. I
mean, you know, there\'s a statute that says, \"You shalt not make false
statements on a government form,\" and it has criminal penalties. It
doesn\'t matter what the government form is, and that was never my sense
that the reporters were making statements that were not true.

What does happen is that there is no definition of who owns emissions,
and there\'s nothing resembling a set of generally accepted emissions
accounting principles that firms can adopt and produce reports that are
consistent even to the degree of, say, financial statements, and we, in
fact, know, those of us who have worked with financial statements, know
that the comparability of financial statements in detail is a little
tricky despite all of the effort expended on it.

So the real problems are not that the claims aren\'t true. It\'s that
what one company claims, another observer might not \-- one observer
might find it credible under one theory of accounting, and another
observer might find it unbelievable under a different theory of
accounting.

Our mission is to make it clear what it is they claim so that people can
form judgments on those questions.

CHAIRMAN MOUNT: Cal, welcome.

MR. KENT: Thank you.

CHAIRMAN MOUNT: So if you want to joint in?

MR. KENT: Just to make a comment, and that is I find this discussion
amusing in that it almost takes me back four years ago when we were
talking about this particular piece of legislation, and I think if we
weren\'t on tape I could be more explicit about the discussions that
went on about this particular \"why are we doing it,\" and basically
this is one of the things that the \"why are we doing it\" is because
this was a compromise between the people who wanted a certification
program so that we could give out gold stars or green trees or whatever
the badge was going to be; those that wanted to use it for regulatory
purposes to say, \"You\'ve got to do it by so much\"; and those that
wanted to be able to claim that there were certain reductions as a
fallout of this particular piece and be able to point that this
particular piece of legislation had had so much positive impact on the
environment so that in international conferences, and so forth, we
could, rather than lying directly like the Europeans do, that we would
be able to have a statistical shroud that we could drape ourselves in
when we made our outrageous claims for how effective we were becoming.

So that was basically why it came into being. The problem is that every
one of the issues that you guys have just raised was discussed in
detail, and basically what the writers and the promoters and the pushers
on both sides of the aisle of this legislation said is, \"We don\'t
care. This is at least something we can point to,\" and the result is it
will be used for mischief. I can assure you it will be used for
mischief.

And when it is used for mischief, I can assure you as in the past EIA
will be blamed.

CHAIRMAN MOUNT: Campbell.

MR. WATKINS: It seems to me a key here is going to be how are you going
to write up and present this material. I saw on the issue paper that you
have, in fact, a report. Have you given any thought to how you\'re going
to or do you have any insights on how you\'re going to present the data?

MR. RYPINSKI: Yeah, actually there are a number of people here who are
anxiously awaiting that report: my bosses.

We propose to in the report \-- basically the report will probably
follow the tone and the spirit that will follow very much the way that I
have presented it, which is to play it straight and say this is what
people are claiming, and to discuss the problems as fully as we can.

MR. WATKINS: But wouldn\'t you want to go a bit beyond that and make
some kind of at least qualitative comments on what it all means.

MR. RYPINSKI: Well \--

MR. WATKINS: Otherwise I think it will be a bit misleading to say,
\"Well, here is what people claim.\" I think you would have to alert the
public to what the problems are with this sort of exercise.

MR. RYPINSKI: Well, I expect that we will, that that is one of the
things that is an objective in the report. As always, preparing these
documents requires a bit of art.

MS. LJUNG: I like Samprit\'s idea of the awareness, or making people
aware, but the problem seems to be that nobody is aware of the survey.
Two households responding, obviously the people probably don\'t know
about it.

CHAIRMAN MOUNT: Jay?

MR. HAKES: I\'d like to make a couple of comments.

One is to put this in context, I think you\'d have to go back to the
Clean Air Amendments of 1990 where we have regulated sulfur emissions
seemingly quite successfully at much less cost than was anticipated and
pretty sizable reductions.

The system that they used was a capping system where you say there\'s
such-and-such limit on sulfur emissions, and people trade emissions.
That has worked well enough that it\'s likely to be the model for any
further emissions. So you would have a system where carbon emissions are
capped and then traded back and forth.

One of the problems with that system is the issue if the baseline is the
point at which the legislation is passed, there is no incentive to
reduce emissions before that point because you don\'t get credit. In
fact, the people who voluntarily reduce emissions are disadvantaged
because they\'ve already taken the easy steps.

So contemplating that there is a chance that the Sulfur Act would be a
model for greenhouse gas reductions, the major emitters would like some
system to allow them to claim credit, and while the methodology is a
little \"loosey-goosey\" at this point, it is transparent as could
possibly be made so that if one decided how one was going to do this,
you could make adjustments within the systems.

I think the value of this system \-- it\'s an uncomfortable situation
for EIA to do this in some ways because it\'s not statistics work as we
know it. On the other hand, I think it\'s work that EIA is probably
better prepared to do than anybody else is.

One of the things I would urge you to look at this as is a definitional
effort. I mean before you measure something, you have to define it, and
if you look at the last two years and our knowledge of what greenhouse
emissions are, how the rank of one emittant compares to another, what
measures are used to mitigate them, we know a lot more now than we did
two years ago.

So that if you\'re going to measure it in the future in a way that is
statistical, at least we have some definitions on which to base those,
and Congress, if they\'re going to pass a law, would have at least some
definitions and some information on which to base that law. Most of the
environmental regulation in this country has been based on pretty good
guesses, you know, educated guesses about things and hoping that it
would work out if there is some chance that there is legislation in this
area, that that would be worked out.

I would also say that the nuclear capacity is going up very rapidly. We
just issued a report within the last two weeks, and it\'s gone up very
year in recent years and is a fairly major driver in the electric
system.

The other is I think this electronic reporting system that Art has
developed can be a model for us of how we ought to be user friendly in
data collection and other areas, but it is not true that Art is being
considered to head the IRS.

(Laughter.)

CHAIRMAN MOUNT: Well, thank you.

So I think that that \-- have you got one more? Sorry. I didn\'t see.
You wanted to address \--

MR. RYPINSKI: I wanted to address very briefly \--

CHAIRMAN MOUNT: We are running a little bit over time. So \--

MR. RYPINSKI: Okay. I\'ll bang through this.

Some of the comments. First, on geographic scope, we got a small number
of overseas forestry projects. Nearly everything that was reported was
domestic. The overseas forestry projects actually were quite
interesting, but by no means did they dominate the landscape.

Forestry and land use turned out to be for the most part that we saw in
the forms through careful forms, and the forestry numbers turned out not
to be that large, in part, because the forms were designed carefully to
prevent people from summing 150 years of savings into the first year.

The next question is what is this for. There are a couple of useful
purposes that I would suggest to you that I think are important. The
first one is that we have today a situation in which the President has
made a major national commitment, which is to stabilize emissions of
greenhouse gases, and he has defined a mechanism, which is a set of
voluntary programs. If people are interested in assessing the
effectiveness of that mechanism and linking back the claims made for
voluntary programs to what companies are actually doing, this database
is an extraordinary, even a unique way of doing that.

So I think that this will ultimately make it possible to assess the
voluntary programs and point a realistic way.

And the second thing is that in it there is an array of all sorts of
interesting information about what I would characterize as unusual
projects: landfill methane reduction, perfluorocarbon emission
reduction, fly ash recycling. And I think there\'s a lot for us to
learn. Certainly there are things to be learned for the national
inventory in the database. There are things to be learned in there, in
fact, about very low cost approaches to controlling emissions of
greenhouse gases in areas where emissions are uncontrolled because
there\'s never been a reason to do it.

So I think there\'s a significant social interest in that now. With that
I\'ll shut up.

CHAIRMAN MOUNT: So some of us are going into caffeine deficit. One brief
comment from Bradley.

MR. SKARPNESS: Brief. You do aggregate these numbers, okay, and as a
consequence, I think that\'s where we have a problem because there\'s no
definitions or there\'s no standard way of calculating these reductions.

MR. RYPINSKI: That chart is pretty good because it\'s number of
projects.

MR. SKARPNESS: Yeah, but this is what people are going to use. Once you
aggregate them, then they\'re going to start using these.

Is there a way \-- I mean in nonattainment areas, there are air
monitoring data being collected. Maybe if you could compare, you know,
aggregate some of those numbers where it\'s done somewhat independently,
you know, with using air monitors, compare trends there that have gone
over the years with what\'s going on, what\'s being claimed here. That
might give some credence to these numbers. I don\'t know.

MR. RYPINSKI: Well, I don\'t think the problem is the numbers that are
claimed are untrue.

MR. SKARPNESS: No.

MR. RYPINSKI: If there\'s an issue, it\'s not going to be that the
monitor says X and the reporter claims Y. The problem is going to be
that the circle the reporter draws around his project will include
things that some people will think should have been out or in.

MS. GUEY-LEE: Can I ask \--

CHAIRMAN MOUNT: Can you speak at the microphone, please?

MS. GUEY-LEE: I\'m sorry.

CHAIRMAN MOUNT: And state your name.

MS. GUEY-LEE: My name is Louise Lee. I worked with Arthur on the
project.

I think that when you look at the reporting, what you saw, we know what
the aggregate statistics in the U.S. is, and it supports what was
reported. A lot of our reports were about improvements in nuclear
availability. We know from the national data that that was a trend in
that period.

So I guess if I characterized this, the reports, you know, you
shouldn\'t think that they were not credible. They were reported within
the rules. For example, General Motors was a manufacturer. They invested
in improving the process of developing cars that were more efficient,
and within the rules of the game, they claim they reduced indirect
emissions. So now that sounds a little better when you look at it that
way. They were within the rules.

Does that help?

MR. SKARPNESS: Well, nobody\'s claiming that they\'re not doing it in a
legal, you know, way, but there is a truth out there that as a
statistician we\'re after that truth.

MS. GUEY-LEE: Okay.

MR. SKARPNESS: Okay? And so that\'s \-- well, we don\'t see the
disconnect between what the numbers are and maybe how they relate to,
quote, unquote, the truth, and that\'s \--

MS. GUEY-LEE: Well, one of the ways that we know we can improve is to
make more efficient appliances, more efficient cars. GM did that.

MR. SKARPNESS: I agree with you.

MS. GUEY-LEE: And they reported it.

MR. SKARPNESS: Okay.

CHAIRMAN MOUNT: So I think that this discussion should continue over
coffee.

(Laughter.)

CHAIRMAN MOUNT: Let\'s try and compromise and have a ten-minute break
because we\'ve got an important topic on restructuring the electric
utility industry after the break.

(Whereupon, a short recess was taken.)

CHAIRMAN MOUNT: So this session on restructuring of the electric power
industry has two papers. The first one, \"An Analysis Agenda for a
Restructured Industry,\" by Doug Hale, Office of Statistical Standards.

MR. GRACE: Tim, I understand there\'s a little problem of hearing in the
back from people making comments at the table.

CHAIRMAN MOUNT: So, members of the Committee, you have to speak into the
microphones. You\'re not doing a good job, and you will be graded on the
second session.

(Laughter.)

CHAIRMAN MOUNT: Those that pass will be allowed to go to dinner, which
is the other things that I should have announced, that we plan to have a
dinner for the Committee at Le Rivage, the usual location, at 6:30, but
we need to have a number count of how many people will be able to go,
and I think the best thing is to decide by the lunchtime and make sure
you tell Tracy.

Okay. So now you\'re on, Doug.

MR. HALE: All right. Thanks, Tim.

My name is Doug Hale. I\'m the author of the paper on electric power
industry restructuring. The world is changing all around us. Last week
they started dancing at Baylor University.

(Laughter.)

PARTICIPANT: They didn\'t when I was there.

MR. HALE: Yesterday the FERC announced new open access rules for the
entire U.S. domestic or U.S. electricity industry. Unless things have
changed for the worse, the far worse, we all know where the dancing is
going to lead up. We don\'t know where the electric pricing is going to
end up, and so the Administrator has given me an opportunity to think
about this for a while with some of my colleagues at EIA, and I want to
present kind of where we are in our thinking.

I must say that there is a huge debate about how this is all going to
shake out. On the one hand, you have people like Paul Joskow claiming in
different ways that the benefits from restructuring and price
competition may, in fact, be minimal.

Joseph Stiglitz yesterday at the announcement said that we got enormous
economic benefits from the deregulation of the telecommunications and
natural gas industries. This is the first comprehensive step in a
similar transformation of the electric industry.

First slide.

What I\'ve tried to do in this paper is present a conceptual framework
for thinking about the restructuring of the industry. I\'ve described
EIA\'s current projects that relate to electric power restructuring, and
I\'ve made some suggestions about a prospective analysis agenda.

This paper has been widely circulated, including to the Department of
Justice, Federal Trade Commission, and others, for the purpose of
getting suggestions about alternative approaches and different
suggestions for projects. I would certainly like your ideas in that
area, but more to the Committee, I\'m concerned with learning about
ongoing work in the area.

I was at a Department of Justice-FTC closed seminar Tuesday dealing with
the merger-acquisition guidelines and their application, and in the
course of that, they had invited people from Enron, Berkeley, several of
the private consulting firms to present research that\'s not been
published, and it\'s very clear to me that there\'s a lot of work going
on that we will have to take advantage of if we\'re going to have an
effective analysis program in this area.

I\'m particularly interested if the Committee has access or knowledge of
ongoing empirical work dealing with such things as nodal pricing,
distribution of nodal prices, approximations to nodal pricing,
elasticities under demand \-- of demand under spot pricing, anything you
may have on transmission economics, and the metering controlled costs
associated with something closer to real time or closer to nodal
pricing.

Most of the economic analysis of this industry is based upon network
models, and we at EIA have not had much experience with them. So we\'re
very interested in any information you have about small scale network
models applicable to the electric power industry that we might use, if
nothing else, as learning devices at this stage.

The Justice Department is also extraordinarily interested in them, and
they have already suggested informally joint projects with us.

Finally, any information you might have on ongoing work to quantify the
price quantity impacts of restructuring in the U.S. markets would be
very much appreciated. We\'re learning more and more about what went
right and what went wrong in the U.K., excellent empirical papers in the
[Journal of Political Economy]{.underline} and other places talking
about the number of competitors, that there are too few. There isn\'t
anything quite ready yet in the U.K., even though there have been a lot
of preliminary pricing experiments in the last few years.

The next slide presents the organization of the paper. The paper is kind
of long, so let me just run you through it.

The first four sections basically deal with the conceptual framework,
the first question being: why do we regulate? Why are we stopping
regulation of the usual sort?

The second deals with the benefits of restructuring. What is there? Are
the benefits real or are they not?

We then look very briefly at three archetypical restructuring proposals.
How do we get from here to there is the issue. Even if there are
benefits from competition, how do we move from the state we\'re in to
this better world?

Finally, an analysis issue is: is there a there there? Again, it\'s not
at all clear. How much of a change there\'s going to be depends upon
empirical facts. It depends upon the equilibrium outcome of this
evolution, and also our ability to quantify it is very much limited by
the data we have.

The final part of the paper deals with the analysis agenda, again, the
first phase being what we\'re doing now and the latter phases of a
prospective nature.

I want to speak just a second about benefits because it\'s the benefits
that set the parameters on how big the changes might be.

Do you want to put that up?

If you look at the literature, most of the benefits are said to come
from five areas. First are improvements in operating efficiency and the
allocation of system-wide resources, including the closing of uneconomic
facilities.

The second thing, which can\'t be over-emphasized, in my opinion, is
that by moving to something closer to nodal pricing or marginal cost
pricing or flexible pricing, you are now able to make beneficial trades
that simply were not possible under average cost pricing. That changes
everything. Okay. That changes when and how the plants \-- you know,
when demands appear, how plants are run, what emissions are like.

The third thing is an asset reevaluation at market values. Utilities are
valued by the market, those publicly owned utilities, but you cannot get
a value for individual facilities apart from book values. The market
will make it very clear as to the values attributable to transmission
versus generation versus better control, better coordination, all sorts
of information features that are simply lost now. It may lead to new
products, new ways of delivering and contracting for electricity.

And finally, the restructuring is said to lead to appropriate signals
for long-term investment, perhaps more investment in storage capacity,
perhaps more investment in incremental changes to upgrade our ability to
transmit power.

I don\'t want to go into the analysis issues as such. They\'re long;
they\'re tedious; and they\'re written in \"economese,\" I guess, but I
do want to mention a couple of things on the empirical analysis issues.

What I\'m talking about here is how big are the potential benefits. How
much might we get out of restructuring? And that\'s why we look at these
sets of questions.

So the question then is: are the utilities actually minimizing the cost
of the services they provide? Are they operating the proper facilities
in the proper sorts of ways?

If, in fact, the great benefit from competition is prices will get
closer to either nodal or marginal costs, then how much do those prices
vary during the day over seasons, by location? If they don\'t vary, then
there\'s less payoff.

Is demand really elastic over the distribution of nodal prices? Can you
really reduce demands 15, 20, 30, 40 percent for a few hours and then
have them met at other times when there\'s plenty of capacity? If you
can\'t, the restructuring benefits are lessened.

Even if there are large benefits, and most people, I think, believe
there are, what are the actual magnitudes of the metering, the
communication, control, and contracting costs associated with moving to
the new world? Are they large enough to swamp the potential benefits?

And finally, what are the opportunities and costs of changing
transmission capability? The FERC strategy for making the industry
competitive is based upon transportation, getting competition through
transportation. Whether or not and how or under what terms you can
improve your transportation system is key to how effective this is going
to be.

Right now, EIA is running four projects, three of which made it on this
graph, one of which didn\'t. So let me start with the important project
that was inadvertently left out.

The first one deals with the status of FERC and PUC regulations with the
status of new entrants into the industry and takes a preliminary look at
transportation capabilities, of opportunities for expanding
transportation capabilities. This is ongoing now in the CNEAF.

The second project, which is prospective and has just been started in a
preliminary way, deals with the efficiency of individual generating
stations. This is an attempt to use NERC data to look at what used to be
called the X efficiency of the operations of plants. How well that
project succeeds depends upon the data. Right now we don\'t know.

And finally, there are two modeling projects. The first, which Art is
going to talk to you about a lot, deals with simulating marginal costs,
pricing, plus adjustments to marginal costs under a regime where you\'re
assuming elasticities and a lot of other things you would like to know
instead of having to just assume.

And another project deals with the effect of removing regulatory
constraints on how our models would view investment over time.

I think these projects, on the whole, are going to be very informative
and successful. I\'m quite excited about Art\'s in particular, but I
think what we\'re going to realize after they\'re done is that our
database is still a bit thin, and so I think for prospective projects
the first thing we\'re going to want to do is get a better handle on how
much the prices vary and how much transition metering costs might really
amount to.

I think we\'re going to find out that with these networks, the costs and
effects of alternative ways to increase transmission capacity is at
least as important, if not far more important, than generation capacity
at least for the next ten or 15 years.

I think also we\'re going to discover in a very profound way how
difficult it is to estimate demands with the information we have
available. There\'s never been anything like the spot market or see
anything approaching a spot market for electricity in this country.
Ergo, there is no data, at least local data, or very little.

The other thing we\'re going to have to start worrying about is scenario
construction. By the time these projects are done, we will have had
basically three years under a slowly moving, perhaps slowly moving
change to a restructured environment. We should be in a better position
to talk about where this is all going to end up, what the equilibrium
outcomes might be like, but right now it\'s too early, but maybe then.
In order to do our analysis, we\'re going to have to make some guesses.

I think finally we\'ll have to do what we\'re going to have to do at
every juncture in this project, and that is to reassess where we are.
This is clearly a one step at a time learning process.

I just want to speculate a little bit as to what would come next, and I
think in this Phase 3 we would start emphasizing doing serious demand
estimates. Right now, as I say, we are just making up elasticities. In
the foreseeable future I think we will be able to sort through which
assumptions are sensible and which are not, but we\'re still far, far
from making serious attempts to estimate elasticities in the context of
this sort of new market.

I think we\'re also going to have to face the network issues head on. I
am proposing that we build three, a couple perhaps toy network models
maybe representing California, the Northeast and the Midwest to try to
get some feel for how much difference the network effects make for the
forecasts and the analysis and the analytical results we get using our
standard tools.

If those effects are large, I suggest then we adjust our models for
these network effects or try to. That may not be possible. In the
distant future, perhaps we will, you know, replace what we have now with
a more network based approach to electric prices. I don\'t know. My
guess is I will have retired by the time we get to that. So we will find
out.

I\'d like to return to my questions one more time. This is an incredible
area of analysis and activity. EIA cannot do it all, should not do it
all. We have to find targets of opportunity where we can add particular
value to the ongoing debates, and one place to start in making those
sortings out is to get a better idea of the work that\'s going on.

So once again, anything you all can do in your own particular knowledge
working in the research organizations to inform us of what\'s going on
would be a great help.

And with that, thank you very much.

CHAIRMAN MOUNT: Thanks a lot, Doug.

We go to the second paper, \"Forecasting Electricity Prices in a
Competitive Environment,\" Art Holland, Office of Integrated Analysis
and Forecasting.

MR. HOLLAND: Good morning. I\'m Art Holland. I appreciate the
opportunity to speak with you this morning.

I\'ll be describing the method that we\'re developing in the Office of
Integrated Analysis and Forecasting to model the price of electricity
generation services under competition.

We\'re up to Number 2, Doug.

I\'ve been asked to review the questions for the reviewers, and they
are: what analysis, data or modeling products do decision-makers in
industry and government need as they face the uncertainty of the future
electric power industry? How can we best use the integrated nature of
NEMS to serve them?

We are in the process of developing a model of a quasi-spot market
pricing mechanism. Are 108 pricing periods enough to simulate a spot
market?

And the next step is to develop a contract pricing mechanism, or maybe
to develop a contract pricing mechanism. Should the contract price be
the average spot market price plus some insurance premium? Should this
premium be based upon the volatility of the spot prices, assumptions
regarding risk aversion and the avoidance of transaction costs? Are
there other factors that we should consider?

Now, the restructuring of the electric power industry is not just an
electricity issue. Fortunately the Office of Integrated Analysis and
Forecasting is able to use NEMS for this analysis effort, and the
strength of NEMS is in its unique systems integration feature.

Now, let me point out before I go any further that what I\'m going to
describe to you today is not integrated into NEMS yet. It\'s in the test
phase. We\'re doing it off line, and the purpose of this phase is to
determine how to integrate it into the NEMS framework, what pieces to
use, and how to hook up all the wires.

On the left in this diagram of NEMS, you\'ll see the supply components;
on the right, the demand components; and in the center, the conversion
components and the system integration piece. What NEMS allows us to do
is gain insights into how changes in the electric power industry will
affect other U.S. energy markets, like coal and natural gas.

So with that, I\'ll try to describe something about what we\'re doing
and how we\'re calculating these competitive prices.

Now, there are at least two, and maybe three, components. Again, this is
the test phase, and keep in mind that this is generation services only.
Transmission and distribution are very important, but we\'re going to
save those for later.

The first component is the energy component. Now, the energy component
is based on the short run operating cost of the last plant dispatched.
There are 108 dispatching periods per year in each region in NEMS.

Now, some people believe that in an over-capacity situation like we are
in right now, this is the only determinant of prices. Now, we anticipate
that most of the times that will be true, but what that means in our
algorithm is the reliability component, which is next, and that may not
be the best name for it, and I\'ll talk a little more about that, will
be small at least on an average annual basis until this over-capacity
situation is relieved.

Now, the reliability component may better be called the market clearing
component. I have to give Doug credit for that name, and it\'s based on
two general calculations or values. First is the value that consumers
place on reliability, reliability specifically of generating supply, and
the second thing is the reduction in unserved energy that\'s contributed
by each kilowatt of generating capacity in the region.

Let me repeat that and talk about it a little bit because it\'s a
slippery concept if you\'re new to it. That\'s the reduction in unserved
energy contributed by each kilowatt of generating capacity. Now,
unserved energy, think of that as a bad thing. That is the extent to
which demand exceeds supply, and anyone that knows anything about
electric power knows that that can\'t happen. So this is an expected
value that we\'re calculating in the model.

Now, one of the questions that I\'m asked about this component is: how
will it be integrated into the prices that consumers see? There are two
possibilities that I can envision.

First would be an independent system operator in that type of a world
enforcing some system reserve requirements.

Another way that it may get in there would be through contracting
mechanisms where consumers purchase greater levels of reliability and,
therefore, pay a higher price for their power on a per kilowatt hour
basis.

The third component \-- I\'m going to talk more about the reliability or
market clearing component before we\'re done \-- the insurance and
convenient component, if that\'s used, will be for end use consumers who
enter into service contracts to avoid the risks and transaction costs at
the spot market. Now, I don\'t know that we\'re going to do this.

One of the purposes of the current test phase is we\'re going to look at
the results we get and then make a determination of this convenience
insurance component.

Now, the first component that I talked about, the energy component of
price, is on the overhead. Now, I know you can\'t see the numbers, but I
wanted to show you this to give you an idea of the range of costs that
we\'re looking at that are translating into prices, and the costs are
color coded.

Now, there are 108 rows in this graph. Those are the 108 dispatching
periods per year in NEMS. Now, each of those dispatching periods is
characterized so that we\'re trying to capture the entire year in load
characteristics that exist on a time basis throughout the year.

The columns, there are 13 columns. These are the 13 regions that the
model uses for its solutions.

Wherever you see white on the chart is where the cost or the short-run
operating costs of the last plant dispatched, which is the price setting
plant for this component, is less than two cents per kilowatt hour.
Where you see yellow, the short-run operating cost of the last plant
dispatched is between two and four cents a kilowatt hour, and everywhere
you see red, it\'s greater than four cents a kilowatt hour.

This tells us that we\'re in the ball park of expectations. These are
the kinds of numbers that we expected to see for the last plant
dispatched to meet load throughout the year.

Doug.

Now, the more contentious piece. The reliability or market clearing
component has three steps basically. There are four up here, but one or
two are done in the model simultaneously.

The first step is to calculate unserved energy for the region for each
of those 108 time slices for which plants are dispatched in the model.
Now, again, unserved energy is not a good thing. That\'s the difference
between supply and demand when demand exceeds supply.

I\'ve been asked why would demand ever exceed supply, and of course, it
can\'t. The idea is that it will not exceed supply if it\'s priced to
reflect shortages properly, and that\'s what we\'re endeavoring to do
with this component of price, and that\'s why market clearing component
might be a better term. It raises the price of electricity when demand
approaches the capacity limits in order to communicate through prices
that capacity shortage.

The unserved energy is calculated based on the capacity of each
generating plant in the region, expected planned enforced outage rates
for each generating plant, and the hourly loads for the region.

Then what we do is we increment capacity a small amount, maybe ten
megawatts, and do it again. This shows us the change in unserved energy
for a change in generating capacity, and the result of those two
calculations gives you the reduction in unserved energy that you get for
each megawatt of capacity in the region, and the assumption is that
every megawatt of capacity that\'s available contributes to the same
degree in that reduction of unserved energy.

If you take those numbers, take the total capacity in the region and an
assumed value of unserved energy or, on the other side, the cost of
unserved energy, multiply those together, and divide by sales, it gives
you the reliability or market clearing component of price.

I think some sensitivities, and to show you some preliminary results,
might help to clear that up a little bit. Now, this was in the early
stages of the model. We didn\'t have the 108 time pieces. All we had is
an average annual number, and as I said, in the current situation of
over-capacity, we\'re going in assuming that that reliability or market
clearing component will be low because demand should very infrequently
approach your capacity limits in the region.

Now, this is New England, and I selected New England because if you look
in the far right-hand column, you\'ll see that with the original data on
an average annual basis, the reliability component of price we were
getting was three and a half cents per kilowatt hour, which is way too
high. That\'s out of the ball park, and something was wrong with that.

So we did some sensitivities. The first thing we did is we doubled the
number of generating plants in the region and halved the capacity of
each, which means you have the same capacity, but it\'s broken up more
finely. That should lower your calculations of unserved energy, which
means an increase in capacity will have a smaller effect on that
unserved energy number, which means your reliability component of price
or your market clearing component of price should be lower when you do
that, and sure enough, it dropped from three and a half to 2.2 cents a
kilowatt hour, but that\'s still too high.

Then we started looking at some of the data that was feeding into the
model, and we saw that in the original data, the availability rates that
were assumed for nuclear power plants were 65 percent. That\'s way too
low. It\'s more reasonable to plug in a number like 80 percent. So
that\'s what we did, and bang, that number dropped from three and a half
cents to .7 cents a kilowatt hour. So clearly the model is very
sensitive to the availability of your large base load plants, which it
should be.

Oh, let\'s skip the next one, Doug.

Now, what I\'d like to do is show you some of the early results. We\'ve
just started getting some what I think are credible results out of this
model this week. So this is very early. We\'ve got a lot more analysis
of the results to do. So please don\'t run out and quote me on these,
and that\'s why I\'m using 1995, so that we can\'t call them forecasts,
and we also know something about the industry in 1995, and we don\'t
know what\'s going to happen later.

Now, the assumptions in these results you\'re seeing are the value of
unserved energy is \$12 a kilowatt hour, and assumed elasticities of
demand are minus .15.

Now, I have two graphs which are lined up here. The top graph \-- and
the reason why they\'re lined up is so you can see how supply and demand
relationships, which are on the top, translate to prices, which are in
the bottom graph.

The top line, the dark blue line, are your seasonal capacity numbers.
Those are adjusted for your maintenance schedule and for your firm
trades. Under that, the lighter kelly green are your hourly demands.
Now, keep in mind that the peaks that you see in demand \-- and those
are broken up, as you can see, under that by season \-- we don\'t know
when those peaks are going to occur. They could occur any time during
the month, but we want to get them in there so that they\'re represented
in the model.

Under that, you\'ll see the red line or the red spikes are the
reliability or market clearing component of price. You\'ll notice that
those line up whenever your green demand lines start getting very close
to your blue capacity lines.

Under that \-- and these, by the way, the three lines on the bottom are
stacked. So the top red line shows you the spot price of electricity at
that point or what we\'re calculating.

The green line is the energy component of price based on the short-run
operating costs of the last plant dispatched, and you\'ll see that\'s
fairly flat in ERCOT. I should have mentioned this is Texas, for those
of you who don\'t know the North American Electric Reliability Council
Regions.

The bottom line is your transmission and distribution component of
price, blue. Now, as I mentioned earlier, we\'re just using the NEMS
numbers for that, which for distribution and transmission means that
that\'s the average imbedded cost. So it\'s going to be flat throughout
the year.

Now, if you take those green, red, and blue lines and you average them
out through the year, your T&D component, which is the same as in the
regulated cases, 1.2 cents a kilowatt hour; your energy component is two
cents a kilowatt hour; and the reliability or market clearing component
averaged throughout the year is .1 cents. That comes out to 3.4 cents
per kilowatt hour. This is a very early number, and there are still some
things that we need to figure out how to get in there, but that does
compare to a 1995 price of six cents a kilowatt hour for this same
region coming out of the cost of service regulation method that\'s in
NEMS now. So we are getting big drops in the price of electricity using
this method.

Another reason I wanted to show you, again, very early, preliminary
results that probably still have some problems with them, this is for
the MAAC region. Now, MAAC, for those of you who aren\'t familiar with
NERC, is Pennsylvania, New Jersey, Delaware, Maryland region. The lines
are the same, seasonal capacity, under that the demands. But you\'ll
notice the difference here is that we\'re getting a rougher, a higher
volatility energy component of price. So the reliability or market
clearing component of price isn\'t kicking in as much. That energy
component is enough to drive the demands down away from the capacity
limit. So we\'re not getting that red piece in there as strongly.

And, again, a price comparison, regulated cost of service was eight
cents in NEMS in \'95. This totals to 4.6 cents.

And, finally, no discussion of electricity restructuring would be
complete without looking at California, and I don\'t know if we should
get any insights from this or not, but I was struck with how flat and
how low the price of electricity in California is from this graph.
There\'s very little volatility in the price, which suggests they have
certainly in excess of capacity there, and they\'re able to use their
cheaper generating plants to meet their demands.

Again, a comparison. I was able to pull nine cents per kilowatt hour out
of the [Electric Power Monthly]{.underline}. That\'s all of California.
This region isn\'t exactly California. There are some slight geographic
differences, and there may be some modeling inconsistencies, but 9.6 is
what came out of NEMS. So you\'ve got nine to nine and a half cents as a
regulated price, and these totaled to 4.7 cents per kilowatt hour.

Doug, you might want to put the questions back up.

Again, thank you very much for the opportunity to speak with you this
morning, and I look forward to your comments.

CHAIRMAN MOUNT: Well, there wasn\'t a rush from the Committee to be a
discussant for these papers. So I hope there are going to be some very
insightful comments after my rather bland remarks are over.

I think it\'s fair to say that it\'s much too early to be able to answer
the questions that have been posed. However, I think that the overall
approach to this problem is correct. Basically people are going to be
interested in prices and demand. Are prices really going to go down? Who
benefits? What\'s going to happen to sales?

In many parts of the country with high reserve margins, the need for new
capacity is not immediate, and the problems about investment incentives
then can be looked up as a sort of next issue that ought to be
addressed.

So I have four issues that are mentioned, but I think are more important
maybe than they are in the paper. The first one is regulatory burden. I
don\'t think it\'s a surprise that the areas of the country that are
interested in competition are the Northeast and California. There\'s an
awful lot to regulatory burden. Some of it is very positive, shall we
say doing things about the environment, but some of it really, certainly
in New York, has been a way of essentially taxing people indirectly
using utility companies, and this has led to frustration from major
customers of electricity who think that they ought to be able to get
power a lot cheaper, and I think that the answer is that they probably
can.

But there\'s one small component of the regulatory burden, and that\'s
the collection of data that also is very important for this. So that I
think that EIA needs to make a major review of the types of data that
are being collected to answer the most obvious question that one\'s
going to get from the Hill about what is actually happening and are
prices going down.

So I think that given the financial pressure that the utility industry
is facing, having a serious review of data collection is sort of
politically sensible. I think that given the fact that we\'re going to
be looking at new forms of financial markets, being able to gather this
type of information electronically is a real possibility. So the burden
will not be as great as it might be without that sort of mechanism.

But I think that the most important thing is that EIA is going to need
new types of information on prices that are not being gathered at the
moment, and I think that what we have to do is to try to avoid getting
into the predicament that there was with natural gas prices that a lot
of the market was missed, not covered.

So I really congratulate EIA for sort of looking at these issues and
trying to grapple with them early at a time when it is not obvious what
exactly is going to happen.

So the second issue that I want to talk about is restructuring rates
that customers pay, and I suppose here one might look at a parallel with
what happened in the airline industry. It costs me about as much to go
and visit my mom in England as it does to make a business trip to come
and visit you folks down here. I also have a vast array of different
pricing systems and different \"thises\" and \"thats\" and I contract
with a broker, my travel agent, to figure out what in the heck is going
on and tell me what to do.

But I think that the bottom line is that the reason that I pay a high
price to come to visit Washington is because I live in a small town,
Ithica, and the reason that I don\'t pay much to visit my mom is that
I\'m going on a big trunk line, the Atlantic run, and I\'m paying very
competitive rates, and I think that\'s basically what it\'s going to be,
that large industrial customers are going to do pretty well or better
than they are now, but I\'m not too optimistic how I\'m going to come
out of this as a residential customer.

So the something that\'s very important about the electric industry and
that the nature of the product makes it possible to use price
discrimination in a way that is completely unrealistic for other types
of products, so that we can have prices that vary by day. We can have
prices for individual customers. We can price the demand from individual
customers so that you pay different amounts for different quantities
purchased, and so essentially multi-part tariffs that can be
approximated by two-part tariffs.

But the ability to come up with a very new, innovative way of pricing
electricity is really substantial, and therefore, just assuming that
getting information about the average price paid is really inadequate
for understanding how these types of price changes are likely to affect
demand.

I think that it\'s important to talk about the sort of vision of what
the institutional structure of pricing is likely to be. Most of the
discussion about competition focuses currently on generation, and the
general view is that some sort of real time spot market will emerge for
generation.

It seems to me that packaged around that, the same way that my travel
agent protects me from all of this stuff, there are going to be a vast
number of brokers representing suppliers and purchasers. There are going
to be many new forms of financial derivatives, forward markets, et
cetera, et cetera, that are going to be important, and it is the prices
that customers pay that determine their demand and the way that they pay
those prices so that customer charges are different form energy charges.

With regard to the actual structure of the industry itself, it seems to
me that we\'re going to have a competitive generation system, and then
we\'re going to add what I\'ve called a regulated wedge, which maybe
covers transmission and distribution, but clearly the utility industry
is very interested in maintaining the ability to recover money for
stranded assets, and how much they\'re going to be able to do that is
really the political issue that hasn\'t been determined.

But I think that that ability, if it exists, to be able to extract more
than a competitive price for electricity will depend on a sort of
quasi-regulated system existing for some part of the industry, and the
most obvious one is for transmission, and I think we\'re going to end up
with the sort of bizarre situation where transmission charges cover the
cost of nuclear power plants.

So I think that this really says that trying to figure out what\'s going
on is going to be extremely difficult, and I don\'t think that there\'s
any easy way to say what type of data are required.

However, I think that there is a difference between the sort of analysis
that we\'re doing at the moment at Cornell and the sort of analysis
that\'s being done at EIA.

A lot of the complication in the presentation this morning is trying to
come up with the equivalent of a capacity charge. Everybody knows what
the energy charge is, the spot market price. I think that there\'s a
good argument that maybe if there is an effective competitive market,
that capacity charges are not going to be needed as a separate item. The
system that the U.K. used, I think \-- I don\'t know what the polite
word is, but \"hokey\" would be a good one \--

(Laughter.)

CHAIRMAN MOUNT: So the bottom line, it\'s not clear to me that worry
about how one ought to measure capacity charges is important at this
time. I think that something that could be done now with NEMS is to just
assume that somebody is going to get a hit somewhere. There\'s going to
be some stranded assets, and you\'re not going to have to recover as
much revenue.

My guess is the big advantage of that is going to go to the industrial
sector, and therefore, an interesting question is if, in fact, rates get
lower in the industrial sector, what happens to demand. Does this deal
with some of the high reserve margin problems that we have?

You know, this is a very sort of straightforward analysis with the
existing model structure.

Then I have one technical argument with Doug over his use of nodal
pricing, point-to-point pricing. Certainly the people that we\'ve talked
to who are trying to set up a market in California for the Mercantile
Exchange think that zonal pricing is the best that we\'ll be able to get
to, but I think that the distinctions of these two are technical.

Clearly one has to take into account capacity constraints in some form
to avoid the problems that the U.K. had.

Given all of these complications with rates, my guess is that the best
data that exist are currently at EIA, and I really hope that that stays
the way, that EIA is able to get the data that they need in order to be
able to monitor this very complicated situation that we\'re getting
into.

So the third point deals with customers with large loads, and this is
somewhat of a side issue in a way, but I think that the utility industry
got into a problem rather like computing groups that hung onto the
mainframe and didn\'t get into distributed computing; that the utility
industry did not realize that there were very real efficiency gains from
using combined cycle turbines and, therefore, having a small distributed
system rather than large plants.

There are probably a few people that still believe that we ought to be
running an energy system based on plutonium, but my guess is that that
view is dying out and that the potential advantage of having this
distributed system is that it makes things like cogeneration a lot more
feasible, and so that the hope that existed under PURPA really can
manifest itself with new technology.

However, the customers that are most likely to be able to do this are
the ones that are going to get all the breaks with competitive prices,
and so I personally think that there\'s a real tension between trying to
do things more efficiently and worry about greenhouse gases, et cetera,
et cetera, and the fact that the large customers are the ones that are
going to be able to negotiate lower rates in the systems that are being
considered at the moment.

And the final point I have is nuclear power. Clearly the nuclear
industry has made tremendous improvements in the last couple of years so
that the many nuclear power plants are now competitive in an operating
sense, but there are still large amounts of debt in the rate base of
many utilities, and this is a major stranded asset or strandable asset,
if you\'re working for the utility companies.

So there\'s an awful lot of restructuring of the industry going on, and
it seems to me that it\'s very important that EIA keep track of what\'s
going on to the nuclear industry. I think that the important thing about
the nuclear industry is that this is not a question of just paying off
what\'s in the rate base.

I tried to think of an analogy for it, and the best I could come up with
was alimony, that basically there\'s a real commitment to pay money to
these plants into the future and decommissioning and the monitoring of
those plants is something that will make companies that have those
liabilities uncompetitive compared to companies that don\'t have those
liabilities.

So the ability to keep a regulatory wedge so that you\'re able to
recover costs for that is one way of dealing with it, but the bottom
line is that I think it\'s very important that we don\'t see a
restructuring of the industry so all of the nuclear pieces go bankrupt,
and then I\'m not too certain who owns them.

But I think that the importance here is that in my view the federal
government cannot cease to be heavily involved in dealing with the
nuclear industry, and that includes paying for a lot of the costs that
are going to come in the future.

Thank you.

So now we open it up to the floor. Campbell.

MR. WATKINS: I have several comments. So maybe I\'ll come up to the
podium.

CHAIRMAN MOUNT: That\'s fine.

MR. WATKINS: First of all, a few comments on Doug\'s paper and then one
or two on Art\'s.

This is a very difficult exercise, of course, but few in Doug\'s paper
\-- one of you mentioned about Paul Joskow\'s estimates that, in fact,
electricity deregulation would not generate a lot of benefits. I\'m not
quite familiar with the paper of his that you\'re referring to, but it
seems that I\'m more with your kind of results, that, in fact, there
would be substantial benefits.

I say that because the mere fact that we\'re talking at all about a
stranded asset problem or perception of it is an indication that there
are a lot of assets out there that are uneconomic. If they\'re
uneconomic, then with a competitive system they\'re not going to be
built in the first place. If they\'re not going to be built in the first
place, they don\'t get in the rate base or whatever, and so your prices
should be lower, and significantly lower, other things equal.

It may be, however, that Joskow\'s comments are more directed about what
may happen under regulation when, in fact, you have incentive regulation
and things like that. So if you do that and if you take account of the
fact that maybe some of these stranded assets have reached the point in
their life in the regulated system where the depreciation schedule is
such that the contribution of those, the cost of services diminishing,
perhaps that\'s how he gets this kind of result.

Doug, you mentioned the point about spot elasticities and the fact that
you\'re really data short on that. Two suggestions there.

What we\'re really talking about is peak shifting in that context, is to
look at international data where, in fact \-- and, of course, the U.K.
would be the prime one \-- where you do have some data that may be
useful.

And, secondly, there is quite a literature on time of day pricing even
with regulation, and you may be able to use that.

A third point, I had a question for you on this question of how pool
delineation. How are you going to handle that? I mean how do you
delineate what is the power pool in a regional locational sense?

On your steps where you talked about \-- on the latter part of your
paper, pages 11 and 12, the various step, when I read those steps I
thought that you have to or my guess would be that what you had as a
portion of Step 2, which has to do with data collection, was more
important than Step 1 that had to do with the NEMS model in that I think
if you\'re going to grapple with this problem, your collection of the
actual data that will be emerging or the mechanisms you have to collect
those data \-- I would put that as Step 1, not as part of Step 2.

A comment Tim made about capacity allocation and the regulated wedge for
transmission, and it\'s frequent to think of that as being in the kind
of natural monopoly category, which is why you may want to have
regulation.

I think you can go a step further than that, however. If you really had
total deregulation, and it\'s a step that natural gas is yet to go to,
that is, where rather than just, in effect, renting capacity rights on
the pipeline and in this case an electric transmission facilities, you,
in fact, acquire property rights on the system, and you had a capacity
release market. You wouldn\'t need to have regulation even of
transmission facilities.

My final comment on Doug\'s paper is that my guess is that if you do
have electricity deregulation, we\'re going to be surprised about how
pervasive it is and how quickly it takes place in that you do have the
natural gas industry to look at, and everybody has been surprised at the
extent to which a competitive market has emerged in a way and to a
degree that hadn\'t been anticipated.

If you look at the recent studies or papers, say, by DeVany and Walls on
the way in which all the prices, the natural gas prices across the
system from one end of it to the other, from Canada right through to
Louisiana, are linked and he degree to which they\'re linked, that has
been a surprise that that has happened that quickly.

Now, dealing with Art\'s paper, I had one question, and I will,
notwithstanding our Chairman\'s caveats have a stab at answering some of
your questions here, but these 108 pricing periods. Were you talking
about are they different prices at different times of the day over
different days or the year or they\'re just different days in the year?
It wasn\'t clear from your paper just what these time slots represented,
the 108.

MR. HOLLAND: They were selected to try to capture the full range of load
characteristics throughout the year.

MR. WATKINS: Okay. So they vary by time of day.

MR. HOLLAND: They vary by time of \--

MR. WATKINS: That\'s seasonal and time of day.

MR. HOLLAND: Correct.

MR. WATKINS: Within the seasons. Okay.

MR. HOLLAND: Seasonal and time of day.

MR. WATKINS: Okay. I understand.

MR. HOLLAND: And they vary in the number of hours, as well.

MR. WATKINS: Right. Well, on your Question 1 about what analysis stage
while modeling products, I jotted down three things here. One is price
formation. The second one, I would suggest, is plant utilization, and
I\'m including in that investment in new plants, and the other one I
jotted down was this question of inter-regional trade that may emerge,
as you rightly pointed out, to a much greater degree with deregulation,
that aspect.

On your Item 4, I think there\'s too much focus here on the spot price.
With a deregulated market, you could have a much greater array of
contracts, of different terms and conditions.

Tim was talking about the airline industry and, you know, the great
array there. You\'re going to have that probably happen.

Also, if you have futures markets that do develop, you may be able to
sign up equivalently for prices one, two, three, four years in advance
using those kinds of mechanisms. So I think your concern with the spot
price is too great. I think you\'re going to have a greater range.

The discussion of the energy prices and your market clearing mechanism,
I wondered why that wasn\'t simpler to think of in terms of just the
distinction between short run and long run marginal costs, but I\'m not
sure I really understood your simulations in terms of your methodology.

Could you put up the first graph again? Was it New England? I mean your
reactor results. You know, you had the ones you had just done this week.

MR. HOLLAND: The ones with the spot prices?

MR. WATKINS: Yeah, and the components of the price.

MR. HOLLAND: The first one was ERCOT.

MR. WATKINS: Now, I mean, I\'m not sure I understand your methodology.
The way I saw it described in the paper because there\'s a gap between
the capacity and the peak demands, ostensibly both the ex ante and ex
post demand is satisfied. So why is there a red component to the charge
at all in the context of your methodology?

And then if there is a red component in there, I don\'t quite understand
some of the relationships. For example \-- I\'ll have to point here \--
this one is almost past here. So why isn\'t this \--

MR. HOLLAND: Actually you should go to the second season. That\'s the
first price in the second season. So you should look at the second
position of the top blue line.

MR. WATKINS: Oh, all right. So it\'s not \-- what I was going to say is
suppose they were aligned. You would expect this one to be much higher.
The gap here is squeezed. So maybe when you align it, perhaps you get
that pattern when you align it, but my main question was I don\'t quite
understand how the calculations emerge if I understand your methodology
properly.

I think that\'s it.

CHAIRMAN MOUNT: Cal?

MR. KENT: Most of the things that I was going to say have already been
covered, but let me just stress a few things that I think need to be
re-emphasized.

The first one is to thank Doug for a very insightful paper. As an
economist, I appreciated reading it very much, and what you\'re
attempting to do with that, I think, is extremely useful.

I did come away though with the feeling that it\'s going to be very
difficult for EIA to respond even to your paper, much less to this
environment, because we don\'t know yet what it is that we need to know,
and that\'s very hard to plan or to model or to even figure out what
data it is until we know what it is that we need to know, and I didn\'t
get the feeling that we were certain enough in our knowledge of what was
going to be expected, what questions were going to be asked, that we
really knew what it was that we ought to be out there looking for.

The second thing is I would reemphasize what Campbell just said, and
that was I think there was an overemphasis in Art\'s paper on spot
markets. I think that the interesting play is going to get to be the
futures markets, which I think are going to develop very, very quickly
in this area, and that that may alleviate some of the problems, Art,
that you talked about as soon as a future market develops, and I
certainly think that they are going to be more important, as they have
proven to be in other energy sources, particularly petroleum markets.

The next thing is just to make a general comment. In one sense natural
gas has kind of led the way, and as I was reading between about four
o\'clock this morning as I was flying in from Los Angeles the paper on
natural gas, it surprised me that some of the questions that were being
asked there about what sort of data issues were being raised by
deregulation I think are going to be exactly some of the same issues
that you all are going to be facing, and there may be quite a bit more
out of the natural gas deregulation that you can mind for indicating to
you or indicating what direction you should go, certainly not to the
degree that took place in natural gas.

The stranded costs have occurred in natural gas, and I\'m babbling now.
So let me see if I can clarify that point.

The deregulation of natural gas has led to stranded costs. They are
certainly not of the magnitude of the major nuclear power plants, but
particularly into East Coast or eastern area gas producers, such as
those in West Virginia, have had terrible losses due to stranded costs
that they have not been able to recover because of the distribution
system that exists in the East as opposed to that that is now in the
South and certainly to the West.

And so I think that what you\'re going to have is that since
historically any time an industry has become more competitive, whether
it\'s through the natural entrance of new competitors or deregulation,
there are always going to be stranded costs that were there and that
were only justified because of a monopoly position, either a natural
monopoly position or a regulated monopoly position.

And I think that the other remark that was made is that we are going to
be surprised at how fast we\'re going to see competition; that this
stranded cost issue may very, very quickly almost get itself solved just
because of the fact that the market is going to force the issue much
more quickly than we may even be able to comprehend it, and my suspicion
is that the end result is going to be as historically it has almost
always been, and that is to say to the people who are stuck with the
stranded costs, \"You get to eat them.\"

And I think if you take a look at the prices of shares of certain
utilities today, you can begin to see that to an extent the market has
already anticipated that result, and so I may be saying that this whole
issue of stranded costs may be one that you\'re not going to have to
worry about modeling as much as you think because it\'s just going to
happen. There are going to be some losers. It\'s going to be quick and
merciless for their stockholders, and then after that the market will
sort its way out.

And I may be wrong on that, but that\'s just my suspicion of the way
things are going to happen.

Then the last comment, and I\'ll shut up, is I think as Doug\'s point as
made in his paper. I think there\'s going to be a tremendous amount of
interest in what us economists call the transactions costs, and I think
this gets back to what Joskow was talking about in his paper, if I
remember it. There\'s going to be phenomenal transactions costs involved
here, particularly in the rather extended transition period we\'re going
to go through, and those transactions costs are going to be much more
interesting than the generating costs that we talked about in the past,
and that may very well be where some of the efficiencies are lost, is
just through these transactions costs.

And you can go back to Coase\'s original work in the area and everything
like this, and I think that a lot of the anomalies that you may see
developing in this market and are probably already developing in the
natural gas markets are due to these transaction costs that we tend to
overlook and we tend not to model.

So that\'s my whole comment.

CHAIRMAN MOUNT: Thank you.

The rest of the Committee is silent. I am amazed.

So how about members of the public back there? Anybody would like to
comment? Would you like to respond?

MR. HALE: Oh, yeah, sure. Just first thank you all very much for your
comments.

First, just a couple of things. Campbell and I will be talking some more
this afternoon, I\'m sure, but there are a couple that are worth talking
about now.

First, the emphasis on spot markets, spot pricing, I agree with Campbell
that what the consumers are going to see are not the spot prices, okay,
but the spot prices to the extent that you get something like that
happening is the grist for the mill. I mean that is what all the futures
contracts, all the deals, all the packages are based upon.

If you don\'t get the spot market prices right, you should go home now
because the rest of it is just \-- you\'ve got nothing to work with. So
that\'s the emphasis on spot.

We\'re not all that optimistic. Well, we find that real hard right now
to even think about getting the spot market prices right without going
to the next level of the packaging that the entrepreneurs are doing
right now.

Second, I think everybody has a lot of priors about how this
restructuring is going to go. You know, as a personal opinion, I believe
that we\'re going to see a huge amount of efficiency gain. I think
we\'re going to see de facto deregulation very, very quickly for
industrials and commercial and people who are lucky enough to be close
to borders between utility districts.

I think you\'re going to see more attempts on the parts of utilities to
insure their monopoly position as best they possibly can be mergers,
acquisitions, joining up with natural gas companies, joining up with the
cable companies, anything they can to control various aspects.

And I also think that what you\'re going to see is, in fact, you won\'t
see nodal prices. You\'ll see something zonal, but we\'ll get to that in
a second.

I believe all this stuff, but when I look at the data I got and I look
at my models and my analytical capability, I\'m not really persuaded
that I\'ve got other than a religious case to make at this stage. I
believe that the answer I\'ve given is the right one. I haven\'t seen it
yet.

MR. WATKINS: Religious must be perfect foresight.

MR. HALE: Once again, I started out by saying I know where dancing
leads. I don\'t know where this leads for sure.

Okay. I really don\'t see any conflict whatsoever between environmental
protection and restructuring. I think that there\'s a real chance that
by pricing electricity much closer to its social cost you\'re going to
see much better resource use, and I think you\'re going to see the old,
ill controlled facilities shutting down much quicker. I think you\'re
going to see electricity used the way it should be, and I think the
environmental benefits could be substantial.

I think this is one where the environmentalists are very, very wrong,
and I hope that I\'m right on this one.

The issue of power pool delineation is critical. It\'s the name of the
game. I don\'t know what to say. I mean the size of the market
determines how well this is going to work, and if the market is
expanding, the larger it is, you know, the better for all of us.

There was a point made that perhaps in the analysis agenda Step 2 should
actually be Step 1, and I think the argument I\'m making or the fact is
Step 1 is already underway, and what I was trying to do in Step 2 is
argue that we should return to some of these data issues just as quickly
as we possibly can.

I\'m not saying we shouldn\'t be doing what we\'re doing, but the data
issues are not going to go away, and they are basic material for coming
to any conclusion at all or contributing to this debate at all.

Joskow, I\'ll either get the proper citation and quote or I\'ll send you
a note recanting. How\'s that?

So I really appreciate your comments and suggestions on the paper, and
I\'m looking forward to talking to you all privately on more technical
matters.

Oh, one other thing I do want to mention. In looking at how spot prices
are calculated, it becomes very, very clear to you that a lot of the
pricing algorithms and a lot of the dispatch operations that are done
now by engineers are done based on rules of thumb, and computer software
and control software that\'s good enough. Why? Because making the
software better, making the calculations better, getting your
approximations more accurate doesn\'t make you any money in an average
cost pricing world. It\'s a wash.

Now I think there are going to be substantial incentives for much better
approximations of the physical constraints, the flows throughout the
system, substantial incentives to go from zonal, you know, to finer and
finer and finer zones because if you have a difference between the
accounting cost and the economic values, that\'s a license to steal, and
they\'re going to do it.

So that\'s my last pitch on that.

MR. CHATTERJEE: Can I ask? This question is for Doug.

MR. HALE: Yeah.

MR. CHATTERJEE: Do you envisage the effect of deregulation to be many
small, efficiently run utility companies, or do you see large mega
utility companies?

MR. HALE: I think you\'re going to see a lot of gorillas, you know,
fighting over each other\'s territory a lot at the fringes. I don\'t
think you\'re going to see atomistic competition. I just don\'t.

But, you know, the key elements to me, at least, are what do they do
with the transmission systems, you know, and control. It has very little
\-- and control has to do with what gets dispatched, who gets turned on
and off, and what conditions, that sort of thing.

I think at least the control part is inherently monopolistic and will
remain that way and will be regulated. Transmission I\'m not so sure of,
at least long distance, you know, the high power transmission. I\'m not
so sure what that is.

So I would imagine you\'re going to have fairly large firms.

MR. CHATTERJEE: The difference in spot pricing, of course, was
diminished at the efficiency of the transmission.

MR. HALE: Absolutely, absolutely. So that\'s our best bet, I think. I
think you\'re going to have a lot of gorillas, you know, big firms, but
if we can get the transmission system working properly, I think there\'s
going to be very effective competition amongst large firms. I just
don\'t see small firms being able to complete. I mean they can\'t even
afford the software to do the calculations I don\'t think.

MR. WATKINS: I think, Doug, you should draw a distinction between the
generation and distribution and transmission. There\'s greater
likelihood of a small firm or a more atomistic competition in
generation.

MR. HALE: Oh, absolutely. Yeah, I\'m sorry. See, I think the real
question has to do with transmission and control of the system. I think
control of the system is probably going to \-- you know, the ISO or
whatever you call it is going to end up viewed as some sort of natural,
indivisible, you know, kind of monopoly that you\'re going to have to
have regulation. Transmission to me is an open game still. I just don\'t
know.

MR. WATKINS: I mean in generation the efficiency of small turbines and
the change associated with that is very strong.

MR. HALE: No, I agree. I think in generation you\'re going to have
opportunities for small firms, but I don\'t think generation is where
the action is going to be in this market. I think you\'re right, but
that\'s not where you\'re going to make your money, in my opinion.
We\'ll see.

MR. HOLLAND: I had just a couple of comments I wanted to make about some
of the things that were said.

This issue about the emphasis on spot prices, there are a couple of
reasons for that. I agree completely that most consumers are probably
not going to see those spot prices. They\'re going to see contracts, and
one of the things that we\'re doing with the spot pricing mechanism or
the spot pricing algorithm is to try to calculate what those contract
prices are going to be.

There\'s a possibility we may even use the spot pricing algorithm to
develop a range of prices over which contracts could exist. Given what
assumptions are done in the particular modeling run, you may even think
of the energy component as defining the floor of the contract prices and
the reliability or market clearing component and possibly the insurance
component as defining the ceiling of those contract prices, depending on
how much reliability different consumers want to purchase from their
generation suppliers.

A second thing or point for the spot pricing is if you reduce the price
of electricity by restructuring the electric power industry and nothing
happens to your demands, what have you accomplished? Possibly some of
your big bangs are going to come from load leveling that will change
your capacity planning decisions and also could change the way you
operate your system.

When Larry Ruff spoke at the NEMS conference, he talked about a system
where as the limits of your capacity start to be reached, you need some
mechanism to jack the price up so that you start getting people who
don\'t value electricity as much off of your system. That keeps you from
having to decide where you shed your loads.

It also is a good load management \-- I think pricing is possibly the
best load management system, but the energy component doesn\'t give you
enough of that. You have to have some other way to calculate how much
you increase your prices. So this algorithm is trying to do that.

We talked about that it was mentioned that there was no capacity. Some
people are using a capacity chart which is based on the cost of building
the capacity, but that\'s doesn\'t get at what the demand \-- what your
consumers\' value is, how much they\'re willing to pay, what the market
will bear is, and that value of unserved energy is a way to estimate or
guess what the consumer or what the market will bear.

That has to be a varying input assumption because everyone has a
different value that they place on electricity during a blackout. A good
example might be to look at what a restaurant owner in Georgetown might
be willing to pay for electricity on Friday night if he\'s got a dining
room full of customers and the lights go out, and then you might look at
a college student with their boyfriend or girlfriend on Friday night
sitting on the sofa in front of the television and the lights go out.

(Laughter.)

MR. HOLLAND: They are very different amounts that they would pay to
bring the lights back on. So that\'s what that value of unserved energy
is trying to get at.

So what\'s all that I wanted to say about that.

CHAIRMAN MOUNT: Anymore comments from the Committee? From the public?

So I thank you both very much, and we are adjourned and we will meet
back here at two o\'clock.

(Whereupon, at 12:22 p.m., the meeting was recessed for lunch, to
reconvene at 2:00 p.m., the same day.)

[AFTERNOON SESSION]{.underline}

(2:05 p.m.)

CHAIRMAN MOUNT: I\'d like to reconvene the meeting of the ASA Committee,
and there will be two presentations on the residential and commercial
demand models in NEMS.

The first one is going to be John Cymbalsky, Office of Integrated
Analysis and Forecasting.

MR. CYMBALSKY: Okay. Thank you, Tim.

Is it okay if I talk like this for the transcript? Okay.

As Tim said, my name is John Cymbalsky with the Office of Integrated
Analysis and Forecasting. I\'ve been with EIA doing residential demand
modeling for almost seven years now. That covers, I think, seven AEOs
I\'ve done since I\'ve been here.

Today\'s presentation, I\'m going to talk about the NEMS residential
model and especially focus on the projections from AEO \'96.

To give those of you who are not familiar with NEMS a little overview of
what it does, okay, the NEMS energy model has 11 supply and demand
models, a macroeconomic activity model, and an emissions model. This is
a general equilibrium model, and it iterates over the years to come to a
partial equilibrium for the energy supplies/demand market.

Now, talking about the residential model in particular, it\'s not an
econometric model. It\'s an energy engineering economic modeling
structure. It\'s very data intensive. It has information about the
housing stock, the equipment stock, appliance efficiencies. We have
technology choices. We have building shell efficiencies, and all of
these numbers vary by the nine Census divisions and three building
types.

We also model at the end use level, and we have about seven end uses,
including lighting, heating, cooling, water heating, and the like.

Now, for AEO \'96 and all AEOs that I\'ve been involved with, we run
generally three cases for the AEO. We have the reference case, which
I\'m going to call the business as usual, and what that means is that we
assume that policy will remain unchanged as it exists today for the next
20 years.

We also have the low and high world oil price cases where we perturb the
oil price projections and see what the response in the energy markets
are.

We also do low and high economic growth cases, and we perturb GDP and
see what happens, again, with the energy markets in these scenarios, and
for the last two AEOs, we\'ve also used stand alone technology cases for
both the residential and commercial models. In these cases, what we did
is we made assumptions about technology, whether it\'s going to be a
high technology case, which is the best technology available, or frozen
technology case where we freeze technology at its 1995 levels and then
we want to see what happens, you know, when you make different
assumptions about technological progress in the future.

Okay. My first graphic here is just showing historical and projected
residential primary and delivered energy. You can see the primary energy
line versus the delivered energy line has been diverging over the past,
say, ten years, and we project this to continue, and this is just
increasing electricity penetration in residential market, and when you
convert that to primary, you can see that the lines diverge even more.

Okay. This graph now shows primary energy intensity for historical and
projected residential energy demand. You can see that on a per household
basis, energy consumption per household has been coming down for the
past 20 years, and also \-- and this is due to efficiency standards and
the like ‑‑ and our forecast you can see also has it slightly declining,
but the thing to note here is if you look on the square foot basis and
the per household basis, you can see the square foot basis, it\'s
declining more in the forecast, and that\'s due to the fact that we\'re
projecting bigger households in the future in terms of the physical
square footage.

Okay. This graphic shows historical and projected primary fuel shares in
the residential sector. From the graph you can see that in 1970
electricity and gas almost had the same share in the residential sector,
and in 1994 you can notice how different the fuel shares have changed.
We have an increasing electrification, and again, this is primary
energy. So electricity has, you know, dominated the market in terms of
market penetration in the last 24 years, and we project this to continue
for the next 20 years or so.

Now, if we look at the end use energy for the residential sector, you
can see that space conditioning, which is heating and cooling, is the
largest end use in the sector, has been and will continue to be in the
projection period.

You can see that the other category is all of the other categories that
aren\'t listed, and this is mostly miscellaneous electric appliances.
You can see that grows enormously in the projection, and this is due to
the fact that they\'re all electric, and there\'s no efficiency
standards for most of these appliances. So as they penetrate the market,
they just increase over time.

And another important fact here is refrigeration as an end use, even
with the addition of the number of refrigerators in the stock in the
past up till now, you can see the actual consumption by refrigerators
has declined, and this is due to the federal energy efficiency standards
for refrigerators in both 1990 and 1993.

Okay. This next slide shows housing growth in the pie, in the projection
period from 1994 to 2015, and what we did with the pie is we wanted to
break out and show you where in the country the housing growth is
projected to be in the next 20 years.

You can see that the South has 47 percent of the growth in housing in
the projection period, and to show you, you know, what\'s going to
happen with fuel use for the next 20 years, it\'s important to know
regionally what fuels are being used, for space heating especially since
it\'s the biggest end use, and you can see that in the South, which has
47 percent of the builds, it also has 56 percent space heating for
electricity and only 43 percent for gas.

Now, nationally \-- well, for the rest of the nation the trend is
different. It\'s 68 percent for gas and only 24 percent for electricity.
So this is very important in the modeling to know that the region in the
country where the houses are being built is very important in terms of
which fuels will be used in the future.

MR. LOCKHART: Is that primarily the heat pumps?

MR. CYMBALSKY: Yeah, heat pumps mostly are in the South. That\'s right.
So most of the new construction in the South are using heat pumps.
Whereas the rest of the country tends to go with gas furnaces.

MR. LOCKHART: Okay.

MR. KENT: Is that the delta that you\'re talking about there or is that
the stock that you\'re looking at?

MR. CYMBALSKY: Right. It is the accumulation of all the new stock from
1994 to 2015.

MR. KENT: But it isn\'t anything \-- I mean what you\'re saying is 2015,
this is where we\'re going to be. It\'s not just talking about what the
change is?

MR. CYMBALSKY: It says that between \'94 and 2015 if you build 20
million homes, of those 20 million that\'s the percentages.

MR. KENT: Okay. So it is the change.

MR. CYMBALSKY: Right, correct.

Yeah, and in the South the reason they use heat pumps obviously, they
have lower heating loads than in the North. The fuel price is less
important. So, you know, they\'ll go for electricity.

Okay. Again, here this is just going to reestablish the growth in the
households that I mentioned in the last slide, and you can see that in
the middle, the south Atlantic, east south central, and the west south
central are the three big growers out of the nine Census divisions, the
south Atlantic especially because that\'s already the biggest Census
division in terms of households and energy use, and you can see that\'s
going to grow the biggest, and it already has the most houses. So
that\'s a major impact in the forecast.

Okay. This is the energy intensity change from \'94 to 2015 as a growth
rate: primary fuel consumption per household, and again, you can see
refrigeration on a per household basis, refrigeration as an end use we
project to decline over three percent per year in intensity, and again,
\"other\" is all of the miscellaneous electricity end uses which do not
have federal efficiency standards associated with them. So they
basically will grow as they penetrate into the housing market.

This graph just shows to give you an idea of the intensity for each of
the three housing types that we have in the model in 1994. There\'s
really no surprise here. Single family, you know, they\'re the biggest
houses; they use the most energy. Multi-family, they, you know, tend to
be apartments and smaller. So they use the least.

Okay. As I mentioned before, we had three side cases that we ran or two
side cases that we ran relative to the reference case in this year\'s
AEO, and this graph is just going to show over time the difference from
the reference case that different technology assumptions \-- what impact
they have on consumption in the sector, and you can see that if there
were on efficiency improvements relative to 1995, we would use about one
and a half quads more by 2015 relative to the reference case.

But if we employed best technology choice in the next 20 years, there\'s
a potential for over three quads of savings in the sector by 2015.

And when we run these cases, basically the cases are run such that
economics are not a factor in the decision making for the technologies
for the side cases, but what I\'ve found interesting and people always
ask is, \"Well, what if you looked at the investment costs for these
technologies? How much would they cost and what would be the incremental
investment needed for the savings?\"

So what this graph does is try to show you what level of investment in
certain appliances is in the AEO reference case and then how much more
it would cost to get the savings that I just mentioned in the previous
slide.

The top line shows in the AEO reference case what Americans are paying
for the fuel costs for these appliances, and you can see it\'s around
\$80 billion in 1991 dollars, and then the next line down is the capital
investment associated with the AEO reference case. So you could see that
Americans are paying somewhere around \$30 billion for this equipment.

And then relative to the base case, we want to look at the best
technology case and ask ourselves, well, how much incremental investment
and how much incremental savings we would get from running the case, and
you can see the red line. In the first years you need a lot more
investment to get the fuel cost savings, and then around 2005 you can
see that the lines intersect, and by the end you have more fuel cost
savings than you do investment.

Now, there is no discounting of the cash flows here. So this is just
undiscounted money.

And the last slide is to look at this same thing, but on an end use
level for electricity and gas. So you could see that the important thing
to note on this is the bottom graph \-- excuse me \-- the top graph
shows that in electric water heating it\'s the only end use where the
savings is greater than the investment, and this basically is because of
the heat pump water heater that is out there and is a viable technology.
It\'s not be purchased much now, but the fuel cost savings associated
with using this electric heat pump water heater far outweigh the
investment needed.

So you can see by all of the end uses sort of what would be needed in
terms of investment to get the savings.

Okay, and I think I\'m out of time. I\'ll turn it over to Erin.

CHAIRMAN MOUNT: The next presentation is going to be by Erin Boedecker,
Office of Integrated Analysis and Forecasting.

MS. BOEDECKER: I\'m John\'s counterpart on the commercial side of the
house, and so I project energy demand in the commercial sector, mainly
dealing with building energy.

Today I\'d like to look at the trends that are shown in AEO \'96 as far
as commercial energy use and intensity, looking at both primary energy,
which includes the electricity losses from production and distribution,
and also delivered energy; look at what types of fuels are used and how
those fuels are used. then I\'d like to look at some of the factors that
contribute to the trends, look at the end uses in more detail, look at
different areas of the country, and also look at the different types of
businesses that are portrayed in the model.

And finally, I\'d like to present the future energy savings that are
projected in AEO \'96 in the reference case, and also show the potential
for additional savings if the commercial sector were to change the way
that they make their investment decisions.

In the commercial sector, we\'re mainly talking about building energy,
and so for our intensity measure we typically use BTUs per square foot
instead of per capita or some other measure.

Over the forecast, you can see that energy intensity is projected to
decline, and as far as primary energy, this reverses the historical
trend for the last 20 years. In the model, this decline is due to
efficiency gains represented through standards, such as the Energy
Policy Act; voluntary programs, such as EPA\'s Green Lights and Energy
Star Programs; and also efficiency gains in electricity generation.

This year the shift in focus from delivered energy to primary energy has
emphasized the commercial sector\'s reliance on electricity, and this
dependence is expected to continue just like it is in the residential
sector.

I\'ll explain a little bit about this slide before I talk about it. The
numbers far to the left are total commercial energy consumption. The top
of the slide is for 1994 and the bottom is for 2015. The pie charts show
us energy consumption by fuel, and here you can see that electricity is
expected to maintain its share at 73 percent. The right-hand side
columns show you the energy consumption by end use.

Now, the fuel shares are projected to be stable, but the composition of
end uses is projected to change slightly.

As you can see, office equipment, which includes PCs, copiers, faxes,
that type of equipment, and also the other use category, which includes
such emerging technologies as medical imaging technology and
telecommunications equipment, are both expected to continue to penetrate
the commercial market.

We made more changes to the model this year to more accurately reflect
the popularity and advances in these areas.

On the other hand, the percent of total energy consumption and also
energy use per square foot, as is shown on this slide, both decrease for
uses that have reached market saturation, such as space conditioning,
water heating, and lighting.

These declines reflect the efficiency gains that I spoke of earlier, and
also in water heating, they reflect the shift from electricity to
natural gas.

The efficiency gains are also expected in office equipment, but these
gains are overshadowed by the project growth.

This slide shows fuel shares by end use, and this more clearly reflects
the shift in water heating from electricity to natural gas as a fuel.
Again, it shows the sector\'s reliability on electricity, and it shows
that the changing end use composition does play an important role in
trends, but there are also other factors that affect energy consumption,
and we\'ll see whether they reinforce the national trend or counteract
each other.

Just as John looked at different areas of the country, we will for the
commercial sector, too, and energy use in different areas of the country
is affected by both climate and demographics. The commercial sector does
operate at the Census division level, which there are nine of, and the
south Atlantic division is the largest sector. It covers the East Coast
from Maryland to Florida, and it\'s also projected to grow the fastest.

The only division in the commercial sector that is not expected to grow
is New England.

Looking at energy intensity by area of the country, it\'s expected to
decline in all areas. However, there are two areas of interest on this
graph. If you look at the east south central division, which includes
Mississippi, Alabama, Tennessee, and Kentucky, it shows very little
change in intensity relative to the other areas of the country, and New
England shows declining use in energy per square foot even though
there\'s no growth in commercial floor space.

This can be explained at least in part by trends exhibited by another
contributing factor, which is building or business type. In NEMS we have
11 commercial building types, and energy use per square foot does vary
by business category. Food sales, service, and health care have the
highest intensities, while warehouse space has the lowest.

Remember in the last slide that energy intensity decreased very little
in the east south central division. It turns out that health care
related space in this division is expected to grow at twice the national
rate, and it happens to be one of the higher intensity end uses and also
is projected to decline in intensity the least.

At the same time, revisiting New England, while no growth is expected
overall, warehouse space, which has the lowest energy intensity, is
projected to increase close to one percent per year over the forecast
period.

This slide just projects floor space by building type kind of for
completeness since we showed it for the Census divisions. You can see
that the mercantile and service category, which contains everything from
retail businesses to auto repair, starts with the largest share of floor
space and is also expected to show the highest rate of growth. It
retains a much greater share of floor space than most of the energy
intensive businesses portrayed.

The next slide shows the considerations that go into choosing equipment
in the commercial sector, which in turn affects the energy use. We\'ve
already looked at the AEO \'96 results by end use, Census division, and
building type. Another factor that certainly affects energy use is the
payback period consumers require to cover their investment.

In NEMS we have a distribution of six implicit or observed discount
rates ranging from just under 20 percent, which is roughly a five-year
payback period, to a rate representing those who only consider capital
costs in their purchase decisions.

The effects of commercial sector discount rates can be observed in the
technology scenarios for AEO \'96. Just as in the residential sector,
for the commercial sector we ran two stand-alone cases with the 1995
technology case assuming no efficiency advances after that year. The
efficiency gains represented in the reference case compared to that case
represent 2.4 percent of energy savings by the year 2015.

However, the potential for much greater savings exists in the high
technology case where commercial consumers would choose only the most
efficient technology available, regardless of the cost. If the
commercial sector started to base investment decisions more on energy
use than the cost of equipment, an additional 12 percent savings could
be expected by 2015.

However, the price paid for energy by the commercial sector is only
expected to rise one tenth of one percent per year over the forecast
horizon. So many changes would have to occur before that potential could
be realized.

Okay. That\'s the end of my presentation. It was requested that we come
up with a few questions to focus discussion. Since this was an
informational briefing, we had a tough time. I did pull a few out.

The first is more specific. The presentation focused on energy use per
square foot as the measure of intensity in commercial buildings. What
additional measures of intensity should be investigated and would add
some type of insights or give a better picture of energy use, if there
is a better picture?

And the second question is up for grabs. Is there a statistical method
of any type that would reconcile the inherent differences between a
short-term, econometric type model and a long-term engineering economic
simulation model when your forecasting years overlap?

This is something that we keep grappling with and would like help with
if there\'s any help out there.

Thank you very much.

CHAIRMAN MOUNT: So the discussant, Campbell Watkins.

Thank you very much, both of you.

MS. BOEDECKER: Thank you.

MR. WATKINS: I don\'t know whether there\'s any help out there, but
you\'ll be pleased to know that I did at least anticipate the second
question in some of the remarks I want to make.

Both these models are very comprehensive and complex and contain a lot
of fine detail, and to my mind or at least to my knowledge are the best
models available to capture a lot of the things that you\'re trying to
capture at this time.

As you just indicated in one of your questions there, there are a mix of
what I\'d call the engineering and econometric approaches, and one can
but hope that the outcome of this marriage is fruitful.

It reminds me a little bit of a story about an Irish playwright of the
first half of this century, George Bernard Shaw, and he was popular on
the circuit at that time. They used to invite all of the famous and the
literati to weekend gatherings, and one of the persons who used to host
these events was a person called Lady Ottiline Morrell, and she became
very enamored of George Bernard Shaw and said, \"Well, you know, really
we should get married because look at what talented children we would
have. They\'d have my looks and your brains.\"

And he said, \"Well, madame, what if the child had your brains and my
looks?\"

So I can but hope that the outcome here is useful to us. I cannot delve
into all the fine details of the model. So what I\'m going to do is
focus on several design aspects.

Concerning particularly the interaction between the engineering and
econometric approaches, I\'m going to focus on the models themselves
rather than your projections because, after all, the projections flow
from the models, and if I focus my attention on the models, that is more
valuable, I think.

I\'m going to talk mainly about the residential model. However, several
of the comments I have with respect to that carry equally over onto the
commercial mode, and I will, however, finish with one or two comments on
the commercial model itself.

Let me deal first of all with the consistency between the econometric
and engineering aspects, and I want to discuss just briefly three
aspects. One is the price elasticities; secondly, the issue of
technological change; and, thirdly, the issue of reversibility.

As I read the description that I have of the models dealing first with
price elasticities, the space heating and cooling energy consumption are
assumed to be the components that are affected by prices in the short
run, and the short run price elasticity of minus 0.15 is currently
employed.

That to my mind raises five issues: firstly, whether other end users,
such as water heaters, dryers, and washers, would not be affected as
well; whether the impacts are confined to just own price effects or
whether they include cross-price effects; whether the short run
elasticity is very strictly speaking short run in being restricted to
intensitive (phonetic) use of the existing stocks; how the longer run
dynamic impacts of price changes are accommodated; and also the origin
and nature of the short-run price elasticity used.

Now, typically a short run elasticity picks up not just changes in
intensity of use in the short run and whether it\'s one year or
whatever, but we\'ll talk about one year, but also it does pick up the
first period impact on replacement and new appliance stocks.

Let me just by illustration put up \-- Richard, if you could just put up
a simple model here, this is just purely for illustrative purposes. I\'m
not suggesting this is the kind of model you would want to use, but
it\'s just to illustrate a point I want to make.

This is a simple dynamic, double logarithmic model where the variables
are expressed in logarithmic terms, and that being the case, the
coefficients of the model represent elasticities. So the short-run
elasticity that you see there for price would be the coefficient B. You
can show that the long-run elasticity will be given by the ratio of B
divided by one minus C.

C is analogous to the retention effect or maybe in certain models it can
actually be one minus the rate of depreciation. So that if C were zero,
that means all of the adjustment is in the first period, and so the
long-run and short-run elasticities are the same.

The larger C is, the longer the period of adjustment to a given price
change.

Now, the point I want to make is that you have a short-run elasticity
which for the sake of argument we\'ll assume is B, is minus 0.15. You
have a price change either up or down. You plug that into your model for
the given category of consumption.

Now, suppose that higher price or lower price is sustained in the next
period. If I read the model description correctly, you wouldn\'t
register any change induced by price because there\'s been no change in
the price between Period 1 and 2. However, if you think of the dynamics
of the model, what you do is you have another impact from the first
period price change, which is how you get all of these dynamic impacts
to asymptotically become B over one minus C eventually as the thing
works through, if the price is maintained at that level throughout.

Now, what is not clear to me is whether that kind of dynamic impact is
imbedded in the model. If so, it\'s not clear from the descriptions that
I have read as to just how it does emerge.

The direction of the impact, of omitting the dynamic impacts, if, in
fact, they are omitted, is in the direction of reducing the impact of
price changes either up or down, but we\'ll see in a moment another
comment I have is maybe an offsetting element to that.

The short-run elasticity reference that I have for how do you get the
minus 0.15 was taken from a report done by Carol Dahl for EIA about a
couple of years ago. I checked the reference there, and it said that
that was the elasticity for natural gas across all sectors. So if my
reference is correct, the question that arises is to what extent is that
appropriate for the residential sector rather than the average across
all sectors, and secondly, the extent to which the natural gas
elasticity is appropriate for other fuels.

As an order of magnitude, when I see that kind of number, it doesn\'t
kind of trigger something in my mind that says, \"Hey, that\'s too
high,\" or too low. It\'s just I\'m trying to reconcile your source,
apparently source, cited for that elasticity and the component of
consumption to which you\'re applying it.

Technological change. As I understand the model, these are largely
treated autonomously. An example here would be how he treated the
building shell efficiencies that gradually improve over time. The
concern I would raise is the interaction between that and any
econometric elements of the models, and that is because if you\'re using
econometric estimates of, say, price elasticities, they themselves are
influenced by changes in energy efficiency over time.

I mean it\'s no surprise that interest in energy efficiency became
stimulated by the substantial increases in prices in the 1970s and early
1980s. So to the extent to which any econometric elasticities are
affected by changes in energy efficiency and, therefore, embody them,
and then you have an autonomous technological change that also is
depicting improved efficiency, there could be some form of double
counting there.

And if that were the case, that would tend to over-estimate impact. So
it would be the opposite side of the coin to the way that I\'ve
suggested, a lack of attention to the long-run dynamic aspects may
underestimate.

Reversibility. That issue arises if you have an increase in energy
prices followed by a decline in energy prices or the other way around.
Are these responses reversible? In other words, if, say, the price goes
up and then returns to its original value and nothing else changes, do
you get back to consumption at the original value?

There are various reasons why that would not be so. Irreversible
technology, and maybe you capture that in model specification; mandated
efficiency standards that don\'t revert, are not responsive to changes
in the conditions. There are new operational modes, become imbedded.
There\'s changes in what is terms bounded rationality, whereby, say, in
the 1960s nobody cared too much about energy budgets, and in the 1970s
they became very concerned about those, and when prices went back down
in real terms to figures that are not far off what they were in the late
\'60s, early \'70s, nevertheless, people are still very aware of energy
costs and, therefore, would not revert to their original behavior.

The other aspect is that just because prices have switched direction in
terms of price expectations, people don\'t necessarily think that\'s
going to be the case throughout. So expectations may affect things as
well.

These are all reasons why you\'re not going to necessarily get
reversibility. I don\'t know what sort of dynamics are imbedded in the
models, but maybe by experimentation you can see whether, in fact, this
imbedded assumption in the model that assumes reversibility. If so, I
think it\'s something that should be adjusted.

Let me make some other comments. On the residential model, there\'s two
housing vintages where you kick off from. There\'s pre-1991 and
post-1990. For the future you keep track of the start by vintage, but as
I read the description in the residential model \-- there\'s a different
aspect in the commercial model which I\'ll come back to \-- you seem to
treat the 1990 stock on a kind of average basis.

If that is the case, it seems to me you could by a process \-- as long
as you have data on housing starts, you could kind of backtrack and get
the historical vintages of the housing start.

You mentioned that the equipment retirement rate in your model depends
on a linear decay function. I assume that the way that function operated
is it operated between the minimum and maximum lives that you have, and
you weren\'t implying that you\'d have an attrition from the first year.
That would be like a flat function and then a reduction.

All equipment in the residential sector, except that for space heating,
seemed to be replaced by the same equipment types, although that may
well have improved efficiency, but I think that assumption is a bit
restrictive in that water heaters, washers, dryers, and ranges could all
switch as well.

The real discount rate that you used of 20 percent in life cycle costing
for choosing among technologies to my mind appears a bit high, but that
may have been picked based on observations about consumer behavior, and
I\'d be interested in your comment on that.

I\'m not sure I fully understood the relationship between the macro
model and these components, both the residential and commercial sector
in that as I understand it energy prices are exogenous to the two
residential and commercial models. What I\'m not clear on is the way in
which changes in the residential and commercial energy demand
aggregates, which of course are a sizable proportion of the total
aggregate, react on the price projections.

You\'ve got around very neatly in the way the problem of what I call
simultaneity in identification, but I wasn\'t clear on the relationship
between aggregate price formation and the demand in the individual
sectors.

I also think there\'s a hidden assumption in the model, and you touched
on this in your presentation, John, that changes in mandated efficiency
standards always act to reduce energy demand. That\'s kind of
intuitively appealing, but there are quite a lot of references in the
literature to why the impact of mandated efficiency standards may be
muted, and that is because if you increase the efficiency of an
appliance in a mandated way, what you do is make the output BTUs cheaper
for the user because he doesn\'t need to buy as many input BTUs to get
the same level of service.

So you have reduced the price of output BTUs. That being the case, some
of the efficiency impacts on energy consumption can be muted and even
conceivably exceeded by the fact that people will use the appliance more
intensively or they may buy a bigger appliance.

So I think that indicates there\'s some caution to be used in assessing
the impact of changes in efficiency standards. It depends on how your
model operates.

Now, let me finish by making a few comments about the commercial sector
that are solely related to it. A lot of the comments I\'ve made on
pricing dynamics, elasticities, et cetera, carry over equally to the
commercial model.

The first one that\'s really commercial specific is cogen electricity.
That is included, at least for commercial establishments, in the
commercial aggregate data. My suggestion would be it would be preferable
for you to exclude that and put it in the power generation sector.

Discount rates, I see those as treated in a much more elaborate way than
in the residential sector. You\'ve got 11 consumer time preference
premia over risk free rates. The question that brings to my mind is why
that sort of or why several premia might not also apply in the
residential sector, and there is some literature on that.

Floor space vintaging. As I read the description, your pre-1989 floor
space are calculated by what you call back-casting, and I think maybe
that is the kind of technique that could be applied to the residential
sector if it hasn\'t, in fact, been done because what you\'ve done is
back off from 1989, your earlier vintages, by using the floor space data
that you have for earlier years as additions to floor space.

Energy price projections. I saw in the description where you\'re doing
the full life cycle analysis that you depend on what you call foresight
routines, that kind of myopic or reductive or perfect foresight. What I
didn\'t understand was that you already have price projections from the
macro model, and there seems to be a distinction between the price
projections that you\'re using for the life cycle analysis and the price
projections that may flow from the macro model. I would have thought you
were going to make them consistent.

A final comment is on DSM links. I see that the commercial sector model
does make specific reference to DSM linkage. No such treatment is
advertised for the residential sector, although I may have missed it in
the description, but if, in fact, it is not included, then I think it
should be because particularly in, for example, the electricity sector,
the impact of DSM programs is certainly noticeable.

CHAIRMAN MOUNT: Thank you, Campbell.

So anybody from the Committee who wants to add to Campbell\'s comments?
Dan.

MR. RELLES: These aren\'t exactly adding to Campbell\'s comments.
They\'re taking a different dimension.

There\'s a lot of statisticians here, and we\'d be guilty of malpractice
if we let you talk about the year 2015 and we just sat here and didn\'t
raise the specter of uncertainty. I want to say a little bit about that.

I know, on the one hand, you do deal with uncertainty by varying your
base cases, and you said that up front, oil prices, GDP, and so forth,
and in fact, the one time I tried to run NEMS on the Web, I was also
given some choices. Do I like these data values? Do I want to change
them?

So I appreciate that, but there are other sources of uncertainty that
don\'t get reflected in these projections. Coefficients have gotten
estimated. Surveys have gotten compiled and models estimated based on
those, and I guess I\'d like to see some discussion in these things
about uncertainty or even see NEMS provide the capability of providing
some kind of uncertainty measures.

I don\'t think it\'s feasible to think about reprogramming NEMS to do
that. I appreciate that it\'s awfully hard to communicate uncertainty in
a bar chart, but the kind of uncertainty I\'d be willing to live with
would be if I could run NEMS; if I could plug my assumptions in and rum
NEMS five or ten times, having it each time vary the parameters a bit
and letting me, you know, select my output and just try to see how it
varies over those different ten iterations.

That idea of varying things ten times and looking at it has gotten well
accepted into the statistical literature. Concepts like multiple
imputation in the end basically say, \"Go ahead and re-impute this
database five or ten times and see if there are major changes.\"

So it\'s a fairly well accepted notion, and I guess I\'d like to put a
plug in for instead of having it ask me do I want to change this data
value, put a plug in to have it ask do you want me to change all of my
parameters by amounts suggested by random variation, and I would most
certainly say yes to that because I really have no idea of what the
uncertainty is when it comes out, when out comes a projection.

MS. COX: Just to remark on his remark, and that is in terms of what he
was saying about or what was said about modeling uncertainty, it is a
good point for some of the things that I do when I\'m just guessing on
the size of parameters or what effects might be, et cetera. I\'ll go in
sometimes and say, \"Well, okay. I thought the cost ratio was going to
be this, but suppose it\'s ten percent more or 25 percent more.\"

If you don\'t see a whole lot of reaction, you feel pretty good, but if
it\'s bouncing all over the place, then you start wondering and not
feeling a lot of confidence in the results.

CHAIRMAN MOUNT: Well, I think it\'s very important that you both raised
this issue of uncertainty since this is a favorite topic of this
Committee, and I think it has got lost.

(Laughter.)

CHAIRMAN MOUNT: And I think it\'s important to bring it back again to
the forefront.

PARTICIPANT: I\'m a little surprised it wasn\'t on the agenda.

(Laughter.)

CHAIRMAN MOUNT: But there\'s a point that I would like to raise about
the use of end use models. In particular, the fact that you are using a
relatively high discount rate in order to approximate what is actually
going on suggests that people are not making energy efficient decisions
when they make appliance choices, and I wondered whether you could say a
little bit more about this particularly for new construction.

Is it because, for example, the people who are making the decisions are
not the people who are actually going to use the buildings or are there
other reasons?

I think that with a model like this, the possibility exists unlike many
econometric models to actually address this type of issue and to say how
well are we making energy appliance and equipment choices now, and in
particular, it\'s surprising how much penetration is being \-- of
electricity into space heating and things like that, given the prices
that we pay for it in the Northeast particularly, where heat pumps
don\'t work very well.

So ny other Committee members want to comment?

Anybody from the public want to make a comment? Would you like to
respond?

MR. CYMBALSKY: I don\'t know where to begin with all of those comments.
I\'ll begin with the one I can answer.

(Laughter.)

MR. CYMBALSKY: The discount rate question has been around forever. So
why don\'t we go for that one first?

As Campbell mentioned, he said the commercial model segments its
population into 11 different discount rates. Six? And his comment was:
why doesn\'t the residential sector do the same thing?

And tying into what Tim said, why do we have these high discount rates?
Well, what we do in the residential model is we have one segment of the
population. That\'s it. We don\'t segment it by owners or leasers or
builders. We have one, but what we do have is good shipment data from
the appliance manufacturer. So we actually know what\'s being purchased
on the market.

I\'m going to agree with Tim and say, well, if you have the new
construction, they tend to do things differently than someone who is
going to live in the house and they stock their own appliances. They
purchase higher efficiency goods. That\'s definitely true.

I think the problem we have is estimating different discount rates in
these different classes. So we used one discount rate per appliance and
said, well, the average of the whole residential sector is, for
instance, 69 percent for refrigerators.

Does that mean everyone has a discount rate of 69 percent? No, but when
a builder builds a house, he doesn\'t care what the efficiency level is.
So he throws in a bad one. A consumer going out may buy a better one,
but on average, you get the 69 percent. So that\'s what we do there.

Uncertainty? I\'m certain that I\'m uncertain about pretty much
everything you said there. We did have a project on uncertainty, and I
think with budget considerations that actually did not go through in our
plans.

You know, to address his point about uncertainty, you can take the
residential PC models and perturb a lot of different things, not just
this data input set. I think you\'re referring to the spread sheet where
you have different housing levels maybe or different prices.

You can change a whole lot of things. It doesn\'t have a switch to just
say, \"Change them all by ten percent.\" You know, that capability may
be a little bit tough to do, and I\'m not sure how the model would
respond to something like that. It may blow up; it may work. I\'ve just
never done it, so I don\'t know.

Let\'s address the housing stock issue in terms of the vintages. There
are data that are available through RECs that segment the existing
housing stock into different vintage levels. When we were developing the
model, we decided to only use the two vintages, basically what existed
the year of the last survey and whatever else is going to be built in
the forecast period. So we have two vintages and we keep those separate
over time.

We can segment the \'90 stock, you know, by different vintages. I don\'t
know what that would really buy us in terms of our projections. That
would be very data intensive. You could see how many pieces of data we
already had, and then you would just segment all of those numbers again
by how many vintages you would want in the existing housing.

The things that would come to mind, you could probably track age of
equipment a little bit better. You can track how efficient the building
shell is for different vintages of housing a little bit better. Yeah,
it\'s a fair concern. I\'m not sure in terms of the amount of data
needed in the model. It would just expand the dimensions by, you know,
probably four or five.

The reversibility issue, I think this is the last one I\'ll talk about.
Maybe Erin wants to address.

The reversibility of all the equipment purchases, they are irreversible,
and you know, once you make a decision to improve your housing shell,
for instance, you don\'t make the decision to take it back out. So once
there\'s a price response, say, to increased insulation in your home,
once you do that, it\'s done forever no matter what else happens to
price after that time period.

And I guess I\'ll end up with my comments on talking about what we call
the rebound effect, and you mentioned, Campbell, that, well, if you do
these efficiency gains, your outputs go down, so you should have some
sort of rebound effect. That is captured in the model now, in fact. So I
think the version of the documentation maybe didn\'t include that, but
we do have the rebound effect in the model.

MS. BOEDECKER: I\'ll be very brief, but to continue what John was just
talking about, the commercial model, too, did just incorporate a rebound
effect for this year. So we do take that into account to a certain
extent.

And as far as reversibility goes, as long as a piece of equipment meets
a standard, the next time that person goes out to buy equipment, they
can go back to a less efficient piece of equipment.

Also, as in the residential sector, if they make shell improvements,
those improvements are there to stay, and they won\'t rip the house back
out.

I\'ll just address one more thing since John touched on most of the
other things. On the price projections, noticing that the forecast
prices used in the life cycle cost calculation were different than those
that we get from the macro model for our consumption calculations, the
forecast prices used in the life cycle cost are supposed to represent
the consumer trying to make a forecast into the future, what they expect
the prices will do.

The prices we get from the macro model we get every year. So if we are
running an integrated run of NEMS, those fuel prices that we get from
the macro model may change from one year to the next, depending on how
demand and supply does change. So they will not be, quote, unquote,
foresight prices until the year of the model run.

CHAIRMAN MOUNT: You\'ve got one follow-up here?

MR. GRACE: One simple comment, maybe not simply; simple in theory,
perhaps more difficult to implement. We\'ve had a lot of success in the
uncertainty area in dealing with these deterministic models to build \--
I\'ll say simply build \-- a shell around the model that allows either
in a more sophisticated sense drawing in a Monte Carlo from
distributions or simple up ten, down ten, up five, down five, but you
don\'t end up having to screw around with the model guts. You\'re just
building a big shell around it that says: go to this array. Draw from
some values. Maybe it\'s a distributional array or maybe it\'s just some
nominated array, but the mechanics of doing it aren\'t as daunting as
they might seem if you\'re thinking, \"Oh, gosh, now I\'ve got to make
this whole general equilibrium model stochastic.\"

And you get pretty much the same effects, at least in stability and some
sort of idea about the uncertainty surrounding it, and they should
propagate forward in time.

CHAIRMAN MOUNT: So we\'ve had a proposal to change the plan slightly and
have a break now before Jerry\'s presentation. So let\'s have a
15-minute caffeine shot and be ready for Jerry.

(Whereupon, a short recess was taken.)

So first I\'ve got an announcement for Committee members who are going
to dinner this evening, that we were not able to get into Le Rivage, and
we\'re now going to go to 701 Pennsylvania Avenue. This is not the same
place. It\'s 701 Pennsylvania Avenue. We have the private dining room,
and we have to order before seven o\'clock to get the theater meal. So
be there on time, please.

MS. COX: To get what?

MR. HAKES: The cheap prices.

MS. COX: Oh.

CHAIRMAN MOUNT: The cheaper prices, less expensive prices, and there\'s
a very important announcement for your TV pleasure this evening, that
Jay Hakes is going to be live on CNN at ten o\'clock to tell us why
gasoline prices are going up.

MR. HAKES: Any help would be welcome.

(Laughter.)

CHAIRMAN MOUNT: He will be soliciting comments at dinner.

MR. LOCKHART: Will it be the same story as this morning or a different
story?

(Laughter.)

CHAIRMAN MOUNT: So the first presentation now is an update on
confidentiality by Jerry Coffey from the Office of Management and
Budget.

MR. COFFEY: Thank you.

As some of you know, I\'ve been working on bits and pieces of this for
most of my professional career. So sometimes I give quite a long speech,
but I\'ll avoid that today and give you the quick version, which is the
version that focuses on the successes.

I\'m going to talk about three things today, two of which are now
public, one of which should be public some time in the next 48 hours, I
hope.

On Wednesday of last week, Alice Rivlin, the Director of OMB,
transmitted to Congress a bill called the Statistical Confidentiality
Act, which I personally have been working on off and on for about 18
years. This bill does a number of things, and I\'ll be happy to go
through some of them.

Really, you all have copies of this. Rest assured that this is the
result of many, many person-months of haggling with our attorneys, and
there are lots of subtleties in the language of both the bill and the
administrative order which bears some thinking about, and I\'m sure as
you look at these things and read them closely, there may be questions
that will occur to you.

What I would like to highlight are some of the issues that are already
being bobbled by some of the pundits. The bill itself has a statement of
findings and purposes, which hasn\'t changed that much over most of the
time I\'ve worked on it. We really are after the same things, and in
fact, recently I pulled out some very old documents and made copies of
them for Cathy Waldman, the Chief Statistician, to remind her just how
long some of these concepts have been around.

I had a page from the 1971 report of the Commission on Federal
Statistics, another page from the \-- actually pages from two different
reports issued in, I think, \'78 and \'79, one by the Privacy Protection
Study Commission and the other by the Paper Work Commission, where you
could find language that looked like we had copied it verbatim, and in
fact, we did in some sections of this bill, particularly the piece that
was written by the Paper Work Commission, which was the last of those
three.

The privacy issue dealt with both statistics and research and had some
complexities in it that we don\'t have to worry about when we\'re
dealing strictly with statistical agencies.

There are a series of definitions. Most of them are fairly
straightforward. A couple of them I would note. We do use an unusual
definition of \"agency.\" It includes most of the things that most laws
treat as agencies, but in addition, covers some unusual agencies, like
some of the groups in the national laboratories are included in this,
and there\'s an explanation of why we did that in the analysis of the
bill.

The other concept over which we have haggled endlessly for several years
now is the idea of an agent. What this really means, there are a lot of
words here. There are other words in the administrative order. There are
explanations in both places. What an agent is in the simplest terms,
it\'s anybody who does anything helpful for a statistical agency and
whose behavior can be controlled at least to the extent of assuring
information security.

The reason we have such a broad definition is that statistical agencies
have come up with ways of solving this problem of how do we handle
contractors or how do we get people with specific skills in that may
help us work on a particular project in so many different ways that we
needed a very broad definition in order to capture them all.

Why do we want to capture them all? We are proposing in this bill to
make agents with this broad definition subject to the penalties of the
Trade Secrets Act. For those of you who may not be familiar with the
Trade Secrets Act, it was actually derived from a piece of tax law, an
old BEA statute, and one other which I\'m not quite sure of many, many
years ago, and essentially it establishes criminal penalties for
employees or officers of federal agencies who make unauthorized
disclosures. It is probably the one most general piece of the criminal
code dealing with disclosures and the penalties for making inappropriate
disclosures.

There are some other changes we are going to make in that statute which
I\'ll get to in a moment.

But the concept of agent didn\'t really authorize anything new. In the
case of the Census Bureau, they have a section in Title 13 that allows
them to sort of swear in people as if they were employees. They use it
to bring people in from universities and lots of other folks and
essentially impose employee discipline on those people.

There are other statistical agencies that are dealt with in this bill
who have many other different strategies. There are licensing
strategies. Certain kinds of contracts are written, and the flavors of
contracts are quite wide in their variety.

There are in a few cases situations where a federal agency actually pays
the salary of someone operating out in the state and supervising
employees of a state agency. This is a NASS agriculture invention which
they\'ve had a long time to think that one up, but it works very well. I
don\'t know if anybody else could come up with a scheme like that any
time in this century, but is has worked very well for NASS.

The responsibilities of the statistical data setters, which are the
agencies that are defined in this act who have special authorities to
get data for exclusively statistical purposes from almost anyone. Their
responsibilities are laid out there. They\'re pretty straightforward.

The agencies themselves are named in Section 4 of the bill. There\'s
Bureau of Economic Analysis, Bureau of the Census, Bureau of Labor
Statistics, the National Agricultural Statistics Service, Department of
Agriculture, the National Center for Education Statistics, National
Center for Health Statistics, the Energy End Use and Integrated
Statistics Division of the Energy Information Administration, and a very
late-comer, the Division of Science Resources of the National Science
Foundation. This was one of the very late changes that we made to the
bill in the last couple of weeks. We\'ve had a very interesting time in
the month of April.

The agencies that are named here are, as I say, authorized to acquire
information for exclusively statistical purposes from almost anyone else
in government. There are a few exceptions which are in the bill.
Particularly national security information is out of bounds. Information
which is controlled by a statute that itself sets specific limits on how
statistical information may be used also have to be observed in any kind
of arrangement for sharing data.

The main body, the main policy of the bill is in Section 6,
confidentiality of information, and this is where we have some of the
phrases that go back for decades. The basic policy: \"data or
information acquired by a statistical data center for exclusively
statistical purposes shall be used only for statistical purposes. Such
data or information shall not be disclosed in identifiable form for any
other purpose without the informed consent of the respondent.\"

There are some slight changes in wording that came along very recently
in there. There are provisions in here for situations that arise in
almost all of these agencies where information is collected for both
statistical and other purposes. In those cases what we want the agencies
to do, the policy that we\'re establishing is that distinctions of that
type have to be made on the record by rule before the data is collected.
You can\'t collect the data for exclusively statistical purposes and
tell all of the respondents that\'s what you\'re doing and then change
your mind later. You\'ve got to do it on the record before the data is
collected.

This also contains some of the exceptions, kinds of information that
cannot be disclosed. It also has a fairly elaborate procedure for data
sharing agreements, and this has a long history, and every word has been
fought over at one time or another. I encourage you to read it. I doubt
if I could explain it in less than an hour.

One of the pieces of sub-text that you should understand and which you
don\'t always see explicitly in the language of this bill, there are a
number of areas where this bill has been written to dovetail with
provisions of the Paper Work Reduction Act. In some cases, we actually
restate policies of the Paper Work Reduction Act, one of them being
Section 6, Subsection F of Section 6, which guarantees that any
restrictions on the use of data in law travel with the data to any other
agency.

An interesting sidelight here is when we originally picked up the
language from the Paper Work Reduction Act, I discovered that there was
a glitch in it. It didn\'t really work. So we fixed our version, and in
the last stages of the reauthorization or the 1995 amendments to the
Paper Work Reduction Act, the House and the Senate decided we were
right, and they changed the Paper Work Reduction Act to match this bill,
though they didn\'t even know this bill existed.

There is a section on coordination oversight. Because a lot of this
approach is procedural, we\'re setting up ways of sharing information,
ground rules for sharing information, permitting agencies to find the
best answers for how they manage information in this kind of an
environment. A lot of this we\'re going to have to invent as we go
along.

One of the things we learned over a decade and a half is that we cannot
write prescriptions that solve all of the problems and remain consistent
with all of the statutes that are on the books already. We tried that in
1983, and we had a bill about this thick, and nobody could read it;
nobody could understand it; and certainly nobody supported it.

So there is a process in here for coordination of oversight. The main
engine of this will be the Director of OMB. There are a number of tasks
there, including the task of reviewing and approving any rules adopted
by any of the agencies that are affected by this bill.

Any agency that donates information for exclusively statistical purpose
to one of the named statistical data center, and of course, the
statistical data centers themselves, may, in fact, be writing policies
as regulations for how they are going to manage this. OMB has the task
of looking at these and assuring that they are done in a consistent
fashion.

What this gives us is the opportunity to look for the kinds of problems
that have gotten in the way for a long time when an agency that has a
solution that works reasonably well for them, but it prevents you from
doing something that\'s very useful somewhere else, and we felt this was
absolutely necessary.

Another important principle is generally rules discovering disclosures
of information that are authorized by this act are promulgated by the
agency that originally collected the information. The presumption here
is the agency that had the mandate to get the information in the first
place probably best understands what the problems are with the data,
what the sensitivities of the respondents may be, what problems they
would have if somebody made a mistake and disclosed something that
shouldn\'t be disclosed. So they are the ones responsible. They have the
primary responsibility for drafting these regulations.

Each agency writes rules for the data that it collects. This was a very
important principle for winning over the Treasury Department, who had
always looked at this as this is a way for OMB to rewrite all of the
Treasury Department\'s rules, but that\'s not what we intended ever.

A good bit of the text of the bill is then in the series of conforming
amendments in Section 9. There are a couple of real short ones for the
Department of Commerce covering the Census Bureau and BEA, one very long
one for the Department of Energy, beautifully drafted. I was told before
I ever got started in this that there were some people in the General
Counsel\'s Office of Department of Energy who were probably the
government\'s best drafts-persons for legislation, and I was certainly
impressed with the job that they did. We only made really one change in
what they sent us, except for adjusting the indentations and things of
that sort, and that was another one of these fire fights that occurred
actually after the bill had been sent forward for Alice to sign off on.
We made one slight change affecting the Department of Energy piece.

We then have some new conforming amendments. These weren\'t in there a
couple of years ago, weren\'t in the version that we circulated last
year, one for the Department of Health and Human Services and a very
short one for the Department of Labor.

Any of you who have seen earlier versions of this, the conforming
amendment for the Department of Labor has gone up and down and up and
down several times. We had a three paragraph version that we wrote with
some members of Janet Norwood\'s staff years ago. Shortly before we went
out for review on the previous iteration of this, which was about a year
and a half ago, Labor said, \"We want to change it,\" and they took the
three paragraphs out to about three pages.

They came in again shortly before we were going to circulate it this
time and they wanted to restore some of the stuff we had taken out last
time. This went round and round. Finally we reminded Labor subtly that,
in fact, no conforming amendment was needed for the Department of Labor
and that if necessary we could go forward without one, and after some
discussions of some very subtle changes in the main language of the
bill, which solved some problems their solicitor was concerned with, we
ended up with an amendment that\'s one sentence long.

The other new feature here, and this was another late riser, was the
amendment for the National Science Foundation. A year ago, when we
talked to NSF, they really weren\'t prepared to become part of this
policy effort. However, they went back and started thinking about it. We
told them, well, maybe the first round of amendments a couple of years
from now we can do something.

They went through the process. They spent a lot of time with their
General Counsel. As a result, when we went out for comment on the bill
about a month ago, their comment came back a completely coordinated,
conforming amendment to the section of Title 42 that deals with the
National Science Foundation, very well written, well put together,
completely coordinated with the General Counsel, and a request that we
add the Science Resources Division to the list of statistical data
centers, and they had done such a beautiful job that we said, \"Fine.
That looks great to us.\"

There is one other important conforming amendment that\'s in Section F
disclosure penalties on page 14 of the handout. Section 1905 of Title
18, this is the Trade Secrets Act. What we have done there, and this
looks a little different from \-- oh, my goodness, this is not even the
right version. Oh, no, that\'s all right. Okay. I have to be careful
with this one because we changed the language on the recommendation to
the Department of Justice in the last three days before we went forward.
Ah, yes, okay. This is the correct one.

Initially what this law said, it imposes penalties on officers or
employees of the government, and there\'s also a special category in
there which are agents of the government, which I think it was put in
there by a unit of the Justice Department for certain kinds of agents
that they have that they wanted added to this.

And what we are doing is simply adding this broad agent concept that
we\'ve defined for these statistical data centers as another form of
agent who can be punished as if they were officers or employees, and
then the other thought we had was, well, this is really old law. It says
you can only be fined \$1,000 for something like this, which isn\'t very
much these days. So we sort of poked around and originally proposed,
well, we ought to increase it to 10,000.

When Justice came back, they said, \"Well, why don\'t you just make it
conform to the current sentencing standard for what this is, which is a
Type A misdemeanor?\"

I said, \"Well, okay. What do you want to do?\"

They said, \"Well, you look in this section and look in this other
section, and by simply tying the fine to the title in which this is
placed, you have the effect of tying it into whatever gets done with the
sentencing standard.\"

I said, \"Well, what is the sentencing standard?\"

Well, it\'s \$100,000, which probably makes a lot of sense considering
what we\'re actually talking about here, but it kind of made a lot of
our agonizing debates over whether it\'s going to be 10,000 or 15,000
look kind of silly in hindsight, and this was, in fact, the issue we
came back to EIA with.

I called Larry Klerr after the bill had been sent forward through
coordination in OMB. It was halfway to the Director\'s office, and I got
a call from LRD and said, \"Hey, there are two of these conforming
amendments which still have this \$10,000 in there. Do you want to keep
them or not?\"

And I said, \"Well, I don\'t know.\" In the case of the National Science
Foundation, they had different reasons for establishing a \$10,000 fine.
In the case of the EIA amendment, they had basically just lifted the
language that we were using at the time that they wrote the amendment,
and I called Larry and said, \"Hey, go talk to your General Counsel and
see if you want to take this ride with us up to 100,000 on the fines.\"

And 20 minutes later Larry had it all coordinated and came back and said
yes, and we sat there on the phone and I got the LRD guide and wrote in
the new language. I then made replacement pages, went up and caught the
package over in the Director\'s suite, and put the new language. It was
that kind of week.

Finally, there was one other very important contribution that of all
people the Department of Defense made. In Section 10, effect on other
laws, we have always had a section in here which acknowledged the
relationship of this law to the Paper Work Reduction Act, particularly
Section 3510, which is the basic piece of law in the federal government
that controls exchanges of information between agencies. It sets the
general rule, and this reinforces the fact that this still applies,
though the way this new law is written, it sets some new boundaries.
It\'s drafted so as to work with the language of the PRA.

The suggestion that we got from the Department of Defense was that it
may still be useful. It\'s sort of \"belt and suspenders\" type of
stuff, but it may be useful to reiterate the fact that what we have
written as policy on confidentiality in Section 6 is intended to
constitute a (b)(3) exemption under the Freedom of Information Act.

We went around a little bit as to exactly how to word it, but everybody
agreed. Even Justice agreed that this was okay to do. What\'s happened
and the reason this is put in there is that there are a number of courts
when they look at FOIA cases who believe because the way FOIA is written
they do not have to look at the legislative history of an act in making
a determination as to whether the (b)(3) exemption applies, which is why
you have sometimes very elegant language which clearly states
restrictions that would qualify a law for the (b)(3) exemption, and then
somewhere down there there\'s another little piece of verbiage that
says, \"This is a (b)(3) exemption in case you didn\'t notice.\" So
that\'s basically what was added here.

The analysis probably does a better job than I\'ve done in the last few
minutes of explaining what all this stuff is about. There is a companion
piece that has not yet gotten out of the agencies. One of the things you
will notice if you are a follower of the various iterations of this bill
over the years is there is no amendment to the tax code in the main
bill.

What happened here \-- well, let me back up a little bit. Over the
years, the Census Bureau and Internal Revenue Service have taken turns
blocking us from getting this thing out of the administration. During
the Bush administration, we came up with a version of the bill that
essentially took all of the chips away from the Census Bureau. It
didn\'t get out of the administration, but the Census Bureau has been
very cooperative in helping us do a reasonable version of this bill ever
since.

On the recommendation of a lot of people, we were encouraged to make the
bill more ambitious than the one in the Bush administration, and as a
result, we tried to come up with a new version of the amendment to the
tax code in the bill that we circulated a year ago. Treasury still
didn\'t like it, but shortly after that bill was circulated, there was a
high level meeting involving the Commissioner of Internal Revenue,
Deputy Secretary of Treasury, and Sally Katzen, my boss\' boss\' boss,
where we came to an agreement on some principles for what might
constitute an agreement on an amendment to the tax code.

What we were offering them was basically some changes in language that
reduced at least theoretically the amount of information that would be
disclosed for statistical purposes, but disclosed kinds of information
that were much more useful than under the current language of the tax
code.

After seven or eight months of negotiation back and forth, IRS came up
with a form of this that we could all live with, and this is now \--
this was also circulated the same week as our bill and has, to my
understanding, now been completely revised to reflect all comments from
all agencies and is back at the Treasury Department awaiting signature
by their General Counsel before it goes to Congress, but I can\'t really
show you copies of that until it becomes official.

If anybody wants to talk about, you know, what the changes are, I\'ll be
happy to talk to you after, but I do want to mention one more piece of
this strategy which started later, but finished earlier than either of
the others, and that was our administrative order which really deals
with a lot of the same concepts that are in the bill.

One of the things that we decided to do when we were having difficulties
many years ago with getting agencies to agree on a strategy for changing
the law was we decided to look at how much we could do without changing
the law, and EIA was, in fact, an important player in determining how
far we could go.

There were some events that some of you in this room will recall in
which there were some disclosures that were being requested. There was a
lot of pushing and shoving going on within the executive branch that was
causing us all kinds of problems, and they really put a fine point on
the extent to which in some cases Congress and the law is not the
problem; it\'s the agencies and the way they behave to each other that
is the problem, and that\'s something we ought to be able to get a
handle on.

As a result of the case that was made all the way up to the White House
by EIA, some of the President\'s key advisors suggested that maybe we
should have an executive order that tells all of the executive agencies
to behave with respect to this particular problem in the future. We
couldn\'t do anything about that immediately. Well, this was in the Bush
administration, and we only had a few months after that before the Bush
administration was no more.

But the idea was planted, and we started drafting little things and
looking at things. What could we do that would work with the strategy
that we were pursuing with the bill?

On January 29th of this year, we put out for comment an order, not an
executive order, but an order providing for the confidentiality of
statistical information. Some of you may be aware that last year we
circulated a proposed executive order. After a lot of discussion with
our General Counsel, we determined that we could actually do more under
our existing authority, which goes back to the Budget and Accounting
Procedures Act of 1950 and has been since buttressed by an executive
order, a congressional requirement to delegate the authority granted to
the President which is reflected in an amendment to that executive
order.

What it all boils down to is we have a strong case that orders adopted
under the authorities that exist have the force of law. If any of you
doubt this, there is one spectacular case, which is Statistical Policy
Directive No. 3. Most of the statistical policy directives sound like
advice and are taken as advice. This one is not. This is the directive
that controls the release of principal economic indicators. It has been
treated as law by six Presidents. It is one of these really strange
situations where lowly working stiffs like me write something, go out
and talk to agencies about it, come up with a policy, and even the
President feels bound by it.

Even though it is based on authority assigned to the President, because
of the way it\'s operated over the years and also because of the good
sense of a lot of presidential advisors who understood why things need
to be done this way, we have a perfect record of compliance all the way
up to the President, and as our General Counsel pointed out, you don\'t
get that with a lot of laws.

So the point he made, one of the other points that was made is that
executive orders frequently change with administrations, which is not
the sort of stability that we were seeking. We really want, after all
these years of living with informal versions of this policy, we really
want to make something permanent happen, and that\'s what the order is
all about.

Tactically also, the order gave us an opportunity to put some of these
principles out on display for the public, out where Congress and
congressional staff would see them, would have a chance to think about
them, to talk about them, to debate them before we go to Congress with a
very similar strategy that requires them to take action and to pass
legislation, and this has generally been very effective.

A number of statistical agency heads and people who have a great deal of
concern with the way the government does its statistical work testified
as a hearing in March, along with Sally Katzen. Cathy Waldman was there,
but she was basically a resource for Sally, and that was one of the
things that clearly had made an impression on Congress.

They had looked at what we were trying to do in the confidentiality
order. There was broad agreement with the principles that we were trying
to pursue there.

Some important differences between what we do in the bill and what we do
in the order. I mentioned before that the bill is a (b)(3) statute. the
reason that the order has a lot more verbiage in it about how you get
things done and a lot of required reviews on the part of the agency as
to what statutes require you to do and what policy options are open to
you is because the order does not assume that you have a (b)(3) FOIA
exemption. What it is doing is, among other things, making a significant
change in the way FOIA policy operates in the case of some designated
agencies.

If any of you are followers of the way Freedom of Information Act law
has developed over decades now, one of the principles had been that
determinations as to what may be disclosed are almost always deferred
until a request for disclosure is made. If you think about it for a
minute, you can see this really makes a hash of a confidentiality
pledge. How can you go out and tell a respondent that this is not going
to be disclosed because it is very important. We know it\'s important to
you and we want you to feel confident the government is not going to
splash this information all over the place. It\'s going to treat you
with respect and respect both your confidentiality privilege and your
privacy rights, and yet at the same time, try to observe a policy that
says we\'re not going to decide until somebody asks us for it.

So part of what we have crafted in the order is a required change on the
part of some designated agencies in the way they deal with FOIA policy.
We require these agencies to make these determinations in advance, which
flies in the face of most FOIA regulations, flies in the face of a
couple of executive orders if you get right down to it, because FOIA
regulations and executive orders have all predominantly dealt with the
case of administrative information. Neither the Privacy Act nor FOIA
have ever really dealt adequately with the problems of statistical
agencies and particularly the concept of statistical confidentiality,
and we\'re trying to carve out some area where we can make sense of this
and make it work.

So this went out in January. The comment period ended the end of March.
I have some extra copies here. If any of you want to see it and you
don\'t get a copy, I can give you the [Federal Register]{.underline}
citation.

The comment period is over, but since I\'ve been busy writing the last
minute changes to the bill, we\'re going to be working on the comments
for a while yet, and if any of you get some inspirations, my E-mail
address is in the notice. It\'s probably easier if I tell you what it is
because the particular font I use here, the lower case Ls and the ones
are identical. The E-mail address if you have any comments on any of
this, and I\'d be happy to see them, is Coffey[jay]{.underline},
C-o-f-f-e-y, underscored Jay, at sign, A1.dop.gov.

As I say, the formal comment process is over. We are still, however,
talking with people who submitted formal comments, and if any of you
have real inspirations, I\'d be happy to look at them because there are
many, many issues that I\'m going to be writing up papers on that came
in in the course of the comment period, and any insight you can provide
me as a personal favor I would greatly appreciate. We can\'t treat them
as formal comments, but I\'d be glad to have them.

That\'s about it. How well did I do? Probably behind time. How much time
do you have left for questions?

CHAIRMAN MOUNT: Does anybody want to? Yes, Cal.

MR. KENT: Is the idea that the order and the law go together or is the
order seen as a separate act that will not require the law to be
affected?

MR. COFFEY: Both. We designed the order so that we could get substantial
benefit from it whether or not Congress passes the bill.

MR. KENT: Whether or not the law passes. Okay.

MR. COFFEY: We also designed it to be consistent and to work effectively
with the bill if Congress chooses to pass it in the way it\'s been
proposed. Some of the things you see in the bill you will eventually see
in the guidance for the order. I don\'t have as big a writing job as it
may appear because I\'m just going to lift whole chunks of definitions
and other things from one straight across to the other.

If you do get a copy of the order, by the way, there\'s one glitch.
There\'s one thing we missed in all of our interminable debates with our
General Counsel. Somewhere in here there\'s a phrase that says about
agents something about \"supervision and control.\" It should have been
\"supervision or control,\" which makes a big difference.

MR. KENT: And can I just ask one more question as a follow-up? Why is
this not applied to all of EIA?

MR. COFFEY: Good question. It was a tactical decision made some time
ago. The first question was could we apply it to any part of EIA.

The choices that we made in this bill were often influenced by what was
perceived to be on the edge of controversy or over the edge. In the case
of EIA, as you well know, changing the disclosure rules that apply
generally to EIA has been a meat grinder since the 1970s.

It appeared to us, at least, that even among the people who were dead
opposed to changing the way information about energy producers is
handled, it\'s to get the big boys, you know, the anti-oil company
mentality, which everybody is aware of, but that has clearly \-- in some
of the actions of Congress that has not applied to small fry, to
households, to small businesses when you\'re talking about retail
gasoline franchises. Congress has from time to time written in special
protections for the interest of these not so big boys.

We basically seized on that and tried to look at the largest chunk we
could define that would avoid that other controversy which had been so
intractable.

The other issue was, and what we have looked for in the designation of
each of these agencies, we are looking for units who, in fact, can do
something useful with data sharing, and the Energy End Use Consumption
division clearly in their work with several other agencies has laid the
ground work for doing some very effective data sharing. They\'ve had to
be very imaginative in many cases in how they did things, but clearly
there was a very real potential there which we could easily explain to
Congress if anybody asked us, and that was the other practical
consideration.

MR. KENT: Right, but this particular bill then would not deal with the
issue that we had in the famous White House meeting?

MR. COFFEY: No, there are other ways to deal with that, particularly
once we get everyone\'s attention with this order. The way disputes over
data sharing are supposed to be managed in the government, most of the
time agencies talk to each other and decide either you\'re going to give
somebody some data or you\'re not, and agencies actually have quite a
lot of latitude in how they make these choices.

The law that applies in these cases is Section 3510 of the Paper Work
Reduction Act, which authorizes agencies to make disclosures that are
not inconsistent with existing law. The way that language is written,
each of the people in this discussion makes a determination as to what
is inconsistent with law. If they can\'t agree, there\'s no agreement.

Congress then provided for another step to resolve such differences,
which is that the Director of OMB may order exchanges of information if
he or she determines that it is no inconsistent with the law. Once
again, this is an independent determination. This is the way we\'ve
interpreted it anyway, along with our General Counsel. If OMB can sit
down, they don\'t have to listen to what anybody else has said in their
regulations or what their policies are. They look at the law and make a
determination as to whether or not this is inconsistent.

Now, one of the dilemmas we\'ve had in this is that without either the
confidentiality order or the confidentiality law, there\'s no
distinction in many instances between statistical uses and
non-statistical uses. So if you read 3510, it seems to say, well, OMB
should be deciding that statistical agencies should be central data
collection agencies for the Federal Trade Commission or, you know,
anybody else who has an interest in similar information.

One of the things we do in both the order and the statute that we\'re
proposing is we finally build into law the concept of functional
separation that\'s been pushed for 25 years now so that when OMB looks
at these determinations under Section 3510, we can legally and
rationally determine that we may have a central collection agency for
statistical uses and another central collection agency for
administrative uses of very similar data, and it would not be
inconsistent because here we would have provided for the basis for
making such a distinction, the functional separation policy.

MR. KENT: So the order would take care \--

MR. COFFEY: You can do it under the order. You can do it even more
explicitly under the bill for any agencies covered in the bill.

MR. SKARPNESS: I\'ve got a sort question, a point of information. So
you\'re saying there\'s even parts of EIA that aren\'t going to be
covered by this?

MR. COFFEY: Right now all of EIA has very little legal back-up for its
confidentiality policies. Yeah, I tried very hard with some very
ingenious regulations some \--

MR. SKARPNESS: So across the street here is DOT.

MR. COFFEY: Okay.

MR. SKARPNESS: You know, were they part of or did they decline getting
involved?

MR. COFFEY: DOT also got into the game late, in part because DOT has
doubled in size and substantially changed its functions in the last six
or eight months.

MR. SKARPNESS: Oh, yeah.

MR. COFFEY: We have been in constant communication with the people in
the BTS, Bureau of Transportation Statistics. If they had been farther
along \--

MR. SKARPNESS: They might have.

MR. COFFEY: \-- this would have been an opportunity to do this. At best
now we hope things will settle out, and maybe the first time we get a
chance to go back and amend this and add another statistical data center
and show Congress how wonderful this whole strategy is working, BTS is
high on the list of the next set of agencies to be added.

MR. SKARPNESS: One last thing.

MR. COFFEY: Yes.

MR. SKARPNESS: There\'s been talk about a statistical clearinghouse
concept, you know, sort of a central location where, you know, this data
is cleared. Does that fit into this in any way? I know that each agency
\--

MR. COFFEY: Well, are you talking about a consolidation of agencies?

MR. SKARPNESS: Well, no, of just data more or less.

MR. COFFEY: I\'m not familiar with that strategy.

MR. SKARPNESS: Okay.

MR. COFFEY: I am familiar with a bill that\'s been discussed on the
Hill.

MR. HAKES: You\'re talking about a central place where you can go for
data?

MR. SKARPNESS: Yeah.

MR. COFFEY: Oh, oh, that\'s a different story.

MR. HAKES: We can create that create that electronically.

MR. SKARPNESS: Right.

CHAIRMAN MOUNT: A virtual one.

MR. SKARPNESS: Yes, where it fits these\--

MR. COFFEY: That\'s happening.

MR. SKARPNESS: Okay.

MR. COFFEY: Yeah, that\'s happening right now.

MR. SKARPNESS: I mean is this facilitated or you know?

MR. COFFEY: It should have happened already, but you know, there\'s been
a lot of back-and-forth in the White House about when they\'re going to
roll it out and who\'s going to be there.

MR. HAKES: It\'s kind of encouraging because it\'s going to be a fairly
prominent part of the White House Home Page. So a person coming into the
federal system will see the statistical resources early in the process
without getting much into the system, which should increase the
visibility of federal statistics.

MR. COFFEY: Right. Our staff and the statistical agencies have all been
working on this one for a while. As I say, we sort of expected that it
was going to happen before now, but for a number of reasons it hasn\'t
quite been scheduled yet, but it\'s about ready. It\'s going to happen
soon.

Any other questions?

MS. COX: I as invisible.

(Laughter.)

MS. COX: I think this might come \-- as I understand it, what you\'ve
handed out here only affects the named agencies.

MR. COFFEY: The act itself names statistical data centers, yes.

MS. COX: Right. Now, for the order, is that \-- I haven\'t seen the
order. Can that impact on \-- yes, I would like a copy of it.

MR. KENT: Have you got another copy? I\'d like one.

MR. COFFEY: I have a few more. If I don\'t have enough I can also give
you the \--

MS. COX: Such as the Bureau of Transportation Statistics or maybe
another group that\'s concerned about the confidentiality for their
survey participants.

MR. COFFEY: Right.

MS. COX: I did work on a survey once where survey data that had been \--
they had said they were going to preserve their confidentiality, but
they had no legal right to do so, and it was subpoenaed under \-- I
can\'t remember. I think this was under EPA.

MR. COFFEY: Which agency?

MS. COX: They had a subpoena and had to release data.

MR. COFFEY: This was EPA?

MS. COX: I think it was EPA.

MR. COFFEY: Yeah, I\'m familiar with that situation.

MS. COX: And it was a very sad situation.

MR. COFFEY: Yeah.

MS. COX: But they had no way of preventing it.

MR. COFFEY: We went through this in great detail with both EPA and the
Justice Department when there were proposals to create a Bureau of
Environmental Statistics. One of the issues, and it is also an issue at
BTS, is whether this agency has a mandate to collect data for
statistical purposes, and we asked both EPA and the Justice Department
to consider whether this is something that should be added because all
of EPA\'s current data collection authorities all derive from regulatory
statutes.

The issue we put to EPA and Justice was: does it make sense? And we
cited some very important examples where, in fact, there were pitched
battles within EPA because they had to, in fact, guarantee very strict
confidentiality and guarantee that certain things were going to be used
only for statistical purposes in order to get some things done.

There were a couple of surveys where clearly they would not have
happened, would not have had good results unless they had done that.
Every one of them was a bloody battle with the General Counsel at EPA
because the general principle that they wanted to observe was anything
we get is available for enforcement purposes, and the point we made was
that if you say that, you\'re not going to get anything from these kinds
of surveys.

So we won the little battles. What happened ultimately though when we
talked to Justice about this was they raised a number of important
questions. Since Justice has a fair chunk of their resources invested in
prosecuting environmental violations, they\'ve got a division over there
that spends all of their time doing this.

The question that arises in that kind of an environment is: how can we
assure that information that needs to be used to prosecute an offense is
not tainted by the terms under which it was collected? And it\'s purely
a pragmatic kind of problem. It\'s can someone who doesn\'t like what
we\'re doing come in and say, \"Well, you didn\'t collect this correctly
because I thought it should have been used for only statistical
purposes,\" and even though the agency may have said otherwise, it\'s a
real hassle. It is something that attorneys have nightmares over, and
basically we just didn\'t want to fight with Justice any further on
that, particularly since the bill wasn\'t going anywhere.

Transportation has some of the same problems. They\'ve inherited two
regulatory mandates, along with staff. If you look at what they\'re
doing, a lot of the reason for the regulatory collection has
disappeared. They haven\'t done nearly the kind of job that EIA did in
the early \'80s to sort out what we need for statistical purposes from
the legacy of regulatory data collections. There\'s still a lot of
people in transportation who still think the same way about what their
information requirements are.

They don\'t quite know how they\'re going to handle that. Clearly
they\'ve got some staff work to do. There is some additional refinement
they\'re going to have to do to their organization. Right now it\'s an
agency that\'s in flux, and the one thing we cannot do is go to Congress
with promises. We\'re going to raise enough controversies as it is.
We\'ve got to go to Congress with agencies who are entirely credible as
statistical agencies.

CHAIRMAN MOUNT: When is the order going to be sent out?

MR. COFFEY: The order? When I have time. I\'m basically the resource for
both of these. We have two other permanent staff and a chief
statistician in our shop, but this is my baby, both of these.

CHAIRMAN MOUNT: Like a month are we talking about?

MR. COFFEY: A month, possibly longer. I have promised the agency heads
some issue papers on some of the extremely interesting comments that we
got. We don\'t anticipate a lot of changes in the order. You can see
that one of the reasons is we have to have certain relationships between
the order and the statute in order to make this whole strategy work, and
those things we\'re going to be very reluctant to change unless there\'s
somebody on the Hill who wants to make a similar change.

What we anticipate is going to happen on the order, we have gotten
comments now. We\'re probably going to go back to the statistical
agencies with some issue papers on a number of the comments, a number of
the issues raised, hash these out further, and then basically make
whatever hopefully small changes will need to be made to the order
itself, and then go to the next important step, which is going to be to
write the guidance that implements the order.

The order is written in fairly general terms. There are a lot of things
in here that we flesh out in the next stage of things, which is how
we\'re going to do this. This says what we\'re going to do. The next
step is how we\'re going to do it.

MS. COX: But who\'s affected? Would EPA come under this order?

MR. COFFEY: They could within 60 days, any time we can to do it. Once
this order is in place, we can amend that list at the end with a 60-day
public comment period. We\'ve got to have reason to do it. We\'ve got to
have whatever.

What was really needed in this case and what we asked the agencies to do
is look at several things. Do you have authority to collect information
for statistical purposes? No point in being on the list if you don\'t.

Second, do you have the wherewithal to provide security for this
information, to make sure it doesn\'t get used for other purposes unless
there\'s some statutory reason for doing that? And in that case, you\'ve
got to go through some other steps to make sure that the public knows
exactly what you\'re doing.

And all of these agencies, we believe, passed these tests. There\'s
still a bit of question about \-- we don\'t have a final opinion from
Chief Counsel at IRS over some questions on the authority of SOI
Division, but we\'re pretty sure that BTS will pass muster. All these
other agencies are going to pass muster.

Under the order it is very easy for us to extend the scope of this, the
effect of this order to other agencies by simply adding to the list. If
you look at the general terms of the order, it doesn\'t even say it has
to be just a particular set of agencies. We\'ve grafted that on to say,
well, the following agencies are the ones we intend for this to apply to
at this point in time.

CHAIRMAN MOUNT: Are you going to be here afterwards?

MR. COFFEY: Unfortunately I\'ve got a command performance back at the
office in about 40 minutes. I wish I could stay around longer. I am
likely to be back. Because I\'ve got to go this afternoon, I\'m likely
to get back tomorrow morning. It was a choice of one or the other. So if
you have some more questions you want to think about tonight and talk
tomorrow, maybe at the break we can discuss any other things you\'re
interested in.

CHAIRMAN MOUNT: Thank you very much, Jerry. That was a very important
topic, and I remember it was one of the first topics that was discussed
when I joined the Committee a number of years ago.

So we should move on now to the last subject for the afternoon \-- I
can\'t even say it \-- pollution control experiments. I\'ve obviously
been here too long this afternoon. Inder Kundra from the Office of
Statistical Standards.

Sorry for squeezing your time.

MR. KUNDRA: I\'m Inder Kundra. It\'s a pleasure to talk to you.

The purpose of my talk is to update the Committee concerning the
experimental economic project undertaken by EIA to analyze the effects
of environmental regulations of energy industries.

In 1993 we informed the Committee that we were in the process of
replicating the results using a software built by Arizona and using a
uniform set of parameters. That\'s what I\'m going to report here.

The Clear Air Act of 1990 instituted variable emission permits for
sulfur dioxide. Each permit with the industry was given to emit one ton
of sulfur dioxide. These permits were the industry could purchase or
sell or even keep these permits for future use. That\'s called banking.

And by 1993, a central market was established for trading these permits
with the provision that to auction about 2.8 percent of the annual
allocation by revenue to discrimination auction (phonetic), this kind of
auction does not generate any money for the central authority.

The Energy Information Administration contracted the University of
Colorado and Arizona for two experimental institutions that mimics the
salient features of the sulfur dioxide market to develop and document
transportable software that implements the experimental institution and
propose to replicate by pots and pots (phonetic), propose to replicate
high pluses (phonetic) of possible future testing with additional
experimental and field data.

The results were presented to the Committee in 1990 and 1991. In \'91,
following the recommendation of the Committee that EIA should pay
attention to the statistical details, we analyzed the Colorado
experiment and presented to the Committee with a finding that there was
no crossover between the replications, and the Colorado experiment was
not subject to any experimental bias.

At that time the Committee stressed the need for replicating these
experiments. Subsequently we funded the Universities of Southern
California and Mississippi to replicate these experiments by using
Arizona software and design and by using a statistical experimental
design Arizona software.

To test the applicability we adopted an experiment, a two-by-two
pictorial design with two supplix (phonetic) within each cell. This
design consisted of two forms, high and low, and two technologies, old
and new. The high forms, the forms I have indicated the amount of the
BTU produced and the technologies, I have indicated the amount of sulfur
dioxide emission per BTU, and with the condition that each from the high
forms we allocated about eight \-- there were two figures. I mean six
\-- 12 figures for the experiment, one through six and seven to 12. One
to six would be allocated eight permits, and we have seen that eight
permits for the high forms and four from seven to 12, and for the low
firms, in those two \-- as is obviously from here, that we did
deliberately introduce variations taking the firms and the technologies.

And to start the experiment, we started in the beginning with about \$20
for each student, and those students then made their profits by selling
the permits. They were the subjects within the firms, made profits
either by redeeming the permits at the pre-assigned redemption values or
by selling the permits to other firms in each of the transaction
periods. The \-- either in a revenue neutral, sealed, bid
discriminatable auction or in a double auction. That is a centralized
exchange where a public bids us and gets prices.

The experiment was done, repeated about eight times in each of the
universities independently, and in each of the replications, we had
different students.

The total profits which was made by these students in each of the
experiments at the end of the 12 periods were used to determine if the
observed differences in estimated profits in universities and
application to other subjects, simply things were obtained for
determining as the basic parameters to use for making some sort of
distinction whether the relation was \-- whether we would replicate
experiments or not.

Next slide.

As is obvious from here, at 18 universities we could see that there was
no difference. Most independent poll, even \-- within applications,
within universities of different technologies, of different subjects
within the given percents.

What we concluded from here that we were able to replicate those
experiments without any problem, without any bias in here left.

I just want to add another slide where I have to thank Dr. David
Bellhause from Canada. He suggested an experiment like this, that we
should have done the analysis with different firms if we see an obvious
infraction between the firms and these, but my main problem was that we
were taking \-- test the only applicability, and the firms were taken
within the universities and all of those things. We did it.

It is not something you say whether you do this or that way. Still we
can come to the conclusions that we can replicate those experiments and
maximum variations which was formed was between the forms, between the
technologies within the universities.

And I am very thankful to him because I did talk to him once or twice on
the phone.

That\'s all I can say about this.

CHAIRMAN MOUNT: Thank you.

The discussant is Greta Ljung.

MS. LJUNG: So I\'ll be just making a couple of comments on the
experiment, and then I\'ll discuss the analysis, and the last
transparency that David had provided covers some of the topics that I
was going to mention with regard to the analysis.

And the first transparency is just a summary of the experiment. We have
a two-factor experiment. The factors are per type and technology type.
We have a two-level design. Each factor has two levels. We are running
the experiments in two locations, and the experiment is replicated eight
times in each location.

And the output variable we\'re interested in is the total profits
realized by each participant.

The objective of the experiment as I read it is to study the effects of
the two factors and also to test the hypothesis that the results can be
replicated, that the results are sort of insensitive to the location and
also to the subject who was used in the experiment.

Now, the two-level design that\'s used here is, of course, very
economical. It includes the fewest factors possible, possible to study
the factor effects. The disadvantage is that we can only study linear
effects, while if there be nonlinear relationships between our output
variable and the factors, we would not get information on those
nonlinearities.

And in the present setting, a potential concern is that we have only
four combinations of the two factors. We have very small experiments.
Within each cell there are only two subjects. So we have a total number
of participants in each trial here is only eight, and so a question
arises: are the results that we get with only eight subjects really
representative of the market where there would be many more players?

Now, as far as the replications, we have a fairly large number of
replications. Eight is a fairly good number. Different students are used
in different replications. I think slightly better design here might
have been to use \-- in each location only use like four sets of
students and replicate the trials for those. That would have given us a
slightly better estimate of experimental error perhaps and might have
given us slightly different information.

Now, the tests that are performed show no differences between
replications, and that to me says that the experiments if tightly
controlled, the same instructions given, somehow those results are not
all that exciting. I mean it would have seemed it would have been a
little more interesting to try to introduce a little more noise into the
system here and to see how sensitive or if the results are sensitive to
changes like changes in parameters, maybe use different types of \--
different instructions or different training or different sort of
subjects, not just these who were all business majors, which are fairly
uniform, just introducing some changes instead of just replicating
everything under constant conditions could have given us a little more
information.

Now, when we talk about replicability, the tests that are done in the
paper basically focus on the average if you compared the two locations.
The tests that are done focus on the average shortage realized at each
location. So there\'s just a comparison of mean values. One might be
interested also in comparing the factor effects. Are the factor effects
the same when we replicate the experiment or did we get different
results? So we might want to compare just the \-- we can do the analysis
or we can think in terms of a mean. The observed \-- for each location
is the mean plus the firm effect, plus the technology effect, and then
we had possible interaction between the two factors, the replication
effect, and then error.

I\'ve calculated those parameters separately for the two locations and
compare not just the overall means for the two locations as done in the
paper here, but we would also be interested in are the firm effects the
same in the two locations and are the technology effects the same, and
we are also comparing two-factor interactions for the two locations.

Now, if we compare the firm effects for the two locations, what we are
really looking at is the university by firm interaction. So it becomes
an interaction effect with the university. And the same, we have an
interaction effect with technology if we compare the 138, actually half
of the difference between 138 and 151. Half of that difference is the
university and technology interaction. Half of the difference of the two
factor interaction becomes the three factor interaction, the interaction
between the two factors and the university, and I think all of those
quantities are of interest.

Now, if we wanted to write out a model that includes location, we would
then \-- that\'s written down on the bottom part there. We have an
overall mean.

Three main effects, if we treat the university effect as a fixed effect,
we have three main effects. We have now three interaction effects, three
two-factor interaction effects and one and then the three-factor
interaction effect, replication effect, and the random error.

And the replication effect here, we are not interested in making
inferences about those specific student groups that we used, but rather
we would treat those as a random sample from a much larger population of
students. So the interaction effect should be treated as a random effect
rather than a fixed effect.

Now, the analysis that we do would then include sums of squares
corresponding to those terms, and that was done at the last transparency
or table that David had provided. David also included interactions
between the factors and replicates. That was the last one that we
showed.

This is then the table that was included in the paper, and when I first
looked at that, a sort of striking feature, I thought, was the 16
degrees of freedom for firms and 16 degrees of freedom for technology,
and this is a little strange, given that we only had two levels of each
factors. There should really be one degree of freedom instead of 16, and
the reason we have 16 here is that the analysis assumed a nested
structure. So we assume that the replicates are nested within
universities, and then the treatments are nested within replications.

But that is sort of not quite the correct assumption because if we
assume firms nested really in replications, we would actually have 16
different firms. That\'s what the nesting would imply, and the same for
technologies, 16 different technologies. Each replicate would basically
use a different level of the factor.

I would note that the levels stay constant across the experiment. There
is no nesting. Those factors are crossed. So that is something that
needs to be changed here.

And then I think we have degree of subjects within cells, and we have an
F ratio. F ratio for that particular component, and then we have an
error term. Now, the error term in this particular case, since we are
already subtracting out the cell variability, typically the cell
variability would provide the error, and sum of squares \-- the error,
sum of squares in this particular table probably comes from the
interaction, from omitted interactions and the interactions probably in
this case are the interactions between firms and technology. That would
give us 16 interactions, 16 degrees of freedom there.

Now, for testing main effects and interactions, my preference would have
\-- no, no, for the analysis down here \-- if you are interested in
comparisons between firms and technology, my preference would have been
to combine and not break out within cell variability from the error,
combine those two into a single error estimate.

Now, David\'s analysis looked quite good. So basically what we need to
do is include main effects and interactions, and that was all done. So I
think his suggestions there were very, very useful, and that\'s probably
the right way to go about the analysis.

Because in the analysis that\'s done on this particular page, the F
ratios all have the same error mean squared, but because the very
experiment was done, it\'s not quite clear that we\'ll have the same
error mean squared for all F statistics, and that\'s something that one
should pay attention to, as well, in the analysis.

But it\'s true of course that although it\'s said the results don\'t
really change, but we still like to have, even if the results are not
affected, we still like to have sort of a sound system.

CHAIRMAN MOUNT: Thank you.

From the Committee? Bradley.

MR. SKARPNESS: I think this brings up a good point in that when you\'re
doing these kinds of analyses that you include a model, and then things
are clarified a little more, exactly what\'s going on and how you\'re
going to do the analysis, and that would help here.

And the other thing is that I was also struck a little bit by the way he
broke up the sums of squares here, and in David\'s approach, I don\'t
think he has a nested effect in there, what he showed. Okay. You do, but
this is still not sort of the standard way that we sort of \-- you know,
from the model you would break up your different components of
variability here.

And then the other thing I was wondering, you know, you do have this
fixed university effect. You know, it could be a random effect, too. You
know, you\'re really not generalizing this. I know it\'s not, but \--

MR. KUNDRA: It\'s not.

MR. SKARPNESS: It would be nice to sort of say, well, we did this across
universities or something, but that\'s just an aside. It is a fixed
effect.

CHAIRMAN MOUNT: Richard.

MR. LOCKHART: Well, I\'d like to put in a word for multivariate analysis
in the form at least of asking a question to reveal that which I don\'t
understand. The eight students are in an experiment together. When you
take instead of eight students two in a high tech., in a large, old
technology firm, and two in a large new technology firm, they\'re all in
one. They\'re run together.

MR. KUNDRA: Right.

MR. LOCKHART: And they buy permits effectively from each other.

MR. KUNDRA: That\'s true.

MR. LOCKHART: So that there are the eight responses that are obtained in
a single replicate of the experiment. It would seem to me to be at least
potentially correlated.

I mean the analysis of variance that\'s been discussed by David and
Greta is one of the univariate ways of \-- there are univariate versions
of MANOVA and then there are multivariate versions of MANOVA, and the
data don\'t actually show much sign of correlation \--

MR. KUNDRA: No, they don\'t.

MR. LOCKHART: \-- between the students within a cell, if you know what I
mean.

MR. KUNDRA: That was the reason that I did it that way, because we
wanted to test whether the students in the cell or not, as well as
reasons because we had \-- but this was the reason. That\'s one of the
reasons we wanted to test whether there\'s a change between students or
not, and that was the first problem that was raised by Dr. Bishop, to
see whether we can test whether there\'s publishing (phonetic) for
students or not. That\'s the way it was designed to test the students.

I understand what she\'s saying, that we should have not nested, but the
question here was that nested or not nested, the main things was we
wanted to test where the replications are. We can repeat the experiments
and all that. We have them or not.

And then one of the other factors is also the error \-- to test the
question that she raised, whether there is interaction with the squares
or not. That\'s why we did that way, but I mean, they would have
suggested the same thing for both of these interactions between the
firms and \-- I mean the firms as the \-- and the interaction between
the firm and technology and all those things.

I had a discussion with him. First he was taking the university, and he
was not even taking the replication test, not nested. Then I discussed
with him, and he changed it. So it was a good discussion with him.

But I do realize what you are saying, that we should have not used
nested because nested \-- the purpose was to see whether we can really
get what we\'re getting.

CHAIRMAN MOUNT: I do have one question. Bill Schulze at COLNOW
(phonetic) has recently set up an experimental lab and has just
introduced the Arizona software. So I\'m sort of interested if you have
evidence about the form of auction in terms of which auctions work best
in terms of coming up with efficient prices in this kind of situation.

PARTICIPANT: Double.

CHAIRMAN MOUNT: Double election.

MR. KUNDRA: We did that.

CHAIRMAN MOUNT: I mean it\'s pretty clear, right? Yeah.

So are there any more comments? Campbell.

MR. WATKINS: Just so I understand Greta\'s comments, on the nesting if
you had eight different technologies within the nest \-- what was the
other factor you mentioned that also should be broken out?

MS. LJUNG: You have firms and technologies.

MR. WATKINS: Yeah, firms and technology. So that then if you did that,
then your interaction effects would proliferate, if you do all of those
combinations, right?

MS. LJUNG: Well \--

MS. BISHOP: Basically we wanted to know whether the change of university
affected any of the conclusions that you would have drawn had you only
done it in one university, and that was the basic purpose of doing it.

CHAIRMAN MOUNT: So have you got a comment, Samprit?

MR. CHATTERJEE: Yeah, I think the same kind of thing. Since other
factors were balanced with regard to the universities, that\'s the
breakdown. So it is a perfectly balanced layout. So the university is
one replication and \--

MS. BISHOP: Except that they\'re different students.

MR. CHATTERJEE: Yes.

CHAIRMAN MOUNT: Any comments from the public?

MR. KUNDRA: I just want to stress the point again we were not suggesting
\-- that was not our purpose because we had deliberately introduced the
variations into those firms and technologies. That\'s how we started.

CHAIRMAN MOUNT: So before we formally close, I just want to ask the
Committee: are you going to be here for lunch tomorrow? We do have some
Committee business, and who\'s not going to be here for lunch?

MR. RELLES: I have to leave right after I give my discussion.

MS. COX: I\'ll be here, but we can\'t talk on and on.

CHAIRMAN MOUNT: I thought that it would be nice if we could break now.
Right? That\'s what I was assuming. Maybe we could meet \-- could we
meet at 8:30 tomorrow morning for breakfast and you try and be there a
little bit earlier so we\'re ready to talk at 8:30 so we can get Dan\'s
input?

MS. COX: I have to leave at 2:30.

CHAIRMAN MOUNT: Yeah, so let\'s do that then. The Committee try to get
down about 8:15 so you have time to eat, and we\'ll just talk for a
while at 8:30 before the meeting at nine, and I think it\'s next door, I
assume, in the Lewis Room.

And to remind you that we\'re going to go to 701 Pennsylvania Avenue for
dinner. We can\'t get in at Le Rivage, and for people who want to walk,
we\'ll meet in the lobby about five past six. Five past six in the
lobby. Who\'s going to walk? You\'re going to walk?

Who\'s walking? Greta, are you going to walk? I just want to know how
many people to look for. Right, you\'re walking.

MS. BISHOP: Where to?

CHAIRMAN MOUNT: To 701 Pennsylvania Avenue.

MS. BISHOP: Well, I have a car in this building. How far is it?

MR. KENT: It\'s right across the Mall.

CHAIRMAN MOUNT: I assume it\'s right across the Mall.

MR. KENT: It\'s right across the Mall.

MS. BISHOP: Well, I think I\'ll drive over if anybody wants a ride.

CHAIRMAN MOUNT: Would the offer of a ride make you change your mind?

Five past six. That will give us time to make it.

(Whereupon, at 4:51 p.m., the Committee meeting was adjourned.)