AMERICAN STATISTICAL ASSOCIATION

COMMITTEE ON ENERGY STATISTICS

FALL MEETING

THURSDAY,

OCTOBER 24, 2002

The meeting was held at 8:30 in Room 8E‑089 of the Department of Energy,
1000 Independence Avenue, S.W., Washington, D.C., Carol A. Gotway
Crawford, Chair, presiding.

[PRESENT]{.underline}:

CAROL A. GOTWAY CRAWFORD, Ph.D. Chair

F. JAY BREIDT, Ph.D. Vice Chair

MARK BERNSTEIN, Ph.D.

JOHNNY BLAIR

MARK BURTON, Ph.D.

JAY EDMONDS, Ph.D.

JAMES K. HAMMITT, Ph.D.

NICHOLAS W. HENGARTNER

WILLIAM G. MOSS, Ph.D.

POLLY A. PHIPPS, Ph.D.

RANDY R. SITTER, Ph.D.

ROY WHITMORE, Ph.D.

[ENERGY INFORMATION ADMINISTRATION REPRESENTATIVES]{.underline}:

GUY F. CARUSO Administrator

MARY J. HUTZLER Acting Deputy Administrator

NANCY J. KIRKENDALL Director, Statistics and

Methods Group

RICHARD BONSKOWSKI

ELIZABETH CAMPBELL

JAY CASSELBERRY

JOHN J. CONTI

FREDERICK L. FREME

DWIGHT FRENCH

DOUGLAS R. HALE

RUEY‑PYNG LU

PRESTON McDOWNEY

RENEE MILLER

ROBERT SCHNAPP

I-N-D-E-X

[AGENDA ITEM]{.underline} [PAGE]{.underline}

[ASA Chair\'s Opening Remarks]{.underline} 4

Carol G. Crawford, Chair,

ASA Committee on Energy Statistics

[Administrator\'s Remarks]{.underline} 8

Guy F. Caruso, EIA Administrator

[Director\'s Remarks]{.underline} 18

Nancy J. Kirkendall, Director,

Statistics and Methods Group, EIA

Questions from the Committee 27

[Brief Update on Commercial Buildings Energy]{.underline} 31

[Consumption Survey (CBECS)]{.underline}

Dwight French, Office of Energy Markets

and End Use, EIA

[Completion of EIA\'s System for the Analysis of]{.underline} 44

[Global Energy Markets (SAGE)]{.underline}

John Conti, Office of Integrated Analysis

and Forecasting, EIA

Committee Questions 53

[Information Quality Guidelines Completed.]{.underline} 55

[What\'s Next?]{.underline}

Jay Casselberry, Statistics and Methods

Group, EIA

ASA Discussant: Nicholas Hengartner 70

Questions from the Committee 75

[Natural Gas Data Program Updates \-- covering the]{.underline} 85

[Weekly Natural Gas Shortage Survey and Changes]{.underline}

[in the Natural Gas Data Collection Program]{.underline}

Introduction by and session chair:

Elizabeth Campbell, Office of Oil and

Gas, EIA

I-N-D-E-X (Continued)

[AGENDA ITEM]{.underline} (Continued) [PAGE]{.underline}

Committee Questions 111

Douglas Hale Summary of [Managing Risk in]{.underline} 120

[Energy Markets]{.underline}

Ruey‑Pyng Lu Summary of [Using Data from]{.underline} 121

[Combined Heat and Power Plants to Estimate]{.underline}

[Natural Gas Industrial Prices]{.underline}

Summaries of advice from the break-out 124

sessions

Frederick L. Freme, Committee Advice on 124

[State Level Coal Forecasting]{.underline}

Preston McDowney, Committee Advice on 125

[Estimating Monthly Data: Creating a]{.underline}

[monthly estimated data series for]{.underline}

[non-utility generation and fuel]{.underline}

[consumption from an annual and monthly]{.underline}

[related time series]{.underline}

Committee Comments 126

[Estimating and Presenting Power Sector Fuel]{.underline} 135

[Use in EIA Publications and Analyses]{.underline}

Presented by Robert Schnapp, CNEAF, and

Renee Miller, SMB

ASA Discussant: Roy Whitmore 153

ASA Discussant: Johnny Blair 161

Questions from the Committee 168

P‑R‑O‑C‑E‑E‑D‑I‑N‑G‑S

(8:34 a.m.)

[ASA CHAIR\'S OPENING REMARKS]{.underline}

CHAIRPERSON CRAWFORD: Good morning. I would like to go ahead and get
started so that we can at least start off getting the meeting on
schedule.

This meeting is being held under the provision of the Federal Advisory
Act. And this is ASA, the American Statistical Association\'s meeting,
not an EIA committee, which periodically provides advice to EIA.

The meeting is open to the public. And public comments are welcome. Time
will be set aside for comments at the end of each morning and each
afternoon session. Written comments are welcome and may be sent to
either ASA or EIA.

All attendees, including guests and EIA employees, should sign the
register in the hall and include their e‑mail addresses. In commenting,
each participant is asked to speak to a microphone. People in the
audience, your microphones are on either side.

Also, committee members and speakers at the head table need to speak
into a microphone so that the transcriber can clearly hear your
comments. There is a lapel microphone up here for speakers. Bill Weinig
can help you get that on if you need it. Members of the committee should
turn their name tents vertically to indicate that they have questions or
comments.

I would like each of us to introduce ourselves. I guess I will start. I
am Carol Gotway with the Centers for Disease Control in Atlanta,
Georgia.

Nick, do you want to go?

MR. HENGARTNER: Thank you.

CHAIRPERSON CRAWFORD: Okay.

MR. HENGARTNER: Nick Hengartner, National Lab at Los Alamos.

DR. BURTON: Mark Burton, Marshall University.

DR. EDMONDS: I\'m Jay Edmonds, Pacific Northwest National Laboratory.

DR. MOSS: Bill Moss, the Brattle Group.

DR. SITTER: Randy Sitter, Simon Fraser University.

DR. BERNSTEIN: Polly Phipps, Washington State Institute for Public
Policy.

DR. BERNSTEIN: Mark Bernstein, RAND.

DR. WHITMORE: Roy Whitmore with RTI International.

VICE CHAIRPERSON BREIDT: Jay Breidt, Colorado State University.

MS. HUTZLER: Mary Hutzler, Energy Information Administration.

MR. CARUSO: Guy Caruso, EIA.

DR. KIRKENDALL: Nancy Kirkendall, EIA.

MR. WEINIG: Bill Weinig, EIA.

CHAIRPERSON CRAWFORD: Renee?

MS. MILLER: Renee Miller, EIA.

MR. FAWZI: Aloylow Fawzi, EIA.

CHAIRPERSON CRAWFORD: The microphone, please.

MR. FAWZI: Aloylow Fawzi, EIA.

MR. TISCHNER: Casey Tischner, EIA.

MR. ZHANG: Bin Zhang, EIA.

MS. SWEENEY: Amy Sweeney, EIA.

MS. TAYLOR: Yvonne Taylor, EIA.

MR. FREEDMAN: Stan Freedman, EIA.

MR. HONEYCUT: Crawford Honeycut, EIA.

MS. BOEDECKER: Erin Boedecker, EIA.

MR. FENNELL: Dean Fennell, EIA.

MS. GEHLIN: Laura Gehlin, SAIC, supporting EIA.

MS. JENNINGS: Alethea Jennings, EIA.

CHAIRPERSON CRAWFORD: Okay. Thank you.

Some more meeting logistics. We will have two break‑out sessions. One of
them on estimating natural gas industrial prices will be held here. The
second one will be in room 5E‑069. So that\'s three floors down,
directly below this room.

Lunch is on the first floor, in room 1E‑226. We have eaten there before.
That is where we usually have lunch. So it is the same place.

Our dinner reservations are at the 701 Restaurant. That is 701
Pennsylvania Avenue, Northwest. I need to see a show of hands for those
who are attending so Bill can get a final head count. So if you are
going to dinner, raise your hand.

Members of the committee who are staying at the hotel, we should leave
the hotel by about 5:45. The reservations are at 6:00 o\'clock.

For your information, Nancy Kirkendall is Designated Federal Officer for
this advisory committee. In this capacity, Dr. Kirkendall may chair but
must attend each meeting, and she is here. She is authorized to adjourn
the meeting if she determines this to be in the public best interest.
She must approve all meetings of the advisory committee and every
agenda. She may designate a substitute in her absence, but she is here.

Now it is my pleasure to recognize Guy Caruso, Administrator of the
Energy Information Administration. He will have a few opening remarks
this morning.

[ADMINISTRATOR\'S REMARKS]{.underline}

MR. CARUSO: Good morning. It\'s good to see everyone. It\'s been six
months, I guess. We met in April. I was observer at that time.

Welcome once again. It\'s great to be here. I appreciate all of you
being here, especially members of the committee. Many have had to travel
a long way, I know. Mark took the overnight flight. So he gets the award
for the most difficult commute.

I also wanted to recognize a couple of our committee members, Carol
Gotway Crawford and Randy Sitter, for having been named Fellows to the
American Statistical Association at the last meeting in August, I think
it was.

And also special good news. Nick Hengartner is now, of course, at Los
Alamos, which is great, but even better news, he\'s become an American
citizen just very recently. So congratulations, Nick. Glad to have you
and look forward to working with you in your new role at Los Alamos.

Since we met last in April, there have been a couple of important
developments, which you will hear more in more detail about from the
individual speakers. One of which is we have initiated the weekly
natural gas storage survey. That has become one of EIA\'s most \-- I
guess \-- anticipated products as it comes out every Thursday.

The other thing, which Nancy will talk a little bit about but Jay
Casselberry will talk even more about, is the EIA or DOE information
quality guidelines that have been released as of the first of October.
So that is an important new development.

As always, there tends to be a lot of short‑term focus on products that
EIA does, of course, the very short‑term being our weekly data that
comes out, as I mentioned, gas but also oil.

More recently, there has been a lot of focus on the short‑term oil
market as it relates to events in the Middle East and concerns about
Iraq. So we have been doing a lot in recent weeks and months about
short‑term oil market outlook. We have a biweekly report called the
Energy Situation Analysis report.

And there have been a number of requests both from within government, in
Congress, and public, for that kind of information. So I think that has
dominated a lot of our effort since we last met with you.

Of course, Mary in her acting capacity over the last ‑‑ well, it was
more than a year ‑‑ had appeared a number of times before Congress and
other public bodies. So EIA is being increasingly asked to do more in
the public forum as well as our internal role. So I think that is not
only continuing but growing.

The other area where I think EIA is being asked to do even more is on
specific analysis of legislative issues, legislative proposals, and
policy analysis. And this is particularly true in the energy bill, which
has been under debate, had been under debate in both houses and in the
conference committee until members went on recess last week.

EIA was asked to do a lot of detailed analysis on issues such as MTBE
bans, the four pollutants bill, four pollutants, ethanol, ANWR. There
are a number of things that EIA was asked to do during that session.

I\'ll speak a little bit later about where that bill is and where it
might be going, but certainly EIA would continue to play a role when
Congress comes back after the elections and certainly into the next
session.

I think the next slide shows in very general terms what some of the
priorities and initiatives that we have had, certainly since we last met
here with you and since I have arrived. That is, you will see data
quality in almost every one of these priorities because it lies behind,
of course, everything that I have mentioned that we are doing and will
do that we certainly require the highest data quality and in all areas
across the fuels.

We are redesigning surveys to reflect these changes in the markets and
changes in the industries that we\'re collecting data on and analyzing.
Nancy will also speak a little bit about that. That occupies most of her
time.

Certainly one thing where this committee can be very helpful is not only
in broad data quality issues, but the need to address the increased
complexity of the data as well. And we\'ll be leaning on this committee
even more in future meetings on that particular issue, as we try to
improve the surveys that we use to collect the data.

In looking out over fiscal \'03, which has begun, even though we are
under a continuing resolution, we don\'t have a budget for fiscal \'03
as we speak, except for this continuing resolution at the fiscal year
\'02\'s level.

We will, of course, continue the surveys, including natural gas survey,
continuing to redesign both the electricity and the natural gas surveys,
which, as I mentioned, are being redesigned in order to reflect changes
that have occurred in both of those sectors of our energy industries.
And, therefore, there are a number of revisions that have been and will
be made as we do these redesigns.

Next week we will release new data on the co‑generation of electricity,
which will be reflected in a number of our statistical publications,
particularly affecting the consumption of natural gas, again, petroleum
data quality issues. And we are also redesigning all of our consumption
surveys: residential, commercial, and industrial.

The other thing that will continue as the high priority in EIA is to use
the Website and Internet to disseminate most of our information. I think
as of the end of this fiscal year, only four EIA publications will be
issued in hard copy. So we are almost going to totally electronic
dissemination.

Again, on the status of the budget, we had requested approximately 83.
Well, since this is a statistical group, \$82.8 million. And the House
had approved that, with an addition of \$500,000 for some coal data work
that they requested EIA to do. The Senate has not acted on that bill
yet. It\'s in the Interior Committee.

So we are not sure whether \-- when they return in lame duck session
after the 11th or on the 12th of November \-- whether they would take
that up or whether the continuing resolution, which now goes through
November 2nd, will be taken up in the lame duck session or carry over
into the next Congress. It has, as you know, important implications for
the amount of new work that we can do, because of the limitations on
spending based on the \'02 budget.

Let me say a few words about my own priorities. As many of you know, I
think when we met in April, I mentioned that I had been a career civil
servant at DOE for 20 years. So I\'m pretty familiar with EIA. So I
don\'t come in here with a lot of \-- I can\'t claim that I don\'t know
much about EIA or even the people and the work that they do. So I come
in with some \--

The good news is that I have been here and I have known a lot of the
people in EIA and the work that they do. So that is one of the reasons I
took the job. I was so impressed by their hard work and dedication over
the 20‑plus years that I have worked with them, but I also bring sort of
some biases. So I guess we will see how that all works out.

But clearly number one priority for me ‑‑ and it\'s influenced by those
years of working in DOE \-- but also the long confirmation process did
two things: enabled me to think a bit more about this, but probably more
importantly, to have heard from a lot of friends and colleagues in and
out of the government, particularly in industry.

And everyone of them had their \-- they had two things to say, how much
they rely on EIA, how highly they regard EIA\'s products and the people
that work in EIA. And the second thing was, \"And when you get there,
I\'ve got these five things I want you to fix.\"

So I have a long list of things that people have approached me about.
Almost every one of them goes to the heart of data quality. That\'s why
I think it\'s appropriate that the priority that Mary and her
predecessors have put on data quality and how that\'s reflected in our
budget for requests for both \'03 and \'04. We plan to devote more
resources to data quality across the board, which goes directly to the
work of this committee and how much I value all of your advice on all of
that, and I hope to lean on you even more in future meetings.

The second area, which is probably more my own personal \-- I guess \--
broad bias, I guess, if you want to put it that way, is on use of models
and modeling in what EIA does. I always get a little bit nervous about
presenting point estimates. So I am very interested in looking into not
only what our results are of our models but what the assumptions are and
how we convey those assumptions in any particular issue. It\'s critical
to maintaining the credibility, the trust, and the independence that EIA
has known over the years.

The third area, broad area, which I have a general idea of what I want
to do, but I could use your advice, whether it be formally in these
committee meetings or offline, on the area of improving the use of what
EIA does, both in terms of data and analysis, in the area of education.

We have done a lot already through the Website, the Kids\' Page, but I
think we can do a lot more. I think there is a tremendous wealth of
information and analysis EIA has at its disposal and is out there, but I
am not sure it\'s getting necessarily through the right people.

So I would be interested in that, both in a formal sense working with
education of younger people, but also the broader use of that term in
educating the general public as well.

So those are kind of, in broad terms, where I am hoping to go. And I
certainly look forward to working with all of you. Before I close my
remarks, I also want to thank Bill Weinig and Nancy for organizing this
meeting and working with this committee, because I think it is also
critical that the kind of relationship that has been established with
all of you. And much of it is due to Bill\'s and Nancy\'s hard work.
Thanks.

If there are any questions, I would be happy to try to answer them, or
we will have plenty of time, I think, in the meeting and in the breaks
to talk even further.

Okay. Carol? Thank you.

(Applause.)

CHAIRPERSON CRAWFORD: So now I would like to turn it over to Nancy
Kirkendall, who is the Director of the Statistics and Methods Division.

[DIRECTOR\'S REMARKS]{.underline}

DR. KIRKENDALL: Well, I would like to welcome you to this meeting of the
ASA Energy Committee. We hope we have an interesting agenda for you and
that we will get some useful advice back for us.

My role at this point is to give you an update on the projects we
discussed at the last meeting, some of the recommendations you made, and
what we have done with those recommendations.

As you recall, the theme of the spring meeting was EIA\'s top ten
priorities. And you will see we don\'t have priority numbers listed, but
as part of this meeting, we do have a number of updates on the specific
topics, on some of those.

There are three exceptions. I\'ll give you more of a road map a little
bit later. They are the Energy Situation Analysis report, extending NEMS
to 2025, and human capital management. So I will give you the update on
those three projects.

For the Energy Situation Analysis report, the ASA discussant was Dr. Cal
Kent, who was also EIA\'s former administrator. I think he was the
administrator when the first ESAR came out. So he has some interest and
biases towards that report.

His recommendation was that we should be cautious about proceeding with
this new product. I\'m not sure he was convinced that the emergency
situation was that important.

But it seems like the environment right now is one of continuing
potential emergency. And, besides that, ESAR seems to have become a
popular product. So we are going to be continuing with the ESAR in spite
of his concerns, I guess.

In terms of extending NEMS to 2025, there were two committee
suggestions. First was to evaluate the model in terms of prediction of
new technology penetration. Scott Sitzer of OIAF has said that they are
planning to write an article on that topic to come out in the spring of
this year. So that comment will be addressed.

The committee also recommended that EIA consider new technologies that
might be available in the future, and EIA has done a lot of this. They
reviewed the technology menu and added some options. They revised their
treatment of new sources for natural gas production, considered
productivity changes in coal, and then they reexamined technology for
the residential, commercial, and transportation sectors.

That work has been done. And the new AEO will come out in November of
this year, with its forecast out to 2025. So this is work that has been
done, and they did consider your recommendations as part of doing that.

The third area that is not on the agenda this time is human capital
management, priority eight. The committee had a lot of suggestions on
this. We haven\'t really acted on all of them because they covered a lot
of ground. We have done a few things.

The committee recommended that we consider college recruiting, and we
are considering that. I am not sure we have actually gone out and done a
lot of that yet, but we have been looking at where to go and how to do
it most effectively. We have also gone to a few career day events,
particularly those that are concentrating on minority groups.

The committee suggested that we make use of more interns. This summer we
had the opportunity to pick up six interns from the joint program in
survey methodology. So that was a plus for us. You get really smart kids
through those programs, and it\'s great to have them available.

Based on exit interviews, however, we still need to work on making sure
that they have interesting projects to work on. Not all of them thought
their projects were interesting. So that\'s a little challenge for us
for the future. That was also one of the recommendations of the
committee. So we didn\'t do quite as well on that one as we should have.

We are expanding the EIA 101 series. I think we talked about that with
the Committee. This is introductory lectures or courses for new
employees to try to tell them what is EIA and how to get around and how
to do things, as well as to give them broader introductions.

This year the statistics and methods group, my group, is going to be
collaborating with the National Energy Information Center to put
together an industry training program. So we will use a combination of
workshops, which my office frequently does, but we will coordinate it
with some site visits and field trips to get actual \-- a little bit
more reinforcement about what is going on in the field in the industries
that we are trying to monitor. Hopefully, we will put something together
for the spring to start that program.

As I said, there is a lot of other good advice that we haven\'t really
acted on yet, but we will continue to look at.

Now, today\'s agenda will start with updates on five of the ten
priorities. These are lectures where we don\'t really have particular
questions to ask you, but, of course, you are welcome to provide any
comments and advice, ask questions about them.

One will be the commercial building survey. Another one will be the SAGE
model. Jay Casselberry will talk about information quality guidelines.
As Guy said, the information quality guidelines was a government‑wide
approach. All federal agencies, including the statistical agencies, put
quality guidelines up on their home pages effective October 1.

What Jay is going to stress is, what are we going to do in the future.
Right now all it is is a product. It sits up there on the Website. If we
don\'t act on it, it isn\'t going to do much.

For EIA, we redid our standards manual, took the opportunity to try to
freshen our standards manual. And that was part of that effort. So now
it\'s: where are we going now? And how can we do a better job on our
data quality? So that would be Jay. He is going to summarize a lot of
the activities that are going to be going on in that area.

The other briefing this morning is on the natural gas program updates.
This is a summary of two priorities. They\'ve rolled them all together.
The first one is the weekly natural gas storage report. And then the
other priority was natural gas data issues. So that has been rolled
together as an update and also a discussion of where they are trying to
go with the natural gas program.

Then finally, in terms of the top ten priorities from the last time, the
last topic on the agenda for today is related to the two priorities that
dealt with electric power. It\'s a concentration on \-- there is more
work that has been done in that area. So you will hear about that.

Two of our break‑out sessions deserve special mention. Ruey‑Pyng Lu and
Preston McDowney of my staff were just assigned to the projects they are
going to be discussing, in August or September. There is a lot of data
manipulation and collection and trying to understand the problem
involved in doing that.

So they are probably a little bit embarrassed to be talking to you
because they don\'t have their ideas firmly set in their minds. So this
is a bit of an experiment, because we wanted to get your advice early
enough in the process so that we could actually do something with it.

Frequently we have given you presentations once the analyst thinks he\'s
done with the work, and he really isn\'t very interested in thinking
about something else. So hopefully, this time you will have a chance to
have an impact on the analysis as we go forward. Who knows? Anyhow, I
hope that that will turn out to be a successful experiment.

For the other two break‑out sessions running opposite the two that I
just mentioned, Doug Hale will be talking about the derivative study.
This is a report that he just completed at the request of the Secretary
of Energy, he and his team. It wasn\'t only him. And then Rick
Bonskowski will be talking about an improved way to estimate weekly coal
production. He is looking for ways to improve it even further, although
they have implemented these changes.

So with that, unless you have some questions, we can begin with the
update presentations from the last meeting.

[QUESTIONS FROM THE COMMITTEE]{.underline}

CHAIRPERSON CRAWFORD: I have five minutes, I think. So I do have a
question.

DR. KIRKENDALL: Sure.

CHAIRPERSON CRAWFORD: My question is: does EIA have any formal documents
describing their data confidentiality procedures and describing what Jay
is going to talk about, the information or data quality guidelines? Is
there a report or something written or anything?

DR. KIRKENDALL: Well, the data quality guidelines on the standards
manual are available on our Website.

CHAIRPERSON CRAWFORD: Okay.

DR. KIRKENDALL: So those are formal documents. There are standards in
there, in the standards manual, on how we do disclosure limitation and
tables. There\'s a new \-- an IT policy statement that is coming around
for signature. It\'s just a page or two, talking about how we can
protect information that is reported electronically. I think the answer
to that is there are probably bits and pieces spread around.

CHAIRPERSON CRAWFORD: So maybe sometime at a break or something, can you
point me to those documents, I guess? The reason it comes up is, I think
some of the people at CDC can actually benefit from knowing what EIA has
been doing. So I could point them to some of these documents and get
their data confidentiality where it needs to be.

DR. KIRKENDALL: Okay. I don\'t know whether you all know \-- actually,
in Cal Kent\'s discussion, he has the paper that he sent out. At the
very end of it, there is a brief discussion of a new law that is on the
floor of Congress right now. The part of it that pertains to EIA \-- if
it actually passes, and who knows \-- would be pretty exciting because
it says \"any federal agencies\" \-- not just statistical agencies \--
that collect data under a pledge of confidentiality can protect the
data; in fact, must protect the data. And there are penalties for
disclosing it.

So that would give EIA its first teeth in actually being able to protect
information. We don\'t have the ability to do that now, because there
are laws that say we have to share data, if requested, for official
purposes, not statistical purposes. We haven\'t given up company‑level
data, but we could be required to. So that would be pretty exciting, if
that law passes.

CHAIRPERSON CRAWFORD: Just a follow‑up question. The disclosure auditing
software that you have been working on, is that available for others to
use?

DR. KIRKENDALL: It is available for any federal agency to use. So if you
wanted to use it at CDC, you could do that.

CHAIRPERSON CRAWFORD: Okay. Thanks.

DR. BURTON: What\'s your personal sense about how the ability to force
confidentiality would do to the quality of the data that is under
request?

DR. KIRKENDALL: I don\'t know whether it would make much difference on
the quality of the data, but one never knows. So that is an interesting
question.

What it might do for some of our surveys, perhaps we would get a little
bit better cooperation. That would be one benefit. So that would affect
the quality of the data \-- if we had better cooperation. Although we
have quite good cooperation on our surveys.

What it would really do is for the consumption surveys, which have
always been treated as confidential ‑‑ and we have to go to extreme
measures to make sure they are treated as confidential ‑‑ for the
residential energy consumption survey and the commercial building
survey, we have a contractor collect the data and promise not to give us
the data, because we can\'t promise to keep it confidential ourselves.

For the manufacturing energy consumption survey, we have to hire the
Census Bureau to do the survey so they can protect the data. If we have
this law \-- and we would, of course, declare those surveys to be
confidential when collecting the information under a pledge of
confidentiality \-- then we would have more options on how we would
conduct the survey. So that is one of the things that would be exciting,
the ability to have some latitude in how we spend our resources. But in
terms of quality other than response rates, where you might consider
that you would do a little better, I am not sure.

Any other questions?

(No response.)

DR. KIRKENDALL: Okay. Thank you.

CHAIRPERSON CRAWFORD: Thank you.

So our next speaker is Dwight French, who will give us an update on the
commercial buildings energy consumption survey.

DR. FRENCH: Excuse me while Mr. Neanderthal here figures out how the
machine works.

DR. KIRKENDALL: While he\'s figuring out the machine, I forgot to
mention that part of the graphics context last time we talked about was
how these little industry graphic‑showing relationships between industry
and our surveys were useful. So they have done one for the electric
power industry. And I\'ll pass this out so you can see them.

[BRIEF UPDATE ON COMMERCIAL BUILDINGS]{.underline}

[ENERGY CONSUMPTION SURVEY (CBECS)]{.underline}

DR. FRENCH: You all have a printed copy of my overheads, I believe. I\'m
supposed to start at 9:15, aren\'t I, according to the schedule. So I\'m
still okay.

My name is Dwight French. I am the Director of the Energy Consumption
Division within the Office of Energy Markets and End Use. At the last
meeting, I gave a presentation on the redesign of EIA\'s commercial
buildings energy consumption survey, which will be used for the first
time next year, for the 2003 version of the survey.

The presentation I gave at the last meeting was to determine what the
committee\'s input would be on three methodological issues. And this
would be on the second slide on your series. The first would be whether
we ought to use population or commercial measure as the measure of size
for selecting first-stage area sampling units, commonly called primary
sampling units.

The second issue was whether the building, which we have historically
used as the sole unit of data collection, should continue to be used or
whether we should use a hybrid approach, combining facilities,
buildings, and tenants.

The third issue is whether or not a national building list available
from the Insurance Services Office should be used as a basic list frame
to supplement the CBECS area sample.

At that time, the committee provided quite a bit of comment and
information. And my purpose with the presentation is now to give you an
update on what has been decided regarding the design.

First, on issue one ‑‑ and this will take us to the next slide ‑‑ in
working with Westat, who is the contractor for the redesign, we have
after looking at various options decided to go with a commercial
building measure of size, as opposed to population as the measure of
size for the first-stage unit.

What happened was the measure of size was determined based upon
employment statistics from county business patterns and
worker‑per‑building ratios based on past CBECS within specific
industries. When I say \"specific industries,\" we talked about using
\-- and we did use \-- the information from county business patterns
translated, which means we worked with NAICS industries.

Now, CBECS collects information on building activity categories. So we
did a rough crosswalk, one-to-one, between CBECS building activities and
NAICS economic industries. And those were the values that were used to
determine what building types are associated with which economic
industry categories. And those were the then employment‑to‑building
ratios that were used within each economic category to eventually
determine, based on economic data, what the measure of size for each
county in the United States would be.

In fact, a first‑stage sample of 108 areas has been selected, based on
those measures of size. Just to give you a little bit of a idea about
the measures of size and what the effect of the commercial measures of
size is, I would comment about some of the counties in the New York City
metropolitan area.

If you look at New York County, which contains Manhattan, it has a
population of about 1.5 million. Its measure of size, its probability
based on that measure of size, is one. It is a certainty all by itself.

On the other hand, Kings and Queens County, which each have a population
of between 2 and 2.5 million, quite a bit larger than the population of
New York County, each have probabilities of selections of only about
two‑thirds, rather than the uncertainties in having a probability of
one.

Bronx County, which has a population over 1.3 million, almost as much as
New York County, which is certainty, only has a probability of selection
of .35.

On the other hand, if you want to compare Bronx County to, let\'s say,
Clark County in Nevada, which is the county that contains Los Vegas,
those two counties are almost identical in population in the 2000
census. But whereas, the Bronx County has a probability of selection of
.35, Clark County, which of course is a well‑known commercial area in
certain ways, has a probability almost twice as large, .63, of
selection.

So these are the types of changes that can be made by using a commercial
measure of size, rather than a population measure of size, at the first
stage. We hope that this will enable us to do a better job of estimating
commercial activity and commercial energy use when we actually conduct
the survey.

Issue 2. This was one that many gave us quite a bit of information and
discussion on last time. As a result of that discussion partially, we
have decided the following.

We are limiting our facility‑level data collection to only two types of
campuses, colleges and health care facilities, which are among the most
complicated and important of the campuses that we come across, and the
most difficult to deal with.

The buildings will be sampled and surveyed on these facilities. And, in
fact, the major data collection for these facilities will still be
building‑level data collection. Facility information will be limited and
supplemental, but we will be able to produce facility‑level energy use,
supplemental to the energy use and the characteristics that we will have
for individual buildings.

Now, on the other hand, we talked about the last time what we were going
to do about malls and strip shopping centers, where a lot of the
information is available, not at the \"building level\" if you want to
think of a building as a free‑standing unit or whatever, surrounded by
walls and a roof line, uniform roof line.

Most of the information there is at the tenant level. What we intend to
do is sample malls and strip center buildings, and then subsample
tenants within and do the primary data collection at the tenant level.
We are going to have a methodology for sampling and determining who to
take regarding anchor stores, common areas in malls, and particular
intensive types of activities within strip centers and malls; that is,
dry cleaners, restaurants, and so forth, as opposed to offices and other
little retail shops, which may not use anywhere near as much energy.

The point is that by collecting data at the tenant level, we can blow
that up to the building level. And, alternatively, we could create a
situation where, if we wanted to provide information for individual
tenants, rather than blowing it up to the building level and showing
actually estimated data in building records, we could consider tenants
to be building equivalents and show those as the primary units.

Now, obviously the situation here is that, regardless of whether we use
the tenants as the primary information or blow it up to the building
level, because we know the probabilities at each stage, we can create
unbiased estimates of floor space and energy consumption and so forth.

The real difficulty in all of this is getting an understandable and
clear definition and information about numbers of buildings. A lot of
people like to know this. They want to know how many buildings there are
in the U.S.

We need to make it clear to the users of our data that, really, floor
space in the commercial sector \-- and the energy use relative to that
floor space \-- is a more important thing to understand than the number
of buildings and energy use per building. The reason I say that is that
what a building is, and where one building stops and another building
begins, can be very much in the eye of the beholder.

And so there is a great deal of \-- this isn\'t sampling error. This is
error of interpretation among various people who would make their own
decisions as to where one building starts and another building ends.

So that\'s what we have done in number two.

DR. SITTER: Excuse me. Before you move on?

DR. FRENCH: Yes, Randy?

DR. SITTER: Could you tell me the difference between those two
samplings, two cases? It sounded like you were doing almost the same
thing. Are you sampling hospitals and then buildings within hospitals?

DR. FRENCH: That is correct. We will be sampling hospitals and college
campuses as such.

DR. SITTER: So maybe your PSU?

DR. FRENCH: No. We already have the 108 areas defined as our PSUs.
Within those areas, we will be sampling colleges and health care
facilities, which we will get off lists.

DR. SITTER: All right. But then you\'d sample those and then you\'d
sample buildings within them?

DR. FRENCH: Sample buildings within them.

DR. SITTER: And strip malls? You\'re going to sample strip malls and
sample tenants within?

DR. FRENCH: But malls and strip shopping centers will probably come
largely through the area sample, rather than through list samples. So
they will come in. And then once we get there and we actually sample
those things, we are not going to actually list tenants at the listing
stage for those buildings.

We are going to list and then sample the malls and strip center
buildings. And then we will list the tenants within, because obviously
you can limit your work quite a bit by listing after you have sampled,
rather than before you have sampled the malls and strip center
buildings.

Issue three pretty well decided itself for us. The Insurance Services
Office \-- after we got done with our research, which they let us do
based on cases that they gave us at \$7 per \-- turned around and said,
\"Oh. Now you maybe want to purchase tens of thousands of units as a
basis for list samples in primary areas around the country. We\'re going
to charge you \$25 per case.\"

We talked with them. They refused to budge. That means it could have
been very easy for us to run up a tab of somewhere between a quarter and
a half a million dollars for selecting buildings from those lists, or
getting list samples to select buildings from those lists.

We said \"Forget that,\" especially since the list\'s incompleteness \--
which we did validate in research studies \-- and the fact that it does
have out‑of‑scope cases, because they don\'t care about clearing cases
off their files. That caused us to give it up as a potential list
source.

We are now looking at information from Dun and Bradstreet, files from
Dun and Bradstreet, which aggregate establishments to buildings based on
address connections, other usual large and special buildings list
sources, lists of government buildings, hospitals, schools, et cetera.
And we may also be supplementing that at the list stage with F. W. Dodge
information to get the newest large building construction.

Those are the three areas that I discussed last time, what we are going
to be doing about them. The committee advised us regarding the first
issue, \"Look where the correlations lie.\" And it looks like the
correlations best lie with the buildings measure, rather than the
population measure. So that is why we chose that. And we can see the
effects of that in the probabilities.

We were advised to try to keep the building as the unit of data
collection to the maximum extent possible. We will do that, although we
want to do something a little bit different for the malls and strip
centers. As I said, this decision was made for us.

Any questions from the committee?

DR. WHITMORE: One question about the malls. If you\'re sampling tenants
in malls, does that give any coverage to the common areas of malls that
don\'t belong to any particular tenant?

DR. FRENCH: When we sample a mall, we intend at this time that the
common area will be a certainty selection as a tenant within the mall.

That does two things. Number one, that is a significant part of the area
of the mall. And, number two, it gives us an \"in\" probably to the mall
manager, whatever, themselves. There\'s useful information we can get
about the mall as a whole from them.

CHAIRPERSON CRAWFORD: Is there another question?

(No response.)

CHAIRPERSON CRAWFORD: Thank you very much.

DR. FRENCH: Sorry for the delay, folks.

CHAIRPERSON CRAWFORD: Do you need the computer or the overhead?

MR. CONTI: Yes. Thank you very much, Dwight, for working out all the
bugs. I never like to go first for that very reason.

CHAIRPERSON CRAWFORD: So, for the record, our next speaker is John
Conti. He is going to talk to us about the completion of EIA\'s system
for the analysis of global energy markets, or SAGE.

[COMPLETION OF EIA\'S SYSTEM FOR THE]{.underline}

[ANALYSIS OF GLOBAL ENERGY MARKETS (SAGE)]{.underline}

MR. CONTI: Good morning, members of the Energy Committee. My name is
John Conti. I am the new director of the Division of International
Economics and Greenhouse Gases Divisions in OIAF.

I am here to talk to you today about SAGE. But the first thing I will
say is that I will not be talking to the completion of EIA\'s system for
analysis of global energy markets, because I would get thrown out of the
Modelers\' Union if I did that.

Instead, I will be giving you a status update on the SAGE system. I will
be reviewing what you talked about in your previous meeting first, your
recommendations, and then talk about our activities since then.

At the last meeting, Pia Hartman described how we had done a
proof-of-concept, where we had taken our templates for supply‑demand and
conversion for a region and we solved for a specific region of the SAGE
system. That was discussed. We discussed our regional structure. I\'ve
listed the regions here because I haven\'t committed them to memory yet.

We also described our software system, which we placed a lot of emphasis
on. It is very important in that we hope we have created a system where
energy analysts can use it to easily modify the model and to do analysis
and to reconcile projections.

Whereas I think in the past a lot of modeling efforts have depended more
on programmers to implement a lot of the modeling changes, what we hope
we have done here is tune a flexible enough system where energy analysts
with not specific training maybe in linear programming could actually
use this system and make changes necessary to implement new technologies
and things of that nature.

At the last meeting, you had given three basic recommendations. You
talked about a probablistic approach to avoid knife‑edge decisions, and
we will talk about that in detail; estimating demand-price elasticities
directly; and including non‑CO~2~ emissions for energy and non‑energy
sources.

Since then, we have developed and tested a market‑sharing algorithm
within the conversion template and model. We have modified the demand
templates to incorporate elasticities. And we have done it in such a way
that we hope that they are easily transferrable among regions and
comparable among regions and easily viewed by other analysts, so we get
expert opinions across this whole modeling community.

We have evaluated the non‑CO~2~ gases. Beyond carbon, we have included
methane at this time. We are contemplating including mercury as time
allows, but we don\'t have any current plans to include the other
non‑energy gases or particulate emissions from the energy sector, as
those are more regional problems and the data doesn\'t exist to really
address them appropriately.

We are in the stage right now where we are trying to prepare for our
International Energy Outlook. We are really under the gun, working the
kinks out of a new modeling system.

We are in the process of reviewing the model structure and the data and
reporting. And an important aspect of this is, we are working with
independent expert reviewers from the Energy Technology and System
Analysis program, which have very specific knowledge about these types
of models and will be able to help us with the different \-- the
technologies and the assumptions that are being used throughout the
model. They will have the regional information. Hopefully that will help
us a bit.

At the last two meetings, you probably have had a flavor for the fact
that the demand templates of the model were fairly far along, and the
other templates \-- the conversion and supply \-- were not. We have
really done a lot of work since then, especially in the supply area but
also in the conversion area.

Some of the things that we have done are, we have broken out OPEC and
non‑OPEC supply, which is often of interest to many of the users of our
projections. We added reserves and resources for unconventional oil. We
have added a capability to merge EIA \-- IEA data streams, and to
evaluate which ones I think might be better sources of data for specific
applications. We have added representation for liquefied natural gas.

We have expanded renewables, especially biomass, to include energy crops
and biomass ethanol. And we developed supply curves for oil, gas, coal,
and renewable energy. We have let some contracts, specifically in the
coal and the renewable area, and we think we have got some very good
information that we hope to get completely implemented over the next few
days. Also, we have expanded the trade capability to include petroleum
products.

In the conversion area, it has been more of a data sort of effort than
it has been a model structure, where we have had to look at the IEA data
and the UDI data for global data on the conversion sector. We used IEA
data to look at fuel consumption and generation, and UDI data for
generation capacity. And we merged these two streams so that we could
have fuel consumption and output by plant type and technology,
technology type being the thing that we were trying to get at.

As you recommended \-- I think in the last meeting \-- retirement
profiles were very important. We have estimated retirement profiles in
the conversion sector. We are advising the technology assumptions, and
so they\'re consistent across all regions. And we are benchmarking
historical information in the model to EIA data.

One of the things that you had recommended \-- a couple of things that
we are going to want to implement in SAGE is going to require us to
introduce algorithms, probably non‑linear in nature, into the LP. We
have implemented a time‑stepped approach to solving LP, where we
basically solve the five‑year increments, then in between increments, we
could make changes to the solutions, or the model itself.

Some of the things that we hope to incorporate by going to this
time‑stepped approach is imperfect foresight in the model, so we could
introduce different cuts of how the markets solve, as opposed to just
dynamically over time.

We are introducing the capital decision market‑sharing algorithm that
was recommended through this methodology. We were able to reduce the
model size significantly. We have been pushing the forefronts of the
software we were using. And it will give us the ability to introduce
other algorithms that are not particularly well‑suited to the LP
framework through this approach.

The next two slides address the market‑sharing algorithm. I will
probably be bouncing back and forth a little bit between them. Basically
we instituted a market‑sharing algorithm in these intervals between
solutions, by specifying a group of technologies in which they compete,
by specifying a parameter by which we determine whether \-- which
technology we will reallocate. We can give it a technology preference
factor if there is some reason to do that. We haven\'t done it at this
point in time.

We look at the reduced costs from the SAGE outputs and create indices.
And then based on these reduced costs and these close‑enough parameters,
we are able to reallocate the solution and put it into the model. And
basically this equation tells us that it\'s based on the relative
solution values, the relative price, the reduced cost in the model.

One of the parameters that is important is the degree of optimization,
whether \-- how far do the solution values have to differ in cost before
they\'re considered. And then we have to go through an algorithm by
which we make sure the solution is consistent.

The last thing that we are currently working on I wanted to talk about
somewhat was learning by doing. This is another one of the sort of
algorithms we borrowed from the NEMS, national energy modeling system.
Basically it helps us get a handle on how technology costs change as we
gain market experience. And we have related it through a doubling of
capacity.

Every time the total cumulative capacity for a specific technology
doubles, what do we expect to happen to the overnight cost of
constructing such capacity? Here we have given the specification for
this algorithm.

And that is all I am here to talk about today. As I said earlier, we are
really concentrating right now on preparing the model for use in the
International Energy Outlook. We are looking seriously at the
projections, comparing them to alternate projections. And we probably
will not do a lot of model enhancements between now and the time it is
published, but then we will pick it up again from there. And that is
all.

CHAIRPERSON CRAWFORD: Are there any questions, comments? Jay?

[COMMITTEE QUESTIONS]{.underline}

DR. EDMONDS: John, is the learning-by- doing the only mechanism by which
technology changes in this model, or is it the only endogenous mode of
technology change?

MR. CONTI: Well, it\'s the only endogenous, right, technology change.
There are different technology costs and parameters at different periods
of time.

DR. EDMONDS: Just one word of caution in that regard, that there is, as
you know, a wide literature on endogenous technological change. And
while there is at the moment a great fashion in learning-by-doing, two
things.

One is that it\'s based primarily on these log‑log plots of cumulative
capacity or something like that against cost, in which the causality is
presumed to be the cumulative experience.

MR. CONTI: Right.

DR. EDMONDS: There is absolutely no reason to exclude the possibility
that it was the reduction in cost which led to an increase in the demand
for the product. And the literature on endogenous technological change
has \-- there are perhaps a half dozen pathways by which endogenous
technological change could be affected.

And so just a word of caution \-- don\'t necessarily structure this with
this one single mechanism as the endogenous technological change
mechanism, because it\'s not \-- you could get caught in a position that
you don\'t want to be in.

MR. CONTI: We have another chicken-and-egg problem here.

DR. EDMONDS: It\'s fundamental. And it\'s not addressed, but in the
community that likes learning-by-doing as the mechanism, they never
address this question of: why do I believe that causality works from
cumulative to the cost, rather than the other way around? Or, some mix.

And you and I can talk about why, about the many ways by which
endogenous technological change can be brought into level. Larry
Goulder, for example, builds a model with two different mechanisms at
work, as an example.

MR. CONTI: I am sure there are many people in the building who say that
we should just model their programs directly, to show how we could have
technologies to decrease the price and increase the penetration.

Any other questions?

(No response.)

MR. CONTI: Thank you very much.

CHAIRPERSON CRAWFORD: Okay. Thank you.

Our next presenter is Jay Casselberry. He will be talking on information
quality guidelines.

[INFORMATION QUALITY GUIDELINES COMPLETED.]{.underline}

[WHAT\'S NEXT?]{.underline}

MR. CASSELBERRY: I\'m Jay Casselberry. I\'m with the statistics and
methods group in EIA. I\'d like to talk about the information quality
guidelines. It\'s a topic we presented to you twice in the past year
already. We talked about it last year, when we were just beginning the
project.

We talked to you in the spring meetings, when we were about halfway
through, where we had developed our draft guidelines and we had also
developed about half of the new standards we were developing. We have
completed that work, and now we are beginning to look at the quality
projects that we are going to undertake to try and help ensure the
quality of EIA\'s information.

The questions for the committee that we would like your opinion on are
suggestions for specific quality projects that we are going to discuss
today. If you have any other suggestions for quality projects that EIA
could undertake that you think would be relevant to the types of
products we disseminate. And considering we have limited resources, if
you have any ideas on where it might be better to focus our resources
that would have the biggest effect on quality.

Unfortunately, I noticed a typo in my second bullet today, which is
probably not a good start for a quality topic.

(Laughter.)

MR. CASSELBERRY: I changed it late last night. I got it right on the
last slide but not on the first slide.

As it was probably discussed a little bit this morning, there was a law
passed that said all agencies should have information quality guidelines
to ensure the quality of the information they disseminate to the public.
OMB was given the lead on that.

They put out guidelines directing all the agencies on what they had to
do. And, in turn, the agencies have put out draft guidelines during the
spring and summer of last year.

We got comments. And we have all issued our final guidelines by October
1st, which begins a process where now the public can comment on what the
agencies are doing and they can ask for correction requests that the
agencies disseminate information that is not of appropriate quality.

In EIA, we have issued our quality guidelines. The quality guidelines
are general in some cases. They inform the public of EIA\'s commitment
to quality, but they do talk about a lot of different activities we
undertake and the things we try to do to ensure quality.

We have our standards manual, which we have issued. And it\'s 32 new
standards. That is directed more towards the EIA contractor staff and
our EIA personnel. It\'s required actions. Hopefully by following those
required actions, it will lead to improved quality in all the work we
do.

With our information quality guidelines, they address five criteria that
were established by OMB in its guidelines for all the agencies. Those
five quality criteria are: objectivity, utility, integrity, transparency
and reproducibility.

All the main agencies, federal agencies, had to put out guidelines. So
the Department of Energy has its own guidelines, which apply to us. But
then our guidelines \-- EIA I think is probably the only DOE component
that came up with its own individual guidelines that sort of go deeper
and just apply to us.

The DOE guidelines also establish the process, when the public comes in
to comment on any information disseminated by a DOE component, what that
process is. You send in a request through the Department of Energy\'s
Chief Information Officer. It comes in. They send it out to the
individual component, disseminate the information, and they take care of
the response.

And there is also an annual report that has to be made each year
beginning in January 2004, telling about what kind of complaints the
agency\'s got and how they dealt with them.

To go a little bit deeper on the core elements of quality, when they
talk about objectivity, they\'re talking about both the substance of the
information that the agency is disseminating, and how it is presented,
just so it is presented in an unbiased manner.

The utility \-- is the information useful to its intended users? And the
integrity, it\'s more into security. Is it free from unauthorized
falsification or corruption?

The guidelines, OMB\'s guidelines, and also the individual agency
guidelines set a higher level, a higher goal, for quality for what they
call influential information. It was up to each agency to define that,
based on some criteria established by OMB.

What DOE defines as influential is any information that we disseminate
that could have greater than a \$100 million impact on the public. And
that\'s usually through rulemakings. EIA is not involved in that.

The second category for DOE information that is influential is any
information that is embargoed because it has a major effect on the
market. And so you are trying to control both who has access to it
before it\'s disseminated and the timing of the dissemination, to make
sure there is fair access by everybody. We have a number of products
that are embargoed, some of our weekly products and the annual energy
outlook and a few others.

For those products, the agencies are supposed to try and strive for a
higher degree of transparency, which means we take more time to explain
the data and methods that were used to come up with the information, and
that we can reproduce it, or that even a third party could come in and
reproduce with access to the same methodology and access to the data.

I think with EIA \-- although it just requires us to meet these two
additional criteria for our influential information \-- I think EIA for
all of the information, we strive to be very transparent, to explain the
data that we put out, its limitations, its sources. And we also have the
capability to reproduce the information that we put out. Although there
is a distinction made in the guidelines, I think for EIA, everything
that we have would probably adhere to these two criteria.

And so we put out the guidelines, told the public what we were going to
do with regard to quality. And we told EIA staff what we expect them to
do. Our mission is to provide high‑quality information.

The offices responsible for the information have lead on the quality
projects. The OMB guidelines also require that the agencies have a
pre‑dissemination review process in place to make sure that any
information that is disseminated has been checked to make sure it
adheres to the quality guidelines. EIA has always had a
pre‑dissemination review process. At the level it\'s done depends on
what information is being put out.

A lot of our more frequent products, like the weekly products, are
reviewed within the offices. Some of the more important products or some
of the less frequent products go through an EIA‑wide review before they
ever get disseminated.

We also use independent expert reviews. When it\'s a highly technical
subject, a lot of times it has more to do with the analytical or
forecasting sides.

Then if we do get any requests from the public to correct information
that they believe we put out that is not of the appropriate quality,
that will be directed to whichever office put out that information.

Also in the OMB guidelines, there is an appeal process whereby if we
respond to a request for correction and the person is not happy with it,
they have the capability to appeal that.

For EIA\'s quality system, the components are we have quality control,
which is done usually as part of the continuing operations of the survey
or model; we have quality assurance, which is typically done by an
independent party. And primarily it\'s the statistics and methods group
or someone else independent, maybe a contractor.

And then with that information, there should be process improvement
built into the information you find out about quality as a result of
these efforts you should build in to try and improve the efforts and
have some mechanisms to track and see if it really is coming about the
desired results.

I would now like to tell you about some projects, some specific
projects, that we are undertaking. Some of them we have started. The
rest of them we are going to be doing in 2003. They are
self‑assessments, data system audits, model documentation evaluations
and assessments.

We have what EIA has described as its essential functions. And we are
evaluating our capabilities to take care of those in the result of some
catastrophic event and our data quality measures. And there are also
co‑performance measures.

With the self‑assessments, that goes back more to the people who are
working on the surveys or models to establish sort of an ongoing survey,
ongoing improvement process. It works into the quality control area
using the data quality measures they have.

Hopefully there will be some periodic assessments by the survey managers
where they actually take some time to sit down to look at how their
surveys are going, where there are problems, to try and identify some
important targets for improvement, and to set up a tracking and
monitoring system to see if they really are improving those.

John Vetter\'s physics and methods group is leading the overall effort
for SMG across EIA and the coal, nuclear, electric and alternate fuels
group in EIA has already undertaken a major project to try and look at
this across this office to try and develop some higher‑level quality
control mechanisms.

We will be doing audits of data systems. We did those back in the 1980s
and I think they probably stopped around 1990 or maybe somewhere in that
area where we used to go in and look at an entire data system from the
collection through processing, through dissemination, and come up with
suggestions on ways to improve it or problems that might be inherent in
it.

What we do when we audit a data system, the first one is going to be
coal, our coal area. We are going to look to see if they have complied
with the new standards and if they don\'t what actions they could take
to come into compliance.

We review the entire process through collection, processing, and
dissemination, and the data quality measures that they have to see if
they have them in place and how they are using them and tracking.

We look at the documentation to see if it describes the current
operations of the system to try and use that also to help evaluate the
soundness of the system or the places where there could be improvements
or enhancements.

Ramesh Dandekar in our statistics and methods group is going to be doing
the coal. After that we are going to look at the standard energy
processing system that is being implemented in the Office of Oil and
Gas.

For our model documentation, that is being led by Inder Kundra in our
statistics and methods group. He used the documentation to help evaluate
the model and also just evaluate how well the documentation describes
it. So he\'s looking to get the theoretical design of the model; the
assumptions; the equations; and any properties, statistical properties,
that might be in there to try and determine how well things fit
together, make any suggestions for improvements. We have already done it
for six models. EIA currently has 26 active models. So that will be
going on in 2003.

For essential functions, we established the continuity of operations
plan for EIA. That describes 43 essential functions that we want to be
able to resurrect in the event of some catastrophic event. They were
broken into four categories. Immediate is something that EIA wants to be
able to restore its capabilities within two hours; short‑term within
seven days; mid‑term within 30 days; long‑term within 90 days. For an
example, a capability might be the ability to put out the weekly
petroleum status report.

So we have 43 of those. What we then did is went back and determined
what functions we would need to be able to do to put out the weekly
petroleum status report because the weekly petroleum status report is a
combination of at least four or five surveys. And so we had to have the
capability to go in and restore those surveys.

So we are in the process now of identifying what documentation exists,
what the personnel requirements are, both in the number of personnel.
And to give a little bit of an idea of the capabilities of the personnel
for a model, it might just be a few highly technical people. For a
survey of a large number of facilities, it\'s done weekly. You need a
lot more people and maybe the telephone capabilities and things like
that. So you\'re looking at your information technology requirements to
resurrect it and also other backup capabilities. If something were to
happen to one of the main buildings that EIA is located in, what
off‑site capabilities do we have to resurrect those and using that
information to identify gaps in our ability to restore these
capabilities. We may test whether or not we can restore a few of them
with the information we have.

The last one is our data quality measures or performance measures. In
the standards we have defined, we describe seven categories. There are
some detailed data quality measures under each one. That\'s frames,
response rates, revisions, how timely we are in getting out the data,
the results of edits, sample surveys, variances, and imputation rates.

And so we are right now in the process of determining what surveys in
EIA have what measures available, what plans they have for possibly
implementing new measures in the future. Initially we are going to focus
on the response rates and the imputation rates as the data quality
measures for the surveys.

And in 2003, we will probably look at some way of layering those so that
the people working on the surveys can get more detailed information and
possibly management can get less detailed information. It gives them
some information on how the surveys are progressing. Our lead on that is
we have a performance measures committee.

So just going back to the questions I have for the committee were any
suggestions you have for the five different types of projects we\'re
undertaking if you think there are other areas we should look into for
quality improvement and if you have any suggestions just based on your
own experiences or own knowledge of areas where a survey organization
might be better to focus its resources.

Thank you.

CHAIRPERSON CRAWFORD: Thank you very much.

Our discussant is Nicholas Hengartner.

MR. HENGARTNER: Well, thank you very much, Jay. It\'s a pleasure to
comment on a well‑thought‑out presentation and a timely topic.

Overall, when I\'m thinking of quality product, there are two things I
have to think about. One is the product, what it is. And the other part
that one should give some thought is who are the clients. And both go
together.

So if the client is the G public, it\'s quite a different matter than if
it\'s a sophisticated user or a member of Congress. One has to be then
specific as to who you want to address or what kind of qualities are
required for those different types of people. So that might be one of
the key things that are not in this report here.

However, on the product side, I think you have done a great job of
dividing it into the three essential components. What is the raw data?
The other one is the methods. And the third is the documentation. Why do
you think that this is really what any data should have, those three
components. And one should make them as transparent as possible.

I mean, I have this vision or this holy grail of what I would call
integrated data products in which at the end of the day, you have a
spreadsheet with the macros that do the calculations that allow you not
only to disseminate the data, the tables, whatever it is, but also the
methodology in which event you make forecasting.

The advantage of doing that is that you talk about cost‑effectiveness
and what is cost‑effective. Well, I am thinking now of sophisticated
users, and that is where I am coming from. So this does not necessarily
apply to G public kind of documents or end users.

For sophisticated users, they will give you feedback. If the method
doesn\'t work, they will tell you. And it\'s a little bit in the
pre‑software mindsets that you are going to have to assist in the effort
as it goes along.

In terms of priorities, where I see it in the data side, it is clearly
that if the raw data stinks, whatever you do afterwards doesn\'t matter.
So this is really where the first focus is.

That\'s the first bullet. As a matter of fact, when you listed them, I
was thinking how would I reorder them. I came up with the same order you
had. So just to say, you are quite on track, but I really would focus on
the data quality as my first standpoint. What do I think about data
quality? You said response rate, quality, timeliness.

There is a mention of sampling frame. And I want to address the sampling
frame issue just very fast. You have at EIA an amazing advantage ‑‑ and
I hope Randy will join in ‑‑ in that not just only one survey but a
series of surveys that evolves over time.

Your sampling frame is also changing over time, but the fact that you
can compare sampling frames versus the survey today to the one that you
are going to do six months down the road allows you to have inherent
quality control over time. And you do mention that you want to track
things through time.

I am suggesting, actually, that quality could be ensured but hopefully
that you will look at the path and gain moral strength. It\'s not each
survey in itself that needs to be important, but the whole history needs
to work. And it needs to jibe together. It\'s a whole view that I would
like to suggest.

The other thing is you mentioned getting data and information from other
agencies. I would suggest that myself. However, I know how hard it is to
get information from other people.

DR. BURTON: That is true.

MR. HENGARTNER: Very much so. And while I think that is something very
important that I would stress to do that, I see difficulties in
achieving that. It sounds funny, but that is my experience.

Finally, I want to come back to this idea of transparency and
reproducibility and this idea of disseminating not just the data but
actually even the methods. And when I think of the methods, clearly you
have to disseminate also the documentation that goes with it. Put it all
together in one package. The advantage of that is it gives you
transparencies and reproducibility because if the macro is out there,
clearly someone can just click the button again and get the answers.

If there wouldn\'t be restrictions about confidentiality, I would even
say put in the macro that calculates imputations. Now, I know this is
not feasible. I am aware of it. But ideally it would be wonderful if I
just clicked on a button and it recalculates imputations, which might be
stochastic and, therefore, introduces the natural variability.

No, you\'re not going to get exactly the same tables, but you are going
to get honest answers and you are going to understand why the numbers
are different but how much they are influenced.

So at this stage, I am thinking of sophisticated users. I agree. But I
think the greatest impact of EIA data is for sophisticated users. I
would like to think everybody is going to look at your Web page. And
they probably should.

But at the end of the day, those who are using this data are
sophisticated people. One should give them the tools. I think you should
be as transparent with the methods as the data knowing that your focus
should be on getting the data at accurate as possible to make the system
run.

So, I mean, you have the costs, but I think of getting a really great
product out there. You already have a great product. I should rephrase
that. I see this holy grail, this integrated data thing. And I think it
might be possible. That would be one of the first things I see in that
direction. I would love to see that happening.

Congratulations.

[QUESTIONS FROM THE COMMITTEE]{.underline}

DR. PHIPPS: I have one thought particularly. The self‑assessment part of
this seems a little bit undefined to me and kind of relates to
Nicholas\' focus on data quality measure. I would think the data quality
measure results and the audit results should drive the self‑assessment.

And I\'m not certain. It sounded like it was more of a starting point,
but I think you have got the real data there on the results and the
quality measures and it should be driving the performance, rather than
something a little bit more subjective driving the assessment from a
management perspective.

DR. BURTON: I just want to sort of reinforce what you said. This is an
issue that I think has come up at least a couple of times. There are a
lot of sophisticated users, academic and sophisticated users.

I think you all did a great job of making this data accessible to almost
anyone who has a curiosity or a need, but sometimes there are things
that can be done or could be done ranging all the way from sort of more
sophisticated issues that you bring up or just data formats that would
make it easier for people who are interested in analytical applications
to take the data and use it to know what they have. And sometimes I wish
it was maybe geared more in that direction.

DR. WHITMORE: One other comment or suggestion for you is I think Nancy
mentioned some difficulty with regard to putting quality guidelines and
standards into practice. I know one way that many places like RTI and
Westat are currently doing it is writing standard operating procedures
and trying to have standard things that are done relate to these
guidelines, have standard procedures for doing them so we\'re sure that
whether it\'s a new employee or somebody very experienced, they\'re
following standard processes to produce types of quality that the
guidelines met and standards are requiring.

It\'s not clear to me if it\'s totally your contractors that would be
writing and using these standard operating procedures or if there are
procedures within EIA that are used to process data, doing imputations,
or basic QC and editing procedures, whatever; they stand to have
standard operating procedures written that would be guidelines to staff
as to how to do things so that the end product would achieve the
guidelines, the standards that you put in place.

CHAIRPERSON CRAWFORD: Did you have some comments?

MR. CASSELBERRY: Sure. I think with the reproducibility, I think on the
modeling systems, where you have been very good about getting
documentation out there, about the available documentation on our
websites, and we do archive the models and they should be available,
with the data methods, because you are talking about the confidentiality
issues and the access to the data, we do try in the technical note to
explain that again.

MR. HENGARTNER: I know you do.

MR. CASSELBERRY: The amount of information is probably less for some of
the things that you are talking about, like imputations. And with the
formats and utility, I think that\'s one thing that is brought up in the
OMB guidelines as an issue of quality, the utility of the information to
the users.

I think when we focus on quality, sometimes maybe we don\'t focus on
that as much. I will agree with that. Sometimes we focus more on making
sure that the process is right and that we have good quality controls in
place, we do checks and sometimes the utility is not always quite as
much.

And with the integrity, we\'re always sort of watching that as an agency
through the IT process, but the utility sometimes may not be quite as
well‑stressed.

I think with the self‑assessments, we are sort of just setting that
process up. So I appreciate your comment because the slide made it sound
like there are self‑assessments and then part of that comes out as
quality measures. You are saying that quality measures should help drive
what we\'re focusing on. That is an important thing. I think we need to
look at the quality measures and sort of determine what we have and
focus on it.

With the standards and guidelines for across EIA with the staff,
sometimes you have heavy contractor involvement in the collection and
processing or the modeling efforts. And sometimes it\'s a lot more EIA
staff who are involved and a lot less contractors.

So we do want them to apply to everyone who is doing these kinds of
activities, but we put them out there. The administrator has approved
them. He\'s told people about how important they are, but we need to put
some processes in place, then, to try and make sure they are adhered to
and make sure that people understand what is being asked of them.

A lot of them are continuations of standards EIA has had in place since
1989. So a lot of the processes are in place and have been checked
before, but I think there are some new things that we have to work with
the offices on and work within the EIA on.

DR. KIRKENDALL: Jay mentioned the activities. This is in response to
your standard operating procedures, which I think is a good idea. They
would have to be done office by office because the offices are all a
little bit different. Some offices probably are already doing those
sorts of things, but not all of them are.

I think that the CNEAF quality effort, the Office of Coal, Nuclear, and
Alternative Fuels, is thinking about that sort of thing. So hopefully
that will result.

DR. WHITMORE: That sounds familiar. We are doing SOPs separately by
divisions within RTI. This depends very specifically on what you do and
the kind of data that you are working with.

DR. KIRKENDALL: And we didn\'t go into much detail on the quality
process that we hope will result in self‑assessments and better quality
measures, whatnot, but that is going to be a collaborative effort with
all of the survey managers to try to get them to agree to and develop a
self‑assessment process based on measures and what we are trying to do
overall.

So that is really kind of off in the future. By the next meeting,
hopefully we can tell you how well that is getting started at least.

CHAIRPERSON CRAWFORD: Are there any other questions, comments? Did you
have one?

MR. CASSELBERRY: I would just add that Jim Joosten, who is back behind
the administrator, is leading the effort in CNEAF. And I don\'t know
whether he might want to add something specifically about what they are
doing.

MR. JOOSTEN: Thanks. We do quite a bit of the data collection, as you
know, within the EIA. We looked at this particular issue and, in fact,
the procedural issue. We know that there have been standards for quite a
few years, but the methods that we have been controlling quality are
largely the ad hoc with respect to data collection and data processing,
more formalized when it comes to publication quality.

So within our office, we have decided to try to create a checklist for
our survey managers to use which would basically have quality
checkpoints at the various steps of the survey process, take place over
an entire year. That survey manager would essentially verify as he is
doing his job that he has answered these certain quality checks and
signed off on them.

The idea is that first it controls the process, rather than leaves it up
to their memory. The second thing is it provides a document for the
management to be able to check the quality at any given point in time.
And the third thing is it provides an audit path at the end of the year
for an independent auditor, like SMG or an outside quality assurance
contractor.

So we are sort of in the process of developing these checklists. Our
goal is to have them done for the survey managers by the end of the
year. That is an informal goal, but that is what we are trying to do.

CHAIRPERSON CRAWFORD: Thank you. Thank you very much.

I have noticed that there have been a few new people come in. So those
of you who have just come in, you need to sign the register in the hall.
If you could take some time to introduce yourself at the microphone for
the record?

MR. RUTCHIK: I\'m Bob Rutchik of the Statistics and Methods Office of
EIA.

MS. MARINER‑VOLPE: My name is Barbara Mariner‑Volpe. I\'m with the
Office of Oil and Gas, the Natural Gas Division.

MR. BUDZIK: I\'m Philip Budzik. I\'m with the Oil and Gas Division of
OIAF, the Office of Integrated Analysis and Forecasting.

MR. WEINIG: Alethea, would you introduce yourself, too? Did you earlier?
Okay. Anyone else?

MR. VAGTS: Hi. I\'m Kenneth Vagts with the Office of Oil and Gas.

MS. GORDON: Janet Gordon, CNEAF.

MR. KASS: Roy Kass, Office of Oil and Gas, Natural Gas Division.

CHAIRPERSON CRAWFORD: Okay. We will move on. Our next presentation is
going to be coordinated by Beth Campbell. She will be providing some
natural gas updates.

[NATURAL GAS DATA PROGRAM UPDATES ‑‑ COVERING THE]{.underline}

[WEEKLY NATURAL GAS SHORTAGE SURVEY AND]{.underline}

[CHANGES IN THE NATURAL GAS DATA COLLECTION PROGRAM]{.underline}

MS. CAMPBELL: My name is Beth Campbell. I am the Director of the Natural
Gas Division in the Office of Oil and Gas.

I am going to be providing kind of an overview introduction that is
giving you an update on some of the projects that we have underway in
the natural gas program. About a third of the way through, I will be
turning it over to Bill Trapman, who will be describing the status of
the weekly underground natural gas storage report, something you had an
earlier presentation on in a previous ASA session.

And then he will be turning it over to Sylvia Norse, who will be talking
to you about the status of our annual survey implementation, the changes
that we have planned for that, about a third of the way through. And so
we will be doing a tag team presentation for you here.

Next slide. So we have a number of ongoing efforts to improve the scope
and performance of the current program. Some of them are in
implementation, such as the weekly natural gas storage program. Some of
them are almost at that point, the steps that we have underway for the
new annual survey.

There is I think a fair amount of interest in improving the natural gas
data program. It does represent about 25 percent of energy use. Now, the
program can be described in terms of its scope; that is, what is its
content, what are the frequency of its products, the geographic detail
that is attempted, and some of the issues of timeliness or
confidentiality that we deal with.

The scope of the program is to provide supply and demand volumes, to
provide prices for certain of these supply and consumption points, and
to have industry capacity and practices information available,
deliverability, storage capacity, those types of things that help you
understand the strength of the industry and to put some of these into
map formats as well for particular use in emergency studies.

Some of our products appear only annually. We have a large and
substantial annual report, a monthly report, and most recently the new
weekly storage survey. For our annual reports, we have U.S. and 50‑state
data. And we have a fair number of 50‑state data series presented in the
monthly as well.

As you move from annual to monthly, you increase the number of sample
surveys relative to census surveys so that as you get more frequent
data, there is more sampling represented.

Now, on the timeliness issue, I want to mention that that certainly
varies by survey and is one of our major issues in operating these
surveys. We have a substantial non‑response, late response problem,
which means that there is some difference between the design of the
program and the execution of the program.

Next slide. When we talk about the overall natural gas data program for
any given time period, a year or a month, we are going to use this basic
framework, supply equals disposition. We have a set of surveys that is
trying to represent these components of supply and components of
disposition.

There may be some slight differences with respect to the annual and the
monthly, but just let me summarize for you some of the challenges that
we face in this area.

Production. Using the AER 2000 data, that is about 81 percent of supply.
That is based on a voluntary survey that is run each month and each year
with reports coming in from the states. We have no direct data
collection from operators or producers in that area.

Net imports. Those data are provided to us from the Office of Fossil
Energy here in the Department of Energy on the basis of a regulatory
program that they run for import and export licenses. We have no direct
contact with importers and exporters on either annual or monthly data.

On storage change, this is the only survey in which we have direct
contact with data respondents. That accounts for about four percent of
supply. So on 81 percent, we are relying on a voluntary survey. And on
15 percent, the net imports, we are relying on a regulatory program.

Now you come to the disposition side. Here we have direct data
collection on an annual basis for residential, commercial, and
industrial sectors through the Natural Gas Division. We have direct data
collection on the electric power sector from the Electric Power
Division. And we also on an annual basis have direct data collection for
the natural gas industry.

On a monthly basis, we have sample survey collection for residential
sector, commercial sector, and industrial sector data from the Natural
Gas Division. We have sample survey coverage from the Electric Power
Division, from the Electric Power Division. And we estimate the natural
gas industry data on a monthly basis.

So on this side of the equation, we have on a monthly basis Natural Gas
Division\'s operating sample surveys for approximately 70 percent of the
supply. The Electric Power Division is operating a sample survey for
about 22 percent of the supply and we are estimating about 8 percent of
the supply.

The difference between the Natural Gas Division and the Electric Power
Division survey does matter because, in fact, the Electric Power
Division surveys are facility‑based surveys; whereas, the gas supply
surveys are where the gas delivery agents are estimating the assignment
of deliveries to sector. So that they are essentially able to account
for their total deliveries, but they make assignments based on their
best judgments with respect to the sectors.

For price data, we obviously have a voluntary survey as the basis of our
wellhead price surveys. We have the regulatory survey as the basis of
the import price survey. There are no prices associated with storage
surveys.

And we have the residential and commercial and industrial sector surveys
coming from the Natural Gas Division\'s prices coming from the Natural
Gas Division survey.

Electric power is coming from a facility‑based survey. And we only do
the pricing information here on an annual basis. We don\'t do it
monthly.

Next slide. So these are some of the program issues that we are dealing
with. Certainly one of them, as I mentioned, is timeliness. I think it
is important to give an example here of our production data survey,
which is coming from the voluntary survey. We request that data
approximately 120 days after the report month.

And at that point in time, we have data from about 8 of the 32 reporting
states. Also we get our data from the minerals management survey for the
federal offshoring. If you\'re familiar with that program, they have not
provided any data since July of 2001. So we have been estimating for all
of these interim data points.

On the issue of completeness, there we have our concern about exactly
what kinds of data elements should be in our program scope. One of the
issues that we are thinking of here is the role of LNG in supply and
certainly in terms of timely supply. We have no LNG information at all
except the import and export data that comes in through the Office of
Fossil Energy. We also have reports on injections and withdrawals but
not inventory on an annual basis but contributes a small amount of
supply.

Now, on a quality issue, some of these issues down here relate to
quality. Certainly we know that the EIA series differ from other
sources. And given, of course, the problems with our production data
sources, production is an example where there are many speculations
about what true natural gas production should be. So that is an area
where there may be problems with what we have to present. And there
certainly is an opportunity for a lot of different alternative
perspectives on what our current production is.

We do have performance measures for several of our data series, which
also suggest that there are some quality problems. Of course, one of
these that I think you heard about in previous presentations is our
industrial price series. In that case, we know that our industrial price
series is based on about 16 to 18 percent of the deliveries in that
series. And that is because basically our price data are only available
to us from local distribution companies. And they only have about 16 to
18 percent of the sales anymore for that sector. So, consequently, that
is all we have to get price data from.

As you have seen in the earlier supply/disposition equation, we are
relying on quite a different number of sources, many of them inside and
outside of our program office. And so this has resulted in this
balancing item or discrepancy item, which is somewhere in the range of
two to five percent.

Next slide. In the previous presentations, I have shown you the figure
which had those blue bars in it. That is essentially what we had as a
balancing item prior to the electric sector revisions that we are
working on right now. And it appears as though based on the AER numbers
that are coming out, that those changes will leave us with a substantial
balancing item problem, just a quite different one. So our work remains
ahead of us.

Next slide. So what are some of the changes that we have underway? I
think you are going to have a separate presentation that talks about the
definition and sources of electric power data.

Now, we, of course, will be adopting the same definition that is being
used EIA‑wide, which is this electric power sector definition. We had
previously had an electric utility presentation of data. And the
non‑utility‑generated data had been part of our industrial sector.

So we are going to be using this new definition of electric power
sector, which is ‑‑ I will let others do a further presentation ‑‑ at
least representing the non‑utility and the electric utility sector.

So that means that we are in the course of changing the sector
presentations for electric power that has an impact on what we have been
providing for industrial sector data and pricing. So each of these we
are in the middle of changing those sector definitions and recalculating
those historical series.

We also are adopting a new source for our annual vehicle fuel use. It
had been our annual survey data. We are now going to be using estimates
coming in from the Office of Alternative Fuels. That will provide
consistent presentations across EIA.

On this fourth item here, we are using a new EIA survey, which we have
been operating for about a year. And by January of \'03, presentation of
data in the residential/commercial sector should include the results of
this survey, which is collecting data from marketers in five states. And
that should mean that we now are reflecting the contributions of gas
marketers as well as local distribution companies in the average price
for those states. It should improve the coverage and quality in those
states.

And then Bill and Sylvia will be talking about the next chart.

Next slide. So the earlier slide represented some of the things that we
are in progress doing right now. These are some of the things that we
are considering and studying. It represents the result of a cross‑office
effort to do some brainstorming and to do some assessment of where we
thought our most urgent needs were to develop alternative approaches and
to get some real information about what we might be able to perform.

One of the most significant of these efforts is we are in the process
right now of looking at the possibility of replacing our voluntary
survey of collecting production data from the states.

We would like to consider whether it is possible to do more timely
direct data collection from some subset of producers or operators. And
we would have to determine whether or not they could be more timely than
the states and how we would define a subset.

Would it be producers, operators? Does it make a difference? What would
we use as the frame for a subset concept? And it has to be subset
because there are thousands of operators and producers and we would very
definitely have to have some subset definition in order to manage being
more timely.

Another issue that we are working with is the sources of our import and
export volumes and prices. As I said, now we are relying upon data from
the Office of Fossil Energy, their regulatory program.

We do receive on our annual survey reports from pipelines on the amount
of gas that they have received at borders, at the U.S. borders. And
there are some differences between what we receive on that annual survey
and what the Office of Fossil Energy provides us. There are also some
differences with what the Customs office has in its data record. So the
three of us have slightly different numbers running.

And it is important for us to get together and start to figure out how
much of this is definitional and how much of this is coverage. But we
have begun the conversation trying to figure out what is going on there.

I think you will have a separate presentation following this on
alternatives for industrial sector price estimation. And I won\'t speak
to that. You can attend that. I think it is right after this
presentation.

I will also skip forward now down to the next to the last item here. We
are also looking at the merit and methods for future liquefied natural
gas industry measurement. This is a task that we have underway.

We do think that right now our measures of liquified natural gas imports
are treated as though they are directly and immediately added to gas
supply. And while that may be true in many cases, it is also important
that some of it is provided in storage and becomes part of the
deliverability capability during winter emergencies. And we would like
to know more about how that industry is actually operating. So that is
an area that we would like to know if it would be possible to develop
surveys there.

So let me conclude this section and ask that Bill step up. My question
to you would be, what kinds of priorities would you have for this work
based on what you as potential data users would have?

MR. TRAPMAN: Good morning. As Beth said, I will give you an update on
the weekly gas storage project. The key features of the weekly project,
just to refresh your memory, you have seen this before, the last set of
meetings that you had here.

We had weekly data collection and release. We thought that was an
essential feature for a weekly program. Mandatory collection is another
feature. We had that to ensure the statistical validity of the program.

It is a sample survey, not a census, like our monthly data are collected
as. We go to 45 respondents. The data is collected on a company‑region
aggregate basis as compared to the monthly survey, which goes out to all
operators for all fields.

Lastly, we collect current working gas volumes. This is significant
because it is working gas volumes, no other volume in storage. And we
feel it is an improvement over the predecessor survey that AGA had in
the sense that it reduces respondent burden. They were asking for three
data elements. We ask for the one.

If there are any changes that need to be reported, of course, we have
asked them for that, too. But on a regular basis, it\'s just working gas
only.

So how has the survey done? Next slide. The survey has benefitted, we
feel, from excellent response rates. You can see here for the first six
months of operation that the response rates were at least 93 percent or
above based on head count of respondents. Based on volume, it\'s 92
percent or above. We feel this is very much in our favor with this
program. Despite the good response rate, we did have a problem or a
number of revisions early on.

Next slide. Now, as background and as a reminder, a revision is when a
prior, previously published estimate would have changed by seven percent
with new information. I\'m sorry. Seven BCF.

Seven billion cubic feet, while that might sound large, is less than
seven percent of the stock measurement that we are estimating. In fact,
as the stocks build up as they have in the last few months, that
percentage diminishes. Right now a seven BCF difference would be less
than one‑fourth of one percent. So it\'s a very small volume.

Now, causes of revisions, we feel there were three major sources or
causes for revisions. One was the reporting responsibility for the
fields. We did quite a bit of outreach with the potential respondents to
ensure consistency in our requirements and their compliance, but there
were some issues.

Certain operators or respondents were submitting information for fields,
a set of fields, that we hadn\'t expected. Where there were joint field
operations and operator owns the field but only a portion of the gas
within sometime, they reported only for their share of the gas, which
was not the intention. It was to be all the gas in the field.

We also had some misreporting of the gas volume itself. Now, we did get
a fan mail from somebody who insisted that people know the difference
between base, working, and total gas. And I wouldn\'t argue that point,
but I would assert that they were reporting working gas, for whatever
reason. We had some issues there.

Now, those three items we feel are not that unusual for a new data
survey. We did have a fourth cause. And the potential cause for future
revisions is how we approach the methodology. We have a reference month
for our estimation parameters.

Next slide, please. This is a slide you saw last time that Roy Kass had
presented. This is the sum of the weekly working gas reported by the
respondents in any given week. And because it\'s a sample, we have to
extrapolate for the missing portion.

We use the assumption that the pattern for or the relative ratio of
total for the region relative to that respondent population in the most
recent monthly report would have been consistent with what we would
expect in the most recent week.

And so this is the total working gas reported in the most recent monthly
for the region. And the denominator is for those respondents who have
reported in any given week the sum of their volumes from the monthly
survey.

Now, there are a few key things here. One is that if the numerator and
denominator change proportionately, that ratio would be fairly stable.
As we go from month to month, if you recall, I said that a revision
would be triggered, would be the equivalent currently of less than
one‑fourth of one percent. So if we updated this ratio based on a more
current month and it changed by a quarter percent or more, we would
trigger a revision.

So based on suggestions that the committee gave us last time, we have
been looking into changes in our approach to the estimation. We are
looking into smoothing techniques. We are looking into ways to separate
the estimation between the certainty portion and the non‑certainty
portion. That work is in development, but we don\'t have any findings to
report right now.

Next slide. Let me leave you with some good news. We feel that we have
eliminated the start‑up difficulties. It was a little bit of a rough go
for the first ten weeks. We had five revisions in ten weeks. People
weren\'t very happy, us included, with having revisions half the time.

The good news is after our release of July 11, in the most recent ‑‑
actually, since it is after 10:30, I can tell you. In the most recent 15
weeks, including today, the release that happened 9 minutes ago, we have
had only one revision. And just as point of fact, this was caused by the
shift in reference month.

Now, as I said, we are addressing the methodology. We have since this
revision in August made a decision to defer further updates in that
parameter until we have finished our quantitative assessment. That work,
as I said, is ongoing. The benefit, the reason why we went through the
revisions, we feel is shown in the improvements in the estimates
themselves.

Initially our weekly and monthly figures differed for the end of March
by about 75 BCF, billion cubic feet. And with changes to the data,
including changes at any point if we had resubmissions in July, we asked
people to resubmit all the way back to March to have a different series.
That difference is now down to 21. And we feel that that is well within
the statistical uncertainty bound.

So we feel we have removed most of the potential for revisions and we
are working on that one outstanding issue. And we feel that we have a
more accurate set of results as a consequence.

Thank you for your attention. Now Sylvia Norris will present her
section.

MS. NORRIS: Good morning. As Beth stated, I am here to discuss the
redesign of EIA‑176 survey. The EIA‑176 survey is an annual census that
was designed to maintain historical data, which tracks consumption by
residential, commercial, and industrial sectors.

Data from this survey dates back to 1912. And it\'s published in our
Natural Gas Annual. And a wide audience, including federal and state
agencies, also use this data.

The EIA‑176 aggregates state‑level data. This is the only source of
tracking flows of gas across state borders, which, for example, are
interstate movement. And this survey tracks approximately 420
underground storage injection fields.

Currently it takes about 20 burden‑hours to complete the EIA Form 176.
In our current clearance package, we reduce those burden‑hours down to
18 hours.

We also have eliminated unneeded data elements by combining
transportation of exchange receipts. For example, there were two
categories of interstate movement on the old survey, which is gas going
from state to state. Currently we are asking for this information under
one category.

The instructions were very complex. We have reduced the number of pages
for the form and instructions with the very important aspect of asking
for what we want in plain English.

The form was reduced from six to four pages. The instructions were
reduced from nine to four pages. Copies are available for your review
outside. Although we have eliminated several data elements, we are still
getting a total supply, at the same time obtaining a supply and
disposition balance.

We went out and conducted cognitive testing to test the redesign and
form and reduced instructions. During this process, we interviewed more
than 15 distribution and/or pipeline companies to test the redesign and
its effectiveness. Our goal was met.

Next. Data quality is an ongoing process. It has been at least five
years since EIA conducted a comprehensive update effort. During the past
year, EIA has conducted a one‑time frames maintenance effort with
supplemental contract funds with someone to review the EIA‑176
respondent list for its completeness.

More than 100 names were reviewed and assessed, mostly small businesses
and distribution companies. Some of the sources reviewed were trade
presses, PUCs, Internet source, sources such as the American Public Gas
Association, and government and state agencies, such as the Federal
Energy Regulatory Commission and the Office of Pipeline Safety and
American Gas Association.

The comprehensive update effort did confirm that our EIA‑176 frame
started out in good shape and only small entities were candidates for
addition to the EIA‑176 survey. This process is now complete.

Current to using or having implemented a new Edit 2 has been
implemented, which is a tracking system. And that is still going on,
such as reformatting a number of data elements for our ties. Our
companies under the old survey, the data was processed through a series
of data edit checks; whereas, the smaller companies received the same
attention as the larger companies. Now we can separate those out. Our
new Edit 2 performs numerous performance measures as well as giving us
an update of our non‑respondents list.

In introducing a new form to respondents, one of our major concerns is
how to make the respondents more conscious of the new EIA‑176 survey. We
are right now pre‑notifying our respondents, encouraging electronic
filing with built‑in edits.

We will provide a crosswalk, and that is from the old survey to the new
survey. And we developed a frequently asked questions list. We are open
to suggestions that would help us introduce a new and simplified,
reduced form and instructions of the EIA‑176.

Next slide. STEPS is a new processing system with a new form. The Census
Bureau uses this system. It is SAS‑based. STEPS will service the frames
management system for all surveys and functions and will analyze large
amounts of statistical data.

Cross‑system surveys will be done in SAS, rather than in STEPS. SAS will
perform statistical analysis sector by sector. For example, SAS will
compare conception data that is collected on a monthly basis from our
EIA Form 857 versus the EIA‑176, which is collected annually. It will
also compare underground storage and injections data that we collect
monthly and annually on the EIA Form 191 versus the 176 survey, which is
collected annually.

STEPS will be able to incorporate ad hoc edits that were developed
during the outside of the system; for example, companies that report for
producers where there is no production in that particular state.

Ad hoc edits will be better automated if incorporated in STEPS. And
STEPS will perform a large range of edit consistency checks. For time
and implementation, we do not have control over that in STEPS right now.
After this reporting cycle, our current contract will not be supporting
our current systems.

Lastly, STEPS must be implemented, tested, and up and running by March
2003 in order to begin processing for our next reporting cycle.

[COMMITTEE QUESTIONS]{.underline}

CHAIRPERSON CRAWFORD: Are there any questions from the committee?

DR. PHIPPS: I have a question. On the weekly survey, you noted that you
have made a number of changes to solve some of the problems. A number of
the changes sounded to me like they were respondent interpretation
problems. And I wonder if you have done anything in terms of changing
the survey form as the sample changes so you won\'t have the same
problems happen again.

MR. TRAPMAN: Well, you\'re right in that the resolution of most of these
problems have evolved with respondents. And we are a few months away
from the re‑sampling. It\'s certainly something we will revisit,
basically a suggestion.

The form, though, is, if not the simplest form EIA offers with
instructions ‑‑

DR. PHIPPS: That is, I guess, my concern. When I saw that one last time,
the instructions were totally separate from the survey form if you want
to integrate some of those critical problems into the survey form, you
would have a better chance of people, you know, people don\'t read the
instructions all the time.

MR. TRAPMAN: Quite true. Thank you.

CHAIRPERSON CRAWFORD: Any other questions or comments? Randy?

DR. SITTER: Can I ask a question concerning the supply equals
disposition? You mentioned a whole bunch of stuff about production, like
81 percent of the supply was production and that you\'re getting
something like 120 days after time of production or report.

What exactly is that survey, those monthly reports? Those are voluntary
reports by the states, correct? They come in more than just 120 days
once, right?

MS. CAMPBELL: They\'re requested to come in approximately 120 days after
the report month. At that time, about 8 out of the 32 states or MMS
sources were provided data. The rest will come in as late responders
over the next year.

DR. SITTER: But just once or do they come in and update them? Is this
the same stuff as the Texas data? Is this the same reporting?

MS. CAMPBELL: Most of them will come in once. We may go back to them
more than once because we may have some questions about their responses,
but most of them are only submitting the data once.

DR. SITTER: Is this the same survey as that?

MS. CAMPBELL: Yes, it is.

DR. KIRKENDALL: It\'s not the same survey.

DR. SITTER: I mean the same information that we have got on Texas that
John Wood says.

DR. KIRKENDALL: That\'s coming from the state. Yes.

MS. CAMPBELL: Same survey.

DR. KIRKENDALL: But it\'s a more complete set of information than we get
from most states.

DR. SITTER: From most states, yes.

MS. CAMPBELL: Texas has a substantial program of their own where
essentially they provide all the data that are needed on the 895 form
and more.

DR. SITTER: So this is the 895, right?

MS. CAMPBELL: Yes, this is the 895.

DR. SITTER: And does this thing not encourage the states to estimate if
they don\'t have the data?

MS. CAMPBELL: It does encourage them.

DR. SITTER: But you have no idea what they are doing when they estimate,
right?

MS. CAMPBELL: That\'s correct.

DR. KIRKENDALL: And, really, the best and most complete data is Texas.

DR. SITTER: Well, the reason I ask this is because obviously I am
looking at the Texas data. And if you were to take the ratio of the
current, say, one month data they give you two months after, three
months after, four months after, and you ratio that to the final value,
the number goes above one quite often.

Now, you wouldn\'t expect that to be able to happen. So the only way
that that is probably happening is whatever estimation procedure they
are using. So you have encouraged them on the form. If you don\'t know
the exact number, then give us an estimate.

MS. CAMPBELL: Right.

DR. SITTER: But you have no idea what they are doing.

MS. CAMPBELL: No, none.

DR. SITTER: Well, obviously what you would do if you didn\'t have all
the data in here, whatever you would do, you would inflate it. So it
seems from the data that they are sometimes overinflating it. And where
that is happening is around 120 days, which is the one you\'re using.

MS. CAMPBELL: For Texas?

DR. SITTER: No, no. I understand.

MS. CAMPBELL: And this is all Texas only.

DR. SITTER: Yes. I understand that. But what I am saying is that, at
least in the Texas case, which is the biggest producer of the ‑‑

MS. CAMPBELL: Right.

DR. SITTER: ‑‑ biggest part of your supply, you are using the point
where they have almost all of their information in and they\'re
inflating it.

MS. CAMPBELL: Yes, they are.

DR. SITTER: And, therefore, they have the potential to overinflate it.
That\'s exactly where they are overinflating it. So I think, though I
haven\'t looked at it from that viewpoint, there could be a positive
bias in that particular point that you introduced into your
supply/disposition equality equation.

CHAIRPERSON CRAWFORD: Jay

VICE CHAIRPERSON BREIDT: Is there any interest in direct surveys of
producers or is that just not in the cards at all?

MS. CAMPBELL: No. That\'s what I was saying. That\'s under study. We are
looking into that. The issue there is essentially developing ‑‑ several
issues. One of them is finding a frame. That is to say, would it be
producers, operators? At what level?

I mean, the best frame that EIA has right now is the EIA‑23, which is
designed for collection of reserves information. And it does current
year reserves, prior year reserve, additions to reserves, et cetera. And
production is a component of that survey, but it\'s really going after
reserves information.

And it has how many times responded?

PARTICIPANT: Fourteen hundred.

MS. CAMPBELL: But that is a subset already. In other words, they have
already chosen a smaller group to collect data on because there are many
more producers than that.

So the issue is essentially what would your frame be and is that the
right group of people to get current production from, where would you
collect it.

The other thing is that right now the same state survey is providing us
whatever we are getting with respect to average wellhead price data. And
so one thing we are looking into is whether or not you could get
production data at the same point that you could get price data or are
you forced to go to two survey populations.

Generally engineers can give you pretty timely production‑type volume
data, but where you do you go for your price data? So if that is the
prospect, then you are splitting these into more than one survey.

So that is the point of the analysis tests that we have going right now,
trying to understand what our options and problems might be.

VICE CHAIRPERSON BREIDT: Thank you.

MS. CAMPBELL: Certainly that\'s what people who are doing their own
studies ‑‑ you see a lot of coverage in the trade press, people\'s
forecasts of production or their statements of what has recently
happened.

Basically they\'re looking at production data that have come out of
10(k)\'s or SEC type reports or some subgroup of people that they know
and talk to directly.

CHAIRPERSON CRAWFORD: Any other questions or comments?

(No response.)

CHAIRPERSON CRAWFORD: Thank you. Thank you very much. We are scheduled
now for a break and to move to the break‑out session. So as you take a
bit of a break, then Mark, Jay Edmonds, Jim, and Bill will go downstairs
with Doug Hale, who is sitting over here in the corner, room 5E‑069.

(Whereupon, the foregoing matter went off the record at 10:57 a.m., to
convene into separate break‑out sessions, and went back into on the
record at 12:11 p.m.)

AMERICAN STATISTICAL ASSOCIATION

COMMITTEE ON ENERGY STATISTICS

FALL MEETING

BREAK‑OUT SESSION:

USING DATA FROM COMBINED HEAT AND

POWER PLANTS TO ESTIMATE

NATURAL GAS INDUSTRIAL PRICES

THURSDAY,

OCTOBER 24, 2002

The meeting was held at 8:30 in Room 8E‑089 of the Department of Energy,
1000 Independence Avenue, S.W., Washington, D.C., Ruey‑Pyng Lu,
presiding.

P‑R‑O‑C‑E‑E‑D‑I‑N‑G‑S

(11:16 a.m.)

MODERATOR LU: Good morning. I\'m Ruey‑Pyng Lu from the statistics and
methods group of EIA.

In the previous hour, Beth gave you the update of the natural gas data
program. One of the analyses underway is trying to find alternatives for
industrial sector price estimation. And this is the topic.

An outline of my presentation will be to give you the statement of the
problem first and then a little bit about the previous work on the
natural gas industrial price, what is the problem with the work being
done. And then we try to give you some little bit of idea we have of the
alternative approaches. And we need your help to give us when we start
this project.

The problem is the percentage of the industrial natural gas deliverables
are delivered to the on‑system sales declined. You can see that on this
picture on the graph since 1989 about 37 percent down to the 18 percent
now. Okay?

So in the table, 24, one‑page table, 24, you have natural gas annual.
You can see that\'s the natural gas annual 2000. In the middle part is
the industrial sector. On the bottom is 18.1 percent. It\'s exactly the
year 2000. The percentage varies from state to state. Alaska, Hawaii
covers almost 100 percent, but 48 continent states, some down to only 2
percent.

What this means, industrial customers buy more natural gas from
marketers or the suppliers than from the local utilities. Local
utilities only deliver to accounts of others.

So when we say the on‑system sales are those of local utilities, local
distribution companies sell the natural gas to the customers.

EIA has asked the committee since 1986 about this problem and get help.
So the committee\'s suggestion was go ahead, survey the end users to get
the quantities of the natural gas purchase and the price and how much
they pay for it.

So two years ago, we contracted the Bureau of Census. So they conducted
a feasibility study to get a monthly industrial sector price and also
the volumes in the manufacture sector. So they have frames of the
manufacture sectors or the companies.

The work result was promising, but it cost a lot of money. They asked
about one million dollars, which can help us to conduct the survey,
monthly survey, to get every state the industrial sector natural gas
price.

So we have put a hold on that and tried to use our data series anywhere
we can get the alternative estimate of the natural gas price in the
industrial sector.

We have five data sources. The first one is the EIA‑423, which is
conducted by our CNEAF Office. It\'s a monthly cost and quality of fuels
for combined heat and power plants. Let me give you an explanation of
what is a combined heat and power plant.

The companies purchase the natural gas so they can generate the heat.
They use the heat to generate electricity. And then after the heat has
generated electricity, it can be used for heating something or they do
other things. So it\'s a combined heat and power.

That\'s EIA‑423, trying to get how much the cost and the quantity of the
fuels. So that is including the coal, natural gas, and the petroleums.

PARTICIPANT: Did you mean quantity or quality?

MODERATOR LU: Quality, yes. They have some ‑‑

MR. FENNELL: It\'s both.

MODERATOR LU: Both.

MR. FENNELL: It\'s quantity and quality.

MODERATOR LU: Yes. We have Dean Fennell. He is the supervisor of the
EIA‑423 data series.

Then we have another data series. The source is FERC‑423. That\'s a
utility. That\'s a regulated power plant. That data is available on the
Web for three years. It is monthly data.

Then in the middle, we have what is called a manufacturing energy
consumption survey, MECS. And it\'s conducted every four years. The
latest one is 1998. In 2002, it will go out to next year. So the data
won\'t be available after 2004. So the most recent one is 1998.

This survey carries the series with the region, census region data. So
go by the SAIC code, the industrial cost, of how much is being consumed
by each different manufacturer companies.

Then we have EIA‑176 and 857. These are the surveys conducted in the
Natural Gas Division. So with these five series, we want to know can we
use this to find a way to get a better estimate of the natural gas
industrial price.

VICE CHAIRPERSON BREIDT: Do these consumption surveys have price
information or just ‑‑

MODERATOR LU: Yes, 846 has price information. And we have MECS manager
here. So he can help me to answer more.

DR. KIRKENDALL: The problem with MECS is that it\'s collected for us by
the Census Bureau. And so we only have access to the data by becoming a
sworn census agency, going out there and looking at the data.

DR. SITTER: That\'s the only one? The one that is held every four years
is the only one with price information?

DR. KIRKENDALL: No. All of them have price information.

DR. SITTER: All of them?

MODERATOR LU: Yes, all of them.

DR. KIRKENDALL: Different strengths and weaknesses for all of them.

MODERATOR LU: Yes. Okay. So let\'s look at the monthly price
information. What do we have? Okay. On the one side, if it\'s power
plant and with a 50 or more megawatt capacity, if it\'s regulated, then
we can get data for FERC‑423. If it\'s unregulated, we call it combined
heat and power plant. And that is in our EIA‑423.

All the other industrial commercial can be found in the EIA‑423 or the
MECS data. So I will give you the partition how we get the monthly
prices.

Then from the Natural Gas Division, the natural gas purchases and
deliverables, these are from the local distribution company. So you can
see one part is on‑system. And, thus, the data shows in 2000, we only
count 18 percent. So we have the volume and the delivery cost, both in
the industrial and the electric power sector.

The off‑system is local distribution company. They deliver the natural
gas to a user, but the charge accounts are for others, which we call the
off‑system. In that part, we only have the volume. We don\'t have the
price because we survey the LDC. They don\'t know the price.

So this means on the off‑system, about one in 80 percent, that\'s what
we are anxious to find. And we wish we could have this kind of a
template to display for the variable information.

Ideally we like to have each state. Okay? If the confidentiality is the
problem, then we have to go to one aggregate level up, which is the
census division. If the divisions do have a problem with the
confidentiality, we may just have to go out to the census region. Okay?

And the numbers here give you the gas‑burning companies. And we can make
another one which is the volume, and another one is the price. So this
is just one example. We have how many companies in each state or each
division. And then there will be another one, how much volume in each
state and then also the price.

CHAIRPERSON CRAWFORD: Did you do this or you are just thinking about
doing this?

MODERATOR LU: Yes, we are thinking we have that available to us but that
we may not have it complete.

CHAIRPERSON CRAWFORD: Okay. So I guess my real question, then, is do you
have a feeling for how complete these tables might be or would they be
mostly complete or mostly incomplete, I mean, that they would actually
be able to report numbers in each one of those cells?

MODERATOR LU: Yes. We can get for these three the number of companies in
each state, yes. And that I can get from the CNEAF people. Those are the
volume. But the price will be the confidential pieces. I have to be very
concerned.

MR. HENGARTNER: Since we\'re talking about electricity‑producing
entities, is it possibly the price of electricity?

MODERATOR LU: Price of electricity. Oh, yes.

MR. HENGARTNER: If you don\'t know the price of gas but you know there
is some electricity and you know the price of the electricity, you know
they probably didn\'t pay more for the gas to get in. So that at least
gets you some ways of guessing what the price might be.

DR. KIRKENDALL: I don\'t know that they all sell it. Sometimes they have
an arrangement with the grid. I don\'t know how that goes on. You know,
a lot of these are manufacturing companies like paper plants. There are
a lot of paper plants that are in there, in the industrial.

It\'s not that we don\'t have the information from them.

MR. HENGARTNER: It\'s not useful.

DR. KIRKENDALL: We hope it\'s useful. We haven\'t done that evaluation
yet.

MODERATOR LU: Thank you. Yes.

DR. KIRKENDALL: But before we decide how much we can use, we have to
look at if there are only three respondents in a state or less than
three respondents in a state, we may not be able to use the price data
anyhow because of confidentiality. So that\'s kind of a constraint on
the final product.

DR. SITTER: Can I clarify Carol\'s question?

MODERATOR LU: Yes.

DR. SITTER: You can fill every box essentially for the number of gas
frame companies in that table.

MODERATOR LU: Almost, yes.

DR. SITTER: And essentially every box per volume.

MODERATOR LU: For the EIA‑423, I think so.

DR. SITTER: But then when it comes to price, I\'m not talking about the
EIA‑423, but you can have a similar table for price.

MODERATOR LU: Yes.

DR. SITTER: So which boxes are empty, and which ones are full,
electricity, industrial, electricity?

DR. KIRKENDALL: We should be able to fill all of those once he gets
access to the data.

DR. SITTER: The price?

DR. KIRKENDALL: Yes. We have the price.

MODERATOR LU: We have the price from ‑‑

DR. KIRKENDALL: It\'s just that it is sensitive. So he is trying to
stress the confidentiality of the data, which is really more of the
final result of the analysis. You know, it\'s whether we could show it
in a public forum or not, rather than whether we can use it or not.

CHAIRPERSON CRAWFORD: So you can use it, but you may not be able to show
it?

DR. KIRKENDALL: Yes.

CHAIRPERSON CRAWFORD: Or you can\'t use it?

DR. KIRKENDALL: We can use it. I mean, we can combine it with other
things. If we could combine it with other things in a way that would
protect it, then we could use it for that. But we do have to watch the
sensitivity.

MR. FENNELL: The EIA‑423 price data are confidential. The FERC‑423 price
data are not. And that is part of the problem. We began collecting the
423, the EIA version, beginning in January of 2002.

And we were just looking at the sensitivity of how we could publish this
in the upcoming issues of Electric Power Monthly. Right now we are
evaluating whether we can publish at the Census Division or state or
whatever avenue, but we know we cannot publish at the detail because of
the fact it\'s confidential. Ruey‑Pyng can have access to it. It\'s just
the availability of what he is going to do with it and how it is going
to be published later that is going to be at issue.

DR. SITTER: I\'m kind of lost. I thought you had an estimation problem
here.

MODERATOR LU: We do.

DR. SITTER: Then you\'ve lost me in the estimation problem.

DR. KIRKENDALL: Okay. This is data that we have.

MODERATOR LU: This is data we have. Many new ‑‑

DR. KIRKENDALL: What we really want is an estimate for prices in the
industrial sector for the manufacturing industrial sector. We don\'t
have that. What we have are the CHP plants. They\'re clearly part of the
industrial sector, but they are not the whole sector.

DR. SITTER: Okay.

DR. KIRKENDALL: So part of the problem is, will these companies and this
data provide a suitable proxy for industrial prices in natural gas?

DR. SITTER: So all of the tables here are showing they only cover the
on‑system stuff is what you are saying?

MODERATOR LU: Yes.

DR. KIRKENDALL: No.

MODERATOR LU: No?

DR. SITTER: They also cover the on‑system?

DR. KIRKENDALL: That is all entirely ‑‑

MODERATOR LU: The on‑system is no problem.

DR. SITTER: The on‑system is no problem.

DR. KIRKENDALL: No.

MODERATOR LU: It\'s off‑system here. Okay? See, we only have a volume.
We don\'t have a price.

DR. SITTER: Which part of this sector are we missing?

MODERATOR LU: We have this from here, most of it from here, from 423.

DR. SITTER: Right.

MODERATOR LU: But in this one, we may have part of that for this reason.
We don\'t know how much.

DR. KIRKENDALL: We may learn that from the MECS.

MODERATOR LU: Yes.

DR. KIRKENDALL: But what you are missing from sure is anybody that does
not generate electricity. So they can use natural gas as part of their
industrial processes, but if they don\'t generate electricity, they
don\'t report on the 423.

DR. SITTER: I see.

MODERATOR LU: This is 18 percent here. This is 82 percent.

DR. SITTER: You know, I understand that. I just didn\'t understand that
you were recovering. Okay.

MODERATOR LU: Yes. So far this is only volume here, no price. Hopefully
we can get a price of this piece from 423.

DR. SITTER: I see.

MODERATOR LU: But how much we can get from 423 on these pieces we don\'t
know.

DR. SITTER: But some of the 423 is in what you are calling industrial,
things like pulp and paper plants that also sell electricity.

MODERATOR LU: Yes.

DR. KIRKENDALL: Yes, right. Yes, definitely.

DR. SITTER: So these cogeneration plant types of things are the ones
that are showing up for you.

MODERATOR LU: Yes.

DR. SITTER: But not the ones that produce no electricity for sale.
That\'s right? Thank you.

PARTICIPANT: Do you have an educated guess? Is it 50 percent of it or is
it 20 percent of it? You have no idea how much 423 covers?

DR. KIRKENDALL: I don\'t know what the percentage is, but that ought to
be available from the MECS because the MECS will tell you whether plant
‑‑ I don\'t know whether they separately identified that, but they know
which plants generate power and which don\'t.

MR. KASS: I can give a body count. The 423 goes to something like 600
different facilities. There are probably something like in excess of
15,000 facilities at the industrial sector that we don\'t have a price
for.

What we are asking for is can we figure out some proxy or some sort of
indicators to help us get enough fuel.

MR. FENNELL: In addition, the 423 only collects information on fossil
fuel data. So if it\'s not coal, oil, or gas, we wouldn\'t be collecting
it.

DR. KIRKENDALL: This one is only interested in natural gas.

MR. ADLER: Another problem with the MECS is though we can separate out
the generators from the non‑generators, we would have some trouble in
determining whether ‑‑

CHAIRPERSON CRAWFORD: Excuse me. I am sorry to interrupt, but would you
go to the microphone? We can\'t hear you down here very well.

MR. ADLER: The problem with the MECS is that although we can separate
out the generators from the non‑generators, we can\'t make, let\'s say,
a hard determination that what is generating is natural gas if there are
multiple fuels being used. That is not the way our survey is currently
set up.

So we could get some good guesses, but right now we would have trouble,
especially in the cogeneration setup, which would be most of the
generation. If it were conventional, we do have that broken out. But
that is a small amount of the generation.

DR. SITTER: And I take it you have on both the 423 and on the MECS some
common covariance?

DR. KIRKENDALL: Probably some.

MODERATOR LU: You mean the covariance?

DR. SITTER: Covariance, common characteristics of companies and so
forth.

DR. KIRKENDALL: There\'s some. You have the SAIC code ‑‑

MODERATOR LU: Yes, but it ‑‑

DR. KIRKENDALL: ‑‑ or NAICS code depending on your year.

MODERATOR LU: Yes, but MECS does not have the 22\'s. They have others,
right? See, they don\'t have the 22\'s, the power plants.

DR. KIRKENDALL: Because they are not manufacturers.

MODERATOR LU: See, MECS may not have electric power, this piece. MECS
have this mostly here, not here.

DR. KIRKENDALL: But that\'s okay because we\'re not interested in the
electric power.

MODERATOR LU: Yes.

DR. KIRKENDALL: We have data on that. It\'s really the industrial one
that we\'re trying to do ‑‑

DR. SITTER: Well, what we really want to know is to get a relationship,
some sort of relationship, between the cogeneration part of industry and
the non. And you have that kind of information on MECS.

And then if you have some kind of common covariates between the 423 and
the MECS on those, then you may be able to use that to predict or to
estimate the relationship. Of course, it\'s only every four years.

MODERATOR LU: Yes.

DR. SITTER: So it\'s only some sort of broad characteristics, not
necessarily anything month. But you do have the volume, which you are
hoping is. But that doesn\'t really control the monthly fluctuation of
prices.

MODERATOR LU: Yes. MECS is a sample survey also.

DR. SITTER: Sample surveys can be lots better than census. Keep that in
mind. Otherwise we\'re out of business.

(Laughter.)

MODERATOR LU: Okay. What I have in mind is to look at the relationship
between the MECS on‑system and off‑system first, try to relate to the
natural gas prices, 857 and 156, and then trying to see the utility
price and the off‑system price in the MECS.

VICE CHAIRPERSON BREIDT: When you say \"utility,\" does that mean a
regulated ‑‑

MODERATOR LU: Yes, regulated. Yes, 423, FERC‑423 data, yes. And then
ideally we want to know if there is a strong relationship between or
what kind of relationship between the utility and non‑utility, the
cogens. So now we want to know.

So put everything together and see if we can derive at a set of
estimates for the off‑system. We have less than 20 percent, what the
other 80 percent would be.

Then if that is feasible, then we can have a procedure to combine those
things together.

DR. SITTER: I thought you have information on some off‑system stuff from
both the MECS and the 423.

MODERATOR LU: Yes. But the pieces are how much they cover each other.

DR. SITTER: Small, you mean?

MODERATOR LU: Yes, very small. And the input on this is much
appreciated. So let me just go back to what do we have here. We want to
estimate a monthly industrial sector natural gas price by state, so 48
states.

You can see that on‑system sale. The percentage is very low. Like Idaho,
Montana, only two percent. Okay? And some states like Michigan are only
17 or 8 percent or something like that, yes, 8 percent. Some, you know,
some industrial states, the percentage is still low, which means that we
have concern.

And also we sort of have a lower bound of the industrial sector price
because from the electric power company, they purchase the bulk of the
natural gas. Usually they have the bargaining power. So they get the
best price.

So we can treat that as a lower bound, but then the upper bound, in the
last year industrial is the average of 4.48 on‑system. The electricity
is only 2.65. So that\'s quite a range. So we are trying to see what
will be the estimate.

CHAIRPERSON CRAWFORD: What about the other sectors, like the residential
sector? Do you do something similar for that? And do you have more
complete information?

MODERATOR LU: Well, see, the residential sector, you can get about 90
percent on‑system sale. In the commercial, you get 63 percent.

DR. KIRKENDALL: But also what they did, ‑‑ I think Beth mentioned this a
bit ‑‑ is they have their new marketer survey that was in place this
year. And they\'re evaluating the quality. That gives them prices
charged by marketers in the residential and commercial sectors but not
in the industrial sectors.

And so, with luck, beginning in January, they will be able to use that
information to fill the gap in those five states and we can move forward
on those sectors.

DR. SITTER: Is this absolutely correct? The way that I view this is
there is no overlap. I mean, do you get information on number one on the
right‑hand side of the 846 as well or is the 846 exclusively ‑‑

MODERATOR LU: You mean in this?

DR. SITTER: Well, it sounded like you got information on the same part
of the frame from both the 846 and the 423 at some point, not on this
table.

MODERATOR LU: Yes.

MR. ADLER: They would intersect at various samples.

DR. SITTER: Which category would those establishments fall? Generate
electricity at 50 or more megawatts?

CHAIRPERSON CRAWFORD: You can sit down here if you want. There are all
these empty chairs.

DR. SITTER: This is supposed to be a discussion, I think.

MR. ADLER: Can I be heard? Essentially the MECS goes after manufacturing
establishments. It\'s a sample. And it really doesn\'t depend initially
on whether they generate electricity or not.

But we do happen to get the largest establishments that would be in the
business of generating electricity as well as manufacturing. And we do
collect the amount that they are generating as part of the energy
picture.

So we would presumably get a lot of the same establishments that would
be in the 423. A difference would be that we are trying to get the fuel
use at the entire establishment, not just for the electricity generation
part.

I assume for the 423, it would be just for the generation facility.

DR. SITTER: So it looks more like big producers of electricity cut into
two pieces, industrial, so cogeneration and not. And then the 846 kind
of overlaps it from two different fronts.

See, because when I was reading the slides on this, it didn\'t seem like
I could get any overlapping information. But it sounds like you have
some overlapping information.

MR. ADLER: Well, the FERC‑423 we should not have anything on the MECS.
It would be the unregulated. The EIA‑423 should have some that we do.
There should be a big overlap.

But with the MECS, we won\'t have the entire industry either, only
manufacturing. So there are things. Whatever would be going on in mining
and construction and agriculture would not be part of the MECS, which is
part of the industrial sector. So that\'s another issue.

DR. KIRKENDALL: So yes. The slide is misleading. You actually have MECS
available also for a number.

DR. SITTER: Yes. Well, you could create three categories probably.

MODERATOR LU: The other one is they combined both.

DR. SITTER: The ones that are both big producers of electricity and are
in the ‑‑

DR. WHITMORE: You\'re saying even where they\'re combined, you\'re
saying that you may not be able to separate out in the MECS the
electricity generation.

DR. KIRKENDALL: Matching respondent by respondent. It\'s frustrating
because there are lots of different reasons why they don\'t report the
same numbers. We have tried to do that on occasion.

VICE CHAIRPERSON BREIDT: But ultimately you just want natural gas
industrial prices. You don\'t care what it is being used for, right?

MODERATOR LU: Yes, yes.

DR. KIRKENDALL: That\'s true. For this purpose, that\'s what we want.

MODERATOR LU: Yes.

CHAIRPERSON CRAWFORD: How badly do you want this?

DR. SITTER: Less than a million dollars worth.

CHAIRPERSON CRAWFORD: I\'ll tell you it certainly sounds easier to me to
ask for more money than figure this problem out.

MODERATOR LU: Yes.

MR. BLAIR: The obstacle to simply collecting these data that are missing
is simply cost?

DR. KIRKENDALL: That\'s why we didn\'t want to go to the Census Bureau,
who has the frame.

MR. BLAIR: Right. But why not, can you control costs by collecting some
subsample of the data that you are interested in and then using that in
conjunction with this procedure that Randy suggested for estimation?

I guess if you have a little bit of information about this missing
sector, that is a lot better than none as far as producing an estimation
procedure.

MODERATOR LU: Yes, but we don\'t have the frame even with trying to do
the sample. Census has.

DR. KIRKENDALL: The other thing that would be interesting, this project
aside, a survey of cogenerators is one that has been developed over the
years and is becoming more and more important to EIA we are going to
hear about this afternoon.

As it gets to be bigger and bigger, one question is well how much of the
manufacturing sector do we have there? The goal is that that frame
should be anybody that generates electricity with some size limits.

And if it\'s true that if we have anybody that generates electricity,
that should be a lot of the manufacturing sector. And if we\'ve got
that, can\'t we use it for a lot of interesting things?

So that\'s kind of a piece of the puzzle. And maybe one of the things we
would sort of like to see is if these cogenerators are a part of MECS,
this particular set of things, then we can use that data for other
purposes.

MR. ADLER: They\'re a big piece of it in terms of energy use because
they are generally the largest establishment. As far as numbers, they
are really not. I think it is about 1,000 that we estimate. So don\'t
hold me to that number, I think the number that they represent in terms
of numbers of establishments is a lot less as a percentage than it is in
terms of the energy.

DR. KIRKENDALL: I always suspect what\'s missing are the small ones that
might hold the lower threshold.

MR. ADLER: These are also what we would characterize as medium to pretty
large. It\'s substantial, and it probably makes up a large portion of
the non‑utility generation, but I don\'t know that those make up as much
of the entire manufacturing sector.

DR. KIRKENDALL: What we would always be missing is the piece that
doesn\'t generate electricity.

MR. ADLER: Right.

DR. SITTER: Well, I think those are the important things to do, to look
at what you are missing. Think of it as if I have this information, how
could I possibly use covariate information, overlapping frames to get an
estimate, then look at the precision of that estimate and ask yourself
where you need the more information, instead of saying, \"Just what is
the best estimate I can get from all of these things?\" because I assume
that periodically, as you say, you may have more than you know you have.
That is, you may have more of the industrial sector than you think you
have.

MODERATOR LU: So that means we may have to gather the data series trying
to match things, whether it\'s being available or what is not, and see
how much is being covered.

DR. SITTER: Well, I think it is just a general exercise. You have some
idea of what you have. You can at least know where you are going and ask
yourself what you need. The answer may be, well, it\'s just as costly to
use all of this information to go to the census and start from scratch.
I don\'t know.

But it may be a matter of changing one of these collections or asking
one more question somewhere or changing the sampling point for one thing
and you\'ve got what you need or it may be that you just need something
smaller to fill in this gap.

DR. KIRKENDALL: Actually, one of the things, the evaluation that we did
with census before, we were asking for state‑level data either monthly
or quarterly.

And, of course, if we back off on state‑level, which we might not be
able to do anyhow because of confidentiality, then maybe that opens up
those discussions again with census for something small.

DR. WHITMORE: Would it be less expensive if we were talking about the
census doing a sample survey versus census?

MODERATOR LU: Census, that\'s a sample survey, yes, monthly sample
survey.

DR. KIRKENDALL: It wouldn\'t be a sample survey. It would be a
subsample.

MODERATOR LU: A subsample of that, yes.

CHAIRPERSON CRAWFORD: You would still need monthly data in the end,
right?

MODERATOR LU: Yes.

DR. KIRKENDALL: Actually, to fill in the data gaps that we\'ve got now,
we would need it.

CHAIRPERSON CRAWFORD: I\'m wondering if the price would go down if
you\'d still have to ‑‑ you know, you\'ve got to do it for a sample size
if it doesn\'t matter if it\'s a monthly collection.

DR. KIRKENDALL: Actually, I think we did ask about quarterly before, and
it really wasn\'t a very big difference to do it quarterly.

MODERATOR LU: Yes.

DR. KIRKENDALL: We were sort of surprised. You would think that doing it
three times more often would be.

CHAIRPERSON CRAWFORD: Yes.

DR. SITTER: But do you think that going away from state‑level is going
to save you anything? If they are already basically piggybacking on a
sampling frame that they are using, you are not going to change it.

DR. KIRKENDALL: Well, the sample, in fact, was developed to get the four
region estimates, right? So it wasn\'t originally designed to give
state‑level information.

DR. SITTER: So they are actually thinking of actually having to go to
more units just to ask the questions you want?

DR. KIRKENDALL: I don\'t know that they actually ‑‑

DR. SITTER: I can\'t imagine that that would only cost a million
dollars.

DR. KIRKENDALL: I don\'t know the details of what they were thinking
about and what they weren\'t thinking about.

MR. ADLER: Partly, it would be whether you need it by industry by state.
If you could just cut out the industry classification, maybe it would be
cheaper. I don\'t know whether that ‑‑

MR. KASS: If I remember correctly, their design included piggybacking on
existing establishment surveys. It was not that they were going to for a
million dollars do a survey for us. They had a monthly survey that we
were going to piggyback on and expand to deal with our needs.

DR. SITTER: So you put a Question 23 on the survey, and then it costs a
million bucks.

MODERATOR LU: Yes, more or less like that. Yes, ask three more questions
along with their existing surveys.

MR. HENGARTNER: If the Census Bureau would do the survey, you would not
get the actual data but a simplified version of it, correct?

DR. KIRKENDALL: Right.

MR. HENGARTNER: So you couldn\'t do the kind of exercise you are doing
now saying, \"Now that we have that data, can we use it somewhere
else?\"

DR. KIRKENDALL: Right.

MODERATOR LU: So this is a challenge to things. We may get a little bit
extra and they are going to pay the million dollars. You won\'t be
getting worse as the off‑system percentage with higher and higher. Your
on‑system will be rough, talking down.

DR. KIRKENDALL: We have projections that say that.

MODERATOR LU: Yes. So you can see that it\'s coming down, down.

DR. KIRKENDALL: It came back up again. States are a lot less excited
about deregulation than marketing with the Enron bust stuff. And so
maybe there will be a little bit of a trend there, but it isn\'t going
to come back far enough so that our prices are breaking, though.

DR. SITTER: Do you know your prices are bent?

DR. KIRKENDALL: We only know that ‑‑ well, we don\'t know anything about
the prices. Actually, that may not be true. Roy, have you done
comparisons with the MECS prices in your prices?

MR. KASS: No. There\'s good reason to believe that it\'s supply side.

DR. KIRKENDALL: That your cost of ‑‑

MR. KASS: It\'s supply side. Verbally, to go through the argument, if I
am a big establishment needing a lot of gas and I can buy for
approximately the same price from my local distribution company, why
would I want to go through the bother of arranging my own deals?

I have to do it for savings. Following that logic, I say that our price
is higher than an overall price. And, as Ruey‑Pyng said, what is a low
estimate? Again, following the logic, the guys who can cut the best
deals are the power companies. And they\'ve got indicators there.

The problem is we\'ve got a difference between 265 and 450 roughly.
That\'s an awful big range.

DR. KIRKENDALL: So 265 is a price for power sector in four?

MODERATOR LU: The power sector in 2000 is a 438. It\'s closer to the
448. But the previous years, they are more apart, like 240 to 314.

DR. KIRKENDALL: That\'s part of the reason why it\'s interesting. There
is such a big gap in there. That would be nice to know a little better
how much industrial, further the industrial prices fall.

They probably don\'t have a leverage of the power sector, but if they
are going off‑system, they are probably doing better than the LBCs.

PARTICIPANT: Ruey‑Pyng, did you say that the industrial prices are
getting closer now?

MODERATOR LU: Yes because that is 18 percent. Remember, that is 18
percent, which is 4.48 here. And electric utility is 4.38.

PARTICIPANT: That\'s kind of interesting. And this is purely anecdotal
with one respondent we talked to when we went out and did the
interviews. When their staff was talking about the different sectors, we
were asking them how they defined them. One of the staff just happened
to mention that they had some special rates for industrial customers so
that they wouldn\'t use them to the marketers.

So I am wondering if as deregulation continues to go on utilities
aren\'t going to get special dispensations from their public service
commissions to cut special deals with industrial customers so they can
remain competitive.

Again, that was one respondent happened to mention that in an interview.
It\'s kind of interesting. They said the percentage has gone up a little
bit, although it\'s still a large percentage.

MODERATOR LU: Yes.

MR. HENGARTNER: So, following up on your suggestion, did you look ever
at the price versus the volume of the data that you have currently? I
mean, if there is a relationship like you suggest, we should be able to
see it in the data, even the one you have collected so far.

MODERATOR LU: You mean ‑‑

DR. KIRKENDALL: That\'s an interesting thought.

MR. HENGARTNER: For each facility for which you have a 423, the MECS,
you have price, volume. They could plot the one against the other. I
don\'t know how it is going to look like.

DR. KIRKENDALL: No. That would be interesting to look at.

MR. HENGARTNER: That\'s exactly your intuition. If there is something
there, we might see it. We might use that as a strawman for now to try
to estimate the price.

MR. ADLER: In terms of the \'98 MECS that 15 percent is a lot ‑‑ well,
we still have a lot more on‑system consumption, call it, in that it
actually looks more like the 1992 ratio.

MR. BLAIR: Those were actually sales, as opposed to on‑system
deliveries, right?

MR. ADLER: What the MECS has is what I was looking at was just direct
purchases, regardless of the consumption. I am not sure what this is.
Deliveries from the LBC or ‑‑

MR. KASS: Delivery and sales from the LBC.

MR. ADLER: They should line up somewhat closer.

PARTICIPANT: So it includes deliveries on behalf of?

MR. KASS: No. It\'s only deliveries and sales from their inventory.

MR. KASS: Right. But it\'s sales.

MR. ADLER: And it could be a problem in that MECS respondents are
calling something a utility that is not a utility.

DR. KIRKENDALL: And they may not know who owned the gas either.

MODERATOR LU: They may just combine together because they may probably
be part of it under the LBC and the part from the marketer. There is a
contracting thing. So they may combine those things to report it.

MR. ADLER: It\'s been a problem to train them. Again, it\'s always the
people who aren\'t doing it that you have to worry about that aren\'t
contracting out. Do they get it right? I don\'t know. We are getting a
higher amount.

MR. FREEDMAN: There was a comment made earlier suggesting that the
electric utilities will always get a better deal than industrials. And,
yet, it seems to me that there are some very large industrials that have
as much pricing power as an electric utility does. I mean fertilizer
plants, petrochemical plants.

I am almost wondering if you can sort of segment the industrial industry
by facility size in a sense and do a match‑up that way. It\'s ‑‑

MR. HENGARTNER: That is the direction we were looking at versus price
and adding two colors, utility or non, that seem to fall in the same
survey.

MR. FREEDMAN: Because there are also small utilities out there.

MR. HENGARTNER: Oh, yes.

MR. FREEDMAN: I mean, there are some very large utilities, but there are
also some small ones.

DR. KIRKENDALL: That\'s why Nick\'s plot would be so interesting.

MR. FREEDMAN: Segmenting the data that way.

DR. KIRKENDALL: He is just saying take the data and plot facility size
as a function of price.

MR. FREEDMAN: Right.

DR. KIRKENDALL: And see if a model leaps out at you.

MR. FREEDMAN: Yes. I think that is a reasonable approach.

MODERATOR LU: The volume versus price.

MR. HENGARTNER: Volume versus price.

DR. KIRKENDALL: Yes.

DR. WHITMORE: How about volume versus price differential? I guess an
individual facility wouldn\'t have both on‑system and off‑system
purchases. I don\'t know if there is any way of getting at a price
differential, on‑system versus off‑system, as a function of volume.

MR. FREEDMAN: The only other variable I would add is the number of
interconnections of that facility to multiple systems because in Texas,
you can have a petrochemical plant that may be tied in to three or four
different transmission systems. And you can also have electric utility
plants that are tied into more than one transmission system. And that
gives them additional bargaining power both with the transporter and
with the producer of the commodity.

MR. HENGARTNER: The other thing is what do you define to be the price of
the gas. I mean, if I am a utility, in winter or fall, I would buy
futures or things like that.

I mean, people will do things like that. They will try to hedge against
all the winter. And now they might buy right now. Then when they use it,
they\'ll say, \"Well, they\'ll be going along through contracting.\" I
would expect even the notion of what the price is that is what they are
going to report to you.

I mean, when you ask, how much did you pay, you know, I have a
combination of having to buy on the spot and the future contract. I
don\'t know what the price is. They probably pay a variety of values for
the gas they have covered over a whole month versus one single number.

DR. KIRKENDALL: Well, what they are supposed to do is report the total
that they paid for gas and the total gas they bought.

MR. HENGARTNER: Okay. That might be a quite complicated exercise for a
big company. I am just thinking aloud that this might not be such a
novelty state. I can see how to calculate the total easily.

MR. FREEDMAN: Well, the other problem you get into is you could have
parties doing swaps. I\'ll sell you electricity, and you provide me the
gas. And what do you value either commodity at?

DR. KIRKENDALL: This one probably doesn\'t get at that.

MR. HENGARTNER: I know, but I\'m just saying there are a variety of
complications. Since we agreed that the price is not clearly defined at
what it is, that we don\'t exactly know what price we are quoting on.

MR. FREEDMAN: Right. Well, but it also goes back to your point that they
may not know what the value is.

MR. HENGARTNER: Yes. That\'s exactly what I am saying. They don\'t tell
us because they don\'t know.

MR. FREEDMAN: Right.

MODERATOR LU: Roy, do we know the percentage of this one?

MR. KASS: What we have is ‑‑

MODERATOR LU: In terms of that?

MR. KASS: Percentage of price?

MODERATOR LU: No. Volume, the off‑system volume in the ‑‑

MR. KASS: We have a weak indicator now, but yes.

DR. KIRKENDALL: What percentage?

MR. KASS: I would have to look that up.

DR. KIRKENDALL: You haven\'t used your own data for a while.

MR. KASS: No. What we have in the 176 is deliveries to electric
utilities and other kind of select utilities. What we publish is the ‑‑

DR. KIRKENDALL: Right.

MODERATOR LU: If we have some kind of relationship mingling here, then
the price may pop out somehow.

DR. SITTER: What has been tried?

DR. KIRKENDALL: He doesn\'t have the data yet.

MODERATOR LU: No. I don\'t yet.

DR. KIRKENDALL: He is still trying to get the data. The EIA‑423 is a new
survey.

MODERATOR LU: Just started in January.

DR. KIRKENDALL: Maybe there are couple of months that are pretty close.
So this is very early in the process. This is one of those talks I was
talking about where this is so early that if you\'ve got good ideas, we
can try it. If you have good ideas, that is what we are looking for.

DR. SITTER: I was just curious. If you hadn\'t plotted volume versus
price, then I was really curious as to what you actually had.

DR. KIRKENDALL: That will be ones of the first things we do.

DR. SITTER: Precisely. And, really, what you are going to be looking for
is other characteristics that you have on all the surveys that you have
in common that you think might be related to price, will be looking for
that.

MR. HENGARTNER: I see quantity is just one of them.

VICE CHAIRPERSON BREIDT: In looking at surveys, there is not a lot in
common. I mean, the 423 and the ‑‑ there is not a lot on it.

MODERATOR LU: No.

VICE CHAIRPERSON BREIDT: It\'s kind of location and pricing quantity and
quality. That\'s it.

MODERATOR LU: That\'s it, yes.

MR. HENGARTNER: Now, I do believe that you are going to find a nice
relationship between price and volume because a lot of the data that you
are missing is cogeneration. I can predict that if the price of natural
gas rises, it\'s just going to switch.

MODERATOR LU: Oh, okay.

MR. HENGARTNER: So now we have is this an explanatory or a response?
That is going to be one of the difficulties we have here because we are
talking about cogeneration.

So I just want to forewarn. While I like the idea myself, I always like
my ideas, I want to be critical, then, as well to avoid pitfalls that we
will see.

So the spot price or whatever, maybe the spot price is what you need to
look at. That is another covariate. And that is known. You know the spot
price. That might give ourselves an indication in price.

DR. SITTER: What\'s the spot price?

MR. HENGARTNER: What\'s currently available at the markets in Chicago.

DR. SITTER: Clearly, that\'s going to be a strong indicator.

MR. HENGARTNER: Clearly, it will be, right.

CHAIRPERSON CRAWFORD: Do you have enough to summarize?

MODERATOR LU: The most interesting thing is price.

DR. KIRKENDALL: And then the other one was looking to see if you can
develop a predictive relationship between cogeneration and non in that
MECS.

MODERATOR LU: Yes.

DR. KIRKENDALL: In general looking to see what you can do with MECS to
try to understand the difference between cogenerators and
non‑cogenerators.

(Whereupon, at 12:11 p.m., the foregoing session was adjourned.)

CHAIRPERSON CRAWFORD: We\'ll get going now to let the break‑out session
leaders summarize the discussion. And we will start off with Doug
Hale\'s first.

[SUMMARY OF MANAGING RISK IN ENERGY MARKETS]{.underline}

MR. HALE: The summary of the discussion is really simple. I
filibustered. So there was very little discussion. The reason I
filibustered is that earlier this year I spoke to the committee about
this project, which is to write a report for the Secretary on
derivatives, how they\'re used in the energy industries.

The Secretary wanted a primer that laid out exactly what these things
are and how they could be used to manage price risk. Then he also wanted
something to talk about how they\'re actually being used in oil, gas,
and electricity industries. And, gee whiz, it would be nice to talk
about whether we should encourage these things or discourage these
things and, if so, what specifically should we be doing.

In looking at that charge, it started to look like a huge tome. We were
able to get it down to a fairly short tome of about 100 pages, 70 or 80
pages of text. And I basically walked the committee through what we did.

The report is on its way to the printer. We believe it will be available
in mid November. We are certainly hopeful we covered everything the
Secretary wanted in a way that doesn\'t put all of the readers to sleep
instantly. We think we have made it accessible and interesting, but only
the market will tell.

So that was pretty much it.

CHAIRPERSON CRAWFORD: Okay. Thank you.

Ruey‑Pyng?

[SUMMARY OF USING DATA FROM COMBINED HEAT AND POWER]{.underline}

[PLANTS TO ESTIMATE NATURAL GAS INDUSTRIAL PRICES]{.underline}

MR. LU: I very much thank you, the committee, giving advice first to
look at price and volume of the companies, either the electric company
or other industrial sector companies, to see the break of the price and
the volumes. And if there\'s a relationship, it could be the clustering
could help us to get to the better idea of how to estimate a price.

The other thing is to look in the MECS, the cogenerators and the
generators relationship if that will help us get a better idea of how
this can relate to the price.

And the third one will be look over the variables data series together
to see if there are any characteristics we can use or modify one of them
to add survey questions, which will then take care of the monthly
industrial price because there are overlaps, but we don\'t know how much
overlaps. But if we can create one, then that will help us to get a
better idea. My work will be to look into what is missing in all the
different pieces of the data source.

Thank you very much. And hopefully later I can report the progress.

CHAIRPERSON CRAWFORD: Does anyone on the committee have any more
questions or comments? Does anybody from EIA have any questions or
comments?

MR. LU: Oh, by the way, if committee members come up with any bright
ideas later on, please e‑mail them to me.

CHAIRPERSON CRAWFORD: Any questions or comments from the public?

(No response.)

CHAIRPERSON CRAWFORD: Finally, Linda Minor says that lunch is ready
downstairs in room 1E‑226. So we can go down there whenever you are
ready.

(Whereupon, at 12:16 p.m., the foregoing matter was recessed for lunch,
to reconvene at 2:40 p.m. the same day.)

AMERICAN STATISTICAL ASSOCIATION

COMMITTEE ON ENERGY STATISTICS

FALL MEETING

BREAK‑OUT SESSION:

STATE LEVEL COAL FORECASTING

THURSDAY,

OCTOBER 24, 2002

The meeting was held at 8:30 in Room 8E‑089 of the Department of Energy,
1000 Independence Avenue, S.W., Washington, D.C., Frederick L. Freme,
presiding.

P‑R‑O‑C‑E‑E‑D‑I‑N‑G‑S

(1:39 p.m.)

MR. BONSKOWSKI: Hi, everybody. My name is Rich Bonskowski. I lived in
Atlanta for a while, didn\'t really care for it.

Anyway, I am presenting today for Bill Watson and Fred Freme. Fred is at
the back of the room there. Bill is hiding in the Caribbean.

I am not the statistician who developed this. That would be Bill Watson.
I was more involved and Fred also has been more involved over the years
in the Legacy method. So we are going to ask for your indulgence today
in handling your questions. If we can\'t address them today, we are
going to ask that you can perhaps get you and Bill in touch next week
when he gets back.

I am just going to review briefly the issues that were in the paper,
presented in the paper, and which led to Bill developing a new weekly
coal production model. And, of course, at the end, we will request your
review and recommendations.

At the end of the presentation, I will also restate the questions that
were included in the paper report. But in the meantime, to summarize,
these are the basic issues.

We were aware over several years that there was a recurring need to
reduce the forecast error and weekly coal production. The Legacy method
was dependent on a number of parameters, which had become increasingly
unavailable or unusable over several years.

And about that same time that we were considering all of these issues, I
discovered some new sources of what we call ground truth data for
Western U.S. coal, which are basically railroad car loadings and unit
train loadings.

So what resulted was this new statistical forecast using both the AAR,
Association of American Railroads, rail data, heating degree days and
cooling degree days with state‑level time trends for the real quarterly
production data, which is both an input and a correction factor.

There were test runs done using the Legacy method and the new method.
And they got 29 percent improvement. And we believe that there could be
50 percent improvement using this method with a bit more work.

It\'s narrower on the screen than it is on here. This indicates the
trend over 12 years of the data. The dark blue line with black on there
is the Mine Safety and Health Administration actual coal production data
based on quarterly surveys at the mine level.

The pink was the weekly coal production estimates prior to any
correction as we made them. Then you can get an idea how they were
tracking. In the yellow, it shows the new EIA estimates based on the new
model when that kicked in. Now ‑‑

MR. HENGARTNER: Can you backtrack and get the yellow for the whole time
series?

MR. BONSKOWSKI: No. I mean, I guess you could, but Bill ran the test
over for eight quarters. There was nothing that was done that I haven\'t
shown here.

This does show that from the earliest date there in 1990 through 1994,
the correlations between the two methods were pretty good. In 1995 and
1996, where there were some causes for concern, we thought perhaps there
were as things started to vary or perhaps it was just an aberration.

Do I have a pointer here? Okay. In these areas here and up through here,
one of the things you notice is that there is an additive effect which
kicks in, which was the effects of our trying to make corrections, which
often the corrections were somewhat out of phase because we were based
on data that was about two quarters old.

We were trying to correct when that was drifting away from when our data
had been drifting away from the actual data. And obviously some of those
were out of phase and actually probably didn\'t help it, then, for the
coming quarter.

Basically what you are looking at are 10 million short ton errors more
or less, 10 million short tons over 260 to 270 million short tons, which
totals 3 to 4 percent for the weekly coal production estimates.

To some people, that might seem acceptable, but it\'s not really because
the percentages were unevenly affected for different key states. Some of
the states for which we were performing badly were key states like
Kentucky, West Virginia, Pennsylvania, and Virginia during that period.

MR. HENGARTNER: So you have a forecast for every state.

MR. BONSKOWSKI: Yes.

MR. HENGARTNER: They\'re probably much more variable than what you have
here.

MR. BONSKOWSKI: Yes. They\'re in the back of this paper.

DR. BURTON: The two hundred million you mentioned is per quarter, right?

MR. BONSKOWSKI: Yes. Does that cover the same period? I think it does.

MR. HENGARTNER: It\'s \'92 to ‑‑

MR. BONSKOWSKI: Yes. It covers the same. The other thing is that around
this time investor interests started to grow in these weekly coal
production estimates. Investors in coal fires at coal‑fired power plants
were looking for better ways to analyze risks. So our data was one of
the things they were looking at.

I also, then, want to mention that in 1997 through \'99, we had a pretty
good quiet period. I think at that point some of the corrections we had
been making had been tempered back. We didn\'t react quite so severely.

And also we had several to some extent management personnel who weren\'t
really involved in running the model week to week who were looking over
our shoulders saying, \"Change it. Fix this.\" So sometimes those may
have helped.

But then in 2000‑2001, the errors increased dramatically and some in the
order of 20 million short tons over a quarter.

Here in Wyoming, you can see a more dramatic portrayal of what happened
between the fourth quarter of 2000 and 2001 first quarter. We had about
an eight million short ton difference a couple of times or about an
eight percent error in what is our largest coal‑producing state and one
that should be relatively easy to model because about 96 percent of the
coal was loaded on trains. Obviously some of our train data wasn\'t too
good.

To summarize the Legacy method, it takes tons per carload. Tons per
carload is carrier‑specific. We also have a tons per carload national
average, which could be used to calculate a U.S. total. The
carrier‑specific data were used in the calculations from the bottom up.

We then took a look at in the center here right below that railroad the
railroad, ship tonnage, compared the railroad, ship tonnage with the
total tonnage of coal produced or shipped.

Now, in all cases, what we are really doing each week and each quarter
is looking at starting with data on shipments of coal and trying to
develop from that an estimate of production of coal. They\'re not always
exactly the same.

MR. HENGARTNER: That is within a week. I would expect a shipment related
to the production of last week instead of ‑‑

MR. BONSKOWSKI: Or any ‑‑

MR. HENGARTNER: Is the last time between shipment and production that
close?

MR. BONSKOWSKI: It\'s, yes, pretty close in general, but ‑‑

DR. BURTON: They try not to put the coal on the ground because it is
expensive to pick it up. If you can avoid putting it on the ground, you
save costs. So you try to load directly to ‑‑ it depends on if they have
to prep it or not or if they\'re shipping it just as it\'s mined.

MR. HENGARTNER: Because that would impact your estimation in here,
wouldn\'t it?

MR. BONSKOWSKI: The other thing is that each week we are trying to
estimate production. The cases where shipments may not equate to
production are generally in place for more than a week. But we don\'t
always know about them at that time.

A good example was around January through March of this year, when we
were tracking AAR car‑loading, which we were taking as new production.
As a matter of fact, those shipments were in that case coming out of the
stockpiles because the utilities and coal‑burning power generators had
to stop taking coal.

Because of the economy and the burns being down, they had actually cut
back on as much as they could accept. Their stockpiles were full.

So there was a delay of two or three months with warrant orders and
other types of contracts that are in place that they had to go ahead and
finish producing coal, but that coal was, in fact, going on the ground.
And so that caused a misalignment of what we were saying was actual
production.

What we are striving for, of course, is production but without enough
knowledge to make some very well‑informed adjustments for differences in
stocks. Without that, we would show it as production. We show it as
production, but it really is shipments. It can often be shipments.

I mentioned these were cars. Here are the railroads shipped versus the
total shipped. And we have various ways to try to estimate what that
portion is.

The other item is, too, what we do in the old system is accumulate the
carriers\' loadings into individual state files because there may be six
or eight railroads that load in one state, especially eight years ago or
so. As time has gone by, that is no longer the case.

The state production share is here, our best estimate of a state\'s rail
ton shipments as compared with that state\'s share of real production.

The objectives of the Legacy system, first of all, when these were good,
they were very, very good. But the objectives tie the forecast to ground
truth to try to capture the time and seasonal trends and to tie the
forecast to data that at one time were easily assembled and were
available closely in time to the forecast period.

There were three big changes that caused most of the problems. One, I
think if you go back in hindsight with the loss of the Interstate
Commerce Commission, the Surface Transportation Board data for tons and
cars, for shipping coal, for the tons and the cars, which is how you get
tons per carload, had been an annual, became an annual, instead of a
quarterly document, it became voluntary or at least it wasn\'t
well‑enforced. So some railroads decided to send it in, and some
didn\'t. And so it was incomplete.

The old Federal Railroad Administration waybill data handed off the
responsibility rather quickly. That was handed off to the Association of
American Railroads became more proprietary, more costly, if you would.
You had to purchase it. And at various times, it was also more utility
than it had been in the past.

Finally, the railroad deregulation itself meant that there was frequent
reshuffling of ownership and trackage rights, which meant that the
states of origin that these different railroads were loading coal in was
shifting.

As they were consolidating, they were really compounding the problem
because if we had six railroads loading in a certain area and five of
those six railroads were down, we could pretty well track which state or
states were causing the problem or had been lower.

As these railroads consolidated into four major carriers, you can\'t do
that anymore. They\'re all loading in about ten different states or
more.

Also, personnel turnover during these meant that just about the time you
had something set up to get data from somebody, they would move on to
another position and your arrangements were gone.

This is the new weekly coal estimation method, AAR coal car loadings.
Its coal car loadings are still used. We use that to get the overall
national level weekly coal production totals. Then the statistical
forecast is used along with that and heating degree days, time and trend
data. There are two different methods at the bottom here.

The weekly coal production estimated on the left side from
state‑specific rail car loadings. That would be the data primarily in
the West that is available in the West through the Union Pacific and
Burlington Northern‑Santa Fe railroads. That\'s two railroads.

And the other on the right is estimating using state shares from
statistical forecasts layered onto the AAR and the total production
levels. That\'s where the CSX and Norfolk Southern railroads and the old
Conrail system operate. So we don\'t have a good real‑time data
arrangement with them at this point. Also, some of the Western states,
most of the Western states, besides Wyoming, are still being forecast
with that method.

DR. BURTON: Now, the state‑share forecast, is that purely a time series
forecast based on a trend? I can\'t remember. I read the paper, and I
can\'t remember. Are there other parameters that go into that forecast?
How are those states ‑‑

MR. BONSKOWSKI: The time series I think is based strictly on past
history of what the real results were.

DR. BURTON: Of shares?

MR. BONSKOWSKI: Yes.

DR. BURTON: Okay.

MR. BONSKOWSKI: The ideas behind this new method on the state‑specific
side where rail car loadings were available, especially this right now
applies to Wyoming, we use those weekly car loadings from the BNSF and
the Union Pacific and we addressed those sources of uncertainty. You
still need to have a factor to convert the car loadings or trains to
tons of coal.

Now, in those two railroads, we have got that very nicely. They supply
one on a weekly basis, and Union Pacific supplies it on a monthly basis.

A factor to account for the production being shipped by other transport
modes, that is up to us to determine. Sometimes we can find it from
examining reports the mines might have on their Web sites. Railroads
sometimes might have given us some data on that.

We\'re talking about how do you account for things like mine mouth power
plants where the mine produces the coal but never hits a railroad. It\'s
shipped over by truck or conveyor. You\'ve got to get a handle on how
much of that there is.

When it says \"EIA‑6A\" there at the bottom, that is one way of doing
it. You can possibly back into that by knowing what the total production
was, either through that or the MSHA data, what the total production is,
and comparing it with those loadings.

None of these have turned out to be foolproof, however. We are trying to
decide. One suggestion I make right now is that another source of
uncertainty may be with these other states besides Wyoming, especially
that there is a changing ratio. So the shipment is by different modes.

And it might be due to a bunch of short‑term contracts going into place,
spot market sales, loss of exports that Utah used to ship to LAX and so
on. A lot of different little things are happening. Those markets are
not really stable.

DR. BURTON: You can\'t pick a harder market to try and work in. There
are so many variables: less metallurgical coal going over Norfolk to
Europe, British Columbia coal coming east instead of going to Asia.
These are things. It\'s an extraordinarily difficult place to work.

I sympathize. I have to do it. So I sympathize.

MR. HENGARTNER: Does the type of coal matter?

DR. BURTON: Yes. Whether it\'s got high Btu or low Btu or high sulfur,
low sulfur, low moisture, yes, it matters a lot.

MR. HENGARTNER: So that should be included, then, because we know where
the production is done, we know the type of coal.

DR. BURTON: The idea that you have in the paper of dividing this into
regions I think may be one of the best short‑term things that you can do
because essentially ‑‑ this isn\'t completely true, but basically
high‑sulfur coal is mostly Eastern coal, low‑sulfur coal is mostly
Western. That\'s not quite 100 percent true, but it would help a lot.

Plus, you have Western coal that goes into different markets than
Eastern coal and a lot of things. Sometimes they compete. Very often
they are going into different markets. So they are affected by different
market forces. Something will happen that will, let\'s say, increase the
demand for Powder River Basin coal but won\'t affect the demand for West
Virginia coal at all.

So I think that to the extent that you can divide this into regions,
that that alone will get ‑‑ you have basically two Eastern railroads and
two Western railroads. So that, again, if you have a way to divide it,
that would hopefully prove your ability. It\'s tough.

MR. HENGARTNER: What are the uses of coal besides heating?

DR. BURTON: Making steel, making steel.

MR. HENGARTNER: So degree days wouldn\'t influence that type of coal.
I\'m just thinking what you have in your model, days, degrees is good
one if it\'s only used for heating, but if it\'s used for steel‑making,
as you say.

DR. BURTON: And to the extent that it is being exported.

MR. BONSKOWSKI: And those are two things that have really fallen off
dramatically, both the coal that is going to steel‑making and because we
import so much coke now, it is really going to making coke, most of it.
And that is being imported as coke now more.

DR. BURTON: A lot of Chinese coke.

MR. BONSKOWSKI: Yes. And the exports are I guess at an all‑time low,
historical low, or something.

DR. BURTON: Over Norfolk, they are at about 50 percent of what they
would have been five years ago. So they are about 13, 14 million tons a
year; whereas, they would have been at 25‑26.

MR. BONSKOWSKI: The other thing, the thing is we are always trying to
especially just analyze the origins of the coal. I mean, coal production
is really the origins of coal. The supply came from where?

So the transportation factors that go into that should tell you that on
the surface, rather than the face, but we don\'t have all the
transportation factors. So that\'s why we sometimes have to look at
downstream as to where it might be going in order to back‑calculate.

DR. BURTON: I loved reading this paper yesterday because this hits
exactly where I live in terms of the stuff that I do. Use of rail cars
is a great idea.

If there is a way to capture the amount that is moving by the other
modes, rather than just sort of doing the proportional thing, then that
would be extraordinarily helpful. I don\'t know.

I don\'t know how much is being barged. And I don\'t know how you would
ever know weekly, how much was loaded to be barged last week. I don\'t
know that anybody collects that information.

Again, in the same way that you use the rail cars, if you knew the
number of loaded barges, you know the amount because it\'s going to be
1,500 tons per barge. And that varies hardly at all. But I don\'t know
where you would get that data from. I don\'t know that anybody ‑‑

MR. BONSKOWSKI: I thought that it was only available monthly or
quarterly from the union, the Corps of Engineers.

DR. BURTON: Yes.

MR. BONSKOWSKI: But I have seen something on their Web site that shows
some weekly statistics.

DR. BURTON: But they are doing the same thing to come up with those
numbers you are. They call them forecast numbers because there is about
six months\' lag in their collection and processing of data.

MR. BONSKOWSKI: What I think is probably not forecast is lock data, the
traffic through locks.

DR. BURTON: The LPMS stuff is available to give as a lag because they
have to collect it and process it. And so the LPMS lock data is
available. You couldn\'t go through and know what was locked through
last week.

I spent a good amount of time yesterday trying to think of where in the
world could you come up with that data. I\'ll ask some other people. I
know a lot of people who know stuff I don\'t. So I\'ll ask.

CHAIRPERSON CRAWFORD: Could you tell me? Maybe this is a question for
Bill, but I am a little confused about how all the statistical models
work together because I see like there is a regression model here
obviously. There is also a forecasting component. But I don\'t know how
it uses the results from the regression. Does that make sense?

MR. BONSKOWSKI: The results from the regression really are to produce
the state‑share multiplier.

CHAIRPERSON CRAWFORD: Okay.

MR. BONSKOWSKI: So the regression looks at those. It looks primarily at
the past production by state history and seasonality and produces a
state‑share multiplier, ‑‑

CHAIRPERSON CRAWFORD: Okay. So it produces the state‑share and then ‑‑

MR. BONSKOWSKI: ‑‑ multiplier, all of which should add up to 1.0, the
whole group for all the different states. The heating degree days and
cooling degree days, by the way, too, are ‑‑ I guess he is using them
pretty much as real‑time indicators. They are slightly leading
indicators.

CHAIRPERSON CRAWFORD: So, then, this model then produces the state‑share
multiplier. And then where does that multiplier go?

MR. BONSKOWSKI: Except for Wyoming, which we are building from the
bottom up, the multipliers are then being applied to the U.S. national
production total that is produced from the AAR car‑loading data. The AAR
car‑loading data is weekly.

CHAIRPERSON CRAWFORD: Okay. So then ‑‑

MR. BONSKOWSKI: And it is real. But it doesn\'t tell you where it came
from. It just says, \"This railroad loaded this many thousands of
carloads of coal.\"

CHAIRPERSON CRAWFORD: So then you use the regression to get this
multiplier. Then you hit a different data source with that multiplier.
And then you run that through an auto‑regressive time series to actually
make the forecasts?

MR. BONSKOWSKI: Does that sound right?

MODERATOR FREME: No, not quite. The way it works is you have the two
models that he has been talking about. The first model, which I guess
he\'s calling it, the one on page 9 about the rail cars that you need
degree days and cooling degree days, the auto‑regressive model is the
one that comes up with the total estimate of production for the U.S. And
it comes up with a total number. Then the other ‑‑

CHAIRPERSON CRAWFORD: So that\'s not done by state or ‑‑

MODERATOR FREME: No, it\'s not. That\'s just total U.S. That\'s
production for the entire week for everybody. Then the state shares
stuff he was talking about, like you said, except for the Wyoming
because we can get Wyoming almost directly from the UP and BNSF data,
the state shares are then applied to that total to come up with the
state estimate of the state production so that they all ‑‑

CHAIRPERSON CRAWFORD: Okay. And then ‑‑

MR. BONSKOWSKI: And I want to say that historically and still in this
system, the U.S. national total is the estimate that we have been the
most successful with. Allocating it among the states has been more
difficult, but he has been able to find a six percent possible increase
in aggrecacy on the U.S. national. It isn\'t a whole lot of change
because it was already pretty good.

CHAIRPERSON CRAWFORD: And so what is it that you forecast, then?

MR. BONSKOWSKI: The forecast is really the estimate. We are making an
estimate of what was produced last week, the forecast in that we don\'t
really know. We are using these other indicators. And we won\'t know
until the end of the quarter.

CHAIRPERSON CRAWFORD: See, that\'s what I am not getting, how you are
using these other indicators in the forecast. That is the part I am
missing.

DR. BURTON: Let me see if I understand it. You are using the
auto‑regressive equation to estimate the national total.

MODERATOR FREME: Right.

DR. BURTON: And then you are using the estimated share equations to
allocate that total among states.

MODERATOR FREME: Right.

CHAIRPERSON CRAWFORD: Right.

DR. BURTON: Does that ‑‑

MODERATOR FREME: That\'s correct. I don\'t know if you\'re hung up on
the word \"forecast\" because I am not sure I particularly like using
the word \"forecast\" in this because it\'s ‑‑

CHAIRPERSON CRAWFORD: My problem is, unfortunately, I looked at the SAS
code. So now I really know what is going on. So that\'s where I decided
I didn\'t understand how things were going because I saw some things
being done with regression.

I saw some things being done with a time series regression. And then I
saw other things being done that ignored all of those variables and just
made a forecast based upon the series history.

So that\'s why I just couldn\'t understand how these components are all
coming together.

DR. BURTON: You looked at the SAS code? You need a hobby. Thank you for
making it available, though. That was very ‑‑

CHAIRPERSON CRAWFORD: I mean, that told me everything I needed to know.
I\'m sorry.

MODERATOR FREME: Well, I think Bill would be much better able.

CHAIRPERSON CRAWFORD: Okay.

MODERATOR FREME: I\'ll have him get in contact with you because he is
the one who wrote the SAS code on this and did that.

DR. BURTON: And I should know the answer to this before I ask, and I am
embarrassed to say that I don\'t, but I think it is worth asking. How
good and how timely are the data on spot prices?

My sense is that they\'re fairly immediately available. Is that, spot
prices for coal?

MODERATOR FREME: For electric utilities?

DR. BURTON: Yes, for steam coal.

MR. BONSKOWSKI: Yes. For purchases of spot coal, they are. They are
available weekly.

DR. BURTON: Have you tried? I mean, I am an economist. I have this crazy
idea that somehow output and prices are correlated, that production will
go up at the same time prices go up.

MR. HENGARTNER: Not necessarily. I think it\'s the relative price of
coal and other sources of energy.

DR. BURTON: That\'s true in the long run, but you can\'t switch. You
can\'t switch in most cases. In most cases, you can\'t switch between
fuels very easily or even switch your dispatching.

I think I would bet that spot prices and production are positively
correlated. I don\'t know if it would be enough to help the estimations.

CHAIRPERSON CRAWFORD: I think I got it now. I understand. So, then, one
part does regression, but it\'s the state‑shares that are done by a pure
forecasting model. Okay.

MR. BONSKOWSKI: So whether that should be called a forecast or an
estimate ‑‑

CHAIRPERSON CRAWFORD: That\'s okay. I mean, that\'s fine.

MR. BONSKOWSKI: But it\'s an important point because of the uses of the
data, to which the data are being put. But I do know and I was going to
say here that Bill Watson did numerous different tests or iterations,
regression iterations, using the 1999‑2000 data. We don\'t have all the
results that he got, but we finally settled on the ones that were
working the best, gave best results.

As it says here, the forecasts are of state‑shares in quarterly coal
production. The quarterly data is from the real data, from MSHA. It\'s a
one‑quarter lag.

And the advantage here for this model is the statistical model captures
the long‑term time trends, the seasonal relationships. And it takes
individual analysts and opinions out of the equation.

DR. BURTON: Can I ask another question that has nothing to do with the
estimation or the statistics? Who is using these estimates? What do they
use them for, the weekly?

MR. BONSKOWSKI: They\'re published. They are picked up in a number of
different places. The National Coal Mining Association, most of the
weekly Coal Week and a number of electric power generation magazines and
newsletters use them and a lot of the traders on Wall Street who trade
in coals commodity are either publishing them or using them.

DR. BURTON: I\'m sure if the other Mark was here, he would jump all over
me for even asking that question because he is real anxious to have
very, very current data. I wasn\'t sure for what purposes.

DR. BERNSTEIN: This Mark is going to ‑‑

DR. BURTON: Oh, you are here. I didn\'t ‑‑

DR. BERNSTEIN: No, no, no. I was going to ask the same question. Why do
we need weekly coal data? I mean, in a way I can understand somewhat
more the weekly gas data because of the markets, but I am not sure.

Then when I look at I assume MSHA is the ‑‑ they do their own estimate.

MR. BONSKOWSKI: Yes. It\'s ‑‑

DR. BERNSTEIN: But you have a graph here that has the data.

MR. BONSKOWSKI: It\'s at the mine level. So it\'s actual production
collected at the mine level quarterly.

DR. BERNSTEIN: On a quarterly basis, which is very close to your point
on the estimates. But they don\'t do it on a weekly basis.

DR. BURTON: No.

DR. BERNSTEIN: Is it needed? I guess is the question.

MR. BONSKOWSKI: A lot of people have asked that.

MS. O\'BRIEN: I\'m Betsy O\'Brien. And whenever we have been a day late
in putting it out, we get flooded with calls is one indication that
people are interested, not just to reproduce it in their publications,
but especially now with a more vibrant spot market for coal and a
commodities market, they find this data really important combined with
stock data for investors. They are some of the biggest people that
complained if we were late.

And we looked at actually changing it. We put some feelers out about a
year ago because we were having trouble with the accuracy to see if it
was critical to people. And we got enough responses back saying that
they wanted it that we felt we should continue.

DR. BURTON: I wasn\'t asking because I knew the answer and wanted to
hear it. I really didn\'t know. I don\'t know how I didn\'t see you
sitting there.

DR. BERNSTEIN: I came in late, and I was slouching.

MR. BONSKOWSKI: He was slumping down.

DR. BERNSTEIN: I guess the other question is, what influence does it
have? Are you comfortable enough with the influence that it does
potentially have on prices? When you said you made the calls? And
what\'s the correlation between what you put out and where prices go?
And that\'s the issue, what it assumes as a relationship.

MR. HENGARTNER: I would even go one step further. Do we need to know
this production at the state level? For market purposes, doesn\'t just
national‑level data suffice?

DR. BURTON: No.

MR. HENGARTNER: No?

MODERATOR FREME: Because it\'s a variation of the ‑‑

DR. BURTON: The characteristics are different, and the transportation
practices are different. And so it really is. It would be nice. It would
be really nice if that was ‑‑

MS. O\'BRIEN: Another thing, too, about 90 percent of this coal is going
in the electric market. So that\'s partly why the heating and cooling
degree days were good forecasters.

People are interested in state because there are different prices by
state and they\'re facing transportation issues. I think it did. I\'m
not sure if the production or production and stock data. It did have
some influence, I think, when we saw coal prices going up a year ago.

The other thing is even though the other data that is actually collected
is collected on a quarterly basis, but it\'s not available to the public
until almost five to six months after the actual time of production. So
there is not much information for traders on a real‑time basis without
this.

DR. BURTON: When oil prices spiked, they carried the spot market price
for coal along with them so that some mines were getting a mine mouth
price of up to \$40‑\$45 a ton, where it had been 21 or 22. Hell, they
were mining coal with a teaspoon at \$40 a ton. So I just can\'t imagine
that using spot market prices wouldn\'t add something to the faucet.

My question is if they are available. Mark?

DR. BERNSTEIN: I haven\'t looked at the coal side.

MR. BONSKOWSKI: Well, spot market prices are being averaged and
published weekly by Platts and by ‑‑ what\'s the other one we use? ‑‑
Coal Daily, Argus.

The two of them are I think the two leaders. Also Hill\'s, Hill and
Associates, has an index. These indexes are calculated slightly
different ways. So it would be worth exploring which are best.

We are publishing one coal type, the prices for one coal type, from each
region. By agreement, Platts is allowing us to publish that, but there
are actually quite a few more. There are different Btu levels and sulfur
levels of coal within each region. So that would probably be the levels
you would want to look at.

They use at least two or three trading companies or more for each region
index.

DR. BURTON: If you have the same time series available for the spa
prices, I would definitely explore that in your national equation. If it
works, call me. If it doesn\'t ‑‑

MR. BONSKOWSKI: You didn\'t say it.

DR. BURTON: That\'s right. Oh, wait. She\'s recording it. I can\'t get
out of it that easily.

MR. BONSKOWSKI: Okay. I\'m going to finish off on this slide by noting
that there are three different ‑‑ get to the right page. Excuse me.

There are three different levels of estimates that we have been talking
about, the national estimate, which Bill found accounts for about eight
percent of the reduction in the average forecast error. The
state‑by‑state estimates have improved by 21 percent. And we expected
and are still hoping for further improvement in the Western states or
primarily in the Western states which could add another 20 percent.

This is a graphic to show during the test period how the current ‑‑
excuse me ‑‑ I say \"current\" method there. That was the Legacy method
there and how that tracked with the new method, resulting in a 29
percent reduction in the forecast error for that test period. From there
on, we are going, of course, with using one method, not using the old
Legacy method now.

That brings us to the area where the question ‑‑ I am going to just
rephrase those questions, reiterate those questions that were asked in
the paper. And some of those I think we have already started to address.

What additional process information or data, if any, are needed to
complete your assessment? I just want to go through these and make sure
that if you have further things to add that we could at least get them
done now and possibly even get back in touch in the future.

So are there any comments on this one, any more comments on this one?

VICE CHAIRPERSON BREIDT: I have a question about the share forecast. It
looks like you are forecasting shares state by state separately by
fitting separate regression, auto‑regressive, models, share on back
shares.

MR. BONSKOWSKI: Are you Roy?

VICE CHAIRPERSON BREIDT: I\'m Jay.

MR. BONSKOWSKI: Jay? Okay.

VICE CHAIRPERSON BREIDT: And then buying those forecasted shares
directly from national coal production data. Is there any step that
takes those shares and assure that they sum to one because you fitted
these separate models? Do you see what I am saying?

MODERATOR FREME: The short answer is yes.

VICE CHAIRPERSON BREIDT: Okay.

MR. BONSKOWSKI: Yes. The shares ‑‑

CHAIRPERSON CRAWFORD: How is that done? Because ‑‑

MODERATOR FREME: I think the way it is done because I am not all
completely familiar with the new method is that once the national total
is arrived, we have got a number, 22 million, whatever. The Wyoming
total is gotten by using the specific data from the UP and the SP.
Basically subtract them, and then you have got what is left over.

The state shares are then applied, and it\'s normalized to that left
over. That is part of the price. So you can never add up to more than
what the total national was.

MR. BONSKOWSKI: And, again, that is because we seem to get the best
results on estimating that national total. That\'s all consistently been
our highest success rate.

MR. HENGARTNER: I want to follow up on Jay\'s question because I didn\'t
realize that. When you make the state model, you don\'t consider
simultaneously all the states together. Each state is done separately.

MODERATOR FREME: Correct, each state is separate.

MR. HENGARTNER: Because one can easily imagine a special ‑‑ I don\'t
want to throw out names, but essentially just modeling the darned thing
altogether.

CHAIRPERSON CRAWFORD: Or at least a multivariate time series.

MR. HENGARTNER: Exactly.

DR. BURTON: It is because it is a spatial equilibrium. The production
amounts at various states is all part of a giant spatial equilibrium
that involves production of coal across states as well as prices of
other fuels.

MR. HENGARTNER: So essentially you should model it jointly, not each one
alone. Each state is not independent of every other state. The reason
why you want to do that is because the variation in the estimated
coefficient might be smaller.

DR. BURTON: Yes, there is going to be the correlation.

MR. HENGARTNER: Yes, there is a correlation. That\'s what accounted for
it.

CHAIRPERSON CRAWFORD: That\'s an important point.

MR. HENGARTNER: And if you get that, then you are going to observe that
the errors are ‑‑ you have a larger error now than you do at the state
level.

The states are more variable than national level, correct? They have to
be. What you are going to observe is that if you model them jointly at a
spatial temporal, essentially trying to have the correlations between
those processes, then I would predict that you will get more precise
observations.

MODERATOR FREME: Okay. If I remember correctly, when I am discussing
this with Bill, ‑‑ I really wish he were here ‑‑ I think he has looked
at some of this because he was talking about correlations. So I will let
him know about this. And I am sure he can get back to you or get back to
everybody and let them know what it was that he has done on that.

CHAIRPERSON CRAWFORD: I think the correlations will be like I would
assume they would end up being where you could even assume it to be like
region‑specific because, you know, probably what is going on in West
Virginia doesn\'t influence very much what is going on in Arizona or
whatever.

MR. HENGARTNER: It could be related by exactly the coal types. The
states that are like Wyoming that have one coal type are going to be
have similarly. And the ones like Virginia is going to be ‑‑

DR. BURTON: That\'s right. West Virginia, Virginia, Kentucky will all be
the same or at least ‑‑

MR. HENGARTNER: At some point one can even ask ourselves do we want to
just aggregate at the region level or do we need to go to the state
level again?

DR. BURTON: Well, you guys already have a lot of stuff by coal region,
not coal regions, right? And those would be a very natural level of
aggregation. Unfortunately, they are not completely state‑specific, but
they are largely.

MR. BONSKOWSKI: Well, currently it\'s the weekly coal production tables
are showing east and west of the Mississippi and the Appalachian, the
Illinois Basin in the West, Western coal fields. The Illinois Basin
interior includes ‑‑ actually, the interior includes the Illinois Basin
and Texas and some of those states just west of the Mississippi.

But the regional aggregation that makes more sense and should be added
to this, not to take anything else away, though, would be the central
Appalachian coal, the northern Appalachian coal because we are then
looking at coal‑producing regions which produce coal of interest to the
markets, the Illinois Basin coal.

DR. BURTON: The only problem I see, you have got two states where you
have got both southern and northern Appalachian coal in West Virginia.
And you\'ve got Illinois Basin and Appalachian coal in Kentucky, right?
Western Kentucky coal is the Illinois Basin coal? So matching things up
on a state‑by‑state basis would be a problem for those two states. But
other than that ‑‑

MR. HENGARTNER: Again, the question is unless we need it at that level,
maybe the basin‑level data is good enough.

MR. BONSKOWSKI: Well, I think the interest from anybody who is dealing
with the markets would be great at those levels. I mean, they are
interested in the central or the northern Appalachian level, they are
looking for certain characteristics.

DR. BURTON: Then you\'re down to 4 or 5 equations, instead of 29.

MR. HENGARTNER: Yes, a lot.

MR. BONSKOWSKI: But at this point, we are doing it by building it up
from state data. I mean, that is the only way we could do it. We
haven\'t started doing that yet, but that is one of the changes we
wanted to make.

MR. HENGARTNER: Now, those regions, the only impediment to doing that by
region is such that it\'s a railroad system. There are two railroad
systems in the same region. Is that correct? That would be the problem?

MR. BONSKOWSKI: Yes. Well, there are. There are two railroads sometimes
in one state. But the solution to that would be if we could get good,
reliable real‑time data from the railroads each week.

From the other railroads, we are now just getting their AAR data, which
just tells us the grand total. We could get it by state. We hope to be
able to do that. That would help a lot. I don\'t think they would give
it to us by region.

DR. BURTON: Does the MSHA data have the production region as well as the
state?

MODERATOR FREME: No.

MR. BONSKOWSKI: What do you mean?

MODERATOR FREME: It\'s at a mine level.

DR. BURTON: But I mean, could I look at West Virginia production and
divide it into northern Appalachian and southern Appalachian?

MODERATOR FREME: Yes. It\'s at a county level. So you know which county
this is in.

DR. BURTON: That\'s right. When you get it, is it fully disaggregated?

MODERATOR FREME: When we get it, we get the mine‑level data, individual
mine.

MR. BONSKOWSKI: You can have it, too. Anybody can have it.

MODERATOR FREME: It\'s on their Web sites.

CHAIRPERSON CRAWFORD: Maybe we should go into the ‑‑

MR. BONSKOWSKI: Well, I\'ll move on.

CHAIRPERSON CRAWFORD: We partially asked this. And Question 3 is ‑‑ I\'m
just going to put them together because you\'ve only got like ten
minutes left ‑‑ \"What problems does the committee see with these
equations?\" I think those two questions are tied together.

MR. BONSKOWSKI: Question 3 shows the equations also.

CHAIRPERSON CRAWFORD: And I guess my thing is I am concerned with the
method of fitting Equation 1. So what they use is just a regular,
ordinary regression with lagged variables in the model like he does
within proc reg. And I think there is a better way to fit that model.

MR. BONSKOWSKI: Are we going to specify what that is?

CHAIRPERSON CRAWFORD: It concerns me that you are putting in variables
that are probably highly correlated with each other in the regression
model. And I think you are not supposed to do that. Not only that, proc
reg is not going to let you deal with any auto‑correlated errors that
might be left over if you haven\'t completely specified the large‑scale
component.

DR. BURTON: You need to use the VAR.

CHAIRPERSON CRAWFORD: So I just wonder why he wasn\'t using some more
sophisticated time series regression model. I don\'t know. What do they
call those in SAS? Is that where they call them transfer functions in
SAS?

DR. BURTON: I\'ve never done any of that in SAS. I\'ve only gone ‑‑

VICE CHAIRPERSON BREIDT: Isn\'t this the auto‑reg step, though, the
fitting Equation 1?

MR. HENGARTNER: Yes. If that\'s the way we end up doing it, that\'s one
way. I don\'t have problems with that. You look at those issues and be
sure that it auto‑correlates.

CHAIRPERSON CRAWFORD: I wasn\'t sure, then, because you\'ve got a proc
reg statement in here right below it. So I am not sure. I mean, maybe he
is just trying to determine who is important.

VICE CHAIRPERSON BREIDT: Oh, I see. I think one might be kind of more a
deterministic seasonal specification versus an auto‑regressive
specification for the core. So I think it\'s just two competing model
specifications. And I don\'t know which one we are actually choosing.

CHAIRPERSON CRAWFORD: Maybe we should tell them we prefer the auto‑reg
to the reg or ‑‑ I don\'t know what he is doing with reg, but it
concerns me this model.

VICE CHAIRPERSON BREIDT: I think you end up with Model 1.

CHAIRPERSON CRAWFORD: What?

VICE CHAIRPERSON BREIDT: I think he does chose Model 1.

MR. BONSKOWSKI: So, Jay, you\'re agreeing that both equations are being
used, right, the one leading into the other, feeding into the other?

VICE CHAIRPERSON BREIDT: Well, I think that for Equation 1, it\'s two
different model specifications, one fitted with proc auto‑reg and the
other fitted with proc reg. I think that in the text, I\'m not sure, but
I think he decides on Model 1, the auto‑reg version, which I think most
of us would seem like that\'s a pretty reasonable model specification
and a reasonable way to fit it.

CHAIRPERSON CRAWFORD: See, I guess that was something that I just
couldn\'t determine from the text. So that would be an open question
because the codes and the text, which output went with what.

If you follow his order, then you\'re right because the order in the SAS
code, if they went that way, then that\'s right. And that would explain
why Model 2 didn\'t give him the results that he thought he should get.

VICE CHAIRPERSON BREIDT: The middle of page 11 says, \"The stronger
structural basis of Model 1 makes the model of choice for estimating
national coal production.\" So I think that\'s ‑‑

MR. BONSKOWSKI: Jay, are you Jay Breidt?

VICE CHAIRPERSON BREIDT: Yes.

MR. BONSKOWSKI: Okay. I just want to make sure we get to contact
everyone. Oh, here it is. Okay.

DR. BERNSTEIN: It appears we are wrapping up at some level. You really
do need to fix this forecast work. This goes back to consistency across
the agency.

When you are estimating weekly production, you are not forecasting
future production. I think that is really important for consistency
across the agency. But it also kept confusing me as I was reading it
expecting to see a forecast. I think that is important both in the
quality control area as well as making sure everybody understands what
it is you are doing and what it is you are not doing.

DR. BURTON: And to go back to something that you said earlier, if you do
have opportunities to fix this through either doing the states or doing
regions simultaneously and perhaps including other explanatory
variables, then it certainly would be worthwhile to do it. If people are
using this to make real decisions on it, you can give them better
information, you will do them a real service.

CHAIRPERSON CRAWFORD: Do you think if they did that, maybe they
wouldn\'t have to do Wyoming separately or is it all ‑‑

DR. BURTON: I wouldn\'t do Wyoming. I would do them all the same.

CHAIRPERSON CRAWFORD: Right. Okay.

MR. HENGARTNER: On the other hand, if you get the data for Wyoming, why
‑‑

DR. BURTON: That\'s true. Why are you less than ‑‑

MR. HENGARTNER: I mean, if you have the option of doing a better job
with Wyoming because you have the information, that would ‑‑

DR. BERNSTEIN: I don\'t know if I missed this, but have you tried
running this pretending you didn\'t have the Wyoming data and do an
estimate on Wyoming and then figure out how close that is to what
actually you get from Wyoming?

MR. BONSKOWSKI: Not since that two‑year period when he did the testing,
2001 data.

DR. BERNSTEIN: Yes, but what you always did is subtract Wyoming off and
then did your state share. Instead, if you pretend you don\'t have the
Wyoming data, create a new model that has state‑share, including sharing
it out to Wyoming based on the primary set, and see how close you are to
the actual.

I mean, you do have the ability to somewhat see how robust the model is
because you do have a deeper point of data that is real. You can pretend
you don\'t have it and estimate it and then compare it to the actual and
see how good you are doing because if you\'re doing okay, fine. But you
find out there are some inconsistencies that may help you do better on
the other states as well.

MODERATOR FREME: We may have already done it.

DR. BERNSTEIN: That would be important to me.

MODERATOR FREME: Yes. I\'ll let you know definitely.

DR. HAMMITT: But the better comparison is to the MSHA data.

DR. BERNSTEIN: Which is on a quarterly basis.

MR. BONSKOWSKI: Our time is the majority of the time, it is operating.

CHAIRPERSON CRAWFORD: No. You actually have one minute.

(Laughter.)

MR. BONSKOWSKI: Let me just present these other three questions because
I did a good job on laying them out here. Question 5, basically I just
want to point out that there are these short‑term changes in the market.

And, of course, the model doesn\'t really pick those up. And I am sure
that the people who are wondering about what is affecting the market
this week versus next week and versus last week would like to see that
reflected. So if you have any suggestions as we contact you, I hope it
will be all right to contact you next week. That would be something we
are looking at.

And, as I say here, some of the examples might be indicators such as
rainfall or snowfall data, rivers data, large traffic, rail advisories
the railroad has put out.

Finally, that is the local mine capacity utilization percentages. These
are only available on an annual basis. We collect that data at the mine
level ourselves, capacity utilization. Perhaps that would affect
regionally.

DR. BURTON: You\'re right. All of those things can affect output at a
specific location at a specific point in time, but I don\'t know any way
statistically.

I think what you do have to do is when you present your estimates, you
have to say, \"These are estimates which do not take into account these
things\" so that the users know that if there is a mine holiday in West
Virginia, that that is not going to be reflected in your estimates of
what last week\'s production was.

But as far as treating it statistically, those are just anomalies.

DR. BERNSTEIN: Well, the question is, how important is it?

MR. BONSKOWSKI: Yes.

DR. BERNSTEIN: If the users say it is really important to try to get a
handle on the chance or the probability or the risk of it being
fluctuated, then you might try to do some analysis to figure out what
precursors tend to lead to those fluctuations.

An analogy is weather forecasting. Tomorrow\'s weather is in the
forecasting realm. It\'s basically today\'s weather with a little bit of
change. It does not capture extremes. And when we did actually a project
for a utility who needed to know when those extremes would likely occur
because they\'ve got to get their plans on line and stuff like that.
There are precursors that could give one a sense of the probability that
this extreme is going to happen. But they really need it, and there was
an important issue there.

You need to determine in the answer to that question is it really
important to know and do users really need to know that. Will it really
make a difference in the market? If not, then don\'t worry about it. If
yes, then come back and try to figure out how to do it.

MR. BONSKOWSKI: It may not be important to know. People want to know it,
but it may not be important because four weeks later, that flip is gone.

DR. BERNSTEIN: Right, exactly.

MR. BONSKOWSKI: You know, it didn\'t really make any difference,
although there was some anxiety at the time.

DR. BERNSTEIN: But if something happens and it is important, say,
because you get a cold front in the Northeast and there is not enough
coal for the power plants or something, maybe there is a reason to do
it, but you need to kind of assess that.

VICE CHAIRPERSON BREIDT: This context is not really forecasting. You
actually know that these events have occurred, right? You knew that this
‑‑

MR. BONSKOWSKI: You know that, yes.

VICE CHAIRPERSON BREIDT: So you are really not trying to determine the
probability that the ‑‑

MR. BONSKOWSKI: You know there was a flood. You know that there were
some rail lines washed out, but you can\'t really model it. You don\'t
know what effect it\'s going to have.

VICE CHAIRPERSON BREIDT: There are intervention models which can take
into account these kinds of anomalous or outlying occurrences and let
them die out in different ways depending on how you think they actually
affect coal production.

MR. BONSKOWSKI: Okay.

MR. HENGARTNER: Just one last question. I think that all of this is on
the premise that there is an overcapacity in railroad cars, that when
you produce something, it is always going to be able to be shipped.
There is never over‑production, and there is no accumulation. Is that
correct?

I am just trying to understand what the basic premise is.

DR. BURTON: That is partially correct. You can cycle cars faster when
you need to. It\'s just expensive. But there is an absolute capacity
limit. So you are making the assumption that there is sufficient excess
capacity to handle any variation. That is not 100 percent true.

CHAIRPERSON CRAWFORD: It\'s not just now.

MR. BONSKOWSKI: Yes. I think that it\'s ‑‑

MR. HENGARTNER: I just want to be sure that we are all aware of that big
assumption because if that assumption fails, then whatever we are
talking about right now is not true.

DR. HAMMITT: I just want to ask something that is sort of related to
this question about why weekly. It seems like it would be easier to
estimate shipments.

Would that be sufficient? Given that that is a pretty close proxy for
production, why not just make that your total to report shipments?

MR. BONSKOWSKI: Yes. I agree. There may be some advantages to
promulgating the fact and emphasizing the fact to the consumers of this
data that these are shipments. More or at least some of the time we are
modeling shipments when we think we are modeling production. So we are
really looking at the shipments.

The other part, which I put here in Question 7, is that where there is a
lot of interest, it\'s in the stockpiles, which is the other part of
that, the other side of that.

Coal is in short supply. It\'s moving. The cars are loaded to capacity
and there is no excess capacity available. Are they acting so that you
can bring more on line?

But when there is an excess of stockpiles, then they\'re building up
either end. And that is something that people would like to know, too.
We may contact you about that next week.

(Whereupon, at 2:46 p.m., the foregoing matter was adjourned.)

A‑F‑T‑E‑R‑N‑O‑O‑N S‑E‑S‑S‑I‑O‑N

(2:46 p.m.)

CHAIRPERSON CRAWFORD: My schedule says, Fred, you were going to
summarize for our group. Is that right or is that changed? Rich?

MR. FREME: No.

CHAIRPERSON CRAWFORD: Go ahead, then.

[SUMMARIES OF ADVICE FROM THE BREAK‑OUT SESSIONS]{.underline}

MR. FREME: A very brief summary since we are running out of time and
several people did come in for a lot of the discussion. We were going
over the estimation of coal production by week, by state that we do.

And some of the comments received, which were very good, relate to
dividing the country into regions and using the regions, instead of
starting out at the national level, using regional level.

Other things to investigate would possibly be using spot prices in the
equation since that is available on a weekly basis. They might help
improve the estimation procedure.

And there was a question as to whether actually weekly coal production
is really needed. And that is there are both sides of the coin there.
And people seem to believe, at least the people I have talked to, the
analysts who have called, seemed to really want that. But that might be
something else that we can think of.

There is just a lot of variability within this system and trying best to
work on that.

CHAIRPERSON CRAWFORD: And for the next group, Preston McDowney.

MR. McDOWNEY: Our break‑out session was dealing with how to estimate
monthly CHP backing power plants, independent power producer data, from
an annual series with three years of monthly data to support it. So we
wanted to estimate over a 10‑year or 12‑year time period with this 3
years at the end of monthly data.

The overall consensus of the group was that that is probably not a good
idea because there would be no way to validate the estimates and might
be misleading because we would be artificially putting a seasonal
component to data without any way of validating it.

With the IPPs, however, it was considered to be a good idea that we
might be able to estimate those with the electric utility data being
that most of them were reclassified electric utilities to begin with.

Some of the comments that were made were that if we were to do it, we
might want to talk to some of the CHP companies themselves and see how
they felt the data would vary over time and with the seasons.

So that\'s about the summary of what we talked about. Did I miss
anything important?

CHAIRPERSON CRAWFORD: Okay. Thank you.

Are there any questions or comments from the committee on either
session? Would you step to the microphone and give the transcriber your
name?

[COMMITTEE COMMENTS]{.underline}

MR. COSTELLO: Yes. Dave Costello, EIA.

This question relates to this issue of estimating or potentially
considering estimating monthly data from some of this annual electric
power data.

My question is I gather the conclusion was it\'s probably not worth
doing without a lot more information. If that is wrong, let me know.

My question would be in a case, for example, that we have a lot of
annual data on things like coal, consumption for generation in plants
that were formerly called non‑utilities but were a mixture of industrial
plants that were burning just, for example, coal or natural gas, either
one, in any case we have a lot of annual data from, say, 1989 up until
1998. We have monthly data after that but just annual data through \'98.

There are some real apparent trends, obviously, in some of that data.
And so if you needed the monthly data, which I happen to need that
information, wouldn\'t it be better to at least try something with
regards to fitting a smooth curve?

We know from year to year it is growing rapidly. Wouldn\'t it be better
to have some kind of trend to fit to or smooth, then whatever
information we could incorporate into it, rather than just, say,
dividing by 12 or something?

Clearly it seems to me that although it\'s not perfect information or
anything, it has got to be better than just doing something simply like
dividing by 12. So that\'s my question.

Some of the smaller things, of course, we don\'t care much about because
they are very small anyway and they don\'t matter, but coal and natural
gas do matter. So, anyway, that is my question.

DR. SITTER: Before you leave the mike, could I ask what you would then
need the monthly data for? You say you need it. That was the question I
asked in the break‑out. What do you need it for?

MR. COSTELLO: Okay. Well, the model that we are using to determine, fuel
splits, for example, for electric power, is monthly. We would like to be
able to enhance the database with this new information as best we can.
So we would like to have the monthly values. We realize that we really
don\'t have ‑‑

DR. SITTER: No, but to do what? To predict the future?

MR. COSTELLO: Right.

DR. KIRKENDALL: This is for the short‑term energy outlook.

MR. COSTELLO: It\'s not just a matter of predicting the future. Yes, I
mean, we have to ‑‑ some of this month‑to‑month information and seasonal
information won\'t be all that useful because it won\'t be real, but, on
the other hand, we also want to show historical patterns of this.

We are going to be showing some aggregates going back a long way. We
would like to do that anyway. We certainly would prefer to show a
picture that was more realistic than just, say, dividing by 12 or not
showing anything at all. I think that is the point.

DR. SITTER: If you\'re talking about showing patterns based on data and
that is what you are doing, you have yearly data and you have monthly
data. If you then use the monthly data to predict the monthly data for
the yearly data and then you put it in another model to build up a big
model, it is just a more complicated model.

If you are using it to backfit something, then it\'s backfitting a
certain amount of information. The danger is if you put it there and now
you treat it like real data, your variabilities are all wrong. If you
are using it to predict the future, then it\'s just self‑validation as
the future includes the data you created it with.

If it\'s just so that the data is in that format, then there is nothing
to stop you from doing that as a data analyst. That\'s not the same as
putting it up on a Web page as it were data and saying, \"We estimated
this best we could.\"

DR. BURTON: Because if somebody thinks that it\'s stochastic and goes
and fits a model for it using that data, they\'re going to get something
that could be very wrong.

DR. SITTER: And look really good.

DR. BURTON: Yes. And the fit is going to be great. That\'s right.

DR. SITTER: That\'s the problem.

MR. COSTELLO: That\'s fine.

DR. SITTER: I think as a modeling exercise, if you\'ve got this kind of
data, all you have to do is just create the data and then put it in the
model and realize that you\'re doing that and you\'ve got these two
levels of stochastic in it.

MR. COSTELLO: That\'s fine.

DR. WHITMORE: I think we did talk about the trend process.

MR. COSTELLO: My interest anyway is just to make sure that whatever we
do, we do it once. And we all have some numbers that we can agree on as
being reasonably good, but absolutely.

I would 100 percent agree that the issue of the variability of the model
as you are going into predictions to the extent it includes this other
information is not correct. We wouldn\'t want people to do that.

DR. SITTER: I understand what you are saying.

MR. COSTELLO: Yes, right.

DR. SITTER: And I think we are saying the same thing.

MR. COSTELLO: Okay.

DR. SITTER: I think they are in two components to what you said because
you said you don\'t want to just divide the number by 12. That would be
essentially fitting a continent. By putting in the trend, then if they
would, they would be increasing in this case.

MR. COSTELLO: Right.

DR. SITTER: So that\'s part of it. But the issue we were really
discussing was, should we make it vary month to month as a seasonal
model frame data that was only over the past three years fitting back?

MR. COSTELLO: Right.

DR. SITTER: So there were sort of two levels of data.

DR. WHITMORE: And the conclusion there was the answer was no?

DR. KIRKENDALL: There were two conclusions, actually. The conclusion was
that it\'s fine for the IPPs. For that section, you can do something
sensible because you\'ve got the history of the utilities.

Some of the utilities moved. They used to be utilities anyhow and you
have monthly data for them. Now they\'re called IPPs. So there\'s stuff
you can do there. And the group thought that made some sense.

It was for the CHPs that they weren\'t convinced that three years of
information should be used to develop seasonal factors that would be
used in the past.

They thought it would be better to just estimate a trend. It was okay to
estimate a trend, but they didn\'t want to do seasonal factors. They
thought that would just be ‑‑

MR. COSTELLO: Okay.

DR. KIRKENDALL: And if you look at some of the data, there is probably
good reason not to do that also because it doesn\'t show strong seasonal
patterns for the two years.

MR. COSTELLO: Okay. Thanks,

CHAIRPERSON CRAWFORD: Are there any other questions or comments? We have
plenty of time.

(No response.)

CHAIRPERSON CRAWFORD: Before we break, I guess would the new members of
the audience identify themselves at the microphone, please, if you
haven\'t already done so?

MS. WAUGH: Shauna Waugh, statistics and methods.

MS. SEIFERLEIN: Kitty Seiferlein.

MR. RATTIDGE: Charlie Rattidge, EIA.

MR. VETTER: John Vetter, SMG.

MS. PAULL: Mary Paull, CNEAF.

CHAIRPERSON CRAWFORD: Okay. Well, I guess we\'re on break, then, until
3:25.

(Whereupon, the foregoing matter went off the record at 2:58 p.m. and
went back on the record at 3:27 p.m.)

CHAIRPERSON CRAWFORD: Our next speakers are Bob Schnapp and Renee
Miller. They will be speaking about estimating and presenting power
sector fuel use in EIA publications and analyses.

MR. SCHNAPP: Thank you very much.

[ESTIMATING AND PRESENTING POWER SECTOR FUEL USE]{.underline}

[IN EIA PUBLICATIONS AND ANALYSES]{.underline}

MR. SCHNAPP: Before I start, I wanted to make sure that everybody had a
copy of our EIA electric industry data collection schema. This was six
months ago. When the committee met, they had suggested we develop
something like this. We have. I wanted to make sure everybody had it.

Roger Sacquety is the person that was mostly responsible for it, but
Bill Weinig helped him a lot in final design, Nancy Kirkendall and Bob
Rutchik. I just want to make sure everybody has that. So talking about
‑‑

MR. HENGARTNER: Is he going to win a prize today?

MR. SCHNAPP: We will definitely be putting him in for an on‑the‑spot
award, but don\'t tell him.

Moving, then, to our presentation that Renee and I put together, we want
to talk to you about the work we have done to edit and change our
non‑utility data.

Previously what I had told you about that what we wanted to do was to
display our non‑utility data in a more consistent manner, consistent
being consistent between each of the publications and forecasts, and
also to improve the accuracy of our data.

The reasons for that were that there has been a lot of change in the
industry. Utilities have been spinning off their generating assets into
separate companies or selling them off to independent power producers.
And there has been a growth of generation in the commercial and the
industrial sector. And we wanted to take a look at that a little more
closely.

So, as a result of that, the change in the industry, we saw that the
categories that we had were becoming less meaningful. So Renee will talk
a little bit about the utility/non‑utility split and how we have kind of
begun to move away from that. Also, there was an inconsistency in how we
presented the information, as I was explaining in the end‑user sectors.

What we have done ‑‑ again, I am just really recapping what we had
talked about a number of months ago ‑‑ is that this has been an EIA‑wide
effort. Every part of EIA has been involved in this.

So it\'s the natural gas and the petroleum people, electric, coal, and
renewables have worked very closely together on this. We have also gone
to the integrated statistics and consumption surveys with the annual
energy review and then later with the monthly energy review.

The short‑term forecast and the integrated analysis with their long‑term
forecasts and the statistical methods group has been an incredible help
in helping us along our way.

So one of the challenges that we faced in our work was how to categorize
the data. And Renee will talk about that, which data sources to use. We
have pretty much come to decide to use the non‑utilities.

Historically our annual non‑utility form, which was the 860B. The moving
forward, it\'s our monthly/annual 906. For the utilities, it was the
759, monthly 759, which has now kind of migrated and has the same exact
form as the non‑utilities in the 906.

So we had a lot of work to do in cleaning the data. In particular was
the generational outlook best of all, the fuel consumption, and then the
useful thermal output.

Finally, the last item, which was we had to estimate how much of the
fuel that cogenerators use, cogeneration produced both electricity and
steam, and they couldn\'t tell us how much of the fuel was used to
produce electricity. So we had to come up with a methodology to split
that fuel into those two end uses. So we came up with that and
implemented it with all the data that we have.

So the last time we asked \"How do we alert our users that we\'re making
massive changes to our data?\"; Johnny Blair suggested that we put
together some documentation and put it on the internet. As he was saying
that, I looked around the room.

And every EIA person that was working on this started shaking their
head, \"Yes.\" In fact, it was the first order of business when we got
back together again. And we spent the past four or five months or six
months, whatever it is, since the last meeting working very hard on
this. I think at this point, we are probably up to version 40 or so of
that. Is that about right?

Just to show you really how hard we are working on it, it is very
important to us. And it will be available with the annual energy review
when that comes out.

Next one. So since the last meeting, we have cleaned the data, the 1989
through 2000 data. We\'re taking one last look at the 2001 data. We have
completed the documentation.

It really is just kind of funny because we were talking about
documentation the other day. We said, \"I don\'t know why we didn\'t
think of that.\" It was just so obvious that we should do that. So,
John, I really want to thank you again for that comment. It really has
helped us move this along a lot.

The annual energy review is in its final stages. On Tuesday, it will be
posted along with this documentation so that anybody can we hope
understand how it is that we have done changes. I think we have provided
copies of the documentation to you folks so that we could get some
feedback on that.

So, with that, let me turn it over to Renee, who will continue to
explain the recategorization and a comparison of how we change the data.

MS. MILLER: Okay. Thank you, Bob.

The annual energy review when it comes out will present data in new
categories. And also it will show data revisions. This is all described
in our documentation. I will talk to you about the recategorization
first.

As Bob pointed out, we realized that the utility/non‑utility split
wasn\'t that meaningful. And so we\'re showing data in terms of new
categories, electricity‑only and combined heat and power plants.

The electricity‑only plants, as their name implies, just produce
electricity. And they include the utilities and the independent power
producers. The combined heat and power plants produce both electricity
and thermal energy.

We used to call them cogenerators. And many of you may know them by that
name, but in our deliberations, we realized that they didn\'t fit the
strict legal definition of cogenerator, which is to produce heat and
power sequentially. Some of our entities, while they produce both heat
and power, they didn\'t do it sequentially. So we thought that combined
heat and power plants better describe them.

Also as part of our recategorization, we decided to break out data for
combined heat and power plants by the end‑use sector that they\'re
associated with.

So, for example, hospitals, for example, combined heat and power plants
that are associated with hospitals, would be shown in the commercial
sector. And we would also have a break‑out for combined heat and power
plants in that sector.

Combined heat and power plants associated with paper mills would be
shown in the industrial sector. And, similarly, there would be a
break‑out for them there. And if they were primarily in the business of
selling electricity, they would be shown in the electric power sector.

In addition, we have removed the independent power producers from the
industrial sector in situations where they were in there like for
natural gas.

I am going to show you examples, but first I just wanted to review the
players involved. And those of you who attended Preston\'s session, this
is going to be very familiar, but just to give over it again.

We have the electricity‑only plants, and we have the combined heat and
power plants that are commercial, industrial, and electric power. All of
these entities produce electricity. And some of them produce thermal
energy as well as electricity.

And then we have the electric power sector, which we define as
electricity‑only plants and the electric power CHPs. So you see that
there is not really a one‑to‑one correspondence between the electric
power sector and the entities that produce electricity.

This differs from the old days, when the electric utilities were the
electric power sector. This made things very simple for us because they
were the only ones who produced electricity. And they produced only
electricity. So we didn\'t have to worry about estimating consumption
for thermal.

Okay. Now, to see examples of what we did, I think all of the committee
members have Table 6.5 of the AER. You will have a sheet that looks like
this. So if you like to look at data, you could kind of follow along. We
are going to be looking at natural gas data as an example. And we are
going to start with the industrial sector.

So we are looking at the AER 2000 page. And we are looking at the column
\"Delivered to Industrial Facilities.\" That column includes data for
combined heat and power plants associated with manufacturers. It also
includes data for independent power producers.

And, to make things more interesting, even more interesting, the data
for independent power producers shown on this table we obtained by
asking suppliers, \"How much did you deliver to independent power
producers?\"; where on our electricity power tables, we obtained the
data by asking the independent power producers how much natural gas did
they consume. And, as you might imagine, they didn\'t necessarily report
the same thing. So we were showing two sets of numbers.

Moving on to how we are doing things now for AER 2001, this is on the
next page of your handout, still looking at Table 6.5. And we will start
with the industrial sector. You could see under this banner \"Other
Industrial.\"

We have the combined heat and power plants shown separately. So the
break‑out is different, but they were included in the industrial sector
previously.

The big difference is the independent power producers are no longer in
the industrial sector. So the industrial sector number for AER 2001,
those numbers are going to be a lot lower than the numbers for AER 2000.

So, as an example, the estimate for 2000 that we are showing in AER
2001, if you\'ll look under the heading \"Other Industrial, Total\" for
2000, you will see 8.25; whereas, on the previous table for the same
year, we showed 9.4. So this is quite a difference. And this has to do
with the removal of the independent power producers.

The independent power producers are now part of the electric power
sector, but it wasn\'t just a matter of moving them from one column to
another because we are now using data from the electric power surveys
for the electric power sector. So we\'re using data that we collected
directly from the independent power producers on how much natural gas
was consumed.

Other aspects of Table 6.5, you can see that for the commercial sector,
we also have a break for the combined heat and power plants.

Another difference between this new version of Table 6.5 and the
previous version, on the previous version, we had an adjustment column.
The reason we needed that column was because, well, AER 2000 was a
transitional year. And we were in the process of making our data
consistent, but we weren\'t quite there.

So we had the non‑utilities in both the industrial sector and the
electric power sector. So we had the adjustment column to take them out
so we wouldn\'t double count them. So for our new Table 6.5, we no
longer need the adjustment column because we have distinct sectors.

So this is all very good news. At least we think so. But if we want to
get the amount of fuel used to generate electricity ‑‑ and you may be
looking at Table 6.5 and you\'re looking, \"Well, where do I get that
from? Is that the electric power sector?\" Well, no. That is not on
Table 6.5 because the electric power sector, as I described, doesn\'t
have all of the entities that produce electricity.

We have the combined heat and power plants in both the industrial and
the commercial sectors. And also the electric power sector has the
combined heat and power plants that are associated with that sector that
are producing, in addition to electricity, thermal output. So if we want
to get the fuel use to generate electricity, we would get that from
Table 8.3A, which is the next table in your handout.

Now, you may wonder, well, is there a link between Table 6.5 and the
electric power tables? I mean, how does this all fit together? We are
pleased to report there is, as seen in the next table.

Table 8.3E shows consumption of fuel for electricity generation and
useful thermal output. We can look at this table and compare it with
Table 6.5 if we want to try this. And we can see that the numbers are
consistent for the electric power sector.

So if you all have Table 8.3E and we look at the top block, \"Electric
Power Sector\" and we look under \"Natural Gas,\" for 2001, you will see
5,261. That\'s in billion cubic feet. Then if you flip back to Table 6.5
and you look at the next to the last column, which is \"Electric Power
Sector,\" you will see 5.26. So we match.

So this is an improvement over what we had previously. We now have a
link between our fuel consumption tables and our electric power tables.
So I gave you examples of the recategorization, and we also talked about
the fact that we have revisions to the data.

Go back to the revisions. You could see that for some categories, the
data have not changed very much. For example, electricity generation,
those numbers are very close in AER 2001 compared with AER 2000.
Similarly, for coal consumption, we don\'t have much of a change. And we
don\'t have much of a change for petroleum consumption.

Where we have large changes for natural gas consumption, we have a three
percent increase over what we previously published. And this is a fairly
sizeable amount in terms of volume. And for renewable energy
consumption, we have a five percent decrease for the 2000 estimates over
what we published previously.

The reason for the change in the natural gas consumption has to do with
the use of different data sources that we are now using the data from
the electric power survey. The change for the renewable energy
consumption has to do with the fact that we are now assigning the fuel
type for consumption and generation purposes at the boiler level, rather
than at the more aggregate plant level.

So before, for instance, if a plant reported using both solar and
natural gas, then we assigned all of the generation and consumption to
renewables; whereas, now we\'re doing it separately at the boiler level.
So we show changes.

For that reason, we looked at the data for the other years. And it
pretty much showed the same pattern, this pretty good indication of the
level of the changes that we\'re showing from one AER to the other.

And we\'ll go on to some issues that came up during this process. We
always like issues. One of them that we would like to hear what you have
to say about is that there is going to be a lag between the release of
the AER and the other annual publications, such as the Electric Power
Annual and the Natural Gas Annual, and the monthlies.

What I mean by that is that the Natural Gas Annual and the Electric
Power Annual won\'t be out until towards the end of this year or maybe
early next year. They will be in the new format. The monthly
publications were going to continue coming out with monthly
publications, but they won\'t be in the new format until possibly April
of the following year.

So the question is, what can we do to help our users in the interim? We
talked about coming up with estimates. We pretty much decided that that
wasn\'t such a very good idea, that we wouldn\'t get very good quality
estimates and it really wouldn\'t be worth the time and it would
possibly mislead our users if we put them in our publication.

So is there anything else we can do? We have the annual data. We see
patterns from the annual data. Is that really the best we can do to tell
users who are interested to look at the annual patterns and let them
make their own judgments about the monthly data? We would be interested
in what you think there.

Some other issues that came up. Some of these will sound familiar. One
has to do with end‑use sector definitions. I\'ve spoken to you about
this, one aspect of this, topic before. That was that the pipelines were
having trouble classifying the data by end‑use sector since it wasn\'t
an important part of their business. But we also noticed from the other
end that facilities that had classified themselves as industrial in one
year have changed their classifications so that they were now in the
electric power sector. And this was because they changed ownership.

So this raised the question, \"Well, should the classification be by
ownership or by how the fuel was used?\" And we used a shorthand \"who
owns, how used.\"

To be consistent with our other surveys, like the manufacturing energy
consumption survey, we thought it should be by the line of business of
the respondents, which would probably correspond to ownership. And we
figured, well, the end user knows what line of business they\'re in.

The problem is that the supplier doesn\'t always. So this leads to the
next issue in resolving differences reported by fuel suppliers and
consumers. So we are talking about adding a question to the electric
power surveys, asking them who delivered the fuel. That will help us
match the electric power data to save the natural gas data to get better
correspondence. I am bringing these issues up because they may be future
topics.

Getting back to the documentation, we have that prepared as Appendix H
to the AER. And when the AER is posted, the documentation will be on the
website with it. The documentation was a group effort. And we welcome
the committee\'s comments on it, and we are interested in any other
comments you may have.

Thank you.

CHAIRPERSON CRAWFORD: Thank you.

Our first ASA discussant is Roy Whitmore.

DR. WHITMORE: I have prepared some transparencies looking at the
documents and the questions. The document that was on the Web site had
some embedded questions, three embedded questions. The first one had to
do with reviewing the documentation that had been prepared and some
specific questions about how accurate that was.

I want to first commend EIA for putting together that documentation on
the changes. I think it could use a little bit of work to make it a
little clearer.

Probably I had more difficulty with it than a lot of the users would
because I am not as familiar with the various energy sectors that we are
talking about here and how things have always been done traditionally
because I basically work in this area about twice a year, when I come to
these committee meetings. And so it\'s always having to refresh my mind
as to how things work. So it was a good review other than that sense of
being someone who has to be educated from the very beginning and
doesn\'t come into this kind of knowing already what needed to be done.

So basically I looked at the three questions that were in the document,
tried to answer those. The first had to do with reviewing sections 1, 2,
and 3 of the documentation and providing comments on whether or not that
was accurately explained, the changes that had been made.

What I didn\'t find there and eventually piece together and even wasn\'t
sure when I put this down if I had gotten it right, unfortunately, from
looking at it today in conjunction with everything else it is going to
present, it looks like we\'ve got it almost right.

But it seemed like it would be helpful to have just an overview of what
the basic categorizations are that are being used. There is the first
break‑out by the industrial sector. \"Industrial\" sector isn\'t the
right word but, anyway, the sector, commercial, industrial, and electric
power, and that is based on the NAIC codes. That wasn\'t very clearly
presented. That was kind of the initial break‑out.

And then, as I read it in the document, ‑‑ I may have misinterpreted ‑‑
I read it as saying within each sector, the electricity generation would
be further broken down into electricity‑only plants and the CHP plants.
That is not exactly right because there aren\'t any electricity‑only
plants within commercial and industrial.

That only appears within electric power. I think some of the other
things that I have seen today, both your current slide and some of the
other stuff I saw earlier, made that clear.

But we need it clearer. I think it\'s not hard to make one clear,
succinct definition of what the categories are that are going to be used
for reporting purposes and how they\'re defined. So that needs to be
kind of up front.

And then we can very quickly say how that differs from what has been
done in the past, where the categorization of the electric generation
was in terms of utility and non‑utility producers. So I am sure that is
in there, but it just wasn\'t as clean and crisp and easy to assimilate
as it should have been.

There was some discussion of use of revised historical data in the
documentation. In particular, it said, \"EIA has revised its historical
data on electric power to resolve anomalies, both regarding EIA\'s
electricity series and its fuel use series.\" It talked about how they
were being ‑‑ well, it didn\'t really talk about the publications.

And the tables that you just presented were very helpful in terms of
seeing how it was. I didn\'t have it before in terms of how it would be
presented under the old and the new categorizations.

I think to some extent, the users will need some tables, at least for an
initial year, where things are presented in both ways. People that are
looking at trends will need to know what the changes they\'re seeing,
are they due to different categorizations or are there real changes in
the data themselves. So it might be useful for certain things. I think I
said publications or maybe tables to be presented in both ways.

Some questions that occurred to me when I looked at this that you might
want to answer in the documentation are, will all relevant data series
be revised to accommodate the new structure? It sounded like from what I
read, that was true, but it seemed like that might be an awful lot of
data. I am not sure I got that right.

Something about quantifying the uncertainty of the data revision it
seems like would be helpful. It was more like \"This is what we did.\"
And I didn\'t get any sense there of how much QC had been done or how
confident you were of the historical data being revised and how well
that could be done.

There was some discussion of at least a few publications continuing both
the old and the new data series because some users wanted the old and
the new data series. I would imagine you are not planning to do that
indefinitely. You might want to say something in there. Does it say
anything about for how long?

So there are some publications that would be continued in both the old
and the new format for a while and some things that might just need
selected tables done both ways for at least one transition year.

In that same section, they\'re talking about use of revised historical
data. The documentation also said an examination of heat rates, capacity
factors, power‑to‑steam ratios across 12 years of reported data were
conducted as a result for non‑utility power producers for 1989 through
2000 have been revised. Again, some characterization of the quality of
those revisions with the confidence that you have and the quality of
those revisions would be helpful.

The documentation also discussed allocation of CHP fuel use, trying to
determine what portion of the fuel used was to produce electricity
versus to produce heat.

The documentation spells out the process that was used, which relies
pretty heavily on an assumed boiler, steam boiler, efficiency rate of 80
percent. And I had no idea.

They\'re all like in the range of 79 to 81. And so assuming they\'re all
80 is great or if they run from 50 to 95 and assuming they\'re all 80,
it\'s really a pretty gross assumption. It might be helpful to say
something about, again, the reliability of that estimation process.

The other question that was brought up in the document is this one
about, well, we can put out the annual series now with the new
categories, but it will be a while before we can incorporate the new
categories into the monthly data series. Do we need to worry about the
fact that it will have an annual publication with the new categories?

And it will be a while before the monthlies catch up. I don\'t know your
data users that well, but it just appeared to me that may not be a huge
problem as long as the data series that goes with each monthly
publication does exist.

If an annual publication uses the new categories, then the data series
for that year ought to be available with those same categories. But if
there is a little bit of a time lag involved in making it available to
the public, I would think folks should understand the need for that.

It didn\'t sound to me like that is a major problem, but that would
really depend on who your users are and how they are using the data.
Others might be able to speak better to that.

The only way around that I could think of is to possibly ‑‑ well, I
guess there was another point here I thought of. If anybody is trying to
track the old data series, having both data series released one year
would be helpful. I think you\'re doing that in most cases.

The final was the final question that Renee just mentioned,
classification of facilities by the owner or how the fuel is being used.
It seemed like if you are trying to classify another fuel being used, it
could be hard to get that accurate. It may have been on who is filling
out the question for an individual facility. It might change from one
year to the next as they concentrate more on one thing in one year and
something else in another year.

It seemed entirely reasonable to me that you can accurately determine
the NAICS classification of the owner of the plant. That clearly puts
the facility unambiguously in one category. And that seemed to be a
reasonable way to do it to me. There won\'t be any measurement error
problems in terms of where you\'re putting the facility. And everybody
will understand what has been done.

Unless there is some great need on the part of the users to classify it
based on how the fuel is being used, I think using the NAIC categories
is a good idea.

So that was it.

CHAIRPERSON CRAWFORD: Thank you.

Our next ASA discussant is Johnny Blair.

MR. BLAIR: First, I am just going to make a few comments on the
documentation on a much more general level, though I certainly agree
with the comments for the most part that Roy made.

You made a suggestion and it has pretty much been accepted as kind of
sort of nowhere to go from there as far as what is peak. And so the
perspective is a little bit different as sort of kind of where you are
now and what are the things that you are going to be doing and what do
you want to do that is forward with this, seeing that this is a living
document in the sense that as users are exposed to it, you are going to
get more comments, probably not as good and as detailed as those of Roy,
but certainly you\'re going to get comments and want to respond to
those.

I think that you have got a very good start given a difficult task of
basically revising how you do a lot of things and trying to convey that
in one place and in this same fashion.

I think a good job has been done. Particularly I thought that the
documentation was strong in the places describing the reasons for the
changes, the rationale for why things are being done differently now
than they had been in the past.

So sort of what do you do now? One is going to be I think responding to
questions that you have and making minor revisions. One point along that
line that I think fits in with Roy\'s comments about perhaps needing
parallel tables for some period of time is I think that there could be
more use of graphics within the documentation as simple graphics, the
kind of before/after sorts of tables that would indicate sort of this is
how the classifications are reporting or the rules or whatever were set
up before, here is how they are done now, and keeping in mind that you
have different levels of readers for any kind of document that you
produce, both in terms of knowledge, some that are young and don\'t know
very much about it and some that are very knowledgeable but also people
that will approach it with wanting to skim and get to certain key points
and others that really want to understand in very fine‑grade detail.

I think that adding fairly simple kinds of tables that are kind of
before/after would help both of those types of readers without it being
a terribly difficult thing to do.

This is going to be, as you mentioned, posted on the Web. I\'m not
entirely clear, but my understanding is that it will be useful also
maintaining a text version of this as well with printed reports.

So one area that you want to give some thought to in planning is over
time how do you want to maintain the parallelism between these two
versions? It\'s natural as what you have done now, at least as I
understand it, starting with a text version and then essentially putting
that up on the Web.

And having that all in one place is important, but obviously also having
links from the particular points in the documentation to the relevant
substantive reports is going to be obviously something that you would
want to do.

In the future, I think that you want to think about do you want to
continue this sort of development from text and then posting on the Web
or really revise the Web document and use that as a way to generate the
text document, rather than the other way around is natural to start or
in trying to do something parallel, as I mentioned in the part of one of
the handouts that once this is up on the Web, of course, you can still
make further changes.

That immediately kind of raises the specter of, well, you have these two
documents now. You have this paper document and this Web document. And
you want to keep them obviously consistent, though they\'ll be different
simply by the nature of a Web document versus a text document and the
kinds of information they convey.

But factually in certain ways, you want them to be consistent. And I
think by having the Web document as the core in generating the text
document to support however you find in the future, then you want to
keep both of those. It is something that is probably the way I would
look at this but something that you want to think about.

The only thing I want to mention is that, again, it was obvious from the
comments that Roy made and other comments I expect that you will get
today that you need to have an assessment of you produced this product
now, which I think is a good one and certainly a very good one for a
start.

And you want to give people a chance to comment on it. You want to do
some assessment of it. And I think there are a couple of ways that you
can do this.

One ‑‑ and you may have already started this ‑‑ is with some small
diverse group of users, to have them do an assessment of sort of what
you have now in terms of an overall evaluation of whether it meets their
needs, in terms of coverage.

Is everything there that they need? Of course, clarity, you know, once
it\'s up as a Web form and there are links to other parts, again, this
is something you can get a lot of mileage out of on not a lot of work
and not a lot of cost to do that kind of a small‑scale assessment.

Lastly, I would think about a kind of ongoing process for opportunity
for users to be able to comment on the website of having whether it\'s
something formal in terms of some little mini questionnaire, which I
would probably prefer because it\'s easy to summarize. It makes sense.
And you get lots of open‑ended comments. You may want to allow for those
as well.

To keep in mind, at least, maybe not forever but maybe certainly because
this is a transition and it won\'t be hopefully forever, but hopefully
for the near future, that you\'re going to be making adjustments and
trying to make it as useable and as effective as possible.

The best way to do that I think is to set up a mechanism where users can
easily without a lot of effort provide feedback on it. Of course, that
is easier to say without actually looking at this stuff that is
providing in deciding what, if anything, to do with it.

Providing that opportunity is also the public relations and that when
people see something new, they like the chance to be able to submit
feedback, \"This is great, but we need this\" or \"Gee, why do you do
this this way.\" And if you can do that simply, which you certainly can
on the Web, I would recommend doing that.

But, again, I think that the key work on this has been done and now
it\'s a matter of adjustments and making it useful over whatever period
of time that you\'re going to maintain it on the Web.

CHAIRPERSON CRAWFORD: Bob, Renee, or Preston, do you have any comments
or questions?

[QUESTIONS FROM THE COMMITTEE]{.underline}

MR. SCHNAPP: Well, I will just address one thing, which is the
suggestion that we put together some graphics and include it in there.
Actually, over the last 36 hours, we have been putting together some for
a briefing that we were doing for the seventh floor here. And so we have
a whole set with three pages of graphics at this point that we can
probably pick and choose from.

There is one graphic for petroleum, one for coal, one for renewables,
non‑utility generation, total generations. I mean, we have a whole set
of things here that we can pick and choose from. A suggestion probably
is pretty good that this will help them understand the extent of the
change.

We put it into a little table up there, but I think it is a little
better graphically. There are a few years. They can kind of see the
pattern.

I am not sure about the others. We\'ll have to kind of assess all the
other comments and then with the larger group to see how to proceed and
kind of impact on resources and publications.

DR. WHITMORE: I would like to second the suggestion of having before and
after tables or at least table shells. Even just one set I think would
be helpful.

MS. MILLER: You\'re talking about shells but not necessarily data?

DR. WHITMORE: You wouldn\'t necessarily have to have the data. You might
want to show the data, but I don\'t think it would be necessary. You are
trying to communicate what the change in format is.

MS. MILLER: Okay. That certainly would be a lot easier.

CHAIRPERSON CRAWFORD: Mark?

DR. BERNSTEIN: Two things. I think you have to put a warning label on
this. A lot of people just don\'t read the whole document. They go right
to the table and start throwing numbers out. And even before they kind
of look down here, there should be these numbers are different than
2000. And then it makes people know, you know, yes, it would be kind of
nice for everybody once a month. And it should be analysis. But my guess
is people don\'t.

The other thing is whether the differences are big enough to affect the
forecasts in AEO. You are showing a three percent difference in natural
gas consumption or a one percent difference in petroleum consumption. Is
that enough to affect forecasts? I don\'t know. Maybe not. But have we
gone back to look?

MS. MILLER: Actually, we\'ve imbedded these in this year\'s forecasts.
It\'s going to come out. The reference case will come out next month. It
turns out I thought natural gas would, but, in fact, it doesn\'t because
the AEO we set up differently than the AER.

We already have the non‑utilities in the generation sector. And when we
pulled them out of the industrial sector, it was sort of imperfect. So
it turned out that it doesn\'t really affect the forecast.

DR. BURTON: I just want to underscore what Mark said. Make them in the
red. Make them blink. Put a warning on the monitor. I\'m exactly the
sort of person that he is talking about.

I think that anything that you can possibly do to help the user be aware
of this, particularly people that haven\'t perhaps used the series
before, would be very, very helpful.

MS. MILLER: So you agree with putting something on each table?

DR. BERNSTEIN: The first year.

DR. BURTON: Yes.

DR. BERNSTEIN: All it has to say is \"This is different than previous
tables\" so people at least know to look.

CHAIRPERSON CRAWFORD: Any other comments or questions?

(No response.)

CHAIRPERSON CRAWFORD: Thank you.

Are there any questions or comments from EIA, the audience? Questions or
comments from the public?

(No response.)

CHAIRPERSON CRAWFORD: Before we adjourn, I just have a couple of
announcements. For those of you who weren\'t here this morning, dinner
reservations are at the 701 Restaurant, which is at 701 Pennsylvania
Avenue, Northwest. And they\'re at 6:00 o\'clock. So if you are staying
in the Holiday Inn Capital, we will meet in the lobby at 5:45. And then
we will figure out whether or not we are walking or taking a cab,
depending on what the weather is like.

Again, for those of you staying in the hotel, we\'ll meet at 7:30 to
come to the meeting here tomorrow.

And we will reconvene here at 8:30 tomorrow morning. So, with that, I
guess our meeting is adjourned.

(Whereupon, at 4:14 p.m., the foregoing matter was recessed, to
reconvene at 8:30 a.m. on Friday, October 25, 2002.