AMERICAN STATISTICAL ASSOCIATION

COMMITTEE ON ENERGY STATISTICS

FALL MEETING

FRIDAY,

OCTOBER 25, 2002

The meeting was held at 8:30 in Room 8E-089 of the Department of Energy,
1000 Independence Avenue, SW, Washington, D.C., Carol G. Crawford,
Chair, presiding.

[PRESENT]{.underline}:

CAROL G. CRAWFORD, Ph.D. Chair

F. JAY BREIDT, Ph.D. Vice Chair

MARK BERNSTEIN, Ph.D.

JOHNNY BLAIR

JAE EDMONDS, Ph.D.

JAMES K. HAMMITT, Ph.D.

NICHOLAS W. HENGARTNER

CALVIN A. KENT, Ph.D.

WILLIAM G. MOSS, Ph.D.

POLLY A. PHIPPS, Ph.D.

RANDY R. SITTER, Ph.D.

ROY WHITMORE, Ph.D.

[ALSO PRESENT]{.underline}:

CALVIN A. KENT, Ph.D. Former Committee Member and EIA Administrator

[ENERGY INFORMATION ADMINISTRATION REPRESENTATIVES:]{.underline}

GUY F. CARUSO Administrator

MARY J. HUTZLER Acting Deputy Administrator

NANCY J. KIRKENDALL Director, Statistics and Methods Group

PAUL McARDLE, Ph.D.

DAVID F. MOREHOUSE

I-N-D-E-X

[AGENDA ITEM]{.underline} [PAGE]{.underline}

[ASA Chair\'s Opening Remarks]{.underline} 3

Carol Gotway Crawford

ASA Committee on Energy Statistics

EIA\'s Administrator: Remarks of Appreciation . . .3

[EIA\'s \"Enhanced\" Voluntary Reporting of]{.underline}

[Greenhouse Gases Programs]{.underline}

Paul McArdle 5

Office of Integrated Analysis and

Forecasting, EIA

ASA Discussant: Jae Edmonds 27

Questions from the Committee 38

[Organization and Delivery of Energy]{.underline}

[Information in Spatially Referenced Forum]{.underline}

David Morehouse 54

Office of Oil and Gas, EIA

ASA Discussant: Mark Bernstein 72

Committee Discussion 78

[ASA Committee on Energy Statistics]{.underline}

[Contributions to EIA]{.underline}

Calvin Kent, ASA Committee 85

Committee Discussion 103

Suggestions for the Fall 2003 Committee Meeting .106

Adjourn 158

P-R-O-C-E-E-D-I-N-G-S

8:05 a.m.

DR. CRAWFORD: Good morning, I\'d like to go ahead and open the meeting
this morning and I\'ll ask any committee member, guest, or EIA staff
member who was not here yesterday to introduce yourself at one of the
microphones.

MR. COPELAND: My name is Brian Copeland. I\'m with BCS, Inc.

MR. TALLY: My name is Roy Tally. I\'m also with BCS, Inc.

DR. CRAWFORD: And anyone who did not sign in yesterday out in the hall
with Linda Minor is asked to sign in this morning at the break or before
you leave. Before we get going with Paul\'s talk, Administrator Caruso
has asked for some time, so I\'ll turn the meeting over to him now.

ADMINISTRATOR CARUSO: This won\'t take very long. I just wanted to
recognize several of the committee members who will be leaving their
services to the committee as of this meeting. First of all our chair,
Carol Gotway Crawford, thank you for your leadership and service for the
last six years. I have a letter signed by Secretary Abraham thanking you
for your service and a certificate signed by less important current
administrator and more important Nancy.

DR. CRAWFORD: Thank you very much. I appreciate it. Thank you.

ADMINISTRATOR CARUSO: And, Roy, your six years of great service has also
been highly appreciated by EIA and Secretary Abraham. Thank you very
much. And last, but not least, one of my illustrious predecessors Cal
Kent. Thank you again for coming to an encore performance, and a
certificate as well.

DR. KIRKENDALL: You get a certificate too. DR. CRAWFORD: Thank you so
much.

ADMINISTRATOR CARUSO: I get to learn from Cal twice. First during the
first Gulf crisis and now again.

DR. KENT: That\'s right.

ADMINISTRATOR CARUSO: Carol thanks very much for that time.

DR. CRAWFORD: Okay Paul, we\'re ready whenever you are. Paul McArdle
will be talking about EIA\'s voluntary reporting of Greenhouse Gases
Program.

DR. MCARDLE: Well thank you. Just let me get this up here. It should be
working. Well, first of all, my name is Paul McArdle. I\'m the program
manager for the voluntary reporting of Greenhouse Gases Program and
it\'s an honor and a pleasure to be here today and to discuss the
program with you, and I appreciate you coming here and looking at the
paper and giving us feedback on the program.

DR. CRAWFORD: Dr. McArdle, would you mind putting on the microphones
please?

DR. MCARDLE:: Oh, okay. This here? Okay.

DR. CRAWFORD: The rustling of papers can obscure you.

DR. MCARDLE: Sure. This just hooks anywhere. It\'s wireless. Okay, well
thank you. I\'ve overcome those technical difficulties but we\'re ready
to go. First of all, presentation objectives, what I\'d like to go over.
First of all, provide some program information, background information,
highlight some of the reasons people report to the program. It is a
voluntary survey. No one is required to report. We\'ll discuss some of
the accounting issues and criticism of the existing program, discuss the
President\'s Climate Change Initiative and the 1605(b) program because
the Climate Change Initiative has specific guidance set up by the
President for us to implement here at EIA and the Department of Energy,
discuss some of the accounting issues in the enhanced 1605(b) program
that we anticipate coming up, and also, to cover the questions from the
committee.

First of all, the program was required by the Energy Policy Act of 1992,
and it gave reporters an opportunity to establish a public record of
greenhouse gas emissions, reductions and commitments to reduce emissions
in the future. It\'s a very flexible program. It permits the reporting
of a broad range of actions, mainly designed to encourage participation.
Reports are self certified by the reporter. A signed letter is required
attesting to the accuracy of their reports. The first data was submitted
in 1994.

And, you may ask why would people voluntarily go through all this
process because it is a costly process to gather all this data. First of
all, it is obviously public recognition for the reporters in terms of
environmental stewardship. They get a certificate signed by the
Secretary of Energy and the EIA administrator, so Guy will be signing
certificates or will be using a signature to sign certificates this
year. It provides a record of achievement, establishes a public record
of what they\'ve done in terms of reducing greenhouse gases. It provides
them experience in estimating and calculating greenhouse gas emissions,
because when we first started a lot of people really didn\'t know how to
do this and many of our reporters are quite expert at it now.

Greenhouse gas technologies, it certainly gives people out there in the
public, as well as reporters, knowledge of what technologies and what
mechanisms are being used to reduce greenhouse gas emissions, and also
familiarize many of the reporters with some of the accounting issues
which I\'ll discuss later on, and are very thorny when you get down to
maybe crediting emissions reductions.

Here are some of the summary of statistics of the program. I would just
like to focus on the top row, which says entities and projects reported,
because the lower portion project level reductions reported had some
definitional issues which I\'ll discuss later, then come back to those
lower tier statistics. But, the first three rows are designed mainly to
give you information on how quickly or by how much the program has grown
since its inception in 1994. It has more than doubled in terms of the
number of reporters reporting to the program. The number of projects
reported to the program have tripled, and the number of folks reporting
at the entity level and that\'s for their mission for their entire
operation has more than doubled from 40 to 100. So, just keep those in
mind. I\'ll come back to the lower project level reductions again. I got
the slide again later on.

And, just to give you some idea of the flexibility that was engendered
by the legislation in the regulations \-- or I shouldn\'t say the
regulations, the guidelines that were promulgated for this program,
it\'s very flexible. It has broad reporting mechanisms. You can report
for your entire entity, your entire firm, or you can report just
specific project activities. You can use two types of baselines,
baselines being what you compare your actions against in order to
calculate your reductions. You\'ve got a basic reference case which is a
historical baseline, a point in time, what your emissions were, and a
modified reference case which is really a counterfactual baseline, what
your emissions would have been had you not not taken this action. So,
that\'s more of a hypothetical or counterfactual.

You can report both direct emissions or indirect emissions, indirect
emissions being mainly this deals with reductions, not emissions, but
indirect emissions reductions whereas if you take some action, I go out
and buy a high-efficiency air conditioner, that doesn\'t reduce my
emissions but it reduces the emissions of someone else at the power
plant. So, that would be an indirect reduction for me, and that\'s just
an individual example.

There are ten categories of emission reduction projects. The form is
very detailed. It\'s about a 40-page form. Not everybody fills it out,
but you only fill what projects apply to you. We cover all the Kyoto
gases, the six major gases. We also allow commitments for future
reductions to be reported. We also have a 1605EZ form, which is
basically like your 1040A. This was designed for small simple projects,
and small reporters, although it\'s far less detailed. It\'s only a
single year reporting. It has no history. You can\'t report
international activities on it. It\'s intended for smaller entities.

So, again that\'s in a very \-- hopefully abbreviated fashion. I gave
you the two forms that are used and the inherent flexibility in those
forms. Here, I just want to highlight some of the key accounting issues,
the very global key accounting issues in greenhouse gas emissions
reduction. First of all, the reporting level, entity versus project
level reporting, they have pluses and minuses which I\'ll touch on
later, baselines, the historical baseline versus the business-as-usual
baseline, and some folks also use a unit of production baseline that is
1990, my greenhouse gases per ton of cement were \"X\". Now in 1995, my
greenhouse gas emissions are X minus Y tons of greenhouse gases per ton
of cement.

So, it\'s a unit of production baseline. I\'ll come back to it later
because it\'s a key component of the President\'s climate change because
that\'s focused on what\'s called greenhouse gas intensity. Reduction
ownership, direct versus indirect, what I was talking about before
obviously there\'s an ownership issue there. If you\'re crediting and
I\'m claiming indirect emissions, I\'m reducing someone else\'s
emissions, and if I claim it, maybe they can claim it too. So, there\'s
an ownership issue.

Verification, as I said before, they self certify at this point. A lot
of people, NGOs, non-government organizations, environmental groups
think verification should be done by an independent third party. Okay,
what is the appropriate level of reporting? And, then I\'m going to
touch on each one of those previous accounting issues. Entity level
reporting, reporting the emissions and/or reductions of an entire
entity. Largely, what\'s inside your fence, you know if you\'re a firm,
kind of use that terminology versus project level reporting where
you\'re just reporting emissions reductions caused by a specific action.
Yes?

DR. PHIPPS: Can you define entity level, what an entity is in the form?

DR. MCARDLE:: We do have a definition in the form. I don\'t have the
form with me though but I could get it for you.

DR. KIRKENDALL: There are probably also different interpretations among
respondents.

DR. MCARDLE: Yes, again, getting back to the guidelines that were
formulated in \'94, and what the administration at that time interpreted
of those guidelines was that Congress was saying we want a flexible
program. We want to get people started. We want to encourage
participation and just get people familiar with this, get them started
in terms of thinking how can I reduce my greenhouse gas emissions and
get people familiarized with the process. There are some people that
report a combination. They report their entire entity, but they also
report on specific projects that they have underway.

MR. HENGARTNER: Is there double counting?

DR. MCARDLE: There is an element B well, first of all, when we report
our summary statistics, we only report project level reductions. So, we
limit it to project level reductions. But, the double counting issue
comes in between direct emissions reductions and indirect emissions
reductions. It\'s basically 80/20, 80 percent direct, 20 percent
indirect, so that if you want to set a boundary condition on the double
counting, maybe it\'s 20 percent of the total project level reduction
stat. I\'ll show you on that table again.

Emission baselines, it\'s a basic or historical baseline and the
difference between emissions and 2000-X, and emissions in an earlier
baseline year. Some of the benefits, it\'s easy to measure and verify,
although it\'s not meaningful for projects or a single facility, and it
measures the outcome, not the cost. In other words, you can\'t B you
really don\'t see causality necessarily when people are reporting at the
entity level. You just see a big number. Maybe it gets smaller, but you
don\'t know exactly why, unless you\'re asking for that type of
information. The modified or business-as-usual baseline is the
difference between actual emissions and what emissions would have been
in the absence of the action. So, there\'s your hypothetical, your
counterfactual.

One of the problems with that, it\'s very difficult to verify the
reference case because it\'s a hypothetical. You can\'t really verify it
per se. You can\'t see it. You just have to look at it and say does it
make sense. Okay, the unit of production baseline again, it\'s easy. As
I was mentioning before, I actually didn\'t mention it, but it\'s easy
to construct for industries that have a homogeneous output, cement,
steel, things like that.

But, once you get into diversified firms, conglomerates, that have a lot
of different products, this is difficult to do and that\'s actually one
of the things the DOE Office of Policy who will be leading this effort
on the 1605(b) enhancements to the program via the President\'s Climate
Change Initiative is dealing with this issue because the President\'s
program is based on reducing greenhouse gas emissions intensity.

So, it\'s great for those firms in the manufacturing sector that have a
homogeneous output, but for some conglomerates, a diversified firm, well
what do you use for a unit of production baseline? Getting again to your
point on emissions reductions in ownership, direct emissions, emissions
from source owned wholly, or in part, or at least by an entity indirect
emissions, and these are definitions I\'m taking in part from the
guidelines and the reporting forms, emissions from sources not owned or
leased by an entity that occur wholly or in part as a result of its
activities.

Now, obviously, double reporting possibilities, inclusion of both direct
and indirect emission reporting yield the possibility of double
reporting. Like I mentioned, boundary case may be 80/20 between direct
and indirect.

MR. HENGARTNER: How do you know?

DR. MCARDLE: Oh, when we do the summary statistics and add up all the
project reductions, we actually classify them by direct and indirect
emissions reductions. So, when you total them all up, 80 percent of the
total is in the direct column, about 20 percent is in the indirect
column.

MR. HENGARTNER: Good, thank you.

DR. MCARDLE: Okay. As I said, the current program requires self
certification. The President\'s Climate Change Initiative calls on
measures to increase the verifiability of the current program, so
that\'s one of the things DOE will be working with in terms of
implementing the President\'s Climate Change Initiative.

Okay, criticisms of the program: We recently went through the OMB
clearance, the information collection request. When we go to OMB to
extend our data collection, we went through this last summer into the
fall and it was a very protracted discussion with OMB on direct
emissions reductions, project versus entity level reporting, potential
for double reporting, the potential of grouping apples and oranges
together, and OMB said after much ado, OMB cleared our data collection
for another two years to Halloween of next year. So, I don\'t know why
they picked that date, but October 31^st^, 2003, and they said okay, you
guys are clear until then. You can keep collecting data, but they said
we want you to look at certain things and these certain things I\'m
going to go over right now, and they said we think you should do these
things unless you can come back to us and say we shouldn\'t.

And the things they mentioned are, number one, and I\'m going to go to
them here. This slide doesn\'t really \-- I\'ve run out of time, and I
didn\'t want use too much time. I probably should have prefaced this
slide better, but they said on baselines you guys should require
historical baselines because that\'s the only way you can tell there\'s
a true emissions reduction and you should only consider cases where
there\'s a counterfactual, a hypothetical baseline may be used if you
can justify it. It\'s not totally dissimilar from what we do, but it
would be a break in a more historical direction, reporting level.

They basically said that you really need to look at entity level
reporting. That way, you don\'t have leakage to other parts of a firm,
or you don\'t have firms just cherry picking projects where their
emissions are going up everywhere else, but those projects are producing
reductions. So, they said, first of all, either require entity level
reporting or, if you do project level reporting, include entity level
energy components in there because, again, energy when you look at total
greenhouse gases, energy related carbon dioxide is probably about 81
percent total U.S. greenhouse gas emissions.

So, three more things, verifications, include a form, include a question
on the form on verification. Did you do independent third party
verification? At least, put it in their mind that hey, why don\'t you go
and consider doing that because that would be a step beyond self
certification, and this would give them a question and we\'d probably
ask who did it, who was the verifier, what was the process, if we
included a question like that.

Double counting: Collect information to identify potential double
counting of direct and indirect emissions. So, somehow we would have to
match up the direct and indirect to see if there\'s overlap. And lastly,
they said data quality. Explore alternative ways to assess and
characterize data, and again, that mainly flows from the fact that some
of our tables in the report, we report total emissions reductions, and
that was rare. I mean in most tables, we always broke out at least
direct and indirect, but there were a few tables that had total
reductions, and it wasn\'t disaggregated by direct or indirect. And,
actually we made some tables in our latest report that actually go
another layer, broken down by four levels, direct, broken in by
reference case. So, you have a four level disaggregation, and that\'s
really the issue that they keep bringing back to us.

So, again, going back now that we\'ve gone over this lower tier, the
project level, this one here, these definitions over there, you can see
better the breakdown. As you can see, just an example for the 2000 year
and these are in million metric tons of carbon dioxide equivalent,
direct emission reductions were 187 million metric tons, and indirect
were 61 million metric tons. So, that gives you a breakout of the direct
and indirect. The potential of double counting there and there\'s two
other categories that are not broken out by indirect and direct. Carbon
sequestration projects, mainly international in scope because that\'s
where there being done but reporting to us at 9 million metric tons.

And the unspecified, the lower one, that\'s our 1605EZ form which again
does not provide a lot of information. That\'s another issue they hit us
on is that don\'t group the EZ with a 1605 because you don\'t have
enough information on the EZ to match it up. So, there\'s 12 million
metric tons there. Okay, that\'s the last time you\'ll see that. You\'ve
seen it twice.

Okay, now we got the guys from OMB but along came the President in
February, Valentine\'s Day, so all these holidays are hooked in with
1605(b). On Valentine\'s Day, the President announced his Climate Change
Initiative, and it has many elements, but one of the elements is
enhancing the 1605(b) program. First of all, he said substantially
improve the 1605 emissions reduction registry and, that\'s what he
referred to it as, the emissions reductions registry although EIA
hadn\'t used that term before.

And, he said improvements are to focus on measurement accuracy,
reliability, and verifiability. And, when we go through this process, he
asked us to look at some of the emerging domestic and international
registries because there are registries coming up in California, a
couple other states. California is the key one because they are almost
ready to go. Actually, they just started. California, the UK, Australia,
Canada have voluntary registry programs and there are some other
protocols being developed by the WRI so all those things he says go out
and look at what they\'re doing and consider incorporating that into the
enhanced 1605(b) program.

And all this is designed to protect and provide transferable credits for
emissions reductions reported to 1605(b). He went on to say
businesses/individuals that register reductions are not penalized, well
actually there\'s a term he said to insure that businesses and
individuals that register reductions are not penalized under future
climate policy and to provide transferable credits. Transferable credits
are given to companies that show real emissions reductions.

MR. HENGARTNER: Are these reductions national or international?

DR. MCARDLE: Well, that\'s a good question because right now under the
current program you can report international activities.

MR. HENGARTNER: You wouldn\'t count it in the U.S.?

DR. MCARDLE: Well, that\'s under the current program but, yes, but in
the discussions so far because actually this process has already started
because the Department of Energy issued a notice of inquiry. I wish I
had a slide on that. I should have. A notice of inquiry in June or July,
I should know, getting initial comments on how this program should be
set up and, one of the issues is international emissions because people
say oh, you should include it because otherwise people will just
offsource, you know, outsource their emissions internationally and show
reductions at home. But, other people come back and say no, don\'t do
that because there\'s potential for double crediting, because if you
give them credit, they may be getting credit under another country\'s
program and the international emissions are going to be very difficult
to verify.

So, you get people coming at that one both ways. But on July 2^nd^,
actually July 2002, I don\'t have the exact date but it was in July of
2002, the Department of Energy along with EPA, the Department of
Commerce, and USDA sent recommendations to the President on how this
program should be enacted or implemented I should say.

These are some of the things they recommended: Transparent accounting,
independent verification, emissions intensity as a baseline, and that\'s
that unit of production baseline I mentioned before, entity-wide
reporting, credits for action to sequester CO2 from the atmosphere and
for emissions reductions, have a process to evaluate the existing
1605(b) tons because we have close to a billion tons reported
cumulatively since 1994, and one of the things they\'re recommending is
a process to look at those tons; and, to put a system in place by
January, 2004, to get this whole process in place.

Enhancing the 1605(b) program accounting issues: There are a number of
them; organizational boundaries, how to define an organization, and
there\'s another issue there in terms of equity share versus management
control. There are a number, particularly in the petroleum industry,
where you have joint ventures and there\'s emissions reductions and the
petroleum industry is very interested in this aspect. How do you account
for emissions reductions there?

And, emissions when you have joint ventures, do you do it by equity
share or is it management control or some other variant? Operational
boundaries, what is the best treatment of direct and indirect emissions?
Should indirect be in? Should it be out?

Reporting level: Should entity level reporting be required, and what is
the appropriate role of project level reporting? All these questions,
the DOE\'s Office of Policy is going to be grappling with in terms of
revising the guidelines, and just as a tangent, the Department of Energy
announced a series of workshops, one in D.C. next month and three
regional workshops in December to go around the country and get public
comment just on these questions I have here, not the exact questions,
but questions like them.

Again, the geographic boundaries, international emissions allowed or
required. I just discussed that. Baseline issues, what is the
appropriate baseline to use? What about intensity based baselines, will
they work? What is the appropriate level verification? And lastly,
emission calculation modules, what you\'re seeing in some of the other
registries that are developing as folks developing what\'s called
transparent emissions calculation modules, kind of like your Schedule A
that the IRS says, or whatever Schedule A is, reductions I guess.

Fill this out, and then we\'ll put it into the form or put it into your
report. Same thing with these emission calculation modules. Here, fill
out this form on stationary fossil fuel combustion and then give us all
that data. Then, we\'ll know you got the right number.

Lastly, questions for the committee: Given the voluntary nature of the
survey, is it possible to develop a statistical frame of the data to
compile summary statistics? Given the voluntary flexible reporting
guidelines, is there anything EIA can do to correct for self-selection
bias? And lastly, what other general recommendations can be offered to
help enhance the measurement, and that\'s getting to what the President
has directed us to do, enhance the measurement accuracy, reliability,
and verifiability, I didn\'t correct that, of the data reported? It says
date, but it\'s data.

DR. CRAWFORD: Thank you very much. Originally, Neha Khanna was going to
be the ASA discussant on Paul\'s work but she was very sick before the
meeting, but she did send written comments and I\'ve given those to
Paul, so you might look at those. And, Jae Edmonds has graciously agreed
to fill in for her and so I\'ll turn it over to him.

DR. EDMONDS: Okay, I think it was causal. Paul, I think you\'ve done an
excellent job in addressing what I would characterize as an inherently
insoluble problem.

DR. MCARDLE: Just what I wanted to hear.

DR. EDMONDS: It seems like EIA always gets these tasks. I started
thinking, I\'ve been thinking about this for ten years and you\'ve done
an excellent job. I mean you\'ve really, I think, pushed it forward in a
way that\'s extremely useful. Let me work my way toward answering your
three questions.

DR. MCARDLE: Okay.

DR. EDMONDS: But, just sort of starting back ten years ago when I first
started thinking about this, and as you pointed out, the origin of this
is the 1992 Energy Policy Act, and there was Section 1605(b) that
created the voluntary reporting of Greenhouse Gases Program, and the
intent was really good. This is a program that is motivated by the best
of intentions.

The motivation is that those who act early should not be penalized
relative to those who came back and don\'t do anything, and so you
should get credit for doing good works rather than being penalized for
doing good works. And, it sort of was a reaction to some of the things
that are going on in the acid rain legislation. And so, EIA actually
took this on. They created a reporting system, established a system, and
it\'s been actually, as you reported, working and becoming increasingly
utilized. It\'s been extremely successful and numerous firms and
projects have been reported even though the system is voluntary.

There\'s no reason that anyone has to report. The key word here is
voluntary. There is no emission allocation or limitation in the United
States for greenhouse gases, and the intent of the present United States
policy is to continue a program based on voluntary actions. So, why are
there any volunteers is actually a question.

And you pointed out, I would put three things as the principal reasons
that firms participated in this. It\'s to show that firms are good,
environmentally friendly, and a responsible entity. Second is to
actually develop some understanding of the how-to of collecting data and
you see this in a number of firms, even that are motivated not by this
but by other considerations, thinking that in fact, because they
operated in an international world that this is not the only place, and
there are, in fact, limits on emissions in other parts of the world.

And so, getting an understanding of this in all the operating
jurisdictions is, in fact, a very useful thing to do. And finally, to
stake a claim to a potentially valuable future asset, and the assets is
a non-trivial asset, and so you, by reporting can stake a claim to it.
As you pointed out, it\'s been criticized for a variety of reasons. One
person reflected to me, he said, all you need is a note from your mother
saying that you\'ve reduced your emissions and you\'re in. And then, you
have a longer list of issues like the cherry picking and the double
counting. I won\'t go into that.

The President\'s Valentine\'s Day speech did charge the Department of
Energy, as he said our government will also move forward immediately to
create a world-class standard for measuring and registering emissions
reductions, and we will give transferable credits to companies that can
show real emissions reductions.

And again, the intent of this is to support the attainment of the
President\'s goal of a reduction in greenhouse gas intensity of 18
percent in a decade as measured by the six Kyoto gases, the carbon
dioxide, the methane, nitrous oxide, the hydrofluorocarbons, the
perfluorocarbons, and the sulfur hexafluoride as weighted by the hundred
year global warming potential coefficients.

Now, the key issue as I see it is trying to create a metric that could
in principle be mapped from emissions mitigations as reported at the
microlevel, the firm or the project, to the national emissions
statistics; that is, the emissions statistics as EIA reports them. And
the question is, is there any way to create a reporting system that is
generalized, and if you got everybody into it would provide information
that\'s inherently consistent with national emissions statistics as
presently reported. And the problem of course is that the parties that
are reporting emissions mitigations and not emissions.

So, here\'s one thing that I think is worth recognizing, and there\'s an
identical problem that is faced by something called the clean
development mechanism. And, the clean development mechanism was created
under the Kyoto Protocol, the framework convention and the climate
change and, of course, as we all know the United States is a party to
the Framework Convention on Climate Change but is not a party to the
Kyoto Protocol and the clean development mechanism creates emissions
mitigation for projects and parties that are credible against national
emissions limitations obligations under the Kyoto Protocol.

So, there is here in place a real B there\'s real money on the table
here. This is real stuff and people are really worried about this, and
you actually have some fairly intense negotiations on that mechanism.

And so, I come back to the central problem that is very simple, that a
voluntary reporting system is an open system, and in an open system the
actions taken by one party or under one project can be undone by actions
of another party or another project, leaving the system unchanged and
when you add everything up, it doesn\'t connect. A farmer who changes
crops to put a field under a no till system to store carbon in soils and
get credit for that, can be undone if he simply goes down the road and
turns a fallow field into a traditionally managed field and that\'s the
product from that reservoir, and the system is unchanged but he\'s
gotten a credit in the process.

There\'s an incentive to voluntarily report any action that reduces
emissions and leave unreported any action that raises emissions. So now,
what does it mean, what does the President mean creating a credible and
transferable entity, credible and transferable for what and to whom and
when? And, what of course, is the emissions mitigation, the credit would
be used presumably at some future point in time to get some future
obligation, and after all the intent was to give credit to those who
were taking early actions to protect the environment.

The presumption is that at some point, a cap and trade or an energy or
emissions tax will be imposed and that EIA credit can be used to satisfy
this future commitment. And, the potential value of these things is
fairly large, from the reports ten million metric tons of carbon
emissions mitigation that can be offset against an obligation, has at
\$10 a ton, \$100 million in assets that has been created.

So, there\'s a lot of value that could be created there, but because the
future system is unknown, neither when it will go into effect, nor its
nature, and because the relationship between the current credits and
future responsibilities is unknown, the value of credit is highly
uncertain and because of the enormous wealth transfer implications of
any emissions control regime the negotiation that would go on
surrounding the creation of the future legislation would doubtless take
into account the existence of the credits, can either recognize them or
not, or recognize them and discount them.

So, in principle, the open system is really quite leaky. So, where does
that leave you? You have your three questions, given the voluntary
nature of the survey is it possible to develop a statistical frame of
the data reported for the purpose of compiling summary statistics? And
here, it depends on what you think the data are measuring. What is it
we\'re trying to measure?

And, if you think the data are measuring emissions mitigations relative
to a referenced case in which the mitigation is caused by the moral
imperative of a stated national goal, something we used to call the
announcement effect, I believe your task is hopeless. Having said that,
implementing the OMB recommendations seems like it would help create
something that is more homogeneous, although what you\'re measuring
still remains a question that I think you have to face up to. It\'s not
clear what you\'re measuring. Again, creating some standardization of
the definitions of emissions mitigations would help, but doesn\'t
overcome the fundamental problem of an open system.

Independent verification would help. As you know, you\'ve got whatever
it is that they said they have but, again, the definition from outside
is really the fundamental problem and there\'s nothing you can do about
that.

Your second question was given that the current flexibility of the
reporting guidelines is there anything EIA can do to correct the
self-selection bias? Again, it depends upon what you\'re measuring.
Again, if you think that the data are measuring emissions mitigation
relative to this referenced case caused by this moral imperative, again
I think your task is absolutely hopeless.

Thinking about it briefly, you can see the dilemma. In the year 2000
there were 200 million metric tons of carbon emissions, carbon dioxide
emissions equivalent reductions. The U.S. emissions in that year were
roughly 7,000 million metric tons of carbon dioxide equivalent, and now,
net U.S. emissions actually rose, so we have 200 million metric tons of
carbon dioxide emissions reductions or emissions in the United States
rose between 1999 in the year 2000 by 165 million metric tons.

And, even if you say let\'s take this constant energy to GNP ratio that
is hypothesized in the 18 percent reduction, you would\'ve raised
emissions by 283 million metric tons and that year\'s intensity
reduction, which is roughly on the order of the decadal average was
about 118 million metric tons.

So, you\'ve reported more emissions mitigation by a factor of two than
you would by that simple model. So, what does that all tell us? And like
I say, not much, except that trying to translate the reporting emissions
reduction into some kind of measure of real reductions is entirely
dependent on the modeling framework to define real, and at this point,
we\'ve entered into a highly subjective world.

Finally, what other general recommendations would I have to EIA and the
first one would be to take a look at the clean development mechanism.
Here there is some real money that\'s on the table. People are actually
getting credit against a national obligation where there is a real test
of whether or not you\'re satisfied that there\'s some pressure to make
sure that only real emissions mitigation is counted. And, being
cognizant of that enterprise would not only provide an experiment from
which learning can occur, but it can also create a situation such that
in the future, you can take two parallel systems that are evolving, one
in the United States and one in the other Annex one countries of the
world, and eventually create something which is consistent so that you
really need to be cognizant of it and have some mapping between the one
and the other.

Finally, until there is some universal reporting, and an explicit point
of reference from which mitigation is to be measured, there is
absolutely no way to create a comprehensive statement of emissions
increases or mitigation, and without that, I think you\'re really
operating in a very awkward position. And, as I said at the beginning, I
think you\'ve done an excellent job being given an impossible task.

DR. CRAWFORD: Thank you very much. Paul, did you get all that?

DR. MCARDLE: Yes, I did.

DR. CRAWFORD: Okay, good.

MR. HENGARTNER: Jae can just give you typed notes.

DR. EDMONDS: Yes, I can give it to you.

DR. CRAWFORD: No, Bill\'s going to bring a disk how\'s that? You can
copy it to disk, is that okay?

DR. EDMONDS: Sure.

DR. MCARDLE: That will be great, very helpful. I\'d really appreciate
it. I don\'t know, am I supposed to respond?

DR. CRAWFORD: Yes, please do if you like. You don\'t have to.

DR. MCARDLE: First of all, I appreciate that very detailed discussion. I
think going in, we knew at least, not we, I should say I had a strong
sense that you may find the statistical abstracts or the statistical
imputations the first two questions would be difficult given the fact
that, again, it\'s voluntary so we all suffer from the fact that it\'s a
select sample and it varies from year-to-year and depending on what
industry.

I mean we\'re well represented in the electric power generation
industry, but once you get away from that, the participation rates kind
of fall away and vary across the board. I was going to ask you a
question, though, can I ask you a question back? Sorry. You mentioned
the CDM.

DR. EDMONDS: Right.

DR. MCARDLE: How does the CDM, that\'s a project based system?

DR. EDMONDS: It is very similar in that it is an open system; that is,
there are projects and entities.

DR. MCARDLE: It\'s open, right.

DR. EDMONDS: And so, it can be in principle implemented at either the
project level, or it could be that a whole nation would report. And so,
again it is very, very similar in the problems that it faces. You\'re
trying to measure emissions mitigation and yet this would, in fact, in a
clean development mechanism, it creates a credit which could actually be
applied against a real national obligation, which is a hard number; that
is, it\'s measured relative to a historical value and so, because of the
negotiated nature of this, you\'ve got entities that want to make sure
that these are in fact real, that is it\'s real in the sense that it\'s
at least as much as if you generalize the approach, you would get not
more than the total emissions reduction relative to the referenced case
as defined from a fixed value.

DR. MCARDLE: And you control leakage because parties have a hard cap
within that country I guess?

DR. EDMONDS: No. See, that\'s where it\'s exactly identical. There are
Annex B nations to the Kyoto protocol. They have hard caps. They
actually have to, through adding, through fixing their national
emissions, plus trades with other nations that have hard caps, plus this
thing called the clean development mechanism, and then there\'s
something called joint implementation as well but that\'s an aside. But
this clean development mechanism is inherently the same thing as the
voluntary reporting system.

DR. MCARDLE: And that\'s because it\'s in developing countries that
don\'t have caps?

DR. EDMONDS: Exactly. Exactly, it\'s in uncapped countries; therefore,
they don\'t have that same explicit frame of reference.

DR. MCARDLE: So they, in theory could, I mean they could have a project
reducing in a developing country but emissions could be outsourced
within the developing country?

DR. EDMONDS: Exactly. It has all the same problems that you face and
again it\'s a system that has to deal with that but it also has to feed
into a system in which the obligations to Annex B parties, which is most
of the developed world, are real and finite.

DR. MCARDLE: Interesting, okay. Thank you.

DR. CRAWFORD: Thank you. Thank you very much.

DR. EDMONDS: All right, thank you.

DR. KIRKENDALL: I want to say something.

DR. CRAWFORD: Sure.

DR. KIRKENDALL: One of the things that came up when Paul gave his talk
in his dry run was that they\'ve actually, and I think it would help us
answer his first two questions, is that they believed that they have and
have done some work to look at this that the electric power sector is
well represented in the Greenhouse Gas Program. Well, we have electric
power surveys that go to the electric power sector and they actually
even report emissions on those forms.

So it seems to me, at least for certain groups of entities, like the
electric power sector, you actually could do something sensible about
saying how representative you think your data are.

DR. MCARDLE: Yes, that is the one area where we have, not only do we
have good coverage, we also have them reporting to EIA already all their
fuel data and emissions, you know, under other authorities under other
surveys so that is a B

DR. KIRKENDALL: So it\'s not totally hopeless? I mean there are pieces.

DR. MCARDLE: There\'s a piece and actually that\'s B

DR. KIRKENDALL: That\'s the best piece.

DR. MCARDLE: That\'s a big piece because that\'s about a third of total
U.S. greenhouse gas emissions.

DR. KIRKENDALL: So, another question, would be okay maybe there is
something sensible you can do there. Are there other groups that have
potential?

DR. MCARDLE: The other groups would be where you\'ve got concentrations
of emissions. That\'s mainly large sources, which are the easiest, and
that would be mainly in industrial sector but we don\'t have nearly as
good coverage there and there\'s also a little trouble there because the
EIA does the Manufacturing Energy Consumption Survey, the MECS, but
that\'s every four years. So, you have a historical marker to compare,
like a universe, but it\'s only every four years. There would be
potential there because maybe you can do some imputation or
interpolation between those years, so you have kind of a universe, a
baseline in which to compare what folks in those particular industries
are reporting.

I mean we have some reporters that are in refining, that are in
chemicals. We don\'t have nearly the coverage. We may have a couple big
ones but we don\'t have everybody; whereas, in the electric power sector
we have very broad or very wide participation.

DR. CRAWFORD: Jim, you had a comment or question?

DR. HAMMITT: Yes, a couple things. One is, I think, responding to your
first point. One proposal is that you report as you\'re dealing I guess,
the universe of responses and you just say these are the voluntary
reports of emission reductions and always sort of be very clear that
it\'s incomplete so that people can\'t sort of pretend this is a total
emission reduction that\'s occurring.

DR. MCARDLE: Okay.

DR. HAMMITT: So, you know, say what you can say the data and don\'t let
people get away with pretending it\'s something different.

DR. MCARDLE: Right.

DR. HAMMITT: If it means something different. That\'s the big point.
Some smaller ones, you asked about organizational boundaries and I
assume organizational boundaries are clearly defined from a tax
perspective.

DR. MCARDLE: Yes.

DR. HAMMITT: And would it make sense to use whatever that definition is
as kind of the legal entity?

DR. MCARDLE: That\'s actually something I have heard is the model or
your entity would mirror what you report financially because that\'s how
you\'re reporting to the government now.

DR. HAMMITT: Right, that would seem pretty clean to coordinate with the
financial side and everything else and then if a firm, you know, spins
off a division or something, they\'d have to decide what share of the
credits went with that division. They\'d have to design a share.

DR. MCARDLE: Right.

DR. HAMMITT: And set some liabilities already.

DR. EDMONDS: Although, actually the way it set up right now, they
wouldn\'t because you only report what you want to report.

DR. HAMMITT: Yes, okay. And then, I didn\'t understand when you had the
two reference cases, a basic and modified reference case and you showed
direct and indirect emissions reductions from that, and then you were
adding those two together but I thought they were like alternatives
categories.

DR. MCARDLE: They are and actually, let me get back there. I\'m not sure
how to show this whole thing. I\'ll go to it. If you notice here, we\'ve
disaggregated them and again, this gets back to the discussion we had
with OMB. In the past, sometimes we\'d have in the very beginning of the
report, we may have grouped together, totaled all the emissions
reductions together and, in some cases, maybe two years ago, we were
guilty. We always do a press release at the end of the year to announce
what the results were and we\'d say X tons of reductions reported.
That\'s one of the things OMB said well you got apples here and these
are oranges.

So what we did, and this is actually a table from the report, these are
all disaggregated. direct 187 million and it\'s broken down by this
modified reference case, which again, that\'s your hypothetical
counterfactual, what emissions would have been had I not taken this
action versus a basic reference case which is a historical point in
time.

This is what they were. Now they\'re this. Take the Delta and then the
indirect, the 187 and 61 plus 9 and 12 would add to a total, but they
don\'t like for us to show a total. So, but that\'s why there\'s no
total here because OMB didn\'t want us to show a total.

DR. HAMMITT: What\'s not clear to me is why you add the 153 and the 35.

DR. MCARDLE: I\'m sorry?

DR. HAMMITT: I guess I was interpreting this as you could calculate
direct emission reductions relative to one reference case or you could
calculate it relative to a different reference case.

DR. MCARDLE: You could.

DR. HAMMITT: Are you\'re reporting it in some different fashion?

DR. MCARDLE: They choose them. You only can use one reference case and
one of the problems with project level reporting is, oftentimes there is
no historical baseline because if I want to go out and put up a wind
turbine and there is no wind turbine there, the emissions were zero
before and they\'re still zero but don\'t I get a reduction relative to
what fossil fuel combustion is?

And so, what many of the renewable folks or utilities, let\'s say I have
a wind turbine, they\'ll say my reference case is the weighted average
of my fossil plants and all my other plants and that\'s 1000 tons per
megawatt hour and so every megawatt hour they produce on the wind
turbine they\'re going to say we get 1000 tons of reduction that because
that\'s why we get 1,000 ton reduction because that\'s why we went out
to build the wind turbine.

And actually, that explains the preponderance in project level
reporting, you\'re going to virtually most of the time you\'re going to
see the counterfactual or hypothetical baseline; whereas, at the entity
level then you can use more of a historical baseline.

DR. HAMMITT: Okay, I understand. So, this tells you sort of what the
people were reporting relative to B some people report relative to one,
other people report relatively different.

DR. MCARDLE: Yes.

DR. HAMMITT: And that\'s why its appropriate to sum them?

DR. MCARDLE: Right, they didn\'t want us because actually previous to
this latest go-around to OMB, which this report came after our
information collection request endeavors in 2001, we had never done a
table like this. We had always had direct, indirect, in sequestration
and unspecified. Actually, that\'s not true. Unspecified used to be
grouped in. But that\'s the EZ form.

So, we did this delineation in response to their comments, and you can
see the only thing are additive, we do add these together. Because of
rounding, it doesn\'t add exactly, but these two are summed to get this.
These two are summed to get this. And, these two are separate and this
all would add up to about 250 or 260, but we can\'t report that. We
don\'t report that number anymore.

DR. CRAWFORD: Okay, Calvin.

DR. HAMMITT: Just one other quick thing. On the indirect effects, I
guess it would be nice to try to the extent possible get the people get
the people doing the reporting to report as comprehensively as they
could indirect emission effects B

DR. MCARDLE: Yes.

DR. HAMMITT: B positive or negative, including outside the US.

DR. MCARDLE: We do ask people to explain their indirects and there\'s a
place on the form in the supplemental text where they can write that up,
and all these forms we always review that they\'re in conformity with
the guidelines, they are arithmetically correct, they\'re also plausible
too. We have to look at does that make sense? If they\'re not, we
usually go back to them and say what\'s going on?

DR. CRAWFORD: Calvin, would you like the last word?

DR. KENT: Just a quick comment to make is that much of this will solve
itself, as Jae pointed out, if these ever become monetized, because if
these start to have some sort of a value, somebody is going to have to
be able to go out there and verify, and if you\'re going to be actually
trading these things, you\'re going to have to have something that you
can definitely trade, and then we have a whole raft of experience with
the emissions tradings that we already have as to how you can actually
go about measuring this.

And, the IRS isn\'t going to give you a tax credit unless you can prove
that you deserve the tax credit. And so, the key to all of this to solve
these problems is for them to start having some sort of a market value,
because then it\'s going to be figured out pretty quickly how to deal
with some of these issues.

But, the overarching problem that you have here is that the reason that
we\'re talking about these credits and trading them and everything is
that we want to be able to prove that we\'re meeting some sort of
politically established goal that we can say gee, we\'ve reduced by X
amount or by X percentage or something like that, and that\'s your whole
problem right now. That\'s what your goal is, is to be able to claim
something.

We\'ll make up any rules you want and then claim victory after we\'ve
played the game and that\'s basically how it\'s been done right now if I
can be cynical enough. You all got to remember who was administrator
back in 1992 and who worked on this monstrosity, so I\'ve always felt
like I ought to crawl under the table when I was listening to some of
these comments. You\'re going to feel that same way too buddy.

ADMINISTRATOR CARUSO: The insoluble takes a little longer.

DR. KENT: Yes, the insoluable takes a little longer than the impossible
but I do think that we have to remember that these problems will become
more soluble if anybody ever decides that we\'re going to place a
monetary value on them, either as a tradable emission or a tax credit or
something. That\'s all I wanted to say.

DR. CRAWFORD: Okay. Thank you and thank you Paul and we\'ll get a copy
of Jae\'s comments.

DR. MCARDLE: That would be great.

DR. CRAWFORD: Okay, thanks.

DR. MCARDLE: Thank you. Let me just get out of this.

DR. CRAWFORD: Our next speaker just left. So, excuse me. I\'ll go see if
he\'s ready to go.

DR. KENT: Well, his time was up, so.

DR. MCARDLE: Did we use all his time?

DR. EDMONDS: Pretty much.

(Background discussion.)

DR. CRAWFORD: We\'d like to get going again. Our next speaker is David
Morehouse and he\'s going to talk to us about the EIA initiatives and
spatially referenced data.

MR. MOREHOUSE: Good morning. I\'m probably David Morehouse because of
the name on the badge. I am the senior geologist in the building. I am a
senior petroleum geologist at EIA and for the last ten years, I\'ve been
representing the Department of Energy on the coordination group of the
Federal Geographic Data Committee, which if you did your homework,
you\'ve read about.

About eight years ago or so, I was sitting chatting with the deputy
secretary and we had no person on the steering committee of the FGDC at
that time and he decided to make the administrator of EIA the
representative to that group, so Guy is now representing us on the
steering committee.

I must say being a geologist it\'s nice to talk to this group for a
change about something that\'s not rocks and their contents. But, what
I\'m about to talk about is kind of part history, part conceptual, part
technological, part economic, part bureaucratic. It tries to be
forward-looking and hopefully it\'s a little bit entertaining here and
there. But the bottom line is, is quite serious because this is
essentially intended as a wake-up call. It deals with not statistics per
se except as individual datums if you will but with how they\'re going
to be delivered in the future and the future is coming very, very
quickly.

That said, I\'ll get into the meat of it real fast. I\'ve got 22 slides
here so we\'re going to really move. Essentially what I\'m going to do
is start talking about paradigms. The paradigm deals with how mankind
preserves updates, disseminates, and retrieves knowledge and, in fact,
how man mostly learns.

The prior technology driven paradigmatic revolution is shown here.
You\'ll have to excuse the fact that this machine is compressing my
slides, so I get strange things like hanging Gs. We have essentially the
printing press. The one on the left is Johannes Gutenberg\'s printing
press, which was roughly in 1450. It is sitting over at the University
of Mainz in Germany right now.

Gutenberg essentially combined the technology of the paper, which we
imported to Europe about 200 years previously from China, the winepress
and movable type which was his real invention (inks had been around for
millennia) to essentially replace scribes, most of whom were monks at
that time, and the press essentially led to widespread book learning
and, in fact, to the university which didn\'t really exist before. We
had books that could be reproduced in numbers.

Five hundred and fifty years later, on the right, we have a Ryobi
four-color offset press. It does essentially the same thing as the one
of the left does, just better. The key paradigmatic point here is that
the fact that printing is the one-to-many technology and it freezes
information in time on the page and that\'s where EIA has been.

I\'ve been here since Day One and we\'re still there. We\'ve
fundamentally organized and delivered our information using the printing
paradigm and it doesn\'t matter whether we do it on paper or whether
it\'s on the Web, that how you get it. You get tables. You get, you
know, text. You get figures. It doesn\'t matter where it is, that\'s how
you get it. Let\'s move next to the revolution that\'s occurring as we
speak. Again, we\'ve got a technological change driving this, as it
always has with respect to knowledge, management and learning. On the
left is the ARPA net as it existed in December of \`69, not much to it.
Thirty years later on the right, is the World Wide Web as it was
represented in the June \`99 Burch Cheswick map of major ISPs. It gets a
little more complicated. What you\'re seeing here is in a sense a
realization of Metcalf\'s law, which is named for Roger Metcalf who was
the inventor of ethernet.

His law says you connect N-computers and you get N-squared potential
value. One might want to even argue you could raise the exponent a bit.
The key point here, however, is that the information age is a
many-to-many technology and essentially it does not freeze information
in time. I don\'t think I can scroll this so I\'m stuck. I didn\'t have
a run-through, by the way. I didn\'t have time to. The other thing is,
of course, that you can constantly update. You can update in realtime
over the web and that\'s something you can\'t do with printed material.

Now, I want to look at pictures and words for a second. You can supply
your own exponent in the top line. It\'s got to be at least one and
it\'s probably somewhere less than seven. There\'s just no better way to
quickly and efficiently convey information than by a picture, especially
when you\'re dealing with complex relationships of different kinds of
data and where rapid discovery of information is crucial.

As it says there, it is no accident that we\'re dealing with
visually-enabled technologies here as opposed to text. Of course, you
have to build your picture correctly to get across the information that
you want to get across and the question is how can that be done for
energy information? We\'ve made stabs at it with various kinds of
figures and charts and graphs, that sort of thing. We\'ve even done some
fairly rudimentary maps in some of our publications but they are really
rudimentary.

You can\'t interrogate them. They\'re just there, and essentially the
answer to the problem is focusing on location. It\'s the only attribute
that we have in common for most of our data and it\'s rare when location
is not the most important attribute of any piece of data that we\'ve
got.

Let me throw a couple of statistics at you. The first one is 20.045
billion barrels of crude oil. Does that mean anything? Can you say
anything about it other than it\'s a larger number? It happens to be the
U.S. crude oil reserves as of December 31, 2001. We\'ve attached a place
to it. It\'s now meaningful, a location to it.

Let\'s try another one. How about 151 megawatt hours? Same problem.
That\'s the amount of electricity generated in one hour according to our
website by the way by Alabama Electric Co-op, McWilliams Alabama Plant
when its operating at its summer peak capacity rate. Until you have a
location, the statistic is meaningless.

That being the case, let\'s put through some definitions real quick.
There are various versions of these around and what I\'ve done is
cobbled some of them together in something that I actually like. Most of
them say pretty much the same thing. Spatial data, which is the most
general category, is data or information that identifies the location in
two, three, sometimes four dimensions, and we\'re going to ignore
cosmology here for the moment and the characteristics of natural
constructive features and boundaries, wherever they may be, spatially
referenced data or spatial data that are not just floating around out
there but are referenced to a standard coordinate system.

Standard coordinate systems have a fixed origin, a known orientation,
and a known metrication scheme of some kind attached to them as well.
The next category is geospatial data, which is one you\'ll keep seeing
the name of fairly frequently or that label. It\'s spatially referenced
data that are pertinent to a planet. Typically, they\'re talking about
Earth, but we also use it for other planets and its immediate environs.

And then, we have geocoding, which may be a new term to some of you.
It\'s essentially the process of attributing a particular datum with
correspondent standardized location information. In other words, it\'s
the process of creating spatially referenced data.

And then, there\'s the term probably everybody\'s heard of, which is
geographic information system or GIS. This is frequently applied
haphazardly to a lot of the other things that we\'ve been defining above
and, in fact, it\'s just a computer tool, digitally based tool for
dealing with spatial data. It\'s very efficient, very effective in doing
that these days.

Now, this is what I started out with when I started doing GIS or what\'s
now called GIS. What we have on the left is an IBM 1401 mainframe. It
was marketed in the early \`60s. It was the first mass-produced all
transistorized digital business computer. The CPU module, which is the
leftmost box to the right of the card reader, measured 5 x 3 x 3 feet.
It had 4K of six-bit plus a parity bit ferrite core memory. The storage
expansion unit, which is the box next to the tape drive that brought it
up to a whopping 16K, and you have the usual stuff, the Hollerith card
reader and punch and a line printer, a couple of tape drives with it.
The cost was about in the six figures.

On the right, is the first Orion electromechanical plotting table. It
happened to be built in Australia. We had one of those. We also had a
CalComp, same basic design, very clunky, and if it worked at all, it
took hours to plot a map. That\'s assuming that the card reader didn\'t
eat your cards. The program did things like raise pin, lower pin, you
know, move along the abscissa, move along the ordinate. You were
constantly programming in pin codes.

In contrast, this is what I\'m working with these days. It\'s a Dell
precision 330 workstation with a 2 GHz processor, 440 MHz databus and a
video graphics card that\'s very speedy and powerful and 270 GB SCSI
drives. On the right, you have a Hewlett-Packard design jet 1055 CM plus
printer, ink jet. The print time for a 5 x 3 foot map is minutes.

The GIS market is booming and the spatial data market is booming. We are
right now at the bottom of the exponential curve shown here. Right about
in here. We\'re right at the bottom of the steep part of the curve. Data
Tech, which is an industry research firm, estimated that in 2000 the GIS
core business revenues comprised of hardware, software, and services
worldwide were about over \$1.5 billion. The software component alone
was about \$1 billion of that.

In recent years, Earth Sciences Research Institute (ESRI) has
experienced a 22 percent growth in business year after year. Don\'t rush
out to call your broker, they\'re privately held. Autodesk recently
reported 27 percent growth in its GIS business. MapInfo had 23.5 percent
average annual growth in its revenues from \`97 to 2001, and in the 2001
tech industry turndown year, it was just a measly 14.4 percent.

DR. CRAWFORD: David, can I interrupt you for a second?

MR. MOREHOUSE: Yes.

DR. CRAWFORD: You don\'t need to make the case about GIS to us. We know
about it.

MR. MOREHOUSE: I understand that. I\'m leading to this though.

DR. CRAWFORD: So, I guess what we\'d really like to know is what does
EIA plan to do with spatial data?

MR. MOREHOUSE: Okay. If you want to cut all the way to that, we will go
a few slides down the line. The point is after we go through a bunch of
this, there\'s an extremely strong growth. It shows no sign of topping
out anytime soon, of applications of spatial data through GIS in
industry and government at all levels. In particular, in the last few
years, there\'s grown a strong and continuously growing corporate
federal emphasis on geospatial data, which was the point of having you
look at a lot of the materials that I had links to in the pre-meeting
notices.

The question is where is EIA in all of this right now, and the answer to
that is that we\'re basically still in diapers. We have exactly, we\'ve
got what, about 300 employees give or take. We have exactly three who
are competent GIS practitioners at the present time. That\'s about one
percent. This is very unlike the situation a decade or slightly less ago
when EIA was three to four years ahead of the rest of the federal
government and most everybody else in moving to distributing and
computing on PCs and use of the Internet. One sort of wonders why this
is the case. Is it the data that we work with or is it that we\'re not
up with what\'s technologically happening and we don\'t have the vision
to see it?

I think it\'s more the latter. In terms of our data, the question is how
geocode-able is EIA\'s energy information? We haven\'t been collecting
latitude and longitude on most of our forms. Some offices are starting
to move in that direction independently of their own volition. But in
the meantime, there\'s a lot of our data, in fact most of it, that can
be geocoded in one manner or another and we\'re not doing it yet. If
we\'re going to be where I think things are going to be ten years from
now, every bit of it will have to be geocoded.

You can always look up from an address location, any standard reference
frame that you really want to. You may not get a precise location but
you\'ll get a centroid of something that\'s close and the conversion
capabilities are being improved dramatically day by day. There\'s lots
of geocoding software available. Most of it is based, these days based
on the TIGER file, which grew out of the old DIME file, which grew out
of censuses.

There are things being done now with respect to homeland security needs
which are going to make it a lot more precise within a relatively short
period of time in terms of location. The TIGER files actually aren\'t
precise enough for some applications. And, here\'s an example, one
manufacturer\'s indication of what can be done with their geocoding
software. It\'s quite powerful. You don\'t get 100 percent hit rates on
this stuff. There are always some cases that are messed up because
somebody misspelled something or whatever and those have to be handled
manually. But, a lot of it\'s automated and it goes very, very quickly.
The software isn\'t expensive.

DR. CRAWFORD: I guess we know, like how to geocode, but what I don\'t
understand is how are you going to use this information because you
can\'t release it. I mean it\'s going to be very, very confidential.

MR. MOREHOUSE: Can\'t release what?

DR. CRAWFORD: Your geocoded.

MR. MOREHOUSE: Sure, you can.

DR. CRAWFORD: Not for some things you can\'t.

MR. MOREHOUSE: For some things you can\'t but most of them you can.

DR. CRAWFORD: For most things I think you can\'t.

MR. MOREHOUSE: Most of it you can. For instance, there is no problem
with plotting up things like, for instance, to just pull something out
of thin air, the service area of a natural gas distributor, and then
there\'s a bunch of data that you can lay on top of that, okay. There\'s
just a tremendous amount of our data that you can put in that way.
There\'s also a lot of things you can do in terms of spatial analysis
data where you don\'t have to show the data itself.

DR. CRAWFORD: I think that\'s what we\'re kind of curious about.

MR. MOREHOUSE: Yes, well we\'ll get it. Essentially, that\'s the last
three slides here or so are about, okay. We sat around, Nancy and I sat
around, it must have been six months ago when there was a chance to
apply for some over-budget funding which never came through for this,
but sat around trying to figure out what kind of a proposal to make, and
that\'s essentially what\'s in these last three slides.

I\'m convinced that ten years from now, most people are going to get
most of their information off the net, by clicking on a map. And, that
will take you down through a bunch of layers, allow you to branch off to
any kind of databases they have around, that sort of thing. It\'s
already happening and it\'s particularly happening big-time over in the
classified side of things and that\'s all going to seep out into the
private sector real quick.

So, essentially for starters I suggested we get somebody on each
office\'s staff mobilized to start looking at their data and
collectively, as a group, look at EIA\'s data and figure out what\'s
doable and when it\'s doable and that sort of thing. So, essentially the
person would serve on a GIS coordination working group or you could call
it a spatial data one. Actually, I would prefer that, the more I look at
it.

Secondly, to be the point person within that office for making sure the
data gets geocoded that can be and there are plans for that in place
and, to act as an advocate, if you will, for the use of spatial data by
GIS in many cases in their products, their reports. The other problem,
of course, we have is compliance with Circular A-16, which several of
our GIS products that we use internally right now, at least, are not
compliant with. They were never really intended for public use in some
cases. They\'re getting some public use and they ought to be standards
compliant particularly with respect to metadata.

Secondly, we need some kind of seed funding and that\'s not going to be
a whole lot of money, I don\'t think, to essentially go through and
assess our future geospatial data needs and hardware and software needs
to develop guidance for use of this technology that makes us compliant
with OMB and FGDC policy and guidelines to look at the geocoding tools
that are available and that are coming onto the market and going to be
coming onto the market and figure out how we want to go on that
department so we don\'t have people using different tools across the
agency.

And, one of the key areas, is to provide training. I\'m sure most
offices will be able to send their own people to GIS training, but in
terms of metadata training, that\'s a whole different matter. We are
working at the CIO level to figure out a way to do that department wide,
in fact.

And third, to start, these are starting ideas I\'ve got up here, to
start putting some funds into a geospatially based energy assessment
program, which looks at particularly the National Energy Atlas as a
project which annually will report on things like county level patterns
of energy production and consumption. This is something we get constant
questions on all the time and we also should probably be looking at
things that are trend related.

And, what I came up with for this was to study the patterns of urban
development versus the requirement for energy. We\'re getting more and
more and more urban or at least suburban and energy use patterns are
going to be changing. They\'ll also change as we shift around between
our industrial and service business bases.

There\'s a lot of things that you can do in GIS that you just simply
can\'t do on a statistical package and you can\'t do it in text, and
that\'s where I think the whole Web is headed.

This by the way is a kind of an interesting little gadget. It\'s a
Compaq Ipaq Pocket PC running ESRI\'s ArcPad software. It\'s selling
like hot cakes right now. You can get one of these for not all that much
money. It provides database access, wireless if you wish, mapping, GIS,
and GPS integration in the field, vector and raster images are
supported, that sort of stuff.

I bet dollars to doughnuts that Guy would like to have this little gem
in his pocket the next time he\'s out in the boonies giving a speech
somewhere and somebody asks him a question that he has to answer with
like about a second, third, or fourth tier statistic that happens to be
spatially discoverable on the EIA web site. I know I\'d like to have one
when I get out there. So, that\'s where the world is headed and the
question is, how does EIA get there? I think you people could probably
give us a lot of advice on that and, in particular, how to get our
statistics in shape and what we might want to do with them.

I\'m hoping that this is the start of a dialogue on the subject and not
just one shot. I\'m not going to come in here and do the golly gee whiz
stuff up on the screen with GIS. If you want that, I\'ll get somebody
from ESRI shop here out in Virginia who can knock your socks off.

DR. CRAWFORD: Thank you. Our ASA discussant is Mark Bernstein.

DR. BERNSTEIN: I did my homework.

MR. MOREHOUSE: Good.

DR. BERNSTEIN: I read all those documents.

MR. MOREHOUSE: It\'s sort of a pain but it does give you a feel for it.

DR. BERNSTEIN: Having been working in government, I understood the
documents, which was really frightening to me.

MR. MOREHOUSE: That\'s frightening.

DR. BERNSTEIN: That doesn\'t mean I thought they were useful, but I
could understand them. And, having spent the last year on a project
being very frustrated with lack of spatial data, or let\'s put it this
way there are \-- 99 percent of the U.S. is spatially mapped for soil
data. There are three gaps. One of them is the study area we were doing,
so just had to vent out that frustration.

Anyway, I think that it\'s very important for EIA, and I think you\'re a
little bit putting the cart before the horse in some sense, and I think
you need to do a strategic assessment before you begin to lay out some
of the issues, particularly when it comes to the hardware/software
issue, though it could go in parallel. I think you need to, and maybe
you\'ve done this but you haven\'t shown it to us. What spatially
referenced limited data exists now? What spatially referenced limited
data would be useful in the near term? So, strategically say okay, you
know, if we we\'re going to pick the high-priority things in the
near-term, what is it that we ought to have spatially?

I agree that data without some spatial acknowledgment is not very useful
but I\'m not sure it is the most important piece. It depends on what
type of analysis you\'re doing. And then, once you get to the point of
knowing, okay, here is the spatially referenced data that we need, how
do you get it? And then, the next question is, okay, what do you think
you\'re going to need in the future?

Okay, I think you should differentiate between what you should really
have now, prioritize it, and what would be nice to have in the future,
and lay out a plan to get there. Then, you\'ll have sort of a good sense
of what it\'s going to take and where you need to be. I think you need
to focus on the data and not the maps now.

MR. MOREHOUSE: I agree.

DR. BERNSTEIN: Maps will come after you have the data. So, I wouldn\'t
worry about getting maps into the AEO anytime soon until you sort of
figure out what you really want. Then, the other question that you face
is, you know, how much do you develop in-house capability and how much
do you farm out? How much do you try to cooperate with other agencies?

You are in the oil and gas area. You know the EPCO stuff and so you\'ve
got that capability and that data in interior. How much do you use of
that and how much do you, you know bring in-house? Those are just
questions you need to ask.

And then, on your last slide, your third, I think these are really good
goals. I think as you\'re looking at the strategic assessment, you know,
there is a goal out there in the future which is to establish a National
Energy Atlas. You should spend some time defining what that would be or
mocking it up and that\'s a good goal to strive for. It will help you
sort of structure the questions here.

I do have a question on the choice of county level data, you know: a)
why; 2) how; and 3) feasibility. You know county level information, you
can get Census county level stuff but it wouldn\'t be able to really get
energy. I mean what\'s it really going to take to get energy data at the
county level? Is it feasible to take state information and, you know,
estimate county level? It may be possible. Some people actually do that.
They do that in California. But, the difference in California is they
spend a lot of money on data stuff and most states don\'t.

So, you really, before sort of choosing county level data, you really
need to assess sort of what do we really needand where do we go? I mean
it would be nice. It would be really great for that next question, which
is the relation to spatial, relationships between land and I wouldn\'t
say urban development. I would say between land use and agency. It\'s an
issue that comes up particularly in the environmental assessments.

You know, we\'re struggling now with a study on ozone precursors into
Sequoia National Park, and you know there is no spatial dimension that
we can use at the moment to where the precursors come from and how it\'s
transported. We actually have to create it. And, that would be really
great to have sort of this relationship between the energy and where it
exists, but then you\'ve, of course, got the issue of transportation and
it goes across boundaries. It would be nice to get there and that\'s
another great goal. I mean to be able to answer that question sometime
in the future would be really nice. Have a date when you want to get
there and try to build the data up to that.

MR. MOREHOUSE: Yes.

DR. BERNSTEIN: So, I think this needs a carefully thought out strategic
planning process. A lot the stuff in the other agencies have not been
thought out that well and they\'re a mess. I mean, yes, they\'re ahead
of you, but they are a mess. They don\'t know what they have. They
don\'t what they don\'t have and you go and you call somebody and say
well, I thought we had that.

So, just really think it out, you know, do a real strategic process here
because it is important. Data is important and you can either spend a
lot of money and waste a lot or you can sort of strategically plan it
out and do it within the budgetary confines that you have, but also try
to get as much sort of cooperation as you can with the other agencies
because there is data in other agencies. The other agencies need data
that you might have, EPA, the interior, and USDA, and work it that way.

DR. CRAWFORD: I just want to add a little bit to what Mark said, and I
think that what I would stress from what I\'ve seen is that before you
even think about, you know, what data you\'re going to collect, you
think about what you\'re going to do with it and particularly with the
software. What analysis are you going to do? Because, a lot of times, I
see people spending an extreme amount of money on all of ESRI\'s
packages and extensions and their analysis capabilities for the types of
things you might want to know are limited.

So, if all you\'re going to do is make maps and you want some, you know,
interactive, Internet mapping stuff, then GIS is really overkill for
that type of application. So, I guess while I would consider
alternatives to ESRI\'s products, some of which are free and very nice
and might be more suitable to the types of work that EIA is going to do.

MR. MOREHOUSE: Okay, let me handle that one first. I know right were
coming from on this, but I don\'t just use ESRIs. I\'ve used MapInfo.
I\'ve used a lot of the other products, okay. I\'ve also used some of
the staff packages that claim to have GIS functions in them.

In fact, about a year ago, I managed to get a hold of some senior people
at the SAS Institute, and similarly some senior people at ESRI and
started banging heads together because there was no way to import
anything from a SAS database into ESRI\'s GIS system. I did the same
thing with MapInfo, okay. They are now working to make it possible to
marry those packages together so you\'ll have the best of both worlds.

DR. CRAWFORD: Well, they\'re not doing it on the analysis point of view.
You can read a SAS-B

MR. MOREHOUSE: I understand that but from the analysis point of view,
that\'s the user function, not theirs, okay. Mark, I very much
appreciate your comments. They were right on, all of them. In terms of
holdings, we do know what we have and it\'s not much quite frankly.

In terms of near-term needs versus long-term needs, definitely that has
to be strategically planned out. In terms of near-term needs, the number
one item is, from my point of view at any rate, is the information on
critical infrastructure, energy infrastructure which is we\'ve got some
of it is in the GIS. It\'s not metadata compliant. Things like that have
to be worked on. Longer-term, the sky is the limit.

In terms of working in or out of the house, that\'s also important. The
county level thing, these were intended as big long-term goals and, yes,
it\'s difficult. But for instance, in the database that we have our
critical infrastructure information, and we have an awful lot at the
county level information in addition to that already, but you will have
a problem with imputing the counties from state level data in some
cases.

In terms of interagency cooperation, that is the essence of what the
FGDC does. It\'s the essence of the geospatial one-stop and so forth.
And, of course, we would be working with other agencies in pulling their
data when we can use it and vice versa. That\'s the way the government
is working now for spatial data at least. So, I agree with you, we have
to sit down and get a strategic plan in place that is well thought out
and that hasn\'t been done yet and that\'s why I\'m here.

DR. CRAWFORD: Johnny.

MR. BLAIR: Yes, I just wanted to make a couple of comments partly in the
context of the strategic plan but also in relation to some of the other
things that were mentioned, and that is, it didn\'t seem to me that
there was enough emphasis, you know, maybe there is and I wasn\'t
hearing it, on the user perspective.

It seems to me that in any new technology, you have kind of early
adopters and then you have people that kind of come along later, and
it\'s when you start to get that second wave, that you get the demand
and the money and these other kinds of things. And secondly, it is
obvious from what you\'re saying that there is a huge amount to be done
and if all you\'ve got are these three folks in the agency, you know,
you\'ve got to get other people on board.

But, as far as setting priorities in deciding sort of what are the
things that need to be done sooner rather than later, I think both in
terms of within the government and users outside of the government, you
know and I\'m not sure exactly how one goes about doing this, but there
needs to be it seems to me more of a perspective of what is it that the
users need? What are the things that are going to find the most use, and
that\'s where the priorities should go in.

I think that needs to be an important part of any strategic planning,
given that you can\'t do everything at once and this is going to happen
over years. Who are the people that you need to bring on board early?
What is it that they need? Who are the ones that are then going to be
the second wave of users that some sort of analysis or, at least,
consideration needs to be given to who is the target audience for all of
this and not simply that we want to do all of this because it can be
done and because it makes sense in terms of some analysis.

But, they\'re going to be people who are doing the analysis and what is
it that they want? What are the needs that are not being met now that
can be met or things that are being done now that can be done better by
this kind of technology and these kinds of data?

MR. MOREHOUSE: Yes, I would argue that even just plain data discovery,
aside from anything that has to do with analysis, this is a better way
to go. I mean I have trouble finding stuff on our own website as well as
I know it. Okay? If I were able to click on a map and then hone down to
where I\'d be able to get it a lot more effectively. Sometimes, I
actually have to call up, know who does it and call on the phone to find
out where the stuff is and that shouldn\'t happen.

DR. CRAWFORD: I think you can also tie this initiative to some of your
goals in education and particularly the Kid\'s Page, and you can do
interactive mapping.

MR. MOREHOUSE: Oh, sure.

DR. CRAWFORD: You know, put a data set there that you know respects the
data confidentiality issues and ask the kids questions and make them
click on the map and see relationships, you know, which state, you know,
has the most of a certain type of coal or something and then that state
can be highlighted and they can link things that way. That might be a
nice tool for them to use.

MR. MOREHOUSE: Yes.

ADMINISTRATOR CARUSO: Congressmen, too.

DR. BERNSTEIN: Congressional district rather than county level. They\'ll
give you the money.

MR. MOREHOUSE: Actually, there\'s a lobby building up there for this
right now.

DR. CRAWFORD: Any other questions, or comments?

MR. MOREHOUSE: Anything more? I appreciate your attention, and look
forward to working with you on things to do with this in the future and
if it\'s not me it will be somebody else I\'m sure.

DR. CRAWFORD: So, we\'re still behind schedule, but maybe we\'ll take a
ten-minute break and reconvene about 10:30.

(Whereupon, the above-entitled matter went off the record.)

DR. CRAWFORD: I\'d like to go ahead and reconvene because I can\'t wait
any longer to hear what Calvin is going to say. He\'s been compiling a
history of this committee from its inception to now and turn it over to
him.

DR. KENT: Am I on?

DR. CRAWFORD: Yes, you are.

DR. KENT: Well, I really appreciated the invitation to be able to come
back and meet with you. I couldn\'t help but remember Shakespeare\'s
play \"MacBeth\", because I think I\'m somewhat like Banquo\'s Ghost. I
keep reappearing and reappearing and reappearing at the most
inconvenient times.

I just happened today when we give it up and I probably ought to take my
cue from the former vice President and just admit it\'s time for me to
go, but I\'m not quite there yet and I did appreciate, well it was about
a year ago when Carol asked me to prepare something for new incoming
members of the committee, that they could use as a reference point so
that they would know what the committee did, so that they would have
some feeling for what the committee had accomplished and what their role
would be.

And so, from that very humble and simple suggestion, this grew into
about a year\'s project on my part and I did have the opportunity
because Bill was kind enough to get them for me out of the national
archives, all of the papers that have ever been presented since the
first meeting of this committee back in 1979. And, I actually, believe
it or not, did go through and read almost all of those. Some of them
were unintelligible but that was due to my limitations and certainly not
to the presenters.

And so, it was really fascinating for me to be able to go back and
reconstruct. Also, I\'ve been attending meetings of this committee since
1989 when I was first nominated to be the administrator for EIA, and I
was just looking back in the period 1989 through the current time. I\'ve
been at over 80 percent of the meetings of this committee in one
capacity or another. So, perhaps I was the appropriate person to do the
chronology and to evaluate what this committee has done.

Just to highlight some of the things that are in the paper, because I
know that we are running behind time, the first thing is that the EIA
was created as part of the Department of Energy when the Department of
Energy was established in 1977. And, EIA brought together the work of
over 50 other statistical agencies that were located in various parts
and various branches of government. And, as I note in the paper, there
were 240 separate databases that EIA had to somehow or another pull
together and amalgamate, databases and models I should say.

And so, there is a very quick statement of what the mission was of EIA
as it was established in 1977. And, I think that EIA has basically
adhered to what their mission statement was. It certainly has gone
without substantial modification, but it has certainly gone with
significant expansion as to what that definition includes.

And then, I also ought to point out that the committee itself was
established under the Federal Advisory Committee Act of 1972, which is
the one that prescribes the framework for all similar such committees. I
think there is over 40 of them that I was able to identify that are
similar to this particular committee that are performing this type of
evaluation and analysis.

And, the specific responsibilities that were assigned to this committee
and that have remained as the responsibilities of this committee are
those three that we have put up there; and that is, to do the periodic
review of the elements of the energy information administration data
collection and analysis programs and the provision of recommendations.

Secondly, advice on priorities of technical and methodological issues
and planning operation and review of energy information administration
statistical programs, and then advice on matters concerning improved
energy, modeling, forecasting tools, particularly regarding their
functioning, relevancy, and results.

It would have been much easier just for the act to have said to look at
everything and discuss everything that EIA does. Now, when I left the
Office of Administrator, right before I did that, and one of the nice
things, one of the few nice things about the election in November in
which George Bush lost his job and I was to lose mine, was that after
November, between November and January when I was still administrator,
nobody paid any attention to me. The phone didn\'t ring. People didn\'t
call up wanting to go to lunch with me, you know. Nobody was offering me
to come to baseball games or football games or anything like that
anymore.

And, EIA people continued to be very nice to me in the same way that you
treat a dying relative, I think. But, be that as it may, it gave me a
good opportunity to go back and to ask myself over the history of EIA
what were the major issues, and I did publish this, that had
characterized EIA and what it did. And, I basically found that there
were six, and these certainly are not distinct, but they were the six
major areas of concern to EIA, and so I used these six areas as a way of
organizing the presentation, the paper that I\'m sure you all read if
you have trouble going to sleep at night.

And, the first one of these is the issue of data quality, which by and
large has been the major concern of this committee. We have had more
sessions, in fact 42 percent of the work that has been done by this
committee has wound up being associated with data quality and these are
the specific issues. Again, I\'m not sure that these are all distinct
because they do significantly overlap, but these are the major problems
in the area of data quality and I\'ve just written down the number of
items that came under each one of the headings, and if they don\'t add
up it\'s because I sometimes double counted if a particular presentation
was in more than one of these areas.

But, what it basically tells you is that we have been heavily, we as a
committee have been very heavily involved with the work of EIA in
improving data quality, which of course is at it should be because if
you don\'t have quality data, then nothing else really matters because
nothing else that you do is going to be of quality either.

The second area was the role of modeling, and actually energy models
have been around, I think we traced them all the way back to the PI
system back in 1974. It was the first identifiable energy model that I
could find, and that one, of course, was one that was inherited from the
old Federal Energy Administration.

But, so EIA has been involved in modeling even though there were various
levels of enthusiasm for modeling in EIA, and certainly when I came to
be administrator there was not a great deal of enthusiasm for modeling
at that time. It was, I should say more tolerated, I think, than
encouraged.

But, as I noted in the paper, the administration, the first Bush
administration, was very serious about the development of a national
energy modeling system. It made it clear to me even before I was
nominated that my principal responsibility was to make sure that a NEMS
was developed and that the modeling capability of EIA was significantly
improved, and if that did not happen, then that function would be
transferred to another part of the Department of Energy.

So, faced with the reality of that threat, we got very heavily involved
in the modeling process. And the committee of course has been very
heavily involved and you\'ve even had presentations at this meeting
concerning the SAGE modeling system. And, the committee I think has well
fulfilled its responsibility here, although from the survey that I took
and some of the comments that I received, I think there is generally not
as much happiness among EIA people with the contributions of the
committee in the modeling area as they are in the other areas with which
the committee has concerned itself.

The next one, of course, is the problem of resources and
responsibilities. EIA has had a very uncertain fiscal future, and I
think I have a table in there that I\'ve given to you and maybe the
graphic, I thought I had a chart made of it somewhere in here about what
our budget responsibilities actually were. I do have one here.

You can see the budget appropriations, just to prove to everybody I do
know how to draw a graph, which some people are not sure about, and I
don\'t. This was done for me by my graduate student, but it does show
you something about the budget appropriation process that immediately
following EIA\'s budget, went very high and then it began to drop and,
except for a little upward blip here, and we won\'t mention who was
administrator during that period of time, EIA\'s budget has consistently
up until just recently in real terms not improved significantly.

And at the same time, there has also been the problem of declining
employment. I think I\'ve got a graphic on that one as well and you can
see again the same pattern, although in the last few years EIA\'s
employment has stabilized at about 375. But, it\'s clearly less than
half of what it was at its peak time. And, at the same time, there has
been a tremendous increase in the responsibilities that have been given
to EIA.

And, if I can get that back up here, I\'ll do it. There have been
terrific pressures from the executive and the legislative branches both.
They want more data. They want more analysis and they want it faster.
Other than that, they have never really given any requests. There have
been two very significant pieces of legislation. The first one to
mention was the Clean Air Act Amendments of 1990, which brought EIA, for
the first time, really into the environmental area and I don\'t think
anybody regularly recognized it at the time, but it was a tremendous
coup because it simply meant that most of the energy environmental
analysis was going to be done in the Department of Energy rather than by
the EPA. And as a result of the Clean Air Act, and that was by the way a
very conscious decision on the part of the administration that that\'s
where they wanted that particular analysis to be done because they
thought it would be more objective, meaning more biased in the direction
that they wished it to go.

The second one was the Energy Policy Act, and as I\'ve already indicated
to you, I was the administrator. I was actually on the team. I\'m sorry
Mary Hutzler isn\'t here because she was part of the team too that was
kind enough to EIA to give these six additional responsibilities and
this was why we got the uptick in the funding that took place, very
short-term uptick in the funding, to meet those six new goals that we
were given or six expanded goals that we were given under the EPACT of
1992, some of which we are still dealing with today as you well know.

The next comment would be that in the area of EIA\'s priorities, as you
recall those of you who were back on the committee in 1999, EIA came to
us and said we\'re trying to prioritize our work. We\'ve got to figure
out how are we going to deal with all of these requests, which are
inundating us and here is where we feel our priorities should be. And,
if you remember, we spent almost a full day back in 1999 working with
them on what their priorities should be, and this basically shows you
the list that resulted from that. And, to my knowledge pretty much, and
the EIA people here can correct me if I\'m wrong, the EIA has pretty
well stuck with that priority list.

The significant thing, of course, is that they have attempted to limit
new areas of data collection having, of course, enough problems with the
data that they were having to collect under the EPACT Act and are still
having difficulty in collecting. An area that was a particular concern
of mine that has continued throughout the years and is hopefully going
to reach a resolution in the Confidentiality Information Protection and
Statistical Efficiency Act of 2002, has been the area of confidentiality
of data.

When EIA began, most of its data series were clearly used for both
statistical and regulatory purposes. Certainly, under the old Federal
Energy Administration, most of the data was being collected not for
analysis but was being collected for regulatory purposes because the
energy industry was so highly regulated at that time. And, it became
very quickly obvious that because of the fact that it was principally
being collected for regulatory purposes, that there were significant
problems with respondents and EIA has been trying to for the last 20
years get rid of that legacy.

And this, of course, has been compounded by the insistence of the
Department of Justice that the Department of Justice has access to
information which any information that EIA has and it was clear at least
when I was administrator, and also clear at other times since then, that
the Justice Department wanted to use EIA data for prosecutorial
purposes.

And as I\'ve detailed in the paper, there have been attempts in the past
to protect EIA\'s data to try to give it the same mantle of security
that census data has and so far those have not been successful;
although, I did not want to detail it in the paper because hopefully it
will be published somewhere. EIA has been very creative in the ways of
making sure that its information did not fall into the hands of the
Department of Justice or other agencies that would have used it for
various nefarious purposes.

But there is now the Confidential Information Protection and Statistical
Efficiency Act of 2002, which would go a very long way in terms of
establishing the ability of EIA to collect data and to use that data
solely for statistical purposes. And, you might at sometime in the
future, if that legislation has a prayer of passage and I understand
that it does, to discuss that in more detail with the committee.

The next thing just to mention is the independence of EIA. This is
something that EIA has valued very highly but it\'s very, very difficult
to maintain. I was talking to Jay Hakes and also had a very fascinating
discussion with Lincoln Moses, who you old-timers remember was the first
administrator of EIA. He\'s still strong and going great. He calls me
and everything like that.

And, he pointed out that this was the original problem was that if you
are serving two masters, if you are serving both Congress and the
administration and they have different objectives, obviously different
political agendas, then it becomes extremely difficult to maintain your
independence, and this has, throughout the history of EIA, been most
acute when there have been spikes in fuel prices such as heating oil and
so forth on a couple of periods, and most recently the electrical
problems out in California.

How do you maintain your objectivity, how do you maintain your
independence? And, of course, one of the major issues, which EIA has my
undying compliments, is how they have been able to handle through
different administrations the questions related to the Kyoto Protocol
and the emissions issue.

When your were handed this political hot potato, and certainly there
wasn\'t anything hotter in the environmental area and certainly right
now it has continued to be more than warm, how did EIA manage to
maintain its independence, maintain its credibility, be the source that
both sides quote in their debate, and I certainly think they have done
an outstanding job on it.

And, I was particularly appreciative of the comments that Jim had when
he was talking about the voluntary reporting system of making sure that
you don\'t over sell your data because that\'s been extremely important
with EIA to be able to maintain its independence. But, EIA has done a
great job of doing this. It is not something that is easy to maintain
because you don\'t want to be viewed as the \"kept woman\" of either
side in a political debate.

One that also I separated out, because it required so much attention, is
the continuing debate between timeliness and accuracy. People always
want to have results very quickly. They want to have data in realtime,
and particularly when faced with emergencies, beginning with the coal
strikes back in the 1970s, through the energy crises, the Gulf War, the
current situation, pressure has always been placed upon EIA to get the
information out there as quickly as possible.

And, of course, the committee itself, they evaluated the Energy
Situational Analysis Report, a couple of meetings ago, which is now
being issued on a bi-weekly basis. We began it during the Gulf War
specifically to try to get information out to calm the markets and to
deal with political issues and to dampen political issues that were
being fanned by rumors and other false information that was out there.

But, EIA has done a great job, particularly through their web site in
getting the material out to individuals as quickly as possible and I
again think that this is an area in which the committee has been
instrumental in providing both advice and guidance to EIA and has made
EIA more effective.

The last thing then that I wanted to talk about was the evaluation of
committee effectiveness, which came from this questionnaire, and what we
proved by this questionnaire was something that all of you have known
and that is I am no survey statistician. And, other than that, we
started off with 60 potential names. We could only find usable
addresses, ways to contact 40 of them. Twenty of them responded, so 13
of them were committee members, including some of you that are sitting
out here, seven of them were EIA people, and I should point out that for
every living administrator responded to this survey, and so that was
very interesting to get all of their things.

Most of them were of their opinions. Most of them, the respondents,
indicated that they did not want to have their names used. Many of them
did not send us their names, did not want to be identified which
surprised me because overall they were very complimentary of EIA and its
work. And, these are of course, the questions that were asked, the first
five, and there were two additional questions, and by and large, as you
can see from the paper, the committee was extremely supportive and EIA
was extremely supportive of the committee.

There were some suggestions that were made by people that were repeated
with sufficient frequency and in some cases with sufficient vehemency
that I think it is well for EIA to take a look at them, particularly the
issue of consistently high quality presentations by EIA staff. There is
a strong feeling that these were very variable in terms of their quality
and that interestingly enough was a comment that was being made by
people who were on the committee back in the 1970s and early 1980s. So,
that evidently has not changed. And, the other one was to try to get the
information out a little bit earlier to the committee.

So, if I was to reach my conclusions, my conclusions were very simple
and that is that the committee has fulfilled its responsibility under
the Advisory Committee Act. It has done what it is set up to do, that
EIA has a low-cost pool of experts which they have used very
effectively. That\'s the report.

DR. CRAWFORD: Thank you, Calvin. Are there any comments or questions
from the committee or from EIA?

DR. KENT: I would appreciate any comments that anybody has on this
paper, any corrections, additions, things that got overlooked or
anything like that because I really hope to do something else with it.

MR. HENGARTNER: How does this committee compare to other committees?

DR. KENT: I did not do anything with the other committees. Many of the
other advisory committees are not statistical advisory committees. I
think there are 11, and maybe a wrong about that, aren\'t there are 11
other statistical advisory committees associated with the Census and
Agriculture and so forth? I did not look at any of their work
specifically.

DR. CRAWFORD: Jim.

DR. HAMMITT: When I saw you had 20 respondents, I thought that was also
further evidence of a thing EIA knows about how hard it is to get people
to respond to surveys.

DR. KENT: That\'s right.

DR. HAMMITT: But when I realized you only surveyed 40, it wasn\'t quite
as bad as I initially thought. But your point about independence and two
masters, I would think it\'s a lot easier to maintain independence if
you have two masters than if you have one.

DR. KENT: You know that\'s an excellent point. That really is, that it
probably, because everybody is checking everybody else. If you never
went up on the Hill, and I don\'t know if Guy is having this experience
but I certainly know that the other administrators did, to meet with any
congressional group, that they didn\'t always slip it in there somehow
or another to remind you of their responsibility.

And, I assume that EIA\'s budget is still determined separately from the
total DOE budget, is it not? Yes. EIA\'s budget has never been part of
the consolidated. I mean it is when the budget document itself comes
out, but the committee that handles us separates us out from the overall
DOE budget just for that reason, to make sure that the independence of
the organization is maintained.

DR. HAMMITT: I\'d like to ask one other thing. You said, I guess you
thought EIA staff and the committee had not been as useful on the
modeling side.

DR. KENT: I sort of got that impression that the replies that we got and
it wasn\'t so much in the statistic, it was more in the comments that we
received, were that they felt that the committee had been more effective
in survey work and more effective in data quality. But that really
didn\'t surprise me because what little expertise I have tends to be in
the modeling area, which was basically the reason I think I was brought
on as administrator to begin with over maybe someone else who has had a
different background because I thought the committee had been very
effective in terms of providing input in this area and it was a minor
comment.

DR. HAMMIT: Do you see reasons or has the committee always had a
reasonable representation of people with modeling expertise?

DR. KENT: I think so. The real place that this committee got most
heavily involved with modeling was with the NEMS, and then the committee
really took so far as the composition of membership was concerned a
decided turn towards including more modelers at that time because that
was what was being demanded. And so, we actually went out and actively
recruited modelers to come on to this committee.

And, prior to that, there had always been some with that orientation but
it became increasingly more important as it was obvious that this was
where the pressure was coming was for EIA to build better and more
responsive, any national, an integrated national energy modeling system
when that was the demand and that was where the money was going. Yes,
but I think that at least since then there has been a sufficient
representation of people who were modelers and I don\'t want to over
emphasize that conclusion because I didn\'t even write it in the paper
because I didn\'t think it was that strong. Yes?

DR. MOSS: You had said of respondents from EIA, you said all of the
former administrators responded. How many does that leave for other
levels of EIA?

DR. KENT: Well, some people like me who responded were both but we had
five of the seven EIA respondents that we used were former
administrators.

DR. MOSS: I guess my question is whether more could\'ve been learned, in
terms of our benefit to EIA, if you surveyed people further down the
working chain?

DR. KENT: I think that would\'ve been a much better way of having done
it, to have actually gone back to the people who headed the various
offices and so forth and have asked them.

DR. KIRKENDALL: Mary\'s input for example, not to identify a respondent
or anything, she asked us for input so there is input in her answers
from me and from various and sundry other people in my office at least.

DR. KENT: Mary gave us a very extensive list of comments.

DR, KIRKENDALL: So, there\'s more in that than just one person because
she got input from others.

DR. KENT: That\'s what I have for you all.

DR. CRAWFORD: What publication plans do you have? What kinds of
journals?

DR. KENT: Well, there\'s at least two journals that have said they would
take a look at it. One of them is the Government Information Quarterly,
which has published things from EIA frequently, and the other one is
Public Administration Review that are interested in this. And so, I\'m
going to work on it and I certainly would appreciate any comments and
anything else that anybody has on how we might improve on this and make
it an even more useful paper. But, I also hope that this serves as the
basis for future committee members at least can read it and get some
sort of a feeling for what goes on the committee meetings.

DR. CRAWFORD: And that\'s great. Thank you so much, Calvin. I guess we
move on now to business and discussion for the next meeting. The first
thing I\'d like to do is announce the date of the spring meeting, so
these are now confirmed for April 3^rd^ and 4^th^ of 2003. The next
thing I\'d like to do is turn it over to Nancy for little bit so she can
talk to you about the Research Fellows Program that EIA has, and I have
some handouts from her.

DR. KIRKENDALL: Okay. What Carol is handing out is our draft of our
solicitation for research fellows to help EIA. What we had originally,
in fact, I think this idea originally came up in talking with the
committee. But, when we first announced this a couple years ago, we were
thinking that we would use it to bring people to EIA to work for three
months in the summer because that was about all we could afford.

But, since then, last year Randy\'s student, we were able to fund her
through this fellowship program as an alternative, and I think that\'s
working out really well to. So, it\'s certainly a great alternative to
bringing somebody here. Why not give a student some funding to work on a
project that\'s of interest to us at their own institution? And, in
fact, that\'s cheaper for us and we can do more of them as we add more
from that area.

So, I\'m not sure our description right now makes it clear that we would
like to make that as a clear alternative that you don\'t have to come to
Washington, although it may be that we would like to include in the
funding at least a trip to Washington or maybe to talk to us about
what\'s being done, maybe coming to this committee to talk about if
it\'s a project that would be of interest everybody.

The descriptions are not yet final. I have a bunch of edits that I\'m
going to make to them. If you have ideas for additional projects that
should be inserted or ways that they could be described better, I would
welcome your input. Those of you who are at universities and have
students, I would encourage you to think about whether you have a
project, if there\'s a project that you\'ve seen that you think that we
might be interested in or some way we can collaborate on these things
because I think that that would be a great way to B we\'re actually
leveraging the committee in another way that way for a mutually
beneficial relationship.

I\'m not sure all these project descriptions are going to end up in the
final brochure; in particular, I didn\'t like all of the last three.
But, that\'s where we\'re still in the draft stage. Any other comments
or suggestions? Any other alternative formats would be welcome too, the
more clear we can be on alternative formats for getting these
fellowships.

DR. BERNSTEIN: The last chapter in the last paragraph of the application
information, is that all, it says: applicants must submit a detailed
research proposal. That\'s all it kind of says. I mean, are you putting
a page B you ought to put a page limit on it. You don\'t want a
sixty-page document. You don\'t want 50 page proposals.

DR. KIRKENDALL: No.

DR. BERNSTEIN: And so, you ought to make it clear, I think you ought to
really define what you want the proposal to look like. And I also think
you more want it to be, I mean wouldn\'t you rather have the opportunity
to work with a student to define the proposal after you\'ve chosen them?
I mean in some sense what you want is you want smart, intelligent
students who really want to get something done.

And so, you take an idea that they have, but then you spend, after
you\'ve said okay we want this person, then you work with them and
refine the proposal. That\'s kind of not what B I think actually getting
a detailed proposal from a student isn\'t as important as somebody who
knows something about the issue and wants to really work on the issues
and has got good recommendations from a faculty member or something and
then, you can work with them to define the project.

DR. KIRKENDALL: I agree with that, particularly when B I mean I think it
was difficult for Randy\'s student and Randy to come up with a proposal
that looked like something that was an application for the fellowship in
spite of the fact we pretty much already decided what she was going to
do and that she was going to get it.

DR. SITTER: Well, it\'s even worse in some sense because you\'re not
asking for anything. Does the student send a resume? Do they send
transcripts? Yes, I know but it doesn\'t say that here. We sort of had
to do this sort of over the phone what exactly do you want? I think I
did more than one draft of the proposal.

DR. BERNSTEIN: A lot of the internships or fellowships that exist for
the government are not really based on a topic. First, it\'s really
based on qualifications or interest and they do generally ask for a two
to three page thought piece on what you think you would do, but that\'s
not necessarily what gets done.

So, personally, you want the potential student to understand the types
of things you\'re interested in but I\'m not sure you really want to
base it on what they think they can do. I think you define with them and
their faculty adviser what they do.

DR. SITTER: Yes. I found it kind of awkward even as a faculty member
when you had it pre-composed to say, you know, I\'m going to come here
and work on this. If you said we have survey design issues and these are
some of the issues we have, it would be more appealing, to be honest,
than for you to have something specific.

DR. KIRKENDALL: Instead of having individual descriptions, something a
little bit more general?

DR. SITTER: I think so. I think for student or faculty or for anyone.

DR. WHITMORE: Something that looks more like a grant application than a
contract proposal.

DR. SITTER: Yes, a little bit more like that, yes.

DR. BERNSTEIN: And what you want to do is then match up the interest and
expertise of the students that you bring in with the topic areas that
you need, and that you do after you kind of know each other, before you
get to the summer of course. You don\'t want to do it after they\'ve
already started but you know after you\'ve chosen them, you say, okay,
we\'ll spend some time getting ready and get some suggestions back and
forth, I also like the idea of them not having to be here in Washington.

DR. KIRKENDALL: You say you like them not having to be?

DR. BERNSTEIN: Well, they don\'t have to you know. Sometimes they\'ll
want to be but I think sometimes yes it is cheaper not have to pay
somebody room and board for three months separate at some level. A visit
here would be good.

DR. KIRKENDALL: Yes maybe two, one at the beginning if you\'re going to
do this refining the project description, and then one of the end to go
over the results at least.

DR. BERNSTEIN: And, you know, good for them to be able to, you know,
maybe get them all, all the interns, you know, here at the same time at
the end, type of thing.

DR. MOSS: Are you doing only one of those?

DR. KIRKENDALL: We were doing only one but when we originally put it out
we were thinking that it would be the one where you bring them here to
work and that\'s more expensive. And so, we could only afford one a
year. But, if we do the one where it\'s more like a grant for somebody
to work at the university, we can probably afford more than one.

DR. KENT: Well, I would certainly recommend that you go that way for
several reasons. You\'ll get more applicants. The applicants usually
will feel more at home. They\'ll be closer to their academic advisor
with whom is probably going to be the one that\'s going to push them to
do it and it would be easier to pay for a couple of trips for them to
come here, and I would think that if you could get three for the price
of one that you ought to go in that direction.

DR. KIRKENDALL: So, you think maybe I should refocus it completely and
not mention the other alternative?

DR. KENT: Yes, I would.

DR. EDMONDS: Well, just as a counterpoint to that, I know from working
with students that unless they\'ve had some prior experience with you or
if their advisors had some prior experience, that they really need more
guidance, particularly if the product is to be integrated in with an
ongoing activity here. So, I\'d bear that in mind as well. I\'ve had
good experiences when I\'ve gotten someone in and worked closely with
them, but typically for students, they just need a lot of B they\'re
always having questions. They\'re always coming back and, if you\'re
right there, it\'s a pretty low cost transaction, but, if it\'s
separate\...

DR. SITTER: Yes, I have to agree. I\'d leave yourself the flexibility. I
mean if you\'re talking about somebody who\'s a student of somebody
who\'s on this committee, then there\'s no problem at all.

DR. KIRKENDALL: Right.

DR. SITTER: But, if you\'re talking about some applicant, a student from
somewhere and them going to stay at their university and work, I
certainly couldn\'t imagine. I mean if it was my student and I knew
nothing about this committee, I would tell them you\'re on your own. I
mean what are you going to do?

DR. CRAWFORD: Ruey-Pyng has a comment. Could you use the microphone,
please?

MR. LU: A couple months ago, I got a request from one graduate student
from Arizona State University. So, you know he\'s very much interested
in this, but I did not know anybody in that area, the connection with us
or even in the energy statistics.

DR. CRAWFORD: How did the student happen to become interested in your
work?

MR. LU: Yes. He is interested in statistics application to the energy
industry.

DR. CRAWFORD: So how much did he or she know about that already when
they contacted you?

MR. LU: Well, he is a second-year graduate student and worked in the
energy industry before.

DR. KIRKENDALL: So, he might be a good candidate really?

MR. LU: Yes, but the scene in this story, if there\'s no advisor in that
area or somehow we may have to bring him here. That could be the
alternative.

DR. CRAWFORD: Nicolas.

MR. HENGARTNER: Yes, I somehow sit between Jae and Calvin and have
somehow positioned between. I think the big advantage is having at the
university is that you get much more mileage out of the student because
it\'s not just the three months in this summer. If he really gets
hooked, he might work on it for his thesis topic, in which case they
actually get really something out of it.

Our experience with summer students at the lab is that it takes so much
energy just get them up to speed that at the end of the day you educate
them and if that\'s what you want to do, fine. But otherwise, getting
them to do actually something that\'s going to be useful is hard. I see
the advantage of having the student close by. One visit or two visits
might be nice. I think I\'d say get them out three weeks.

In three weeks, if they cannot understand the problem, then it\'s a lost
cause, you know, and it\'s probably somewhere in between that in finding
some flexibility, and I think you have a lot of flexibility that would
make this a very successful program. Get them out three weeks and then
let them come back a few times maybe for the end results, but keep them
at the university, because if they come back with a good problem to
their advisor, they might actually do something, right Randy?

DR. SITTER: She\'s worked on it for probably ten months.

DR. HENGARTNER: Right, instead of three, so.

DR. SITTER: Exactly and she probably will work on it. If we could get
microdata out of John afterwards, she\'s starting her Ph.D. with me. She
would subsequently work on it at times during her Ph.D. if we could get
microdata from John Woods that would be good.

DR. KIRKENDALL: John can\'t provide the microdata but the state of Texas
might.

DR. SITTER: Might, and that\'s one of the reasons I was asking.

DR. KIRKENDALL: Yes.

DR. SITTER: Not for current work but she\'s going to be spending three
or four years as a Ph.D. student with me. I think there is some
interesting research level work here that you may or may not be
interested in but she certainly would be and then you\'d have another
person who\'s interested in energy statistics and that never hurts.

DR. KIRKENDALL: And knows about data.

DR. CRAWFORD: Jim.

DR. HAMMITT: Yes, two comments, one on this here-away issue. If part of
the goal here is to recruit potential future staff, I think having them
come and be immersed would be very valuable, and also then they don\'t
have access to their home academic advisor potentially but they would
work closely with somebody here who is their advisor, and one example of
this is the RAND Student Summer Program which brings people in. Maybe
Mark or Polly knows more, but I think they sort of pick people who look
good who are interested.

People at RAND have to have a project they want a student to work on,
but then you just kind of pick what look like good students, get them
there, turn them loose on your project, and I think often they do get
something useful done. There is the issue in various areas. I think
that\'s much more into the culture, knowing people here, so I think
it\'s a lot better for recruiting potential staff.

DR. KIRKENDALL: We do have the intern program that we work, the Joint
Program on Survey Methodology. I think I mentioned yesterday that we had
six of them the summer.

DR. HAMMITT: Okay.

DR. KIRKENDALL: And, we\'ve managed to hire one since. It\'s been going
on for like three years.

DR. HAMMITT: Great. Well, then the other thing is a totally different
comment is, I\'d suggest reformatting this so it\'s like one page, which
is the general information and how to apply, and then the rest of it as
an attachment and sort of examples of interesting topics.

DR. CRAWFORD: Randy, did you have something?

DR. SITTER: One of the things was that is the term of this, is it
flexible or does it have to be four months?

DR. KIRKENDALL: No, it\'s totally flexible. It can be anything we want
it to be.

DR. SITTER: See in some sense talking of your internship, I mean we have
lots of students that would be valuable to you in an internship type
level but would not be eligible for your internship because they\'re not
American citizens. They wouldn\'t be employable to you right away
afterwards either. But, still it can be used in that way if you want a
student here for a longer period of time, so it has that flexibility.

DR. CRAWFORD: Ruey-Pyng did you have another comment?

MR. LU: Eventually, this brochure will be put on the ASA web site and
right now what I\'m trying to do is marketing this, you know brought to
the committee member. Hopefully we can solicit more applicants to
submitting their proposals. Thank you.

DR. MOSS: Are you doing this as a recruiting tool for both statisticians
and economists or do see this more as targeted towards statisticians?

DR. KIRKENDALL: No, economists too. In fact, the first person who came
for the summer was somebody who works at, in fact he\'s already a Ph.D.
I think, who works at the University of Maryland who came and worked on
the NEMS System on convergence issues related to NEMS. So, that\'s not
really statistics or whatever.

But, if it sounds like B if there\'s a good applicant, and his was just
by far the best application that year. It was a real easy choice. So,
we\'re certainly interested in any problems that solve, any applications
that sound like they will address problems and we\'ve got problems all
over the map.

DR. MOSS: It sounds like he had the best of both worlds though. He could
be close to his advisor.

DR. KIRKENDALL: Well, there\'s an advantage to being local that way, you
can work wherever you want and still come in whenever you want.

DR. HAMMITT: It obviously affects where you post this and how you
advertise.

DR. KIRKENDALL: That is another thing. I\'m not sure that we have gotten
that as widely distributed as we should.

DR. HAMMITT: There\'s a thing the economists, there\'s something called
JOE, Job Opportunities for Economists which is run by the American
Economic Association and it\'s online. I don\'t know if its monthly or
daily updates, and I\'m not quite sure if these things go in because
it\'s mostly jobs. But that would be a place to check because that\'s
where economics grad students look for jobs.

DR. CRAWFORD: There\'s a list of internships in statistics that comes
out in a timely way now. I forget if it\'s in a page in the AMSTAT News
now or something and I\'m not sure if this was on it. I wasn\'t paying
attention the last time but I know a lot of Masters level faculty look
to that because they want to place their students in an internship if
they know they\'re ending at their Masters Degree so they can get some
real world opportunities.

When I was on faculty at Nebraska, I mean almost all of our students did
an internship of some sort and we had trouble finding them. And so, now
it seems like they\'re more of them and more plentiful and this one
would have been a great one for them. I\'m sure you would\'ve gotten
some takers from our department had anyone there known about it.

DR. KENT: If you really are serious about wanting somebody to do some
work with the 10(k)s, which you mention in here as your last thing, you
also ought to be looking for people in finance.

MR. HENGARTNER: I would add people in operations research in some of the
problem so there\'s INFORMS that has probably a web site where you can
post things.

DR. CRAWFORD: Okay. Any other questions or comments for Nancy? I\'d like
to turn it over to Jay. You know, I\'m retiring as the chair of this
committee and Jay will take over effective January 1^st^. And so, he
will be the chair for the spring meeting and he needs to select a vice
chair and he has told me that he has done that.

DR. BREIDT: Well, yes the process for vice chair is to have a committee
vote followed by ASA approval and I\'d like to nominate Nicolas
Hengartner as vice chair, and he is willing to do it, so he has met the
major criteria, in addition to that I think you\'d be a great vice
chair. So, I\'d like to make that motion.

DR. CRAWFORD: Okay, and I second that motion. All in favor? Anyone
opposed? Okay, so then you\'ll just contact ASA and tell them what
we\'ve recommended.

DR. HENGARTNER: Just give me time to renew my membership.

DR. SITTER: So now, Nick, you know exactly when to resign from this
committee.

DR. CRAWFORD: So then, the real reason that we sort of scheduled this
time was for us to have some input and suggestions for the spring
meeting. So, are there topics that you would like to see presented,
continuing topics from this time that you\'d to follow up on, maybe from
two times ago or totally

new topics? And, before you answer any of that, Randy are you going to
be wanting to be on the spring agenda for sure? And, this way you\'ll be
etched in stone and you won\'t be bumped.

DR. SITTER: Yes, either me and/or my student whose name is Crystal, by
the way.

DR. CRAWFORD: Crystal. Mark.

DR. BERNSTEIN: I\'m sorry. Mark Burton and I were chatting about stuff
with I think Nancy and Guy or Bill and Guy or whatever, some combination
of people. I think some of the folks are here. The issue of what impact
the weekly reporting like gas, oil and coal has on the market and trying
to understand that, and if it does have an impact, doing a little bit
more on evaluation and assessment of the estimates that are being made.

We just have a little bit of time on the coal estimates this time but
not enough to, you know, if it is true as we heard from some of the EIA
staff that the people make decisions based on this weekly data, and I
think we need to spend more time evaluating the equations, the
assessments and estimates that are provided. And what that means is
having a lot more detailed description significantly far in advance of
the meeting because if you do a pyramid you have to get into the
equations and stuff.

DR. KENT: There was a lot of work done in the early \`90s by EIA not on
coal but on the impact of those weeklies on oil and gas, particularly
futures for gasoline and things of this nature. And, it did show that
there was a very strong relationship. There was a big announcement
effect and a pre-announcement effect and some of that could be the
starting point. I remember that work was done and I actually presented
it over in Geneva at a conference, so I know that that work is out there
but I think that\'s very worthwhile to go back and to look at what
impact just the announcements themselves have.

DR. KIRKENDALL: Do you know where that was published or where we could
track down a copy of it?

DR. KENT: John B oh, gosh. I can\'t remember his name right now. I
don\'t think he\'s retired.

DR. KIRKENDALL: Pierson?

DR. KENT: No, no. It will come to me and I\'ll give it to you. Yes, I
mean there was a lot of work that was done and I can probably find it in
my files somewhere.

DR. MOSS: I have a question with respect to the gas anyway, the weekly
gas, well the weekly gas reports you come out with are primarily the
storage, right?

DR. KIRKENDALL: Yes.

DR. MOSS: How would that help you? I mean, everybody knows these prices
are used in the market because of the gasp, then you could hear
throughout the country when AGA said we\'re not going to do it anymore,
everybody said well we need it. You know how are we going to fill the
void? And you got the nod so I mean I don\'t think there\'s any question
that people think they need it and there are contracts that are indexed
to some of these series. And so, I guess knowing that there may be some
announcement effects, does that really affect what are going to do?

DR. KIRKENDALL: It may not affect what we\'re going to do but one of the
things that this Information Quality Act we had to designate some data
as \"influential.\" And so, what they designated as influential was
anything that\'s embargoed before it comes out. We had a number of
products like that but our weekly products are embargoed and so that
means they are already called influential. One of the things this little
analysis would do would be to demonstrate that they are influential.
Those are demonstratable from data.

DR. BERNSTEIN: The concern that I had was based on the conversation we
were having the other day on coal stuff is that there is some
uncertainty of how well the estimates are done and this committee could
probably help make sure that they are done as good as can be so that
there is no bias associated or as little bias as possible to its impact
on the market. So, maybe it is that all the weekly reports are important
and we should systematically take a look at them and take a look at how
the estimates are being done, how they\'ve been reviewed, and maybe some
other subjects, spending more time.

DR. KIRKENDALL: Part of my thinking of why that\'s a good idea, is if
these are influential, which we think we can demonstrate anyhow, and
besides we\'ve called them that, is that we would have, if we went
through this process that Mark was describing then we could say upon our
quality guidelines that we\'ve had a detailed review of energy committee
who agree that, if you do, that what we\'re doing is sound and as good a
job as you can expect and that would be worth something to be able to
say that there\'s been an unbiased review by an outside committee
that\'s taken a good, solid look at it.

MR. HENGARTNER: I mean the fact that they are influential is, I think,
clearly established. What I would like to see since there are variable,
they are estimates is how much of their abilities actually reflected in
the change? You know how much does the precision matter? I mean it does
matter but how much? And, I think that would be useful for us to know,
would be useful for EIA to know since we know it\'s influential. The
question is how much does the precision matter?

DR. KIRKENDALL: Actually, it\'s sort of interesting. EIA, you know, we
think we ought to make the numbers as good as possible. I think Beth
told you yesterday that in the weekly natural gas storage, they did some
revisions. They had a lot of complaints from industry and these are the
users, the people out in the financial markets who use the data. Don\'t
revise those numbers. Don\'t make them any better. I just don\'t want to
see them change.

What we were trying to do was to make our weekly numbers do a better job
matching our monthly numbers so that we understand what is we\'ve really
got. So it was sort of interesting.

DR. KENT: Did you any feedback why? Did you ask them?

DR. KIRKENDALL: Maybe Beth did. I don\'t remember why.

DR. KENT: That\'s an amazing sort of thing because I can\'t imagine
anybody in a money market, for example, saying that they want to have a
constant set that is never revised of monetary. My first reaction is
this makes no sense.

DR. KIRKENDALL: Well, I agree.

DR. KENT: Unless it somehow or another makes their life more difficult
because they bet on something, invested \$2 million of their company
money, and then lost when your revisions came out.

DR. MOSS: Well, that\'s unanticipated. Unanticipated changes are what
affect the market. I mean you\'ll see immediate. If they\'re anticipated
then as soon as they\'re anticipated they\'re built into the market
problem.

DR. KENT: There is a huge body of information in publications in the
finance journals on these very things. I don\'t think they apply to
energy so much, but there\'s a lot of work that has been done in
financial areas.

DR. CRAWFORD: Are there any other suggestions? What about the GIS? Is
that something that you\'d like to follow up on next time or maybe the
fall?

DR. BERNSTEIN: Well, I think that it\'s really important for EIA to get
a strategic plan together. I\'d certainly like to see that.

DR. CRAWFORD: All right.

DR. BERNSTEIN: We could help them at some level, but I\'m not sure we
need to report back on it as I indicated earlier. We could just get a
report, you know, a written report in as to here is our strategic plan
and any of us who want to comment on it, we can.

DR. CRAWFORD: Okay.

DR. KIRKENDALL: Actually, we are supposed to go through a strategic
planning exercise in the spring and it\'s supposed to be pretty much
done by the next meeting, so we can always just report to you on what we
think we\'re going to do in that area and get your comments. And,
that\'s an area that maybe we can report again sometime, but we need to
have good, specific questions, or thoughts, or ideas to throw out.

DR. HAMMIT: I have sort of a general area, which is probably the
perennial question which is to ask whether it would be useful to take an
overview of what your various products are and the time resolution in
these estimates and frequency with which they are released and the
substance of what\'s covered.

You know, it seems you\'re always in this position of balancing what it
is you do, what are the most important areas, are there some things that
have outlived their usefulness or are there some things that ought to be
recombined because of the way energy markets have developed? So, I\'m
not sure if that\'s part of this broad, strategic plan, or if you just
meant specifically in the geo referenced.

DR. KIRKENDALL: I\'m not sure what all is in the strategic planning
either because I didn\'t

take part in the last one. I think Renee took part in part of at least.
Well, I really don\'t know. I really can\'t say much about what we do in
strategic planning. As part of the thing about an overview of the EIA
products and efforts, I think it would be useful to have a brief
document that talked about those things to use as background information
for new committee members, and maybe some of the old ones too.

We asked you a lot of questions and, of course, before you know anything
about EIA it\'s hard to put into context, so I know we need to do a
better job on good introductory material for you and that is a topic
that Bill has worked on a little bit and we had it on the agenda for
this time and then it was overcome with events. So and again, that may
or may not be a briefing but we certainly could provide information to
you.

DR. CRAWFORD: Does that tie in at all with EIA 101? Are some of the
documents the same or would this be something more specialized for the
committee?

DR. KIRKENDALL: Well, just starting with what we\'ve got for EIA 101
isn\'t a bad idea.

DR. CRAWFORD: Okay.

DR. KIRKENDALL: We need to have little, simple descriptions of what we
do anyway.

MR. WEINIG: Some of the things that are involved in EIA 101 have more to
do, and I stress some of the things, with personnel, where\'s the
bathroom, you know, that kind of stuff that you don\'t want to know
about or care about. But, the bathroom is important. But, such things as
where EIA got its start that came up. You know, those kinds of bench
marking and then we can move on. And, Paul Staller in the Office of
Resource Management has already developed a much larger list of things
for the EIA 101 from which he and I are beginning to draw from.

DR. CRAWFORD: So then, the other thing I guess I can do is I\'ll send an
electronic copy of Cal\'s report to Jay and maybe that could be
forwarded to the new committee members to read, because that does a good
job of addressing some of the statistical and economic issues that new
committee members are going to have to face when they come on the
committee so they\'ll know to put their data quality hats on or their
modeling hats on when they come.

DR. PHIPPS: So, is this going to be a subject of discussion, training
for new members, because particularly when you\'re putting a six-year
investment in people, it would have been really useful to have like
either a briefing book or even a half-day training session when you came
in just on what your surveys are, things like that, and every time I
kind of struggle through and look back through previous ones when I\'m
trying to find the information they\'ve got. It\'s so easy to forget
between times, so if there was anything like parts of that, the 101
course, or an introduction to the different program areas and surveys,
it would be really great.

DR. SITTER: I have two comments. One is in the natural gas data program
there was a slide on analysis underway, merits and methods to replace
voluntary survey states for production volume prices, sources of
import/export. It was related to update on natural gas data, current
programming changes underway or under study. Is the spring meeting a
good time for some update on that, on what\'s been accomplished there?

DR. KIRKENDALL: I certainly hope so. They have a contract in place right
now to do some of the background investigation and I believe they\'re
supposed to be done by that time. So, I hope we have time then to show
you what we\'ve learned from the contract and talk about what proposals
there are. There\'s a lot of stuff going on in the natural gas area and
it would be good to get input as we go forward with it.

DR. SITTER: And related to this training document and so forth, and this
isn\'t just for new members, but I mean since they\'ve been on the
committee and obviously since Cal\'s been on the committee, the
committee sort of consistently complains about material arriving on
time. So, I\'m going to say that what would be nice is to have the
material from the previous meeting that\'s related to the material from
this meeting. So, like I just said, if we\'re going to update this one,
it wouldn\'t be bad to have this stuff, not the whole package from this
meeting but the stuff that\'s pertinent or from the previous. And, this
you can get to me really early. You already have it.

DR. HENGARTNER: To some extent, we could have them archived.

DR. SITTER: An archive, yes.

DR. HENGARTNER: An archive, the old stuff that you put on the Web this
time goes into the archive and when something is pertinent, there\'s a
link so you can get to it.

DR. SITTER: Just link back to it, exactly.

MR. WEINIG: We maintain, just for your general information, we maintain
those with the links each meeting has and this one and the previous two
are maintained over time. So, you can get to any of that at your
leisure.

DR. SITTER: Perfect, but what you can do is that what happens is you
already have titles for your talks well ahead of when we get the
material. You can put them up and link them back.

DR. CRAWFORD: It\'s the linking back I think we really need because
it\'s going to be hard for us to tell from the title of the talk what
was it that we saw last time or from the time before that\'s going to
tie-in.

MR. WEINIG: Gotcha.

DR. CRAWFORD: So, you\'re going to have to blatantly point us to where
you need us to go.

DR. SITTER: Just pretend we\'re silly enough to take a vice chair
position on this committee.

DR. BERNSTEIN: If I might add on that, the timeliness of it, my
preference would be that if the staff can\'t get their presentation done
in enough advance to run it through with one of you guys, then it\'s
just taken off the agenda. I think one of the issues that we heard from
Calvin is there\'s this inconsistency in presentations built into the
content. You guys know the audience and a lot of times the staff do not
and if you can\'t have it scripted, if it wasn\'t done enough then you
just scrub it.

DR. KIRKENDALL: We only had one of those this time and you probably
forget which one that was.

DR. BERNSTEIN: But there was still some inconsistency.

DR. KIRKENDALL: Well, there\'s always going to the inconsistencies.

DR. BERNSTEIN: But there was, the one before was too long.

DR. KIRKENDALL: Yes.

DR. BERNSTEIN: It was not just what was being said was too long, it was
too long, and so there needs to be B

DR. KIRKENDALL: And, that was our fault because was it was also too long
in the dry run and we didn\'t make him shorten it. So, we can do that.

DR. BERNSTEIN: That\'s your agenda.

DR. CRAWFORD: I saw too and I didn\'t want to cut off the discussion
because I thought it was an interesting topic and so I let it go on.

DR. KIRKENDALL: And, on his briefing, we could have worked with him and
he was very nice about presenting it with us and he did do a dry run,
and so the fact that it was too long really was our fault. We didn\'t
make him change it. We didn\'t give him specific suggestions for how to
make it shorter. We\'ve done that with others but we didn\'t with him
for some reason. We were sleeping or something, I don\'t know.

MR. HENGARTNER: I would have given him more time.

DR. KIRKENDALL: Well, and that\'s it, we had trouble getting a paper
from him in the beginning that people thought was useful and there was
some discussion of taking him off the agenda because of that. But, you
know, next time will be too late on that. Well, it\'s not really too
late but now is a good time to talk about the issues.

DR. CRAWFORD: And, I think he got some good comments. I think Jae\'s
comments are excellent, and Neha\'s, I\'ve read hers, they\'re very good
also. Cal.

DR. KENT: I would make a comment that if somebody\'s gone back and read
all of these, the quality has improved immeasurably over the years of
what was presented to the committee. As I told some of you before, the
early papers, it honestly looks like somebody, oh my God tomorrow
morning I got to make a presentation in front of that committee, and so
they sat down at their typewriters, because a lot of the early ones were
not even word processed and banged out what was ever off the top of
their mind and brought it in. And, we\'ve taken giant steps, or the EIA
has taken giant steps in terms of doing that. But consistency is still
something I think that needs to be addressed there. But, if you want to
see some of the early papers, I now have them all, believe it or not.

DR. CRAWFORD: I do remember when I first came on the committee, we did
have papers, and for me papers, well they provide more information when
you\'re just looking at the outline of talks, and that\'s what we\'ve
gotten to now is just an outline of talks. It\'s hard for me to follow.
I don\'t really know what\'s going to be said, particularly B I mean
some of these were more detailed and the slides were crowded which was
good because you get more information. But, I mean if you just have a
list of bulleted items, it\'s really hard to know where that fits in and
to provide a discussion around that. And so, Jae said something last
night about Bill Watson\'s paper being almost ideal for us because it
was text. It had data that you could see. It had graphs that were
clearly labeled, and then of course it had SAS codes and we knew exactly
what models were fitted and how they were done. But then, Nancy said his
paper was late, and so.

DR. KIRKENDALL: Was Watson\'s paper late?

DR. CRAWFORD: Yes, it was.

DR. KIRKENDALL: He\'s the one who knows about the timing of these
things.

MR. WEINIG: But, if I understood Carol, on the break it read quickly
because of the way it was organized and the level of detail.

DR. CRAWFORD: Yes.

MR. WEINIG: And so, while it was late, and it was, it may have been more
useful than some.

DR. CRAWFORD: That type of paper can be late but if you take something
that\'s a topic that is a little bit more general that we haven\'t seen
before and put it all in bulleted format and then make it late, then
that makes it hard.

DR. KIRKENDALL: One of the things I was trying to do with a couple of
the breakout sessions was more under my control, I guess, and that
would\'ve been Ruey-Pyng\'s and Preston\'s. Now one reason why we
couldn\'t give you a full paper on that is they only got the assignment
to do the work in early September. But on the other hand, we got really
good suggestions in the breakout sessions and I think for both of them,
particularly for Preston\'s but his is a simpler project.

DR. MOSS: But, he was the one where we didn\'t know where he was coming
from it all. You know, I mean, we saw the problem but there is no
indication of how he was trying to solve it, so we didn\'t think about
it before we got in there and just strictly kind of reacted on it.

DR. KIRKENDALL: He only gave you data. Well, you had found a lot of
stuff on the Web and looked at it.

DR. MOSS: He gave us a spreadsheet right and I tinkered with that a
little bit, but there was no indication of the different approaches he
was taking.

DR. KIRKENDALL: Well, I think B go ahead.

DR. MOSS: But like you say, he only got the assignment a month or so
before, so it\'s not fair to him.

DR. KIRKENDALL: I\'ve been telling people, I\'ve been terrorizing my
staff by having him do this. He liked it. He thought it was a positive
experience coming and talking to you. So, it was good that way and I
think he got really good input. So, the end product was really good,
even if frustrating for all concerned.

DR. SITTER: We do have to be cautious because you know we\'re always
telling people come to us early, come to us early. That\'s what we\'re
saying. I thought that one wasn\'t bad. I thought it was a pretty good
fit. We had some data. He\'d done some plots but he did a pretty good
job, and I think that if he hadn\'t come to us then, he might have done
a lot of stuff that we would have said, our reaction would have been,
gee you probably shouldn\'t have done that.

DR. KIRKENDALL: No, as a result of talking with the committee, he now
has a much clearer idea how to solve the problem. He\'ll be able to do
it fast and I think everybody will buy into it. So, it has really
simplified the project that we came in and talked to you on that one.

DR. MOSS: I think there\'s a difference between having a clear
presentation and having a finished product. I mean, if we have finished
product were not getting contribution. That was like Doug Hale\'s
presentation, which I thought was really interesting. I always like his
stuff, but it was essentially B

DR. KIRKENDALL: It\'s done.

DR. MOSS: Yes. Here\'s what is the result of what we talked about two
sessions earlier, maybe one.

DR. CRAWFORD: Renee, did you have a comment?

MS. MILLER: No, Randy pretty much covered it.

DR. BERNSTEIN: You know I think that\'s a good point in the sense that
maybe if you get somebody started on this too early and they\'re already
finished, then it\'s also helpful maybe. You know, I\'m not as concerned
with the polish. I\'m concerned with is there a real question that
you\'re trying to get at? Get to the question first.

Show some data or some model or some equations underneath it and give us
a chance to review it, and maybe even give us a chance to have a
dialogue by e-mail with the person even before we get here. Sometimes
whoever on the committee is going to be the respondent if they have
enough time could actually maybe as encouragement get some questions on
e-mail before the session.

DR. KIRKENDALL: Well, as an example of those two, Preston\'s work is
going to be done before the next meeting, so you probably won\'t hear it
again except maybe in our little summary paragraph to tell you what we
did and how it turned out. But Ruey-Pyng\'s, I\'m sure you\'re going to
hear about. That\'s a much more B it\'s a much bigger project. I doubt
very seriously he\'s going to be done with it by next time. But, by next
time, he might have some data and more interesting things to show you.
So, that would be one that I think that you probably will hear about
again because I\'m sure we won\'t have answered all the questions but
we\'ll have a better discussion for you and you all already have a
little background now because we struggled through this one.

DR. BERNSTEIN: I\'m sure you\'ve probably addressed this at least in the
datafile, but is the timing of these meetings appropriate for the
scheduled things you need to do and when you need the feedback? Or, is
there just no commonality among it so it doesn\'t really matter when
these meetings are? But, I don\'t need an answer for that now, but
thinking about: a) is the timing good from the point of getting the
feedback, and is the timing good in terms of when these guys are too
overburdened to begin with?

DR. KIRKENDALL: Well, that\'s always a problem.

DR. BERNSTEIN: It\'s always a problem. There may be no down times.

DR. CRAWFORD: Bill had a comment or question.

MR. WEINIG: I wanted to go back to Bill Moss for a minute. Bill, Hale\'s
study was done but we wanted to get back to the economist and
forecasting advice that we\'ve gotten and so for that reason we wanted
to get back to the committee. But because there wasn\'t another place to
put it, we put it in a breakout sessions to kind of get that group, your
group of folks into it, and get a reaction.

And, we thought given the importance of the project and the level of
sophistication, these are my words not Hale\'s, that maybe doing it in a
breakout session which will allow us on the order of an hour rather than
several minutes to include in Nancy\'s introductory remarks on Thursday
morning might be a sensible way to handle that.

And, to come back over in here, a lot of the modeling and forecasting
staff is developing the annual energy outlook in the fall. So, if
there\'s a time for them, at least this year, when they weren\'t raising
their hand for another opportunity to come to the committee, it could
have had something to do with that.

DR. CRAWFORD: So, are there any final comments or suggestions for
particular topics for the next meeting?

DR. HENGARTNER: There\'s one topic. I don\'t know if we need to address
it in the next meeting but I\'m curious to know what is available and
what is the impact of the data we collect here for homeland security?

DR. CRAWFORD: So then when you talk with Guy this afternoon at noon, and
I\'m going to adjourn the meeting soon so you can get there, maybe you
could be in touch with Jae and Nancy and Bill and see if that isn\'t
something that could come up at the next meeting depending on what he
says.

DR. HENGARTNER: I mean it\'s something that is of interest to me. If no
one else cares then that\'s it, but I\'ll bring it up with Guy and see
there.

DR. CRAWFORD: Okay and so then just let Jae and Nancy know what comes
out of that because I do think it\'s a fascinating and very timely
question that I think everybody would be interested in.

DR. HENGARTNER: Thank you.

DR. EDMONDS: Just to say, it may be obvious, but I would like to
maintain this argument we\'ve had on the modeling that has gone on in
the two sessions that I\'ve attended. I think it\'s been useful,
particularly in light of Cal\'s comments that that\'s a place where we
need to work to make sure that we give good useful input. Having that
dialogue going on I think is important.

DR. KIRKENDALL: I think it\'s an important topic too. The question is in
coming up with questions for the committee, and I think if we\'re not
getting good advice from the committee, it\'s probably because we
haven\'t come to you with questions that are appropriate for you to
address.

DR. KENT: I didn\'t say that you weren\'t getting good advice. I was
saying that they felt there was a feeling that the relationship and
modeling was not as strong as in the survey area or in the data area,
and that might be a question to be asked because I really do think it\'s
a case if the right questions aren\'t being asked and the right answers
are coming from EIA, then you\'re not going to get much response. I
would think that you should have something every time, at least on some
modeling, but I think it needs to be asked, not are the questions the
ones the committee is interested in, but are the questions the ones that
EIA is interested in?

DR. CRAWFORD: Well, I know and I don\'t want the current modelers to
take that personally either. I don\'t know how time sensitive that
comment was, but I know that EIA did have that comment when I was chair
and I\'ve been actively recruited people like Jae to be on the committee
to serve that need. So, I think that the committee has been responsive
to that. We just might be, it\'s a little slow for us to move in that
direction.

DR. BERNSTEIN: I don\'t know about Jae but maybe having a discussion
about how you estimate the impact of environmental regulations would be
sort of narrowed down and then how that\'s done. That might be useful
for the modelers and us, both to get a sense of really what is happening
in NEMS and do that estimate and then get some feedback from the
modelers here on both how the results will look, how to present it, and
whether or not there should be other elements in there and, I wouldn\'t
mind a lengthy discussion about it.

DR. EDMONDS: Well, in fact, just to generalize your comment, the
problems that are being put in front of the modelers and how they feel
about, do they have the resources? I mean the other possibility is that
the modelers know what they\'re doing and know where they\'re heading
and the committee is not necessarily B they don\'t need the committee is
another possibility.

DR. KIRKENDALL: Well, they probably haven\'t been trained on how to use
the committee.

DR. BERNSTEIN: But, I think maybe we could think about before the next
meeting what is the question that you\'re likely to get asked about from
either the administration or Congress on some issue? I mean we could
brainstorm a little bit sort of off-line and sort of pushed by Jae here
to get us to think about some questions, and say okay here\'s a question
we think you may get asked. Let\'s have dialogue about how you should
address that.

DR. MOSS: Well, let me be the naysayer on that because from what I hear
you\'re overworked already and what that suggestion means, we\'re going
to think of new tasks for you to take on, it seems to me one of our
benefits to EIA is to augment your resources to help you deal with the
projects you have in a better way.

DR. EDMONDS: Actually, I didn\'t hear him say that, which if that had
been what he was saying, I would agree with you. But, I heard Mark
saying, and you might say it better yourself, but it was let\'s think
about, anticipate the kinds of things you may have coming down at you so
that to the extent you\'re prepared in anticipation then it makes your
life a little easier. That was what I heard you say.

DR. CRAWFORD: Maybe there can be some more discussion about this over
lunch. I\'d like to B oh I\'m sorry Johnny, do you have a comment?

MR. BLAIR: Just a quick suggestion or question. We had, I\'m not sure
how many meetings ago, a discussion about emergency surveys or
short-term surveys, whatever. I\'d be interested in getting an update on
what, if anything, has happened with that or what the current plans are.
It may be related to the homeland security, but it would be interesting
to know just sort of hearing what happened, what was the follow-up to
the discussion and to the suggestions that were made at that time.

DR. KIRKENDALL: I think one of the suggestions was to document lessons
learned from doing the emergency surveys and I believe that paper was
prepared, and I thought we distributed it to you last time. If we
didn\'t, we should distribute it to you.

DR. SITTER: Just give us a link back if you did.

DR. KIRKENDALL: And, if we didn\'t, there was an oversight and we need
to do that because we did follow through and do a paper and I think they
even thought about the lessons learned when they did the next iteration
on a new survey, so it was useful advice.

DR. CRAWFORD: So, finally, just to wrap up I really want to express my
deep appreciation to everybody on the committee that made my job as
chair easy. I got a lot of people to volunteer, even if they were
guilt-ridden volunteers. I\'ve learned a tremendous amount from
everybody about a lot of different topics and I also appreciate all the
help I\'ve received from EIA, from Bill, from Nancy, from Stan Freedman
who\'s not here, from Renee, from Mary. It\'s made my time as chair very
nice and I hope to see you around in future meetings.

So, with that, we\'ll take some questions from the public and then close
the meeting. So, lunch is ready for us downstairs for any of you that
can stay.

(Whereupon, the above-entitled matter was concluded at 12:03 p.m.)