May 19, 1993

MEMORANDUM

SUBJECT:  Technical Comparison Document--Phelps Dodge Smelter

FROM:     Joseph A. Tikvart, Chief
          Source Receptor Analysis Branch, TSD (MD-14)

TO:       Gerald Fontenot, Chief
          Air Programs Branch, Region VI (6A-AP)

     As follow up to telephone conversations with your staff and in
response to the request from Shawn Kendall of Phelps Dodge, the Model
Clearinghouse has reviewed the subject document.  Although we have a few
comments, which are provided below, in general we believe that the plan for
model evaluation in the document is sound and the Company should plan to go
ahead with the data collection as planned.  Resolution of the relatively
minor issues we have identified can take on whatever schedule you want to
set up with the source over the next several months.

Comments on the Technical Comparison Document (TCD)

1.  Comments from Bill Cox are attached.  His first comment refers to the
role of a reference model.  The normal protocol for determining the most
appropriate model for a given application through a performance evaluation
is to compare performance of the proposed model and a reference model.  If
the proposed model performs better than the reference model, based on
criteria set up in the scoring scheme, then it may be used for the
application.  If the proposed model does not perform better, then the
reference model should be used.  However, the Interim Procedures for
Evaluating Air Quality Models (Revised) does allow for the use of other
technical criteria to make a decision in the case of comparable
performance.  But the basis for making such a judgment is usually highly
subjective and could be argued either way in most cases.  

     Even though we believe that the reference model should generally be
the default, unless the proposed model clearly performs better, an
exception to this philosophy may be appropriate in the Phelps Dodge case. 
We tend to agree that the MPDM model may be used in the event of a
performance evaluation tie.  This is the case because MPDM appears to be a
technically better model for this application since it handles transport in
complex terrain in a more rigorous manner.  The real key to its technical
superiority is, however, the fact that the sophisticated data bases
necessary to run the model should be available from the field study.

    Bill's other two comments are self explanatory and should be addressed
in revisions to the document.

2.  An important part of the criteria that we have insisted be included in
model evaluation protocols of the type being discussed here is to include
provisions to guard against underprediction of the design concentration(s). 
Phelps Dodge should include such a provision in their application scheme
for MPDM should that model be determined to be appropriate, yet still
underpredict the critical concentrations.  Their consultant, ENSR, is
familiar with underprediction adjustment factors used in other situations,
and should be able to develop an acceptable scheme.

3.  As a general observation, we agree with the monitor siting and with the
performance evaluation criteria in the TCD, and we do not have a better
scheme to suggest.  However, we really wonder whether the state of the
science is commensurate with the degree of sophistication used in
developing the scoring schemes.  In other words, while we agree that it is
important to consider such things as concentration patterns, weighting of
monitor readings, and cross-cutting between averaging times, we question
whether having a highly sophisticated rigorous procedure for accomplishing
these considerations leads to an unwarranted sense of certainty about the
believability of the results.  Perhaps a simpler scheme, based more on
subjectivity, would better convey to the public the true state of the
science in developing a scoring scheme.  While these are our thoughts, we
are not insisting or even recommending that any changes be made in the
scheme contained in the TCD.

     In summary, we are in general agreement with the material contained in
the Phelps Dodge TCD.  The Company should address our comments in a
revision to the document.  We are particularly interested in reviewing the
underprediction scheme before the Environmental Protection Agency agrees to
the protocol.

     If you have any questions please contact Dean Wilson at 919-541-5683.

Attachment (1 p.)

cc:  G. Blais, MD-15
     W. Cox, MD-14
     T. Diggs, Region VI
     J. Yarbrough, Region VI

To:       Dean Wilson                                        April 20, 1993
Subject:  Comments on the ENSR-Phelps Dodge Project
From:     Bill Cox

     The criteria for model selection compares each competing model with
"the most technically sound model" which they have determined to be MPDM
(page 11-1).  They propose to use MPDM unless the MCM calculation of at
least some other model is statistically superior to MPDM.  Question:  I
thought that comparisons had to be made against a reference model--a model
I assumed was other than the model being proposed.  Have I missed
something?

     ENSR apparently follows the EPA approach for model comparison fairly
closely with the exception of (1) weighting assigned to monitoring stations
and (2) inclusuion of a 3-hour fractional correlation statistic.  I
personally am luke warm to the 3-hour correlation statistic (see page 10-8)
and do not see that it has any particular value.  If they wish to include
it, I am not sure we should necessarily object; however, I do believe that
they could present a more cogent argument for it.

     They should include the Atmospheric Environment reference in addition
to the EPA reference for comparing models.  It is:

     Cox, William M. and Tikvart, J.A. (1990).  A Statistical Procedure for
          Determining the Best Performing Air Quality Simulation Model. 
          Atmospheric Environment, 24A.