CALPUFF: Status and Up

Dennis Atkinson*

Meteorologist (NOAA)

*in partnership with US EPA, OAQPS

Tyler Fox and Roger Brode

USEPA/OAQPS

Regional/State/Local Modelers Workshop

May 16, 2007
Outline

• CALPUFF Modeling System Background

– Scorecard of CALPUFF/CALMET versions

• First update of EPA regulatory version of 

CALPUFF/CALMET

• Initial efforts to update to VISTAS version

– Engaging TRC through VISTAS

• Results of update process • Findings and next steps
Scorecard of CALPUFF 

Versions

- 5.7 April, 2003 (promulgated version)- 5.711a July, 2004 (current regulatory)- 5.711b December, 2005- 5.754 February, 2006 (original VISTAS- 5.756 August, 2006 (latest VISTAS)- 6.112 April, 2006 (MMS)

April, 2003 July, 2004 Dec., 2005

Feb., 2006

April, 2006 

5.7 5.711a 5.711b

5.754

6.112 
CALPUFF/CALMET Versio

Linkages

- 5.7.......... 5.53

- 5.711a.... 5.53a 

- 5.711b.... 5.53b- 5.754...... 5.724- 5.756...... 5.726- 6.112...... 6.211 
Regulatory Approval of

CALPUFF Modeling Syst

• Question about regulatory status of CALM

(by community, model developer)

• EPA approves each component of the 

CALPUFF modeling system

• Regulatory approval is for both, e.g. curre

regulatory version is 5.711a/5.53a
CALPUFF Update Proce

• Incumbent upon EPA to perform an indep

assessment of CALPUFF when updating versions

• CALPUFF requires extensive assessmen

understanding of changes made; approvamade by EPA, not developer 

• CALPUFF Update Tool – Introduced at 8

Modeling Conference in Sept 2005
CALPUFF Update Too

• Developed by MACTEC• Compares 2 CALPUFF versions; 

proposed (beta) vs. current version 

• Determines code changes and differ

in results across 10 scenarios

• Provides a standardized methodolog

evaluating changes in model codes
Analysis

Tool

Sources

Meteorology

Terrain

Differences

?

Reso

BETABASE

CALPUFF Update Too
Description of Scenario

1. Large-scale domain in Pacific NW-NWS 2. Same as 1 with MM5 NOOBS option3. Medium-scale domain in Pacific NW with 

MM5/NWS

4. Medium-scale domain near Shenandoah 5. Small-scale complex flow with deep valle6. Idealized hill with steady-state met – sim7. Idealized hill with steady-state met – PG8. Flat terrain with steady-state met – simila9. Flat terrain with steady-state met – PG10. Idealized hill with simulated wind shear
Description of Sources

• Four core sources included in all 

scenarios:

(1) Ground-level area source (20m x 

200m)

(2) 10m volume source(3) 30m non-buoyant point source(4) 65m buoyant point source
CALPUFF Regulatory Upda

Version 5.7 to 5.711a

• Model Change Bulletin-A provided 

description of changes

– Only minor changes in coding

• Application of Update Tool showed 

minimal differences/impacts on concentrations

– Clearly attributed differences to bug fix

• Approved 5.711a - June 15, 2006
CALPUFF Regulatory Upda

Version 5.7 to 5.711a
CALPUFF Regulatory Upda

Version 5.7 to 5.711a
FEDs Discussions on CALPU

Spring 2006

• Reason: EarthTech sells CALPUFF rights to TR

2006) and EPA experiences difficulties in updatiVISTAS version 5.754

• Representatives from Federal Agencies discuss

current situation with CALPUFF (status and reguupdates)

– EPA– FLM– MMS

• Discussed the following issues:

– lack of adequate documentation– response time from developer slow– developer requires funding to address any issues from 

community, despite regulatory classification of the mo
EPA contacts TRC:

June 30, 2006

• CALPUFF must meet Appendix W 

requirements for regulatory models, Section 3.1.1(b)-(d):

– *.exe to run on a common platform– current model documentation – input/output test data w/model– users can operate model from available 

documentation

– documentation must include compariso

w/tracer data

– model/code cannot be proprietary
TRC to EPA response:

July 11, 2006

• TRC provides copy of Use Agreeme• States that ... “Permission to modify 

Fortran computer codes is granted tusers with the provision that modifieversions of the model codes must be provided promptly to TRC, along witappropriate written documentation ainstructions for use”
TRC to EPA response:

July 11, 2006

• Use Agreement also states...

“It is the user's responsibility to dete

if the CALPUFF modeling system is appropriate for any particular applicaand the user assumes all responsibiand liability for its use.”
CALPUFF Regulatory Upd

• Began update process with v5.754 in Jun

2006

• Engaged with TRC in discussions to unde

5.711a vs. 5.754 differences

• August 15, 2006—Received e-mail from T

indicating a new VISTAS v5.756 would breleased

• OAQPS decided to focus on update for n

VISTAS v5.756

– newest features, latest corrections, etc. for us

community (including MMS updates for over 
CALPUFF Regulatory Upda

5.711a -> 5.756

• Found large ‘unresolved’ differences 

between 5.711a/5.756

– CALMET versions (5.53a vs. 5.726)—u

83% for scenarios 2, 3, (Pacific NW), 5 valley)

– CALPUFF versions (5.711a vs. 5.756)— 

7% for scenario 5 (deep valley)

• PROBLEM = incomplete documenta

– Missing Model Change Bulletins (MCB– No in-code comparisons– Inadequate model documentation
While we wait for response 

• BART usage

– “Good faith” effort to use “best” availab

model version

– Attempt to be “consistent” with App W 

‘wall off’ any potential bad precedents

• OAQPS becomes increasingly conc

about NSR/PSD permit modeling

– Appendix W provides “rules of the gam
Clearing the Air on CALPU

• OAQPS makes clear position statement a

monthly call with RO modelers (January 2

• Region 4 communicates with VISTAS so 

Brewer contacts OAQPS and facilitates discussion with TRC (Feb 2007)

• Request for documentation from TRC via 

VISTAS (Feb. 2007)

• Received documentation from TRC on M

15, 2007
CALPUFF Update Process

Initial Assessment of v5.75

• Significant differences in high ranked 

between v5.711a and v5.756

– Largest differences (up to 83%) attribu

CALMET (v5.53a vs. v5.726) for Scena3 (Pacific NW), and 5 (near field/comp

– Smaller differences attributable to CAL

(v5.711a vs. v5.756); up to 7% for Sce(near field/complex flow)

• Much larger differences found acros

modeling domain – LRT distances
CALPUFF Update Process

Initial Assessments of v5.75

• Results from initial assessment of VISTAS 

range of percent differences of high rankeacross all sources and averaging periods scenario:

– Scenario 1: -22.0 to +23.3 – Scenario 2: -0.1 to +83.4– Scenario 3: -17.7 to +60.8– Scenario 4: -13.6 to +28.1– Scenario 5: -46.0 to +21.1– Scenario 6: -10.3 to +6.3– Scenario 7: -1.7 to +1.0– Scenario 8: -10.0 to +5.4– Scenario 9: -1.2 to +1.0– Scenario 10: No differences
Maximum Percent Differences by Averaging Period Due to CALMET:

Short Term Averages:Max % Diffs by Aver Period for Scenario 1:

ARE1 VOL1 P301 P651 ARE2 VOL2 P302 

----- ----- ----- ----- ----- ----- ----

1 -0.019 0.107 0.675 15.212 1.641 1.879 -15.804 

3 0.011 0.081 -1.132 -10.811 0.218 -1.139 -22.012 

24 -1.757 -0.272 2.609 12.242 0.676 0.800 12.437 

8 -0.041 0.075 1.985 9.376 -0.062 0.908 21.914 

Max % Diffs by Aver Period for Scenario 2:

ARE1 VOL1 P301 P651 ARE2 VOL2 P302 

----- ----- ----- ----- ----- ----- ----

1 22.278 54.430 0.153 42.447 -0.084 -0.091 -0.133 

3 60.666 51.456 0.137 34.825 -0.047 0.393 -0.070 

24 59.770 83.390 3.771 46.302 0.837 0.803 0.185 

8 68.987 39.749 1.448 63.646 -0.202 0.732 0.472 

Max % Diffs by Aver Period for Scenario 3:

ARE VOL P30 P65 P99----- ----- ----- ----- -----

1 0.008 0.023 2.704 60.830 5.198

3 0.177 -0.141 0.175 25.405 17.063

24 0.715 0.440 5.879 21.892 -7.893

8 -0.025 2.368 4.788 -17.722 11.605

Maximum Percent Differences for High Ranked Va

Scenario and Source Due to CALMET v5.53a vs. v
Maximum Percent Differences by Averaging Period Due to CALMET:

Short Term Averages:Max % Diffs by Aver Period for Scenario 4:

ARE VOL P30 P65 BAR----- ----- ----- ----- -----

1G 4.244 6.878 -3.408 7.026 -6.991

3G 16.207 9.008 17.049 4.782 -11.002

24G 15.403 16.381 16.672 13.379 3.346

8G 18.695 16.323 18.239 5.697 -13.627

1D -6.718 -8.323 6.937 6.176 3.585

3D 17.231 20.079 22.897 8.938 -3.995

24D 22.332 23.710 16.899 21.473 -9.353

8D 21.152 12.227 19.777 28.060 -3.349

Max % Diffs by Aver Period for Scenario 5:

ARE1 VOL1 P301 P651 ARE2 VOL2 P302 

----- ----- ----- ----- ----- ----- ----

1 -0.020 -0.188 2.522 -33.268 -19.460 -0.015 16.66

3 -0.003 -0.069 3.798 -21.227 -5.749 -0.058 11.72

24 -0.558 2.006 19.485 -27.681 11.601 1.498 11.31

8 0.005 -0.300 17.110 -31.322 0.133 0.304 21.10

Maximum Percent Differences for High Ranked Va

Scenario and Source Due to CALMET v5.53a vs. v
Scenario 2 - 65m Point Source, Salem Location

Percent Difference - 1st Highest, 1-Hour Averages

-400 -300 -200 -100 0 100 200 300 400

-400-300-200-100

0

100200300400
710 712 714 716 718 720 722 724

524252445246524852505252

Scenario 5, Area Source

West Side of River Near Rock Island Dam

Percent Differences - 1st Highest, 1-Hour Average
CALPUFF Update Process

Additional Assessments of v5

• Under auspices of VISTAS, TRC provided 

additional documentation regarding model 

– New MCB-C and MCB-D (not released yet)– Tables of in-code documentation annotated to 

category of change and whether change may aresults (previous MCB’s only included bug fixes

– Summary of TRC model tests, claiming that “th

code is equivalent to the EPA-approved code ocoding errors in the EPA code are corrected.”
CALPUFF Update Process

Additional Assessments of v5

• Annotated in-code documentation id

several categories of changes, inclu

– Bug fixes– Non-optional technical enhancements– Optional technical enhancements– Non-technical enhancements– Enhancement adjustments– Coordinate conversion fixes
CALPUFF Update Process

Additional Assessments of v5

• TRC provided the following interim ve

to facilitate isolating impacts due to (fixes and (2) non-optional technical enhancements:

– CALMET v5.53c = CALMET v5.53a + bug fix– CALMET v5.53c2 = CALMET v5.53c + non-op

technical enhancements

– CALPUFF v5.711c = CALPUFF v5.711a + bug 
CALPUFF Update Process

Additional Assessments of v5

• Additional potential source of differences i

(indirectly) by TRC, namely “new default parameters for optional technical enhance

• User must manually override these new de

parameters to maintain consistency with Eapproved version

• CALMET and CALPUFF output list files do 

document these new parameters unless thhas specified them in the input file
CALPUFF Update Process

Additional Assessments of v5

• New default parameters for optional techn

enhancements in CALMET include:

– IMIXH = -1 for convective mixing height option based on 

Carson for land cells only and original OCD mixing heig(default value of 1 will select Maul-Carson for land and 

IMIXH is omitted);

– THRESHL = 0.0 for threshold buoyancy energy flux per 

boundary layer required for Mixing Growth over land (deof 0.05 W/m

2/m will be selected if THRESHL is omitted)

– THRESHW = 0.0 for threshold buoyancy energy flux pe

boundary layer required for Mixing Growth over water (dof 0.05 W/m

2/m will be selected if THRESHW is omitted

– ICOARE = 0 for option to use delta-T method for the Co

Ocean Atmosphere Response Experiment (COARE) bufor computing Z

o, u*, L and Q

H overwater
CALPUFF Update Process

Additional Assessments of v5

• TRC later identified the following new defa

parameter for an optional technical enhanin CALPUFF:

– SVMIN(7:12) = 0.50 for minimum sigma-v value (m/s) fo

cells for stability classes 1 through 6 (default value of 0.be used for each stability class for over water grid cells parameters are omitted)
CALPUFF Update Process

Additional Assessments of v5

• EPA conducted several tests in order to co

and isolate affects of the following categorchanges:

– Bug fixes– Non-optional technical enhancements– New default parameters for optional technical 

enhancements

• Test No. 1 designed to test TRC’s assertio

“the VISTAS code is equivalent to the EPAapproved code once the coding errors in tcode are corrected”

– CALPUFF v5.711c/CALMET v5.53c vs. curren

version (CALPUFF v5.756/CALMET v5.726)
CALPUFF Update Process

Additional Assessments of v5

• Results from Test No. 1 - range of percent 

differences of high ranked values across asources and averaging periods by scenari

– Scenario 1: -22.0 to +24.5 – Scenario 2: -21.7 to +81.8– Scenario 3: -17.9 to +60.3– Scenario 4: -3.3 to +30.5– Scenario 5: -41.8 to +19.5– Scenario 6: -9.7 to +6.3– Scenario 7: -1.7 to +1.0– Scenario 8: -9.3 to +5.6– Scenario 9: -1.2 to +1.0– Scenario 10: No differences
CALPUFF Update Process

Additional Assessments of v5

• Conclusion? Range of differences similar 

assessment so at odds with the assertion all about the bug fixes.

• Moving on, Test No. 4 was designed to eli

documented causes of differences relative VISTAS version

– 1) bug fixes; 2) non-optional technical enhance

new default parameters for optional technical enhancements

– CALPUFF v5.711c/CALMET v5.53c2 vs. curre

version (CALPUFF v5.756/CALMET v5.726), wmodified inputs to override new default param
CALPUFF Update Process

Additional Assessments of v5

• Results from Test No. 4 - range of percent 

differences of high ranked values across asources and averaging periods by scenari

– Scenario 1: -17.7 to +0.6– Scenario 2: -20.9 to +10.3– Scenario 3: -0.4 to +5.1– Scenario 4: No differences– Scenario 5: No differences– Scenario 6: No differences– Scenario 7: No differences– Scenario 8: No differences– Scenario 9: No differences– Scenario 10: No differences
CALPUFF Update Process

Additional Assessments of v5

• Conclusion? Some progress made (no dif

for Scenarios 4 – 10), but significant unresdifferences remain for Scenarios 1 – 3

• While range of differences is smaller than 

tests, much larger differences are found adomain for LRT distances

• The following example shows one source 

Scenario 2
Figure 1: Contour Plot of Percent Differences for Scenario 2

Jordan Valley 65m Point Source; 4th-Highest 24-Hour Averages

Test 4 - CALPUFF (5.711c) and CALMET (5.53c2) vs. VISTAS (5.756/5.726) 

(with modified inputs to override new default parameter)

0 100 200 300

-200-100

0

100200
CALPUFF Update Process

Additional Assessments of v5

• Given results from Tests 1 through 4, addi

tests were performed to further isolate diffdue to each of the three factors contributindifferences

• Results from Tests 5 through 7 are shown 

following table as the range of high ranked by scenario
CALPUFF Update Process

Additional Assessments of v5

Table 1. Range of Maximum Absolute Percent Differences by Sce

 Test 5 – Bug Fixes 

Test 6 – Non-Optional 

Technical Enhancements 

Test 7 

P

Scenario 1 -1.0 to +22.6 -18.6 to +1.9 Scenario 2 -15.4 to +27.8 -30.5 to +0.7 Scenario 3 -5.1 to +6.0 -0.8 to +1.7 Scenario 4 -18.4 to +8.4 -0.7 to +4.2 Scenario 5 -32.4 to +9.5 -9.2 to +0.6 Scenario 6 -1.4 to +0.6 -2.0 to +6.3 Scenario 7 No differences > |0.01| -1.7 to +1.0 Scenario 8 -1.4 to +0.6 -2.0 to +3.1 Scenario 9 No differences > |0.01| -1.2 to +1.0 No diScenario 10 No differences No differences 
CALPUFF Update Process

Additional Assessments of v5

• Conclusions? 

– All three factors cause significant differences– New default parameters appear to cause the l

differences overall

– Differences vary significantly across different s

and source types, with no significant overall bi

• Reminder: These summaries of difference

high ranked values are only the tip of the iand may significantly understate magnituddifferences for LRT portions of domain
CALPUFF Update Process

Additional Assessments of v5

• Given the unresolved differences found in 

EPA sent a copy of Test 4 to TRC on Apri

• TRC identified four (4) additional bug fixes 

(2) additional non-optional technical enhanin the VISTAS version that were not includprevious interim versions “c” and “c2”

• EPA received a DVD from TRC on May 11

containing test results showing no differenacross all scenarios once these additional were incorporated
CALPUFF Update Process

Additional Assessments of v5

• TRC also sent a new version of CALMET, 

– Level 070501, containing the six (6) addchanges

• EPA has requested that TRC provide a ne

5.53c of CALMET containing only the four fixes in order to isolate differences due to category of change

• Once new version 5.53c – Level 07???? o

CALMET is received, at a minimum EPA wTests 5 through 7 to document impact of ecategory
CALPUFF Update Process

Additional Assessments of v5

• Beyond all of the tests and multiple versio

CALMET and CALPUFF, EPA has serious technical concerns regarding how the optitechnical enhancements for mixing height been implemented in CALMET

• These concerns are related to the new de

parameters THRESHL and THRESHW for threshold buoyancy energy flux over land water, respectively

• According to TRC, these new thresholds w

implemented as part of the MMS-sponsoreenhancements, to prevent unlimited growtconvective boundary layer over water
CALPUFF Update Process

Additional Assessments of v5

• These new thresholds are defined (in the 

documentation available) as the “threshold buoyancy flux required to sustain convectiheight growth”

• However, the code indicates that when the 

heat flux falls below the threshold, the conmixing height is immediately assigned to 0that grid cell, eliminating convective turbul

• A new convective boundary layer may form 

subsequent hours
CALPUFF Update Process

Additional Assessments of v5

• This behavior is masked somewhat by oth

defaults within CALMET, including the defminimum mixing height of 50m, the use of maximum of convective and mechanical mheights as the overall mixing height (similaAERMOD), and the default option for upwaveraging of mixing heights 

• The default threshold is 0.05 W/m

2/m. Fo

example, if Zic = 3000m at 2pm, and H = 1

at 3pm, then Zic = 0m at 3pm. A new CBL likely form at 4pm.

• The following figures from Scenario 4 illus

behavior
Convective Mixing Heights for Scenario 4 – 10/18/1992 Ho
Convective Mixing Heights for Scenario 4 – 10/18/1992 Ho
Convective Mixing Heights for Scenario 4 – 10/18/1992 Ho
Convective Mixing Heights for Scenario 4 – 10/18/1992 Ho
Convective Mixing Heights for Scenario 4 – 10/18/1992 Ho
Current Options for 

Regulatory Approval

• Approval of latest VISTAS version “a

is simply NOT an option at present s

• Step 1: Approve “de-bugged” versio

5.711c

– Addressed main concern by all which is 

of version with bugs & have understandimplications

• Step 2: Continue process to approve 

latest VISTAS version with pre-set regulatory defaults by resolving issueproblems w/ implementation of tech enhancements
How we expect to procee

• Step 1: Approve “debugged” version 5.7

Level 07???? 

– Replicate and document review and apply up

to revised versions from TRC

• Step 2: Engage with model developer an

community on appropriateness of technicenhancements

– Approval process will include revised MOU re

responsibilities and expectations of model dewrt regulatory version and interaction with uscommunity
Lessons Learned