CALPUFF Modeling System:Science and Implementation IssuesRoger W. BrodeU.S. EPA/OAQPSAir Quality Modeling Group9thConference on Air Quality ModelingOctober 9, 2008Research Triangle Park, NCOutline• Assessment of VISTAS version of CALPUFF• Unresolved technical issues with v5.8• Near-field Clarification Memo• Some examples of complex wind situations• Discussion of technical issues and concerns relatedto CALPUFF near-field applications• Other CALPUFF implementation and science issues• Summary and conclusionsCALPUFF Update Process• Incumbent upon EPA to perform anindependent assessment of CALPUFFwhen updating to new versions• CALPUFF requires extensive assessmentand understanding of changes made;approvals made by EPA, not developer• CALPUFF Update Tool – Introduced at 8thModeling Conference in Sept 2005CALPUFF Update Tool• Compares 2 CALPUFF versions;proposed (beta) vs. current version (base)• Determines code changes and differencesin results across 10 scenarios• Provides a standardized methodology forevaluating changes in model codesAnalysisToolSourcesMeteorologyTerrainDifferences?Resolve DocumentBETABASECALPUFF Update ToolCALPUFF Update Process:Initial Assessments of v5.756Table 1: Range of Maximum Absolute PercentDifferences by Scenario for Initial Application ofAssessment ToolScenario 1 -22.0 to +23.3Scenario 2 -0.2 to +83.4Scenario 3 -17.7 to +60.8Scenario 4 -13.6 to +28.1Scenario 5 -46.0 to +21.1Scenario 6 -10.3 to +6.3Scenario 7 -1.7 to +1.0Scenario 8 -10.0 to +5.4Scenario 9 -1.2 to +1.0Scenario 10 No differencesEPA, 2008: Assessment of the “VISTAS” Version of the CALPUFF ModelingSystem, EPA-454/R-08-007, available on SCRAM website.CALPUFF Update Process:Additional Assessments of v5.756• TRC provided the following interim versions,to facilitate isolating impacts due to (1) bugfixes and (2) non-optional technicalenhancements:– CALMET v5.53c = CALMET v5.53a + bug fixes– CALMET v5.53c2 = CALMET v5.53c + non-optionaltechnical enhancements– CALPUFF v5.711c = CALPUFF v5.711a + bug fixesCALPUFF Update Process:Additional Assessments of v5.756• EPA conducted several tests in order tocompare and isolate affects of the followingcategories of changes:– Bug fixes– Non-optional technical enhancements– New default parameters for optional technicalenhancementsCALPUFF Update Process:Summary of Significant Findings•Significant differences attributed to each of thethree known factors:–Bug Fixes–Non-optional Technical Enhancements–New Default Parameters for Optional TechnicalEnhancements•Of the three known factors, the New DefaultParameters for Optional Technical Enhancementscaused the largest differences overall•Differences varied significantly across differentscenarios and source types, with no significantoverall bias evidentCALPUFF Update Process:Summary of Differences – v5.8 FinalNo differencesNo differencesNo differencesNo differencesScenario 10No differences > |0.01|No differences > |0.01|-1.2 to +1.0No differences > |0.01|Scenario 9-1.4 to +0.6-12.1 to +5.5-2.0 to +3.1-1.4 to +0.6Scenario 8No differences > |0.01|0.0 to +0.4-1.7 to +1.0No differences > |0.01|Scenario 7-1.4 to +0.6-12.6 to +5.5-2.0 to +6.3-1.4 to +0.6Scenario 6-32.4 to +9.5-41.8 to +20.9-9.2 to +0.6-32.4 to +9.5Scenario 5-18.4 to +8.4-3.2 to +30.5-0.7 to +4.2-18.4 to +8.4Scenario 4-4.0 to +7.3-19.3 to +60.8-0.8 to +1.7-5.1 to +6.0Scenario 3-21.8 to +29.2-21.8 to +118.3-30.5 to +0.7-15.4 to +27.8Scenario 2-1.1 to +22.6-22.0 to +24.4-18.6 to +1.9-1.0 to +22.6Scenario 1Test 8 – Base 5.711a vs.Beta 5.8 (Final Test)1Test 7 – New DefaultParametersTest 6 – Non-OptionalTechnical EnhancementsTest 5 – Bug FixesRange of Maximum Absolute Percent Differencesby Scenario1Test 8 differences are due to bug fixes, but include some additional bug fixes compared to Test 5.Table 6. CALPUFF Comparison Results for Scenarios 1 - 5 for Base v5.711a vs. Beta v5.8Scenario5-0.88 -32.4365 m Point Source348.40 -4.55 -4.95.No differences in highest ranked (design) values0.6899 m Point SourceBuoyant Area SourceSource TypeArea Source Volume Source 30 m Point Source-5.10 -7.3612-21.78 -6.710.02 -0.300.32 0.12-18.37 -13.02Differences in one or more design values-0.02 0.00 -0.05 7.26 -3.982.35 0.58 25.2429.24-2.840.480.287.5922.630.120.410.201.07CALPUFF Assessment Results from Previous Update –v5.7 to 5.711aScenario 2 - 65m Point Source, Salem LocationPercent Difference - 1st Highest, 1-Hour Averages-400 -300 -200 -100 0 100 200 300 400-400-300-200-1000100200300400Figure 1: Contour Plot of Percent Differences for Scenario 2Jordan Valley 65m Point Source; 4th-Highest 24-Hour AveragesTest 4 - CALPUFF (5.711c) and CALMET(5.53c2) vs. VISTAS (5.756/5.726)(with modified inputs to override new default parameter)0 100 200 300-200-1000100200CALPUFF Update Process:Unresolved Issues with v5.8• EPA has unresolved technical concerns regardinghow the optional technical enhancements formixing height have been implemented in CALMET• These concerns are related to the new defaultparameters THRESHL and THRESHW for thethreshold buoyancy energy flux over land andwater, respectively• These new thresholds were implemented as part ofthe MMS-sponsored enhancements, to preventunlimited growth of the convective boundary layerover waterCALPUFF Update Process:Unresolved Issues with v5.8• These new thresholds are defined (in the limiteddocumentation available) as the “thresholdbuoyancy flux required to sustain convective mixingheight growth”• However, the code indicates that when the sensibleheat flux falls below the threshold, the convectivemixing height is immediately assigned to 0m forthat grid cell, eliminating convective turbulence• A new convective boundary layer may form onsubsequent hoursCALPUFF Update Process:Unresolved Issues with v5.8• This behavior is masked somewhat by otherdefaults within CALMET, including the defaultminimum mixing height of 50m, the use of themaximum of convective and mechanical mixingheights as the overall mixing height, and the defaultoption for upwind averaging of mixing heights• The default threshold is 0.05 W/m2/m.– For example, if Zic = 3000m at 2pm, and H = 149 W/m2at 3pm, then Zic = 0m at 3pm. A new CBL will likely format 4pm.• The following figures from Scenario 4 illustrate thisbehaviorCALMET v5.8 Convective Boundary LayerHeights (THRESHL = 0.05)Time series of convective mixingheight from CALMETCALPUFF Update Process:Resolution of Issues (Partial)•TRC agreed to implement the following changesin the CALPUFF modeling system code, allowingEPA to approve v5.8:–Incorporate the non-optionaltechnical enhancements under theoptional technical enhancements, removing non-optional technicalenhancements as a potential source of differences;–Incorporate a new regulatory default switch (MREG=1) in CALMETto allow the optional technicalenhancements to be included in themodel, but to require the user tooverride the default options toexercise these optional technical enhancements; and–Modify the CALPUFF model code toinclude the minimum sigma-vof 0.5 m/s over water as part ofthe regulatory default setting.CALPUFF Update Process:Summary of Differences – v5.8 FinalNo differencesNo differencesNo differencesNo differencesScenario 10No differences > |0.01|No differences > |0.01|-1.2 to +1.0No differences > |0.01|Scenario 9-1.4 to +0.6-12.1 to +5.5-2.0 to +3.1-1.4 to +0.6Scenario 8No differences > |0.01|0.0 to +0.4-1.7 to +1.0No differences > |0.01|Scenario 7-1.4 to +0.6-12.6 to +5.5-2.0 to +6.3-1.4 to +0.6Scenario 6-32.4 to +9.5-41.8 to +20.9-9.2 to +0.6-32.4 to +9.5Scenario 5-18.4 to +8.4-3.2 to +30.5-0.7 to +4.2-18.4 to +8.4Scenario 4-4.0 to +7.3-19.3 to +60.8-0.8 to +1.7-5.1 to +6.0Scenario 3-21.8 to +29.2-21.8 to +118.3-30.5 to +0.7-15.4 to +27.8Scenario 2-1.1 to +22.6-22.0 to +24.4-18.6 to +1.9-1.0 to +22.6Scenario 1Test 8 – Base 5.711a vs.Beta 5.8 (Final Test)1Test 7 – New DefaultParametersTest 6 – Non-OptionalTechnical EnhancementsTest 5 – Bug FixesRange of Maximum Absolute Percent Differencesby Scenario1Test 8 differences are due to bug fixes, but include some additional bug fixes compared to Test 5.EPA, 2008: Assessment of the “VISTAS” Version of the CALPUFF Modeling System,EPA-454/R-08-007, available on SCRAM website.CALPUFF Implementation Issues• Lack of adequate documentation– New MREG option in CALMET not well-documented; CALMET.INP sampleinput file indicates “no default” and assigns value of MREG = 0, i.e., don’tenforce regulatory defaults– Many important technical details are not documented, except in the code• Magnitude of differences between v5.8 of CALPUFF andprevious version raises questions regarding “validity” oforiginal model evaluations for CALPUFF• PG vs. turbulence dispersion option in CALPUFF– Addressed in March 2006 Model Clearinghouse Memo: not an automaticswitch with promulgation of AERMOD, requires separate assessment– Assessment is underway, but has uncovered other PG-class dependencieswith turbulence option• Pending assessment of v5.8’n’ and v6 updates– Currently “on hold” due to need to address broader issues• CALPUFF near-field Clarification MemoCALPUFF Near-fieldClarification Memorandum• Summary of Key Points:–The EPA-preferred model for near-fieldregulatory applications (less than 50kilometers) for simple andcomplex terrainis AERMOD.The AERMOD model shouldbe used for all near-field regulatory applications, unless an adequate determination ismade that AERMOD isnot appropriate for thatapplication or is clearly less appropriatethan an alternative model.[See paragraph 4.2.2(b) of Appendix W – “For a wide range ofregulatory applications in all types of terrain, the recommended model is AERMOD.”]–CALPUFF is notthe EPA-preferred model for near-field applications, but may beconsidered as an alternative model on a case-by-case basis for near-fieldapplications involving “complex winds,” subject to approval by the reviewingauthority.The approval of CALPUFF for near-fieldregulatory applications should bebased on case-specific justification, including necessary documentation and an adequatedetermination that AERMODis not appropriate or clearly less appropriate thanCALPUFF. Generalized approval of CALPUFF for near-field applications based onreference to other cases where CALPUFF has been approved for near-field use is notacceptable, unless suchcases are similar enough to theapplication under review to beapplicable, and are adequately documented tosupport that determination.[Seeparagraph 7.2.8(a) of Appendix W – “the CALPUFF modeling system(described inAppendix A) may be applied on a case-by-casebasis for air quality estimates in suchcomplex non-steady-state meteorological conditions.”]CALPUFF Near-fieldClarification Memorandum• From Preamble to April 2003 FR Noticepromulgating CALPUFF:–“We will require approval to be obtained prior toaccepting CALPUFF for complex wind situations, as thiswill ensure that a protocol is agreed to between theparties involved, and that all are willing to accept theresults as binding. As experience is gained in usingCALPUFF for complex wind situations, acceptance willbecome clear and those cases that are problematic willbe better identified.” (pp. 18441-2)CALPUFF Near-fieldClarification Memorandum• From Section 7.2.8 of Appendix W:–“The setup and application of the model should bedetermined in consultation with the appropriate reviewingauthority (paragraph 3.0(b)) consistent with limitations ofparagraph 3.2.2(e).”–Reference to paragraph 3.2.2(e) places CALPUFF in thestatus of an alternative model.CALPUFF Near-fieldClarification Memorandum• Paragraph 3.2.2(e) of Appendix W:“e. Finally, for condition (3) inparagraph (b) of this subsection[preferred model is less appropriatefor the specific application, orthere is no preferred model], analternative refined model may beused provided that:i. The model has received a scientific peer review;ii. The model can be demonstrated to be applicable to the problem on atheoretical basis;iii. The data bases which are necessary to perform the analysis areavailable and adequate;iv. Appropriate performance evaluations of the model have shown thatthe model is not biased toward underestimates; andv. A protocol on methods and procedures to be followed has beenestablished.”CALPUFF Near-fieldClarification Memorandum• Summary of main steps:1) a determination that treatment ofcomplex winds is critical toestimating design concentrations;2) a determination that the preferredmodel is not appropriate or lessappropriate than CALPUFF; and3) a demonstration that the five criteria listed in paragraph 3.2.2(e)for use of an alternative model are adequately addressed.• Each of these steps involves case-specificconsiderationsCALPUFF Near-fieldClarification Memorandum• Importance of consistency, spelled out inparagraph 1(b) of the Guideline:“The model that most accurately estimates concentrations in thearea of interest isalways sought. However,it is clear from the needsexpressed by the States andEPA Regional Offices, by many industriesand trade associations, and also by thedeliberations of Congress, thatconsistency in the selection and application ofmodels and data bases should alsobe sought, even in case-by-caseanalyses. Consistency ensures that air quality control agencies and the generalpublic have a common basis for estimating pollutant concentrations, assessingcontrol strategies and specifying emission limits. Suchconsistency is not, however,promoted at the expense of model and data base accuracy. TheGuidelineprovides a consistent basis for selection of the most accurate models and databases for use in air quality assessments.”Examples of Complex Winds• Examples of complex winds are described inparagraph 7.2.8(a) of Appendix W:– “a.Inhomogeneous Local Winds.In many parts of theUnited States, the ground is neither flat nor is the groundcover (or land use) uniform. These geographical variationscan generate local winds and circulations, and modify theprevailing ambient winds and circulations. Geographiceffects are most apparent when the ambient winds arelight or calm. In general these geographically induced windcirculation effects are named after the source location ofthe winds, e.g., lake and sea breezes, and mountain andvalley winds. In very rugged hilly or mountainous terrain,along coastlines, or near large land use variations, thecharacterization of the winds is a balance of variousforces, such that the assumptions of steady-state straightline transport both in time and space are inappropriate.”Examples of Complex Winds• Valley flows– Down-slope/down-valley flows under light windstable conditions– Cross-valley circulations due to differential heatingunder convective conditions– Valley channeling, may be driven by differentconditions– Stagnation conditions– Significant horizontal and vertical discontinuities inwind, temperature, etc.– Grid resolution and availability of representative metdata may be significant issuesSlope Flows• Part of local diurnalwind pattern that isthermally induced.• Formation is favoredunder weak synopticpressure gradients.• Depends uponsurface temperaturecontrasts due todaytime heating andnighttime cooling.Mountain-Plain Circulations• Mountain-Plaincirculations also developdue to thermal effects.• Rising air over mountainsas upslope winds developcreate a weak pressuregradient which causes airfrom the plains to flowtoward the mountain tocompensate for the rising• At night, the circulationreverses, and the windsflow from the mountainsonto the plains.Nocturnal Thermal Structure• As air over mountain slopescools, it becomes more densethan its surroundingenvironment and begins todrain down the gradient of theterrain slopes.• Cold air accumulates in thevalley basin and creates atemperature inversion at thetop of the valley.• Air within valley is stablystratified and pollution canbecome trapped andaccumulate.Examples of Complex Winds• Coastal influences– Land/sea-breeze (lake-breeze) circulations drivenby differential heating between land and water– Thermal internal boundary layer (TIBL) near coastduring daytime onshore flow– Grid resolution and representative met data maybe significant issues– TIBL effects most important for elevated releasesnear the coast, and may be addressed throughsubgrid-scale modulesComplex Winds - Issues• General issues– Complex wind influences will vary significantlybased on source characteristics and location– Plume recirculation may occur under variousconditions, such as cross-valley and coastalcirculations and stagnation– Availability of representative meteorological inputsto “inform” the system– Ability of modeling system to effectively utilizesite-specific information– Model performance/uncertainty for spatiotemporalpairing of model results is not well documented orunderstoodComplex Winds - Issues• More specific issues– Near-field, localized plume terrain interactionsmay be significantly misrepresented due to gridresolution limitations– Ability of modeling system to effectively utilizesite-specific information appears to besignificantly lacking – important information maybe effectively destroyed or misrepresented– Recent Staff Memorandum, dated September 26,2008, provides additional details regarding theseissuesComplex Winds – PerformanceEvaluations• CALPUFF modeling system performance fornear-field complex wind applications is notwell-documented• IWAQM Phase 2 report includes someCALPUFF evaluation results for Kincaid (flatterrain) and Lovett (complex terrain)• Evaluation results for Lovett show goodperformanceCALPUFF Lovett Evaluation Resultsfrom IWAQM Phase 2 ReportComplex Winds – PerformanceEvaluations• However, CALPUFF was applied with CTDMPLUSmet inputs, bypassing CALMET• This is not consistent with motivation for CALPUFFnear-field applications under paragraph 7.2.8 ofAppendix W, which is to “fully treat the time andspace variations of meteorology effects on transportand dispersion.”• Therefore, these evaluation results are not relevantto near-field applications under paragraph 7.2.8 ofAppendix WCALPUFF/CALMET Evaluationsfor Lovett• Various methods, including graphical and statisticalmeasures, are used for evaluating model performance fornear-field regulatory models, such as AERMOD or theIndustrial Source Complex (ISC) models.• The Cox-Tikvart method uses the Fractional Bias (FB) as thestatistical measure of agreement between the observed andpredicted concentration;– The FB is bounded between -2 and +2, with a value of 0(zero) indicating perfect agreement. An absolute value ofless than 0.667 for FB indicates agreement within a factorof 2Near-Field EvaluationProcedure Cont’d• Fractional bias calculated from Robust HighestConcentration (RHC):• Model comparisons based on Composite PerformanceMeasure (CPM) and Model Comparison Measure (MCM)[],21-3NX(N)-X+X(N)=RHC⎥⎦⎤⎢⎣⎡ln( ),2)(AFB+)(AFB32+AFB31=CPM243s,r⎥⎦⎤⎢⎣⎡,CPMCPM=MCM)2()1(−Lovett Near-Field ComplexTerrain• CALMET 3-D Windfields Developed– 3 Level Tower (10m, 50m, 100m) simulated as 3 surface stationswith corresponding anemometer heights– Upper air data from Albany, NY– 125 meter grid spacing using SRTM-1 (terrain) and NLCD92(landuse).• Various CALPUFF configurations tested– P-G– AERMOD turbulence– Plume Half-Height Adjustment– CALPUFF Strain Based Adjustment– AERMOD surface and profile dataModel Configurations Evaluated• CALPUFF1:– P-G Dispersion– Plume Half Height Adjustment• CALPUFF2:– AERMOD Turbulence– Plume Half Height Adjustment• CALPUFF3:– P-G Dispersion– CALPUFF Strain Based Adjustment• CALPUFF4:– AERMOD Turbulence– CALPUFF Strain Based Adjustment• CALPUFF5:– AERMOD Profile Data– Plume Half Height Adjustment• AERMOD (Version 07026)Robust Highest Concentrations-2.202E-02190.79AERMOD8.778E-02170.94CALPUFF5-0.9107498.73CALPUFF4-0.1424215.26CALPUFF3-0.5597331.69CALPUFF2-0.3681270.83CALPUFF1186.63OBSERVEDFBRHCMODEL-8.16E-0352.19AERMOD0.135645.20CALPUFF5-0.528088.91CALPUFF4-0.430780.19CALPUFF33.42E-0250.03CALPUFF2-0.428279.98CALPUFF151.77OBSERVEDFBRHCMODEL3-Hour24-HourComposite Performance Measure (CPM)Model CPM+/- C.I. +/- C.I.90% 95%CALPUFF1 0.5956 0.1280 0.1526CALPUFF2 0.6005 0.1708 0.2036CALPUFF3 0.4763 0.1210 0.1442CALPUFF4 0.7884 0.2349 0.2800CALPUFF5 0.4382 0.1008 0.1202AERMOD 0.42340.0980 0.1168Model Comparison Measure(MCM)Model - Model MCM +/- C.I.90%CALPUFF1-CALPUFF2 -4.943E-03 0.3247CALPUFF1-CALPUFF3 0.1192 0.1709CALPUFF1-CALPUFF4 -0.19280.3939CALPUFF1-CALPUFF5 0.15740.3377CALPUFF1-AERMOD0.1722 0.2679CALPUFF2-CALPUFF3 0.1242 0.3195CALPUFF2-CALPUFF4 0.1879 0.4114CALPUFF2-CALPUFF5 0.16240.3563CALPUFF2-AERMOD0.1771 0.3353CALPUFF3-CALPUFF4 -0.31200.3897CALPUFF3-CALPUFF53.819E-02 0.3319CALPUFF3-AERMOD5.295E-02 0.2619CALPUFF4-CALPUFF5 0.35030.4839CALPUFF4-AERMOD 0.36500.4344CALPUFF5-AERMOD1.476E-02 0.21073-Hour Q-Q Plot - Lovett24-Hour Q-Q Plot - LovettObservations – Near FieldEvaluation• CALPUFF model performance exhibits significantsensitivity to dispersion and terrain options.• More “advanced” options (turbulence and strain-basedterrain adjustment) exhibited poorest performance.• CALPUFF with AERMOD profiles (i.e. no CALMET)performed best of CALPUFF configurations.• CAVEAT: These evaluation results are very preliminaryand will be updated based onadditional insights intotreatment of tower data in CALMETQuestions?