Long Range Transport ModelEvaluationsReassessment of InteragencyWorkgroup on Air Quality Modeling(IWAQM) RecommendationsBackground• 8thModeling Conference – EPA recognizesCALPUFF model science has evolved anddiscusses intent to update InteragencyWorkgroup on Air Quality Modeling (IWAQM)Phase 2 Guidance– Formed CALPUFF Workgroup in 2005 to discussissues and prioritize tasks– Discussed need to perform updated modelperformance evaluationto examine new scienceenhancements to model (e.g. puff splitting, P-G v.turbulence, etc.) which are not reflected in currentguidance.LRT Model Evaluation Project• EPA is performing five tasks for this project:– Assemble a tracer and meteorological database for usewith LRT model evaluations– Develop a comprehensive evaluation framework(methodologies and tools) for both meteorological(prognostic and diagnostic) and LRT models for promotionof more scientifically robust and consistent evaluation ofmeteorological and LRT models– Exercising and testing meteorological and LRT models forthe assembled tracer database– Provide full documentation of model evaluation measuresand results from meteorological and LRT evaluations– Updating existing EPA LRT modeling guidance (IWAQMPhase 2) to reflect lessons learned from this projectIWAQM Guidance Goals• Examine science evolution of CALPUFF modelingsystem to incorporate recent enhancements to modelsystem in updated guidance. Overarching questions:1. Can puff-splitting extend the effective range ofCALPUFF beyond recommended distance of 200-300km?2. Can guidance migrate to recommend turbulencebased dispersion (CALPUFF and AERMOD options)over P-G?3. How best to supply meteorological data to CALPUFF?(Best CALMET Options, MM5CALPUFF, etc.)Mesoscale Tracer Experiments• Great Plains Tracer Experiment (GP80) –1980 (currently evaluating)• Savannah River Laboratory TracerExperiment (SRL) – 1975 (underway)• Cross-Appalachian Tracer Experiment(CAPTEX) – 1983 (TBD)• European Tracer Experiment (ETEX) –1994 (completed)Additional Tracer Studies to BeIncluded• Project MOHAVE (Vimont, Koracin)• VTMX (Urban 2000) (?)• Dipole Pride 26 (Hanna & Chang)• Overland Along-Wind Dispersion (OLAD)(Hanna & Chang)Evaluation of Additional Models• Evaluation of additional modeling technology to aidunderstanding of additional LRT modeling and to providea framework to understand how well can any modelreasonably do with these experiments– Lagrangian particle models (models routinelyavailable with wide meteorological compatibility)• FLEXPART (Norwegian Institute for Air Research)• HYSPLIT (Air Resources Laboratory, NOAA)– Eulerian grid models• Comprehensive Air Quality Model with Extensions(CAMx)Evaluation Methods• Irwin Method– Plume arrival– Arc transit time– Fitted maximum– Observed maximum– Crosswind integratedconcentration– Plume sigma-y– Plume Azimuth• ATMES-II statistical measures– Scatter• Normalized Mean SquareError• Factor of Exceedance• Factor of 2/5• Correlation– Bias• Fractional Bias• Mean Bias– Spatial• Figure of Merit in Space• Probability of Detection• False Alarm Rate• Threat Score– Cumulative distribution• Kolomogorov-SmirnovParameter– Overall Model RankGreat Plains Mesoscale Tracer Experiment• One of several originalEPA tracer study forCALPUFF,documented in EPA454/R-98-009• Two perflourocarbaontracer releases fromNorman, OK July 8and July 11, 1980.Synoptic Meteorology – July 8-9, 1980Model Experiment Design-500 -400 -300 -200 -100 0 100 200 300 400 500 600 700-500-400-300-200-1000100200300400500• Three Dispersion Models1. CALPUFF:– CALMET Meteorology• Observation only• Hybrid (MM5 + Obs)• NOOBS=1 (Obs surface, MM5 aloft)• NOOBS=2 (No Observations, onlyMM5)– MM5CALPUFF Meteorology– All Dispersion Options• P-G Turbulence• AERMOD Turbulence• CALPUFF Turbulence– Puff-Splitting on 600 km simulation, none for100 km– 100 km and 600 km arcs of receptors, 0.25°increments, 361 total receptors for each.– 2 Domains: 100km, 600 km arcs– 20 km CALMET for 600 km– 4 km CALMET for 100 km2. FLEXPART Lagrangian Particle Model3. HYSPLIT Lagrangian Particle ModelModel Receptor ArcsMM5 Configuration• MM5 Version 3.7.4• 3 nested domains– 108 km– 36 km– 12 km• 34 vertical layers• ICBC: NCEP/NCARReanalysis Dataavailable every 6hours at 2.5° x 2.5°• Physics:– Kain-Fritsch IICumulus– ETA PBL– NOAH LSM– RRTM Radiation– Simple IceMicrophysicsObserved MaximumAERMODCALPUFFP-GCALPUFF Dispersion OptionObservedOmax (ppt) – actual maximum0.16380.17080.15610.38Great Plains, July 8, 600 km, MM5 Winds0.16890.16860.16470.38Great Plains, July 8, 600 kmCALMET NOOBS=2 Winds0.16480.16680.15930.38Great Plains, July 8, 600 kmCALMET NOOBS=1 Winds0.13470.13310.14890.38Great Plains, July 8, 600 kmCALMET Hybrid Winds0.15660.15430.867E-10.38Great Plains, July 8, 600 kmCALMET Obs Only Winds2.252.271.781.05Great Plains, July 8, 100 km, MM5 Winds**2.0141.9462.3621.05Great Plains, July 8, 100 km, CALMET**Winds** Round 1 Results – will be updatedPlume Sigma-yCALPUFF Dispersion OptionAERMODCALPUFFP-GObserved33.9235.4243.8519.5Great Plains, July 8, 600 km, MM5 Winds28.3127.9248.6619.5Great Plains, July 8, 600 kmCALMET NOOBS=2 Winds23.9623.6736.7319.5Great Plains, July 8, 600 kmCALMET NOOBS=1 Winds34.5935.1852.919.5Great Plains, July 8, 600 kmCALMET Hybrid Winds30.4630.9152.219.5Great Plains, July 8, 600 km,CALMET Obs Only Winds6.66.68.39.1Great Plains, July 8, 100 km,MM5 Winds6.46.46.39.1Great Plains, July 8, 100 km, CALMETWindsCmax (ppt) – fitted maximumPlume Arrival Time on Arc191:0300191:0300191:0200191:0200Great Plains, July 8, 600 km,MM5 Winds191:0200191:0200191:0200191:0200Great Plains, July 8, 600 kmCALMET NOOBS=2 Winds191:0100191:0200191:0100191:0200Great Plains, July 8, 600 kmCALMET NOOBS=1 Winds191:0200191:0200191:0100191:0200Great Plains, July 8, 600 kmCALMET Hybrid Winds191:0200191:0100191:0200191:0200Great Plains, July 8, 600 kmCALMET Obs Only Winds190:1700190:1700190:1700190:1545Great Plains, July 8, 100 km,MM5 Winds190:1600190:1600190:1600190:1545Great Plains, July 8, 100 km,CALMET WindsAERMODCALPUFFP-GObservedCALPUFF Dispersion OptionArrival Time at Arc(Julian day: hour LST)Arc Transit TimeAERMODCALPUFFP-GCALPUFF Dispersion OptionObservedLength of Plume Passage (hours)14151515Great Plains, July 8, 600 km, MM5 Winds66615Great Plains, July 8, 600 kmCALMET NOOBS=2 Winds66615Great Plains, July 8, 600 kmCALMET NOOBS=1 Winds88915Great Plains, July 8, 600 kmCALMET Hybrid Winds88815Great Plains, July 8, 600 kmCALMET Obs Only Winds4445Great Plains, July 8, 100 km,MM5 Winds5555Great Plains, July 8, 100 km, CALMETWindsPlume CenterlineAERMODCALPUFFP-GCALPUFF Dispersion OptionObservedPlume Centerline (degrees from north)17.8017.4417.99.8Great Plains, July 8, 600 km, MM5 Winds28.3128.0120.159.8Great Plains, July 8, 600 kmCALMET NOOBS=2 Winds25.1225.0125.039.8Great Plains, July 8, 600 kmCALMET NOOBS=1 Winds26.7226.5225.919.8Great Plains, July 8, 600 kmCALMET Hybrid Winds26.5726.6329.599.8Great Plains, July 8, 600 kmCALMET Obs Only Winds-8.9-8.78-11.331Great Plains, July 8, 100 km,MM5 Winds25.1525.0428.621Great Plains, July 8, 100 km, CALMETWindsCALPUFF SimulationsOBS ONLY – AERMOD DISPERSIONMM5 ONLY – AERMOD DISPERSIONEuropean Tracer Experiment(ETEX)• ETEX initiated in 1992 by theEuropean Commission (EC),International Atomic Energy Agency(IAEA), and World MeteorologicalOrganization (WMO) to address manyof the questions that arose from the1986 Chernobyl accident regarding thedevelopment of LRT models• ETEX was designed to validate longrange transport models used foremergency response situations and todevelop a database which could beused for model evaluation purposes.• Two releases of perflourocarbon (PFC)tracer were made in October andNovember 1994 from France.– The PFC was released at aconstant rate for 12 hours– Air concentrations were sampledat 168 monitoring sites in 17European countries with asampling frequency of every 3hours for approximately 90 hours.A01A02A03A04A05A06A07A08B01B02B03B04B05BG01BG02CH01CH02CH03CH04CR01CR02CR03CR04D01D02D03D04D05D06D07D08D09D10D11D12D13D14D15D16D17D18D19D20D21D22D23D24D25D26D27D28D29D30D31D32D33D34D35D36D37D38D39D40D41D42D43D44D45D46D47DK01DK02DK03DK04DK05DK06DK07DK08DK09DK10DK11F01F02F03F04F05F06F07F08F09F10F11F12F13F14F15F16F17F18F19F20F21F22F23F24F25F26F27H01H02H03H04N01N02N04N05N06N07NL01NL02NL03NL04NL05NL06NL07NL08NL09PL01PL02PL03PL04PL05PL06PL07PL08R01R02S01S02S03S04S05S06S07S08S09S10S11S12S13S14S15S16S17S18SF01SF02SR01SR02SR03SR04UK01UK02UK03UK04UK05UK06UK07Experiment Design• Meteorology– MM5 Version 3.74• Initialized with NCEPReanalysis Data (available 6-hourly at 2.5 x 2.5 grid)• Kain-Fritsch II Cumulus• ETA PBL• NOAH LSM• RRTM Radiation• LRT Models– CALPUFF Version 5.8– HYSPLIT Version 4.8– FLEXPART Version 6.2(MM5)• Each model coupled directly toMM5 to facilitate betterintercomparisonFLEXPART Time Series24 HR48 HR36 HR60 HRCALPUFF Time Series00.010.020.050.10.20.512510205000.010.020.050.10.20.512510205000.010.020.050.10.20.512510205000.010.020.050.10.20.512510205024 HR 36 HR48 HR 60 HRHYSPLIT Time Series010205010020050010002000500010000200005000001020501002005001000200050001000020000500000102050100200500100020005000100002000050000010205010020050010002000500010000200005000024 HR 36 HR48 HR 60 HRSpatial Statistics: ETEX0.200.540.2730.07FLEXPART0.420.360.5541.4HYSPLIT0.060.740.078.35CALPUFFThreat Score(TS)False AlarmRate(FAR)Probability ofDetection(POD)Figure of Meritin Space(FMS)Statistical MeasureModelGlobal Statistics: ETEX1.241.780.66RANK49.0039.0079.00KSP-16.61-10.65-37.48FOEX11.8623.441.99FA55.3410.050.5FA2108.0618.12202.12NMSE1.200.691.27FB-0.010.320.02PCCFLEXPARTHYSPLITCALPUFFStatisticalMeasureInitial Observations• Insufficient number of tracer experiments to draw any conclusionsfrom current data• Great Plains Tracer Experiment– CALPUFF/CALMET 100 km results performed well except for plumeazimuth. MM5 results better for azimuth, but worse for time of arrivaland duration on 100 km arc.• Unable to replicate 600 km arc statistics from original GP80 and SRLstudies conducted by EPA in 1997 despite using same rawmeteorological data, horizontal, andvertical grid configurations.• Two major differences from original EPA study are updated terrain andlanduse from old CALPUFF 1.0 distribution and use of lambert conformalprojection for GP80 and SRL, all other CALMET options remainedconstant.• CALPUFF performance varied due to variations in CALMET optionsselected. CALPUFF results appearsensitive to manner in whichmeteorology is supplied to the model.• European Tracer Experiment– CALPUFF performs reasonably compared to particle models for first24 hours, has more difficulty further into transport simulation– Puff-splitting did not change CALPUFF performance significantly.• Additional sensitivities for puff-splitting (eliminating mixing heightrestrictions) increased number ofpuffs, but did not augment modelperformance.Next Steps• Project results shown today are work-inprogress– Rebuilding database and simulation results aftercritical hard drive failure in early summer– Model evaluation protocol drafted– Full documentation and data availability necessary.– Need to engage with model developer to help usunderstand some of our observations• Did we go wrong in model setup? What can we do better?• Has the model changed since the previous evaluations?