**Long Range Transport Model Evaluations** **Reassessment of Interagency Workgroup on Air Quality Modeling** **(IWAQM) Recommendations Background** **• 8 th** **Modeling Conference – EPA recognizes CALPUFF model science has evolved and** **discusses intent to update Interagency Workgroup on Air Quality Modeling (IWAQM)** **Phase 2 Guidance – Formed CALPUFF Workgroup in 2005 to discuss** **issues and prioritize tasks – Discussed need to perform updated model** **performance evaluation to examine new science enhancements to model (e.g. puff splitting, P-G v.** **turbulence, etc.) which are not reflected in curren t** **guidance. LRT Model Evaluation Project** **• EPA is performing five tasks for this project: – Assemble a tracer and meteorological database for** **use with LRT model evaluations** **– Develop a comprehensive evaluation framework (methodologies and tools) for both meteorological** **(prognostic and diagnostic) and LRT models for prom otion** **of more scientifically robust and consistent evalua tion of** **meteorological and LRT models – Exercising and testing meteorological and LRT mode** **ls for the assembled tracer database** **– Provide full documentation of model evaluation mea sures** **and results from meteorological and LRT evaluations – Updating existing EPA LRT modeling guidance (IWAQM** **Phase 2) to reflect lessons learned from this proje ct** **IWAQM Guidance Goals • Examine science evolution of CALPUFF modeling** **system to incorporate recent enhancements to model system in updated guidance. Overarching questions:** **1. Can puff-splitting extend the effective range of CALPUFF beyond recommended distance of 200-300** **km? 2. Can guidance migrate to recommend turbulence** **based dispersion (CALPUFF and AERMOD options) over P-G?** **3. How best to supply meteorological data to CALPUFF ?** **(Best CALMET Options, MM5CALPUFF, etc.) Mesoscale Tracer Experiments** **• Great Plains Tracer Experiment (GP80) – 1980 (currently evaluating)** **• Savannah River Laboratory Tracer Experiment (SRL) – 1975 (underway)** **• Cross-Appalachian Tracer Experiment (CAPTEX) – 1983 (TBD)** **• European Tracer Experiment (ETEX) – 1994 (completed)** **Additional Tracer Studies to Be Included** **• Project MOHAVE (Vimont, Koracin) • VTMX (Urban 2000) (?)** **• Dipole Pride 26 (Hanna & Chang) • Overland Along-Wind Dispersion (OLAD)** **(Hanna & Chang) Evaluation of Additional Models** **• Evaluation of additional modeling technology to ai d** **understanding of additional LRT modeling and to pro vide** **a framework to understand how well can any model reasonably do with these experiments** **– Lagrangian particle models (models routinely available with wide meteorological compatibility)** **• FLEXPART (Norwegian Institute for Air Research) • HYSPLIT (Air Resources Laboratory, NOAA)** **– Eulerian grid models • Comprehensive Air Quality Model with Extensions** **(CAMx) Evaluation Methods** **• Irwin Method – Plume arrival** **– Arc transit time – Fitted maximum** **– Observed maximum – Crosswind integrated** **concentration – Plume sigma-y** **– Plume Azimuth • ATMES-II statistical measures** **– Scatter • Normalized Mean Square** **Error • Factor of Exceedance** **• Factor of 2/5 • Correlation** **– Bias • Fractional Bias** **• Mean Bias – Spatial** **• Figure of Merit in Space • Probability of Detection** **• False Alarm Rate • Threat Score** **– Cumulative distribution • Kolomogorov-Smirnov** **Parameter – Overall Model Rank** **Great Plains Mesoscale Tracer Experiment • One of several original** **EPA tracer study for CALPUFF,** **documented in EPA 454/R-98-009** **• Two perflourocarbaon tracer releases from** **Norman, OK July 8 and July 11, 1980.** **Synoptic Meteorology – July 8-9, 1980 Model Experiment Design** **-500 -400 -300 -200 -100 0 100 200 300 400 500 600 700 -500** **-400 -300** **-200 -100** **0 100** **200 300** **400 500** **• Three Dispersion Models 1. CALPUFF:** **– CALMET Meteorology • Observation only** **• Hybrid (MM5 + Obs) • NOOBS=1 (Obs surface, MM5 aloft)** **• NOOBS=2 (No Observations, only MM5)** **– MM5CALPUFF Meteorology – All Dispersion Options** **• P-G Turbulence • AERMOD Turbulence** **• CALPUFF Turbulence – Puff-Splitting on 600 km simulation, none for** **100 km – 100 km and 600 km arcs of receptors, 0.25°** **increments, 361 total receptors for each. – 2 Domains: 100km, 600 km arcs** **– 20 km CALMET for 600 km – 4 km CALMET for 100 km** **2. FLEXPART Lagrangian Particle Model 3. HYSPLIT Lagrangian Particle Model** Model Receptor Arcs**MM5 Configuration • MM5 Version 3.7.4** **• 3 nested domains – 108 km** **– 36 km – 12 km** **• 34 vertical layers • ICBC: NCEP/NCAR** **Reanalysis Data available every 6** **hours at 2.5° x 2.5° • Physics:** **– Kain-Fritsch II Cumulus** **– ETA PBL – NOAH LSM** **– RRTM Radiation – Simple Ice** **Microphysics Observed Maximum** AERMODCALPUFFP-GCALPUFF Dispersion OptionObservedOmax (ppt) – actual maximum0.16380.17080.15610.38Great Plains, July 8, 600 km, MM5 Winds0.16890.16860.16470.38Great Plains, July 8, 600 kmCALMET NOOBS=2 Winds0.16480.16680.15930.38Great Plains, July 8, 600 kmCALMET NOOBS=1 Winds0.13470.13310.14890.38Great Plains, July 8, 600 kmCALMET Hybrid Winds0.15660.15430.867E-10.38Great Plains, July 8, 600 kmCALMET Obs Only Winds2.252.271.781.05Great Plains, July 8, 100 km, MM5 Winds**2.0141.9462.3621.05Great Plains, July 8, 100 km, CALMET**Winds** Round 1 Results – will be updated**Plume Sigma-y**CALPUFF Dispersion OptionAERMODCALPUFFP-GObserved33.9235.4243.8519.5Great Plains, July 8, 600 km, MM5 Winds**28.31 27.92** **48.66**19.5Great Plains, July 8, 600 kmCALMET NOOBS=2 Winds**23.96 23.67** **36.73**19.5Great Plains, July 8, 600 kmCALMET NOOBS=1 Winds**34.59 35.18** **52.9**19.5Great Plains, July 8, 600 kmCALMET Hybrid Winds**30.46 30.91** **52.2**19.5Great Plains, July 8, 600 km,CALMET Obs Only Winds6.66.68.39.1Great Plains, July 8, 100 km,MM5 Winds6.46.46.39.1Great Plains, July 8, 100 km, CALMETWindsCmax (ppt) – fitted maximum**Plume Arrival Time on Arc**191:0300191:0300191:0200191:0200Great Plains, July 8, 600 km,MM5 Winds191:0200191:0200191:0200191:0200Great Plains, July 8, 600 kmCALMET NOOBS=2 Winds191:0100191:0200191:0100191:0200Great Plains, July 8, 600 kmCALMET NOOBS=1 Winds191:0200191:0200191:0100191:0200Great Plains, July 8, 600 kmCALMET Hybrid Winds191:0200191:0100191:0200191:0200Great Plains, July 8, 600 kmCALMET Obs Only Winds190:1700190:1700190:1700190:1545Great Plains, July 8, 100 km,MM5 Winds190:1600190:1600190:1600190:1545Great Plains, July 8, 100 km,CALMET WindsAERMODCALPUFFP-GObservedCALPUFF Dispersion OptionArrival Time at Arc(Julian day: hour LST)**Arc Transit Time**AERMODCALPUFFP-GCALPUFF Dispersion OptionObservedLength of Plume Passage (hours)14151515Great Plains, July 8, 600 km, MM5 Winds66615Great Plains, July 8, 600 kmCALMET NOOBS=2 Winds66615Great Plains, July 8, 600 kmCALMET NOOBS=1 Winds88915Great Plains, July 8, 600 kmCALMET Hybrid Winds88815Great Plains, July 8, 600 kmCALMET Obs Only Winds4445Great Plains, July 8, 100 km,MM5 Winds5555Great Plains, July 8, 100 km, CALMETWinds**Plume Centerline**AERMODCALPUFFP-GCALPUFF Dispersion OptionObservedPlume Centerline (degrees from north)17.8017.4417.99.8Great Plains, July 8, 600 km, MM5 Winds28.3128.0120.159.8Great Plains, July 8, 600 kmCALMET NOOBS=2 Winds25.1225.0125.039.8Great Plains, July 8, 600 kmCALMET NOOBS=1 Winds26.7226.5225.919.8Great Plains, July 8, 600 kmCALMET Hybrid Winds26.5726.6329.599.8Great Plains, July 8, 600 kmCALMET Obs Only Winds-8.9-8.78-11.331Great Plains, July 8, 100 km,MM5 Winds25.1525.0428.621Great Plains, July 8, 100 km, CALMETWinds**CALPUFF Simulations**OBS ONLY – AERMOD DISPERSION MM5 ONLY – AERMOD DISPERSION**European Tracer Experiment (ETEX)** **• ETEX initiated in 1992 by the European Commission (EC),** **International Atomic Energy Agency (IAEA), and World Meteorological** **Organization (WMO) to address many of the questions that arose from the** **1986 Chernobyl accident regarding the development of LRT models** **• ETEX was designed to validate long range transport models used for** **emergency response situations and to develop a database which could be** **used for model evaluation purposes. • Two releases of perflourocarbon (PFC)** **tracer were made in October and November 1994 from France.** **– The PFC was released at a constant rate for 12 hours** **– Air concentrations were sampled at 168 monitoring sites in 17** **European countries with a sampling frequency of every 3** **hours for approximately 90 hours. A01** **A02 A03** **A04 A05** **A06 A07** **A08 B01** **B02 B03** **B04 B05** **BG01 BG02** **CH01 CH02** **CH03 CH04** **CR01 CR02** **CR03 CR04** **D01 D02** **D03 D04** **D05 D06** **D07 D08** **D09 D10** **D11 D12** **D13 D14** **D15 D16** **D17 D18** **D19 D20** **D21 D22** **D23 D24** **D25 D26** **D27 D28** **D29 D30** **D31 D32** **D33 D34** **D35 D36** **D37 D38** **D39 D40** **D41 D42** **D43 D44** **D45 D46** **D47 DK01** **DK02 DK03** **DK04 DK05** **DK06 DK07** **DK08 DK09** **DK10 DK11** **F01 F02** **F03 F04** **F05 F06** **F07 F08** **F09 F10** **F11 F12** **F13 F14** **F15 F16** **F17 F18** **F19 F20** **F21 F22** **F23 F24** **F25 F26** **F27 H01** **H02 H03** **H04 N01** **N02 N04** **N05 N06** **N07 NL01** **NL02 NL03** **NL04 NL05** **NL06 NL07** **NL08 NL09** **PL01 PL02** **PL03 PL04** **PL05 PL06** **PL07 PL08** **R01 R02** **S01 S02** **S03 S04** **S05 S06** **S07 S08** **S09 S10** **S11 S12** **S13 S14** **S15 S16** **S17 S18** **SF01 SF02** **SR01 SR02** **SR03 SR04** **UK01 UK02** **UK03 UK04** **UK05 UK06** **UK07 Experiment Design** **• Meteorology – MM5 Version 3.74** **• Initialized with NCEP Reanalysis Data (available 6-** **hourly at 2.5 x 2.5 grid) • Kain-Fritsch II Cumulus** **• ETA PBL • NOAH LSM** **• RRTM Radiation • LRT Models** **– CALPUFF Version 5.8 – HYSPLIT Version 4.8** **– FLEXPART Version 6.2 (MM5)** **• Each model coupled directly to MM5 to facilitate better** **intercomparison FLEXPART Time Series** 24 HR48 HR36 HR60 HR**CALPUFF Time Series 0** **0.01 0.02** **0.05 0.1** **0.2 0.5** **1 2** **5 10** **20 50** **0 0.01** **0.02 0.05** **0.1 0.2** **0.5 1** **2 5** **10 20** **50 0** **0.01 0.02** **0.05 0.1** **0.2 0.5** **1 2** **5 10** **20 50** **0 0.01** **0.02 0.05** **0.1 0.2** **0.5 1** **2 5** **10 20** **50**24 HR 36 HR48 HR 60 HR**HYSPLIT Time Series 0** **10 20** **50 100** **200 500** **1000 2000** **5000 10000** **20000 50000** **0 10** **20 50** **100 200** **500 1000** **2000 5000** **10000 20000** **50000 0** **10 20** **50 100** **200 500** **1000 2000** **5000 10000** **20000 50000** **0 10** **20 50** **100 200** **500 1000** **2000 5000** **10000 20000** **50000**24 HR36 HR48 HR60 HR**Spatial Statistics: ETEX**0.200.540.2730.07FLEXPART0.420.360.5541.4HYSPLIT0.060.740.078.35CALPUFFThreat Score(TS)False AlarmRate(FAR)Probability ofDetection(POD)Figure of Meritin Space(FMS)Statistical MeasureModel**Global Statistics: ETEX**1.241.780.66RANK49.0039.0079.00KSP-16.61-10.65-37.48FOEX11.8623.441.99FA55.3410.050.5FA2108.0618.12202.12NMSE1.200.691.27FB-0.010.320.02PCCFLEXPARTHYSPLITCALPUFFStatisticalMeasure**Initial Observations • Insufficient number of tracer experiments to draw** **any conclusions from current data** **• Great Plains Tracer Experiment – CALPUFF/CALMET 100 km results performed well excep** **t for plume azimuth. MM5 results better for azimuth, but worse** **for time of arrival and duration on 100 km arc.** **• Unable to replicate 600 km arc statistics from ori ginal GP80 and SRL** **studies conducted by EPA in 1997 despite using same raw** **meteorological data, horizontal, and vertical grid configurations.** **• Two major differences from original EPA study are u pdated terrain and** **landuse from old CALPUFF 1.0 distribution and use of lambert conformal** **projection for GP80 and SRL, all other CALMET option s remained** **constant. • CALPUFF performance varied due to variations in CAL** **MET options selected. CALPUFF results appear sensitive to mann** **er in which meteorology is supplied to the model.** **• European Tracer Experiment – CALPUFF performs reasonably compared to particle m** **odels for first 24 hours, has more difficulty further into transpor** **t simulation – Puff-splitting did not change CALPUFF performance** **significantly. • Additional sensitivities for puff-splitting (elimin** **ating mixing height restrictions) increased number of puffs, but did not** **augment model performance.** **Next Steps • Project results shown today are work-in** **progress – Rebuilding database and simulation results after** **critical hard drive failure in early summer – Model evaluation protocol drafted** **– Full documentation and data availability necessary .** **– Need to engage with model developer to help us understand some of our observations** **• Did we go wrong in model setup? What can we do bet ter?** **• Has the model changed since the previous evaluatio ns?**