**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call** 

# CENRAP Modeling Update: **CMAQ versus CAMx Model Performance**  **Evaluation** Ralph Morris, Gerard Mansell, Bonyoung Koo and Abby Hoats, ENVIRON International Corporation Gail Tonnesen, Mohammed Omary, Chao - Jung Chien

and Yingqun Wang UC Riverside CENRAP Modeling Work Group Meeting Baton Rouge, Louisiana February 7, 2006**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call**



# Today's **Topics**
• Model Performance Evaluation (MPE) • CMAQ versus CAMx

– Are we ready to pick the Lead and the Corroborative Model? [wait for 2018 projection comparisons] 

### • 36 km versus 12 km Grid

– Does the 12 km grid produce improved results? • Is it worth the additional costs and time?

– Does running with 12 km met but 36 km emissions buy most of the benefits of the 12 km grid? 

### • 2018 Visibility Projections

– CMAQ 36 km only at this time 

### • Source Apportionment Plans • BART VOC/NOx and Control Strategy Modeling**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call** 

# Model Performance **Evaluation**
• CMAQ versus CAMx 36 km Grid

– CMAQ V4.4 SOAmods • Includes SOAmods update that adds SOA from sesquiterpenes, isoprene, etc .



### – CAMx V4.3beta

– Evaluate for monitoring sites in the CENRAP States • Other RPOs also examined

– on website



### – Focus on Visibility PM Species Components • SO4, NO3, OMC, EC, Soil and CM • IMPROVE, CASTNet, STN, NADP**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call** 

# Model Performance **Evaluation**
• Numerous model performance statistics and graphics have been generated:

– Stacked Bar Charts of Obs and Pred • 24 - Hour Extinction • 20% Best and 20% Worst Days

– Scatter Plots

– Time Series plots

– Monthly Bias/Error Bar Plots

– Bugle Plots

– Spatial Maps w/ Superimposed Observations 

### • **See:** 

### http://pah.cert.ucr.edu/aqm/cenrap/cmaq.shtml**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Stacked**  **Extinction Big Bend**  **National Park**  Observed  CMAQ**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Stacked**  **Extinction Big Bend**  **National Park**  Observed  CAMx**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Stacked**  **Extinction Caney**  **Creek**  Observed  CMAQ**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Stacked**  **Extinction Caney**  **Creek**  Observed  CAMx**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Stacked**  **Extinction Vo** **yageurs**  Observed  CMAQ**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Stacked**  **Extinction Vo** **yageurs**  Observed  CAMx**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Big Bend National Park,**  **Te xas**  **Wo rst 20%** **D ays**  “Relaxed in Ti me” Comparisons Select Highest Modeled Extinction on Same Day; Within 3 Days; Within 1 We ek; Within 2 Weeks of Observed Worst 20% Days**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Boundary** **W aters,**  **Minnesota Wo**  **rst 20% D**  **ays** “Relaxed in Ti me” Comparisons SO4 Good NO3 Overstated OMC Understated EC Overstated No Obs/Model Soil No Modeled CM**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Breton Island,**  **Louisiana Wo**  **rst 20% D**  **ays** “Relaxed in Ti me” Comparisons SO4 FairlyGood NO3 Understated OMC Understated EC Overstated No Obs/Model Soil No Modeled CM**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Mingo,**  **Missouri Wo**  **rst 20% D**  **ays** “Relaxed in Ti me” Comparisons SO4 Good NO3 Overstated OMC Understated EC Overstated No Obs/Model Soil No Modeled CM**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Wi**  **chita Mountains, Oklahoma**  **Wo rst 20%** **D ays**  “Relaxed in Ti me” Comparisons Select Highest Modeled Extinction on Same Day; Within 3 Days; Within 1 We ek; Within 2 We eks of Observed Wo rst 20% Days**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Wi**  **chita Mountains Best 20%** **D ays**  “Relaxed in Ti me” Comparisons Select Lowest Modeled Extinction on Same Day; Within 3 Days; Within 1 Week; Within 2 Weeks of Observed Worst 20% Days**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call**



## SO4 CENRAP**IMPROVE SITES CMAQ**  **versus CAMx**  February August**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Sulfate (SO4)**  **Monthly Fractional**  **Bias**  **CMA Q**  **V4.4 SOA mods**  **CA Mx**  **better Feb -** **Sep; CMA** **Q better in Oct; Jan,**  **Nov a**  **nd Dec mixed CA** **Mx V4.3beta**   **Winter ov** **erestimation, Summer**  **underestimation ENVIRON/UCR**  **CENRAP Modeling Team Conference Call** 

 CMAQ  CAMx Fractional Bias as function of average concentration

**SO4 Bugle**  **Plots ENVIRON/UCR**  **CENRAP Modeling Team Conference Call** 



## NO3 CENRAP**IMPROVE SITES CMAQ**  **versus CAMx**  February August**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Nitrate (NO3)**  **Monthly Fractional**  **Bias**  **CMA Q**  **V4.4 SOA mods**  **Winter Ov erestimation**  **Summer Underestimation CA** **Mx V4.3beta**   **CMA Q**  **performance slightly improv** **ed ov** **er CA** **Mx ENVIRON/UCR**  **CENRAP Modeling Team Conference Call**  **NO3 Bugle Plots**   CMAQ Winter overestimation more concern due to higher mass; summer underestimation less concern due to low mass  CAMx**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call**



## Organic **Matter Carbon (OMC) CENRAP** **IMPROVE SITES CMAQ**  **versus CAMx**  February August**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Organic Matter**  **Carbon (OMC)** Fractional Bias  CMAQ V4.4 SOAmods Obs: OMC = 1.4 x OC
CAMx V4.3beta  CMAQ Winter Overestimation, CAMx large Summer Underestimation (no SOAmods)**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call OMC**  **Bugle Plots**  CMAQ Summer summer underestimation of most concern  CAMx**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call** 

# CMAQ SOAmods Update

• December 2004 VISTAS CMAQ performance very poor summer OMC performance • Identified several processes missing in CMAQ/CAMx Secondary Organic Aerosol (SOA) modules

– SOA from

sesquiterpenes

– SOA from

isoprene

– Polymerization of SOA into non- volatile particles

– Acid catalyzed reactions • Added first three of these processes to CMAQ (SOAmods) and OMC performance was greatly improved • Plans to add a similar but enhanced approach to CAMx

– Initial prototype March 2006**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call** 

## Elemental Carbon (EC) CENRAP**IMPROVE**

**CMAQ versus**  **CAMx** February August**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Elemental Carbon**  **(EC)** Fractional Bias  CMAQ V4.4 SOAmods

CAMx V4.3beta  To o high in Winter, To o low i n Summer CAMx slightly better than CMAQ**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call EC**  **Bugle Plots**  CMAQ EC Meets Performance Goals  CAMx**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Soil (Other PM**  **2.5 )**  Fractional Bias  CMAQ V4.4 SOAmods

CAMx V4.3beta  CMAQ and CAMx large overestimation bias except in summer**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Coarse Matter**  **(CM or PM 2.5**  **- 10**  **)** Fractional Bias  CMAQ V4.4 SOAmods

CAMx V4.3beta  CMAQ and CAMx large underestimation bias except CMAQ in w inter**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call Bugle Plots**  IMPROVE Network All Species  CMAQ  CAMx CM large underestimation Soil overestimation NO3 Winter Overestimation OC Summer Underestimation**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call** 

## Conclusions: CMAQ vs. CAMx Model **Performance using 36 km Grid** 
• No one model performing consistently better than the other across all species, locations and seasons • Sulfate performance mixed

– Both models performing fairly well • CMAQ winter nitrate overprediction not as severe as seen for CAMx • CMAQ OMC performance substantially better than CAMx

– SOAmods update improves summer performance • CAMx slightly better for EC but both models fairly good • Soil (overprediction) and CM (underprediction) performance for both models poor**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call** 

## Recommendations: CMAQ vs. CAMx
• Resource and time constraints dictates that a lead model be selected with the second model used for corroboration • Selection Criteria

– Science

– Model Performance

– Ease of Use • Run Times • Disk Space

– Source Apportionment Capability

– Other RPO Selections

– Other Factors**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call** 

## Recommendations: CMAQ vs. CAMx
• Model performance slight nod toward CMAQ

– SOAmods update in CMAQ greatly improves OMC performance and improves science 

### • CMAQ elapsed run times faster than CAMx due to availability of MPI that can use more CPUs

– CAMx MPI still months off 

### • CMAQ uses much more disk space than CAMx • CAMx includes flexi -nesting (e.g., 12 km) • CAMx includes PM Source Apportionment (PSAT)

– CMAQ TSSA approach not available in V4.5**ENVIRON/UCR CENRAP Modeling Team Conference**  **Call** 

## Recommendations: CMAQ vs. CAMx
• Other neighboring RPOs model selection mixed

– MWRPO selected CAMx

– VISTAS selected CMAQ

– WRAP currently using both

– CENRAP States using CAMx for 8 -hour ozone (TX & MO) 

### • At this time recommend carry both models through 2018 Visibility Projections and then make decision

– likely recommendation based on current situation

– CMAQ SOAmods for sensitivity and sensitivity/strategy runs

– CAMx for PM source apportionment, 12km flexi -nesting and corroborative runs