ENVIRON/UCR CENRAP Modeling Team Conference Call

CENRAP Modeling Update: 

CMAQ versus CAMx Model 

Performance Evaluation

Ralph Morris, Gerard Mansell, Bonyoung Koo and 

Abby Hoats, ENVIRON International Corporation

Gail Tonnesen, Mohammed Omary, Chao-Jung Chien 

and Yingqun Wang UC Riverside

CENRAP Modeling Work Group Meeting

Baton Rouge, Louisiana

February 7, 2006
ENVIRON/UCR CENRAP Modeling Team Conference Call

Today's Topics

• Model Performance Evaluation (MPE)

• CMAQ versus CAMx

– Are we ready to pick the Lead and the Corroborative 

Model? [wait for 2018 projection comparisons]

• 36 km versus 12 km Grid

– Does the 12 km grid produce improved results?

• Is it worth the additional costs and time?

– Does running with 12 km met but 36 km emissions buy 

most of the benefits of the 12 km grid?

• 2018 Visibility Projections

– CMAQ 36 km only at this time

• Source Apportionment Plans

• BART VOC/NOx and Control Strategy Modeling
ENVIRON/UCR CENRAP Modeling Team Conference Call

Model Performance Evaluation

• CMAQ versus CAMx 36 km Grid

– CMAQ V4.4 SOAmods

• Includes SOAmods update that adds SOA from 

sesquiterpenes, isoprene, etc.

– CAMx V4.3beta

– Evaluate for monitoring sites in the CENRAP 

States

• Other RPOs also examined – on website

– Focus on Visibility PM Species Components

• SO4, NO3, OMC, EC, Soil and CM

• IMPROVE, CASTNet, STN, NADP
ENVIRON/UCR CENRAP Modeling Team Conference Call

Model Performance Evaluation

• Numerous model performance statistics and graphics 

have been generated:

– Stacked Bar Charts of Obs and Pred

• 24-Hour Extinction

• 20% Best and 20% Worst Days

– Scatter Plots

– Time Series plots

– Monthly Bias/Error Bar Plots

– Bugle Plots

– Spatial Maps w/ Superimposed Observations

• See:

http://pah.cert.ucr.edu/aqm/cenrap/cmaq.shtml
ENVIRON/UCR CENRAP Modeling Team Conference Call

Stacked 

Extinction

Big Bend 

National Park

Observed

CMAQ
ENVIRON/UCR CENRAP Modeling Team Conference Call

Stacked 

Extinction

Big Bend 

National Park

Observed

CAMx
ENVIRON/UCR CENRAP Modeling Team Conference Call

Stacked 

Extinction

Caney Creek

Observed

CMAQ
ENVIRON/UCR CENRAP Modeling Team Conference Call

Stacked 

Extinction

Caney Creek

Observed

CAMx
ENVIRON/UCR CENRAP Modeling Team Conference Call

Stacked 

Extinction

Voyageurs

Observed

CMAQ
ENVIRON/UCR CENRAP Modeling Team Conference Call

Stacked 

Extinction

Voyageurs

Observed

CAMx
ENVIRON/UCR CENRAP Modeling Team Conference Call

Big Bend National Park, 

Texas

Worst 20% Days

“Relaxed in Time” 

Comparisons

Select Highest Modeled 

Extinction on Same Day; 

Within 3 Days; Within 1 

Week; Within 2 Weeks of 

Observed Worst 20% 

Days
ENVIRON/UCR CENRAP Modeling Team Conference Call

Boundary Waters, 

Minnesota

Worst 20% Days

“Relaxed in Time” 

Comparisons

SO4 Good

NO3 Overstated 

OMC Understated

EC Overstated

No Obs/Model Soil

No Modeled CM
ENVIRON/UCR CENRAP Modeling Team Conference Call

Breton Island, 

Louisiana

Worst 20% Days

“Relaxed in Time” 

Comparisons

SO4 FairlyGood

NO3 Understated 

OMC Understated

EC Overstated

No Obs/Model Soil

No Modeled CM
ENVIRON/UCR CENRAP Modeling Team Conference Call

Mingo, Missouri

Worst 20% Days

“Relaxed in Time” 

Comparisons

SO4 Good

NO3 Overstated 

OMC Understated

EC Overstated

No Obs/Model Soil

No Modeled CM
ENVIRON/UCR CENRAP Modeling Team Conference Call

Wichita Mountains, 

Oklahoma

Worst 20% Days

“Relaxed in Time” 

Comparisons

Select Highest 

Modeled Extinction 

on Same Day; Within 

3 Days; Within 1 

Week; Within 2 

Weeks of Observed 

Worst 20% Days
ENVIRON/UCR CENRAP Modeling Team Conference Call

Wichita Mountains

Best 20% Days

“Relaxed in Time” 

Comparisons

Select Lowest 

Modeled Extinction on 

Same Day; Within 3 

Days; Within 1 Week; 

Within 2 Weeks of 

Observed Worst 20% 

Days
ENVIRON/UCR CENRAP Modeling Team Conference Call

SO4 CENRAP IMPROVE SITES

CMAQ versus CAMx

February August
ENVIRON/UCR CENRAP Modeling Team Conference Call

Sulfate (SO4)

Monthly Fractional 

Bias

CMAQ V4.4 SOAmods

CAMx better Feb-Sep; 

CMAQ better in Oct; Jan, 

Nov and Dec mixed

CAMx V4.3beta 

Winter 

overestimation, 

Summer 

underestimation
ENVIRON/UCR CENRAP Modeling Team Conference Call

CMAQ

CAMx

Fractional Bias as 

function of average 

concentration

SO4

Bugle 

Plots
ENVIRON/UCR CENRAP Modeling Team Conference Call

NO3 CENRAP IMPROVE SITES

CMAQ versus CAMx

February August
ENVIRON/UCR CENRAP Modeling Team Conference Call

Nitrate (NO3)

Monthly Fractional 

Bias

CMAQ V4.4 SOAmods

Winter Overestimation

Summer Underestimation

CAMx V4.3beta 

CMAQ performance 

slightly improved 

over CAMx
ENVIRON/UCR CENRAP Modeling Team Conference Call

NO3

Bugle Plots

CMAQ

Winter 

overestimation 

more concern due 

to higher mass; 

summer 

underestimation 

less concern due to 

low mass

CAMx
ENVIRON/UCR CENRAP Modeling Team Conference Call

Organic Matter Carbon (OMC)

CENRAP IMPROVE SITES

CMAQ versus CAMx

February August
ENVIRON/UCR CENRAP Modeling Team Conference Call

Organic Matter 

Carbon (OMC)

Fractional Bias

CMAQ V4.4 SOAmods

Obs: OMC = 1.4 x OC

CAMx V4.3beta 

CMAQ Winter 

Overestimation, CAMx 

large Summer 

Underestimation (no 

SOAmods)
ENVIRON/UCR CENRAP Modeling Team Conference Call

OMC

Bugle Plots

CMAQ

Summer summer 

underestimation of 

most concern

CAMx
ENVIRON/UCR CENRAP Modeling Team Conference Call

CMAQ SOAmods Update

• December 2004 VISTAS CMAQ performance very poor 

summer OMC performance

• Identified several processes missing in CMAQ/CAMx 

Secondary Organic Aerosol (SOA) modules

– SOA from sesquiterpenes

– SOA from isoprene

– Polymerization of SOA into non-volatile particles

– Acid catalyzed reactions

• Added first three of these processes to CMAQ (SOAmods) 

and OMC performance was greatly improved

• Plans to add a similar but enhanced approach to CAMx

– Initial prototype March 2006
ENVIRON/UCR CENRAP Modeling Team Conference Call

Elemental Carbon (EC) CENRAP IMPROVE

CMAQ versus CAMx

February August
ENVIRON/UCR CENRAP Modeling Team Conference Call

Elemental Carbon 

(EC)

Fractional Bias

CMAQ V4.4 SOAmods

CAMx V4.3beta 

Too high in Winter, Too 

low in Summer

CAMx slightly better than 

CMAQ
ENVIRON/UCR CENRAP Modeling Team Conference Call

EC

Bugle Plots

CMAQ

EC Meets 

Performance Goals

CAMx
ENVIRON/UCR CENRAP Modeling Team Conference Call

Soil (Other PM2.5)

Fractional Bias

CMAQ V4.4 SOAmods

CAMx V4.3beta 

CMAQ and CAMx large 

overestimation bias except 

in summer
ENVIRON/UCR CENRAP Modeling Team Conference Call

Coarse Matter 

(CM or PM2.5-10)

Fractional Bias

CMAQ V4.4 SOAmods

CAMx V4.3beta 

CMAQ and CAMx large 

underestimation bias 

except CMAQ in winter
ENVIRON/UCR CENRAP Modeling Team Conference Call

Bugle Plots

IMPROVE Network

All Species

 CMAQ

CAMx

CM large underestimation

Soil overestimation

NO3 Winter Overestimation

OC Summer Underestimation
ENVIRON/UCR CENRAP Modeling Team Conference Call

Conclusions: CMAQ vs. CAMx Model 

Performance using 36 km Grid

• No one model performing consistently better than the other 

across all species, locations and seasons

• Sulfate performance mixed

– Both models performing fairly well

• CMAQ winter nitrate overprediction not as severe as seen 

for CAMx

• CMAQ OMC performance substantially better than CAMx

– SOAmods update improves summer performance

• CAMx slightly better for EC but both models fairly good

• Soil (overprediction) and CM (underprediction) 

performance for both models poor
ENVIRON/UCR CENRAP Modeling Team Conference Call

Recommendations: CMAQ vs. CAMx

• Resource and time constraints dictates that a lead 

model be selected with the second model used for 

corroboration

• Selection Criteria

– Science

– Model Performance

– Ease of Use

• Run Times

• Disk Space

– Source Apportionment Capability

– Other RPO Selections

– Other Factors
ENVIRON/UCR CENRAP Modeling Team Conference Call

Recommendations: CMAQ vs. CAMx

• Model performance slight nod toward CMAQ

– SOAmods update in CMAQ greatly improves OMC 

performance and improves science

• CMAQ elapsed run times faster than CAMx due to 

availability of MPI that can use more CPUs

– CAMx MPI still months off

• CMAQ uses much more disk space than CAMx

• CAMx includes flexi-nesting (e.g., 12 km)

• CAMx includes PM Source Apportionment (PSAT)

– CMAQ TSSA approach not available in V4.5
ENVIRON/UCR CENRAP Modeling Team Conference Call

Recommendations: CMAQ vs. CAMx

• Other neighboring RPOs model selection mixed

– MWRPO selected CAMx

– VISTAS selected CMAQ

– WRAP currently using both

– CENRAP States using CAMx for 8-hour ozone (TX & MO)

• At this time recommend carry both models through 

2018 Visibility Projections and then make decision –

likely recommendation based on current situation

– CMAQ SOAmods for sensitivity and sensitivity/strategy 

runs 

– CAMx for PM source apportionment, 12km flexi-nesting 

and corroborative runs