NOAA Technical Memorandum NWS SR – 227 

___________________________________________________________________________ 

NORTH AMERICAN MESOSCALE MODEL -- COMMUNITY 

MULTISCALE AIR QUALITY MODEL OZONE FORECAST 

VERIFICATION STUDY FOR KNOXVILLE, TENNESSEE 

(SUMMER 2005) 

Loren C. Marz 

National Weather Service 

Weather Forecast Office 

Knoxville/Tri-Cities, Tennessee 

Scientific Services Division 

Southern Region 

Fort Worth, Texas 

December 2006 

UNITED STATES National Oceanic and National Weather Service 

DEPARTMENT OF COMMERCE Atmospheric Administration David L. Johnson, Assistant 

Carlos M. Gutierrezs, Secretary Conrad C. Lautenbacher Administrator for Weather Services 
This publication has been reviewed 

and is approved for publication by 

Scientific Services Division 

Southern Region 

 David B. Billingsley, Chief 

 Scientific Services Division 

 Fort Worth, Texas 

i
TABLE OF CONTENTS

____________________________________________________ 

PAGE 

I. Inroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

II. Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 

III. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 

IV. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 

V. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 

VI. References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 

ii
I. Introduction 

In the summer of 2004, the National Weather Service (NWS) launched experimental ozone 

forecasts for the northeastern quadrant of the Continental United States (CONUS). The 

domain of this experimental ozone forecast capability encompassed most of Tennessee. In the 

summer of 2005, this domain was expanded to include the entire eastern part of the CONUS 

through the Mississippi Valley. The portion of the domain that had been in development 

during 2004, and experimentally tested in 2005 was approved for operational deployment in 

August, 2005. Thus, all of Tennessee is now part of the operational domain (Davidson 2005).

The National Centers for Environmental Prediction's (NCEP) North American Mesoscale 

(NAM) and Community Multi-scale Air Quality (CMAQ) models form the backbone of the 

new NOAA-EPA air quality forecast capability. The NCEP runs the linked model system to 

provide the ozone forecast guidance data to state and local air quality forecasters and the 

public. State and local air quality forecasters issue the official air quality forecasts for specific 

regions of their respective states: some 300 cities nationwide issue air quality alerts based on 

ozone forecasts, and about 100 of them include information on airborne particulate matter. 

State air quality forecasters have historically used statistical models to provide air quality 

forecasts. These statistical approaches relate forecasted temperatures and sky cover to 

forecasted ambient ozone levels. (Personal communication with State of Tennessee air quality 

forecasters.) 

As described by Otte, et al., (2005) the CMAQ model is coupled to the NAM model (12 km 

grid spacing) to provide forecast guidance for ground-level ozone. The NAM model provides 

forecast meteorological parameters to the air quality modules PREMAQ and CMAQ, including 

temperature, winds, mixing heights, and cloud cover. A pre-processor, called PREMAQ, 

modifies estimated emissions (based on EPA’s National Emission Inventory) for weatherdependence and, with other interface processors, converts the NAM output to a form that can 

be used by the chemical transport model, CMAQ. The CMAQ then performs atmospheric 

reactive chemical transport simulations, which are used to predict ozone concentrations 

(McQueen et al. 2005). 

The ground-level ozone is commonly formed when nitrogen dioxide (NO2), which is one of 

several nitrogen oxide species collectively called NOx, photo-dissociates into nitric oxide (NO) 

and atomic oxygen (O) (Cooper and Alley 2002). The resulting atomic oxygen combines with 

molecular oxygen (O2) to form ozone (O3). However, the NO that remains from the initial 

photo-dissociation immediately reacts with ambient O3 to reform NO2 and O2. Thus, there is 

no net ozone production from the photo-dissociation of NO2 . Volatile Organic Compounds 

(VOCs) play a significant role in ground-level ozone production in that atmospheric 

decomposition of VOCs produces peroxy radicals (RO2

.

 and HO2

.

). These peroxy radicals 

oxidize NO (another component of NOx) into NO2, resulting in more NO2 for O3 formation 

and less NO for O3 depletion (Davidson, 2003). Relatively high ambient levels of ground-level 

ozone are considered a human health hazard in that O3 causes inflammation of lung tissues. 

The Environmental Protection Agency (EPA) has set a maximum National Ambient Air 

Quality Standard (NAAQS) of 0.08 parts per million (ppm) 8-hour average (U.S. 

Environmental Protection Agency National Ambient Air Quality Standards). A maximum 1-

1 
hour value of 0.12 ppm was used prior to implementation of the 8-hour average, and is still 

used for areas classified as Early Action Compact (EAC) areas (EPA Green Book). 

A study was conducted for the 2005 ozone season, which is nominally between May 1 and 

September 30, to assess performance of the NAM-CMAQ ozone forecast guidance. The city 

of Knoxville, Tennessee, was used as the forecast point. The NAM-CMAQ ozone predictions 

for Knoxville were compared to observed data for the corresponding days. Maximum onehour and eight-hour average ozone forecast guidance concentrations were used. Ozone 

concentrations are given in parts per billion (ppb). Due to the photochemical nature of ozone 

production diurnal, in addition to seasonal cycles, are typical. Diurnal maximum 1-hour and 8-

hour ozone levels typically occur during the afternoon hours. 

II. Procedure 

Ozone forecast guidance for Knoxville, Tennessee, was obtained each day through the study 

period from the NWS ozone forecast guidance data base on the operational Web site 

(http://www.weather.gov/aq/). The grid point used for Knoxville corresponded to latitude: 

36.00 N, longitude: -83.90 W. Generally, data from the 1200 UTC run were used, although 

other model run times were used in isolated instances when 1200 UTC data were not available 

or where circumstances prevented obtaining the 1200 UTC data. The ozone forecast guidance 

analyzed and the highest predicted one-hour and eight-hour values were determined. These 

values were then compiled for each day through the study period (see Table 1). A graphical 

depiction of the maximum 8-hour predicted vs. actual ozone values is provided in Figure 1. 

Forecast guidance data were occasionally not recorded (on a total of 16 days). An entry of 

“MM” was made for each of those missing values. 

2
MAX 8-HOUR OZONE VALUES - PREDICTED VS 

ACTUAL

0

20

40

60

80

100

120

5/1

5/15

5/29

6/12

6/26

7/10

7/24

8/7

8/21

9/4

9/18

Date

Ozone Values (ppb)

Predicted Value

Actual Value

Figure 1. Variation of predicted and observed maximum 8-hour average ozone values through 

the study period. 

Official ozone monitoring data were obtained from the State of Tennessee Department of 

Environment and Conservation (TDEC) and the Knox County Department of Air Quality 

Management. There were two monitors in the Knoxville area, one monitor located at 9315 

Rutledge Pike, Mascot, Tennessee (36.01944 N, -83.87361 W), and the other located at 4625 

Mildred Drive, Knoxville, Tennessee (36.084722 N, -83.764722 W). Monitoring values for 

the two sites were averaged. The monitoring data were also included in the matrix (Table 1) 

on the corresponding days. The NAM-CMAQ ozone forecast verification was based on the 

data from these two monitors. 

The predicted ozone values were compared to the monitoring results. Mean Absolute Error 

(MAE) and Mean Bias Error (MBE) were subsequently calculated for each predicted value, for 

both 1-hour and 8-hours. 

III. Results 

The MAE for the entire study period was approximately 10.3 ppb for both the 1-hour and 8-

hour maximum average values. The maximum 1-hour ambient ozone levels averaged 66.9 

ppb, and the maximum 8-hour ambient ozone levels averaged 57.6 ppb through the study 

period. Therefore the 10.3 ppb MAE equates to approximately 15% and 18% forecast errors, 

respectively. 

3
The NAM-CMAQ forecast guidance for Knoxville showed a high bias, which suggests a 

tendency to overpredict ozone levels. The MBE for the maximum 1-hour average values was a 

+2.91 ppb. The MBE for the maximum 8-hour average values was +5.91 ppb. This 

overprediction bias has been reported previously in statistical evaluations for the entire Eastern 

domain conducted in a previous study of the NAM-CMAQ during developmental testing in 

2004 (Otte, et al. 2002). 

The NAM-CMAQ forecast that 10 days would exceed the maximum 8-hour NAAQS limit for 

ozone, which is 0.084 ppm (or 84 ppb) through the study period. Only one day actually 

exceeded the 8-hour value, based on the average of the two Knoxville ozone monitors. That 

occurred on July 26, when the average concentration of the ozone monitors was 92 ppb. The 

NMA-CMAQ predicted a value of 96 ppb for that day. There were no days which exceeded 

the now-defunct (except for EAC areas) maximum 1-hour NAAQS value of 0.124 ppm (124 

ppb) and none were predicted by the NAM-CMAQ. 

Some spatial variability was noted between the two ozone monitors. There were two instances 

through the study period when one of the monitors, but not the other, exceeded the 8-hour 

ozone NAAQS, and the average of the two monitors was at or below 84 ppb. These instances 

occurred on June 24, when the Mildred Drive monitor recorded 88 ppb, and on August 12, 

when the same monitor recorded 86 ppb. 

The maximum 1-hour observed values ranged from 29 ppb on July 7 to 103 ppb on July 26. 

The maximum 8-hour observed values ranged from 18 ppb on September 26 to 92 ppb on July 

26. The maximum forecast error occurred on June 9 when the predicted ozone value was 

missed by 60 ppb for 1-hour and 46 ppb for 8-hours. The NAM-CMAQ predicted a maximum 

1-hour value of 119 and a maximum 8-hour value of 94 ppb, while the actual ambient ozone 

concentrations were 59 ppb and 48 ppb, respectively, based on the ozone monitoring data. 

These errors were caused by a bug in the interface processing between the newly updated 

ground-level global forecast system NWP model (GFS) that provides boundary and initial 

conditions to the NAM. The bug was diagnosed as a result of its impacts on predicted groundlevel ozone, and the resulting crisis fix corrected the situation by mid-June Paula Davidson, 

personal communication). 

Another large high bias error was noted on August 14, when NAM-CMAQ missed the 8-hour 

ozone value by 40 ppb (91 ppb predicted, 51 ppb actual). The large error that resulted on this 

day was probably related to unusual conditions not predicted by the NAM model. A review of 

the weather conditions reported at the Knoxville, Tennessee, airport (TYS) revealed 

temperatures were near the climatological normal for the date, but with fog reported until 1000 

LST, and broken to overcast sky conditions in all but one observation during the remaining 

daylight hours. 

The only observation that did not report a ceiling occurred at 1600 LST, which is fairly late in 

the afternoon, after the typical peak diurnal ozone production time. Based on raw NAM model 

data from that date, the predicted temperature (91o F) was close to the actual temperature (89o

F). However, the predicted relative humidity (RH) values were generally below 70% at all 

levels (925 millibars to 300 millibars), suggesting that clouds were likely not predicted by the 

4
model through the peak ozone-generating daylight hours, thus leading to the relatively high 

ozone values forecast by NAM-CMAQ. 

At any rate, it is clear that weather conditions, particularly cloud cover, play a major role in 

ambient ozone production. Since ozone production is a photochemical process, any errors in 

the NAM predicted cloud cover would have a significant effect on CMAQ forecast ozone 

concentrations. 

The largest low bias occurred on August 20 for both the 1-hour and 8-hour categories. The 

NAM-CMAQ predicted 51 ppb (1 hour maximum) and 48 ppb (8 hour maximum) while the 

corresponding observed values were 85 ppb and 71 ppb, respectively. The weather 

observations for TYS on that day indicated few to scattered clouds through the daylight hours. 

The raw NAM data show quite high RH values (>85%) at 850 millibars throughout the 

daylight hours, suggesting the possibility that the NAM overpredicted clouds at that level. 

This would have caused the CMAQ to significantly underpredict the ozone levels. 

The most accurate ozone forecasts occurred for May 24 and July 15 when the NAM-CMAQ 

maximum 8-hour values were 50 ppb and 54 ppb, respectively, the same as the observed 

values. The 1-hour forecast guidance was only 1 ppb from the observed values on those days. 

A breakdown of the NAM-CMAQ forecast errors by percentage in various categories is 

provided in Table 2. The NAM-CMAQ had a low bias for both 1-hour and 8-hour ozone 

levels early in the season (May). There were 19 days and 16 days, respectively, where the 

NAM-CMAQ underpredicted ozone levels during May, while only seven days (out of 26 days 

sampled or 27%) and nine days (35%), respectively, were overpredicted. 

The NAM-CMAQ tended to persistently overpredict ozone levels during June - September. 

The months with the greatest number of days in which NAM-CMAQ overpredicted ozone 

levels were July for 1-hour levels (21 out of 28 days sampled, or 75%), and August for the 8-

hour levels (25 out of 29 days sampled, or 86%). 

IV. Conclusion 

In general, the NAM-CMAQ did a good job in predicting maximum ambient ozone 

concentrations for the following day, and could provide useful guidance of air quality. There 

was a tendency for the NAM-CMAQ to overpredict ozone levels, with a few days when large 

errors were noted. Those errors were most likely related to errors in the underlying NAM 

weather prediction, and could potentially be mitigated by incorporating knowledge of the 

expected performance of the NAM. 

V. Acknowledgements 

The author thanks the staff of the State of Tennessee Department of Environment and 

Conservation and the Knox County Department of Air Quality Management for providing the 

ozone monitoring data. Thanks also go to the staff of the National Weather Service Forecast 

Office at Morristown, Tennessee, especially David Hotz, Science and Operations Officer, and 

5
to the staff of the National Weather Service Office of Science and Technology, especially Dr. 

Paula Davidson, National Weather Service Program Manager for Air Quality Forecasting, for 

helpful suggestions with the study and manuscript. 

V. References 

Cooper, C. David and F. C. Alley, 2002: Air Pollution Control; A Design Approach, 3d ed. 

Waveland Press, 739 pp. 

Davidson, Paula M., 2003: National Air Quality Forecast Capability: First Steps Toward 

Implementation, May 7, 2003, Office of Science and Technology, National Weather Service, 

31pp. [Available online at http://www.nws.noaa.gov/ost/air_quality/Davidson.pdf.] 

________________, 2005: National Air Quality Forecasting Capability: Progress and Plans, 

June 17, 2005, 18pp. [Available online at 

http://www.nws.noaa.gov/ost/air_quality/NAQFC_update_0605.pdf.]

McQueen, Jeffrey T., Pius Lee, Marina Tsidulko, Geoff DiMego, Rohit Mather, Tanya Otte, 

Jon Pleim, George Pouliot, Daiwen Kang, Ken Schere, Jerry Gorline, Michael Schenk, and 

Paula Davidson, 2005: Update of the Eta-CMAQ forecast model run at NCEP operations and 

its performance for the summer 2004. Preprints, Third Annual Models-3 User's Conference, 

Chapel Hill, NC, Environ. Protection Agency, [Available online at 

http://www.cmascenter.org/conference/2004/abstracts/Forecasting/mcqueen_abstract.pdf.] 

Otte, Tanya L., George Pouliot, Jonathan E. Pleim, Jeffrey O. Young, Kenneth L. Schere, 

David C. Wong, Pius C. S. Lee, Marina Tsidulko, Jeffery T. McQueen, Paula Davidson, Rohit 

Mathur, Hui-Ya Chuang, Geoff DiMego, and Nelson L. Seaman, 2005: Linking the Eta Model 

with the Community Multiscale Air Quality (CMAQ) Modeling System to Build a National 

Air Quality Forecasting System, Weather and Forecasting, Vol. 20, No. 3, pp. 367–384. 

U.S. Environmental Protection Agency, National Ambient Air Quality Standards (NAAQS). 

[Available online at http://www.epa.gov/air/criteria.html .] 

_______________________________ Green Book Nonattainment Areas for Criteria 

Pollutants 1-Hour Ozone Information. [Available online at 

http://www.epa.gov/oar/oaqps/greenbk/oindex.html#List1 .] 

6
Table 1: NAM-CMAQ ozone forecast and observed data. Ozone levels are in PPB (max 1-hour 

average/max 8-hour average). Observed data are the average of two available monitors in the 

Knoxville area. See text for details. 

Date Forecast (1hr/8hr) Observed (1hr) Observed (8hr)

5/01 52/51 56 54 

5/02 52/50 52 48 

5/03 54/51 55 50 

5/04 MM 58 54 

5/05 51/44 44 37 

5/06 55/51 63 57 

5/07 68/62 82 77 

5/08 67/62 75 71 

5/09 65/62 81 71 

5/10 64/57 50 41 

5/11 64/55 68 61 

5/12 84/75 93 81 

5/13 72/67 90 78 

5/14 MM 61 50 

5/15 60/55 58 54 

5/16 61/57 53 49 

5/17 74/66 74 68 

5/18 75/64 87 82 

5/19 67/62 92 82 

5/20 61/57 57 49 

5/21 MM 56 51 

5/22 52/46 66 49 

5/23 MM 63 60 

5/24 56/50 57 50 

5/25 51/50 46 44 

5/26 MM 60 54 

5/27 66/59 75 67 

5/28 45/44 60 54 

5/29 66/58 50 41 

5/30 45/37 59 55 

5/31 60/58 68 63 

7
Date Forecast (1hr/8hr) Observed (1hr) Observed (8hr)

6/01 56/50 53 44 

6/02 62/48 47 42 

6/03 72/63 38 29 

6/04 65/54 65 58 

6/05 72/67 66 58 

6/06 72/65 76 64 

6/07 77/68 53 49 

6/08 93/90 58 48 

6/09 119/94 59 48 

6/10 70/60 54 45 

6/11 49/45 47 37 

6/12 49/45 38 27 

6/13 56/52 54 44 

6/14 71/66 68 62 

6/15 60/58 63 58 

6/16 55/53 54 50 

6/17 52/48 65 61 

6/18 71/58 70 67 

6/19 76/71 74 68 

6/20 84/73 67 53 

6/21 89/83 79 70 

6/22 68/63 81 74 

6/23 84/79 82 77 

6/24 90/84 92 84 

6/25 94/86 89 81 

6/26 85/74 73 62 

6/27 61/50 66 51 

6/28 51/45 67 51 

6/29 69/64 83 67 

6/30 73/64 80 66 

8
Date Forecast (1hr/8hr) Observed (1hr) Observed (8hr)

7/01 70/60 99 70 

7/02 62/55 65 58 

7/03 60/59 52 37 

7/04 75/69 74 60 

7/05 58/54 54 46 

7/06 79/74 37 33 

7/07 47/35 29 25 

7/08 61/58 59 51 

7/09 86/79 78 70 

7/10 73/72 70 59 

7/11 56/55 25 24 

7/12 47/38 61 57 

7/13 51/49 52 46 

7/14 54/49 52 45 

7/15 60/54 61 54 

7/16 44/41 40 37 

7/17 49/47 34 31 

7/18 51/47 54 39 

7/19 56/50 50 38 

7/20 74/68 57 47 

7/21 92/79 67 50 

7/22 79/64 77 70 

7/23 71/65 64 61 

7/24 95/92 94 80 

7/25 MM 89 78 

7/26 114/96 103 92

7/27 MM 68 61 

7/28 MM 54 47 

7/29 52/48 51 47 

7/30 MM 59 52 

7/31 66/64 58 52 

9
Date Forecast (1hr/8hr) Observed (1hr) Observed (8hr)

8/01 74/72 61 56 

8/02 80/77 72 68 

8/03 87/81 79 71 

8/04 103/95 96 74 

8/05 93/86 78 72 

8/06 105/95 91 66 

8/07 54/52 30* 35* 

8/08 54/50 38 30 

8/09 65/56 54 43 

8/10 84/78 68 61 

8/11 85/80 94 76 

8/12 89/86 102 78 

8/13 93/83 78 62 

8/14 98/91 74 51 

8/15 87/74 88 64 

8/16 79/70 74 60 

8/17 80/72 59 46 

8/18 54/48 63 40 

8/19 57/52 76 63 

8/20 51/48 85 71 

8/21 77/70 59 54 

8/22 MM 74 61 

8/23 86/67 62 53 

8/24 77/75 71 63 

8/25 73/57 82 61 

8/26 62/55 73 63 

8/27 72/67 76 62 

8/28 66/63 70 60 

8/29 61/55 50 45 

8/30 43/39 42 35 

8/31 70/67 69 62 

*since 8-hour average exceeded 1-hour average, data were suspect and were not used. 

10
Date Forecast (1hr/8hr) Observed (1hr) Observed (8hr)

9/01 83/70 73 65 

9/02 83/71 67 65 

9/03 MM 66 63 

9/04 MM 64 58 

9/05 MM 71 65 

9/06 71/67 70 62 

9/07 78/70 69 62 

9/08 MM 67 62 

9/09 84/75 75 72 

9/10 91/84 84 76 

9/11 98/89 81 75 

9/12 78/72 84 71 

9/13 84/79 79 75 

9/14 84/82 100 83 

9/15 88/71 77 69 

9/16 58/51 48 38 

9/17 65/60 64 56 

9/18 63/59 75 60 

9/19 87/77 98 78 

9/20 73/61 100 80 

9/21 91/82 93 80 

9/22 76/72 83 71 

9/23 87/74 69 58 

9/24 MM 74 61 

9/25 61/56 60 50 

9/26 43/39 30 18 

9/27 64/58 60 53 

9/28 61/55 66 54 

9/29 MM 46 41 

9/30 76/66 56 48 

Note – Bold numbers indicate values exceeding the NAAQS limit for ozone 

11
Table 2. Percentage distribution of NAM-CMAQ ozone forecast errors by category. Observed 

data are the average of two available monitors in the Knoxville area. 

Forecast-Observed 1-hour 8-hour 

(high bias) 

> +16 ppb 16.2% 19.1% 

+13 to +16 ppb 5.9% 6.6% 

+10 to +12 ppb 6.6% 7.4% 

+7 to +9 ppb 9.5% 11.8% 

+4 to +6 ppb 8.1% 14.0% 

+1 to +3 ppb 15.4% 11.8% 

0 ppb 2.2% 2.9% 

-1 to -3 ppb 7.4% 7.4% 

-4 to -6 ppb 5.9% 5.1% 

-7 to -9 ppb 5.9% 2.9% 

-10 to -12 ppb 3.7% 3.7% 

-13 to -16 ppb 6.6% 0.7% 

< -16 ppb 6.6% 6.6% 

(low bias) 

12