Appendix I 

Meteorological Model Performance 
TABLE OF CONTENTS 

I. INTRODUCTION ....................................................................................................................................1 

II. 12 KM MM5 PERFORMANCE IN NORTH CAROLINA ...................................................................1 

III. MM5 MODEL PERFORMANCE AT GREENSBORO, NC ...............................................................7 

IV. UPPER AIR MODEL PERFORMANCE .............................................................................................8 

V. SUMMARY..........................................................................................................................................12 
LIST OF FIGURES 

Figure I-1: Monthly plots of modeled 1.5 meter temperature, bias and absolute error..............................2 

Figure I-2: Monthly plots of modeled 1.5 meter mixing ratio, bias and absolute error.............................2 

Figure I-3: Monthly plots of modeled 1.5 meter relative humidity, bias and absolute error .....................3 

Figure I-4: Monthly plots of modeled cloud coverage, bias and absolute error. .......................................4 

Figure I-5: Monthly plots of modeled wind direction, bias and absolute error. ........................................4 

Figure I-6: Monthly plots of modeled wind speed, bias and absolute error...............................................5 

Figure I-7: Monthly plots of modeled wind speed with calm observations omitted, bias and absolute 

error.......................................................................................................................................................6 

Figure I-8: Monthly plots of modeled wind speed with modeled wind speed below 1.5 m/s omitted, bias

and absolute error..................................................................................................................................6 

Figure I-9: Modeled data compared to surface observations at the Greensboro, NC ASOS site (KGSO) 

for two weeks typical of winter conditions...........................................................................................7 

Figure I-10: Modeled data compared to surface observations at the Greensboro, NC ASOS site (KGSO) 

for two weeks typical of summer conditions. .......................................................................................8 

Figure I-11(a)..............................................................................................................................................10 

Figure I-11(b)..............................................................................................................................................10 

Figure I-11(c)..............................................................................................................................................11 

Figure I-11: The Charlotte, NC (CHANC) profiler winds co-plotted with the 12-km MM5 winds for (a) 

12-23 UTC on January 17, 2002, (b) 00-11 UTC on January 19, 2002, and 00-11 UTC on January 

20, 2002...............................................................................................................................................11

Figure I-12(a)..............................................................................................................................................11 

Figure I-12(b)..............................................................................................................................................12 

Figure I-12: The Charlotte, NC (CHANC) profiler winds co-plotted with the 12-km MM5 winds for (a) 

00-11 UTC on November 10, 2002, and (b) 12-23 UTC on November 17, 2002 ..............................12
1 

METEOROLOGICAL MODEL PERFORMANCE 

I. INTRODUCTION 

 The VISTAS (Visibility Improvement State and Tribal Association of the Southeast) MM5 

meteorological model performed quite well at the 12 km grid resolution. In a majority of cases, the model 

statistics fell within the expected ranges of error. In reviewing all available data, SCDHEC agrees with 

VISTAS/ASIP and NCDAQ’s conclusions that the VISTAS MM5 results for the 2002 modeled core 

ozone season will produce credible inputs for CMAQ modeling intended as the basis for the Metrolina 

Area 8-hour ozone attainment demonstration. 

 The attainment demonstration for the Charlotte-Gastonia-Rock Hill, NC-SC 8-hour ozone 

nonattainment area (referred to as the Metrolina area) used the meteorological modeling from the 

VISTAS regional haze modeling. VISTAS is run by the ten Southeast states: Alabama, Florida, Georgia,

Kentucky, Mississippi, North Carolina, South Carolina, Tennessee, Virginia, and West Virginia. The 

meteorological model used for this project was the Pennsylvania State University/National Center for 

Atmospheric Research Mesoscale Meteorological Model (MM5). 

 The sections that follow summarize the meteorological model performance for North Carolina on the 

12 kilometer (km) grid domain. York County, SC, as part of the Metrolina Area, is included in the model 

performance data as an extended part of the North Carolina review, since it contains the only part of SC 

for which this document applies. SCDHEC has evaluated the balance of this work and agrees with its 

conclusions in their entirety. 

 The overall VISTAS meteorological model development and model performance was documented 

by the VISTAS contractor Baron Advanced Meteorological Systems, LLC, and is attached to this 

Appendix. 

II. 12 KM MM5 PERFORMANCE IN NORTH CAROLINA 

 In general, the MM5 performance for North Carolina (including York County, SC) was very similar 

to the performance for the entire VISTAS modeling domain. The temperature bias was near zero in May, 

June, and August. July had a slight negative bias near –0.25 Kelvin (K), and September had a negative

bias of –0.1 K. The absolute temperature error hovered around 1.5 K from May-September. Figure I-1 

displays the overall temperature, temperature bias and absolute error for North Carolina and the VISTAS 

modeling domain. 
2 

Figure I-1: Monthly plots of modeled 1.5 meter temperature, bias and absolute error. 

 The mixing ratio bias in North Carolina was closer to neutral as compared to the entire VISTAS 

domain. The mixing ratio bias in North Carolina was near 0 gram/kilogram (g/kg) in May through July, 

then fell to –0.2 g/kg in August and to -0.6 g/kg in September. The absolute error was only slightly 

higher in North Carolina than in the entire VISTAS domain, peaking at 1.8 g/kg in July. Figure I-2 

displays the overall mixing ratio, mixing ratio bias and absolute error for North Carolina and the VISTAS 

modeling domain. 

Figure I-2: Monthly plots of modeled 1.5 meter mixing ratio, bias and absolute error. 

1.5m Temperature

275

280

285

290

295

300

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

NovDec

Month

Temperature (K)

NC

VISTAS

1.5m Temperature Bias

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

Temperature (K)

NC

VISTAS

1.5m Temperature (K) Absolute Error

0

0.5

1

1.5

2

2.5

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

Abs Error (K)

NC

VISTAS

 Mixing Ratio Qv 

4

6

8

10

12

14

16

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

g/kg

NC

VISTAS

Mixing Ratio Qv Bias

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

NovDec

Month

g/kg

NC

VISTAS

Mixing Ratio Qv Absolute Error

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

Abs Error (g/kg)

NC

VISTAS
3 

 The relative humidity bias in North Carolina was about 0.5% higher when compared to the entire 

VISTAS domain. The relative humidity bias in North Carolina generally hovered around ± 3%. The 

absolute error was slightly higher in North Carolina than the entire VISTAS domain, holding steady 

around 10% during the ozone season. Figure I-3 displays the overall relative humidity, relative humidity 

bias and absolute error for North Carolina and the VISTAS modeling domain. 

Figure I-3: Monthly plots of modeled 1.5 meter relative humidity, bias and absolute error 

 The cloud coverage bias in North Carolina was 2 – 3% higher compared to the entire VISTAS 

domain. The cloud coverage bias peaked near 10% in July, with all other months with a bias less than 

5%. The absolute error was generally 1 – 2% lower than the entire VISTAS domain, peaking at 31% in 

September. Figure I-4 displays the overall cloud coverage, cloud coverage bias and absolute error for

North Carolina and the VISTAS modeling domain. 

Relative Humidity 

50

55

60

65

70

75

80

85

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

%

NC

VISTAS

Relative Humidity Bias

-6

-4

-2

0

2

4

6

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

%

NC

VISTAS

Relative Humidity Absolute Error

0

2

4

6

8

10

12

14

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Abs Error (%)

NC

VISTAS
4 

Figure I-4: Monthly plots of modeled cloud coverage, bias and absolute error. 

 Wind direction was the most erratic of the measurements. The direction bias in North Carolina was 

more pronounced, being more negative May through July, and more positive in August and September. 

The absolute error was close to the entire VISTAS domain, peaking at 35 degrees in July when the 

lightest winds are experienced. Figure I-5 displays the overall wind direction, wind direction bias and 

absolute error for North Carolina and the VISTAS modeling domain. 

Figure I-5: Monthly plots of modeled wind direction, bias and absolute error. 

Cloud Coverage

0

10

20

30

40

50

60

70

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

%

NC

VISTAS

Cloud Coverage Bias

-5

0

5

10

15

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

%

NC

VISTAS

Cloud Cover (%) Absolute Error

0

5

10

15

20

25

30

35

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

NovDec

Abs Error (%)

NC

VISTAS

Wind Direction

0

60

120

180

240

300

360

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

Degrees

NC

VISTAS

Wind Direction Bias

-80

-60

-40

-20

0

20

40

60

80

100

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

NovDec

Month

Degrees

NC

VISTAS

Wind Direction Absolute Error

0

5

10

15

20

25

30

35

40

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

NovDec

Month

Degrees

NC

VISTAS
5 

 The wind speed bias in North Carolina was 0.3 to 0.4 meter/second (m/s) higher than wind speeds 

across the entire VISTAS domain. When considering all wind measurements, the wind speed was 0.8 to 

1.0 m/s too strong. The absolute error is near 1.2 m/s, just above the entire VISTAS domain. When 

omitting calm observations, the bias falls to 0.2 to 0.5 m/s. The absolute error is near 1.2 m/s, very close 

to the entire VISTAS domain. When omitting modeled wind speeds below the threshold of the 

anemometer (<1.5 m/s), the bias is 0.5 to 0.8 m/s. The absolute error is near 1.3 m/s, just above the entire 

VISTAS domain. Figure I-6 displays the overall wind speed, wind speed bias and absolute error for 

North Carolina and the VISTAS modeling domain. Figure I-7 displays the wind speed when the calm 

observations are omitted as well as the bias and absolute error. Figure I-8 displays the wind speed with 

modeled wind speeds below 1.5 m/s omitted. 

Figure I-6: Monthly plots of modeled wind speed, bias and absolute error. 

Wind Speed

1

1.5

2

2.5

3

3.5

4

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

NovDec

Month

m/s

NC

VISTAS

Wind Speed Bias

0

0.2

0.4

0.6

0.8

1

1.2

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

NovDec

Month

m/s

NC

VISTAS

Wind Speed Absolute Error

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Abs Error (m/s)

NC

VISTAS
6 

Figure I-7: Monthly plots of modeled wind speed with calm observations omitted, bias and 

absolute error. 

Figure I-8: Monthly plots of modeled wind speed with modeled wind speed below 1.5 m/s omitted, 

bias and absolute error. 

 Overall, excess wind speeds, increased relative humidity, and cloud cover likely lead to underprediction of the daily maximum peak ozone concentration, which will be discussed in the air quality 

model performance (Appendix J). 

Wind Speed - no calm 

1

1.5

2

2.5

3

3.5

4

4.5

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

m/s

NC

VISTAS

Wind Speed - no calm Bias

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Month

m/s

NC

VISTAS

Wind Speed - no calm Absolute Error

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Abs Error (ms/)

NC

VISTAS

Wind Speed - 'minicalm'

1

1.5

2

2.5

3

3.5

4

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

NovDec

Month

m/s

NC

VISTAS

Wind Speed - 'minicalm' Bias

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Jan

FebMar

Apr

May

Jun

Jul

Aug

Sep

Oct

NovDec

Month

m/s

NC

VISTAS

Wind Speed - 'minicalm' Absolute Error

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Abs Error (m/s)

NC

VISTAS
7 

III. MM5 MODEL PERFORMANCE AT GREENSBORO, NC 

 An additional way to evaluate model performance is to compare the various parameters to surface 

observations for a site-specific evaluation. This methodology, when results are statistically sound, should 

provide a good surrogate for model performance in areas containing similar climatology to the evaluated 

station. (In this case, the Greensboro site is used as a surrogate due to its similarity to the Metrolina 

Area). Figures I-9 and I-10 show a time-series model and observed meteorological data for the 

Greensboro, North Carolina Automated Surface Observing System (ASOS) site (KGSO). The time series 

include modeled surface values for temperature, mixing ratio, wind speed, wind direction, relative 

humidity, cloud cover, and precipitation (blue dashed lines). The model-predicted values are compared to 

actual observations from the Greensboro site (solid black lines). Figure I-9 depicts two different winter 

weeks (January 15-21 and November 12-18, 2002), while Figure I-10 depicts two summer weeks (July 

20-26 and August 19-25, 2002). It is important to look at both summer and winter events, as the forcing 

mechanisms for synoptic features can differ from season to season. 

Figure I-9: Modeled data compared to surface observations at the Greensboro, NC ASOS site 

(KGSO) for two weeks typical of winter conditions. 
8 

Figure I-10: Modeled data compared to surface observations at the Greensboro, NC ASOS site 

(KGSO) for two weeks typical of summer conditions. 

 Generally the time series reflect what was seen on the monthly plots. Overall, the plots reflect good 

model performance for 2002 when compared to modeling standards. Wind direction was close to 

observed values and switched at the appropriate time (when frontal boundaries passed through the area). 

Wind speeds were captured well by the model during the daylight hours, but were often too strong at 

night. 

 Model performance was better at capturing the wintertime diurnal temperature trends than the 

summertime trends. For the summer temperatures, the model was generally too cool with the afternoon 

high temperatures. The mixing ratio was close to observations in the winter, but, again, the model was 

more variable in the summertime. The diurnal trends in relative humidity were reasonably well captured, 

though the diurnal ranges were less in the model than with observations. 

IV. UPPER AIR MODEL PERFORMANCE 

 To further quantify model performance, upper air data from the model was compared to data from 

the Charlotte, North Carolina radar profiler. A radar profiler is an instrument designed to accurately
9 

measure various meteorological variables from the surface up through the first couple of thousand meters 

of the atmosphere (commonly known as the “Planetary Boundary” or “Mixed” Layer). The profiler, 

similar in function to weather radar, sends pulses of electromagnetic radiation up into the atmosphere, 

interpreting the return echoes into useable information, especially wind speed and direction. The 

Charlotte profiler is owned and operated by the State of North Carolina. 

 Profilers yield results at a much finer vertical and temporal resolution than do standard rawinsondes

(balloons with attached meteorological equipment used to take upper air readings). The profiler data are 

not used to nudge or correct MM5 modeling results, and in fact, cannot effectively be used in that 

capacity without additional quality control to remove or correct erroneous data. Since the model results 

will not be artificially biased toward the profiler data because of nudging, and, since the profiler has a 

high data resolution, it makes an excellent source of data to judge model performance. Figure I-11 shows 

a series of profiler plots from the Charlotte, North Carolina profiler. 

 Figures I-11 and I-12 compare model predicted winds (purple wind barbs) with profiler-derived 

winds (black wind barbs) over the lowest 2,500 meters of the atmosphere. Each plot contains 12 hours of 

data, with the hour of the observation labeled near the plot bottom and with the hours increasing from left 

to right. The wind barbs follow the meteorological standard, with a full barb representing a 10-knot (kt) 

wind, a half barb representing a 5-kt wind, and a full flag representing a 50-kt wind. 

 Figure I-11(a) is from the period of 12 to 23 Coordinated Universal Time (UTC) on January 17, 

2002, and depicts the typical wind flow pattern prior to frontal passage. Figure I-11(b) is 00-11 UTC on 

January 19, 2002, and shows the disruption to the winds field as a cold front passes through the area, with 

Figure I-11(c) (00-11 UTC on January 20, 2002) illustrating the northerly flow typically seen after front 

passage in the region. The model captures the wind direction fairly well through out the atmosphere. The 

model winds do become disjointed from the observations in the mid levels during the early hours of the

frontal passage on the 19th [Figure I-11(b)]. 

 Figure I-12(a) represents the time period from 00 UTC to 11 UTC on November 10, 2002, and 

shows the modeling capturing uniform flow throughout the atmosphere. Figure I-12(b) is from seven 

days later (12-23 UTC on November 17, 2002) and demonstrates the model capturing the disturbance of 

the uniform flow in the upper levels. 

 Overall, these Charlotte, North Carolina profiler plots show typical performance, in that the model 

generally matches the profiler winds albeit not perfectly. Upper level winds are captured very well, as are 

the wind shifts associated with frontal passages. In the subset of days presented here, the model winds are 

approximately within 20 degrees of the profiler observed winds and typically are much closer. 

Unfortunately, it is difficult to know if this slight wind direction bias indicates a model flaw or an issue 

with the profiler data being representative. It is likely that there are physical mechanisms in the real 

world of which the model is unaware, which in this case are not being compensated for via nudging. 
10 

Figure I-11(a) 

Figure I-11(b)
11 

Figure I-11(c)

Figure I-11: The Charlotte, NC (CHANC) profiler winds are co-plotted with the 12-km MM5 

winds for (a) 12-23 UTC on January 17, 2002, (b) 00-11 UTC on January 19, 2002, and 00-11 UTC 

on January 20, 2002. 

Figure I-12(a) 
12 

Figure I-12(b) 

Figure I-12: The Charlotte, NC (CHANC) profiler winds are co-plotted with the 12-km MM5 

winds for (a) 00-11 UTC on November 10, 2002, and (b) 12-23 UTC on November 17, 2002 

 

V. SUMMARY 

 In general, the VISTAS MM5 meteorological model performed quite well at the 12 km grid 

resolution. In a majority of cases, the model statistics fell within the expected ranges of error. In 

reviewing all available data, SCDHEC agrees with VISTAS and NCDAQ’s conclusions that the VISTAS 

MM5 results will produce credible inputs for CMAQ modeling intended as the basis for the Metrolina 

Area 8-hour ozone attainment demonstration.