Meteorological Model Performance Evaluation of an 

Annual 2002 MM5 (version 3.6.3) Simulation 

Matthew T. Johnson 

Iowa Department of Natural Resources 

2004 - 2007 

v2.0.4 
Table of Contents 

1. Introduction ................................................................................................................. 3 

1.1 Background....................................................................................................... 3 

1.2 Model Selection ................................................................................................ 3 

2. Sensitivity Projects...................................................................................................... 4 

3. Modeling System Configuration ................................................................................. 5 

3.1 Overview........................................................................................................... 5 

3.1.1 Terrain......................................................................................................... 5 

3.1.2 PREGRID/REGRIDDER ........................................................................... 5 

3.1.3 LIITLE_R ................................................................................................. 10 

3.1.4 INTERPF .................................................................................................. 10 

3.2 MM5 ............................................................................................................... 12 

3.3 Computational Summary ................................................................................ 13 

4. Model Performance Evaluation................................................................................. 14 

4.1 Background..................................................................................................... 14 

4.2 Methods........................................................................................................... 14 

4.3 Statistical Measures ........................................................................................ 15 

5. Surface Evaluation (36 km)....................................................................................... 18 

5.1 Great Lakes ..................................................................................................... 18 

5.2 Northern and Southern Cenrap ....................................................................... 18 

5.3 Ohio Valley..................................................................................................... 19 

5.4 Iowa................................................................................................................. 19 

5.5 Eastern Regions .............................................................................................. 19 

5.6 Western Regions ............................................................................................. 19 

6. Twelve Kilometer Evaluation ................................................................................... 30 

6.1 Daily Averaged Statistics................................................................................ 30 

6.2 Hourly Statistics.............................................................................................. 39 

7. Upper Air Evaluation ................................................................................................ 43 

7.1 Soundings........................................................................................................ 43 

7.2 Precipitation .................................................................................................... 43 

7.3 PBL Depths..................................................................................................... 45 

8. Conclusions ............................................................................................................... 48

9. References ................................................................................................................. 49 

Appendix A....................................................................................................................... 51 

Appendix B ....................................................................................................................... 52 

Eastern Subdomains................................................................................................... 52 

Western Subdomains ................................................................................................. 58 

Appendix C ....................................................................................................................... 66 

CenrapN: January 2002 ............................................................................................ 66 

CenrapN: June 2002 ................................................................................................. 68 

OhioVal: January 2002............................................................................................. 70 

OhioVal: June 2002 .................................................................................................. 72 

Iowa: January 2002................................................................................................... 74 

Iowa: June 2002........................................................................................................ 76 

2
1. INTRODUCTION 

1.1 BACKGROUND

Projects pursuing PM2.5, 8-hour ozone, and regional haze are generating modeling requirements 

at spatial and temporal scales only recently confronted within the regulatory air quality 

community. The scope of recent legislative and executive decisions has created the need to 

implement sophisticated models developed for regional scale multi-pollutant environments 

encompassing diverse climatological regimes. Computational limitations have historically 

bound the modeler’s ability to investigate broad and complex scenarios with sufficient 

resolution. Exponential growth in computational efficiency has partially minimized this hurdle. 

As scientific theory and model complexity evolve, computational innovations remain moderately 

offset. Currently, a balance has been achieved which permits the development of large modeling 

databases such as annual continental scale simulations. 

Annual continental scale air quality simulations require the implementation of a triumvirate 

modeling system composed of meteorological, emissions, and air quality models. 

Meteorological modeling is the first component addressed as meteorological data supports both 

the emissions and air quality models. In preparation for regulatory requirements involving 

regional haze, PM2.5, and ozone, the Iowa Department of Natural Resources (IDNR) developed 

a continental scale annual meteorological dataset designed for use in air quality applications. 

This document details the methods employed to create the annual meteorological simulation and 

provides performance evaluation results. 

1.2 MODEL SELECTION

Due to scientific progression, historical application, community support, and availability, the 

Fifth Generation Penn State University/National Center for Atmospheric Research Mesoscale 

Model (MM5) was selected for the development of an annual meteorological dataset. Originally 

formulated in the 1970s at Penn State and first documented by Anthes and Warner (1978), the 

MM5 modeling system maintains its status as a state-of-the-science1

 model through 

enhancements provided by a broad user community (e.g. Chen and Dudhia, 2001; Dudhia, 1993; 

Stauffer and Seaman, 1990; Stauffer and Seaman, 1991; Xiu and Pleim, 2000). The MM5 

modeling system is routinely employed in operational forecasting frameworks as well as research 

applications spanning meteorological disciplines from synoptic to mesoscale. Utilization of 

MM5 within air quality applications is also a conventional practice. The MM5 modeling system 

was recently selected to generate three continental scale annual simulations: 1996, 2001, and 

2002. The 1996 and 2001 simulations were conducted through EPA contracts (Olerud et al., 

2000; McNally, 2003). The 2002 simulation was conducted in support of regional haze 

modeling for the Visibility Improvement – State and Tribal Association of the Southeast 

(VISTAS) regional planning organization (RPO) (Olerud and Sims, 2004). This list is not 

exhaustive as both public and private organizations continue to pursue annual meteorological 

modeling episodes. 

Additional information regarding MM5 is available at: http://www.mmm.ucar.edu/mm5/

 

1

 True during project implementation. MM5 is no longer regularly updated as the focus has shifted to WRF. 
2. SENSITIVITY PROJECTS 

The MM5 modeling system consists of several pre-processors, the core prognostic model, and 

post-processing tools. Each component contains highly configurable control files; together they 

control the aspects of grid structure, first-guess fields, model physics, temporal operation, and 

ultimately results visualization. The inherit complexity of the MM5 modeling system 

complicates the development of a sound model configuration suitable for regional scale annual 

episode air quality applications. Although the complete matrix of configuration options reduces 

in size as inappropriate options are eliminated, a large matrix of potentially acceptable model 

configurations remains with most applications. The first step in developing the annual MM5 

dataset was therefore completion of a series of sensitivities studies designed to identify the 

configuration yielding optimum results. 

The first sensitivity study project began in 2002 and involved a collaborative project lead by 

Kirk Baker with the Lake Michigan Air Directors Consortium (LADCO) and Matthew Johnson 

(IDNR). Wyat Appel and Mike Abraczinskas with the North Carolina Division of Air Quality 

participated through the generation of a summary analysis for select sensitivity runs. The project 

was conducted in coordination with sensitivity work performed by Dennis McNally (with Alpine 

Geophysics). Components evaluated included, for example, PBL schemes, microphysical 

schemes, convective parameterizations, land surface parameterizations, and snow models. Two 

one-month long episodes were selected for evaluation, January and July of 2001. The 

performance evaluation of each sensitivity run included, but was not limited to, temperatures, 

wind vectors, cloud cover, precipitation, and mixing ratios. 

Following the sensitivity study, the IDNR completed a 2002 annual simulation. This simulation 

utilized surface moisture and temperature nudging. Within implementation of the Pleim-Xiu 

(PX) land surface model (LSM), soil moisture and soil temperatures were modeled in continuum 

from one 5-day episode block to the next. The model performance evaluation revealed an 

extreme cold bias over the Central U.S. While unrelated to the cold bias, utilization of surface 

nudging techniques was abandoned following discussion with the modeling community, as this 

practice has lead to the generation of super-adiabatic lapse rates near the surface. The optimum 

IDNR/LADCO configuration was thus modified accordingly and this annual simulation was 

deemed unsuitable for use in air quality modeling projects. 

In a similar timeframe, VISTAS contracted with Baron Advanced Meteorological Systems, LLC 

(BAMS) for the development of an annual MM5 dataset (Olerud and Sims, 2004). The work of 

VISTAS (through Olerud and Sims, 2004) also included a series of sensitivity studies. 

Independent results from the VISTAS project yielded findings similar to the conclusions reached 

by IDNR and LADCO. The compilation of all project results subsequently produced the 

configuration utilized by the IDNR in development of an annual metrological dataset suitable for 

regional scale air quality modeling. 

4
3. MODELING SYSTEM CONFIGURATION 

3.1 OVERVIEW

Version 3.6.3 of the MM5 modeling system was utilized in the second1 (and final) 2002 IDNR 

annual meteorological simulation. The 3.6.3 release represented the most current version 

available at the time of project inception. Other than the necessary configuration parameters, no 

modeling system code modifications capable of altering results were rendered. 

3.1.1 TERRAIN 

The terrain processor is used to define grid structure and assign various surface features. Terrain 

elevation, the dominant landuse category, and vegetative and soil data were assigned using the 2-

minute 24-category USGS data. The horizontal grid structure consists of a 36 km domain 

conforming to the RPO meteorological grid specifications. A nested 12 km grid was also 

included. The RPO 36 km meteorological domain consists of a Lambert Conic Conformal 

projection centered at 90o W longitude, 40o N latitude, with true latitudes of 33 and 45o N. The 

horizontal extent of the RPO domain was engineered according to the bounds of the Eta 212 

grid. Domain development involved the implementation of TERRAIN through a series of 

sensitivity runs designed to extract the largest domain which remains within the borders of the 

Eta 212 grid. The 12 km grid was designed to achieve a balance between computational 

resources while maximizing coverage of Iowa-centric upwind and downwind flows. Both grid 

structures are described in Table 3.1 and depicted in Figure 3.1. 

Table 3.1. Grid data, referencing MM5 terminology specifications refer to dot points. 

Grid Resolution 

(km) NX NY 

Nest 

Location 

(x,y) 

Southwest 

Coordinate 

(km offset) 

1 36 165 129 1,1 (-2952, -2304) 

2 12 193 199 66,30 (-612, -1260) 

3.1.2 PREGRID/REGRIDDER 

The PREGRID processor prepares archived gridded meteorological data for use within MM5 

through conversion to an intermediate data format readable by MM5. REGRIDDER invokes a 

horizontal interpolation scheme to translate data to the MM5 domain. The 3-hour Eta analysis 

and surface fields (ds609.2) were used to supply initial and boundary conditions to MM5. As the 

Eta analysis fields obtained from NCAR are a compressed (tar) file, the data were first 

uncompressed prior to use within PREGRID. The tar files also include the undesirable 12 hourly 

cold start files. All cold start files (*.tm12) were deleted prior to running PREGRID. 

 

1

 The first simulation was deemed unsuitable for use in air quality modeling projects and has been deleted. 

5
Figure 3.1. Twelve and 36 km domains utilized in the IDNR 2002 MM5v363 annual simulation. 

In the first IDNR 2002 MM5 simulation, NCEP data was included in PREGRID to supply timevariant sea-surface temperature (SST) data, as the Eta surface files supply only a time-invariant 

SST approximation known as skin-temperature. Upon further examination of SST data sources, 

the temporally variable NCEP SST data was found to lead to unrealistic diurnal temperature 

profiles over the Great Lakes and near shorelines. Figure 3.2 shows the NCEP-based Great 

Lakes SSTs for July 4, 2002, at 12 and 18Z. Over this 6-hour span, temperature fluctuations 

over many areas of the Great Lakes (particularly Lake Erie, and most shorelines) reach 20o F. 

While some variability is expected along shorelines and other shallow areas, the magnitudes 

observed through use of the NCEP data are unrealistic. Observed SST data from buoy 45007 

(located in the southern end of Lake Michigan yet far removed from the shoreline, see Figure 

3.3) for the period July 4 – July 9 are provided in Figure 3.4. The maximum temperature 

variation throughout July 4 at this site was less than 3o F. Figure 3.5 depicts the 5-day SST 

timeseries produced using the NCEP SST data within REGRIDDER for the 36 km grid cell 

corresponding to the location of buoy 45007. The NCEP data yields a diurnal temperature range 

of approximately 7o F in this cell on July 4. The NCEP data also generates unrealistic diurnal 

profiles with a net upward trend in SST over this five-day period. In contrast, the observed data 

show less variability and a downward trend in SST. Utilization of the Eta skin-temperature data 

produces the constant SST boundary conditions shown in Figure 3.6. The corresponding Eta 

6
skin-temperature for the location of buoy 45007 is ~294 K. While this yields warmer surface 

temperatures than observed throughout the July 4 – July 9 period, no questionable diurnal 

variability or artificial warming trends are present. 

Figure 3.2. Lake temperature variability across a 6 hour span, from 12Z 7/4/2002 to 18Z 

7/4/2002, using the NCEP SST data. 

Figure 3.3. Great Lake buoy locations. 

7
Observed SST Data

Buoy 45007 - Lake Michigan

289

290

291

292

293

294

07/04/02 07/05/02 07/06/02 07/07/02 07/08/02 07/09/02

Date

Sea Surface Temperature

(K)

Figure 3.4 Observed SST temperature data for buoy 45007. 

Figure 3.5 NCEP derived SST profile for the grid cell corresponding to the location of buoy 

45007. 

8
Figure 3.6. Constant SST data derived from Eta skin-temperatures for the period 12Z 7/4/2002 

through12Z 7/9/2002. 

9
3.1.3 LIITLE_R 

LITTLE_R was originally designed to improve the REGRIDDER output by using objective 

analysis techniques to blend observational data into the gridded first-guess fields. Following 

traditional practices, the NWS upper air (ds353.4) and surface (ds464.0) datasets supply the 

observations. As the Eta fields already contain these NWS datasets, the implementation of 

LITTLE_R is viewed as partially redundant. However, LITTLE_R also generates the files used 

in both the four-dimensional data assimilation (FDDA) and Pleim-Xiu soil moisture nudging 

schemes and therefore must be invoked. The implementation of LITTLE_R does not negatively 

affect model performance when the Eta surface and analysis data provide the first-guess fields 

(Baker, 2002). 

3.1.4 INTERPF 

The IDNR 2002MM5v363 simulation uses a 34 vertical layer structure defined through the 

INTERPF preprocessor. The layer interfaces, provided in Table 3.2, were designed through 

coordination with Dennis McNally to parallel the vertical structure in use by EPA. INTERPF 

interpolates the pressure level data developed in the previous preprocessors to MM5’s native 

vertical system - terrain following sigma coordinates. Sigma levels are defined according to Eq. 

3.1, where ps equals the surface pressure, and pt equals the pressure at model top. The model top 

was defined at 100 mb, or approximately 14,662 meters above ground level. Approximate sigma 

heights are calculated using Eqs. 3.1 – 3.3, with the user-defined variables assigned the following 

values: ps = 1000 mb; pt = 100 mb; Ts = 275 K; A = 50 K. R and g represent the gas and 

gravitational constants of 287 J/(kg K) and 9.8 m/s2

, respectively. 

ts

t

pp

pp

−

− σ = (3.1) 

( ) − + pppp tts =σ ⋅ (3.2) 

⎥

⎥

⎦

⎤

⎢

⎢

⎣

⎡

⎟

⎟

⎠

⎞

⎜

⎜

⎝

⋅ ⎛

+ ⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛ ⋅ ⋅ −=

s

s

s p

p

g

TR

p

p

g

AR

z ln ln

2

2

 (3.3) 

10
Table 3.2. Details of the 34-layer vertical structure. 

Level Sigma Height (m) p (mb) Depth (m)

34 0.000 14662 100 1841 

33 0.050 12822 145 1466 

32 0.100 11356 190 1228 

31 0.150 10127 235 1062 

30 0.200 9066 280 939 

29 0.250 8127 325 843 

28 0.300 7284 370 767 

27 0.350 6517 415 704 

26 0.400 5812 460 652 

25 0.450 5160 505 607 

24 0.500 4553 550 569 

23 0.550 3984 595 536 

22 0.600 3448 640 506 

21 0.650 2942 685 480 

20 0.700 2462 730 367 

19 0.740 2095 766 266 

18 0.770 1828 793 259 

17 0.800 1569 820 169 

16 0.820 1400 838 166 

15 0.840 1235 856 163 

14 0.860 1071 874 160 

13 0.880 911 892 158 

12 0.900 753 910 78 

11 0.910 675 919 77 

10 0.920 598 928 77 

9 0.930 521 937 76 

8 0.940 445 946 76 

7 0.950 369 955 75 

6 0.960 294 964 74 

5 0.970 220 973 74 

4 0.980 146 982 37 

3 0.985 109 987 37 

2 0.990 73 991 36 

1 0.995 36 996 36 

0 1.000 0 1000 0 

11
3.2 MM5

An overview of the physics parameterization configuration used in the IDNR 2002MM5v363 

simulation is provided in Table 3.3. As previously discussed, the configuration emerges from 

the cumulative efforts of several sensitivity studies, in combination with guidance from the AdHoc Meteorological Modeling community. In comparison with the original IDNR 2002 

simulation, the cessation of continuous soil field techniques within the PX LSM is one of the 

most notable modifications.1 With the PX LSM no longer restricted to sequential operation, the 

annual simulation was generated from 95 independent simulations initialized at 12Z and 

integrated through five days (versus 5-day blocks arranged in quarterly sequential simulations in 

the original run). This temporal structure allows maximum air quality modeling flexibility as 

photochemical simulations can be initialized using midnight local time or midnight GMT 

without the need to split any given 24-hour period across multiple MM5 simulation blocks. 

While this methodology does increase the number of runs required to complete an annual 

simulation (versus initialization at 00Z with a 5.5 day run time), the increased computational 

requirements are not prohibitive. An example of the temporal structure is provided in Appendix 

A. To allow for approximately a two week photochemical model spin-up period, the simulation 

started at 12/16/2001 12Z. The completion date occurred at 12Z on 1/1/2003. A 90 second 

timestep was used with output written every hour. The output files were split every 24 hours to 

simplify the post-processing (and photochemical pre-processing) stages. 

Table 3.3 Description of the options selected within the IDNR 2002 annual MM5v363 run.

Option Configuration Details 

Microphysics Mixed-Phase (Reisner I) 

Cumulus Scheme Kain-Fritsch 2 

PBL Asymmetric Convective Model * Required by Pleim-Xiu LSM 

Radiation RRTM Calculated every 15 minutes 

Land Surface Model Pleim-Xiu No continuous soil fields 

Shallow Convection Not enabled 

SST Data source Eta Skin-Temperature 

Snow Cover Effects Considered IFSNOW=1 

Timestep 90 seconds (PX uses an internal 40s timestep)

*

The Asymmetric Convective Model (ACM) is also referred to as the Pleim-Chang PBL. The 

ACM parameterization is a derivative of the Blackadar scheme (Pleim and Chang, 1992). 

 

1

 While discussion of the complete list of configuration variability between the original and 2020MM5v363 

simulations is beyond the scope of this document, additional key updates include: the abandonment of NCEP SST 

data in favor of Eta-Skin temperatures; the addition of the 12 km domain; use of a more recent modeling system 

release; and a new temporal structure. 

12
Additional configuration details include the following: Sea surface temperatures remained 

constant during the simulation as Eta skin temperatures were used as surrogate sea surface 

temperatures. Snow cover effects were considered. Analysis nudging of the temperature, 

mixing ratio, and wind fields was applied above the PBL. At the surface only the wind field was 

nudged. The default nudging strengths of 2.5 x 10-4 and 1.0 x 10-4 were used for the temperature 

and wind fields at 36 and 12 km, respectively. A nudging coefficient of 1.5 x 10-5 was 

established for the mixing ratios at both 36 and 12 km. The rotational wind field was not 

nudged, nor were observational nudging techniques applied. Optimal observational nudging 

methods require a station density not available across a continental scale annual simulation. 

Referencing Baker et al. (2004) the following details are provided: 

Vertical moisture and temperature advection are set to use linear interpolation. Other 

options incorporated include: moist vertical diffusion in clouds, temperature advection 

using potential temperature, diffusion using perturbation temperature, and an upper 

radiative boundary condition. The Pleim-Xiu land surface module requires the addition 

of three variables in the MM5 deck: ISMRD, NUDGE, and IFGROW. ISMRD was set 

to use soil moisture fields from the ETA analyses. NUDGE was assigned to adjust the 

soil moisture data to the analyses fields. Finally, IFGROW was set to option 2, which 

takes vegetative growth into account based on vegetative fraction data from the 

TERRAIN file. 

The configuration of the 12 km grid pictured in Figure 3.1 closely resembles the 36 km grid 

methodology. The explicit exceptions include a decrease in the wind and temperature nudging 

strengths. While the terminology is questionable, the nesting technique employed is commonly 

referred to as “a two-way nested run without feedback”. In this method, the 12 km model 

solution is not feed back to the master domain, but the grids are run simultaneously to allow the 

fine grid to receive boundary condition updates at every timestep. 

3.3 COMPUTATIONAL SUMMARY

Seven dual CPU Linux workstations were acquired to complete the annual simulation. Six 

machines were equipped with dual 3.06 GHz Intel Pentium Xeon processors, with the final 

machine a dual 2.0 GHz processor. Each machine1

 was equipped with 2 Gb of RAM, and Ultra 

320 SCSI local hard drives for model I/O. Upon completion of each run, output data was 

transferred via NFS to a SCSI-IDE RAID array. In summation, 41 wall-clock days were 

required to complete the annual simulation. This represents each machine computing two 

independent simulations simultaneously (essentially each CPU was tasked with one simulation at 

any given time). Open MP was not an available option due to the implementation of PX. 

Approximately 100 wall-clock hours was required for a 3.06 GHz machine to complete two 

simulations running simultaneously. Storage requirements reached 1.1 terabytes, with the 36 km 

simulation occupying 400 Gb and the 12 km data using 700 Gb. 

 

1

 The 2.0 GHz machine had only 1 Gb of onboard RAM. 

13
4. MODEL PERFORMANCE EVALUATION 

4.1 BACKGROUND

No rigid guidelines exist for systematically and objectively evaluating the quality of 

meteorological simulations. However, sound comprehensive philosophies exist. A seven point 

approach outlined by Tesche (1994) provides the framework for a thorough model performance 

evaluation. The framework can be classified into two components: an operational evaluation 

and a scientific evaluation (Emery and Tai, 2001). The scientific evaluation requires rigorous 

examinations of model formulation and algorithm development, methods beyond the scope of 

most modeling projects. Historical development and applications of MM5 within the scientific 

community (including air quality and prognostic projects published through peer-reviewed 

journal articles) must then serve to support the scientific evaluation. Thus the performance 

evaluation of the IDNR 2002MM5v363 annual simulation will focus upon operational criteria. 

4.2 METHODS

Climatic variability, complex mesoscale meteorological phenomena, and scientific unknowns 

contribute to meteorological modeling difficulties and force modelers to take a subjective 

approach to model performance. Objective statistical measures which offer a quantitative model 

assessment exist, but implementation of the metrics is subjective to a degree. For example, 

defining the area over which domain averaged metrics are calculated is a subjective decision, 

buffered only through guidelines. In general, metrics averaged over large meteorological 

modeling domain are avoided, as error cancellation dilutes relevance. Conversely, splitting the 

modeling domain into small subdomains renders sample sizes unrepresentative. The logical 

approach falls well within the bounds of the extremes, leaving optimum subdomain definition 

open to interpretation. As one means of addressing the issue, a subjective grid decomposition 

technique was applied, resulting in the twelve rectangular1

 subdomains pictured in Figure 4.1. 

Model performance measures must also minimally include a review of upper air features in 

tandem with surface statistics. Upper air features are key variables in terms of air quality 

modeling given the importance of fields such as three dimension wind flows and PBL depths. 

Evaluation of the upper atmosphere also introduces a level of complexity exceeding the 

difficulty associated with assessing surface features. The sheer volume of upper air model data, 

in combination with a relatively sparse observing network gathering only twice daily soundings, 

creates problems in terms of scale. A limited set of data analysis tools also restricts the review 

process. In an attempt to achieve a balance between available resources and the level of detailed 

review, the upper air evaluation includes review of PBL features and focuses upon observed 

versus modeled soundings. To improve the efficiency and simplify the review of soundings, a 

new software tool was developed in-house: RAOBPLOT. In the final aspect of the upper air 

evaluation, an independent review of precipitation prediction, conducted by Kirk Baker, is 

briefly summarized. While technically a surface feature, the precipitation evaluation indirectly 

enhances the upper air review given the three dimensional nature of precipitation events. 

 

1

 Processing requirements necessitated that subdomains be simple rectangles defined only through a southwest and 

northeast grid coordinate. 

14
PacificNW

SW 

North 

DesertSW 

CenrapS 

CenrapN GL 

SE 

NE 

MidAtlantic 

OhioVal

Iowa 

Figure 4.1. Decomposition of the continental scale MM5 domain into simple rectangular 

subregions designed for targeted model performance evaluation. Areas of overlap are shaded 

differently and outlines have been added to highlight individual subdomain boundaries. 

4.3 STATISTICALMEASURES

Within the statistical degrees of freedom available to the meteorological modeler, a subset of 

standard statistical measures has emerged, outlined in Table 4.1. These metrics are calculated 

based upon data contained within a given subdomain (See Figure 4.1). Metrics are calculated 

using hourly and daily averages. While no strict criteria establishing acceptable model 

performance exist, the general guidelines established by Emery and Tai (2001) provide a 

community adopted frame of reference. A summary of the guidelines is provided in Table 4.2. 

15
Table 4.1. List of statistical measures commonly discussed in meteorological model evaluations. 

The DH designation represents that both daily and hourly averaged values are calculated for a 

particular metric. Conversely, D or H indicates that the value is available only on a daily or 

hourly average, respectively. 

Statistical Measure Wind Speed Wind 

Direction Temperature Humidity 

Obs. vs Predicted Timeline DH DH DH DH 

Bias DH DH DH DH 

Gross Error D D D D 

Total RMSE DH DH DH 

Systematic RMSE DH DH DH 

Unsystematic RMSE DH DH DH 

Index of Agreement DH DH DH 

Table 4.2. Guidelines for meteorological model performance. Source: Meteorological 

Modeling and Performance Evaluation of the September 13-20, 1999 Ozone Episode (Emery 

and Tai, 2001). Data pertain to daily averaged values.

Wind Speed Wind Direction Temperature Humidity 

RMSE ≤ 2 m/s 

Mean Bias ≤ ±0.5 m/s 

IOA ≥ 0.6 

Gross Error ≤ 30 deg 

Mean Bias ≤± 10 deg 

Gross Error ≤ 2 K 

Mean Bias ≤ ±0.5 K 

IOA ≥ 0.8 

Gross Error ≤ 2 g/kg 

Mean Bias ≤ ±1 g/kg 

IOA ≥ 0.6 

An overview of the significance for each metric is provided by Baker et al. (2004): 

“Bias error (bias) is the degree of correspondence between the mean prediction and the 

mean observation, with lower numbers indicative of better performance. Values less than 

0 indicate under-prediction. The gross error, or mean absolute error, is the mean of the 

absolute value of the residuals from a fitted statistical model. Lower numbers indicate 

better model performance. 

Root Mean Square Error (RMSE) is a good overall measure of model performance. The 

weighting of (prediction-observation) by its square tends to inflate RMSE, particularly 

when extreme values are present. With respect to a good model the root mean square 

error should approach zero. RMSE can be divided into a systematic and unsystematic 

component by least-squares regression. Since differences described by systematic RMSE 

can be described by a linear function, they should be relatively easy to dampen by a new 

parameterization of the model. Unsystematic RMSE can be interpreted as a measure of 

potential accuracy or noise level (Emery et al., 2001). With respect to a good model the 

systematic difference should approach zero while the unsystematic difference approaches 

RMSE. 

16
Index of Agreement is a relative measure of the degree of which predictions are errorfree. The denominator accounts for the model's deviation from the mean of the 

observations as well as to the observations deviation from their mean. It does not provide 

information regarding systematic and unsystematic errors. The index of agreement 

approaches one when model performance is best.” 

The basis of the statistical analysis if formed through a comparison of the modeled fields with 

the Techniques Data Laboratory U.S. and Canada surface hourly observations (ds472.0). Hourly 

and daily averaged bias, error, RMSE (total, systematic, and unsystematic), and index of 

agreement metrics for wind speed, wind direction, temperature and humidity were generated 

using the Metstat program and MS Excel post-processing macro developed by Environ. Time 

series of modeled and observed conditions were also prepared via Metstat. As continental-scale 

domain averaged statistical measures are susceptible to error cancellation, metrics were 

calculated over the twelve subdomains illustrated in Fig. 4.1. 

The volume of data associated with the annual simulation can quickly overwhelm standard time 

series displays or similar attempts at numerical data presentation. As a solution Kirk Baker 

developed and ingenious method of data display. PAVE is used to plot daily metrics, aligned 

vertically by month, and horizontally by date. This allows for an annual graphical display of 

daily averaged metrics in a single plot, simplifying the identification of error trends or pervasive 

biases. Even with this method of simplification, a detailed discussion of all twelve subdomains 

becomes excessive. The statistical analysis therefore focuses upon those regions encompassing 

the CENRAP and Midwest RPO states, primarily the regions: CenrapN, CenrapS, GL (Great 

Lakes), OhioVal (Ohio Valley), and Iowa. 

17
5. SURFACE EVALUATION (36 KM) 

The daily averaged metrics described below are provided graphically in the form of a “Bakergram”. 

The Bakergram, developed by Kirk Baker, allows for the meaningful depiction of an annual set of 

daily averaged statistical values in a single plot. For example, Figure 5.1 consists of a compilation 

of four Bakergrams, one each for the wind speed bias, wind speed error, wind direction bias, and 

wind direction error. Focusing on the wind speed bias Bakergram in Figure 5.1 (top left), 365 daily 

averaged metrics are provided. Twelve columns are provided, which each column containing a 

monthly dataset. The individual days are provided in rows, with the first of the month displayed at 

the top, with days descending from top to bottom. The concept is repeated (for example, see Figure 

5.2) with temperature and mixing ratio metrics plotted. 

5.1 GREAT LAKES

In previous sensitivity studies, the Pleim-Chang/Pleim-Xiu PBL/LSM configuration was found to 

improve wind vector performance versus the use of alterative PBL parameterizations. Consistent 

with this discovery, the wind vector performance in the GL region is encouraging. Wind speed 

metrics are generally favorable, and no clear trends in error or bias are evident (see Figure 5.1). A 

notable caveat, daily metrics may hide inconsistencies occurring within the diurnal profile.1

 

Turning to the wind direction evaluation, again results are satisfactory, with one exception found, an 

increase in the summertime gross error. 

In the Great Lakes region, the problems of greatest concern lie in the wintertime cold temperature 

biases, the warm summertime biases, and the summertime positive moisture biases (See Figure 5.2). 

Examining the temperature biases from a diurnal2 perspective, the warm bias is predominantly 

caused by nighttime temperatures remaining warmer than observed. The cold wintertime 

temperature bias is often traced to underpredicted high temperatures, evening temperatures falling 

too rapidly, and nighttime lows often colder than observed. Caution should be exercised when 

generalizing the wintertime bias trends though, as exceptions are more abundant than with the 

summertime warm biases. Turning to the mixing ratio (humidity) evaluation (see Figure 5.2), 

although the gross error metrics are generally within the statistical guidelines, the summertime 

positive bias is a concerning trend. Only on rare occasions do negative biases occur. The likely 

culprit is MM5’s tendency to overpredict precipitation. 

5.2 NORTHERN AND SOUTHERN CENRAP

In general, the statistical evaluation for the CenrapN subdomain (Figures 5.3 - 5.4) yields results 

similar to the Great Lakes region. A notable exception being the nearly consistently negatively 

biased wind speeds. Examining the wind speed bias in greater detail (through diurnal profiles), this 

fault is predominantly influenced by the underprediction in the daily peak wind speeds. Keeping 

these errors in perspective, the magnitude of the underprediction typically remains below 1 m/s. 

Examining the mixing ratio performance, the most serious issue remains the abundance of 

summertime surface moisture. While arguable trivial, CenrapN does differ from the GL subdomain 

during May, where several surface moisture underpredictions occur. 

 

1

 Diurnal metrics are examined in Chapter 6. 

2

 Ibid. 
Turning to the Southern CENRAP subdomain (Figures 5.5 - 5.6), wind direction performance 

remains encouraging, similar to the performance for the CenrapN and GL regions. As found in 

CenrapN, wind speeds are generally negatively biased, but more pronounced in this region. The 

mixing ratio biases reveal excess moisture, although a drier than observed fall was predicted. 

Examining temperature performance, late winter/early spring temperatures yielded positively biased 

trends, in contrast to the pervasive cold winter biases found in the CenrapN and GL regions. 

Examination of the diurnal profiles revealed the biases were attributable to warm nighttime lows. 

5.3 OHIO VALLEY

Once again, the wind speeds are generally too low, however, the associated error is well within the 

acceptable guidelines. Wind directions errors are also generally small, but an increase in error is 

found in the summer months. Mixing ratios are consistently too moist, except in the mid-October 

timeframe. As in the Great Lakes regions, a cold winter bias is found, while summer temperatures 

remain too warm (predominantly over the nighttime hours). The results are depicted in Figures 5.7 

- 5.8. 

5.4 IOWA 

Within the Iowa subdomain wind vector performance is favorable, with wind speed bias and error 

measures predominantly meeting the statistical goals. Wind directions exhibit greater errors in the 

late summer/early fall timeframe versus the CenrapN and GL subdomains, but are not cause for 

severe alarm (see Figure 5.9). As is common, cold winter and warm summer biases are present 

(Figure 5.10). In terms of the moisture bias, the Iowa domain exhibits greater springtime negative 

moisture bias versus CenrapN, otherwise similar performance is shown (this result is not 

unexpected, given the superposition of the Iowa subdomain over CenrapN). 

5.5 EASTERN REGIONS

A detailed discussion of model performance for all areas is beyond the scope of this document. 

Alternatively, summary remarks are provided. Over the MidAtlantic, no serious abnormalities are 

found beyond the errors identified previously in Central U.S. subdomains. As is common to MM5, 

a positive moisture bias exists, affecting both the MidAtlantic and SE regions. Examining the NE 

region, wind speed, and wind direction errors approach the upper extreme of acceptable 

performance. Again, the moisture bias is positively biased, with errors maximized over the summer 

months. Given moisture carrying capacity is a non-linear function of temperature, the relatively 

small mixing ratio gross errors occurring in the wintertime of regions with colder climates should 

not be interpreted as superior model performance. The daily averaged statistical results are 

provided in Appendix B for each of the individual Eastern subregions. 

5.6 WESTERN REGIONS

The daily averaged statistical results for the western subdomains are also provided in Appendix B. 

The complex topography found in the Western United States clearly introduces a degree of 

modeling difficulty not found in other regions. Performance metrics are discouraging when viewed 

initially, however, the appropriateness of the statistical measures are questionable as model 

resolution is not designed to capture the topographically induced near-field flows affecting many of 

the local observations. 

19
Figure 5.1. Daily averaged wind speed/direction metrics for the Great Lakes (GL) subdomain. 
Figure 5.2. Daily averaged temperature and mixing ratio metrics for the Great Lakes (GL) 

subdomain. 

21
Figure 5.3. Daily averaged wind speed/direction metrics for the CenrapN subdomain. 

22
Figure 5.4. Daily averaged temperature and mixing ratio metrics for the CenrapN subdomain. 

23
Figure 5.5. Daily averaged wind speed/direction metrics for the CenrapS subdomain 

24
Figure 5.6. Daily averaged temperature and mixing ratio metrics for the CenrapS subdomain. 

25
Figure 5.7. Daily averaged wind speed/direction metrics for the OhioVal subdomain. 

26
Figure 5.8. Daily averaged temperature and mixing ratio metrics for the OhioVal subdomain. 

27
Figure 5.9. Daily averaged wind speed/direction metrics for the Iowa subdomain. 

28
Figure 5.10. Daily averaged temperature and mixing ratio metrics for the Iowa subdomain. 

29
6. TWELVE KILOMETER EVALUATION 

6.1 DAILY AVERAGED STATISTICS

Generalizing the impacts of the 12 km domain upon the Great Lakes region, in terms of daily 

averaged metrics, a decrease in simulation accuracy during the winter months is found, while 

only negligible changes occur across the remainder of the year. This trend is prevalent for wind 

speed, wind direction1

 and temperature errors. The wintertime temperature cold bias (found at 

36 km resolution) is thus even more pronounced in the 12 km domain. Mixing ratio statistics 

were generally uninfluenced by domain resolution. These results are depicted in Figures 6.1 – 

6.2, where the Bakergram concept is maintained, however, the results are presented in terms of 

the differences between the 36 and 12 km results. The plots were generated by subtracting the 

36 km daily averaged statistical values from the 12 km data. As the comparison only involves 

gross and root mean square error metrics, negative values indicate an improvement in model 

performance at 12 km resolution. This methodology is maintained for Figures 6.1 - 6.8.

The CenrapN regions shows only minor variations in the temperature fields, with the greatest 

change concentrated to the cooler months, with slight performance disbenefits. Wind direction 

metrics produced a drastically different trend, as nearly all days showed poorer performance. 

Figures 6.3 – 6.4 provide a graphical depiction of the 12 km domain impacts upon the daily 

averaged metrics for this subdomain. 

Over the Ohio Valley, only minor differences were calculated between the 12 and 36 km daily 

averaged statistical results, in general. A slight improvement in the mixing ratio fields was 

computed. As in CenrapN, wind direction gross errors encountered widespread performance 

degradation during the winter and early spring months. Keeping the increasing errors in 

perspective, additional error remained below 3.5 degrees. See Figures 6.5 - 6.6. 

In terms of daily averaged statistical measures, the Iowa subdomain receives few benefits from 

increased resolution. Wind speeds generally exhibit slightly greater error in the winter, spring, 

and fall, while demonstrating little variability during the summer. Consistent with nearby 

subdomains, wind direction performance suffers. While mixing ratios impacts were negligible, 

most months exhibited days with increased temperature error, particularly in the winter. 

Fortunately, gross error degradation remained below 0.5 K. The results are depicted in Figures 

6.7 - 6.8. 

Due to the spatial extent of the 12 km domain, neither the CenrapS domain, nor any other 

subdomain, is eligible for comparison. 

 

1

 With additional errors occurring into the early spring months. 
Figure 6.1. Twelve km domain daily averaged statistical performance for selected wind metrics 

in relation to the 36 km grid for the Great Lakes (GL) subdomain. 

31
Figure 6.2. Twelve km domain daily averaged statistical performance for selected temperature 

and mixing ratio metrics in relation to the 36 km grid for the Great Lakes (GL) subdomain. 

32
Figure 6.3. Twelve km domain daily averaged statistical performance for selected wind metrics 

in relation to the 36 km grid for the CenrapN subdomain. 

33
Figure 6.4. Twelve km domain daily averaged statistical performance for selected temperature 

and mixing ratio metrics in relation to the 36 km grid for the CenrapN subdomain. 

34
Figure 6.5. Twelve km domain daily averaged statistical performance for selected wind metrics 

in relation to the 36 km grid for the OhioVal subdomain. 

35
Figure 6.6. Twelve km domain daily averaged statistical performance for selected temperature 

and mixing ratio metrics in relation to the 36 km grid for the OhioVal subdomain. 

36
Figure 6.7. Twelve km domain daily averaged statistical performance for selected wind metrics 

in relation to the 36 km grid for the Iowa subdomain. 

37
Figure 6.8. Twelve km domain daily averaged statistical performance for selected temperature 

and mixing ratio metrics in relation to the 36 km grid for the Iowa subdomain. 

38
6.2 HOURLY STATISTICS

Additional comparisons between the 36 and 12 km simulations are provided below through 

review of hourly timeseries. Modeled (both 36 and 12 km) versus observed conditions are 

plotted below, with the associated bias also depicted. The hourly time series evaluation 

eliminates the statistical smoothing associated with the daily averaging periods. These charts 

also serve as the diurnal profile data source referenced in previous chapters, however, the 

discussion below will primarily focus upon differences between the 12 and 36 km simulations. 

Assessing the timeseries from a winter (January) and summer (June) monthly subset of the 

annual simulation for the Great Lakes region (Figures 6.9 - 6.10) leads to a general conclusion 

that improvement occurs in the daytime wind speed biases with implementation of the 12 km 

grid, while nighttime disbenefits are observed. At 12 km resolution, the wintertime cold bias is 

even more pronounced versus the 36 km domain, as nighttime low temperatures dip further 

below observed values (Figure 6.11). The ultimate cause for the low temperature bias is 

unknown, but this is not an uncommon feature of MM5 simulations (Ad-Hoc Meteorological 

Modelers Meeting group discussion, 2007). For the GL region, no significant differences are 

found in either temperature or humidity during the summer month of June (Figure 6.12). 

Appendix C provides additional January and June hourly 12 versus 36 km statistical charts for 

the Great Lakes, OhioVal, CenrapN, and Iowa subdomains. 

Observed/Predicted Windspeed

2

3

4

5

6

7

8

 1/ 1

1/ 2

1/ 3

 1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

 1/14

1/15

 1/16

 1/17

 1/18

1/19

 1/20

 1/21

1/22

 1/23

 1/24

1/25

 1/26

 1/27

 1/28

1/29

 1/30

 1/31

m/s

ObsWndSpd 36km Predicted 12km Predict ed

Bias Windspeed

-1.5

-1

-0.5

0

0.5

1

1.5

 1/ 1

 1/ 2

 1/ 3

 1/ 4

 1/ 5

 1/ 6

 1/ 7

 1/ 8

 1/ 9

 1/10

 1/11

 1/12

 1/13

 1/14

 1/15

 1/16

 1/17

 1/18

 1/19

 1/20

 1/21

 1/22

 1/23

 1/24

 1/25

 1/26

 1/27

 1/28

 1/29

 1/30

 1/31

m/s

36km Bias 12km Bias

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

 1/ 1

 1/ 2

 1/ 3

 1/ 4

 1/ 5

 1/ 6

1/ 7

 1/ 8

 1/ 9

 1/10

 1/11

 1/12

 1/13

 1/14

 1/15

1/16

 1/17

 1/18

 1/19

 1/20

 1/21

 1/22

 1/23

 1/24

 1/25

 1/26

1/27

 1/28

 1/29

 1/30

 1/31

deg

ObsWndDir 36km Predict ed 12km Predict ed

Bias Wind Direction

-30

0

30

 1/ 1

 1/ 2

 1/ 3

 1/ 4

1/ 5

 1/ 6

 1/ 7

1/ 8

 1/ 9

 1/10

 1/11

 1/12

 1/13

 1/14

 1/15

 1/16

1/17

 1/18

 1/19

 1/20

 1/21

 1/22

1/23

 1/24

 1/25

 1/26

 1/27

 1/28

 1/29

 1/30

 1/31

deg

36km Bias WD 12km Bias WD

bias shifted 

Great Lakes Domain

Figure 6.9. Twelve and 36 km hourly wind vector statistics for the Great Lakes subdomain for 

January, 2002. 

39
Observed/Predicted Windspeed

2

4

6

8

 6/ 1

 6/ 2

 6/ 3

6/ 4

 6/ 5

 6/ 6

6/ 7

 6/ 8

 6/ 9

6/10

 6/11

 6/12

6/13

 6/14

 6/15

 6/16

 6/17

 6/18

 6/19

 6/20

 6/21

 6/22

 6/23

 6/24

 6/25

 6/26

6/27

 6/28

 6/29

 6/30

m/s

ObsWndSpd 36km Predict ed 12km Predict ed

Bias Windspeed

-2

-1

0

1

2

 6/ 1

 6/ 2

 6/ 3

6/ 4

 6/ 5

 6/ 6

6/ 7

 6/ 8

 6/ 9

6/10

 6/11

 6/12

6/13

 6/14

 6/15

 6/16

 6/17

 6/18

 6/19

 6/20

 6/21

 6/22

 6/23

 6/24

 6/25

 6/26

6/27

 6/28

 6/29

 6/30

m/s

36 km Bias 12 km Bias

Bias Wind Direction

-30

0

30

 6/ 1

 6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

 6/12

 6/13

 6/14

 6/15

 6/16

6/17

 6/18

 6/19

 6/20

6/21

 6/22

 6/23

 6/24

6/25

 6/26

 6/27

 6/28

 6/29

 6/30

deg

36km Bias 12km Bias

Great Lakes Domain

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

 6/ 1

 6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

 6/ 7

6/ 8

 6/ 9

 6/10

 6/11

6/12

 6/13

 6/14

6/15

 6/16

 6/17

 6/18

 6/19

 6/20

 6/21

 6/22

 6/23

 6/24

 6/25

 6/26

 6/27

 6/28

 6/29 6/30

deg

ObsWndDir 36 km PredWnd 12 km PredWndDir

Figure 6.10. Twelve and 36 km hourly wind vector statistics for the Great Lakes subdomain for 

June, 2002. 

40
Bias Temperature

-5

-4

-3

-2

-1

0

1

2

1/ 1

1/2

 1/ 3

 1/4

 1/ 5

1/6

 1/ 7

 1/ 8

 1/ 9

 1/10

1/11

 1/12

1/13

 1/14

 1/15

 1/16

1/17

 1/18

1/19

 1/20

1/21

1/22

1/23

 1/24

 1/25

 1/26

1/27

1/28

 1/29

 1/30

1/31

K

36 km Bias T 12 km Bias T

Observed/Predicted Temperature

255

260

265

270

275

280

 1/ 1

1/2

 1/ 3

 1/4

 1/ 5

1/6

 1/ 7

 1/ 8

 1/ 9

 1/10

1/11

 1/12

1/13

 1/14

 1/15

 1/16

1/17

 1/18

1/19

 1/20

1/21

1/22

1/23

 1/24

 1/25

 1/26

1/27

 1/28

 1/29

 1/30

1/31

K

ObsTemp 36 km Pred_T 12 km Pred_T

Predicted/Observed Humidity

0

1

2

3

4

5

6

1/1

1/2

 1/ 3

 1/ 4

 1/ 5

1/6

1/7

 1/ 8

 1/ 9

 1/10

 1/11

 1/12

1/13

 1/14

 1/15

 1/16

 1/17

1/18

1/19

 1/20

 1/21

1/22

1/23

1/24

 1/25

 1/26

 1/27

1/28

 1/29

 1/30

 1/31

g/kg

ObsHum 36km Pred q 12 k Pred q

Bias Humidity

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1/1

1/2

 1/ 3

 1/ 4

 1/ 5

1/6

1/7

 1/ 8

 1/ 9

 1/10

 1/11

 1/12

1/13

 1/14

 1/15

 1/16

 1/17

1/18

1/19

 1/20

 1/21

1/22

1/23

1/24

 1/25

 1/26

 1/27

1/28

 1/29

 1/30

 1/31

g/kg

36 km Bias q 12 km Bias q

Predicted/Observed Humidity

0

1

2

3

4

5

6

1/1

1/2

 1/ 3

 1/ 4

 1/ 5

1/6

1/7

 1/ 8

 1/ 9

 1/10

 1/11

 1/12

1/13

 1/14

 1/15

 1/16

 1/17

1/18

1/19

 1/20

 1/21

1/22

1/23

1/24

 1/25

 1/26

 1/27

1/28

 1/29

 1/30

 1/31

g/kg

ObsHum 36km Pred q 12 k Pred q

Bias Humidity

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1/1

1/2

 1/ 3

 1/ 4

 1/ 5

1/6

1/7

 1/ 8

 1/ 9

 1/10

 1/11

 1/12

1/13

 1/14

 1/15

 1/16

 1/17

1/18

1/19

 1/20

 1/21

1/22

1/23

1/24

 1/25

 1/26

 1/27

1/28

 1/29

 1/30

 1/31

g/kg

36 km Bias q 12 km Bias q

12km even colder

Great Lakes Domain

Figure 6.11. Twelve and 36 km hourly temperature and moisture statistics for the Great Lakes 

subdomain for January, 2002. 

41
Observed/Predicted Temperature

270

275

280

285

290

295

300

305

310

 6/ 1

 6/ 2

 6/ 3

6/4

6/5

6/ 6

 6/7

6/8

6/9

 6/10

6/11

6/12

 6/13

6/14

6/15

6/16

6/17

 6/18

6/19

6/20

 6/21

 6/22

 6/23

6/24

 6/25

 6/26

6/27

6/28

6/29

6/30

K

ObsTemp 36 km 12 km

Bias Temperature

-3

-2

-1

0

1

2

3

 6/ 1

 6/ 2

 6/ 3

6/4

6/5

6/6

 6/7

6/8

6/9

 6/10

6/11

6/12

 6/13

6/14

6/15

6/16

6/17

 6/18

6/19

6/20

 6/21

 6/22

 6/23

6/24

 6/25

 6/26

6/27

6/28

6/29

6/30

K

36 km Bias 12km Bias

Great Lakes Domain

Predicted/Observed Humidity

0

2

4

6

8

10

12

14

16

18

6/ 1

 6/ 2

 6/ 3

 6/ 4

 6/5

6/6

 6/ 7

 6/ 8

 6/9

 6/10

 6/11

 6/12

6/13

 6/14

 6/15

 6/16

 6/17

6/18

 6/19

 6/20

 6/21

6/22

6/23

6/24

 6/25

6/26

6/27

6/28

6/29

6/30

g/kg

ObsHum 36 km Predicted 12k m Predict ed

Bias Humidity

-2

-1

0

1

2

3

4

 6/ 1

 6/ 2

 6/ 3

6/4

6/5

6/ 6

 6/7

6/8

6/9

 6/10

6/11

6/12

 6/13

6/14

6/15

6/16

6/17

 6/18

6/19

6/20

 6/21

 6/22

 6/23

6/24

 6/25

 6/26

6/27

6/28

6/29

6/30

g/kg

36 km bias 12 km Bias

no significant changes

Figure 6.12. Twelve and 36 km hourly temperature and moisture statistics comparison for the 

Great Lakes subdomain for June, 2002. 

42
7. UPPER AIR EVALUATION 

7.1 SOUNDINGS

A comprehensive assessment of model performance cannot be completed through the evaluation 

of surface statistical measures alone. A rigorous evaluation requires the examination of 

additional features such as precipitation fields, PBL depths, and vertical profiles of temperature, 

moisture, and wind vectors. As readily available tools have not been identified which yield 

objective measures of such parameters, evaluations are typically subjective. A precipitation 

evaluation of the 36 km dataset has been completed by Kirk Baker (Baker et al., 2004) and is 

summarized below. In combination with the precipitation evaluation, the most efficient method 

available for an upper air analysis is to focus upon radiosonde observations. To aid in the review 

of upper air feature, the IDNR created the RAOBPLOT software tool that efficiently displays 

modeled versus observed radiosonde upper air measurements. With twice-daily soundings 

available from approximately 70 observing stations, roughly 51,100 modeled versus observed 

soundings are available for examination from the 36 km annual simulation alone. Clearly a 

complete examination is resource prohibitive. The volume of data available, in combination 

with only inefficient subjective methods for evaluations highlights a current deficiency in annual 

scale regional modeling applications. While inelegant, the immediately practicable solution 

requires a targeted review of specific data. 

A brief review of the modeled versus observed sounding for many sites in the Central U.S. was 

conducted, with no terminal deficiencies discovered. A more focused evaluation upon the 

Davenport, Iowa, station was completed over the simulated summer months, with the following 

conclusions reached: Upper level wind vectors are well simulated. The temperature fields 

below approximately 900 mb yielded a tendency toward underprediction at 0Z, while the 

moisture fields were generally overstated during the same region and time. At 12Z, temperatures 

were generally underpredicted below 900 mb. In terms of estimated PBL depths, the mixed 

layer commonly appears shallower than observed. While error is never desired, in terms of 

modeling air quality (in a conservative sense) a shallow PBL is preferred versus excessive depth. 

A sample of the observed versus modeled sounding produced by RAOBPLOT is provided in 

Figure 7.1. 

7.2 PRECIPITATION

Kirk Baker with the Lake Michigan Air Directors Consortium was provided a complete copy of 

the 36 km meteorological dataset and subsequently completed a model performance evaluation 

examining precipitation fields. In summary, both rainfall totals and precipitation spatial 

coverage are generally well simulated in the fall, winter, and springtime periods. As is common 

with many MM5 simulations, summertime precipitation events produce an excess of 

precipitation. Rainfall patterns also exhibit greater spatial coverage than observed. Additional 

detail, including graphical representation of predicted and observed rainfall, is available in Baker 

et al., 2004. 

43
Figure 7.1. Sample ROABPLOT observed versus predicted (36 km domain) sounding for 

Davenport Iowa, on July 13, 2007, at 0Z. Wind speed and directions are accurately simulated 

throughout the depth of the sounding. The temperature profile performance is more than 

adequate. As is common, a positive moisture bias exists at (and above the surface), while the 

estimated PBL depth remains too shallow. 

44
7.3 PBLDEPTHS

Additional upper air analyses included a limited comparison of the 36 and 12 km predicted PBL 

heights. Figure 7.2 provides an example comparison. As expected, the degree of agreement 

between the 36 and 12 km results exceeds variability. Areas in Western Illinois and Eastern 

Texas (among others) do display deviations. In Eastern Texas, MM5 predicts a precipitation 

event (which is weakly supported by observations, see Figures 7.3 - 7.4). The reduction in PBL 

heights in Western Illinois would appear to be precipitation driven as well, but no convective or 

non-convective rainfall was predicted by MM5 during this time. The observed radar 

reflectivities also suggests no precipitation occurred during this time. In summary, the 12 km 

grid yields improved feature detail yet the accuracy of such fields, across a continental scale 

annual simulation, is difficult to assess within a reasonable timeframe.

Figure 7.2. PBL heights predicted by MM5 for June 23, 2002, at 19Z. for the 12 and 36 km 

modeling domains. 

45
Figure 7.3. Observed conditions on June 24, 0Z. 

46
Figure 7.3. Radar reflectivity on June 23, 2002 at 19Z. 

47
8. CONCLUSIONS 

In the northern half of the Central U.S. through the Ohio River valley, the surface statistical 

evaluation reveals a dominant wintertime cold bias, with cool conditions typically present in the 

evening hours, while overly aggressive nighttime lows and weak high temperatures also 

contribute to the cold bias. The summer months exhibit a warm bias, attributable to the 

overprediction of nighttime temperatures. Wind speed and direction predictions over the central 

and northern Central U.S. exhibit low statistical error and provide for an increase in model 

confidence. Continuing the evaluation into the Western U.S. yields a reduction in model 

confidence, as error measures increase across all fields. As discussed, this result is not 

completely unexpected given complex Western topography. Regions within the Eastern U.S. 

demonstrate prediction skill above Western regions, yet statistical accuracy falls below that 

found in the Midwest. 

Expanding the evaluation into upper air features reveals no fundamental flaws jeopardizing the 

adequacy of the simulation in terms of air quality modeling. A tendency to slightly underpredict 

summertime PBL depths over Eastern Iowa was discovered. In subjective terms, such error is 

acceptable as perfect model performance is unattainable. A similar conclusion is reached for the 

precipitation shortfalls discussed by Baker et al., 2004. 

Within the Central U.S, increasing the horizontal resolution from 36 to 12 km yielded no benefits 

from a surface-feature statistical evaluation perspective. Within the Great Lakes subdomain, the 

12 km simulation appears to improve daytime wind speed predictions, however, nighttime 

predictions suffer. Overall, wind speed error showed little variability between the 36 and 12 km 

domains. Beyond the statistical evaluation, additional field detail is resolved by the 12 km 

domain as expected. As in the upper air analysis for the 36 km grid, no fundamental flaws were 

identified in review of 12 km upper air features. 

In summary, the statistical evaluation yields results predominantly within acceptable guidelines 

for the principal regions of interest (the States near and within LADCO and the northern twothirds of CENRAP). Concurrently, no major simulation deficiencies were revealed during the 

upper air review. The 36 and 12 km Iowa DNR 2002MM5v363 datasets are thus judged 

acceptable for use in regional scale air quality modeling studies focused within the central United 

States. 

48
9. REFERENCES 

Anthes, R. A. and T. T. Warner, 1978: The Development of Mesoscale Models Suitable for Air 

Pollution and Other Mesometeorological Studies. Monthly Weather Review, 106, 1045-

1078. 

Baker, K., 2002: MM5 Sensitivity for the National RPO Grid, Presented at the Ad-Hoc 

Meteorological Modeling Meeting, Des Plaines, IL, 2002. 

Baker, Kirk, Matthew Johnson and Steven King, 2004: Meteorological Modeling Performance 

Summary For Application to PM2.5/Haze/Ozone Modeling Projects. 

http://ladco.org/tech/photo/docs/mm5_protocol_dec2004.pdf

Chen, F., and J. Dudhia, 2001: Coupling an advanced land-surface/hydrology model with the 

Penn State/NCAR MM5 modeling system. Part I: Model implementation and sensitivity. 

Monthly Weather Review, 129, 569-585. 

Dudhia, J., 1993: A Non-hydrostatic Version of the Penn State/NCAR Mesoscale Model: 

Validation Tests and Simulation of an Atlantic Cyclone and Cold Front. Monthly Weather 

Review, 121, 1493-1513. 

Emery, C., and Edward Tai, 2001: Meteorological Modeling and Performance Evaluation of the 

September 13-20, 1999 Ozone Episode. Prepared for the Texas Near Non-Attainment 

Areas through the Alamo Area Council of Governments, by ENVIRON International 

Corp, Novato, CA. 

McNally, Dennis, 2003: Annual Application of MM5 for Calendar Year 2001. Prepared for Pat 

Dolwick, USEPA, March 31st 2003. 

Olerud, Donald, K. Alapaty, and N. Wheeler, 2000: Meteorological Modeling of 1996 for the 

United States with MM5. Prepared for Normal Possiel, Client Representative, U.S. 

EPA/OAQPS. EPA Task Order Number CAA689805 

Olerud, Don and Aaron Sims, 2004: Draft Protocol for Annual MM5 Modeling in Support of 

VISTAS (Visibility Improvement – State and Tribal Association) (DRAFT), Prepared for 

Mr. Mike Abraczinskas, VISTAS Technical Analysis Workgroup. 

Pleim, J. E., and J. S. Chang, 1992: A non-local closure model for vertical mixing in the 

convective boundary layer. Atmospheric Environment, 26A, 965-981. 

Stauffer, D. R., and N. L. Seaman, 1990: Use of Four Dimensional Data Assimilation in a 

Limited-Area Mesoscale Model. Part I: Experiments with Synoptic Data. Monthly 

Weather Review, 118, 1250-1277. 

49
50

Stauffer, D. R., and N. L. Seaman. 1991: Use of Four Dimensional Data Assimilation in a 

Limited-Area Mesoscale Model. Part II: Effects of Data Assimilation within the Planetary 

Boundary Layer. Monthly Weather Review, 119, 734-754 

Tesche, T. W. 1994. “Evaluation Procedures for Regional Emissions, Meteorological, and 

Photochemical Models.” Presented at the 86th Annual Meeting of the Air and Waste 

Management Association, 14-18 June, Denver, CO. 

Xiu, A., and J. E. Pleim, 2000: Development of a land surface model. Part I: Application in a 

mesoscale meteorology model. Journal of Applied Meteorology, 40, 192-209. 
APPENDIX A 

Temporal structure example for the 2002 annual simulation. 

Start BLOCK End BLOCK Start (Z) End (Z) Filename M

Start (Z) 

12/28/2001 12:00 1/2/2002 12:00 12/28/2001 12:00 12/28/2001 12:00 MMOUT_DOMAIN1_00 

12/28/2001 13:00 12/29/2001 12:00 MMOUT_DOMAIN1_01 12/29/2001 012/29/2001 13:00 12/30/2001 12:00 MMOUT_DOMAIN1_02 12/29/2001 112/30/2001 13:00 12/31/2001 12:00 MMOUT_DOMAIN1_03 12/30/2001 1

12/31/2001 13:00 1/1/2002 12:00 MMOUT_DOMAIN1_04 12/31/2001 1

1/1/2002 13:00 1/2/2002 12:00 MMOUT_DOMAIN1_05 1/1/2002 13

1/1/2002 12:00 1/6/2002 12:00 1/1/2002 12:00 1/1/2002 12:00 MMOUT_DOMAIN1_00 

1/1/2002 13:00 1/2/2002 12:00 MMOUT_DOMAIN1_01 1/2/2002 0:

1/2/2002 13:00 1/3/2002 12:00 MMOUT_DOMAIN1_02 1/2/2002 13

1/3/2002 13:00 1/4/2002 12:00 MMOUT_DOMAIN1_03 1/3/2002 131/4/2002 13:00 1/5/2002 12:00 MMOUT_DOMAIN1_04 1/4/2002 131/5/2002 13:00 1/6/2002 12:00 MMOUT_DOMAIN1_05 1/5/2002 13

1/5/2002 12:00 1/10/2002 12:00 1/5/2002 12:00 1/5/2002 12:00 MMOUT_DOMAIN1_00 

1/5/2002 13:00 1/6/2002 12:00 MMOUT_DOMAIN1_01 1/6/2002 0:

1/6/2002 13:00 1/7/2002 12:00 MMOUT_DOMAIN1_02 1/6/2002 13

1/7/2002 13:00 1/8/2002 12:00 MMOUT_DOMAIN1_03 1/7/2002 131/8/2002 13:00 1/9/2002 12:00 MMOUT_DOMAIN1_04 1/8/2002 131/9/2002 13:00 1/10/2002 12:00 MMOUT_DOMAIN1_05 1/9/2002 13
APPENDIX B 

Daily averaged metrics from the 36 km simulation. 

Eastern Subdomains

52
53
54
55
56
57
Western Subdomains

58
59
60
61
62
63
64
65
APPENDIX C 

Hourly statistical results for both the 36 and 12 km grids. 

CenrapN: January 2002

Observed/Predicted Windspeed

0

2

4

6

8

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

m/s

ObsWndSpd 36 km winds 12 km winds

Bias Windspeed

-2.5

-1.5

-0.5

0.5

1.5

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

m/s

36km Bias 12km Bias

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

 1/ 1

1/ 2

1/ 3

 1/ 4

1/ 5

 1/ 6

1/ 7

 1/ 8

1/ 9

1/10

 1/11

1/12

 1/13

1/14

 1/15

 1/16

1/17

1/18

 1/19

1/20

 1/21

1/22

 1/23

1/24

 1/25

 1/26

1/27

 1/28

1/29

 1/30

1/31

deg

ObsWndDir 36km Predict ed 12km Predict ed

Bias Wind Direction

-30

0

30

 1/ 1

1/ 2

 1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

1/ 8

 1/ 9

1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

1/18

 1/19

1/20

 1/21

1/22

 1/23

1/24

 1/25

1/26

 1/27

1/28

 1/29

 1/30

1/31

deg

36km Bias WD 12km Bias WD

66
Observed/Predicted Temperature

240

245

250

255

260

265

270

275

280

285

290

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

K

ObsTemp 36 km Pred_T 12 km Pred_T

Bias Temperature

-5

-4

-3

-2

-1

0

1

2

3

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

K

36 km Bias T 12 km Bias T

Predicted/Observed Humidity

0

1

2

3

4

5

6

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

g/kg

ObsHum 36km Pred q 12 k Pred q

Bias Humidity

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

g/kg

36 km Bias q 12 km Bias q

67
CenrapN: June 2002

Observed/Predicted Windspeed

0

5

10

 6/ 1

6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

6/ 7

6/ 8

 6/ 9

6/10

6/11

 6/12

6/13

 6/14

6/15

 6/16

6/17

6/18

 6/19

6/20

 6/21

 6/22

6/23

 6/24

 6/25

6/26

 6/27

6/28

6/29

 6/30

m/s

ObsWndSpd 36 km winds 12 km winds

Bias Windspeed

-2

-1

0

1

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

m/s

36 km Bias 12 km Bias

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

 6/ 1

6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

6/ 7

6/ 8

 6/ 9

6/10

6/11

 6/12

6/13

 6/14

6/15

 6/16

6/17

6/18

 6/19

6/20

 6/21

 6/22

6/23

 6/24

 6/25

6/26

 6/27

6/28

6/29

 6/30

deg

ObsWndDir 36 km PredWnd 12 km PredWndDir

Bias Wind Direction

-30

0

30

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

deg

36km Bias 12km Bias

68
Observed/Predicted Temperature

270

275

280

285

290

295

300

305

310

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

K

ObsTemp 36 km Pred_T 12 km Pred_T

Bias Temperature

-2

-1

0

1

2

3

4

 6/ 1

6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

6/ 7

6/ 8

 6/ 9

6/10

6/11

 6/12

6/13

 6/14

6/15

 6/16

6/17

6/18

 6/19

6/20

 6/21

 6/22

6/23

 6/24

 6/25

6/26

 6/27

6/28

6/29

 6/30

K

36 km Bias T 12 km Bias T

Predicted/Observed Humidity

0

5

10

15

20

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

g/kg

ObsHum 36km Pred q 12 k Pred q

Bias Humidity

-2

-1

0

1

2

3

 6/ 1

6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

6/ 7

6/ 8

 6/ 9

6/10

6/11

 6/12

6/13

 6/14

6/15

 6/16

6/17

6/18

 6/19

6/20

 6/21

 6/22

6/23

 6/24

 6/25

6/26

 6/27

6/28

6/29

 6/30

g/kg

36 km Bias q 12 km Bias q

69
OhioVal: January 2002

Observed/Predicted Windspeed

0

2

4

6

8

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

m/s

ObsWndSpd 36 km winds 12 km winds

Bias Windspeed

-2

-1

0

1

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

m/s

36 km Bias 12 km Bias

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

deg

ObsWndDir 36 km PredWnd 12 km PredWndDir

Bias Wind Direction

-30

0

30

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

deg

36km Bias 12km Bias

70
Observed/Predicted Temperature

245

250

255

260

265

270

275

280

285

290

295

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

K

ObsTemp 36 km Pred_T 12 km Pred_T

Bias Temperature

-5

-4

-3

-2

-1

0

1

2

3

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

K

36 km Bias T 12 km Bias T

Predicted/Observed Humidity

0

1

2

3

4

5

6

7

8

9

10

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

g/kg

ObsHum 36km Pred q 12 k Pred q

Bias Humidity

-1

-0.5

0

0.5

1

1.5

2

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

g/kg

36 km Bias q 12 km Bias q

71
OhioVal: June 2002

Observed/Predicted Windspeed

0

2

4

6

8

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

m/s

ObsWndSpd 36 km winds 12 km winds

Bias Windspeed

-1.5

-1

-0.5

0

0.5

1

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

m/s

36 km Bias 12 km Bias

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

deg

ObsWndDir 36 km PredWnd 12 km PredWndDir

Bias Wind Direction

-30

0

30

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

deg

36km Bias 12km Bias

72
Observed/Predicted Temperature

275

280

285

290

295

300

305

310

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

K

ObsTemp 36 km Pred_T 12 km Pred_T

Bias Temperature

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

K

36 km Bias T 12 km Bias T

Predicted/Observed Humidity

0

2

4

6

8

10

12

14

16

18

20

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

g/kg

ObsHum 36km Pred q 12 k Pred q

Bias Humidity

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

3.5

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

g/kg

36 km Bias q 12 km Bias q

73
Iowa: January 2002

Observed/Predicted Windspeed

0

5

10

15

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

m/s

ObsWndSpd 36 km winds 12 km winds

Bias Windspeed

-3.5

-2.5

-1.5

-0.5

0.5

1.5

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

m/s

36km Bias 12km Bias

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

 1/ 1

1/ 2

1/ 3

 1/ 4

1/ 5

 1/ 6

1/ 7

 1/ 8

1/ 9

1/10

 1/11

1/12

 1/13

1/14

 1/15

 1/16

1/17

1/18

 1/19

1/20

 1/21

1/22

 1/23

1/24

 1/25

 1/26

1/27

 1/28

1/29

 1/30

1/31

deg

ObsWndDir 36km Predict ed 12km Predict ed

Bias Wind Direction

-30

0

30

 1/ 1

1/ 2

 1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

1/ 8

 1/ 9

1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

1/18

 1/19

1/20

 1/21

1/22

 1/23

1/24

 1/25

1/26

 1/27

1/28

 1/29

 1/30

1/31

deg

36km Bias WD 12km Bias WD

74
Observed/Predicted Temperature

230

240

250

260

270

280

290

300

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

K

ObsTemp 36 km Pred_T 12 km Pred_T

Bias Temperature

-5

-4

-3

-2

-1

0

1

2

3

4

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

K

36 km Bias T 12 km Bias T

Predicted/Observed Humidity

0

1

2

3

4

5

6

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

g/kg

ObsHum 36km Pred q 12 k Pred q

Bias Humidity

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1/ 1

 1/ 2

1/ 3

1/ 4

 1/ 5

1/ 6

 1/ 7

 1/ 8

1/ 9

 1/10

 1/11

1/12

 1/13

1/14

 1/15

1/16

 1/17

 1/18

1/19

 1/20

1/21

1/22

 1/23

1/24

1/25

 1/26

1/27

 1/28

 1/29

1/30

 1/31

g/kg

36 km Bias q 12 km Bias q

75
Iowa: June 2002

Observed/Predicted Windspeed

0

5

10

15

 6/ 1

6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

6/ 7

6/ 8

 6/ 9

6/10

6/11

 6/12

6/13

 6/14

6/15

 6/16

6/17

6/18

 6/19

6/20

 6/21

 6/22

6/23

 6/24

 6/25

6/26

 6/27

6/28

6/29

 6/30

m/s

ObsWndSpd 36 km winds 12 km winds

Bias Windspeed

-4

-2

0

2

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

m/s

36 km Bias 12 km Bias

Observed/Predicted Wind Direction

0

60

120

180

240

300

360

 6/ 1

6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

6/ 7

6/ 8

 6/ 9

6/10

6/11

 6/12

6/13

 6/14

6/15

 6/16

6/17

6/18

 6/19

6/20

 6/21

 6/22

6/23

 6/24

 6/25

6/26

 6/27

6/28

6/29

 6/30

deg

ObsWndDir 36 km PredWnd 12 km PredWndDir

Bias Wind Direction

-30

0

30

6/ 1

 6/ 2

6/ 3

6/ 4

 6/ 5

6/ 6

 6/ 7

 6/ 8

6/ 9

 6/10

 6/11

6/12

 6/13

6/14

 6/15

6/16

 6/17

 6/18

6/19

 6/20

6/21

6/22

 6/23

6/24

6/25

 6/26

6/27

 6/28

 6/29

6/30

deg

36km Bias 12km Bias

76
Observed/Predicted Temperature

270

275

280

285

290

295

300

305

310

 6/ 1

6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

6/ 7

6/ 8

 6/ 9

6/10

6/11

 6/12

6/13

 6/14

6/15

 6/16

6/17

6/18

 6/19

6/20

 6/21

 6/22

6/23

 6/24

 6/25

6/26

 6/27

6/28

6/29

 6/30

K

ObsTemp 36 km Pred_T 12 km Pred_T

Bias Temperature

-3

-2

-1

0

1

2

3

4

 6/ 1

6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

6/ 7

6/ 8

 6/ 9

6/10

6/11

 6/12

6/13

 6/14

6/15

 6/16

6/17

6/18

 6/19

6/20

 6/21

 6/22

6/23

 6/24

 6/25

6/26

 6/27

6/28

6/29

 6/30

K

36 km Bias T 12 km Bias T

Predicted/Observed Humidity

0

5

10

15

20

 6/ 1

6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

6/ 7

6/ 8

 6/ 9

6/10

6/11

 6/12

6/13

 6/14

6/15

 6/16

6/17

6/18

 6/19

6/20

 6/21

 6/22

6/23

 6/24

 6/25

6/26

 6/27

6/28

6/29

 6/30

g/kg

ObsHum 36km Pred q 12 k Pred q

Bias Humidity

-4

-3

-2

-1

0

1

2

3

4

 6/ 1

6/ 2

 6/ 3

 6/ 4

6/ 5

 6/ 6

6/ 7

6/ 8

 6/ 9

6/10

6/11

 6/12

6/13

 6/14

6/15

 6/16

6/17

6/18

 6/19

6/20

 6/21

 6/22

6/23

 6/24

 6/25

6/26

 6/27

6/28

6/29

 6/30

g/kg

36 km Bias q 12 km Bias q

77