# Meteorological Model Performa nce Evaluation of an Annual 2002 MM5 (version 3.6.3) Simulation

Matthew T. Johnson Iowa Department of Natural Resources

2004 - 2007

v2.0.4

# Table of Contents

1. Introduction ................................................................................................................ . 3 1.1 Background....................................................................................................... 3

1.2 Model Selection ................................................................................................ 3 2. Sensitivity Projects...................................................................................................... 4 3. Modeling System Configuration ................................................................................. 5

3.1 Overview........................................................................................................... 5

3.1.1 Terrain......................................................................................................... 5

3.1.2 PREGRID/REGRIDDER ........................................................................... 5
3.1.3 LIITLE_R ................................................................................................. 10

3.1.4 INTERPF .................................................................................................. 10

3.2 MM5 ............................................................................................................... 12 3.3 Computational Summary ................................................................................ 13

4. Model Performance Evaluation................................................................................. 14 4.1 Background..................................................................................................... 14

4.2 Methods........................................................................................................... 14

4.3 Statistical Measures ........................................................................................ 15 5. Surface Evaluation (36 km)....................................................................................... 18

5.1 Great Lakes ..................................................................................................... 18 5.2 Northern and Southern Cenrap ....................................................................... 18 5.3 Ohio Valley..................................................................................................... 19

5.4 Iowa................................................................................................................. 19 5.5 Eastern Regions .............................................................................................. 19 5.6 Western Regions ............................................................................................. 19

6. Twelve Kilometer Evaluation ................................................................................... 30 6.1 Daily Averaged Statistics................................................................................ 30 6.2 Hourly Statistics.............................................................................................. 39 7. Upper Air Evaluation ................................................................................................ 43 7.1 Soundings........................................................................................................ 43 7.2 Precipitation .................................................................................................... 43 7.3 PBL Depths..................................................................................................... 45

8. Conclusions ............................................................................................................... 48

9. References ................................................................................................................. 4 9 Appendix A....................................................................................................................... 51 Appendix B ....................................................................................................................... 52

Eastern Subdomains................................................................................................... 52

Western Subdomains ................................................................................................. 58 Appendix C ....................................................................................................................... 66 CenrapN: January 2002 ............................................................................................ 66 CenrapN: June 2002 ................................................................................................. 68 OhioVal: January 2002............................................................................................. 70

OhioVal: June 2002 .................................................................................................. 72 Iowa: January 2002................................................................................................... 74

Iowa: June 2002........................................................................................................ 76

2

# 1. INTRODUCTION

1.1 B ACKGROUND


# Projects pursuing PM2.5, 8-hour ozone, and regiona l haze are generating modeling requirements at spatial and temporal scales only recently confronted within the regulatory air quality community. The scope of recent legislative and executive decisi ons has created the need to implement sophisticated models developed fo r regional scale multi-pollutant environments encompassing diverse climatological regimes. Computational limitations have historically bound the modeler’s ability to investigate br oad and complex scenarios with sufficient resolution. Exponential growth in computational efficiency has par tially minimized this hurdle. As scientific theory and model complexity evolve, computational innovations remain moderately offset. Currently, a balance has been achieved which permits the development of large modeling

databases such as annual c ontinental scale simulations.

Annual continental scale air quality simulations require the implementation of a triumvirate modeling system composed of meteorolog ical, emissions, and air quality models. Meteorological modeling is the first component a ddressed as meteorological data supports both the emissions and air quality models. In pr eparation for regulator y requirements involving regional haze, PM2.5, and ozone, the Iowa Depart ment of Natural Resources (IDNR) developed a continental scale annual meteorological dataset designed for use in air quality applications. This document details the methods employed to create the annual meteor ological simulation and provides performance evaluation results.



# 1.2 M ODEL 

# S ELECTION


# Due to scientific progression, historical applica tion, community support, and availability, the Fifth Generation Penn State University/National Center for Atmospheric Research Mesoscale Model (MM5) was selected for the development of an annual meteorological dataset. Originally

formulated in the 1970s at Penn State and fi rst documented by Anthes and Warner (1978), the MM5 modeling system maintains its status as a state-of-the-science 1 

# model through enhancements provided by a broad user community ( e.g. Chen and Dudhia, 2001; Dudhia, 1993; Stauffer and Seaman, 1990; Stauffer and Se aman, 1991; Xiu and Pleim, 2000). The MM5 modeling system is routinely employed in operati onal forecasting frameworks as well as research applications spanning meteorologi cal disciplines from synoptic to mesoscale. Utilization of MM5 within air quality applications is also a conventional practice. The MM5 modeling system was recently selected to generate three c ontinental scale annual simulations: 1996, 2001, and

2002. The 1996 and 2001 simulations were conducte d through EPA contracts (Olerud et al., 2000; McNally, 2003). The 2002 simulation was conducted in support of regional haze modeling for the Visibility Improvement – Stat e and Tribal Association of the Southeast (VISTAS) regional planning organization (RPO) (Olerud and Sims, 2004). This list is not exhaustive as both public and pr ivate organizations continue to pursue annual meteorological modeling episodes.

Additional information regarding MM5 is available at: http://www.mmm.ucar.edu/mm5/

1 

# True during project implementation. MM5 is no longer regularly updated as the focus has shifted to WRF.

# 2. SENSITIVITY PROJECTS

The MM5 modeling system consists of several pre-processors, the core prognostic model, and post-processing tools. Each component contains hi ghly configurable control files; together they control the aspects of grid structure, first-guess fields, model physics, temporal operation, and ultimately results visualization. The inheri t complexity of the MM5 modeling system complicates the development of a sound model c onfiguration suitable fo r regional scale annual episode air quality applications. Although the comp lete matrix of configuration options reduces in size as inappropriate options are eliminated, a large matr ix of potentially acceptable model configurations remains with mo st applications. The first st ep in developing the annual MM5 dataset was therefore completion of a series of sensitivities studies designed to identify the configuration yielding optimum results.

The first sensitivity study project began in 2002 and involved a collaborat ive project lead by

Kirk Baker with the Lake Mi chigan Air Directors Consorti um (LADCO) and Matthew Johnson (IDNR). Wyat Appel and Mike Abraczinskas w ith the North Carolina Division of Air Quality participated through the generation of a summary analysis for select sensitivity runs. The project was conducted in coordination with sensitivity work performed by Dennis McNally (with Alpine Geophysics). Components evaluated included, for example, PBL schemes, microphysical schemes, convective parameterizations, land su rface parameterizations, and snow models. Two one-month long episodes were selected for evaluation, January and July of 2001. The performance evaluation of each se nsitivity run included, but was not limited to, temperatures, wind vectors, cloud cover, precipitation, and mixing ratios.

Following the sensitivity study, the IDNR comple ted a 2002 annual simulation. This simulation utilized surface moisture and temperature nudgi ng. Within implementation of the Pleim-Xiu (PX) land surface model (LSM), soil moisture and soil temperatures were modeled in continuum from one 5-day episode block to the next. The model performance evaluation revealed an

extreme cold bias over the Central U.S. While unr elated to the cold bias, utilization of surface nudging techniques was abandoned following discussi on with the modeling community, as this practice has lead to the generation of super-adia batic lapse rates near th e surface. The optimum IDNR/LADCO configuration was thus modified accordingly and this annual simulation was deemed unsuitable for use in air quality modeling projects.

In a similar timeframe, VISTAS contracted with Baron Advanced Meteor ological Systems, LLC (BAMS) for the development of an annual MM5 dataset (Olerud and Sims, 2004). The work of VISTAS (through Olerud and Sims, 2004) also included a series of sensitivity studies. Independent results from the VISTAS project yiel ded findings similar to the conclusions reached by IDNR and LADCO. The compilation of all project results subsequently produced the configuration utilized by the IDNR in development of an annual metrological dataset suitable for regional scale air quality modeling.

4

# 3. MODELING SYSTEM CONFIGURATION

3.1 O VERVIEW


# Version 3.6.3 of the MM5 modeling sy stem was utilized in the second 1 

# (and final) 2002 IDNR annual meteorological simulation. The 3.6.3 rel ease represented the mo st current version available at the time of project inception. Other than the necessary configuration parameters, no modeling system code modifications capab le of altering results were rendered.

3.1.1 TERRAIN The terrain processor is used to define grid structure and assign va rious surface features. Terrain elevation, the dominant landuse cate gory, and vegetative and soil da ta were assigned using the 2- minute 24-category USGS data. The horizontal gr id structure consists of a 36 km domain conforming to the RPO meteorol ogical grid specifications. A nested 12 km grid was also included. The RPO 36 km meteorological doma in consists of a Lambert Conic Conformal projection centered at 90o W longit ude, 40o N latitude, with true la titudes of 33 and 45o N. The horizontal extent of the RPO domain was engi neered according to th e bounds of the Eta 212 grid. Domain development involved the impl ementation of TERRAIN through a series of sensitivity runs designed to extract the largest domain which remains within the borders of the Eta 212 grid. The 12 km grid was designed to achieve a balance between computational resources while maximizing coverage of Iowa-cen tric upwind and downwind flows. Both grid

structures are described in Tabl e 3.1 and depicted in Figure 3.1.

Table 3.1. Grid data, referencing MM5 terminol ogy specifications refer to dot points.

Grid Resolution (km) NX NY Nest

Location (x,y) Southwest

Coordinate (km offset)

1 36 165 129 1,1 (-2952, -2304)

2 12 193 199 66,30 (-612, -1260)

3.1.2 PREGRID/REGRIDDER The PREGRID processor prepares archived gri dded meteorological data for use within MM5 through conversion to an inte rmediate data format readab le by MM5. REGRIDDER invokes a horizontal interpolation scheme to translate data to the MM5 domain. The 3-hour Eta analysis and surface fields (ds609.2) were used to supply initial and boundary conditions to MM5. As the Eta analysis fields obtained from NCAR are a compressed (tar) file, the data were first uncompressed prior to use within PREGRID. The tar files also include the undesirable 12 hourly

cold start files. All cold start files (*.tm12) were dele ted prior to running PREGRID.

1 

# The first simulation was deemed unsuitable for use in air quality modeling projects and has been deleted.



# 5

# Figure 3.1. Twelve and 36 km domains utilized in the IDNR 2002 MM5v363 annual simulation.

In the first IDNR 2002 MM5 simulation, NCEP da ta was included in PREGRID to supply timevariant sea-surface temperature (SST) data, as th e Eta surface files supply only a time-invariant SST approximation known as skin-temperature. U pon further examination of SST data sources, the temporally variable NCEP SST data was f ound to lead to unrealisti c diurnal temperature profiles over the Great Lakes and near shorel ines. Figure 3.2 shows the NCEP-based Great Lakes SSTs for July 4, 2002, at 12 and 18Z. Over this 6-hour span, temperature fluctuations over many areas of the Great Lakes (particularly La ke Erie, and most shorelines) reach 20o F. While some variability is expe cted along shorelines and other shallow areas, the magnitudes observed through use of the NCEP data are unr ealistic. Observed SST data from buoy 45007 (located in the southern end of Lake Michigan yet far removed from the shoreline, see Figure 3.3) for the period July 4 – July 9 are provi ded in Figure 3.4. The maximum temperature variation throughout July 4 at this site was less than 3o F. Figure 3.5 depicts the 5-day SST timeseries produced using the NCEP SST data within REGRIDDER for the 36 km grid cell corresponding to the location of buoy 45007. The NC EP data yields a diurnal temperature range of approximately 7o F in this cell on July 4. The NCEP data also generates unrealistic diurnal profiles with a net upward trend in SST over this five-day peri od. In contrast, the observed data show less variability and a downward trend in SST. Utilization of the Eta skin-temperature data produces the constant SST boundary conditions shown in Figure 3.6. The corresponding Eta

6

# skin-temperature for the location of buoy 45007 is ~294 K. While this yields warmer surface temperatures than observed throughout the Ju ly 4 – July 9 period, no questionable diurnal variability or artificial warming trends are present.

Figure 3.2. Lake temperature variability ac ross a 6 hour span, from 12Z 7/4/2002 to 18Z 7/4/2002, using the NCEP SST data.

Figure 3.3. Great Lake buoy locations.

7

## Observed SST Data Buoy 45007 - Lake Michigan

289 290 291 292 293 294

07/04/02 07/05/02 07/06/02 07/07/02 07/08/02 07/09/02

DateSea Surface Temperature(K)



# Figure 3.4 Observed SST temperature data for buoy 45007.

Figure 3.5 NCEP derived SST profile for the gr id cell corresponding to the location of buoy

45007.

8

# Figure 3.6. Constant SST data derived from Et a skin-temperatures for the period 12Z 7/4/2002 through12Z 7/9/2002.

9

# 3.1.3 LIITLE_R LITTLE_R was originally designed to improve the REGRIDDER output by using objective analysis techniques to blend obs ervational data into the gridde d first-guess fields. Following traditional practices, the NWS upper air (ds353.4) and surface (ds464.0) datasets supply the observations. As the Eta fields already cont ain these NWS datasets, the implementation of LITTLE_R is viewed as partially redundant. Howe ver, LITTLE_R also generates the files used

in both the four-dimensional da ta assimilation (FDDA) and Pl eim-Xiu soil moisture nudging

schemes and therefore must be invoked. The implementation of LITTLE_R does not negatively affect model performance when th e Eta surface and analysis data provide the first-guess fields (Baker, 2002).

3.1.4 INTERPF

The IDNR 2002MM5v363 simulation uses a 34 vert ical layer structur e defined through the INTERPF preprocessor. The layer interfaces, pr ovided in Table 3.2, were designed through coordination with Dennis McNally to parallel th e vertical structure in use by EPA. INTERPF interpolates the pressure level data developed in the previous preprocessors to MM5’s native vertical system - terrain following sigma coordina tes. Sigma levels are defined according to Eq. 3.1, where p

s 

# equals the surface pressure, and p t 

# equals the pressure at model top. The model top was defined at 100 mb, or approximately 14,662 meters above ground level. Approximate sigma heights are calculated using Eqs. 3.1 – 3.3, with the user-defined variab les assigned the following values: p

s 

# = 1000 mb; p t 

# = 100 mb; T

s 

# = 275 K; A = 50 K. R and g represent the gas and gravitational constants of 287 J/(kg K) and 9.8 m/s 2 

# , respectively.

ts t



# pp pp

− −

= σ 

# (3.1)



# ( )

tts 

# pppp +

− ⋅ = σ (3.2)



# ⎥ ⎥ ⎦ ⎤

⎢ ⎢ ⎣ ⎡ ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

⋅ + ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛

⋅ ⋅

−=

s s

s 

# p p

g TR

p p

g AR

z ln ln 2 2 

# (3.3)

10

# Table 3.2. Details of the 34-layer vertical structure.

Level Sigma Height (m) p (mb) Depth (m)

34 0.000 14662 100 1841

33 0.050 12822 145 1466

32 0.100 11356 190 1228

31 0.150 10127 235 1062

30 0.200 9066 280 939

29 0.250 8127 325 843

28 0.300 7284 370 767

27 0.350 6517 415 704

26 0.400 5812 460 652

25 0.450 5160 505 607

24 0.500 4553 550 569

23 0.550 3984 595 536

22 0.600 3448 640 506

21 0.650 2942 685 480

20 0.700 2462 730 367

19 0.740 2095 766 266

18 0.770 1828 793 259

17 0.800 1569 820 169

16 0.820 1400 838 166

15 0.840 1235 856 163

14 0.860 1071 874 160

13 0.880 911 892 158

12 0.900 753 910 78

11 0.910 675 919 77

10 0.920 598 928 77

9 0.930 521 937 76

8 0.940 445 946 76

7 0.950 369 955 75

6 0.960 294 964 74

5 0.970 220 973 74

4 0.980 146 982 37

3 0.985 109 987 37

2 0.990 73 991 36

1 0.995 36 996 36

0 1.000 0 1000 0



# 11

# 3.2 MM5

An overview of the physics parameterization c onfiguration used in the IDNR 2002MM5v363 simulation is provided in Table 3.3. As previous ly discussed, the configuration emerges from the cumulative efforts of several sensitivity studi es, in combination with guidance from the AdHoc Meteorological Modeling co mmunity. In comparison with the original IDNR 2002 simulation, the cessation of contin uous soil field techniques with in the PX LSM is one of the

most notable modifications. 1 

# With the PX LSM no longer restri cted to sequential operation, the annual simulation was generated from 95 indepe ndent simulations initialized at 12Z and integrated through five days (ver sus 5-day blocks arranged in quart erly sequential simulations in the original run). This temporal structure allo ws maximum air quality modeling flexibility as photochemical simulations can be initialized using midnight local time or midnight GMT without the need to split any given 24-hour pe riod across multiple MM5 simulation blocks. While this methodology does increase the number of runs required to complete an annual simulation (versus initialization at 00Z with a 5.5 day run time), the increased computational requirements are not prohibitive. An example of the temporal structure is provided in Appendix A. To allow for approximately a two week ph otochemical model spin-up period, the simulation started at 12/16/2001 12Z. The completion da te occurred at 12Z on 1/1/2003. A 90 second timestep was used with output written every hour. The output files were split every 24 hours to simplify the post-processing (and photo chemical pre-processing) stages.

Table 3.3 Description of the options selected within the IDNR 2002 annual MM5v363 run.



# Option Configuration Details
Microphysics Mixed-Phase (Reisner I)

Cumulus Scheme Kain-Fritsch 2

PBL Asymmetric Convective Model * 

# Required by Pleim-Xiu LSM

Radiation RRTM Calculated every 15 minutes

Land Surface Model Pleim-Xiu No continuous soil fields

Shallow Convection Not enabled

SST Data source Eta Skin-Temperature

Snow Cover Effects Considered IFSNOW=1 Timestep 90 seconds (PX uses an internal 40s timestep)

* 

# The Asymmetric Convective Model (ACM) is also referred to as the Pleim-Chang PBL. The ACM parameterization is a deri vative of the Blackadar scheme (Pleim and Chang, 1992).

1 

# While discussion of the complete list of configuration variability between the original and 2020MM5v363 simulations is beyond the scope of this document, additional key updates include: the abandonment of NCEP SST data in favor of Eta-Skin temperatur es; the addition of the 12 km domain; use of a more recent modeling system release; and a new temporal structure.



# 12

# Additional configuration details include the following: S ea surface temperatures remained constant during the simulation as Eta skin temperatures were used as surrogate sea surface temperatures. Snow cover effects were cons idered. Analysis nudging of the temperature, mixing ratio, and wind fields was applied above the PBL. At the surface only the wind field was nudged. The default nudging strengths of 2.5 x 10 -4 

# and 1.0 x 10 -4 

# were used for the temperature and wind fields at 36 and 12 km, respec tively. A nudging coefficient of 1.5 x 10 -5 

# was established for the mixing ratios at both 36 a nd 12 km. The rotational wind field was not nudged, nor were observational nudging techniques applied. Optimal observational nudging methods require a station density not available across a continen tal scale annual simulation.

Referencing Baker et al. (2004) th e following details are provided: Vertical moisture and temperature advection are set to use linear interpolation. Other options incorporated include: moist verti cal diffusion in clouds, temperature advection using potential temperature, diffusion usi ng perturbation temperature, and an upper radiative boundary condition. The Pleim-Xiu land surface module requires the addition

of three variables in the MM5 deck: ISMRD, NUDGE, a nd IFGROW. ISMRD was set

to use soil moisture fields from the ETA an alyses. NUDGE was assigned to adjust the soil moisture data to the analyses fields. Finally, IFGROW was se t to option 2, which takes vegetative growth into account base d on vegetative fraction data from the

TERRAIN file.

The configuration of the 12 km grid pictured in Figure 3.1 closely resembles the 36 km grid methodology. The explicit exceptions include a decrease in the wind and temperature nudging strengths. While the terminology is questionabl e, the nesting technique employed is commonly referred to as “a two-way nested run without f eedback”. In this method, the 12 km model solution is not feed back to the master domain, but the grids are run simultaneously to allow the fine grid to receive boundary c ondition updates at every timestep.



# 3.3 C OMPUTATIONAL 

# S UMMARY


# Seven dual CPU Linux workstations were acquired to complete the annual simulation. Six machines were equipped with dual 3.06 GHz Inte l Pentium Xeon processors, with the final machine a dual 2.0 GHz processor. Each machine 1 

# was equipped with 2 Gb of RAM, and Ultra 320 SCSI local hard drives for model I/O. Upon completion of each run, output data was

transferred via NFS to a SC SI-IDE RAID array. In summation, 41 wall-clock days were required to complete the annual simulation. Th is represents each machine computing two independent simulations simultaneously (essentially each CPU was tasked with one simulation at any given time). Open MP wa s not an available option due to the implementation of PX. Approximately 100 wall-clock hours was require d for a 3.06 GHz machine to complete two simulations running simultaneously. Storage requirements reached 1.1 terabytes, with the 36 km simulation occupying 400 Gb and the 12 km data using 700 Gb.

1 

# The 2.0 GHz machine had only 1 Gb of onboard RAM.



# 13

# 4. MODEL PERFORMANCE EVALUATION

4.1 B ACKGROUND


# No rigid guidelines exist for systematical ly and objectively eval uating the quality of meteorological simulations. However, sound comp rehensive philosophies exist. A seven point approach outlined by Tesche (1994) provides th e framework for a thorough model performance

evaluation. The framework can be classified in to two components: an operational evaluation and a scientific evaluation (Emery and Tai, 20 01). The scientific ev aluation requires rigorous examinations of model formulation and algorit hm development, methods beyond the scope of most modeling projects. Historic al development and applications of MM5 within the scientific community (including air quality and prognos tic projects published through peer-reviewed journal articles) must then serve to support th e scientific evaluation. Thus the performance

evaluation of the IDNR 2002MM5v363 annual simu lation will focus upon operational criteria.



# 4.2 M ETHODS


# Climatic variability, complex mesoscale meteorological phenom ena, and scientific unknowns contribute to meteorological m odeling difficulties and force modelers to take a subjective approach to model performance. Objective statistical measures which offer a quantitative model assessment exist, but implementation of the metrics is subjective to a degree. For example, defining the area over which domain averaged me trics are calculated is a subjective decision, buffered only through guidelines. In general, metrics averag ed over large meteorological modeling domain are avoided, as error cancellation dilutes relevance. Conversely, splitting the modeling domain into small subdomains renders sample sizes unrepresen tative. The logical approach falls well within the bounds of the extremes, leaving optimum subdomain definition open to interpretation. As one means of addres sing the issue, a subjective grid decomposition technique was applied, resulti ng in the twelve rectangular 1 

# subdomains pictured in Figure 4.1.

Model performance measures must also minimally include a review of upper air features in tandem with surface statistics. Upper air featur es are key variables in terms of air quality modeling given the importance of fields such as three dimension wind flows and PBL depths. Evaluation of the upper atmosphere also intr oduces a level of complexity exceeding the difficulty associated with asse ssing surface features. The sheer volume of upper air model data, in combination with a relatively sparse observing network gathering on ly twice daily soundings, creates problems in terms of scale. A limited set of data analysis tools also restricts the review process. In an attempt to achieve a balance betw een available resources and the level of detailed review, the upper air evaluation includes review of PBL features and focuses upon observed versus modeled soundings. To improve the effi ciency and simplify the review of soundings, a new software tool was developed in-house: RAOB PLOT. In the final aspect of the upper air evaluation, an independent revi ew of precipitation prediction, conducted by Kirk Baker, is briefly summarized. While technica lly a surface feature, the precipitation ev aluation indirectly enhances the upper air review given the three dimensional na ture of precipitation events.

1 

# Processing requirements necessitated that subdomains be simple rectangles defined only through a southwest and northeast grid coordinate.



# 14

# PacificNW



# SW North

DesertSW

CenrapS CenrapN GL

SE NE

MidAtlantic

OhioVal Iowa

Figure 4.1. Decomposition of th e contin ental scale MM5 domain into s i mple rectangu lar subregions designed for targeted model perfor mance evaluation. Areas of overlap are shaded differently a nd outlin es have been a dded to high light individ ual subdomain boundar ies.



# 4.3 S TAT I S T I C A L 

# M EAS U RES



# W ithin the statistica l degrees of freedom available to the m e teorological m odeler, a subset of

standard statistical m easures ha s em erged, outlined in Table 4.1. These m e trics are calculated based upon data contained within a given subdom a in (See Figure 4.1). Metrics are calculated using hourly and daily averages. W h ile no st rict criteria establis hing acceptable m odel perform a nce exist, the genera l guidelines established by Em ery and Tai (2001) provide a community adopted fram e of reference. A summ ary of the guidelines is provided in Table 4.2.

15

# Table 4.1. List of statistical me asures commonly discussed in mete orological model evaluations. The DH designation represents that both daily an d hourly averaged values are calculated for a particular metric. Conversely, D or H indicates that the value is ava ilable only on a daily or hourly average, respectively.

Statistical Measure Wind Speed Wind

Direction Temperature Humidity

Obs. vs Predicted Timeline DH DH DH DH

Bias DH DH DH DH

Gross Error D D D D

Total RMSE DH DH DH
Systematic RMSE DH DH DH
Unsystematic RMSE DH DH DH
Index of Agreement DH DH DH

Table 4.2. Guidelines for meteorologi cal model performance. Source: Meteorological Modeling and Performance Evaluation of the September 13-20, 1999 Ozone Episode (Emery and Tai, 2001). Data pertain to daily averaged values.

Wind Speed Wind Direction Temperature Humidity

RMSE ≤ 2 m/s

Mean Bias ≤ ± 0.5 m/s

IOA ≥ 0.6 Gross Error ≤ 30 deg

Mean Bias ≤± 10 deg Gross Error ≤ 2 K

Mean Bias ≤ ± 0.5 K

IOA ≥ 0.8 Gross Error ≤ 2 g/kg

Mean Bias ≤ ± 1 g/kg

IOA ≥ 0.6



# An overview of the significance for each me tric is provided by Baker et al. (2004):

“ Bias error (bias) is the degree of corresponden ce between the mean prediction and the mean observation, with lower numbers indicative of better performance. Values less than 0 indicate under-prediction. The gross error , or mean absolute erro r, is the mean of the

absolute value of the residuals from a fitted statistical model. Lower numbers indicate better model performance.

Root Mean Square Error (RMSE) is a good overall measure of model performance. The weighting of (prediction-observation) by its s quare tends to inflate RMSE, particularly when extreme values are present. With re spect to a good model th e root mean square error should approach zero. RMSE can be divided into a systematic and unsystematic component by least-squares regression. Since differences described by systematic RMSE can be described by a linear function, they should be relatively easy to dampen by a new parameterization of the model. Unsystematic RMSE can be interpre ted as a measure of potential accuracy or noise level (Emery et al., 2001). With resp ect to a good model the systematic difference should approach zero wh ile the unsystematic difference approaches

RMSE.

16

# Index of Agreement is a relative measure of the degr ee of which pred ictions are errorfree. The denominator accounts for the mo del's deviation from the mean of the

observations as well as to the observations de viation from their mean. It does not provide information regarding systematic and unsys tematic errors. The index of agreement approaches one when model performance is best.”

The basis of the statistical analysis if formed through a comparison of the modeled fields with the Techniques Data Laboratory U.S. and Canada surface hourly observations (ds472.0). Hourly and daily averaged bias, error, RMSE (total , systematic, and unsystematic), and index of agreement metrics for wind speed, wind directi on, temperature and humidity were generated using the Metstat program and MS Excel post- processing macro devel oped by Environ. Time

series of modeled and observed conditions were al so prepared via Metstat. As continental-scale domain averaged statistical measures are su sceptible to e rror cancellation, metrics were calculated over the twelve subdomains illustrated in Fig. 4.1.

The volume of data associated with the annual simulation can quickly overwhelm standard time series displays or similar attempts at numeri cal data presentation. As a solution Kirk Baker developed and ingenious method of data display. PAVE is used to plot daily metrics, aligned vertically by month, and horizonta lly by date. This allows for an annual graphical display of daily averaged metrics in a singl e plot, simplifying the identificati on of error trends or pervasive biases. Even with this method of simplificatio n, a detailed discussion of all twelve subdomains becomes excessive. The statistical analysis therefore focuses upon those regions encompassing the CENRAP and Midwest RPO states, primarily the regions: CenrapN, CenrapS, GL (Great Lakes), OhioVal (Ohio Valley), and Iowa.

17

# 5. SURFACE EVALUATION (36 KM)

The daily averaged metrics described below are provi ded graphically in the form of a “Bakergram”. The Bakergram, developed by Kirk Baker, allows fo r the meaningful depiction of an annual set of daily averaged statistical values in a single plot . For example, Figure 5.1 consists of a compilation of four Bakergrams, one each for the wind speed bi as, wind speed error, wind direction bias, and wind direction error. Focusing on the wind speed bias Bakergram in Figure 5.1 (top left), 365 daily averaged metrics are provided. Twelve columns are provided, which each column containing a monthly dataset. The individual days are provided in rows, with the first of the month displayed at the top, with days descending from top to bottom. The concept is repeated (f or example, see Figure 5.2) with temperature and mixi ng ratio metrics plotted.



# 5.1 G REAT 

# L AKES


# In previous sensitivity studies, the Pleim-Cha ng/Pleim-Xiu PBL/LSM conf iguration was found to improve wind vector performance versus the use of alterative PBL parameterizations. Consistent with this discovery, the wind vect or performance in the GL region is encouraging. Wind speed metrics are generally favorable, and no clear trends in erro r or bias are evident (see Figure 5.1). A notable caveat, daily metrics may hide inconsis tencies occurring within the diurnal profile. 1


# Turning to the wind direction evaluation, again re sults are satisfactory, with one exception found, an increase in the summertime gross error.

In the Great Lakes region, the problems of greates t concern lie in the wintertime cold temperature biases, the warm summertime biases, and the summe rtime positive moisture biases (See Figure 5.2). Examining the temperature biases from a diurnal 2 

# perspective, the warm bias is predominantly caused by nighttime temperatures remaining warmer than observed. The cold wintertime temperature bias is often traced to underpredicted high temperatures, evening temperatures falling too rapidly, and nighttime lows often colder than observed. Caution should be exercised when generalizing the wintertime bias trends though, as exceptions are more abundant than with the summertime warm biases. Turning to the mixi ng ratio (humidity) evaluation (see Figure 5.2), although the gross error metrics ar e generally within the statisti cal guidelines, the summertime positive bias is a concerning trend. Only on rare occasions do negative biases occur. The likely culprit is MM5’s tendency to overpredict precipitation.



# 5.2 N ORTHERN AND 

# S OUTHERN 

# C ENRAP


# In general, the statistical evaluation for the Ce nrapN subdomain (Figures 5.3 - 5.4) yields results similar to the Great Lakes regi on. A notable exception being the nearly consistently negatively biased wind speeds. Examining the wind speed bias in greater detail (thr ough diurnal profiles), this fault is predominantly influenced by the underpredi ction in the daily peak wind speeds. Keeping these errors in perspective, th e magnitude of the underprediction typically remains below 1 m/s. Examining the mixing ratio performance, the most serious issue remains the abundance of summertime surface moisture. While arguable trivia l, CenrapN does differ from the GL subdomain during May, where several surface mo isture underpredictions occur.

1 

# Diurnal metrics are examined in Chapter 6.

2 

# Ibid.

# Turning to the Southern CENR AP subdomain (Figures 5.5 - 5.6 ), wind direction performance remains encouraging, similar to the performance for the CenrapN and GL regions. As found in CenrapN, wind speeds are generally negatively bi ased, but more pronounced in this region. The mixing ratio biases reveal excess moisture, alt hough a drier than observed fall was predicted. Examining temperature performance, late winter/ear ly spring temperatures yielded positively biased trends, in contrast to the perv asive cold winter bias es found in the CenrapN and GL regions. Examination of the diurnal profiles revealed the biases were attributable to warm nighttime lows.



# 5.3 O HIO 

# V ALLEY


# Once again, the wind speeds are generally too low, however, the associated er ror is well within the acceptable guidelines. Wind directions errors are al so generally small, but an increase in error is found in the summer months. Mixing ratios are cons istently too moist, ex cept in the mid-October timeframe. As in the Great Lakes regions, a cold winter bias is found, while summer temperatures remain too warm (predominantly over the nighttime hours). The results are depicted in Figures 5.7

- 5.8.



# 5.4 I OWA


# Within the Iowa subdomain wind vector performan ce is favorable, with wind speed bias and error measures predominantly meeting the statistical goals. Wind directions exhibi t greater errors in the late summer/early fall timeframe versus the Ce nrapN and GL subdomains, but are not cause for severe alarm (see Figure 5.9). As is common, cold winter and warm summ er biases are present (Figure 5.10). In terms of the moisture bias, the Iowa domain e xhibits greater springtime negative moisture bias versus CenrapN, otherwise sim ilar performance is shown (this result is not unexpected, given the superposition of the Iowa subdomain over CenrapN).



# 5.5 E ASTERN 

# R EGIONS


# A detailed discussion of model performance for a ll areas is beyond the scope of this document. Alternatively, summary remarks are provided. Ov er the MidAtlantic, no serious abnormalities are found beyond the errors identified previously in Central U.S. subdomains. As is common to MM5, a positive moisture bias exists, affecting both th e MidAtlantic and SE regi ons. Examining the NE region, wind speed, and wind direction errors approach the upper extreme of acceptable performance. Again, the moisture bias is positiv ely biased, with errors maximized over the summer months. Given moisture carrying capacity is a non-linear function of temp erature, the relatively small mixing ratio gross errors oc curring in the wintertime of regions with colder climates should not be interpreted as superior model performan ce. The daily averaged statistical results are provided in Appendix B for each of the individual Eastern subregions.



# 5.6 W ESTERN 

# R EGIONS


# The daily averaged statistical results for the west ern subdomains are also pr ovided in Appendix B. The complex topography found in the Western Un ited States clearly introduces a degree of modeling difficulty not found in other regions. Pe rformance metrics are di scouraging when viewed initially, however, the appropriateness of the st atistical measures are questionable as model resolution is not designed to captu re the topographically induced nea r-field flows affecting many of

the local observations.

19

# Figure 5.1. Daily averaged wind speed/direction metrics for the Great Lakes (GL) subdomain.

# Figure 5.2. Daily averaged temperature and mixi ng ratio metrics for th e Great Lakes (GL)

subdomain.

21

# Figure 5.3. Daily averaged wind speed/dir ection metrics for the CenrapN subdomain.

22

# Figure 5.4. Daily averaged temperature and mi xing ratio metrics for the CenrapN subdomain.

23

# Figure 5.5. Daily averaged wind speed/dir ection metrics for the CenrapS subdomain

24

# Figure 5.6. Daily averaged temperature and mi xing ratio metrics for the CenrapS subdomain.

25

# Figure 5.7. Daily averaged wind speed/dir ection metrics for the OhioVal subdomain.

26

# Figure 5.8. Daily averaged temperature and mi xing ratio metrics for the OhioVal subdomain.

27

# Figure 5.9. Daily averaged wind speed/dir ection metrics for the Iowa subdomain.

28

# Figure 5.10. Daily averaged temperature and mi xing ratio metrics for the Iowa subdomain.

29

# 6. TWELVE KILOMETER EVALUATION

6.1 D AILY 

# A VERAGED 

# S TATISTICS


# Generalizing the impacts of the 12 km domain upon the Great Lakes region, in terms of daily averaged metrics, a decrease in simulation accu racy during the winter months is found, while only negligible changes occur across the remainder of the year. This trend is prevalent for wind speed, wind direction 1 

# and temperature errors. The wintertime temperature cold bias (found at 36 km resolution) is thus even more pronounced in the 12 km domain. Mixing rati o statistics were generally uninfluenced by domain resolution. These result s are depicted in Figures 6.1 – 6.2, where the Bakergram concept is maintained, how ever, the results are presented in terms of

the differences between the 36 and 12 km results. The plots were generated by subtracting the 36 km daily averaged statistical values from th e 12 km data. As the comparison only involves gross and root mean square error metrics, nega tive values indicate an improvement in model performance at 12 km resolution. This me thodology is maintained for Figures 6.1 - 6.8.

The CenrapN regions shows only mi nor variations in the temperat ure fields, with the greatest change concentrated to the cooler months, with slight performance disbenefits. Wind direction metrics produced a drastically di fferent trend, as nearly all days showed poorer performance. Figures 6.3 – 6.4 provide a graphi cal depiction of the 12 km domain impacts upon the daily averaged metrics for this subdomain.

Over the Ohio Valley, only minor differences we re calculated between the 12 and 36 km daily averaged statistical results, in general. A sl ight improvement in the mixing ratio fields was computed. As in CenrapN, wi nd direction gross errors encount ered widespread performance degradation during the winter and early spring months. Keep ing the increasing errors in perspective, additional error remained below 3.5 degrees. See Figures 6.5 - 6.6.

In terms of daily averaged statistical measures , the Iowa subdomain receives few benefits from increased resolution. Wind speeds generally exhibit slightly greater error in the winter, spring, and fall, while demonstrating little variability during the summer. Consistent with nearby subdomains, wind direction performance suffers. While mixing ratios impacts were negligible, most months exhibited days with increased temp erature error, particularly in the winter. Fortunately, gross error degradation remained belo w 0.5 K. The results are depicted in Figures

6.7 - 6.8.

Due to the spatial extent of the 12 km do main, neither the CenrapS domain, nor any other subdomain, is eligible for comparison.

1 

# With additional errors occurring into the early spring months.

# Figure 6.1. Twelve km domain daily averaged statis tical performance for selected wind metrics in relation to the 36 km grid fo r the Great Lakes (GL) subdomain.

31

# Figure 6.2. Twelve km domain daily averaged statis tical performance for selected temperature and mixing ratio metrics in relation to the 36 km grid for the Great Lakes (GL) subdomain.

32

# Figure 6.3. Twelve km domain daily averaged statis tical performance for selected wind metrics in relation to the 36 km gr id for the CenrapN subdomain.

33

# Figure 6.4. Twelve km domain daily averaged statis tical performance for selected temperature and mixing ratio metrics in relation to the 36 km grid for the CenrapN subdomain.

34

# Figure 6.5. Twelve km domain daily averaged statis tical performance for selected wind metrics in relation to the 36 km gr id for the OhioVal subdomain.

35

# Figure 6.6. Twelve km domain daily averaged statis tical performance for selected temperature and mixing ratio metrics in relation to the 36 km grid for the OhioVal subdomain.

36

# Figure 6.7. Twelve km domain daily averaged statis tical performance for selected wind metrics in relation to the 36 km gr id for the Iowa subdomain.

37

# Figure 6.8. Twelve km domain daily averaged statis tical performance for selected temperature and mixing ratio metrics in relation to the 36 km grid for the Iowa subdomain.

38

# 6.2 H OURLY 

# S TATISTICS


# Additional comparisons between the 36 and 12 km simulations are provided below through review of hourly timeseries. Modeled (bot h 36 and 12 km) versus observed conditions are plotted below, with the associated bias also depicted. The hourly time series evaluation eliminates the statistical smoothing associated w ith the daily averaging periods. These charts also serve as the diurnal profile data source referenced in previous chapters, however, the discussion below will primarily focus upon differences between the 12 and 36 km simulations.

Assessing the timeseries from a winter (January) and summer (June) monthly subset of the annual simulation for the Great Lakes region (Figur es 6.9 - 6.10) leads to a general conclusion that improvement occurs in the daytime wind sp eed biases with implementation of the 12 km grid, while nighttime disbenefits are observed. At 12 km resolution, the wintertime cold bias is even more pronounced versus the 36 km domai n, as nighttime low temperatures dip further below observed values (Figure 6.11). The ultimate cause for the low temperature bias is unknown, but this is not an uncommon feature of MM5 simulations (Ad-Hoc Meteorological Modelers Meeting group discussion, 2007). For the GL region, no significant differences are found in either temperature or humidity duri ng the summer month of June (Figure 6.12). Appendix C provides additional January and June hourly 12 versus 36 km statistical charts for the Great Lakes, OhioVal, CenrapN, and Iowa subdomains.

Observed/Predicted Windspeed

2 3 4 5 6 7 8

1/ 1 1 / 2 1 / 3

1/

4

1/ 5 1 / 6

1/

7

1/ 8 1 / 9 1/10 1/11
1 / 1 2 1/13 1/14

1 / 1 5

1/1

6

1/17 1/18

1 / 1 9

1/20 1/21
1 / 2 2

1/23 1/24

1 / 2 5

1/2

6

1/27 1/28

1 / 2 9

1/30 1/31m/s ObsWndSpd 36km Predicted 12km Predict ed



## Bias Windspeed

-1.5

-1 -0.5

0 0.5

1 1.5

1/ 1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1 /10

1/11 1/12 1/13 1/14 1/15 1/16 1/1

7 1/18 1/19 1/20 1/21 1/22 1/23 1/24 1/25 1/26 1/27 1/28 1/29 1/30 1/31m/s 36km Bias 12km Bias



# Observed/Predicted Wind Direction

0 60 120 180 240 300 360

1/ 1 1

/ 2

1/ 3 1

/ 4

1/ 5 1

/ 6

1 / 7

1/ 8 1/ 9

1/10 1/11 1/12 1

/ 13

1

/14

1/15

1 /16

1/17 1

/1

8 1

/ 19

1

/2

0 1/21 1

/ 22

1/23 1

/2

4 1

/25

1

/2

6

1 /27

1/28 1

/2

9 1

/ 30

1

/3

1deg ObsWndDir 36km Predict ed 12km Predict ed



# Bias Wind Direction

-30

0 30

1/

1

1/ 2 1/

3

1

/ 4

1/

5 1

/ 6 1/

7

1/

8

1/

9 1/10 1

/1

1 1/1

2 1

/1

3 1/14 1/1

5 1/16 1/1

7 1

/1

8 1

/1

9 1/

2 0 1/2 1 1

/2

2

1/2

3 1

/2

4 1/25 1/

2 6 1/27 1

/2

8 1/

2 9 1

/3

0 1/31deg 36km Bias WD 12km Bias WD 

# bias shifted 

# Great Lakes Domain

Figure 6.9. Twelve and 36 km hourly wind vector stat istics for the Great Lakes subdomain for January, 2002.

39

# Observed/Predicted Windspeed

2 4 6 8

6/ 1 6/ 2 6/ 3 6/ 4 6

/ 5

6/

6

6/ 7 6

/ 8

6/

9

6 /10

6/11 6

/ 12

6 /13

6/14 6/15 6

/1

6 6

/1

7 6

/18

6

/ 19

6

/2

0 6

/21

6

/22

6

/23

6/24 6

/25

6

/26 6 /27

6/28 6

/ 29

6

/3

0m/s ObsWndSpd 36km Predict ed 12km Predict ed



# Bias Windspeed

-2 -1 0 1 2

6/ 1 6/ 2 6/ 3 6/ 4 6

/ 5

6/

6

6/ 7 6

/ 8

6

/ 9

6/10 6

/11

6

/12

6/13 6

/14

6

/15

6/1

6 6/1

7 6/18 6/

19

6/2

0 6/21 6/22 6/23 6

/24

6/25 6/26 6/27 6

/28

6/

29

6/3

0m/s 36 km Bias 12 km Bias



# Bias Wind Direction

-30

0 30

6/ 1 6/ 2 6/ 3 6/ 4 6/ 5 6

/ 6

6/ 7 6/ 8 6/ 9

6

/10

6

/11

6

/12

6

/1

3 6

/14

6

/15

6

/16 6 /17

6/18 6

/19

6

/20 6 /21

6/22 6/23 6

/ 24

6 /25

6/26 6/27 6

/ 28

6

/2

9 6

/3

0deg 36km Bias 12km Bias 

# Great Lakes Domain

Observed/Predicted Wind Direction

0 60 120 180 240 300 360

6/

1

6/ 2 6

/ 3

6/ 4 6/ 5 6/ 6 6/

7

6/ 8 6

/ 9

6

/10

6/11 6/12 6/13 6

/14

6/15 6

/16

6/17 6/

18

6/1

9 6

/20

6/21 6/

22

6/2

3 6

/24

6/25 6/26 6/2

7 6

/28

6/29 6/30deg ObsWndDir 36 km PredWnd 12 km PredWndDir



# Figure 6.10. Twelve and 36 km hourly wind vector st atistics for the Great Lakes subdomain for
June, 2002.

40

### Bias Temperature

-5 -4 -3 -2 -1 0 1 2

1/ 1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1/10 1/ 11 1/12 1/ 13

1/14 1/

1 5 1/16 1/ 17

1/18 1/ 19

1/20 1/ 21 1/ 22 1/ 23

1/24 1/

2 5 1/26 1/ 27 1/ 28

1/29 1/30 1/ 31K 36 km Bias T 12 km Bias T 

### Observed/Predicted Temperature

255 260 265 270 275 280

1/ 1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1/

10 1/11 1/ 1 2 1/13 1/

14

1/15 1/

16 1/17 1/

18 1/19 1/

20 1/21 1/22 1/23 1/

24

1/25 1/

26 1/27 1/

28

1/

29 1/ 30 1/31K ObsTemp 36 km Pred_T 12 km Pred_T



### Predicted/Observed Humidity

0 1 2 3 4 5 6

1/

1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1/

10

1/

1 1 1/ 12 1/13 1/

14

1/

15 1/ 16

1/17 1/18 1/19 1/

20

1/

2 1 1/22 1/23 1/24 1/

25 1/ 26

1/27 1/28 1/

29 1/ 30

1/

3 1g/kg ObsHum 36km Pred q 12 k Pred q



### Bias Humidity

-0.4 -0.2

0 0.2 0.4 0.6 0.8

1 1.2

1/

1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1/

10

1/

1 1 1/ 12 1/13 1/

14

1/

15 1/ 16

1/17 1/18 1/19 1/

20

1/

2 1 1/22 1/23 1/24 1/

25 1/ 26

1/27 1/28 1/

29 1/ 30

1/

3 1g/kg 36 km Bias q 12 km Bias q 

### Predicted/Observed Humidity

0 1 2 3 4 5 6

1/

1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1/

10

1/

1 1 1/ 12 1/13 1/

14

1/

15 1/ 16

1/17 1/18 1/19 1/

20

1/

2 1 1/22 1/23 1/24 1/

25 1/ 26

1/27 1/28 1/

29 1/ 30

1/

3 1g/kg ObsHum 36km Pred q 12 k Pred q



### Bias Humidity

-0.4 -0.2

0 0.2 0.4 0.6 0.8

1 1.2

1/

1 1/ 2 1/ 3 1/ 4 1/ 5 1/ 6 1/ 7 1/ 8 1/ 9 1/

10

1/

1 1 1/ 12 1/13 1/

14

1/

15 1/ 16

1/17 1/18 1/19 1/

20

1/

2 1 1/22 1/23 1/24 1/

25 1/ 26

1/27 1/28 1/

29 1/ 30

1/

3 1g/kg 36 km Bias q 12 km Bias q 

# 12km even colder 

# Great Lakes Domain

Figure 6.11. Twelve and 36 km hourly temperature and moisture statistics for the Great Lakes subdomain for January, 2002.

41

### Observed/Predicted Temperature

270 275 280 285 290 295 300 305 310

6/ 1 6/ 2 6/ 3 6/ 4 6/ 5 6/ 6 6 / 7 6/ 8 6/ 9

6 / 10 6/ 11 6/ 1 2

6 / 13 6/ 14 6/ 15 6/16 6/17 6/18 6/19 6/20 6/21 6/22 6/23 6/ 24

6/25 6/26 6/ 27 6/ 28 6/29 6/30K ObsTemp 36 km 12 km



### Bias Temperature

-3 -2 -1 0 1 2 3

6/ 1 6/ 2 6/ 3 6/ 4 6/ 5 6/ 6 6

/ 7 6/ 8 6/ 9

6 / 10 6/ 11 6/ 12

6 / 13 6/ 14 6/ 15 6/16 6/17 6/18 6/ 1 9 6/20 6/21 6/22 6/23 6/ 24

6/25 6/26 6/ 27 6/ 28 6/ 2 9 6/30K 36 km Bias 12km Bias 

# Great Lakes Domain

Predicted/Observed Humidity

0 2 4 6 8 10 12 14 16 18

6/ 1 6/ 2 6/ 3 6/ 4 6 / 5 6/ 6 6/ 7 6/ 8 6 / 9

6/10 6/11 6/12 6/13 6/14 6/15 6/16 6 / 17 6/ 18

6/19 6/20 6 / 21 6/ 22 6/ 23 6/ 2 4

6 / 25 6/ 26 6/ 27 6/ 2 8 6/29 6/30g/kg ObsHum 36 km Predicted 12k m Predict ed



### Bias Humidity

-2 -1 0 1 2 3 4

6/ 1 6/ 2 6/ 3 6/ 4 6/ 5 6/ 6 6 / 7 6/ 8 6/ 9

6 / 10 6/ 11 6/ 1 2

6 / 13 6/ 14 6/ 15 6/16 6/17 6/18 6/19 6/20 6/21 6/22 6/23 6/ 24

6/25 6/26 6/ 27 6/ 28 6/29 6/30g/kg 36 km bias 12 km Bias 

# no significant changes



# Figure 6.12. Twelve and 36 km hourly temperature and moisture statistics comparison for the Great Lakes subdomain for June, 2002.

42

# 7. UPPER AIR EVALUATION

7.1 S OUNDINGS


# A comprehensive assessment of model performance cannot be completed through the evaluation

of surface statistical measures alone. A rigorous evaluation requires the examination of additional features such as precip itation fields, PBL depths, and ve rtical profiles of temperature, moisture, and wind vectors. As readily availabl e tools have not been identified which yield objective measures of such parameters, evaluations are typically subjective. A precipitation evaluation of the 36 km dataset has been comple ted by Kirk Baker (Baker et al., 2004) and is summarized below. In combination with the pr ecipitation evaluation, the most efficient method available for an upper air analysis is to focus upo n radiosonde observations. To aid in the review of upper air feature, the IDNR created the RAOBPL OT software tool that efficiently displays modeled versus observed radiosonde upper air measurements. With twice-daily soundings available from approximately 70 observing sta tions, roughly 51,100 modeled versus observed soundings are available for examination from the 36 km annual simulation alone. Clearly a complete examination is resource prohibitive. The volume of da ta available, in combination with only inefficient subjective methods for evalua tions highlights a current deficiency in annual scale regional modeling applications. While inelegant, the immediately practicable solution requires a targeted review of specific data.

A brief review of the modeled versus observed sounding for many sites in the Central U.S. was conducted, with no terminal deficiencies disc overed. A more focused evaluation upon the Davenport, Iowa, station was completed over the simulated summer months, with the following conclusions reached: Upper level wind vectors are well simulated. The temperature fields below approximately 900 mb yielded a tendenc y toward underprediction at 0Z, while the moisture fields were generally overstated during the same region and time. At 12Z, temperatures were generally underpredicted below 900 mb. In terms of es timated PBL depths, the mixed layer commonly appears shallower than observed. While error is never desired, in terms of modeling air quality (in a conserva tive sense) a shallow PBL is pr eferred versus excessive depth. A sample of the observed versus modeled s ounding produced by RAOBPL OT is provided in Figure 7.1.



# 7.2 P RECIPITATION


# Kirk Baker with the Lake Michigan Air Direct ors Consortium was provide d a complete copy of the 36 km meteorological datase t and subsequently completed a model performance evaluation examining precipitation fields. In summary, both rainfall totals and precipitation spatial coverage are generally well simulated in the fall, winter, and springtime periods. As is common with many MM5 simulations, summertime pr ecipitation events produce an excess of precipitation. Rainfall patterns also exhibit grea ter spatial coverage th an observed. Additional detail, including graphical repres entation of predicted and observed rainfall, is available in Baker et al., 2004.

43

# Figure 7.1. Sample ROABPLOT observed versu s predicted (36 km domain) sounding for Davenport Iowa, on July 13, 2007, at 0Z. Wind speed and directions are accurately simulated throughout the depth of the sounding. The temp erature profile performance is more than adequate. As is common, a positive moisture bias exists at ( and above the surface), while the estimated PBL depth remains too shallow.

44

# 7.3 PBL D EPTHS


# Additional upper air analyses included a limited comparison of the 36 and 12 km predicted PBL heights. Figure 7.2 provides an example comparis on. As expected, the degree of agreement

between the 36 and 12 km results exceeds variabili ty. Areas in Western Illinois and Eastern Texas (among others) do display deviations. In Eastern Texas, MM5 predicts a precipitation event (which is weakly supporte d by observations, see Figures 7.3 - 7.4). The re duction in PBL heights in Western Illinois would appear to be precipitation driven as well, but no convective or non-convective rainfall was predicted by MM 5 during this time. The observed radar reflectivities also suggests no pr ecipitation occurred during this ti me. In summary, the 12 km grid yields improved feature deta il yet the accuracy of such fields, across a continental scale annual simulation, is difficult to a ssess within a reas onable timeframe.

Figure 7.2. PBL heights predicted by MM5 fo r June 23, 2002, at 19Z. for the 12 and 36 km modeling domains.

45

# Figure 7.3. Observed conditions on June 24, 0Z.

46

# Figure 7.3. Radar reflectivity on June 23, 2002 at 19Z.

47

# 8. CONCLUSIONS

In the northern half of the Cent ral U.S. through the Ohio River valley, the surface statistical

evaluation reveals a dominant wint ertime cold bias, with cool c onditions typically present in the evening hours, while overly aggressive night time lows and weak high temperatures also

contribute to the cold bias. The summer months exhibit a warm bias, attributable to the overprediction of nighttime temperatures. Wind speed and direction predictions over the central

and northern Central U.S. exhibi t low statistical error and provi de for an increase in model confidence. Continuing the evaluation into the Western U.S. yields a reduction in model confidence, as error measures increase across al l fields. As discusse d, this result is not completely unexpected given complex Western t opography. Regions within the Eastern U.S. demonstrate prediction skill above Western regions, yet statistic al accuracy falls below that

found in the Midwest.

Expanding the evaluation into upper air features reveals no fundamental flaws jeopardizing the adequacy of the simulation in terms of air quality modeling. A tendency to slightly underpredict summertime PBL depths over Eastern Iowa was disc overed. In subjective terms, such error is acceptable as perfect model performa nce is unattainable. A simila r conclusion is reached for the precipitation shortfalls di scussed by Baker et al., 2004.

Within the Central U.S, increasing the horizontal resolution from 36 to 12 km yielded no benefits

from a surface-feature statistical evaluation perspective. Within the Great Lakes subdomain, the 12 km simulation appears to improve daytim e wind speed predictions, however, nighttime predictions suffer. Overall, wind speed error sh owed little variability between the 36 and 12 km domains. Beyond the statistical evaluation, add itional field detail is resolved by the 12 km domain as expected. As in the upper air analys is for the 36 km grid, no fundamental flaws were identified in review of 12 km upper air features.

In summary, the statistical evaluation yields re sults predominantly within acceptable guidelines for the principal regions of interest (the Stat es near and within LADCO and the northern twothirds of CENRAP). Concurrently, no major si mulation deficiencies were revealed during the upper air review. The 36 and 12 km Iowa DNR 2002MM5v363 datasets are thus judged acceptable for use in regional scale air quality m odeling studies focused within the central United

States.

48

# 9. REFERENCES

Anthes, R. A. and T. T. Warner, 1978: The Deve lopment of Mesoscale Models Suitable for Air Pollution and Other Mesometeorological Studies. Monthly Weather Review

, 106 , 1045-

1078.

Baker, K., 2002: MM5 Sensitivity for the Na tional RPO Grid, Presented at the Ad-Hoc Meteorological Modeling Meet ing, Des Plaines, IL, 2002.

Baker, Kirk, Matthew Johnson and Steven Ki ng, 2004: Meteorological Modeling Performance Summary For Application to PM2.5/Haze/Ozone Modeling Projects. http://ladco.org/tech/pho to/docs/mm5_protocol_dec2004.pdf

Chen, F., and J. Dudhia, 2001: Coupling an ad vanced land-surface/hyd rology model with the Penn State/NCAR MM5 modeling system. Part I: Model implementation and sensitivity. Monthly Weather Review

, 129 , 569-585.

Dudhia, J., 1993: A Non-hydros tatic Version of the Penn State/NCAR Mesoscale Model: Validation Tests and Simulation of an Atlantic Cyclone and Cold Front . Monthly Weather

Review

, 121 , 1493-1513.

Emery, C., and Edward Tai, 2001: Meteorological Modeling and Performance Evaluation of the September 13-20, 1999 Ozone Episode. Prepar ed for the Texas Near Non-Attainment Areas through the Alamo Area Council of Governments, by ENVIRON International Corp, Novato, CA.

McNally, Dennis, 2003: Annual Application of MM5 for Calendar Year 2001. Prepared for Pat Dolwick, USEPA, March 31 st 

# 2003.

Olerud, Donald, K. Alapaty, and N. Wheeler, 2000: Meteorological Modeling of 1996 for the United States with MM5. Prepared for Norm al Possiel, Client Representative, U.S. EPA/OAQPS. EPA Task Order Number CAA689805

Olerud, Don and Aaron Sims, 2004: Draft Protocol for Annua l MM5 Modeling in Support of VISTAS (Visibility Improvement – State and Tr ibal Association) (DRAFT), Prepared for Mr. Mike Abraczinskas, VISTAS Technical Analysis Workgroup.

Pleim, J. E., and J. S. Chang, 1992: A non-lo cal closure model for vertical mixing in the convective boundary layer. Atmospheric Environment

, 26A , 965-981.

Stauffer, D. R., and N. L. Se aman, 1990: Use of Four Dimens ional Data Assimilation in a Limited-Area Mesoscale Model. Part I: Experiments with Synoptic Data. Monthly

Weather Review

, 118 , 1250-1277.

49

# 50 Stauffer, D. R., and N. L. Seaman. 1991: Us e of Four Dimensional Data Assimilation in a Limited-Area Mesoscale Model. Part II: Effects of Data Assimilation within the Planetary Boundary Layer. Monthly Weather Review

, 119 , 734-754

Tesche, T. W. 1994. “Evaluation Procedures for Regional Emissions, Meteorological, and

Photochemical Models.” Presented at the 86th Annual Meeting of the Air and Waste Management Association, 14-18 June, Denver, CO.

Xiu, A., and J. E. Pleim, 2000: Development of a land surface model. Part I: Application in a mesoscale meteorology model. Journal of Applied Meteorology

, 40 , 192-209.51APPENDIX ATemporal structure examplefor the 2002 annual simulation.Start BLOCKEnd BLOCKStart (Z)End (Z)FilenameMetstat UsableStart (Z)End (Z)12/28/2001 12:001/2/2002 12:0012/28/2001 12:0012/28/2001 12:00MMOUT_DOMAIN1_0012/28/2001 13:0012/29/2001 12:00MMOUT_DOMAIN1_0112/29/2001 0:0012/29/2001 12:0012/29/2001 13:0012/30/2001 12:00MMOUT_DOMAIN1_0212/29/2001 13:0012/30/2001 12:0012/30/2001 13:0012/31/2001 12:00MMOUT_DOMAIN1_0312/30/2001 13:0012/31/2001 12:0012/31/200113:001/1/200212:00MMOUT_DOMAIN1_0412/31/2001 13:001/1/2002 12:001/1/2002 13:001/2/2002 12:00MMOUT_DOMAIN1_051/1/2002 13:001/1/2002 23:001/1/2002 12:001/6/2002 12:001/1/2002 12:001/1/2002 12:00MMOUT_DOMAIN1_001/1/2002 13:001/2/2002 12:00MMOUT_DOMAIN1_011/2/2002 0:001/2/2002 12:001/2/2002 13:001/3/2002 12:00MMOUT_DOMAIN1_021/2/2002 13:001/3/2002 12:001/3/2002 13:001/4/2002 12:00MMOUT_DOMAIN1_031/3/2002 13:001/4/2002 12:001/4/2002 13:001/5/2002 12:00MMOUT_DOMAIN1_041/4/2002 13:001/5/2002 12:001/5/2002 13:001/6/2002 12:00MMOUT_DOMAIN1_051/5/2002 13:001/5/2002 23:001/5/2002 12:001/10/2002 12:001/5/2002 12:001/5/2002 12:00MMOUT_DOMAIN1_001/5/2002 13:001/6/2002 12:00MMOUT_DOMAIN1_011/6/2002 0:001/6/2002 12:001/6/2002 13:001/7/2002 12:00MMOUT_DOMAIN1_021/6/2002 13:001/7/2002 12:001/7/2002 13:001/8/2002 12:00MMOUT_DOMAIN1_031/7/2002 13:001/8/2002 12:001/8/2002 13:001/9/2002 12:00MMOUT_DOMAIN1_041/8/2002 13:001/9/2002 12:001/9/200213:001/10/200212:00MMOUT_DOMAIN1_051/9/2002 13:001/9/2002 23:00

# APPENDIX B

Daily averaged metrics from the 36 km simulation.



# Eastern Subdomains

52

# 53

# 54

# 55

# 56

# 57

# Western Subdomains

58

# 59

# 60

# 61

# 62

# 63

# 64

# 65

# APPENDIX C

Hourly statistical results for both the 36 and 12 km grids.



# CenrapN: January 2002

Observed/Predicted Windspeed

0 2 4 6 8

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/15 1/

16

1/17 1/18 1/

19

1/20 1/

21

1/

22

1/23 1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31m/s ObsWndSpd 36 km winds 12 km winds



# Bias Windspeed

-2.5 -1.5 -0.5 0.5 1.5

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/20 1/

21

1/

22

1/23 1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31m/s 36km Bias 12km Bias



# Observed/Predicted Wind Direction

0 60 120 180 240 300 360

1/ 1 1/ 2 1/ 3

1/ 4 1/

5

1/ 6 1/

7

1/ 8 1/

9

1/

10

1/11 1/

12

1/13 1/

14

1/15 1/16 1/

17

1/

18

1/19 1/

20

1/21 1/

22

1/23 1/

24

1/25 1/26 1/

27

1/28 1/

29

1/30 1/

31deg ObsWndDir 36km Predict ed 12km Predict ed



# Bias Wind Direction

-30

0 30

1/ 1 1/ 2

1/ 3 1/

4

1/ 5 1/

6

1/ 7 1/

8

1/ 9 1/

10

1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1 /18

1/

19

1/

20

1/21 1/

22

1/23 1/

24

1/25 1/

26

1/27 1/

28

1/29 1/30 1/

31deg 36km Bias WD 12km Bias WD



# 66

# Observed/Predicted Temperature

240 245 250 255 260 265 270 275 280 285 290

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31K ObsTemp 36 km Pred_T 12 km Pred_T



# Bias Temperature

-5 -4 -3 -2 -1 0 1 2 3

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31K 36 km Bias T 12 km Bias T



# Predicted/Observed Humidity

0 1 2 3 4 5 6

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31g/kg ObsHum 36km Pred q 12 k Pred q



# Bias Humidity

-0.4 -0.2

0 0.2 0.4 0.6 0.8

1

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31g/kg 36 km Bias q 12 km Bias q



# 67

# CenrapN: June 2002

Observed/Predicted Windspeed

0 5 10

6/ 1 6/ 2

6/ 3 6/ 4 6/

5

6/ 6 6/

7

6/

8

6/ 9 6/

10

6/

11

6/12 6/

13

6/14 6/

15

6/16 6/

17

6/

18

6/19 6/

20

6/21 6/22 6/

23

6/24 6/25 6/

26

6/27 6/

28

6/

29

6/30m/s ObsWndSpd 36 km winds 12 km winds



# Bias Windspeed

-2 -1 0 1

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/

15

6 /16

6/

17

6/

18

6/

19

6/20 6/

21

6/

22

6/23 6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30m/s 36 km Bias 12 km Bias



# Observed/Predicted Wind Direction

0 60 120 180 240 300 360

6/ 1 6/ 2

6/ 3 6/ 4 6/

5

6/ 6 6/

7

6/

8

6/ 9 6/

10

6/

11

6/12 6/

13

6/14 6/

15

6/16 6/

17

6/

18

6/19 6/

20

6/21 6/22 6/

23

6/24 6/25 6/

26

6/27 6/

28

6/

29

6/30deg ObsWndDir 36 km PredWnd 12 km PredWndDir



# Bias Wind Direction

-30

0 30

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/

15

6 /16

6/

17

6/

18

6/

19

6/20 6/

21

6/

22

6/23 6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30deg 36km Bias 12km Bias



# 68

# Observed/Predicted Temperature

270 275 280 285 290 295 300 305 310

6/

1

6/

2

6/

3

6/

4

6/

5

6/

6

6/

7 6/

8

6/

9 6/1

0 6/1 1

6/

1 2 6/1

3

6/

1 4 6/15 6/

1 6 6/17 6/18 6/

1 9 6/2

0

6/

2 1 6/

2 2 6/2

3

6/

2 4 6/

2 5 6/2

6

6/

2 7 6/2

8 6/2

9

6/

3 0K ObsTemp 36 km Pred_T 12 km Pred_T



# Bias Temperature

-2 -1 0 1 2 3 4

6/ 1 6/ 2

6/ 3 6/ 4 6/

5

6/ 6 6/

7

6/

8

6/ 9 6/

10

6/

11

6/12 6/

13

6/14 6/15 6

/ 16

6/17 6/18 6/19 6/

2 0 6/21 6/22 6/

2 3 6/24 6/25 6/

26

6/27 6/

28

6/

29

6/30K 36 km Bias T 12 km Bias T



# Predicted/Observed Humidity

0 5 10 15 20

6/

1

6/

2

6/

3

6/

4

6/

5

6/

6

6/

7 6/

8

6/

9 6/1

0 6/1 1

6/

1 2 6/1

3

6/

1 4 6/15 6/

1 6 6/17 6/18 6/

1 9 6/2

0

6/

2 1 6/

2 2 6/2

3

6/

2 4 6/

2 5 6/2

6

6/

2 7 6/2

8 6/2

9

6/

3 0g/kg ObsHum 36km Pred q 12 k Pred q



# Bias Humidity

-2 -1 0 1 2 3

6/ 1 6/ 2

6/ 3 6/ 4 6/

5

6/ 6 6/

7

6/

8

6/ 9 6/

10

6/

11

6/12 6/

13

6/14 6/15 6

/ 16

6/17 6/18 6/19 6/

2 0 6/21 6/22 6/

2 3 6/24 6/25 6/

26

6/27 6/

28

6/

29

6/30g/kg 36 km Bias q 12 km Bias q



# 69

# OhioVal: January 2002

Observed/Predicted Windspeed

0 2 4 6 8

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/15 1/

16

1/17 1/18 1/

19

1/20 1/

21

1/

22

1/23 1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31m/s ObsWndSpd 36 km winds 12 km winds



# Bias Windspeed

-2 -1 0 1

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/20 1/

21

1/

22

1/23 1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31m/s 36 km Bias 12 km Bias



# Observed/Predicted Wind Direction

0 60 120 180 240 300 360

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/15 1/

16

1/17 1/18 1/

19

1/20 1/

21

1/

22

1/23 1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31deg ObsWndDir 36 km PredWnd 12 km PredWndDir



# Bias Wind Direction

-30

0 30

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/20 1/

21

1/

22

1/23 1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31deg 36km Bias 12km Bias



# 70

# Observed/Predicted Temperature

245 250 255 260 265 270 275 280 285 290 295

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31K ObsTemp 36 km Pred_T 12 km Pred_T



# Bias Temperature

-5 -4 -3 -2 -1 0 1 2 3

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31K 36 km Bias T 12 km Bias T



# Predicted/Observed Humidity

0 1 2 3 4 5 6 7 8 9 10

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31g/kg ObsHum 36km Pred q 12 k Pred q



# Bias Humidity

-1 -0.5

0 0.5

1 1.5

2

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31g/kg 36 km Bias q 12 km Bias q



# 71

# OhioVal: June 2002

Observed/Predicted Windspeed

0 2 4 6 8

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/15 6/

16

6/17 6/18 6/

19

6/20 6/

21

6/

22

6/23 6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30m/s ObsWndSpd 36 km winds 12 km winds



# Bias Windspeed

-1.5

-1 -0.5

0 0.5

1

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/

15

6 /16

6/

17

6/

18

6/

19

6/20 6/

21

6/

22

6/23 6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30m/s 36 km Bias 12 km Bias



# Observed/Predicted Wind Direction

0 60 120 180 240 300 360

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/15 6/

16

6/17 6/18 6/

19

6/20 6/

21

6/

22

6/23 6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30deg ObsWndDir 36 km PredWnd 12 km PredWndDir



# Bias Wind Direction

-30

0 30

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/

15

6 /16

6/

17

6/

18

6/

19

6/20 6/

21

6/

22

6/23 6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30deg 36km Bias 12km Bias



# 72

# Observed/Predicted Temperature

275 280 285 290 295 300 305 310

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/

15

6 /16

6/

17

6/

18

6/

19

6/2 0

6/

21

6/

22

6/2

3

6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30K ObsTemp 36 km Pred_T 12 km Pred_T



# Bias Temperature

-2 -1.5

-1 -0.5

0 0.5

1 1.5

2 2.5

3

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/

15

6 /16

6/

17

6/

18

6/

19

6/2 0

6/

21

6/

22

6/2

3

6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30K 36 km Bias T 12 km Bias T



# Predicted/Observed Humidity

0 2 4 6 8 10 12 14 16 18 20

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/

15

6 /16

6/

17

6/

18

6/

19

6/2 0

6/

21

6/

22

6/2

3

6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30g/kg ObsHum 36km Pred q 12 k Pred q



# Bias Humidity

-2 -1.5

-1 -0.5

0 0.5

1 1.5

2 2.5

3 3.5

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/

15

6 /16

6/

17

6/

18

6/

19

6/2 0

6/

21

6/

22

6/2

3

6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30g/kg 36 km Bias q 12 km Bias q



# 73

# Iowa: January 2002

Observed/Predicted Windspeed

0 5 10 15

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/15 1/

16

1/17 1/18 1/

19

1/20 1/

21

1/

22

1/23 1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31m/s ObsWndSpd 36 km winds 12 km winds



# Bias Windspeed

-3.5 -2.5 -1.5 -0.5 0.5 1.5

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/20 1/

21

1/

22

1/23 1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31m/s 36km Bias 12km Bias



# Observed/Predicted Wind Direction

0 60 120 180 240 300 360

1/ 1 1/ 2 1/ 3

1/ 4 1/

5

1/ 6 1/

7

1/ 8 1/

9

1/

10

1/11 1/

12

1/13 1/

14

1/15 1/16 1/

17

1/

18

1/19 1/

20

1/21 1/

22

1/23 1/

24

1/25 1/26 1/

27

1/28 1/

29

1/30 1/

31deg ObsWndDir 36km Predict ed 12km Predict ed



# Bias Wind Direction

-30

0 30

1/ 1 1/ 2

1/ 3 1/

4

1/ 5 1/

6

1/ 7 1/

8

1/ 9 1/

10

1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1 /18

1/

19

1/

20

1/21 1/

22

1/23 1/

24

1/25 1/

26

1/27 1/

28

1/29 1/30 1/

31deg 36km Bias WD 12km Bias WD



# 74

# Observed/Predicted Temperature

230 240 250 260 270 280 290 300

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31K ObsTemp 36 km Pred_T 12 km Pred_T



# Bias Temperature

-5 -4 -3 -2 -1 0 1 2 3 4

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31K 36 km Bias T 12 km Bias T



# Predicted/Observed Humidity

0 1 2 3 4 5 6

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31g/kg ObsHum 36km Pred q 12 k Pred q



# Bias Humidity

-0.6 -0.4 -0.2

0 0.2 0.4 0.6 0.8

1 1.2 1.4

1/

1

1/ 2 1/

3

1/

4

1/ 5 1/

6

1/ 7 1/ 8 1/

9 1/10 1/11 1/

12

1/13 1/

14

1/

15

1 /16

1/

17

1/

18

1/

19

1/2 0

1/

21

1/

22

1/2

3

1/

24

1/

25

1/26 1/

27

1/28 1/29 1/

30

1/31g/kg 36 km Bias q 12 km Bias q



# 75

# Iowa: June 2002

Observed/Predicted Windspeed

0 5 10 15

6/ 1 6/ 2

6/ 3 6/ 4 6/

5

6/ 6 6/

7

6/

8

6/ 9 6/

10

6/

11

6/12 6/

13

6/14 6/

15

6/16 6/

17

6/

18

6/19 6/

20

6/21 6/22 6/

23

6/24 6/25 6/

26

6/27 6/

28

6/

29

6/30m/s ObsWndSpd 36 km winds 12 km winds



# Bias Windspeed

-4 -2 0 2

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/

15

6 /16

6/

17

6/

18

6/

19

6/20 6/

21

6/

22

6/23 6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30m/s 36 km Bias 12 km Bias



# Observed/Predicted Wind Direction

0 60 120 180 240 300 360

6/ 1 6/ 2

6/ 3 6/ 4 6/

5

6/ 6 6/

7

6/

8

6/ 9 6/

10

6/

11

6/12 6/

13

6/14 6/

15

6/16 6/

17

6/

18

6/19 6/

20

6/21 6/22 6/

23

6/24 6/25 6/

26

6/27 6/

28

6/

29

6/30deg ObsWndDir 36 km PredWnd 12 km PredWndDir



# Bias Wind Direction

-30

0 30

6/

1

6/ 2 6/

3

6/

4

6/ 5 6/

6

6/ 7 6/ 8 6/

9 6/10 6/11 6/

12

6/13 6/

14

6/

15

6 /16

6/

17

6/

18

6/

19

6/20 6/

21

6/

22

6/23 6/

24

6/

25

6/26 6/

27

6/28 6/29 6/

30deg 36km Bias 12km Bias



# 76

# Observed/Predicted Temperature

270 275 280 285 290 295 300 305 310

6/ 1 6/ 2

6/ 3 6/ 4 6/

5

6/ 6 6/

7

6/

8

6/ 9 6/

10

6/

11

6/12 6/

13

6/14 6/15 6

/ 16

6/17 6/18 6/19 6/

2 0 6/21 6/22 6/

2 3 6/24 6/25 6/

26

6/27 6/

28

6/

29

6/30K ObsTemp 36 km Pred_T 12 km Pred_T



# Bias Temperature

-3 -2 -1 0 1 2 3 4

6/ 1 6/ 2

6/ 3 6/ 4 6/

5

6/ 6 6/

7

6/

8

6/ 9 6/

10

6/

11

6/12 6/

13

6/14 6/15 6

/ 16

6/17 6/18 6/19 6/

2 0 6/21 6/22 6/

2 3 6/24 6/25 6/

26

6/27 6/

28

6/

29

6/30K 36 km Bias T 12 km Bias T



# Predicted/Observed Humidity

0 5 10 15 20

6/ 1 6/ 2

6/ 3 6/ 4 6/

5

6/ 6 6/

7

6/

8

6/ 9 6/

10

6/

11

6/12 6/

13

6/14 6/15 6

/ 16

6/17 6/18 6/19 6/

2 0 6/21 6/22 6/

2 3 6/24 6/25 6/

26

6/27 6/

28

6/

29

6/30g/kg ObsHum 36km Pred q 12 k Pred q



# Bias Humidity

-4 -3 -2 -1 0 1 2 3 4

6/ 1 6/ 2

6/ 3 6/ 4 6/

5

6/ 6 6/

7

6/

8

6/ 9 6/

10

6/

11

6/12 6/

13

6/14 6/15 6

/ 16

6/17 6/18 6/19 6/

2 0 6/21 6/22 6/

2 3 6/24 6/25 6/

26

6/27 6/

28

6/

29

6/30g/kg 36 km Bias q 12 km Bias q



# 77