Appendix E.2



MM5 Modeling Protocol

Tennessee Department of Environment and Conservation 

Division of Air Pollution Control 

9th Floor L&C Annex 

401 Church Street 

Nashville, TN 37243

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 1

January 18, 2008
Draft

Protocol for Annual MM5 Modeling in Support of VISTAS (Visibility 

Improvement – State and Tribal Association) 

(DRAFT)

Task 3a Deliverable

Prepared for:

Mr. Mike Abraczinskas

VISTAS Technical Analysis Workgroup

NC Division of Air Quality

1641 Mail Service Center

Raleigh, NC 276990-1641

Prepared by:

Mr. Don Olerud

Mr. Aaron Sims

Baron Advanced Meteorological Systems, LLC

P.O. Box 12889, 3021 Cornwallis Road

Research Triangle Park, North Carolina 27709

February 9, 2004

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 2

January 18, 2008
Draft

ii

1 Introduction..................................................................................................................................................... 1

2 Description of the Meteorological Modeling Approach................................................................................. 1

3 Evaluation Approach....................................................................................................................................... 3

4 Results............................................................................................................................................................. 5

5 Auxiliary Results........................................................................................................................................... 52

6 Conclusions from Sensitivity Modeling ..................................................................................................... 109

7 Methodology for Annual Modeling Effort ................................................................................................. 110

A. Modeling Approach................................................................................................................................ 110

B. Model Execution..................................................................................................................................... 112

Terrain......................................................................................................................................................... 112

REGRID (pregrid)....................................................................................................................................... 112

REGRID (regridder) ................................................................................................................................... 112

FETCH (adp_sfc)........................................................................................................................................ 113

FETCH (adp_upa)....................................................................................................................................... 113

LITTLE_R (Objective Analysis)................................................................................................................ 113

INTERPF .................................................................................................................................................... 113

MM5............................................................................................................................................................ 113

C. Model Evaluation Procedures................................................................................................................. 114

D. Computing Resources............................................................................................................................. 124

E. Project Schedule...................................................................................................................................... 124

8 References................................................................................................................................................... 124

9 Appendices...................................................................................................................................................... 1

A. Terrain namelist ...................................................................................................................................... 1

B. Pregrid script........................................................................................................................................... 2

C. Regridder 36-km sample namelist ........................................................................................................ 10

D. FETCH (adp_sfc) sample namelist....................................................................................................... 11

E. FETCH (adp_upa) sample namelist...................................................................................................... 11

F. LITTLE_R sample 36-km namelist ...................................................................................................... 11

G. INTERPF sample 36-km namelist ........................................................................................................ 14

H. MM5 sample mmlif .............................................................................................................................. 15

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 3

January 18, 2008
Draft

1

1 Introduction

In recent years visibility concerns have come to the forefront in the air quality community. Millions of 

visitors to national parks in the United States have their views obstructed by pollution-induced haze. The 

USEPA reports that average visibility in the east has been reduced from 90 miles to 15-25 miles 

(http://www.epa.gov/oar/visibility/what.html). To address this issue, the USEPA in 1999 instituted policies to 

improve visibility in the national parks. As part of this initiative, five multi-state regional planning 

organizations (RPO) were formed. The RPO governing visibility issues in the southeastern US is the Visibility 

Improvement State and Tribal Association of the Southeast (VISTAS) (http://www.vistas-sesarm.org/). 

VISTAS recognizes the regional nature of haze, and has therefore set up a modeling approach to address the 

problem in the southeast US. Ultimately pollution controls will be enacted based upon chemical modeling 

results over the region of interest. To support this modeling effort, Baron Advanced Meteorological Systems 

(BAMS) is tasked with conducting the meteorological modeling. A 12-month modeling period is deemed 

necessary to cover an adequate range of visibility impairment. Prior to investing the resources to produce 

meteorological results at 36-km and 12-km resolution for the full 12-month period, BAMS executed a series of 

sensitivity tests to determine the optimal meteorological setup for the annual modeling. This report details the 

results from this sensitivity testing, leading directly to the protocol we will use for this project. 

2 Description of the Meteorological Modeling Approach

The meteorological model used in this study is the PSU/NCAR Mesoscale Model (MM5 version 3.6, Grell 

et al., 1994, MPP version). In order to build on prior relevant MM5 modeling results funded by the EPA and 

other RPOs, those studies serve to establish the initial model configuration for this effort, as best can be 

determined from available reports. Those findings are summarized in Olerud, 2003a. The modeling results 

indicate that MM5 is most sensitive to the selection of planetary boundary layer (PBL) and soil schemes. 

Therefore a series of sensitivity tests are recommended in Olerud, 2003b. Given limited time and budget 

resources, a series of five sensitivity tests are laid out testing primarily the model response to the selection of 

PBL scheme and soil model. These are the tests:

• px_acm: Pleim-Xiu land surface model, asymmetric convective mixing PBL (Xiu and 

Pleim, 2000), interppx to link runs.

• px_acm2: Like px_acm, except no interppx linkage.

• noah_mrf: Noah land-surface scheme (Chen and Dudhi, 2001) with the medium range 

forecast (mrf) PBL (Hong and Pan, 1996). 

• multi_blkdr: Multi-layer soil scheme with Blackadar PBL and Zilitinkevich thermal 

roughness length.

• noah_eta-my: Noah land-surface scheme with the ETA Mellor-Yamada PBL (IMVDIF=0).

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 4

January 18, 2008
Draft

2

The common options for all sensitivity tests include Kain-Fritsch 2 cumulus parameterization (Kain and 

Fritsch, 1993; Kain, 2002), mixed phase (Reisner 1) microphysics (Reisner et al, 1998), and Rapid Radiative 

Transfer Model (RRTM) radiation (Mlawer et al, 1997). Snow effects are turned on (IFSNOW=1). Note that the 

use of the ETA M-Y pbl scheme necessitates moist vertical diffusion being turned off. The runs are made with 

analysis nudging coefficients set as follows (36-km and 12-km resolutions):

Winds (aloft): 2.5E-4,1.0E-4,

Winds (surface): 2.5E-4,1.0E-4,

Temp (aloft): 2.5E-4,1.0E-4,

Temp (surface): Not employed

Moisture (aloft): 1.0E-5,1.0E-5

Moisture (surface): Not employed

In the cross-sensitivity plots that follow we compare the px_acm2 sensitivity run to the noah_mrf, 

multi_blkdr, and noah_eta-my sensitivities for episode 1. Episodes 2 and 3 are similar except px_acm will serve 

as the base configuration. 

The runs are executed in 2-way mode with feedback turned off. The five sensitivity runs are executed for 

three separate episodes listed below:

• Episode 1: January 2-21, 2002

• Episode 2: July 13-28, 2001

• Episode 3: July 13-22, 1999.

Each episode is preceded by a spin-up period (7, 7, and 4 days, respectively) that will not be discussed in 

this report. The runs are made in 5.5-day segments, each starting at 00 UTC, with the first 12 hours of each 

segment serving as spin-up. Figure 1 shows the modeling domains used in this study.

Figure 1. VISTAS 36-km/12-km MM5 modeling domains are shown.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 5

January 18, 2008
Draft

3

3 Evaluation Approach

It is common in the air quality community to use surface statistics of the base meteorological variables as 

the dominant metrics to determine acceptable model performance. Often statistics for only temperature, mixing 

ratio and wind speed are calculated. Obviously it is important for the model to accurately represent these 

variables, but there are additional variables that also become important when one considers that the results will 

be used to improve visibility. As such we have added cloud cover, relative humidity, and precipitation to the 

performance suite. While we calculate metrics separately for wind direction and wind speed, we also calculate 

the mean error vector as perhaps the single best metric to quantify overall wind performance. 

Recognizing that qualitative analyses of the model output are as important as standard quantitative analyses, 

we enable the systematic visualization of model fields with observations overlaid whenever possible. To do this 

we process the MM5 output through EPA’s MCIP2 program. MCIP2 transforms the data into NetCDF format 

while also calculating a few fields (e.g. low, middle, and high CFRAC) that are not readily available in the raw 

MM5 output. MCIP2 also interpolates temperature and wind speed to observation height (1.5m and 10m, 

respectively) for more accurate evaluation. Even though MCIP2 outputs a total cloud fraction, CMAQ uses this 

quantity to estimate optical depth. Accordingly its value can be markedly different than what meteorologists 

typically regard as cloud fraction. To make things as consistent as possible between the model and observations, 

the cloud fractions presented in this report represent the maximum of the low, middle, and high cloud fractions. 

We also use MCIP2 to cull a minimum of six cells about the domain periphery to minimize edge effects. The 

reduced domain precisely matches the domain used in the air quality modeling. The 36-km analysis domain 

thus contains 148 columns, 112 rows, and 34 layers. The 12-km analysis domain covers 168 columns, 177 rows, 

and 34 layers.

The observations used for statistics come primarily from UCAR’s ds472.0 (TDL) archive 

(http://dss.ucar.edu/datasets/ds472.0/). These data are quality controlled and converted to NetCDF format, thus 

allowing the data to be visualized on the model fields via PAVE 

(http://www.cep.unc.edu/empd/EDSS/pave_doc/index.shtml). Unfortunately the precipitation values in this 

dataset are not reliable, so we calculate precipitation statistics based on the 24-h gridded accumulations 

available from the Climate Prediction Center (CPC) 

(http://www.cpc.ncep.noaa.gov/products/precip/realtime/retro.html). These fields, originally at 0.25- degree 

resolution, undergo grid transformation to match our 36-km and 12-km domains. Since the CPC analyses are 

derived primarily from rain gauges, the statistics are only calculated over cells that MM5 deems to be land. 

For aloft analyses we process standard sounding observations from the NCEP ds353.4 archive 

(http://dss.ucar.edu/datasets/ds353.4/). These observations are quality controlled and used to produce 

model/observation skewT sounding plots for selected sites. Additionally we integrate the observations into 

sigma levels that match the MM5 specifications, after which we can statistically analyze performance at sigma 

levels 9, 17, and 22 (~500m, ~1600m, ~3400m, respectively). Qualitative profiler plots showing 

model/observed hourly winds are also created based upon the data stored at the Forecast Systems Lab 

(http://www.profiler.noaa.gov/jsp/). These results, along with much more, will not be presented here. The reader 

is referred to the VISTAS meteorological website (http://www.baronams.com/projects/VISTAS/) for additional 

evaluation details and results. 

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 6

January 18, 2008
Draft

4

The number of analysis plots available on the above website is truly daunting. To keep this report at a 

somewhat manageable level, we will focus primarily on cross-sensitivity plots and surface statistics. Except for 

precipitation (as mentioned above), these statistics are calculated at the sites (color-coded by RPO) shown in 

Figure 2. Statistics are calculated and stored at each observing site, and we routinely aggregate these results to 

produce statistical time series plots and tables for every appropriate RPO region. This approach also enables us 

to produce station-specific statistical quantities that can be plotted in a similar manner to Figure 2. The VISTAS 

web page even shows an animation of how these quantities change throughout an episode-composite day. The 

results shown in this document focus on statistics aggregated only over the VISTAS portion of the 12-km 

domain, and the US portion of the 36-km domain. The cross-sensitivity plots are shown only for the 12-km 

VISTAS domain.

Figure 2. Surface observing network color-coded to represent Regional Planning Organization areas. Dark blue diamonds 

are in the VISTAS RPO, green diamonds are in the MANE-VU RPO, light blue diamonds are in the MIDWEST RPO, 

yellow diamonds are in the CENRAP RPO, and red diamonds are in the WRAP RPO.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 7

January 18, 2008
Draft

5

4 Results

The initial results for the px_acm run for episode 1 are quite discouraging. The run showa a significant cold 

bias over much of the eastern US, including the VISTAS region as illustrated in figure 3. While mixing ratio, 

clouds, precipitation, and winds are modeled reasonably well, the large cold bias is unexpected based on prior 

findings from other RPO’s and the EPA. Note in figure 3 that the first couple of days show very little 

temperature bias, but the bias increases as the mean temperature rises. After much investigation it becomes 

apparent that the deep soil temperature is initiated during an extreme cold event in the eastern US. Since the 

model soil temperatures and moisture are passed from one model segment to another via the interppx 

preprocessor, the cold soil acts as a continuous drag on the atmosphere that the model physics can never quite 

overcome. 

The bias problem is significantly reduced by simply running each model segment independently, thus 

limiting the cold drag to actual cold conditions. Figure 4 shows the statistical time series for this new case, 

px_acm2. This px_acm2 configuration will henceforth be considered the base case episode 1 for crosssensitivity purposes, while px_acm will be used for episodes 2 and 3. For the statistical difference tables that 

make up the bulk of the table portion of this document, we employ px_acm2 as the de facto base case. 

Figure 3. Episode 1 temperature (1.5 m) statistical time series plot for the 12-km VISTAS region, px_acm sensitivity. The 

top panel shows the mean of the observations (blue) and the model (red), the middle plot shows the model bias (blue) and 

the absolute error (red), while the bottom plot shows the index of agreement (blue) and coefficient of determination (red).

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 8

January 18, 2008
Draft

6

Figure 4. Episode 1 temperature (1.5 m) statistical time series plot for the 12-km VISTAS region, px_acm2 sensitivity. 

The top panel shows the mean of the observations (blue) and the model (red), the middle plot shows the model bias (blue) 

and the absolute error (red), while the bottom plot shows the index of agreement (blue) and coefficient of determination 

(red).

Figure 5 shows the daytime PBL heights for January 10, 2002 at 12-km resolution over the VISTAS region. 

It should be noted that the noah_eta-my PBL heights can erroneously become negative over small spatial areas; 

we set all such negative values to zero before averaging. Due to the small areal extent of these negative PBL 

values, we do not anticipate any qualitative assessments to be affected by those artifacts. The January 10, 2002 

PBL heights are rather typical of winter PBL heights. The noah_mrf heights are significantly higher and 

smoother than those in the other sensitivities. Generally speaking, the noah_eta-my daytime PBL heights are 

lower than they are in the other sensitivity runs. The px_acm2 heights tend to be more in the middle of those 

extremes, though they also “bottom out” more than the other runs. 

Figure 6 shows the cross-sensitivity daytime precipitation plot for this same day. The low PBL heights in 

the px_acm2 run are closely correlated to precipitation in the Ohio Valley, while melting snow and clouds 

(figure 7) might inhibit mixing over the northern Mid-Atlantic States. 

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 9

January 18, 2008
Draft

7

Figure 5. Daytime (18-21 UTC) average PBL heights for the 12-km VISTAS region for January 10, 2002 are displayed. 

The px_acm2 sensitivity is shown in the upper left, the noah_mrf in the upper right, the multi_blkdr in the lower left, and 

the noah_eta-my in the lower right. Note that the time value (0:00:00) is only a placeholder and has no physical meaning.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 10

January 18, 2008
Draft

8

Figure 6. Like figure 5, except for precipitation. 

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 11

January 18, 2008
Draft

9

Figure 7. Like figure 5, except for cloud coverage. 

Figure 8 shows the daily average temperature for this same day. Note that the px_acm2 case is generally 

warmer than the other cases, while the noah_eta-my run is the coldest. Figure 9 shows that for daily averaged 

mixing ratio the patterns are very similar for all runs. The combination of warmer daytime temperatures and 

similar mixing ratios results in lower daytime relative humidity for the px_acm2 case compared with the other 

sensitivity runs (figure 10). Finally, figure 11 shows that the daytime wind speeds are similar in all cases. 

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 12

January 18, 2008
Draft

10

Figure 8. Like figure 5, except for daily average temperature.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 13

January 18, 2008
Draft

11

Figure 9. Like figure 5, except for daily average temperature.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 14

January 18, 2008
Draft

12

Figure 10. Like figure 5, except for relative humidity.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 15

January 18, 2008
Draft

13

Figure 11. Like figure 5,except for wind speed.

Traditionally nighttime PBL heights have not been considered very important for air quality modeling, but 

they certainly may be for visibility/particulate modeling. Figure 12 shows the nighttime (07-10 UTC) PBL 

heights for January 10, 2002. Notice that the px_acm2 produces lower PBL heights than the rest of the 

sensitivity cases do, thus trapping surface-based emissions in a smaller volume of air than would occur in 

another MM5 configuration. The noah_mrf run again produces the highest PBL heights at night, while the 

multi_blkdr and noah_eta-my runs are somewhere in the middle. 

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 16

January 18, 2008
Draft

14

Figure 12. Nighttime (07-10 UTC) average PBL heights for the 12-km VISTAS region for January 10, 2002 are 

displayed. The px_acm2 sensitivity is shown in the upper left, the noah_mrf in the upper right, the multi_blkdr in the 

lower left, and the noah_eta-my in the lower right. Note that the time value (0:00:00) is only a placeholder and has no 

physical meaning.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 17

January 18, 2008
Draft

15

Nighttime cloud cover is shown in figure 13. The most striking observation from this figure is the cloud 

deck over Tennessee in the noah_eta-my run that does not exist to the same extent in the other sensitivity runs. 

Figure 13. Like figure 12, except for cloud cover.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 18

January 18, 2008
Draft

16

Figure 14 shows the nighttime relative humidity plot. One might expect that the px_acm2 case would show 

the highest relative humidity, given the low PBL heights as indicated by figure 12. The opposite is actually the 

case. The warmer temperatures in this run counteract the increased stability such that the relative humidity 

values are the lowest of all the runs. The noah_eta-my run easily exhibits the highest relative humidity. 

Figure 14. Like figure 12, except for relative humidity.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 19

January 18, 2008
Draft

17

Figure 15 shows the nighttime wind speed. Speeds are lowest in the noah_eta-my run, followed by 

px_acm2, noah_mrf, and multi_blkdr. In fact, the latter two cases seem to show an inappropriate diurnal pattern 

in that their nighttime wind speeds are higher than their daytime wind speeds (figure 12). 

Figure 15. Like figure 12, except for wind speed.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 20

January 18, 2008
Draft

18

Many of the same observations reported above are also valid for the summer episodes. To save time we will 

only show spatial 4-panel plots for PBL heights for a sample summer day, July 19, 2001. Figure 16 shows the 

daytime average, while figure 17 shows the nighttime average.

Figure 16. Daytime (18-21 UTC) average PBL heights for the 12-km VISTAS region for July 19, 2001 are displayed. The 

px_acm sensitivity is shown in the upper left, the noah_mrf in the upper right, the multi_blkdr in the lower left, and the 

noah_eta-my in the lower right. Note that the time value (0:00:00) is only a placeholder and has no physical meaning.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 21

January 18, 2008
Draft

19

Figure 17. Nighttime (07-10 UTC) average PBL heights for the 12-km VISTAS region for July 19, 2001 are displayed. 

The px_acm sensitivity is shown in the upper left, the noah_mrf in the upper right, the multi_blkdr in the lower left, and 

the noah_eta-my in the lower right. Note that the time value (0:00:00) is only a placeholder and has no physical meaning.

Now that a qualitative understanding of these sensitivity runs has been established, the remainder of this 

report will focus on quantitative comparisons between the sensitivity cases. Figure 18 shows the temperature 

statistical time series plot for episode 1. While the general performance of the model is very similar across the 

sensitivity runs, close examination reveals that the px_acm2 case performs the best. Figure 19 shows the 

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 22

January 18, 2008
Draft

20

corresponding plot for episode 2. The noah_eta-my run clearly performs the worst, while the other sensitivity 

runs show similar performance. The episode 3 plot (not shown) reveals similar responses. 

Figure 20 shows the 12-km mixing ratio statistical time series plot for episode 1 over the VISTAS region. 

Overall the noah_mrf case performs the best, followed by px_acm2. The multi_blkdr case is clearly the poorest 

performing. The corresponding episode 2 plot (figure 21) reveals a different result in that the noah_mrf case is 

negatively biased in mixing ratio. This weakness presumably stems from dry air being mixed down from aloft 

as the PBL becomes too high. The other cases are relatively similar. The negative mixing ratio bias is also 

evident for noah_mrf in episode 3 (not shown). 

The wind direction plot for episode 1 is shown in figure 22. The direction bias and error plots show similar 

performance among the sensitivity cases, but the magnitude of the error vector plot (bottom panel) shows that 

the noah_eta-my is the best performing run, especially at night. The px_acm2 run is generally second best. 

Figure 23 shows that similar results are seen in episode 2, as well as in episode 3 (not shown). 

The cloud cover statistical plots (figures 24-25) show very little difference in performance among the 

sensitivity runs. Figure 26 reveals that relative humidity for episode 1 is best modeled by either px_acm2 or 

noah-mrf, with the multi_blkdr case performing the worst. The episode 2 plot (figure 27) shows a strong diurnal 

signature with the sensitivity runs generally being negatively biased at night and positively biased during the 

day. The diurnal signature is interestingly the weakest for the noah_eta-my run, leading to that sensitivity 

possibly performing the best for this quantity. The episode 3 plot (not shown) has the noah_eta-my run 

displaying the poorest performance, no doubt due to its negative temperature bias. 

Figure 28 shows the precipitation statistics for the full 12-km grid for episode 1. The bias blip on January 10 

resulted from there being very few grid cells that actually observed measurable precipitation on that day. 

Sensitivity px_acm2 clearly outperforms the other cases for this episode. Figure 29 shows the corresponding 

plot for episode 2. Sensitivity noah_eta-my seems to be relatively unbiased, while the other sensitivity runs 

show a slight low bias. Nevertheless the skill plots show little difference in performance among the runs. The 

px_acm case appears to show slightly better results than do the other runs. Similar results are found for episode 

3 (not shown). 

Figure 30 is designed to show which sensitivity case statistically performs the best at each valid observation 

site. This particular image represents a composite of 1.5m temperatures for all hours, with absolute error being 

the defining metric. Note that the px_acm2 run performs best for a majority of the sites. The noah_eta-my run 

appears to do quite well over Florida. Figure 31 shows the corresponding plot for episode 2. Again the px_acm 

case seems to perform best overall, though noah_eta-my again performs best in Florida and along the 

southeastern coastline. The results for episode 3 (not shown) reveal no best performing sensitivity. 

Figure 32 shows a similar type of plot for mixing ratio. The noah_mrf case appears to perform best for the 

largest number of sites, followed by the px_acm2 case. The episode 2 results (figure 33) indicate just the 

opposite, as the noah_eta-my or multi_blkdr cases seem to perform best for most of the sites. The episode 3 plot 

(not shown) is a mixed bag with the noah_eta-my and px_acm cases seemingly performing best.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 23

January 18, 2008
Draft

21

Figure 18. Episode 1 (Jan 2-21, 2002) cross-sensitivity statistical time series plot for temperature is shown. The top panel 

shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm2 case is shown in 

blue, noah_mrf in red, multi_blkdr in black, and noah_eta-my in purple.

Figure 19. Episode 2 (Jul 13-28, 2001) cross-sensitivity statistical time series plot for temperature is shown. The top panel 

shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm case is shown in blue, 

noah_mrf in red, multi_blkdr in black, and noah_eta-my in purple.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 24

January 18, 2008
Draft

22

Figure 20. Episode 1 (Jan 2-21, 2002) cross-sensitivity statistical time series plot for mixing ratio is shown. The top panel 

shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm2 case is blue,

noah_mrf red, multi_blkdr black, and noah_eta-my purple.

Figure 21. Episode 2 (Jul 13-28, 2001) cross-sensitivity statistical time series plot for mixing ratio is shown. The top panel 

shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm case is blue, noah_mrf 

red, multi_blkdr black, and noah_eta-my purple.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 25

January 18, 2008
Draft

23

Figure 22. Episode 1 (Jan 2-21, 2002) cross-sensitivity statistical time series plot for winds is shown. The top panel shows 

wind direction bias, the second panel absolute wind direction error, and the bottom panel the magnitude of the error wind 

vector. The px_acm2 case is shown in blue, noah_mrf in red, multi_blkdr in black, and noah_eta-my in purple.

Figure 23. Episode 2 (Jul 13-28, 2001) cross-sensitivity statistical time series plot for winds is shown. The top panel 

shows wind direction bias, the second panel absolute wind direction error, and the bottom panel the magnitude of the error 

wind vector. The px_acm case is shown in blue, noah_mrf in red, multi_blkdr in black, and noah_eta-my in purple.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 26

January 18, 2008
Draft

24

Figure 24. Episode 1 (Jan 2-21, 2002) cross-sensitivity statistical time series plot for cloud coverage is shown. The top 

panel shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm2 case is blue, 

noah_mrf red, multi_blkdr black, and noah_eta-my purple.

Figure 25. Episode 2 (Jul 13-28, 2001) cross-sensitivity statistical time series plot for cloud coverage is shown. The top 

panel shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm case is blue, 

noah_mrf red, multi_blkdr black, and noah_eta-my purple.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 27

January 18, 2008
Draft

25

Figure 26. Episode 1 (Jan 2-21, 2002) cross-sensitivity statistical time series plot for relative humidity is shown. The top 

panel shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm2 case is blue, 

noah_mrf red, multi_blkdr black, and noah_eta-my purple.

Figure 27. Episode 2 (Jul 13-28, 2001) cross-sensitivity statistical time series plot for relative humidity is shown. The top 

panel shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm case is blue, 

noah_mrf red, multi_blkdr black, and noah_eta-my purple.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 28

January 18, 2008
Draft

26

Figure 28. Episode 1 (Jan 2-21, 2002) cross-sensitivity statistical time series plot for 24-h measurable precip is shown. 

The top panel shows bias, the second panel accuracy, and the bottom panel equitable threat score. The px_acm2 case is 

blue, noah_mrf red, multi_blkdr black, and noah_eta-my purple.

Figure 29. Episode 2 (Jul 13-28, 2001) cross-sensitivity statistical time series plot for 24-h measurable precip is shown. 

The top panel shows bias, the second panel accuracy, and the bottom panel equitable threat score. The px_acm case is 

blue, noah_mrf red, multi_blkdr black, and noah_eta-my purple.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 29

January 18, 2008
Draft

27

Figure 30. Episode 1 (Jan 2-21, 2002) cross-sensitivity 1.5m temperature absolute error comparison plot is shown. 

Stations for which px_acm2 show the smallest composite error are plotted in blue, noah_mrf in green, multi_blkdr in 

yellow, and noah_eta-my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders 

and should be ignored.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 30

January 18, 2008
Draft

28

Figure 31. Episode 2 (Jul 13-28, 2001) cross-sensitivity 1.5m temperature absolute error comparison plot is shown. 

Stations for which px_acm2 show the smallest composite error are plotted in blue, noah_mrf in green, multi_blkdr in 

yellow, and noah_eta-my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders 

and should be ignored.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 31

January 18, 2008
Draft

29

Figure 32. Episode 1 (Jan 2-21, 2002) cross-sensitivity mixing ratio absolute error comparison plot is shown. Stations for 

which px_acm2 show the smallest composite error are plotted in blue, noah_mrf in green, multi_blkdr in yellow, and 

noah_eta-my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders and should 

be ignored.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 32

January 18, 2008
Draft

30

Figure 33. Episode 2 (Jul 13-28, 2001) cross-sensitivity mixing ratio absolute error comparison plot is shown. Stations for 

which px_acm2 show the smallest composite error are plotted in blue, noah_mrf in green, multi_blkdr in yellow, and 

noah_eta-my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders and should 

be ignored.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 33

January 18, 2008
Draft

31

The corresponding series of plots for the magnitude of the error wind vector (figures 34-35) show the clear 

superiority of the noah_eta-my runs. The px_acm(2) runs perform a distant second best for all sensitivities. The 

main reason why noah_eta-my performs so well is its ability to calm its wind speeds at night (figure 15) relative 

to the other model configurations. 

Figure 34. Episode 1 (Jan 2-21, 2002) cross-sensitivity error vector magnitude comparison plot is shown. Stations for 

which px_acm2 show the smallest composite error are plotted in blue, noah_mrf in green, multi_blkdr in yellow, and 

noah_eta-my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders and should 

be ignored.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 34

January 18, 2008
Draft

32

Figure 35. Episode 2 (Jul 13-28, 2001) cross-sensitivity error vector magnitude comparison plot is shown. Stations for 

which px_acm2 show the smallest composite error are plotted in blue, noah_mrf in green, multi_blkdr in yellow, and 

noah_eta-my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders and should 

be ignored.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 35

January 18, 2008
Draft

33

The remainder of this report will focus primarily on statistical tables. These tables are episode-composites 

and are divided into three sections. The top section contains bias, absolute error, coefficient of determination 

(r2), index of agreement, and root mean square error for temperature (1.5 m), water vapor mixing ratio, relative 

humidity, wind speed (10 m), and cloud coverage. The second section contains additional statistics for winds, 

including wind direction bias, absolute wind direction error, U-wind component error, V-wind component error, 

and the magnitude of the average error wind vector. The latter metric is perhaps the single best means of 

assessing model wind performance. The final section contains precipitation statistics at various thresholds (0.01, 

0.05, 0.10, 0.25, 0.5, and 1.0 inches) for daily precipitation accumulations. The metrics presented are accuracy, 

bias, false alarm ratio (FAR), probability of detection (POD), critical success index (CSI, i.e. threat score), 

equitable threat score (ETS), “true skill score” (TSS, i.e. Hanssen and Kuipers score), and Heidke skill score 

(HSS). These precipitation statistics are only generated for approximately the US portion of the grids; there are 

no RPO-specific statistics for this variable. Since an RPO-specific statistical summary is incomplete without 

some sort of precipitation metric, we have decided to use the precipitation statistics described above for every 

statistical composite we produce. The reader should understand the inconsistencies that this approach entails. 

Note also that the precipitation observational grids are obtained, in part, by objectively analyzing rain gauge 

data. This tends to “spread out” the measurable precipitation beyond where it actually occurs. Since a model 

does not do that, a perfect precipitation forecast might appear to be slightly low biased at the 0.01-inch 

threshold. At higher thresholds the number of valid occurrences may not reach a robust level.

 Sensitivity px_acm2 is the base case for all of the statistical tables that follow. Table 1 shows the base case 

statistics for the US portion of the 36-km domain for episode 1, while table 2 reports statistics for the VISTAS 

12-km domain for episode 1. Recall the meteorological statistical benchmarks reported by Emery (2001):

Wind speed: RSME <= 2 m/s, Bias <= +/- 0.5 m/s, IA >= 0.6

Wind direction: Gross Error <= 30 deg, Bias <= +/- 10 deg.

Temperature: Gross Error <= 2 K, Bias <= +/- 0.5 K, IA >= 0.8

Humidity: Gross Error <= 2 g/kg, Bias <= +/- 1 g/kg, IA >= 0.6

Note that the only metrics that fail to meet these benchmarks are temperature bias and error. It should be

understood that the above benchmarks are based primarily on meteorological modeling of summertime 

episodes. While ideally we want less temperature bias and error, the results seen here are not unusually bad. 

Note that the index of agreement temperature statistic easily betters the benchmark value for both domains. No 

benchmarks exist for relative humidity, cloud cover, and precipitation. It is encouraging that these quantities are 

all relatively unbiased and appear to show reasonable skill.

Tables 3-4 show the episode 2 statistical tables for px_acm2 for the selected domains. The only base 

variable that fails to meet the above benchmark is wind direction gross error at 36-km resolution, and that value 

(32.27o) is certainly in the ballpark. Given the weak synoptic forcing in this episode, it is likely that subsynoptic forcing (e.g. terrain flows, thunderstorm outflows, etc...) unresolved adequately at this resolution 

degrades the wind direction performance. The other variables are modeled reasonably well, with a couple of 

possible exceptions. Note the positive cloud coverage bias (>7%) in table 4, and the positive precipitation bias 

at higher thresholds in both table 3 and table 4. The model appears to overestimate the extent and intensity of 

summertime convection. Similar results are seen for episode 3 (tables 5-6). The cloud biases are less than in 

episode 2, though the positive precipitation biases at higher thresholds are accentuated at 36-km resolution. 

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 36

January 18, 2008
Draft

34

Total stats bias abserr r2 ia rmse

Temperature (K) -1.14 2.30 0.879 0.963 3.0613 

Mixing Ratio (g/kg) 0.07 0.53 0.876 0.967 0.7673 

Rel. Humidity (%) 3.54 12.62 0.451 0.814 16.4208 

Wind Speed (m/s) -0.02 1.33 0.510 0.808 1.7307 

Clouds (%) -4.00 26.12 0.340 0.763 37.6462 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 1.913 25.11 1.2520 1.2552 1.7729 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.8360 1.0061 0.2760 0.7284 0.5701 0.4379 0.6102 0.6091 

0.05 0.9038 1.1156 0.3204 0.7581 0.5585 0.4914 0.6898 0.6590 

0.10 0.9352 1.1405 0.3308 0.7632 0.5541 0.5115 0.7187 0.6768 

0.25 0.9703 1.0628 0.2959 0.7484 0.5693 0.5502 0.7309 0.7099 

0.50 0.9845 1.0006 0.3133 0.6872 0.5232 0.5140 0.6792 0.6790 

1.00 0.9940 0.8873 0.3833 0.5472 0.4083 0.4054 0.5446 0.5769 

Table 1. Episode 1 composite statistical summary for base case px_acm2 for the US portion of the 36-km domain is 

shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -1.03 2.25 0.858 0.957 2.9596 

Mixing Ratio (g/kg) 0.12 0.62 0.914 0.977 0.8708 

Rel. Humidity (%) 3.36 12.79 0.422 0.802 16.7355 

Wind Speed (m/s) 0.43 1.35 0.407 0.764 1.7320 

Clouds (%) 0.95 22.80 0.419 0.809 34.9631 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 2.216 22.74 1.1604 1.2419 1.6996 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.8735 1.0329 0.2086 0.8174 0.6725 0.5513 0.7170 0.7108 

0.05 0.9148 1.0648 0.2238 0.8265 0.6675 0.5955 0.7643 0.7465 

0.10 0.9363 1.0381 0.2145 0.8154 0.6669 0.6159 0.7741 0.7623 

0.25 0.9635 0.9532 0.1842 0.7776 0.6615 0.6343 0.7599 0.7763 

0.50 0.9756 0.9480 0.2568 0.7045 0.5665 0.5511 0.6929 0.7106 

1.00 0.9907 0.7899 0.3092 0.5457 0.4385 0.4337 0.5423 0.6050 

Table 2. Episode 1 composite statistical summary for base case px_acm2 for the VISTAS portion of the 12-km domain is 

shown.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 37

January 18, 2008
Draft

35

Total stats bias abserr r2 ia rmse

Temperature (K) -0.16 1.62 0.877 0.964 2.1443 

Mixing Ratio (g/kg) -0.17 1.47 0.809 0.947 1.9714 

Rel. Humidity (%) -0.28 9.78 0.672 0.904 13.0680 

Wind Speed (m/s) -0.20 1.24 0.429 0.772 1.6149 

Clouds (%) 1.38 27.52 0.134 0.635 37.9526 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 2.010 32.37 1.2469 1.3037 1.8040 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.7375 0.8822 0.2501 0.6616 0.5420 0.3067 0.4663 0.4694 

0.05 0.7578 1.0815 0.4105 0.6376 0.4415 0.2794 0.4470 0.4368 

0.10 0.7807 1.2392 0.5089 0.6085 0.3732 0.2511 0.4363 0.4014 

0.25 0.8387 1.5406 0.6509 0.5378 0.2685 0.2008 0.4137 0.3345 

0.50 0.9052 1.6829 0.7565 0.4098 0.1803 0.1482 0.3416 0.2582 

1.00 0.9715 1.5577 0.8696 0.2030 0.0862 0.0782 0.1849 0.1450 

Table 3. Episode 2 composite statistical summary for base case px_acm2 for the US portion of the 36-km domain is 

shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.05 1.45 0.815 0.938 1.8473 

Mixing Ratio (g/kg) -0.17 1.38 0.749 0.924 1.7851 

Rel. Humidity (%) -0.72 9.44 0.541 0.858 12.4555 

Wind Speed (m/s) 0.41 1.24 0.415 0.776 1.5632 

Clouds (%) 7.27 30.11 0.170 0.660 39.2499 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -1.254 29.29 1.1294 1.2068 1.6529 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.7787 0.8714 0.1843 0.7108 0.6125 0.3855 0.5553 0.5564 

0.05 0.7785 1.0044 0.3065 0.6966 0.5326 0.3524 0.5216 0.5211 

0.10 0.7803 1.0860 0.3857 0.6672 0.4702 0.3176 0.4942 0.4821 

0.25 0.8045 1.2707 0.5345 0.5915 0.3522 0.2503 0.4426 0.4003 

0.50 0.8585 1.4779 0.6821 0.4699 0.2340 0.1784 0.3678 0.3028 

1.00 0.9398 1.7059 0.8631 0.2336 0.0945 0.0774 0.1929 0.1436 

Table 4. Episode 2 composite statistical summary for base case px_acm2 for the VISTAS portion of the 12-km domain is 

shown.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 38

January 18, 2008
Draft

36

Total stats bias abserr r2 ia rmse

Temperature (K) -0.19 1.64 0.862 0.957 2.1492 

Mixing Ratio (g/kg) -0.25 1.49 0.771 0.934 1.9660 

Rel. Humidity (%) -0.22 9.77 0.640 0.893 12.9109 

Wind Speed (m/s) -0.34 1.24 0.490 0.801 1.6120 

Clouds (%) -0.59 26.66 0.179 0.672 36.8433 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 2.458 32.13 1.2014 1.2878 1.7612 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.7199 0.8128 0.2526 0.6075 0.5040 0.2749 0.4265 0.4313 

0.05 0.7517 1.0015 0.4218 0.5791 0.4071 0.2520 0.4028 0.4026 

0.10 0.7820 1.1542 0.5201 0.5539 0.3461 0.2305 0.3959 0.3746 

0.25 0.8447 1.4487 0.6681 0.4808 0.2443 0.1813 0.3679 0.3070 

0.50 0.9114 1.7663 0.7772 0.3936 0.1659 0.1371 0.3292 0.2412 

1.00 0.9687 2.0344 0.9079 0.1874 0.0658 0.0579 0.1654 0.1095 

Table 5. Episode 3 composite statistical summary for base case px_acm2 for the US portion of the 36-km domain is 

shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.18 1.52 0.821 0.933 1.9103 

Mixing Ratio (g/kg) 0.01 1.35 0.488 0.833 1.7674 

Rel. Humidity (%) 1.35 8.58 0.524 0.842 11.1546 

Wind Speed (m/s) 0.13 1.16 0.282 0.699 1.4752 

Clouds (%) 3.91 29.83 0.169 0.664 38.7202 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 1.595 34.59 1.0827 1.1372 1.5702 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.7015 0.8579 0.2708 0.6256 0.5077 0.2512 0.4006 0.4015 

0.05 0.7219 0.9406 0.3852 0.5783 0.4245 0.2378 0.3791 0.3842 

0.10 0.7461 0.9913 0.4560 0.5393 0.3714 0.2241 0.3651 0.3661 

0.25 0.8068 1.0958 0.5801 0.4602 0.2813 0.1924 0.3351 0.3227 

0.50 0.8806 1.2216 0.6944 0.3734 0.2020 0.1569 0.2986 0.2712 

1.00 0.9536 1.5517 0.8494 0.2337 0.1008 0.0872 0.2037 0.1605 

Table 6. Episode 3 composite statistical summary for base case px_acm2 for the VISTAS portion of the 12-km domain is 

shown.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 39

January 18, 2008
Draft

37

To augment the statistical tables described above, we have also produced tables of statistical differences 

from the base case px_acm2. The table setup is precisely the same as above, but the differences reported are not 

simple differences. Instead we present differences designed to indicate whether the sensitivity produces better 

results than the base case. For some metrics (e.g. r2, IA, CSI) larger numbers indicate improved performance. 

For these metrics we calculate differences: SENS – BASE. Positive results are color-coded green, indicating

that the sensitivity improved performance. Negative results are color-coded red, indicating that the sensitivity 

degraded performance. For some other metrics (e.g. abserr, uverr, FAR) smaller numbers indicate improved 

performance. Thus the difference table shows BASE – SENS, and the color-coding described above still 

applies. Similarly the differences for the bias statistics are calculated relative to optimum performance (0 

normally, 1 for precipitation). In this manner one can readily ascertain statistical benefits/disbenefits for all 

sensitivity runs. One should keep in mind that very small non-zero statistical changes are color-coded the same 

as significant changes, so the actual numerical change should be considered more important than the colorcoding.

Tables 7-8 show the statistical difference results for sensitivity px_acm for episode 1. The most significant 

differences are found in the temperature metrics, revealing that the px_acm configuration is significantly more 

cold-biased than is px_acm2 for this wintertime episode. Most of the other surface variables also show degraded 

performance. The px_acm run does seem to very slightly improve cloud coverage and precipitation, but the 

improvements are very slight and are probably less meaningful than the temperature degradation. These 

performance patterns hold for both the US 36-km results and the VISTAS 12-km results. Tables 9-10 indicate 

very little performance difference between the two configurations for episode 2. Precipitation performance is 

slightly enhanced for this summertime period, while interestingly it actually slightly degrades in the other 

summertime episode (tables 11-12). Overall px_acm seems to be a less desirable configuration for annual 

modeling.

Tables 13-14 show the statistical difference results for the noah_mrf sensitivity for episode 1. The 

performance of mixing ratio and relative humidity for both grids seem to be improved slightly in this 

configuration. The remaining variables are degraded almost universally. Perhaps the most striking result seen in 

the episode 2 tables (tables 15-16) is the degraded mixing ratio performance, countermanding the wintertime 

result. The noah_mrf sensitivity is significantly more low-biased for this variable than is the base case px_acm2, 

especially for the 12-km VISTAS domain. At 36-km this seems to help the run actually improve its cloud and 

precipitation performance relative to the base. The 12-km results are more ambiguous for these variables. 

Generally similar results are found for episode 3 (tables 17-18). These findings, combined with the likelihood 

that the PBL heights are overestimated under this configuration, lead one to reject this configuration for annual 

modeling. 

The multi_blkdr results for episode 1 are displayed in tables 19-20. With few exceptions this sensitivity 

shows degraded performance for all variables for both grid resolutions. The low temperature bias is perhaps the 

most significant issue with this configuration. The summertime tables for multi_blkdr (tables 21-24) show 

results that are generally degraded from px_acm2. This configuration clearly degrades performance compared 

with the base case px_acm2. 

The noah_eta-my results are found in tables 25-30. As mentioned earlier this configuration tends to improve 

performance for winds, but the other variables tend to be degraded. Temperature especially is biased low for all 

grids/resolutions. Still, the superior surface wind performance of the noah_eta-my run deserves additional 

attention. To compare the performance of this run versus px_acm(2), we computed composite statistics at 

various sounding locations. These statistics considered only the 00 UTC (or 12 UTC) data in the composite. A 

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 40

January 18, 2008
Draft

38

plot of this type is shown in figure 36 for Greensboro, NC, episode 1, 00 UTC. Note that at the surface the wind 

errors are less in the noah_eta-my case than they are in the px_acm2 case, but for the majority of the lower 

portions of the atmosphere the opposite occurs. The large temperature/dew point biases/errors in the noah_etamy are expected and corroborate what is seen in the surface statistics. Figure 37 shows that for this site similar 

results are found in the wind profile for a summer episode. Performance at other upper air stations vary, and for 

Florida stations the noah_eta-my results appear to be slightly improved over the px_acm results. Generally 

speaking, though, the improved wind performance for noah_eta-my appears to be relegated to the lowest model 

layer. For a multitude of reasons, therefore, this configuration is not recommended for the annual modeling. 

Figure 36. Episode 1 (Jan 2-21, 2002) composite vertical statistics for all 00 UTC times for Greensboro, NC are shown. 

The magnitude of the error vector is plotted in the leftmost panel, followed by temperature bias and error, then dew point 

bias and error. Blue represents the px_acm2 case, while dashed red shows the noah_eta-my results.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 41

January 18, 2008
Draft

39

Figure 37. Episode 2 (Jul 13-28, 2001) composite vertical statistics for all 00 UTC times for Greensboro, NC are shown. 

The magnitude of the error vector is plotted in the leftmost panel, followed by temperature bias and error, then dew point 

bias and error. Blue represents the px_acm2 case, while dashed red shows the noah_eta-my results.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 42

January 18, 2008
Draft

40

Total stats bias abserr r2 ia rmse

Temperature (K) -0.92 -0.60 -0.021 -0.017 -0.6456 

Mixing Ratio (g/kg) -0.01 0.00 -0.002 -0.001 0.0023 

Rel. Humidity (%) 0.26 -0.08 -0.017 -0.006 -0.1239 

Wind Speed (m/s) -0.02 -0.03 -0.022 -0.015 -0.0392 

Clouds (%) -0.23 0.12 0.002 0.001 0.0373 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.424 -0.06 -0.0205-0.0178 -0.0270 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0035 -0.0140 0.0120 -0.0073 0.0028 0.0057 0.0008 0.0054 

0.05 0.0023 0.0235 0.0104 -0.0045 0.0045 0.0057 -0.0010 0.0051 

0.10 0.0014 0.0238 0.0097 -0.0051 0.0039 0.0045 -0.0029 0.0040 

0.25 0.0006 0.0221 0.0103 -0.0049 0.0039 0.0042 -0.0038 0.0034 

0.50 0.0000 -0.0111 0.0030 -0.0056 -0.0016 -0.0015 -0.0054 -0.0013 

1.00 0.0000 -0.0051 -0.0010-0.0041 -0.0027 -0.0028 -0.0040 -0.0028 

Table 7. Episode 1 composite statistical comparison of sensitivity px_acm with base case px_acm2 for the US portion of 

the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -1.05 -0.57 -0.015 -0.019 -0.6223 

Mixing Ratio (g/kg) 0.04 -0.01 -0.006 -0.001 -0.0164 

Rel. Humidity (%) -0.28 -0.39 -0.026 -0.013 -0.5072 

Wind Speed (m/s) 0.00 -0.04 -0.031 -0.019 -0.0431 

Clouds (%) 0.16 0.07 0.001 0.001 0.0341 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.222 -0.24 -0.0291-0.0208 -0.0351 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0059 0.0231 0.0219 -0.0121 0.0072 0.0125 0.0022 0.0103 

0.05 0.0049 0.0370 0.0215 -0.0067 0.0112 0.0145 0.0013 0.0113 

0.10 0.0024 0.0299 0.0160 -0.0073 0.0065 0.0080 -0.0031 0.0061 

0.25 0.0006 -0.0251 0.0118 -0.0095 0.0007 0.0011 -0.0079 0.0008 

0.50 0.0000 -0.0165 0.0041 -0.0084 -0.0030 -0.0030 -0.0079 -0.0025 

1.00 -0.0001 -0.0202 0.0006 -0.0136 -0.0085 -0.0085 -0.0134 -0.0083 

Table 8. Like table 7, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 43

January 18, 2008
Draft

41

Total stats bias abserr r2 ia rmse

Temperature (K) 0.04 0.05 0.004 0.002 0.0393 

Mixing Ratio (g/kg) -0.24 -0.04 0.003 -0.002 -0.0335 

Rel. Humidity (%) -1.56 -0.13 -0.003 -0.002 -0.2838 

Wind Speed (m/s) 0.02 0.00 -0.001 0.000 -0.0004 

Clouds (%) 0.06 -0.07 -0.002 -0.002 -0.0969 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.074 0.07 -0.0015-0.0012 -0.0019 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0026 -0.0124 0.0068 -0.0034 0.0012 0.0043 0.0045 0.0050 

0.05 0.0031 0.0170 0.0063 -0.0034 0.0019 0.0038 0.0025 0.0046 

0.10 0.0032 0.0169 0.0059 -0.0010 0.0030 0.0042 0.0033 0.0053 

0.25 0.0032 0.0281 0.0066 0.0002 0.0040 0.0047 0.0038 0.0064 

0.50 0.0014 0.0371 0.0027 -0.0047 0.0005 0.0008 -0.0030 0.0012 

1.00 0.0004 0.0356 0.0006 -0.0036 -0.0003 -0.0003 -0.0032 -0.0005 

Table 9. Episode 2 composite statistical comparison of sensitivity px_acm with base case px_acm2 for the US portion of 

the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.01 0.05 0.007 0.004 0.0413 

Mixing Ratio (g/kg) -0.20 -0.05 0.003 -0.002 -0.0506 

Rel. Humidity (%) -1.33 -0.13 0.004 -0.001 -0.3367 

Wind Speed (m/s) -0.02 0.00 -0.001 -0.001 -0.0069 

Clouds (%) 0.07 -0.01 0.000 0.000 0.0241 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.092 0.04 -0.0035-0.0038 -0.0051 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0029 -0.0150 0.0090 -0.0046 0.0015 0.0054 0.0055 0.0057 

0.05 0.0030 -0.0130 0.0085 -0.0068 0.0010 0.0039 0.0019 0.0043 

0.10 0.0040 0.0275 0.0093 -0.0071 0.0018 0.0045 0.0014 0.0052 

0.25 0.0049 0.0391 0.0099 -0.0059 0.0035 0.0053 0.0014 0.0069 

0.50 0.0048 0.0489 0.0119 0.0013 0.0067 0.0077 0.0064 0.0109 

1.00 0.0011 0.0331 0.0047 0.0032 0.0027 0.0028 0.0042 0.0049 

Table 10. Like table 9, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 44

January 18, 2008
Draft

42

Total stats bias abserr r2 ia rmse

Temperature (K) -0.14 -0.01 0.002 0.001 0.0048 

Mixing Ratio (g/kg) -0.25 -0.02 0.001 -0.002 -0.0184 

Rel. Humidity (%) -0.99 0.09 0.010 0.003 0.0700 

Wind Speed (m/s) 0.01 0.00 -0.001 -0.001 0.0011 

Clouds (%) 0.18 -0.15 -0.004 -0.002 -0.1736 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.009 0.08 -0.0012-0.0004 -0.0012 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0166 0.0050 -0.0061-0.0013 -0.0037 -0.0216 -0.0237 -0.0270 

0.05 -0.0168 -0.0082 -0.0071-0.0136 -0.0102 -0.0197 -0.0268 -0.0256 

0.10 -0.0126 0.0113 -0.0130-0.0203 -0.0145 -0.0190 -0.0286 -0.0254 

0.25 -0.0067 -0.0385 -0.0312-0.0335 -0.0250 -0.0246 -0.0384 -0.0360 

0.50 0.0003 -0.1315 -0.0330-0.0335 -0.0240 -0.0220 -0.0341 -0.0348 

1.00 0.0027 0.0539 -0.0273-0.0592 -0.0208 -0.0200 -0.0568 -0.0364 

Table 11. Episode 3 composite statistical comparison of sensitivity px_acm with base case px_acm2 for the US portion of 

the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.09 0.06 0.005 0.008 0.0655 

Mixing Ratio (g/kg) -0.22 0.00 -0.007 -0.004 -0.0056 

Rel. Humidity (%) 1.01 0.17 0.022 0.013 0.2309 

Wind Speed (m/s) -0.03 -0.01 -0.004 -0.002 -0.0088 

Clouds (%) -0.11 -0.12 -0.002 -0.001 -0.0867 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.083 0.22 -0.0043-0.0073 -0.0082 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0320 -0.0381 -0.0245-0.0478 -0.0425 -0.0475 -0.0625 -0.0631 

0.05 -0.0218 -0.0299 -0.0461-0.0604 -0.0526 -0.0474 -0.0661 -0.0643 

0.10 -0.0139 -0.0286 -0.0642-0.0774 -0.0636 -0.0535 -0.0771 -0.0746 

0.25 0.0057 -0.0350 -0.0865-0.0832 -0.0664 -0.0530 -0.0770 -0.0781 

0.50 0.0193 -0.0746 -0.0733-0.0723 -0.0511 -0.0399 -0.0600 -0.0617 

1.00 0.0152 0.1611 -0.0075-0.0347 -0.0100 -0.0055 -0.0237 -0.0095 

Table 12. Like table 11, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 45

January 18, 2008
Draft

43

Total stats bias abserr r2 ia rmse

Temperature (K) -0.14 -0.17 -0.012 -0.005 -0.1357 

Mixing Ratio (g/kg) 0.01 0.04 0.009 0.002 0.0382 

Rel. Humidity (%) -0.40 0.22 0.006 0.000 0.3528 

Wind Speed (m/s) -0.13 -0.02 0.001 0.025 -0.0283 

Clouds (%) -2.68 -1.58 -0.042 -0.029 -1.4727 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -2.704 -0.27 -0.0108-0.0156 -0.0187 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0153 -0.0554 -0.0359 0.0020 -0.0213 -0.0302 -0.0207 -0.0298 

0.05 -0.0042 -0.0243 -0.0154-0.0009 -0.0109 -0.0128 -0.0058 -0.0116 

0.10 -0.0015 -0.0225 -0.0093 0.0042 -0.0042 -0.0050 0.0021 -0.0044 

0.25 -0.0027 -0.0400 -0.0299-0.0049 -0.0223 -0.0234 -0.0073 -0.0198 

0.50 -0.0011 -0.0105 -0.0224-0.0155 -0.0217 -0.0220 -0.0161 -0.0195 

1.00 -0.0006 0.0437 -0.0464-0.0162 -0.0290 -0.0293 -0.0167 -0.0303 

Table 13. Episode 1 composite statistical comparison of sensitivity noah_mrf with base case px_acm2 for the US portion 

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.29 -0.08 0.009 0.000 -0.0457 

Mixing Ratio (g/kg) 0.00 0.04 0.007 0.002 0.0375 

Rel. Humidity (%) 1.54 0.98 0.035 0.015 1.1957 

Wind Speed (m/s) 0.18 -0.11 -0.060 -0.005 -0.1359 

Clouds (%) -1.84 -0.72 -0.027 -0.016 -0.5780 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -1.433 -1.71 -0.0155-0.0637 -0.0575 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0325 -0.0745 -0.0657-0.0137 -0.0561 -0.0767 -0.0549 -0.0671 

0.05 -0.0071 -0.0054 -0.0174-0.0144 -0.0221 -0.0258 -0.0196 -0.0207 

0.10 -0.0041 0.0143 -0.0089-0.0203 -0.0198 -0.0217 -0.0214 -0.0168 

0.25 -0.0015 0.0054 -0.0103-0.0054 -0.0107 -0.0114 -0.0066 -0.0087 

0.50 0.0002 -0.0472 0.0152 -0.0214 -0.0055 -0.0053 -0.0201 -0.0044 

1.00 -0.0003 0.0821 -0.0289 0.0315 0.0073 0.0070 0.0309 0.0068 

Table 14. Like table 13, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 46

January 18, 2008
Draft

44

Total stats bias abserr r2 ia rmse

Temperature (K) -0.37 -0.27 -0.021 -0.005 -0.3041 

Mixing Ratio (g/kg) -0.67 -0.05 0.017 -0.005 -0.0018 

Rel. Humidity (%) -2.50 -0.02 0.016 -0.002 0.3326 

Wind Speed (m/s) -0.31 -0.20 -0.104 -0.047 -0.2402 

Clouds (%) 0.83 0.50 0.005 0.001 0.6075 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -1.086 -0.53 -0.0358-0.0394 -0.0531 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0003 -0.0157 0.0042 -0.0082 -0.0034 -0.0009 -0.0015 -0.0010 

0.05 0.0058 0.0206 0.0109 -0.0007 0.0058 0.0085 0.0079 0.0103 

0.10 0.0076 0.0410 0.0145 -0.0027 0.0072 0.0100 0.0077 0.0127 

0.25 0.0150 0.1450 0.0331 -0.0045 0.0179 0.0214 0.0130 0.0292 

0.50 0.0151 0.3081 0.0500 -0.0064 0.0243 0.0272 0.0098 0.0403 

1.00 0.0047 0.2763 0.0575 0.0378 0.0318 0.0324 0.0419 0.0542 

Table 15. Episode 2 composite statistical comparison of sensitivity noah_mrf with base case px_acm2 for the US portion 

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.08 -0.18 -0.026 0.001 -0.2689 

Mixing Ratio (g/kg) -0.90 -0.18 -0.008 -0.024 -0.2066 

Rel. Humidity (%) -4.76 -0.32 0.041 -0.015 -0.0816 

Wind Speed (m/s) 0.29 -0.27 -0.195 -0.084 -0.3222 

Clouds (%) 4.64 0.49 -0.013 -0.007 0.5665 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.317 -1.61 -0.0994-0.0984 -0.1397 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0069 -0.0472 0.0095 -0.0306 -0.0180 -0.0136 -0.0147 -0.0143 

0.05 0.0031 -0.0423 0.0149 -0.0213 -0.0042 0.0013 -0.0043 0.0015 

0.10 0.0049 0.0566 0.0143 -0.0201 -0.0021 0.0026 -0.0049 0.0030 

0.25 0.0085 0.0674 0.0179 -0.0098 0.0065 0.0097 0.0028 0.0124 

0.50 0.0094 0.0540 0.0288 0.0238 0.0217 0.0232 0.0316 0.0327 

1.00 0.0023 -0.0311 0.0312 0.0583 0.0249 0.0248 0.0591 0.0419 

Table 16. Like table 15, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 47

January 18, 2008
Draft

45

Total stats bias abserr r2 ia rmse

Temperature (K) -0.25 -0.13 -0.017 0.000 -0.1497 

Mixing Ratio (g/kg) -0.74 -0.08 0.020 -0.009 -0.0576 

Rel. Humidity (%) -3.06 0.00 0.018 -0.001 0.1712 

Wind Speed (m/s) -0.32 -0.20 -0.084 -0.038 -0.2410 

Clouds (%) -1.72 -0.03 -0.008 -0.007 -0.0666 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.471 -0.27 -0.0195-0.0240 -0.0309 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0017 -0.0352 0.0089 -0.0194 -0.0096 -0.0037 -0.0055 -0.0047 

0.05 0.0086 -0.0425 0.0190 -0.0082 0.0051 0.0103 0.0075 0.0130 

0.10 0.0110 0.0771 0.0229 -0.0123 0.0066 0.0115 0.0048 0.0150 

0.25 0.0154 0.1750 0.0351 -0.0134 0.0145 0.0185 0.0055 0.0261 

0.50 0.0144 0.3705 0.0420 -0.0240 0.0165 0.0193 -0.0077 0.0293 

1.00 0.0073 0.6216 0.0406 0.0000 0.0184 0.0194 0.0074 0.0341 

Table 17. Episode 3 composite statistical comparison of sensitivity noah_mrf with base case px_acm2 for the US portion

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) 0.13 0.01 -0.032 0.008 -0.0685 

Mixing Ratio (g/kg) -0.95 -0.11 0.010 -0.036 -0.1291 

Rel. Humidity (%) -2.84 -0.35 0.028 -0.003 -0.3691 

Wind Speed (m/s) -0.17 -0.30 -0.159 -0.085 -0.3600 

Clouds (%) 2.88 0.55 0.001 -0.004 1.1355 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 1.036 -1.65 -0.0814-0.0266 -0.0759 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0044 -0.0586 0.0225 -0.0248 -0.0064 0.0065 0.0080 0.0083 

0.05 0.0116 -0.0506 0.0250 -0.0089 0.0067 0.0147 0.0141 0.0190 

0.10 0.0112 -0.0575 0.0244 -0.0085 0.0069 0.0129 0.0104 0.0171 

0.25 0.0111 0.0901 0.0257 -0.0171 0.0043 0.0091 -0.0004 0.0128 

0.50 0.0100 0.1478 0.0306 -0.0124 0.0088 0.0117 -0.0004 0.0173 

1.00 0.0047 0.1971 0.0281 0.0083 0.0138 0.0145 0.0130 0.0242 

Table 18. Like table 17, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 48

January 18, 2008
Draft

46

Total stats bias abserr r2 ia rmse

Temperature (K) -0.36 -0.22 -0.023 -0.010 -0.3301 

Mixing Ratio (g/kg) -0.31 -0.14 -0.033 -0.017 -0.1974 

Rel. Humidity (%) -11.03 -4.38 -0.080 -0.147 -5.0237 

Wind Speed (m/s) -0.26 -0.05 -0.002 0.024 -0.0604 

Clouds (%) 2.38 -1.57 -0.042 -0.020 -1.3545 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -1.938 -0.38 -0.0476-0.0389 -0.0611 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0282 -0.1382 0.0682 0.0220 -0.0318 -0.0495 -0.0275 -0.0496 

0.05 -0.0058 -0.0490 0.0231 0.0065 -0.0124 -0.0152 -0.0017 -0.0138 

0.10 -0.0028 -0.0385 0.0168 0.0060 -0.0085 -0.0099 0.0021 -0.0087 

0.25 -0.0012 -0.0291 0.0154 0.0036 -0.0080 -0.0086 0.0023 -0.0072 

0.50 -0.0004 -0.0099 0.0091 -0.0025 -0.0068 -0.0069 -0.0028 -0.0060 

1.00 -0.0001 -0.0284 0.0009 -0.0183 -0.0106 -0.0106 -0.0182 -0.0108 

Table 19. Episode 1 composite statistical comparison of sensitivity multi_blkdr with base case px_acm2 for the US

portion of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.75 -0.38 -0.025 -0.017 -0.5501 

Mixing Ratio (g/kg) -0.30 -0.12 -0.008 -0.007 -0.1244 

Rel. Humidity (%) -11.86 -4.44 -0.072 -0.159 -5.5613 

Wind Speed (m/s) -0.04 -0.09 -0.003 0.019 -0.1052 

Clouds (%) 0.45 -0.03 -0.003 -0.002 -0.0372 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.167 -1.15 -0.0333-0.0397 -0.0518 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0496 -0.1959 0.1099 0.0200 -0.0707 -0.1048 -0.0620 -0.0935 

0.05 -0.0072 -0.0412 0.0260 0.0032 -0.0174 -0.0219 -0.0067 -0.0175 

0.10 -0.0035 -0.0233 0.0168 0.0005 -0.0119 -0.0139 -0.0038 -0.0108 

0.25 -0.0017 0.0177 0.0156 -0.0007 -0.0108 -0.0117 -0.0026 -0.0089 

0.50 -0.0010 0.0061 0.0137 -0.0084 -0.0132 -0.0137 -0.0091 -0.0115 

1.00 -0.0003 0.0150 0.0154 -0.0021 -0.0075 -0.0076 -0.0022 -0.0074 

Table 20. Like table 19, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 49

January 18, 2008
Draft

47

Total stats bias abserr r2 ia rmse

Temperature (K) 0.02 -0.14 -0.019 -0.006 -0.1609 

Mixing Ratio (g/kg) 0.15 -0.02 -0.030 -0.010 -0.0399 

Rel. Humidity (%) -0.65 -0.48 -0.032 -0.021 -0.2737 

Wind Speed (m/s) 0.10 -0.12 -0.073 -0.018 -0.1302 

Clouds (%) 0.04 0.91 0.011 0.005 0.9705 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.841 -0.24 -0.0488-0.0678 -0.0828 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0254 0.0395 0.0400 -0.0073 -0.0257 -0.0413 -0.0487 -0.0499 

0.05 -0.0254 -0.0682 0.0421 -0.0083 -0.0276 -0.0377 -0.0411 -0.0475 

0.10 -0.0229 -0.0816 0.0399 -0.0125 -0.0276 -0.0342 -0.0382 -0.0449 

0.25 -0.0107 -0.0272 0.0284 -0.0350 -0.0250 -0.0263 -0.0426 -0.0373 

0.50 0.0007 0.1146 0.0144 -0.0505 -0.0176 -0.0169 -0.0471 -0.0261 

1.00 0.0023 0.1937 -0.0100-0.0116 0.0019 0.0023 -0.0092 0.0040 

Table 21. Episode 2 composite statistical comparison of sensitivity multi_blkdr with base case px_acm2 for the US 

portion of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.22 -0.02 -0.019 0.002 -0.0627 

Mixing Ratio (g/kg) -0.23 0.05 -0.009 -0.002 0.0662 

Rel. Humidity (%) -1.53 0.68 0.060 0.014 1.1438 

Wind Speed (m/s) -0.11 -0.23 -0.108 -0.044 -0.2627 

Clouds (%) 4.25 0.65 -0.007 -0.004 0.8315 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.872 -0.25 -0.0930-0.1375 -0.1640 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0136 -0.0485 -0.0018-0.0381 -0.0276 -0.0261 -0.0280 -0.0276 

0.05 -0.0051 -0.0353 -0.0016-0.0291 -0.0162 -0.0142 -0.0204 -0.0156 

0.10 -0.0035 0.0398 0.0015 -0.0260 -0.0139 -0.0125 -0.0203 -0.0145 

0.25 -0.0005 0.0463 0.0024 -0.0245 -0.0101 -0.0089 -0.0196 -0.0114 

0.50 0.0020 0.0806 0.0027 -0.0294 -0.0089 -0.0078 -0.0242 -0.0113 

1.00 0.0059 0.2326 -0.0171-0.0067 0.0065 0.0074 -0.0005 0.0128 

Table 22. Like table 21, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 50

January 18, 2008
Draft

48

Total stats bias abserr r2 ia rmse

Temperature (K) 0.16 -0.03 -0.011 -0.001 -0.0470 

Mixing Ratio (g/kg) -0.12 0.01 -0.016 -0.008 0.0215 

Rel. Humidity (%) -0.31 -0.16 -0.026 -0.015 -0.0298 

Wind Speed (m/s) 0.10 -0.13 -0.081 -0.017 -0.1462 

Clouds (%) -2.60 0.38 0.001 -0.002 0.2603 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -1.000 -0.37 -0.0601-0.0617 -0.0861 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0193 0.0011 0.0256 -0.0200 -0.0250 -0.0310 -0.0386 -0.0391 

0.05 -0.0119 -0.0047 0.0197 -0.0240 -0.0213 -0.0230 -0.0308 -0.0299 

0.10 -0.0080 0.0283 0.0177 -0.0335 -0.0220 -0.0220 -0.0348 -0.0296 

0.25 0.0034 0.1115 0.0018 -0.0394 -0.0115 -0.0095 -0.0310 -0.0138 

0.50 0.0079 0.2509 -0.0122-0.0374 -0.0009 0.0008 -0.0274 0.0011 

1.00 0.0052 0.5231 -0.0052-0.0403 -0.0036 -0.0028 -0.0345 -0.0050 

Table 23. Episode 3 composite statistical comparison of sensitivity multi_blkdr with base case px_acm2 for the US

portion of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.14 0.07 -0.009 0.007 0.0185 

Mixing Ratio (g/kg) -0.40 -0.02 -0.004 -0.006 -0.0280 

Rel. Humidity (%) -0.41 -0.12 0.014 0.008 -0.0297 

Wind Speed (m/s) 0.02 -0.22 -0.119 -0.050 -0.2555 

Clouds (%) 2.84 0.87 0.013 0.006 1.2969 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.528 0.09 -0.0884-0.0628 -0.1066 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0069 -0.1108 -0.0247-0.0624 -0.0319 -0.0115 -0.0155 -0.0148 

0.05 0.0041 -0.1003 -0.0205-0.0445 -0.0159 -0.0040 -0.0137 -0.0052 

0.10 0.0012 -0.0871 -0.0066-0.0415 -0.0174 -0.0108 -0.0239 -0.0145 

0.25 0.0001 0.0839 0.0063 -0.0417 -0.0187 -0.0157 -0.0333 -0.0223 

0.50 0.0033 0.1390 0.0062 -0.0492 -0.0176 -0.0153 -0.0412 -0.0232 

1.00 0.0033 0.2018 -0.0040-0.0250 -0.0033 -0.0026 -0.0210 -0.0045 

Table 24. Like table 23, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 51

January 18, 2008
Draft

49

Total stats bias abserr r2 ia rmse

Temperature (K) -1.30 -0.90 -0.046 -0.030 -1.1408 

Mixing Ratio (g/kg) -0.06 -0.08 -0.022 -0.007 -0.0803 

Rel. Humidity (%) -10.88 -4.71 -0.077 -0.125 -5.2197 

Wind Speed (m/s) -0.06 0.03 0.026 0.028 0.0442 

Clouds (%) 1.19 -0.98 -0.016 -0.011 -0.2897 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.668 0.35 0.0371 0.0318 0.0487 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0434 -0.2155 -0.0990 0.0351 -0.0465 -0.0732 -0.0418 -0.0747 

0.05 -0.0015 -0.0123 -0.0063 0.0013 -0.0036 -0.0043 -0.0008 -0.0039 

0.10 0.0003 0.0009 0.0015 0.0011 0.0017 0.0017 0.0013 0.0015 

0.25 -0.0001 0.0093 0.0016 -0.0050 -0.0019 -0.0018 -0.0047 -0.0016 

0.50 -0.0002 -0.0047 -0.0015-0.0056 -0.0041 -0.0042 -0.0056 -0.0036 

1.00 -0.0001 -0.0142 -0.0051-0.0132 -0.0095 -0.0096 -0.0132 -0.0098 

Table 25. Episode 1 composite statistical comparison of sensitivity noah_eta-my with base case px_acm2 for the US 

portion of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -1.08 -0.56 -0.012 -0.018 -0.7253 

Mixing Ratio (g/kg) -0.07 -0.06 -0.010 -0.003 -0.0629 

Rel. Humidity (%) -8.65 -2.57 -0.041 -0.092 -3.3834 

Wind Speed (m/s) 0.29 0.09 0.050 0.042 0.1122 

Clouds (%) -0.05 0.00 0.001 0.001 -0.0017 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.568 -0.15 0.0837 0.0833 0.1179 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0759 -0.3018 -0.1553 0.0315 -0.1011 -0.1512 -0.0944 -0.1393 

0.05 -0.0080 -0.0418 -0.0277 0.0018 -0.0196 -0.0244 -0.0087 -0.0195 

0.10 -0.0023 -0.0065 -0.0088-0.0040 -0.0090 -0.0103 -0.0060 -0.0079 

0.25 -0.0009 0.0020 -0.0058-0.0039 -0.0067 -0.0071 -0.0045 -0.0054 

0.50 -0.0007 0.0135 -0.0115-0.0010 -0.0073 -0.0077 -0.0017 -0.0064 

1.00 -0.0002 0.0539 -0.0179 0.0221 0.0065 0.0063 0.0217 0.0061 

Table 26. Like table 25, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 52

January 18, 2008
Draft

50

Total stats bias abserr r2 ia rmse

Temperature (K) -1.15 -0.71 -0.043 -0.022 -0.8789 

Mixing Ratio (g/kg) 0.00 0.05 0.009 0.003 0.0895 

Rel. Humidity (%) -5.67 -0.74 0.032 -0.008 -0.5429 

Wind Speed (m/s) -0.24 0.00 0.026 0.002 -0.0108 

Clouds (%) -0.54 0.45 -0.006 -0.008 0.2731 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 1.138 0.31 0.0335 0.0337 0.0475 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0176 0.1064 -0.0459 0.0344 -0.0036 -0.0269 -0.0293 -0.0321 

0.05 -0.0149 -0.1029 -0.0288 0.0265 -0.0046 -0.0146 -0.0062 -0.0180 

0.10 -0.0181 -0.1142 -0.0304 0.0150 -0.0128 -0.0197 -0.0122 -0.0255 

0.25 -0.0085 -0.0389 -0.0208-0.0193 -0.0169 -0.0182 -0.0264 -0.0256 

0.50 0.0045 0.1814 -0.0016-0.0465 -0.0104 -0.0091 -0.0393 -0.0139 

1.00 0.0040 0.3590 0.0130 -0.0311 -0.0014 -0.0006 -0.0268 -0.0010 

Table 27. Episode 2 composite statistical comparison of sensitivity noah_eta-my with base case px_acm2 for the US 

portion of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.81 -0.45 -0.025 -0.014 -0.6594 

Mixing Ratio (g/kg) 0.04 0.05 -0.001 0.003 0.0776 

Rel. Humidity (%) -1.75 1.04 0.105 0.029 1.7577 

Wind Speed (m/s) 0.29 0.09 0.023 0.019 0.0956 

Clouds (%) 4.90 0.36 -0.013 -0.007 0.4730 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -1.641 0.26 0.0688 0.0883 0.1115 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0024 0.0915 -0.0481 0.0853 0.0290 0.0056 0.0073 0.0059 

0.05 -0.0126 -0.1823 -0.0444 0.0737 0.0113 -0.0074 0.0121 -0.0081 

0.10 -0.0169 -0.1763 -0.0389 0.0592 0.0027 -0.0113 0.0109 -0.0131 

0.25 -0.0111 -0.0996 -0.0201 0.0189 -0.0054 -0.0098 0.0012 -0.0125 

0.50 0.0045 0.1230 0.0012 -0.0375 -0.0091 -0.0072 -0.0288 -0.0105 

1.00 0.0101 0.4790 0.0112 -0.0519 -0.0057 -0.0035 -0.0401 -0.0060 

Table 28. Like table 27, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 53

January 18, 2008
Draft

51

Total stats bias abserr r2 ia rmse

Temperature (K) -1.08 -0.65 -0.052 -0.022 -0.7857 

Mixing Ratio (g/kg) 0.24 0.07 0.013 0.006 0.1111 

Rel. Humidity (%) -5.41 -0.81 0.018 -0.011 -0.8111 

Wind Speed (m/s) -0.23 -0.02 0.032 0.000 -0.0192 

Clouds (%) -3.24 0.25 -0.004 -0.005 0.0162 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 1.051 0.49 0.0283 0.0293 0.0407 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0211 0.1203 -0.0560 0.0376 -0.0031 -0.0305 -0.0353 -0.0385 

0.05 -0.0115 -0.0834 -0.0240 0.0222 -0.0018 -0.0099 -0.0033 -0.0128 

0.10 -0.0108 -0.0680 -0.0201 0.0081 -0.0076 -0.0118 -0.0076 -0.0157 

0.25 -0.0009 0.0351 -0.0070-0.0215 -0.0093 -0.0088 -0.0200 -0.0128 

0.50 0.0058 0.2353 -0.0001-0.0527 -0.0102 -0.0087 -0.0441 -0.0137 

1.00 0.0061 0.5801 0.0139 -0.0332 0.0012 0.0022 -0.0267 0.0038 

Table 29. Episode 3 composite statistical comparison of sensitivity noah_eta-my with base case px_acm2 for

the US portion of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.52 -0.27 -0.052 -0.005 -0.4031 

Mixing Ratio (g/kg) -0.12 0.00 -0.003 0.000 -0.0028 

Rel. Humidity (%) -1.43 -0.02 0.022 0.006 -0.1437 

Wind Speed (m/s) -0.03 0.03 0.033 0.014 0.0470 

Clouds (%) 0.01 1.11 0.020 0.010 1.3794 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.358 0.67 0.0526 0.0891 0.1007 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0031 0.0764 -0.0417 0.1071 0.0419 0.0064 0.0095 0.0082 

0.05 -0.0133 -0.1067 -0.0382 0.0940 0.0256 0.0017 0.0218 0.0022 

0.10 -0.0267 -0.1942 -0.0475 0.0579 0.0005 -0.0175 -0.0013 -0.0237 

0.25 -0.0168 -0.0750 -0.0384-0.0136 -0.0223 -0.0264 -0.0309 -0.0380 

0.50 0.0068 0.1725 0.0080 -0.0444 -0.0108 -0.0077 -0.0331 -0.0115 

1.00 0.0093 0.5063 0.0305 -0.0444 0.0012 0.0034 -0.0339 0.0056 

Table 30. Like table 29, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 54

January 18, 2008
Draft

52

5 Auxiliary Results

The results from the five main sensitivity runs indicate that the px_acm2 configuration is overall the 

preferred option for annual modeling. Still there are numerous model options that are not tested in the above 

configurations. Accordingly we have conducted a variety of auxiliary sensitivity tests with limited analyses. 

These include:

• px_acm3: Like px_acm2, except with soil moisture/temperature initialization for P-X soil 

layer 2 from EDAS soil layer 3 (40-100 cm).

• px_acm4: Like px_acm2, except it uses the SFCFDDA file from RAWINS and 

MMINPUT files processed using RAWINS data rather than REGRID data.

• px_acm5: Like px_acm2, except it uses Reisner 2 microphysics rather than Reisner 1. 

• px_acm6: Like px_acm2, except analysis nudging strengths aloft for wind/temps are 

increased from 2.5E-4, 1.0E-4 (36/12 km) to 3.0E-4, 3.0E-4.

• px_acm7: Like px_acm2, except the SFCFDDA files from RAWINS are used instead of 

those from LITTLE_R.

• px_acm8: Like px_acm2, except the MMINPUT files are processed using data from 

LITTLE_R rather than from REGRID.

• px_acm9: Like px_acm3, except the initialization for soil temperature/moisture is based 

on a 25% weighting of EDAS soil layer 1 (0-10 cm) values and a 75% 

weighting from EDAS soil layer 3 (40-100 cm). Code supplied by Aijun Xiu; 

executed for episode 3 only. 

• px_acm_n: Like px_acm2, except surface analysis nudging is turned on for temp/moisture 

at a strength of 2.5E-4, 1.0E-4 (36, 12 km temps) and 1.0E-4, 1.0E-4 (36, 12 

km moisture).

• px_acm_n2: Like px_acm_n, except surface analysis nudging for moisture ss reduced to 

1.0E-5, 1.0E-5 (36, 12 km). Run only for episode 2.

• s0px_acm2: Like px_acm2, except snow cover effects are not considered (IFSNOW=0) as 

opposed to IFSNOW=1 in px_acm2. Run only for episode 1.

• s2px_acm2: Like px_acm2, except the simple snow model is employed (IFSNOW=2) as 

opposed to IFSNOW=1 in px_acm2. Run only for episode 1.

• n0_px_acm2: Like px_acm2, except soil moisture nudging is turned off. 

For the most part the analyses of these runs will be statistical and will follow the statistical table format 

detailed in the prior section. The goal is to quickly determine whether the option being tested proves beneficial. 

The analyses are limited to the wintertime episode 1 and the longer of the two summertime episodes, episode 2. 

Recall that the px_acm2 run, while performing the best of the original five sensitivities, failed to meet the 

statistical “benchmark” for temperature for episode 1. The px_acm3 configuration is an attempt to rectify this 

bias by initializing the model from the single most appropriate EDAS soil layer (40-100 cm). This run included 

an error correction pass that reduced the effect of “missing” values in the EDAS data set. Tables 31-32 show the

results of this sensitivity for episode 1 for both grid resolutions modeled. Note the vast improvement in 

temperature bias in both tables. Most of the other variables are relatively unaffected.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 55

January 18, 2008
Draft

53

Total stats bias abserr r2 ia rmse

Temperature (K) 0.65 0.17 -0.002 0.003 0.2029 

Mixing Ratio (g/kg) 0.05 0.00 -0.002 0.000 -0.0035 

Rel. Humidity (%) 2.53 0.18 0.025 0.015 0.1032 

Wind Speed (m/s) -0.03 0.02 0.010 0.009 0.0192 

Clouds (%) -0.02 -0.01 0.001 0.000 0.0497 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.643 -0.02 0.0072 0.0026 0.0070 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0028 -0.0257 -0.0101 0.0082 -0.0014 -0.0038 0.0007 -0.0037 

0.05 -0.0016 -0.0193 -0.0075 0.0047 -0.0026 -0.0034 0.0019 -0.0031 

0.10 -0.0012 -0.0252 -0.0086 0.0069 -0.0023 -0.0030 0.0048 -0.0026 

0.25 -0.0007 -0.0199 -0.0092 0.0039 -0.0038 -0.0041 0.0031 -0.0035 

0.50 -0.0002 -0.0251 -0.0083 0.0086 0.0000 -0.0001 0.0082 -0.0001 

1.00 0.0000 0.0152 -0.0014 0.0081 0.0039 0.0038 0.0081 0.0039 

Table 31. Episode 1 composite statistical comparison of sensitivity px_acm3 with base case px_acm2 for the US portion 

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) 0.96 0.13 -0.004 0.002 0.1753 

Mixing Ratio (g/kg) 0.03 0.00 0.003 0.001 0.0160 

Rel. Humidity (%) 3.05 0.39 0.043 0.025 0.2853 

Wind Speed (m/s) -0.09 0.00 0.022 0.011 0.0014 

Clouds (%) 0.19 -0.09 -0.002 -0.001 -0.0258 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.913 0.13 0.0089 -0.0115 -0.0025 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0028 -0.0285 -0.0119 0.0100 -0.0021 -0.0049 0.0013 -0.0041 

0.05 -0.0013 -0.0182 -0.0076 0.0059 -0.0019 -0.0029 0.0027 -0.0023 

0.10 -0.0013 -0.0242 -0.0105 0.0079 -0.0024 -0.0034 0.0049 -0.0026 

0.25 -0.0009 0.0187 -0.0113 0.0043 -0.0044 -0.0050 0.0029 -0.0038 

0.50 -0.0004 0.0186 -0.0093 0.0049 -0.0022 -0.0026 0.0043 -0.0021 

1.00 -0.0002 0.0146 -0.0105 0.0016 -0.0032 -0.0032 0.0015 -0.0031 

Table 32. Like table 31, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 56

January 18, 2008
Draft

54

Unfortunately the improvements noted in the wintertime episode do not translate to the summer episode 

(tables 33-34). In fact the results indicate a degraded bias result, with the temperatures more significantly coldbiased for both resolutions modeled. The remaining variables tend to be relatively unaffected, generally 

showing a very slight degradation in performance. If this sensitivity had produced superior results for all 

episodes, we would recommend further efforts to possibly improve the initialization of the P-X module in 

MM5. Given the ad hoc nature of this sensitivity test that only yields mixed-bag results at best, we do not 

recommend applying this approach for the annual modeling.

An alternative approach to using LITTLE_R in the MM5 preprocessing stream is to use RAWINS. 

Functionally they serve the same purpose, which is to use observations to “improve” the first-guess and analysis 

fields that drive MM5. NCAR recommends that users employ LITTLE_R, which is a newer, more flexible 

program, but also somewhat more difficult to implement. (Unfortunately when this project started the 

documentation for LITTLE_R was less than stellar. The updated documentation indicates that upper air data 

“probably should” be added to the sfc_obs files; we plan to do just that for the annual modeling, but these tests 

will not have that “benefit”.) Prior modeling results found little difference between using these programs and 

not using them at all. To use P-X in the recommended manner (i.e. with indirect soil nudging) requires that the 

SFCFDDA file be available, which necessitates running one of those objective analysis preprocessors. The pxacm2 configuration used LITTLE_R only for that purpose. The px_acm4 configuration is actually a two-fold 

sensitivity in that it tests the effects of 1) running RAWINS rather than LITTLE_R to generate the SFCFDDA 

files, and 2) using objective analysis (RAWINS, in this case) output as input to INTERPF, as opposed to using 

REGRID output. Tables 35-36 show the results for the wintertime episode. In general we see improvement for 

most of the variables, especially for the VISTAS 12-km domain. Precipitation results are very slightly degraded, 

but not significantly so. The summertime results for this sensitivity are shown in tables 37-38, and they indicate 

no clear overall improvement/degradation.

The Reisner 2 microphysics package is sophisticated and computationally expensive, but is believed to 

incorporate the appropriate science necessary to properly model precipitation over a variety of conditions. 

Tables 39-40 show the statistical differences for the wintertime episode. Interestingly enough this px_acm5 

sensitivity yields improved temperature and cloud coverage performance, but the precipitation skill scores show 

a very slight degradation. Summertime performance for this sensitivity (tables 41-42) show generally degraded 

performance. These results do not justify the increased computational/storage cost of implementing Reisner 2 

microphysics.

Sensitivity test px_acm6 is designed to show the effects of nudging strength. The very slight increase in 

nudging strength at 36-km is not expected to have a large effect on the results, and the episode 1 statistical 

difference table (table 43) bears this out. The 12-km results (table 44) are a little more interesting since the 

nudging strength aloft is increased more dramatically. The idea is that above the planetary boundary layer 

synoptic conditions prevail, thus making analysis nudging at a reasonably strong level appropriate. In the 

boundary layer itself the wind nudging is kept at a relatively weak level. We do note an improvement in the 

temperature and precipitation metrics, but a slight degradation in mixing ratio and winds. The summer case 

expectedly reveals little change at 36-km resolution (table 45), while the 12-km results (table 46) show 

decreased skill for almost all variables, with temperature and winds being the notable exceptions. Overall it is 

hard to justify this configuration as an improvement over the base case.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 57

January 18, 2008
Draft

55

Total stats bias abserr r2 ia rmse

Temperature (K) -0.62 -0.23 -0.018 -0.007 -0.2704 

Mixing Ratio (g/kg) -0.25 -0.03 -0.006 -0.004 -0.0304 

Rel. Humidity (%) -0.09 -0.18 -0.009 -0.003 -0.2014 

Wind Speed (m/s) 0.00 -0.01 -0.008 -0.005 -0.0114 

Clouds (%) 0.15 0.06 0.001 0.001 -0.0039 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.147 0.07 -0.0061-0.0036 -0.0068 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0068 -0.0401 0.0033 -0.0273 -0.0168 -0.0126 -0.0159 -0.0149 

0.05 0.0005 0.0599 0.0060 -0.0292 -0.0110 -0.0069 -0.0160 -0.0085 

0.10 0.0043 0.0794 0.0079 -0.0297 -0.0071 -0.0034 -0.0162 -0.0044 

0.25 0.0075 0.1285 0.0101 -0.0305 -0.0022 0.0003 -0.0184 0.0004 

0.50 0.0061 0.1842 0.0082 -0.0326 -0.0025 -0.0010 -0.0245 -0.0016 

1.00 0.0031 0.2807 0.0104 -0.0232 -0.0004 0.0001 -0.0198 0.0003 

Table 33. Episode 2 composite statistical comparison of sensitivity px_acm3 with base case px_acm2 for the US portion 

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.37 -0.05 -0.006 -0.001 -0.0514 

Mixing Ratio (g/kg) -0.08 0.01 0.001 0.001 0.0144 

Rel. Humidity (%) 0.60 -0.11 -0.005 -0.002 -0.0932 

Wind Speed (m/s) 0.01 0.00 -0.011 -0.006 -0.0085 

Clouds (%) -0.25 0.09 0.003 0.003 0.0358 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.273 -0.07 -0.0054-0.0047 -0.0071 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0050 -0.0409 0.0094 -0.0255 -0.0141 -0.0099 -0.0107 -0.0103 

0.05 0.0019 -0.0455 0.0138 -0.0246 -0.0068 -0.0014 -0.0076 -0.0015 

0.10 0.0050 0.0672 0.0159 -0.0252 -0.0039 0.0015 -0.0077 0.0017 

0.25 0.0080 0.0981 0.0162 -0.0267 -0.0009 0.0031 -0.0110 0.0040 

0.50 0.0071 0.1388 0.0097 -0.0312 -0.0031 -0.0008 -0.0203 -0.0012 

1.00 0.0051 0.2605 -0.0003-0.0361 -0.0067 -0.0056 -0.0299 -0.0096 

Table 34. Like table 33, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 58

January 18, 2008
Draft

56

Total stats bias abserr r2 ia rmse

Temperature (K) 0.08 0.07 0.004 0.002 0.0950 

Mixing Ratio (g/kg) -0.02 0.01 0.003 0.000 0.0093 

Rel. Humidity (%) -0.18 0.09 0.009 0.003 0.1488 

Wind Speed (m/s) 0.02 -0.03 -0.024 -0.010 -0.0397 

Clouds (%) -0.22 -0.15 -0.006 -0.003 -0.2251 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.255 -0.62 -0.0339-0.0300 -0.0452 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0033 -0.0153 -0.0087 0.0022 -0.0041 -0.0062 -0.0035 -0.0061 

0.05 -0.0010 -0.0074 -0.0042 0.0004 -0.0026 -0.0031 -0.0009 -0.0028 

0.10 -0.0010 -0.0073 -0.0050-0.0009 -0.0039 -0.0044 -0.0018 -0.0038 

0.25 -0.0001 0.0203 0.0032 -0.0110 -0.0043 -0.0042 -0.0104 -0.0036 

0.50 0.0000 -0.0346 0.0079 -0.0170 -0.0055 -0.0054 -0.0165 -0.0048 

1.00 0.0001 -0.0264 0.0130 -0.0051 0.0028 0.0028 -0.0049 0.0028 

Table 35. Episode 1 composite statistical comparison of sensitivity px_acm4 with base case px_acm2 for the US portion 

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.05 0.05 0.009 0.002 0.0709 

Mixing Ratio (g/kg) 0.03 0.01 0.002 0.001 0.0147 

Rel. Humidity (%) 0.12 0.11 0.008 0.003 0.1409 

Wind Speed (m/s) -0.01 0.00 0.003 0.001 0.0005 

Clouds (%) 0.34 0.25 0.006 0.003 0.2262 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.039 0.04 -0.0010 0.0036 0.0019 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0035 -0.0029 -0.0061-0.0040 -0.0071 -0.0092 -0.0072 -0.0077 

0.05 -0.0002 0.0202 0.0049 -0.0106 -0.0034 -0.0032 -0.0081 -0.0025 

0.10 -0.0019 0.0205 -0.0001-0.0162 -0.0110 -0.0117 -0.0155 -0.0091 

0.25 -0.0018 -0.0322 0.0004 -0.0259 -0.0186 -0.0192 -0.0253 -0.0147 

0.50 0.0001 -0.0513 0.0145 -0.0250 -0.0083 -0.0081 -0.0237 -0.0068 

1.00 -0.0004 0.0103 -0.0209-0.0097 -0.0145 -0.0146 -0.0098 -0.0143 

Table 36. Like table 35, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 59

January 18, 2008
Draft

57

Total stats bias abserr r2 ia rmse

Temperature (K) -0.10 0.01 0.005 0.001 0.0265 

Mixing Ratio (g/kg) 0.02 0.01 0.003 0.001 0.0200 

Rel. Humidity (%) 0.18 0.22 0.016 0.005 0.3339 

Wind Speed (m/s) -0.01 -0.03 -0.030 -0.016 -0.0438 

Clouds (%) 0.05 -0.13 -0.004 -0.003 -0.1747 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.087 -0.91 -0.0344-0.0341 -0.0484 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0027 -0.0096 0.0060 -0.0020 0.0018 0.0044 0.0048 0.0051 

0.05 0.0030 0.0180 0.0063 -0.0040 0.0016 0.0036 0.0021 0.0043 

0.10 0.0002 0.0281 0.0000 -0.0137 -0.0052 -0.0041 -0.0098 -0.0053 

0.25 0.0033 0.0683 0.0031 -0.0193 -0.0031 -0.0019 -0.0132 -0.0026 

0.50 0.0018 0.0780 -0.0014-0.0213 -0.0050 -0.0044 -0.0182 -0.0068 

1.00 0.0012 0.0465 0.0179 0.0211 0.0118 0.0118 0.0219 0.0202 

Table 37. Episode 2 composite statistical comparison of sensitivity px_acm4 with base case px_acm2 for the US portion 

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.18 0.01 0.003 0.002 0.0145 

Mixing Ratio (g/kg) -0.06 0.02 0.004 0.002 0.0246 

Rel. Humidity (%) 0.37 0.14 0.009 0.003 0.2255 

Wind Speed (m/s) 0.01 0.00 -0.008 -0.005 -0.0036 

Clouds (%) 0.50 0.01 -0.001 -0.001 0.0901 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.021 -0.16 -0.0117-0.0020 -0.0094 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0002 -0.0212 0.0081 -0.0104 -0.0033 0.0002 0.0000 0.0003 

0.05 0.0014 -0.0212 0.0079 -0.0131 -0.0031 -0.0001 -0.0034 0.0000 

0.10 0.0010 0.0379 0.0057 -0.0174 -0.0055 -0.0030 -0.0088 -0.0035 

0.25 0.0060 0.0729 0.0120 -0.0196 -0.0005 0.0025 -0.0079 0.0033 

0.50 0.0076 0.1079 0.0156 -0.0131 0.0048 0.0067 -0.0034 0.0096 

1.00 0.0050 0.1819 0.0174 0.0015 0.0082 0.0090 0.0066 0.0154 

Table 38. Like table 37, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 60

January 18, 2008
Draft

58

Total stats bias abserr r2 ia rmse

Temperature (K) 0.12 0.07 0.003 0.002 0.0972 

Mixing Ratio (g/kg) -0.01 0.00 0.000 0.000 0.0004 

Rel. Humidity (%) -0.01 -0.06 -0.001 0.000 -0.0488 

Wind Speed (m/s) 0.00 0.01 0.001 0.001 0.0018 

Clouds (%) 2.28 0.95 0.024 0.017 0.6766 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.046 -0.01 0.0017 0.0008 0.0018 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0031 -0.0799 0.0283 -0.0408 -0.0094 -0.0030 -0.0190 -0.0029 

0.05 0.0033 0.0860 0.0247 -0.0329 -0.0025 0.0001 -0.0228 0.0001 

0.10 0.0018 0.0700 0.0190 -0.0265 -0.0017 -0.0006 -0.0213 -0.0005 

0.25 0.0007 0.0505 0.0169 -0.0186 -0.0002 0.0002 -0.0168 0.0001 

0.50 -0.0003 -0.0442 0.0046 -0.0269 -0.0133 -0.0132 -0.0264 -0.0117 

1.00 -0.0001 -0.0325 -0.0015-0.0213 -0.0126 -0.0126 -0.0212 -0.0129 

Table 39. Episode 1 composite statistical comparison of sensitivity px_acm5 with base case px_acm2 for the US portion 

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) 0.07 0.05 0.003 0.001 0.0578 

Mixing Ratio (g/kg) -0.01 0.00 0.000 0.000 -0.0024 

Rel. Humidity (%) 0.01 -0.02 0.006 0.002 -0.0119 

Wind Speed (m/s) 0.00 0.00 0.001 0.000 0.0018 

Clouds (%) -1.42 0.26 0.011 0.005 0.1818 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.055 -0.02 0.0006 0.0017 0.0016 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0059 -0.0605 0.0509 -0.0538 -0.0044 0.0049 -0.0200 0.0040 

0.05 0.0037 0.0531 0.0305 -0.0292 0.0019 0.0053 -0.0169 0.0041 

0.10 0.0014 0.0130 0.0232 -0.0271 -0.0025 -0.0010 -0.0204 -0.0008 

0.25 -0.0006 -0.0400 0.0103 -0.0232 -0.0104 -0.0104 -0.0215 -0.0079 

0.50 0.0001 -0.0454 0.0133 -0.0217 -0.0067 -0.0065 -0.0205 -0.0055 

1.00 0.0001 -0.0020 0.0067 0.0038 0.0053 0.0053 0.0040 0.0052 

Table 40. Like table 39, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 61

January 18, 2008
Draft

59

Total stats bias abserr r2 ia rmse

Temperature (K) 0.07 -0.03 -0.002 -0.001 -0.0285 

Mixing Ratio (g/kg) 0.01 -0.02 -0.004 -0.002 -0.0248 

Rel. Humidity (%) -0.20 -0.12 -0.009 -0.003 -0.1517 

Wind Speed (m/s) -0.02 0.00 -0.001 -0.002 -0.0034 

Clouds (%) -0.30 -0.11 -0.001 0.000 -0.2593 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.122 -0.02 0.0000 -0.0005 -0.0003 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0066 -0.0324 0.0013 -0.0232 -0.0150 -0.0121 -0.0151 -0.0143 

0.05 -0.0023 0.0426 0.0000 -0.0252 -0.0122 -0.0099 -0.0177 -0.0122 

0.10 0.0006 0.0525 0.0006 -0.0250 -0.0092 -0.0072 -0.0175 -0.0092 

0.25 0.0046 0.1108 0.0029 -0.0344 -0.0072 -0.0052 -0.0249 -0.0074 

0.50 0.0060 0.1887 0.0071 -0.0354 -0.0037 -0.0022 -0.0272 -0.0034 

1.00 0.0024 0.2219 0.0059 -0.0210 -0.0017 -0.0013 -0.0184 -0.0021 

Table 41. Episode 2 composite statistical comparison of sensitivity px_acm5 with base case px_acm2 for the US portion 

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) 0.04 0.00 0.001 -0.001 -0.0038 

Mixing Ratio (g/kg) 0.00 -0.02 -0.003 -0.001 -0.0145 

Rel. Humidity (%) -0.21 -0.06 -0.006 -0.003 -0.0767 

Wind Speed (m/s) 0.01 0.00 -0.004 -0.001 -0.0015 

Clouds (%) -0.90 -0.13 0.001 0.001 -0.3080 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) -0.189 -0.06 -0.0011-0.0002 -0.0008 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0087 -0.0369 0.0034 -0.0273 -0.0186 -0.0169 -0.0180 -0.0177 

0.05 -0.0034 -0.0169 0.0002 -0.0176 -0.0102 -0.0093 -0.0130 -0.0101 

0.10 -0.0022 0.0294 -0.0004-0.0185 -0.0095 -0.0084 -0.0140 -0.0097 

0.25 0.0020 0.0616 0.0029 -0.0252 -0.0075 -0.0055 -0.0172 -0.0070 

0.50 0.0073 0.1220 0.0128 -0.0214 0.0011 0.0032 -0.0112 0.0046 

1.00 0.0038 0.1704 0.0055 -0.0149 -0.0001 0.0006 -0.0106 0.0011 

Table 42. Like table 41, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 62

January 18, 2008
Draft

60

Total stats bias abserr r2 ia rmse

Temperature (K) 0.01 0.00 0.000 0.000 0.0053 

Mixing Ratio (g/kg) 0.00 0.00 0.000 0.000 -0.0009 

Rel. Humidity (%) 0.07 0.07 0.003 0.002 0.0799 

Wind Speed (m/s) -0.02 0.00 -0.002 -0.002 -0.0041 

Clouds (%) -0.03 0.02 0.001 0.000 0.0167 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.145 -0.04 -0.0006-0.0011 -0.0011 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0001 0.0034 0.0010 -0.0014 -0.0003 0.0000 -0.0006 0.0000 

0.05 0.0003 0.0041 0.0013 -0.0012 0.0002 0.0004 -0.0007 0.0003 

0.10 0.0004 0.0025 0.0023 0.0009 0.0021 0.0021 0.0013 0.0019 

0.25 0.0002 0.0000 0.0018 0.0019 0.0023 0.0024 0.0020 0.0019 

0.50 -0.0001 -0.0105 -0.0025 0.0046 0.0012 0.0011 0.0045 0.0010 

1.00 0.0000 0.0031 -0.0021 0.0000 -0.0009 -0.0010 0.0000 -0.0010 

Table 43. Episode 1 composite statistical comparison of sensitivity px_acm6 with base case px_acm2 for the US portion 

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) 0.16 0.11 0.008 0.004 0.1259 

Mixing Ratio (g/kg) -0.03 -0.03 -0.010 -0.003 -0.0530 

Rel. Humidity (%) 0.33 0.38 0.023 0.011 0.4391 

Wind Speed (m/s) 0.11 -0.01 -0.026 -0.017 -0.0069 

Clouds (%) 0.28 -0.19 -0.006 -0.003 -0.1165 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 1.201 -0.22 -0.0119 0.0096 -0.0012 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0051 0.0104 0.0108 0.0028 0.0097 0.0132 0.0090 0.0108 

0.05 0.0041 0.0060 0.0110 0.0070 0.0128 0.0151 0.0104 0.0118 

0.10 0.0041 -0.0031 0.0119 0.0149 0.0187 0.0208 0.0170 0.0158 

0.25 0.0012 0.0277 -0.0022 0.0204 0.0132 0.0136 0.0197 0.0100 

0.50 0.0006 0.0197 -0.0140 0.0483 0.0218 0.0216 0.0466 0.0177 

1.00 -0.0001 0.0324 -0.0090 0.0149 0.0058 0.0057 0.0148 0.0056 

Table 44. Like table 43, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 63

January 18, 2008
Draft

61

Total stats bias abserr r2 ia rmse

Temperature (K) 0.00 0.00 0.001 0.001 0.0059 

Mixing Ratio (g/kg) -0.01 -0.01 -0.002 -0.001 -0.0132 

Rel. Humidity (%) -0.06 0.01 0.001 0.001 0.0190 

Wind Speed (m/s) 0.00 0.00 -0.001 -0.001 -0.0023 

Clouds (%) 0.01 -0.14 -0.003 -0.003 -0.2124 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.026 -0.12 -0.0029-0.0027 -0.0039 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0005 0.0045 -0.0019 0.0017 0.0002 -0.0007 -0.0008 -0.0008 

0.05 -0.0023 -0.0094 -0.0042 0.0009 -0.0019 -0.0031 -0.0028 -0.0038 

0.10 -0.0031 -0.0116 -0.0057-0.0014 -0.0038 -0.0048 -0.0050 -0.0061 

0.25 -0.0018 -0.0141 -0.0041-0.0014 -0.0027 -0.0031 -0.0033 -0.0043 

0.50 -0.0015 -0.0280 -0.0046-0.0011 -0.0028 -0.0030 -0.0027 -0.0046 

1.00 -0.0005 -0.0485 -0.0017 0.0037 -0.0001 -0.0002 0.0030 -0.0003 

Table 45. Episode 2 composite statistical comparison of sensitivity px_acm6 with base case px_acm2 for the US portion 

of the 36-km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) -0.07 0.04 0.007 0.004 0.0450 

Mixing Ratio (g/kg) -0.53 -0.26 -0.081 -0.034 -0.3480 

Rel. Humidity (%) -2.23 -0.75 -0.035 -0.020 -1.0033 

Wind Speed (m/s) 0.13 0.01 -0.015 -0.004 0.0137 

Clouds (%) -2.24 -2.06 -0.039 -0.025 -2.9210 

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.618 -1.08 -0.0026 0.0070 0.0034 

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 -0.0258 0.0287 -0.0392-0.0118 -0.0306 -0.0478 -0.0512 -0.0515 

0.05 -0.0290 -0.0430 -0.0461-0.0185 -0.0374 -0.0508 -0.0534 -0.0577 

0.10 -0.0311 -0.0461 -0.0518-0.0304 -0.0443 -0.0548 -0.0618 -0.0659 

0.25 -0.0251 -0.0470 -0.0517-0.0462 -0.0445 -0.0494 -0.0666 -0.0658 

0.50 -0.0170 -0.0812 -0.0497-0.0518 -0.0387 -0.0409 -0.0653 -0.0610 

1.00 -0.0069 -0.2206 -0.0253-0.0186 -0.0152 -0.0160 -0.0252 -0.0280 

Table 46. Like table 45, except for 12-km VISTAS domain.

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 64

January 18, 2008
Draft

62

Recall from the discussion of sensitivity px_acm4 that a slight improvement was found by using RAWINS 

output to drive both INTERPF and MM5. Since that sensitivity involved changing two variables at once, we do 

not know which change (or both) led to the improvement. Sensitivities px_acm7 and px_acm8 are designed to 

answer that question. In px_acm7 we change only the SFCFDDA file that MM5 reads in, using the RAWINS 

output rather than LITTLE_R output. Tables 47-50 show the results for both grids and for both episodes. Very 

slight performance degradation is seen in all tables. Tables 51-54 show the results for px_acm8, a sensitivity in 

which LITTLE_R outputs are used as inputs both to MM5 and INTERPF. While the wintertime tables show 

very slight performance degradation for precipitation, the remaining variables show universal improvement. 

Similar improvements are seen in the summertime cases with no performance degradation in the precipitation 

statistics. This configuration appears to be an improvement over the base case and should be incorporated into 

the annual modeling protocol.

The MM5 (http://www.mmm.ucar.edu/mm5/documents/MM5_tut_Web_notes/MM5/mm5.html) 

documentation states that the IFSNOW option applies when the LSM is not used. So this flag was not deemed 

to be important for P-X applications. Of course when MM5 documentation refers to the LSM it normally means 

the NOAH LSM, not the P-X LSM. So we performed a couple of wintertime only sensitivities to determine the 

effect the IFSNOW flag may have on a P-X run. Table 55 shows the results for the 36-km domain if the 

IFSNOW flag is set to 0. Note the bias improvement of 0.07 oC for temperature for this sensitivity s0px_acm2. 

Perhaps snow effects are being double-counted in P-X if this flag is turned on? The other variables are relatively 

unaffected, though it is interesting to note the almost universal slight degradation of precipitation skill. Since 

snow is a relatively rare occurrence in the VISTAS domain, little change is seen at 12-km resolution for the 

VISTAS domain, as illustrated in table 56.

Tables 57-58 show the statistical difference tables for the s2px_acm2 configuration. This sensitivity turns 

the simple snow model on. These runs are very cold-biased, even for the 12-km VISTAS domain. Clearly this 

configuration is not recommended for annual modeling. 

Tables 59-60 show the shorter summertime episode (episode 3) results for the px_acm9 sensitivity. The idea 

for this test is to see if an integration of EDAS soil temperature/moisture might yield similar wintertime benefits 

that we saw in the px_acm3 sensitivity, while not showing the degraded summertime performance that prior 

sensitivity showed. Admittedly more work could be done along these lines, but the results for this px_acm9 

sensitivity show the summertime performance degradation we had hoped would not exist. This configuration is 

not recommended for annual modeling. 

The next few sensitivity runs quantify the effects of nudging temperature/moisture near the surface. One 

advantage of the P-X module is that it allows indirect soil moisture nudging to be applied, thereby in essence 

“nudging” the lowest layers of the atmosphere without the instabilities that might result from directly applying 

surface analysis nudging of temperature/moisture. To quantify the effects this indirect soil nudging has on the 

results, we turned it off in sensitivity n0px_acm2 and applied the model only for episode 2 at 36-km resolution 

(table 61). Not surprisingly we see general performance degradation, though the character of the simulation is 

not changed dramatically. For example, the temperature error metric still shows better performance than what 

we saw in the noah_mrf, multi_blkdr, and noah_eta-my runs (tables 15, 21, and 27, respectively). 

Pre-Hearing Draft

Regional Haze SIP for TN Class I Areas

Appendix E.2 - 65

January 18, 2008