# Appendix E

. 2

M M5 M odeling Protocol

Tennessee Department of Environment and Conservation

Division of Air Pollution Control

9 th Floor L&C Annex

401 Church Street

Nashville, TN 37243Draft



## Protocol for Annual MM5 Modeling in Support of VISTAS (Visibility

Improvement

–State and Tribal Association)

(DRAFT)

Task 3a Deliverable

Prepared for:

Mr. Mike Abraczinskas

VISTAS Technical Analysis Workgroup

NC Division of Air Quality

1641 Ma il Service Center

Raleigh, NC 276990

- 1641

Prepared by:

Mr. Don Olerud

Mr. Aaron Sims

Baron Advanced Meteorological Systems, LLC

P.O. Box 12889, 3021 Cornwallis Road

Research Triangle Park, North Carolina 27709

February 9, 2004Draft

ii 1 Introduction

................................ ................................ ................................ ................................ ..................... 1

2 Description of the Meteorological Modeling Approach

................................ ................................ ................. 1

3 Evaluation Approac h

................................ ................................ ................................ ................................ ....... 3

4 Results

................................ ................................ ................................ ................................ ............................. 5

5 Auxiliary Results

................................ ................................ ................................ ................................ ........... 52

6 Conclusions from Sensitivity Modeling

................................ ................................ ................................ ..... 109

7 Methodology for Annual Modeling Effort

................................ ................................ ................................ . 110

A. Modeling Approach

................................ ................................ ................................ ................................ 110

B. Model Execution

................................ ................................ ................................ ................................ ..... 112

Terrain

................................ ................................ ................................ ................................ ......................... 112

REGRID (pregrid)

................................ ................................ ................................ ................................ ....... 112

R EGRID (regridder)

................................ ................................ ................................ ................................ ... 112

FETCH (adp_sfc)

................................ ................................ ................................ ................................ ........ 113

FETCH (adp_upa)

................................ ................................ ................................ ................................ ....... 113

LITTLE_R

(Objective Analysis)

................................ ................................ ................................ ................ 113

INTERPF

................................ ................................ ................................ ................................ .................... 113

MM5

................................ ................................ ................................ ................................ ............................ 113

C. Model Evaluation Proc edures

................................ ................................ ................................ ................. 114

D. Computing Resources

................................ ................................ ................................ ............................. 124

E. Project Schedule

................................ ................................ ................................ ................................ ...... 124

8 Refere nces

................................ ................................ ................................ ................................ ................... 124

9 Appendices

................................ ................................ ................................ ................................ ...................... 1

A. Terrain namelist

................................ ................................ ................................ ................................ ...... 1

B. Pregrid script

................................ ................................ ................................ ................................ ........... 2

C. Regridder 36

- km sample namelist

................................ ................................ ................................ ........ 10

D. FETCH (adp_sfc) sample namelist

................................ ................................ ................................ ....... 11

E. FETCH (adp_upa) sample namelist

................................ ................................ ................................ ...... 11

F. LITTLE_R sample 36

- km namelist

................................ ................................ ................................ ...... 11

G. INTERPF sample 36

- km namelist

................................ ................................ ................................ ........ 14

H. MM5 sample mmlif

................................ ................................ ................................ .............................. 15Draft

1 

## 1 Introduction

In recent years visibility concerns have come to the forefront in the air quality community. Millions of

visitors to national parks in the Unit ed States have their views obstructed by pollution

- induced haze. The

USEPA reports that average visibility in the east has been reduced from 90 miles to 15

- 25 miles

( http://www.epa.gov/oar/visibil ity/what.html ). To address this issue, the USEPA in 1999 instituted policies to

improve visibility in the national parks. As part of this initiative, five multi

- state regional planning

organizations (RPO) were formed. The RPO governing visibility issues in the southeastern US is the Visibility

Improvement State and Tribal Association of the Southeast (VISTAS) ( http://www.vistas

- sesarm.org/ ).

VISTAS recognizes the regional nature of haze, and has therefore s et up a modeling approach to address the

problem in the southeast US. Ultimately pollution controls will be enacted based upon chemical modeling

results over the region of interest. To support this modeling effort, Baron Advanced Meteorological Systems

(BA MS) is tasked with conducting the meteorological modeling. A 12

- month modeling period is deemed

necessary to cover an adequate range of visibility impairment. Prior to investing the resources to produce

meteorological results at 36

- km and 12

- km resolution for the full 12

- month period, BAMS executed a series of

sensitivity tests to determine the optimal meteorological setup for the annual modeling. This report details the

results from this sensitivity testing, leading directly to the protocol we will use for this project.



## 2 Description of the Meteorological Modeling Approach

The meteorological model used in this study is the PSU/NCAR Mesoscale Model (MM5 version 3.6, Grell

et al., 1994, MPP version). In order to build on prior relevant MM5 modeling results funded by the EPA and

other RPOs, those studies serve to establish the initial model configuration for this effort, as best can be

determined from available reports. Those findings are summarized in Olerud, 2003a. The modeling results

indicate that MM5 is most sensitive to the selection of planetary boundary layer (PBL) and soil schemes.

Therefore a series of sensitivity tests are recommended in Olerud, 2003b. Given limited time and budget

resources, a series of five sensitivity tests are laid out testing primarily the model response to the selection of

PBL scheme and soil model. These are the tests:

• px_acm : Pleim

- Xiu land surface model, asymmetric convective mixing PBL (Xiu and

Pleim, 2000), interppx to link runs.

• px_acm2 : Like px_acm, except no interp px linkage.

• noah_mrf : Noah land

- surface scheme (Chen and Dudhi, 2001) with the medium range

forecast (mrf) PBL (Hong and Pan, 1996).

• multi_blkdr : Multi

- layer soil scheme with Blackadar PBL and Zilitinkevich thermal

roughness length.

• noah_eta

- my : Noah land

- surface scheme with the ETA Mellor

- Yamada PBL (IMVDIF=0).Draft

2

The common options for all sensitivity tests include Kain

- Fritsch 2 cumulus parameterization (Kain and

Fritsch, 1993; Kain, 2002), mixed phase (Reisner 1) microphysics (Reisner et al, 1998), and Rapid Radiative

Transfer Model (RRTM) radiation (Mlawer et al, 1997). Snow effects are turned on (IFSNOW=1). Note that the

use of the ETA M

- Y pbl scheme necessitates moist vertical diffusion being turned off. The runs are made with

analysis nudging coe

fficients set as follows (36

- km and 12

- km resolutions):

Winds (aloft): 2.5E

- 4,1.0E

- 4,

Winds (surface): 2.5E

- 4,1.0E

- 4,

Temp (aloft): 2.5E

- 4,1.0E

- 4,

Temp (surface): Not employed

Moisture (aloft): 1.0E

- 5,1.0E

- 5

Moisture (surface): Not employed

In the cr oss

- sensitivity plots that follow we compare the px_acm2 sensitivity run to the noah_mrf,

multi_blkdr, and noah_eta

- my sensitivities for episode 1. Episodes 2 and 3 are similar except px_acm will serve

as the base configuration.

The runs are executed in 2

- way mode with feedback turned off. The five sensitivity runs are executed for

three separate episodes listed below:

• Episode 1: January 2

- 21, 2002

• Episode 2: July 13

- 28, 2001

• Episode 3: July 13

- 22, 1999.

Each episode is preceded by a spin

- up period (7, 7, and 4 days, respectively) that will not be discussed in

this report. The runs are made in 5.5

- day segments, each starting at 00 UTC, with the first 12 hours of each

segment serving as spin

- up. Figure 1 shows the modeling domains used in this study.

Figure 1. VISTAS 36

- km/12

- km MM5 modeling domains are shown.Draft

3 

## 3 Evaluation Approach

It is common in the air quality community to use surface statistics of the base meteorological variables as

the dominant metrics to determine acceptable model performance. Often statistics for only temperature, mixing

ratio and wind speed are calculated. Obviously it is important for the model to accurately represent these

variables, but there are additional variables that also become important when one considers that the re sults will

be used to improve visibility. As such we have added cloud cover, relative humidity, and precipitation to the

performance suite. While we calculate metrics separately for wind direction and wind speed, we also calculate

the mean error vector as perhaps the single best metric to quantify overall wind performance.

Recognizing that qualitative analyses of the model output are as important as standard quantitative analyses,

we enable the systematic visualization of model fields with observations ov erlaid whenever possible. To do this

we process the MM5 output through EPA’s MCIP2 program. MCIP2 transforms the data into NetCDF format

while also calculating a few fields (e.g. low, middle, and high CFRAC) that are not readily available in the raw

MM5 ou tput. MCIP2 also interpolates temperature and wind speed to observation height (1.5m and 10m,

respectively) for more accurate evaluation. Even though MCIP2 outputs a total cloud fraction, CMAQ uses this

quantity to estimate optical depth. Accordingly its v alue can be markedly different than what meteorologists

typically regard as cloud fraction. To make things as consistent as possible between the model and observations,

the cloud fractions presented in this report represent the maximum of the low, middle, and high cloud fractions.

We also use MCIP2 to cull a minimum of six cells about the domain periphery to minimize edge effects. The

reduced domain precisely matches the domain used in the air quality modeling. The 36

- km analysis domain

thus contains 148 co lumns, 112 rows, and 34 layers. The 12

- km analysis domain covers 168 columns, 177 rows,

and 34 layers.

The observations used for statistics come primarily from UCAR’s ds472.0 (TDL) archive

( http://dss. ucar.edu/datasets/ds472.0/ ). These data are quality controlled and converted to NetCDF format, thus

allowing the data to be visualized on the model fields via PAVE

( http://www.cep.unc.e du/empd/EDSS/pave_doc/index.shtml ). Unfortunately the precipitation values in this

dataset are not reliable, so we calculate precipitation statistics based on the 24

- h gridded accumulations

available from the Climate Prediction Center (CPC)

( http://www.cpc.ncep.noaa.gov/products/precip/realtime/retro.html ). These fields, originally at 0.25

- degree

resolution, undergo grid transformation to match our 36

- km and 12

- km domains. Since the CPC analyses are

derived primarily from rain gauges, the statistics are only calculated over cells that MM5 deems to be land.

For aloft analyses we process standard sounding observations from the NCEP ds353.4 archive

( http://dss.ucar.edu/datasets/ds353.4/ ). These observations are quality controlled and used to produce

model/observation skewT sounding plots for selected sites. Additionally we integrate the observations into

sigma levels that match the MM5 specifications, after which we can statistically analyze performance at sigma

levels 9, 17, and 22 (~500m, ~1600m, ~3400m, respectively). Qualitative profiler plots showing

model/observed hourly winds are also created based upon the data stor ed at the Forecast Systems Lab

( http://www.profiler.noaa.gov/jsp/ ). These results, along with much more, will not be presented here. The reader

is referred to the VISTAS meteorological website ( http://www.baronams.com/projects/VISTAS/ ) for additional

evaluation details and results.Draft

4

The number of analysis plots available on the above website is truly daunting. To keep this report at a

somewhat manag

eable level, we will focus primarily on cross

- sensitivity plots and surface statistics. Except for

precipitation (as mentioned above), these statistics are calculated at the sites (color

- coded by RPO) shown in

Figure 2. Statistics are calculated and stored at each observing site, and we routinely aggregate these results to

produce statistical time series plots and tables for every appropriate RPO region. This approach also enables us

to produce station

- specific statistical quantities that can be plotted in a similar manner to Figure 2. The VISTAS

web page even shows an animation of how these quantities change throughout an episode

- composite day. The

results shown in this document focus on statistics aggregated only over the VISTAS portion of the 12

- km

domain , and the US portion of the 36

- km domain. The cross

- sensitivity plots are shown only for the 12

- km

VISTAS domain.

Figure 2. Surface observing network color

- coded to represent Regional Planning Organization areas. Dark blue diamonds

are in the VISTAS RPO , green diamonds are in the MANE

- VU RPO, light blue diamonds are in the MIDWEST RPO,

yellow diamonds are in the CENRAP RPO, and red diamonds are in the WRAP RPO.Draft

5 

## 4 Results

The initial results for the px_acm run for episode 1 are quite discouraging. The r un showa a significant cold

bias over much of the eastern US, including the VISTAS region as illustrated in figure 3. While mixing ratio,

clouds, precipitation, and winds are modeled reasonably well, the large cold bias is unexpected based on prior

finding s from other RPO’s and the EPA. Note in figure 3 that the first couple of days show very little

temperature bias, but the bias increases as the mean temperature rises. After much investigation it becomes

apparent that the deep soil temperature is initiated during an extreme cold event in the eastern US. Since the

model soil temperatures and moisture are passed from one model segment to another via the interppx

preprocessor, the cold soil acts as a continuous drag on the atmosphere that the model physics can never quite

overcome.

The bias problem is significantly reduced by simply running each model segment independently, thus

limiting the cold drag to actual cold conditions. Figure 4 shows the statistical time series for this new case,

px_acm2. This px_acm

2 configuration will henceforth be considered the base case episode 1 for cross

sensitivity purposes, while px_acm will be used for episodes 2 and 3. For the statistical difference tables that

make up the bulk of the table portion of this document, we empl oy px_acm2 as the de facto base case.

Figure 3. Episode 1 temperature (1.5 m) statistical time series plot for the 12

- km VISTAS region, px_acm sensitivity. The

top panel shows the mean of the observations (blue) and the model (red), the middle plot sh ows the model bias (blue) and

the absolute error (red), while the bottom plot shows the index of agreement (blue) and coefficient of determination (red).Draft

6 Figure 4. Episode 1 temperature (1.5 m) statistical time series plot for the 12

- km VISTAS region, px_acm2 sensitivity.

The top panel shows the mean of the observations (blue) and the model (red), the middle plot shows the model bias (blue)

and the absolute error (red), while the bottom plot shows the index of agreement (blue) and coefficient of determ ination

(red).

Figure 5 shows the daytime PBL heights for January 10, 2002 at 12

- km resolution over the VISTAS region.

It should be noted that the noah_eta

- my PBL heights can erroneously become negative over small spatial areas;

we set all such negative values to zero before averaging. Due to the small areal extent of these negative PBL

values, we do not anticipate any qualitative assessments to be affected by those artifacts. The January 10, 2002

PBL heights are rather typical of winter PBL heights. The noah_mrf heights are significantly higher and

smoother than those in the other sensitivities. Generally speaking, the noah_eta

- my daytime PBL heights are

lower than they are in the other sensitivity runs. The px_acm2 heights tend to be more in the middle of those

extremes, though they also “bottom out” more than the other runs.

Figure 6 shows the cross

- sensitivity daytime precipitation plot for this same day. The low PBL heights in

the px_acm2 run are closely correlated to precipitation in the Ohio Valle y, while melting snow and clouds

(figure 7) might inhibit mixing over the northern Mid

- Atlantic States.Draft

7 Figure 5. Daytime (18

- 21 UTC) average PBL heights for the 12

- km VISTAS region for January 10, 2002 are displayed.

The px_acm2 sensitivity is shown in the upper left, the noah_mrf in the upper right, the multi_blkdr in the lower left, and

the noah_eta

- my in the lower right. Note that the time value (0:00:00) is only a placeholder and has no physical meaning.Draft

8 Figure 6. Like figure 5, except for pr ecipitation.Draft

9 Figure 7. Like figure 5, except for cloud coverage.

Figure 8 shows the daily average temperature for this same day. Note that the px_acm2 case is generally

warmer than the other cases, while the noah_eta

- my run is the coldest. Figure 9 shows that for daily averaged

mixing ratio the patterns are very similar for all runs. The combination of warmer daytime temperatures and

similar mixing ratios results in lower daytime relative humidity for the px_acm2 case compared with the other

sensit ivity runs (figure 10). Finally, figure 11 shows that the daytime wind speeds are similar in all cases.Draft

10 Figure 8. Like figure 5, except for daily average temperature.Draft

11 Figure 9. Like figure 5, except for daily average temperature.Draft

12 Figure 10. Like figure 5, except for relative humidity.Draft

13 Figure 11. Like figure 5,except for wind speed.

Traditionally nighttime PBL heights have not been considered very important for air quality modeling, but

they certainly may be for visibility/particula te modeling. Figure 12 shows the nighttime (07

- 10 UTC) PBL

heights for January 10, 2002. Notice that the px_acm2 produces lower PBL heights than the rest of the

sensitivity cases do, thus trapping surface

- based emissions in a smaller volume of air than wou ld occur in

another MM5 configuration. The noah_mrf run again produces the highest PBL heights at night, while the

multi_blkdr and noah_eta

- my runs are somewhere in the middle.Draft

14 Figure 12. Nighttime (07

- 10 UTC) average PBL heights for the 12

- km VISTAS r egion for January 10, 2002 are

displayed. The px_acm2 sensitivity is shown in the upper left, the noah_mrf in the upper right, the multi_blkdr in the

lower left, and the noah_eta

- my in the lower right. Note that the time value (0:00:00) is only a placehold er and has no

physical meaning.Draft

15

Nighttime cloud cover is shown in figure 13. The most striking observation from this figure is the cloud

deck over Tennessee in the noah_eta

- my run that does not exist to the same extent in the other sensitivity runs.

Figure 13. Like figure 12, except for cloud cover.Draft

16

Figure 14 shows the nighttime relative humidity plot. One might expect that the px_acm2 case would show

the highest relative humidity, given the low PBL heights as indicated by figure 12. The opposite is actually the

case. The warmer temperatures in this run counteract the increased stability such that the relative humidity

values are the lowest of all the runs. The noah_eta

- my run easily exhibits the highest relative humidity.

Figure 14. Like fig ure 12, except for relative humidity.Draft

17

Figure 15 shows the nighttime wind speed. Speeds are lowest in the noah_eta

- my run, followed by

px_acm2, noah_mrf, and multi_blkdr. In fact, the latter two cases seem to show an inappropriate diurnal pattern

in that t heir nighttime wind speeds are higher than their daytime wind speeds (figure 12).

Figure 15. Like figure 12, except for wind speed.Draft

18

Many of the same observations reported above are also valid for the summer episodes. To save time we will

only show sp atial 4

- panel plots for PBL heights for a sample summer day, July 19, 2001. Figure 16 shows the

daytime average, while figure 17 shows the nighttime average.

Figure 16. Daytime (18

- 21 UTC) average PBL heights for the 12

- km VISTAS region for July 19, 20 01 are displayed. The

px_acm sensitivity is shown in the upper left, the noah_mrf in the upper right, the multi_blkdr in the lower left, and the

noah_eta

- my in the lower right. Note that the time value (0:00:00) is only a placeholder and has no physical me aning.Draft

19 Figure 17. Nighttime (07

- 10 UTC) average PBL heights for the 12

- km VISTAS region for July 19, 2001 are displayed.

The px_acm sensitivity is shown in the upper left, the noah_mrf in the upper right, the multi_blkdr in the lower left, and

the noah _eta

- my in the lower right. Note that the time value (0:00:00) is only a placeholder and has no physical meaning.

Now that a qualitative understanding of these sensitivity runs has been established, the remainder of this

report will focus on quantitative comparisons between the sensitivity cases. Figure 18 shows the temperature

statistical time series plot for episode 1. While the general performance of the model is very similar across the

sensitivity runs, close examination reveals that the px_acm2 case performs the best. Figure 19 shows theDraft

20 corresponding plot for episode 2. The noah_eta

- my run clearly performs the worst, while the other sensitivity

runs show similar performance. The episode 3 plot (not shown) reveals similar responses.

Figure 20 shows the 12

- km mixing ratio statistical time series plot for episode 1 over the VISTAS region.

Overall the noah_mrf case performs the best, followed by px_acm2. The multi_blkdr case is clearly the poorest

performing. The corresponding episode 2 plot (figure 21) reveals a different result in that the noah_mrf case is

negatively biased in mixing ratio. This weakness presumably stems from dry air being mixed down from aloft

as the PBL becomes too high. The other cases are relatively similar. The negative mixing rat io bias is also

evident for noah_mrf in episode 3 (not shown).

The wind direction plot for episode 1 is shown in figure 22. The direction bias and error plots show similar

performance among the sensitivity cases, but the magnitude of the error vector plo t (bottom panel) shows that

the noah_eta

- my is the best performing run, especially at night. The px_acm2 run is generally second best.

Figure 23 shows that similar results are seen in episode 2, as well as in episode 3 (not shown).

The cloud cover statis tical plots (figures 24

- 25) show very little difference in performance among the

sensitivity runs. Figure 26 reveals that relative humidity for episode 1 is best modeled by either px_acm2 or

noah

- mrf, with the multi_blkdr case performing the worst. The epi sode 2 plot (figure 27) shows a strong diurnal

signature with the sensitivity runs generally being negatively biased at night and positively biased during the

day. The diurnal signature is interestingly the weakest for the noah_eta

- my run, leading to that sensitivity

possibly performing the best for this quantity. The episode 3 plot (not shown) has the noah_eta

- my run

displaying the poorest performance, no doubt due to its negative temperature bias.

Figure 28 shows the precipitation statistics for the ful l 12

- km grid for episode 1. The bias blip on January 10

resulted from there being very few grid cells that actually observed measurable precipitation on that day.

Sensitivity px_acm2 clearly outperforms the other cases for this episode. Figure 29 shows the corresponding

plot for episode 2. Sensitivity noah_eta

- my seems to be relatively unbiased, while the other sensitivity runs

show a slight low bias. Nevertheless the skill plots show little difference in performance among the runs. The

px_acm case appears to show slightly better results than do the other runs. Similar results are found for episode

3 (not shown).

Figure 30 is designed to show which sensitivity case statistically performs the best at each valid observation

site. This particular image repres ents a composite of 1.5m temperatures for all hours, with absolute error being

the defining metric. Note that the px_acm2 run performs best for a majority of the sites. The noah_eta

- my run

appears to do quite well over Florida. Figure 31 shows the correspo nding plot for episode 2. Again the px_acm

case seems to perform best overall, though noah_eta

- my again performs best in Florida and along the

southeastern coastline. The results for episode 3 (not shown) reveal no best performing sensitivity.

Figure 32 shows a similar type of plot for mixing ratio. The noah_mrf case appears to perform best for the

largest number of sites, followed by the px_acm2 case. The episode 2 results (figure 33) indicate just the

opposite, as the noah_eta

- my or multi_blkdr cases se em to perform best for most of the sites. The episode 3 plot

(not shown) is a mixed bag with the noah_eta

- my and px_acm cases seemingly performing best.Draft

21 Figure 18. Episode 1 (Jan 2

- 21, 2002) cross

- sensitivity statistical time series plot for temperature is shown. The top panel

shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm2 case is shown in

blue, noah_mrf in red, multi_blkdr in black, and noah_eta

- my in purple.

Figure 19. Episode 2 (Jul 13

- 28, 2001) c ross

- sensitivity statistical time series plot for temperature is shown. The top panel

shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm case is shown in blue,

noah_mrf in red, multi_blkdr in black, and noah_et a

- my in purple.Draft

2 2 Figure 20. Episode 1 (Jan 2

- 21, 2002) cross

- sensitivity statistical time series plot for mixing ratio is shown. The top panel

shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm2 case is blue,

noah_mrf red, multi_blkdr black, and noah_eta

- my purple.

Figure 21. Episode 2 (Jul 13

- 28, 2001) cross

- sensitivity statistical time series plot for mixing ratio is shown. The top panel

shows bias, the second panel absolute error, and the bottom panel in dex of agreement. The px_acm case is blue, noah_mrf

red, multi_blkdr black, and noah_eta

- my purple.Draft

23 Figure 22. Episode 1 (Jan 2

- 21, 2002) cross

- sensitivity statistical time series plot for winds is shown. The top panel shows

wind direction bias, the seco nd panel absolute wind direction error, and the bottom panel the magnitude of the error wind

vector. The px_acm2 case is shown in blue, noah_mrf in red, multi_blkdr in black, and noah_eta

- my in purple.

Figure 23. Episode 2 (Jul 13

- 28, 2001) cross

- sensi tivity statistical time series plot for winds is shown. The top panel

shows wind direction bias, the second panel absolute wind direction error, and the bottom panel the magnitude of the error

wind vector. The px_acm case is shown in blue, noah_mrf in red, multi_blkdr in black, and noah_eta

- my in purple.Draft

24 Figure 24. Episode 1 (Jan 2

- 21, 2002) cross

- sensitivity statistical time series plot for cloud coverage is shown. The top

panel shows bias, the second panel absolute error, and the bottom panel index of a greement. The px_acm2 case is blue,

noah_mrf red, multi_blkdr black, and noah_eta

- my purple.

Figure 25. Episode 2 (Jul 13

- 28, 2001) cross

- sensitivity statistical time series plot for cloud coverage is shown. The top

panel shows bias, the second panel a bsolute error, and the bottom panel index of agreement. The px_acm case is blue,

noah_mrf red, multi_blkdr black, and noah_eta

- my purple.Draft

25 Figure 26. Episode 1 (Jan 2

- 21, 2002) cross

- sensitivity statistical time series plot for relative humidity is shown. The top

panel shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm2 case is blue,

noah_mrf red, multi_blkdr black, and noah_eta

- my purple.

Figure 27. Episode 2 (Jul 13

- 28, 2001) cross

- sensitivity statistical time series plot for relative humidity is shown. The top

panel shows bias, the second panel absolute error, and the bottom panel index of agreement. The px_acm case is blue,

noah_mrf red, multi_blkdr black, and noah_eta

- my purple.Draft

26 Figure 28. Episode 1 (J an 2

- 21, 2002) cross

- sensitivity statistical time series plot for 24

- h measurable precip is shown.

The top panel shows bias, the second panel accuracy, and the bottom panel equitable threat score. The px_acm2 case is

blue, noah_mrf red, multi_blkdr black, and noah_eta

- my purple.

Figure 29. Episode 2 (Jul 13

- 28, 2001) cross

- sensitivity statistical time series plot for 24

- h measurable precip is shown.

The top panel shows bias, the second panel accuracy, and the bottom panel equitable threat score. The px_a cm case is

blue, noah_mrf red, multi_blkdr black, and noah_eta

- my purple.Draft

27 Figure 30. Episode 1 (Jan 2

- 21, 2002) cross

- sensitivity 1.5m temperature absolute error comparison plot is shown.

Stations for which px_acm2 show the smallest composite error ar e plotted in blue, noah_mrf in green, multi_blkdr in

yellow, and noah_eta

- my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders

and should be ignored.Draft

28 Figure 31. Episode 2 (Jul 13

- 28, 2001) cross

- sensitivit y 1.5m temperature absolute error comparison plot is shown.

Stations for which px_acm2 show the smallest composite error are plotted in blue, noah_mrf in green, multi_blkdr in

yellow, and noah_eta

- my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders

and should be ignored.Draft

29 Figure 32. Episode 1 (Jan 2

- 21, 2002) cross

- sensitivity mixing ratio absolute error comparison plot is shown. Stations for

which px_acm2 show the smallest composite error are plotted in blu e, noah_mrf in green, multi_blkdr in yellow, and

noah_eta

- my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders and should

be ignored.Draft

30 Figure 33. Episode 2 (Jul 13

- 28, 2001) cross

- sensitivity mixing ratio a bsolute error comparison plot is shown. Stations for

which px_acm2 show the smallest composite error are plotted in blue, noah_mrf in green, multi_blkdr in yellow, and

noah_eta

- my in red. The date/time/max/min information at the bottom of the plot serves o nly as placeholders and should

be ignored.Draft

31

The corresponding series of plots for the magnitude of the error wind vector (figures 34

- 35) show the clear

superiority of the noah_eta

- my runs. The px_acm(2) runs perform a distant second best for all sensitivi ties. The

main reason why noah_eta

- my performs so well is its ability to calm its wind speeds at night (figure 15) relative

to the other model configurations.

Figure 34. Episode 1 (Jan 2

- 21, 2002) cross

- sensitivity error vector magnitude comparison p lot is shown. Stations for

which px_acm2 show the smallest composite error are plotted in blue, noah_mrf in green, multi_blkdr in yellow, and

noah_eta

- my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders and sh ould

be ignored.Draft

32 Figure 35. Episode 2 (Jul 13

- 28, 2001) cross

- sensitivity error vector magnitude comparison plot is shown. Stations for

which px_acm2 show the smallest composite error are plotted in blue, noah_mrf in green, multi_blkdr in yellow, an d

noah_eta

- my in red. The date/time/max/min information at the bottom of the plot serves only as placeholders and should

be ignored.Draft

33

The remainder of this report will focus primarily on statistical tables. These tables are episode

- composites

and are div ided into three sections. The top section contains bias, absolute error, coefficient of determination

(r 2 ), index of agreement, and root mean square error for temperature (1.5 m), water vapor mixing ratio, relative

humidity, wind speed (10 m), and cloud co verage. The second section contains additional statistics for winds,

including wind direction bias, absolute wind direction error, U

- wind component error, V

- wind component error,

and the magnitude of the average error wind vector. The latter metric is perh aps the single best means of

assessing model wind performance. The final section contains precipitation statistics at various thresholds (0.01,

0.05, 0.10, 0.25, 0.5, and 1.0 inches) for daily precipitation accumulations. The metrics presented are accuracy

, bias, false alarm ratio (FAR), probability of detection (POD), critical success index (CSI, i.e. threat score),

equitable threat score (ETS), “true skill score” (TSS, i.e. Hanssen and Kuipers score), and Heidke skill score

(HSS). These precipitation stat istics are only generated for approximately the US portion of the grids; there are

no RPO

- specific statistics for this variable. Since an RPO

- specific statistical summary is incomplete without

some sort of precipitation metric, we have decided to use the p recipitation statistics described above for every

statistical composite we produce. The reader should understand the inconsistencies that this approach entails.

Note also that the precipitation observational grids are obtained, in part, by objectively anal yzing rain gauge

data. This tends to “spread out” the measurable precipitation beyond where it actually occurs. Since a model

does not do that, a perfect precipitation forecast might appear to be slightly low biased at the 0.01

- inch

threshold. At higher th

resholds the number of valid occurrences may not reach a robust level.

Sensitivity px_acm2 is the base case for all of the statistical tables that follow. Table 1 shows the base case

statistics for the US portion of the 36

- km domain for episode 1, while table 2 reports statistics for the VISTAS

12

- km domain for episode 1. Recall the meteorological statistical benchmarks reported by Emery (2001):

Wind speed: RSME <= 2 m/s, Bias <= +/

- 0.5 m/s, IA >= 0.6

Wind direction: Gross Error <= 30 deg, Bias <= +/

- 10 deg.

Temperature: Gross Error <= 2 K, Bias <= +/

- 0.5 K, IA >= 0.8

Humidity: Gross Error <= 2 g/kg, Bias <= +/

- 1 g/kg, IA >= 0.6

Note that the only metrics that fail to meet these benchmarks are temperature bias and error. It should be

understood that the above benchmarks are based primarily on meteorological modeling of summertime

episodes. While ideally we want less temperature bias and error, the results seen here are not unusually bad.

Note that the index of agreement temperature st atistic easily betters the benchmark value for both domains. No

benchmarks exist for relative humidity, cloud cover, and precipitation. It is encouraging that these quantities are

all relatively unbiased and appear to show reasonable skill.

Tables 3

- 4 sho w the episode 2 statistical tables for px_acm2 for the selected domains. The only base

variable that fails to meet the above benchmark is wind direction gross error at 36

- km resolution, and that value

(32.27 o ) is certainly in the ballpark. Given the weak s ynoptic forcing in this episode, it is likely that sub

synoptic forcing (e.g. terrain flows, thunderstorm outflows, etc...) unresolved adequately at this resolution

degrades the wind direction performance. The other variables are modeled reasonably well, wit h a couple of

possible exceptions. Note the positive cloud coverage bias (>7%) in table 4, and the positive precipitation bias

at higher thresholds in both table 3 and table 4. The model appears to overestimate the extent and intensity of

summertime convec tion. Similar results are seen for episode 3 (tables 5

- 6). The cloud biases are less than in

episode 2, though the positive precipitation biases at higher thresholds are accentuated at 36

- km resolution.Draft

34

Total stats bias abserr r2 ia rmse

Temperatu

re (K)

- 1.14 2.30 0.879 0.963 3.0613

Mixing Ratio (g/kg) 0.07 0.53 0.876 0.967 0.7673

Rel. Humidity (%) 3.54 12.62 0.451 0.814 16.4208

Wind Speed (m/s)

- 0.02 1.33 0.510 0.808 1.7307

Clouds (%)

- 4.00 26.12 0.340 0.763 37.6462

Wind stats bi as abserr uerr verr uverr

Wind (deg, m/s) 1.913 25.11 1.2520 1.2552 1.7729

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.8360 1.0061 0.2760 0.7284 0.5701 0.4379 0.6102 0.6091

0.05 0.9038 1.1156 0.3204 0.7581 0.5585 0.4914 0.6898 0.65 90

0.10 0.9352 1.1405 0.3308 0.7632 0.5541 0.5115 0.7187 0.6768

0.25 0.9703 1.0628 0.2959 0.7484 0.5693 0.5502 0.7309 0.7099

0.50 0.9845 1.0006 0.3133 0.6872 0.5232 0.5140 0.6792 0.6790

1.00 0.9940 0.8873 0.3833 0.5472 0.4083 0.4054 0.5446 0.5769

Table 1. Episode 1 composite statistical summary for base case px_acm2 for the US portion of the 36

- km domain is

shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 1.03 2.25 0.858 0.957 2.9596

Mixing Ratio (g/kg) 0.12 0.62 0.914 0.977 0.8 708

Rel. Humidity (%) 3.36 12.79 0.422 0.802 16.7355

Wind Speed (m/s) 0.43 1.35 0.407 0.764 1.7320

Clouds (%) 0.95 22.80 0.419 0.809 34.9631

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 2.216 22.74 1.1604 1.2419 1.6996

Pcp T hreshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.8735 1.0329 0.2086 0.8174 0.6725 0.5513 0.7170 0.7108

0.05 0.9148 1.0648 0.2238 0.8265 0.6675 0.5955 0.7643 0.7465

0.10 0.9363 1.0381 0.2145 0.8154 0.6669 0.6159 0.7741 0.7623

0.25 0.9635 0.9532 0.1842 0.7776 0.6615 0.6343 0.7599 0.7763

0.50 0.9756 0.9480 0.2568 0.7045 0.5665 0.5511 0.6929 0.7106

1.00 0.9907 0.7899 0.3092 0.5457 0.4385 0.4337 0.5423 0.6050

Table 2. Episode 1 composite statistical summary for base case px_acm2 for the VISTAS portion of the 12

- km domain is

shown.Draft

35

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.16 1.62 0.877 0.964 2.1443

Mixing Ratio (g/kg)

- 0.17 1.47 0.809 0.947 1.9714

Rel. Humidity (%)

- 0.28 9.78 0.672 0.904 13.0680

Wind Speed (m/s)

- 0.20 1 .24 0.429 0.772 1.6149

Clouds (%) 1.38 27.52 0.134 0.635 37.9526

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 2.010 32.37 1.2469 1.3037 1.8040

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.7375 0.8822 0.2501 0.6616 0 .5420 0.3067 0.4663 0.4694

0.05 0.7578 1.0815 0.4105 0.6376 0.4415 0.2794 0.4470 0.4368

0.10 0.7807 1.2392 0.5089 0.6085 0.3732 0.2511 0.4363 0.4014

0.25 0.8387 1.5406 0.6509 0.5378 0.2685 0.2008 0.4137 0.3345

0.50 0.9052 1.6829 0.7565 0.4098 0.180 3 0.1482 0.3416 0.2582

1.00 0.9715 1.5577 0.8696 0.2030 0.0862 0.0782 0.1849 0.1450

Table 3. Episode 2 composite statistical summary for base case px_acm2 for the US portion of the 36

- km domain is

shown.

Total stats bias abserr r2 ia rmse

Temper ature (K)

- 0.05 1.45 0.815 0.938 1.8473

Mixing Ratio (g/kg)

- 0.17 1.38 0.749 0.924 1.7851

Rel. Humidity (%)

- 0.72 9.44 0.541 0.858 12.4555

Wind Speed (m/s) 0.41 1.24 0.415 0.776 1.5632

Clouds (%) 7.27 30.11 0.170 0.660 39.2499

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 1.254 29.29 1.1294 1.2068 1.6529

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.7787 0.8714 0.1843 0.7108 0.6125 0.3855 0.5553 0.5564

0.05 0.7785 1.0044 0.3065 0.6966 0.5326 0.3524 0.5216 0 .5211

0.10 0.7803 1.0860 0.3857 0.6672 0.4702 0.3176 0.4942 0.4821

0.25 0.8045 1.2707 0.5345 0.5915 0.3522 0.2503 0.4426 0.4003

0.50 0.8585 1.4779 0.6821 0.4699 0.2340 0.1784 0.3678 0.3028

1.00 0.9398 1.7059 0.8631 0.2336 0.0945 0.0774 0.1929 0.143 6

Table 4. Episode 2 composite statistical summary for base case px_acm2 for the VISTAS portion of the 12

- km domain is

shown.Draft

36

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.19 1.64 0.862 0.957 2.1492

Mixing Ratio (g/kg)

- 0.25 1.49 0.771 0 .934 1.9660

Rel. Humidity (%)

- 0.22 9.77 0.640 0.893 12.9109

Wind Speed (m/s)

- 0.34 1.24 0.490 0.801 1.6120

Clouds (%)

- 0.59 26.66 0.179 0.672 36.8433

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 2.458 32.13 1.2014 1.2878 1.7612

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.7199 0.8128 0.2526 0.6075 0.5040 0.2749 0.4265 0.4313

0.05 0.7517 1.0015 0.4218 0.5791 0.4071 0.2520 0.4028 0.4026

0.10 0.7820 1.1542 0.5201 0.5539 0.3461 0.2305 0.3959 0.3746

0.25 0.84 47 1.4487 0.6681 0.4808 0.2443 0.1813 0.3679 0.3070

0.50 0.9114 1.7663 0.7772 0.3936 0.1659 0.1371 0.3292 0.2412

1.00 0.9687 2.0344 0.9079 0.1874 0.0658 0.0579 0.1654 0.1095

Table 5. Episode 3 composite statistical summary for base case px_acm2 for the US portion of the 36

- km domain is

shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.18 1.52 0.821 0.933 1.9103

Mixing Ratio (g/kg) 0.01 1.35 0.488 0.833 1.7674

Rel. Humidity (%) 1.35 8.58 0.524 0.842 11.1546

Wind Speed (m/s) 0 .13 1.16 0.282 0.699 1.4752

Clouds (%) 3.91 29.83 0.169 0.664 38.7202

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 1.595 34.59 1.0827 1.1372 1.5702

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.7015 0.8579 0.2708 0.6 256 0.5077 0.2512 0.4006 0.4015

0.05 0.7219 0.9406 0.3852 0.5783 0.4245 0.2378 0.3791 0.3842

0.10 0.7461 0.9913 0.4560 0.5393 0.3714 0.2241 0.3651 0.3661

0.25 0.8068 1.0958 0.5801 0.4602 0.2813 0.1924 0.3351 0.3227

0.50 0.8806 1.2216 0.6944 0.3734 0.2020 0.1569 0.2986 0.2712

1.00 0.9536 1.5517 0.8494 0.2337 0.1008 0.0872 0.2037 0.1605

Table 6. Episode 3 composite statistical summary for base case px_acm2 for the VISTAS portion of the 12

- km domain is

shown.Draft

37

To augment the statistical tables des cribed above, we have also produced tables of statistical differences

from the base case px_acm2. The table setup is precisely the same as above, but the differences reported are not

simple differences. Instead we present differences designed to indicate w hether the sensitivity produces better

results than the base case. For some metrics (e.g. r 2 , IA, CSI) larger numbers indicate improved performance.

For these metrics we calculate differences: SENS

–BASE. Positive results are color

- coded green, indicating

that the sensitivity improved performance. Negative results are color

- coded red, indicating that the sensitivity

degraded performance. For some other metrics (e.g. abserr, uverr, FAR) smaller numbers indicate improved

performance. Thus the difference tab le shows BASE

–SENS, and the color

- coding described above still

applies. Similarly the differences for the bias statistics are calculated relative to optimum performance (0

normally, 1 for precipitation). In this manner one can readily ascertain statistic al benefits/disbenefits for all

sensitivity runs. One should keep in mind that very small non

- zero statistical changes are color

- coded the same

as significant changes, so the actual numerical change should be considered more important than the color

coding

.

Tables 7

- 8 show the statistical difference results for sensitivity px_acm for episode 1. The most significant

differences are found in the temperature metrics, revealing that the px_acm configuration is significantly more

cold

- biased than is px_acm2 for this wintertime episode. Most of the other surface variables also show degraded

performance. The px_acm run does seem to very slightly improve cloud coverage and precipitation, but the

improvements are very slight and are probably less meaningful than the temperature degradation. These

performance patterns hold for both the US 36

- km results and the VISTAS 12

- km results. Tables 9

- 10 indicate

very little performance difference between the two configurations for episode 2. Precipitation performance is

slightly enhanced for this summertime period, while interestingly it actually slightly degrades in the other

summertime episode (tables 11

- 12). Overall px_acm seems to be a less desirable configuration for annual

modeling.

Tables 13

- 14 show the statistical differe nce results for the noah_mrf sensitivity for episode 1. The

performance of mixing ratio and relative humidity for both grids seem to be improved slightly in this

configuration. The remaining variables are degraded almost universally. Perhaps the most strik ing result seen in

the episode 2 tables (tables 15

- 16) is the degraded mixing ratio performance, countermanding the wintertime

result. The noah_mrf sensitivity is significantly more low

- biased for this variable than is the base case px_acm2,

especially for

the 12

- km VISTAS domain. At 36

- km this seems to help the run actually improve its cloud and

precipitation performance relative to the base. The 12

- km results are more ambiguous for these variables.

Generally similar results are found for episode 3 (tables 17

- 18). These findings, combined with the likelihood

that the PBL heights are overestimated under this configuration, lead one to reject this configuration for annual

modeling.

The multi_blkdr results for episode 1 are displayed in tables 19

- 20. With few exceptions this sensitivity

shows degraded performance for all variables for both grid resolutions. The low temperature bias is perhaps the

most significant issue with this configuration. The summertime tables for multi_blkdr (tables 21

- 24) show

results t hat are generally degraded from px_acm2. This configuration clearly degrades performance compared

with the base case px_acm2.

The noah_eta

- my results are found in tables 25

- 30. As mentioned earlier this configuration tends to improve

performance for winds , but the other variables tend to be degraded. Temperature especially is biased low for all

grids/resolutions. Still, the superior surface wind performance of the noah_eta

- my run deserves additional

attention. To compare the performance of this run versus px_acm(2), we computed composite statistics at

various sounding locations. These statistics considered only the 00 UTC (or 12 UTC) data in the composite. ADraft

38 plot of this type is shown in figure 36 for Greensboro, NC, episode 1, 00 UTC. Note that at the surf ace the wind

errors are less in the noah_eta

- my case than they are in the px_acm2 case, but for the majority of the lower

portions of the atmosphere the opposite occurs. The large temperature/dew point biases/errors in the noah_eta

my are expected and corr oborate what is seen in the surface statistics. Figure 37 shows that for this site similar

results are found in the wind profile for a summer episode. Performance at other upper air stations vary, and for

Florida stations the noah_eta

- my results appear to be slightly improved over the px_acm results. Generally

speaking, though, the improved wind performance for noah_eta

- my appears to be relegated to the lowest model

layer. For a multitude of reasons, therefore, this configuration is not recommended for the annual modeling.

Figure 36. Episode 1 (Jan 2

- 21, 2002) composite vertical statistics for all 00 UTC times for Greensboro, NC are shown.

The magnitude of the error vector is plotted in the leftmost panel, followed by temperature bias and error, then dew point

bias and error. Blue represents the px_acm2 case, while dashed red shows the noah_eta

- my results.Draft

39 Figure 37. Episode 2 (Jul 13

- 28, 2001) composite vertical statistics for all 00 UTC times for Greensboro, NC are shown.

The magnitude of the error v ector is plotted in the leftmost panel, followed by temperature bias and error, then dew point

bias and error. Blue represents the px_acm2 case, while dashed red shows the noah_eta

- my results.Draft

40

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.92

- 0.60

- 0.021

- 0.017

- 0.6456

Mixing Ratio (g/kg)

- 0.01 0.00

- 0.002

- 0.001 0.0023

Rel. Humidity (%) 0.26

- 0.08

- 0.017

- 0.006

- 0.1239

Wind Speed (m/s)

- 0.02

- 0.03

- 0.022

- 0.015

- 0.0392

Clouds (%)

- 0.23 0.12 0.002 0.001 0.0373

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.424

- 0.06

- 0.0205

- 0.0178

- 0.0270

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0035

- 0.0140 0.0120

- 0.0073 0.0028 0.0057 0.0008 0.0054

0.05 0.0023 0.0235 0.0104

- 0.0045 0.0045 0.0057

- 0.0010 0.0051

0.10 0.0014 0.0238 0.0097

- 0.0051 0.0039 0.0045

- 0.0029 0.0040

0.25 0.0006 0.0221 0.0103

- 0.0049 0.0039 0.0042

- 0.0038 0.0034

0.50 0.0000

- 0.0111 0.0030

- 0.0056

- 0.0016

- 0.0015

- 0.0054

- 0.0013

1.00 0.0000

- 0.0051

- 0.0010

- 0.0041

- 0.0027

- 0. 0028

- 0.0040

- 0.0028

Table 7. Episode 1 composite statistical comparison of sensitivity px_acm with base case px_acm2 for the US portion of

the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 1.05

- 0.57

- 0.015

- 0.019

- 0. 6223

Mixing Ratio (g/kg) 0.04

- 0.01

- 0.006

- 0.001

- 0.0164

Rel. Humidity (%)

- 0.28

- 0.39

- 0.026

- 0.013

- 0.5072

Wind Speed (m/s) 0.00

- 0.04

- 0.031

- 0.019

- 0.0431

Clouds (%) 0.16 0.07 0.001 0.001 0.0341

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.222

- 0.24

- 0.0291

- 0.0208

- 0.0351

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0059 0.0231 0.0219

- 0.0121 0.0072 0.0125 0.0022 0.0103

0.05 0.0049 0.0370 0.0215

- 0.0067 0.0112 0.0145 0.0013 0.0113

0.10 0.0024 0.0 299 0.0160

- 0.0073 0.0065 0.0080

- 0.0031 0.0061

0.25 0.0006

- 0.0251 0.0118

- 0.0095 0.0007 0.0011

- 0.0079 0.0008

0.50 0.0000

- 0.0165 0.0041

- 0.0084

- 0.0030

- 0.0030

- 0.0079

- 0.0025

1.00

- 0.0001

- 0.0202 0.0006

- 0.0136

- 0.0085

- 0.0085

- 0.0134

- 0.0083

Table 8. Like table 7, except for 12

- km VISTAS domain.Draft

41

Total stats bias abserr r2 ia rmse

Temperature (K) 0.04 0.05 0.004 0.002 0.0393

Mixing Ratio (g/kg)

- 0.24

- 0.04 0.003

- 0.002

- 0.0335

Rel. Humidity (%)

- 1.56

- 0.13

- 0.003

- 0.002

- 0.2838

Wind Speed (m/s) 0.02 0.00

- 0.001 0.000

- 0.0004

Clouds (%) 0.06

- 0.07

- 0.002

- 0.002

- 0.0969

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.074 0.07

- 0.0015

- 0.0012

- 0.0019

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0026

- 0.0124 0.0068

- 0.0034 0.0012 0.0043 0.0045 0.0050

0.05 0.0031 0.0170 0.0063

- 0.0034 0.0019 0.0038 0.0025 0.0046

0.10 0.0032 0.0169 0.0059

- 0.0010 0.0030 0.0042 0.0033 0.0053

0.25 0.0032 0.0281 0.0066 0.0002 0.0040 0.0047 0.0038 0.0064

0.50 0.0014 0.0371 0.0027

- 0.0047 0.0005 0.0008

- 0.0030 0.0012

1.00 0.0004 0.0356 0.0006

- 0.0036

- 0.0003

- 0.0003

- 0.0032

- 0.0005

Table 9. Episode 2 composite statistical comparison of sensitivity px_acm with base case px_acm2 for the US portion of

the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.01 0.05 0.007 0.004 0.0413

Mixing Ratio (g/kg)

- 0.20

- 0.05 0.003

- 0.002

- 0.0506

Rel. Humidity (%)

- 1.33

- 0.13 0.004

- 0.001

- 0.3367

Wind Speed (m/s)

- 0.02 0.00

- 0.001

- 0.001

- 0.0069

Clouds (%) 0.07

- 0.01 0.000 0.000 0.0241

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.092 0.04

- 0.0035

- 0.0038

- 0.0051

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0029

- 0.0150 0.0090

- 0.0046 0.0015 0.0054 0.0055 0.0057

0.05 0.0030

- 0.0130 0.0085

- 0.0068 0.0010 0.0039 0.0019 0.0043

0.10 0.0040 0.0275 0.0093

- 0.0071 0.0018 0.0045 0.0014 0.0052

0.25 0.0049 0.0391 0.0099

- 0.0059 0.0035 0.0053 0.0014 0.0069

0.50 0.0048 0.0489 0.0119 0.0013 0.0067 0.0077 0.0064 0.0109

1.00 0.0011 0.0331 0.0047 0.0032 0.0027 0.0028 0.0042 0.0049

Table 10. Like table 9, except for 12

- km VISTAS domain.Draft

42

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.14

- 0.01 0.002 0.001 0.0048

Mixing Ratio (g/kg)

- 0.25

- 0.02 0.001

- 0.002

- 0.0184

Rel. Humidity (%)

- 0.99 0.09 0.010 0.003 0.0700

Wind Speed (m/s) 0.01 0.00

- 0.001

- 0.001 0.0011

Clouds (%) 0.18

- 0.15

- 0.004

- 0.002

- 0.1736

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.009 0.08

- 0.0012

- 0.0004

- 0.0012

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0166 0.0050

- 0.0061

- 0.0013

- 0.0037

- 0.0216

- 0.0237

- 0.0270

0.05

- 0.0168

- 0.0082

- 0.0071

- 0.0136

- 0.0102

- 0.0197

- 0.0268

- 0.0256

0.10

- 0.0126 0.0113

- 0.0130

- 0.020 3

- 0.0145

- 0.0190

- 0.0286

- 0.0254

0.25

- 0.0067

- 0.0385

- 0.0312

- 0.0335

- 0.0250

- 0.0246

- 0.0384

- 0.0360

0.50 0.0003

- 0.1315

- 0.0330

- 0.0335

- 0.0240

- 0.0220

- 0.0341

- 0.0348

1.00 0.0027 0.0539

- 0.0273

- 0.0592

- 0.0208

- 0.0200

- 0.0568

- 0.0364

Table 11. Episode 3 composite statistical comparison of sensitivity px_acm with base case px_acm2 for the US portion of

the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.09 0.06 0.005 0.008 0.0655

Mixing Ratio (g/kg)

- 0.22 0. 00

- 0.007

- 0.004

- 0.0056

Rel. Humidity (%) 1.01 0.17 0.022 0.013 0.2309

Wind Speed (m/s)

- 0.03

- 0.01

- 0.004

- 0.002

- 0.0088

Clouds (%)

- 0.11

- 0.12

- 0.002

- 0.001

- 0.0867

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.083 0.22

- 0.0 043

- 0.0073

- 0.0082

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0320

- 0.0381

- 0.0245

- 0.0478

- 0.0425

- 0.0475

- 0.0625

- 0.0631

0.05

- 0.0218

- 0.0299

- 0.0461

- 0.0604

- 0.0526

- 0.0474

- 0.0661

- 0.0643

0.10

- 0.0139

- 0.0286

- 0.0642

- 0.0774

- 0.0636

- 0.0535

- 0.0771

- 0.0746

0.25 0.0057

- 0.0350

- 0.0865

- 0.0832

- 0.0664

- 0.0530

- 0.0770

- 0.0781

0.50 0.0193

- 0.0746

- 0.0733

- 0.0723

- 0.0511

- 0.0399

- 0.0600

- 0.0617

1.00 0.0152 0.1611

- 0.0075

- 0.0347

- 0.0100

- 0.0055

- 0.0237

- 0.0095

Table 12. L ike table 11, except for 12

- km VISTAS domain.Draft

43

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.14

- 0.17

- 0.012

- 0.005

- 0.1357

Mixing Ratio (g/kg) 0.01 0.04 0.009 0.002 0.0382

Rel. Humidity (%)

- 0.40 0.22 0.006 0.000 0.3528

Wind Speed (m

/s)

- 0.13

- 0.02 0.001 0.025

- 0.0283

Clouds (%)

- 2.68

- 1.58

- 0.042

- 0.029

- 1.4727

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 2.704

- 0.27

- 0.0108

- 0.0156

- 0.0187

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0153

- 0.0554

- 0.0359 0.0020

- 0.0213

- 0.0302

- 0.0207

- 0.0298

0.05

- 0.0042

- 0.0243

- 0.0154

- 0.0009

- 0.0109

- 0.0128

- 0.0058

- 0.0116

0.10

- 0.0015

- 0.0225

- 0.0093 0.0042

- 0.0042

- 0.0050 0.0021

- 0.0044

0.25

- 0.0027

- 0.0400

- 0.0299

- 0.0049

- 0.0223

- 0.0234

- 0.0073

- 0.0198

0.50

- 0.0011

- 0.0105

- 0.0224

- 0.0155

- 0.0217

- 0.0220

- 0.0161

- 0.0195

1.00

- 0.0006 0.0437

- 0.0464

- 0.0162

- 0.0290

- 0.0293

- 0.0167

- 0.0303

Table 13. Episode 1 composite statistical comparison of sensitivity noah_mrf with base case px_acm2 for the US portion

of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.29

- 0.08 0.009 0.000

- 0.0457

Mixing Ratio (g/kg) 0.00 0.04 0.007 0.002 0.0375

Rel. Humidity (%) 1.54 0.98 0.035 0.015 1.1957

Wind Speed (m/s) 0.18

- 0.11

- 0.060

- 0.005

- 0.1359

Clouds (%)

- 1.84

- 0.72

- 0.027

- 0.016

- 0.5780

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 1.433

- 1.71

- 0.0155

- 0.0637

- 0.0575

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0325

- 0. 0745

- 0.0657

- 0.0137

- 0.0561

- 0.0767

- 0.0549

- 0.0671

0.05

- 0.0071

- 0.0054

- 0.0174

- 0.0144

- 0.0221

- 0.0258

- 0.0196

- 0.0207

0.10

- 0.0041 0.0143

- 0.0089

- 0.0203

- 0.0198

- 0.0217

- 0.0214

- 0.0168

0.25

- 0.0015 0.0054

- 0.0103

- 0.0054

- 0.0107

- 0.0114

- 0.0066

- 0.0087

0.50 0.0002

- 0.0472 0.0152

- 0.0214

- 0.0055

- 0.0053

- 0.0201

- 0.0044

1.00

- 0.0003 0.0821

- 0.0289 0.0315 0.0073 0.0070 0.0309 0.0068

Table 14. Like table 13, except for 12

- km VISTAS domain.Draft

44

Total stats bias abserr r2 ia rmse

Temperature ( K)

- 0.37

- 0.27

- 0.021

- 0.005

- 0.3041

Mixing Ratio (g/kg)

- 0.67

- 0.05 0.017

- 0.005

- 0.0018

Rel. Humidity (%)

- 2.50

- 0.02 0.016

- 0.002 0.3326

Wind Speed (m/s)

- 0.31

- 0.20

- 0.104

- 0.047

- 0.2402

Clouds (%) 0.83 0.50 0.005 0.001 0.6075

Wind st ats bias abserr uerr verr uverr

Wind (deg, m/s)

- 1.086

- 0.53

- 0.0358

- 0.0394

- 0.0531

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0003

- 0.0157 0.0042

- 0.0082

- 0.0034

- 0.0009

- 0.0015

- 0.0010

0.05 0.0058 0.0206 0.0109

- 0.0007 0.0058 0.0085 0.0079 0.0103

0.10 0.0076 0.0410 0.0145

- 0.0027 0.0072 0.0100 0.0077 0.0127

0.25 0.0150 0.1450 0.0331

- 0.0045 0.0179 0.0214 0.0130 0.0292

0.50 0.0151 0.3081 0.0500

- 0.0064 0.0243 0.0272 0.0098 0.0403

1.00 0.0047 0.2763 0.0575 0.0378 0.0318 0 .0324 0.0419 0.0542

Table 15. Episode 2 composite statistical comparison of sensitivity noah_mrf with base case px_acm2 for the US portion

of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.08

- 0.18

- 0.026 0.001

- 0.2689

Mixing Ratio (g/kg)

- 0.90

- 0.18

- 0.008

- 0.024

- 0.2066

Rel. Humidity (%)

- 4.76

- 0.32 0.041

- 0.015

- 0.0816

Wind Speed (m/s) 0.29

- 0.27

- 0.195

- 0.084

- 0.3222

Clouds (%) 4.64 0.49

- 0.013

- 0.007 0.5665

Wind stats bias abserr uerr verr u verr

Wind (deg, m/s) 0.317

- 1.61

- 0.0994

- 0.0984

- 0.1397

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0069

- 0.0472 0.0095

- 0.0306

- 0.0180

- 0.0136

- 0.0147

- 0.0143

0.05 0.0031

- 0.0423 0.0149

- 0.0213

- 0.0042 0.0013

- 0.0043 0.0015

0. 10 0.0049 0.0566 0.0143

- 0.0201

- 0.0021 0.0026

- 0.0049 0.0030

0.25 0.0085 0.0674 0.0179

- 0.0098 0.0065 0.0097 0.0028 0.0124

0.50 0.0094 0.0540 0.0288 0.0238 0.0217 0.0232 0.0316 0.0327

1.00 0.0023

- 0.0311 0.0312 0.0583 0.0249 0.0248 0.0591 0.0419

Table 16. Like table 15, except for 12

- km VISTAS domain.Draft

45

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.25

- 0.13

- 0.017 0.000

- 0.1497

Mixing Ratio (g/kg)

- 0.74

- 0.08 0.020

- 0.009

- 0.0576

Rel. Humidity (%)

- 3.06 0.00 0.018

- 0.001 0.1712

Wind Speed (m/s)

- 0.32

- 0.20

- 0.084

- 0.038

- 0.2410

Clouds (%)

- 1.72

- 0.03

- 0.008

- 0.007

- 0.0666

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.471

- 0.27

- 0.0195

- 0.0240

- 0.0309

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0017

- 0.0352 0.0089

- 0.0194

- 0.0096

- 0.0037

- 0.0055

- 0.0047

0.05 0.0086

- 0.0425 0.0190

- 0.0082 0.0051 0.0103 0.0075 0.0130

0.10 0.0110 0.0771 0.0229

- 0.0123 0.0066 0.0115 0.0048 0.0150

0.25 0.0154 0.1750 0.0351

- 0.0134 0.0145 0.0185 0.0055 0.0261

0.50 0.0144 0.3705 0.0420

- 0.0240 0.0165 0.0193

- 0.0077 0.0293

1.00 0.0073 0.6216 0.0406 0.0000 0.0184 0.0194 0.0074 0.0341

Table 17. Episode 3 composite statistical comparison of sensitivity noah_mrf with base case px_acm2 for the US portion

of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) 0.13 0.01

- 0.032 0.008

- 0.0685

Mixing Ratio (g/kg)

- 0.95

- 0.11 0.010

- 0.036

- 0.1291

Rel. Humidity (%)

- 2.84

- 0.35 0.028

- 0.003

- 0.3691

Wind Speed (m/s)

- 0.17

- 0.30

- 0.159

- 0.085

- 0.3600

Clouds (%) 2.88 0.55 0.001

- 0.004 1.1355

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 1.036

- 1.65

- 0.0814

- 0.0266

- 0.0759

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0044

- 0.0586 0.0225

- 0.0248

- 0.0064 0.0065 0.0080 0.0083

0.05 0.0116

- 0.0506 0.0250

- 0.0089 0.0067 0.0147 0.0141 0.0190

0.10 0.0112

- 0.0575 0.0244

- 0.0085 0.0069 0.0129 0.0104 0.0171

0.25 0.0111 0.0901 0.0257

- 0.0171 0.0043 0.0091

- 0.0004 0.0128

0.50 0.0100 0.1478 0.0 306

- 0.0124 0.0088 0.0117

- 0.0004 0.0173

1.00 0.0047 0.1971 0.0281 0.0083 0.0138 0.0145 0.0130 0.0242

Table 18. Like table 17, except for 12

- km VISTAS domain.Draft

46

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.36

- 0.22

- 0.023

- 0.010

- 0.3301

Mixing Ratio (g/kg)

- 0.31

- 0.14

- 0.033

- 0.017

- 0.1974

Rel. Humidity (%)

- 11.03

- 4.38

- 0.080

- 0.147

- 5.0237

Wind Speed (m/s)

- 0.26

- 0.05

- 0.002 0.024

- 0.0604

Clouds (%) 2.38

- 1.57

- 0.042

- 0.020

- 1.3545

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 1.938

- 0.38

- 0.0476

- 0.0389

- 0.0611

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0282

- 0.1382 0.0682 0.0220

- 0.0318

- 0.0495

- 0.0275

- 0.0496

0.05

- 0.0058

- 0.0490 0.0231 0.0065

- 0.0124

- 0.0152

- 0.0017

- 0.0138

0.10

- 0.0028

- 0.0385 0.0168 0.0060

- 0.0085

- 0.0099 0.0021

- 0.0087

0.25

- 0.0012

- 0.0291 0.0154 0.0036

- 0.0080

- 0.0086 0.0023

- 0.0072

0.50

- 0.0004

- 0.0099 0.0091

- 0.0025

- 0.0068

- 0.0069

- 0.0028

- 0.0060

1.00

- 0.0001

- 0.0284 0.0009

- 0.0183

- 0.0106

- 0.0106

- 0. 0182

- 0.0108

Table 19. Episode 1 composite statistical comparison of sensitivity multi_blkdr with base case px_acm2 for the US

portion of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.75

- 0.38

- 0.025

- 0.017

- 0.5 501

Mixing Ratio (g/kg)

- 0.30

- 0.12

- 0.008

- 0.007

- 0.1244

Rel. Humidity (%)

- 11.86

- 4.44

- 0.072

- 0.159

- 5.5613

Wind Speed (m/s)

- 0.04

- 0.09

- 0.003 0.019

- 0.1052

Clouds (%) 0.45

- 0.03

- 0.003

- 0.002

- 0.0372

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.167

- 1.15

- 0.0333

- 0.0397

- 0.0518

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0496

- 0.1959 0.1099 0.0200

- 0.0707

- 0.1048

- 0.0620

- 0.0935

0.05

- 0.0072

- 0.0412 0.0260 0.0032

- 0.0174

- 0.0219

- 0.0067

- 0.0175

0.10

- 0.0035

- 0.0233 0.0168 0.0005

- 0.0119

- 0.0139

- 0.0038

- 0.0108

0.25

- 0.0017 0.0177 0.0156

- 0.0007

- 0.0108

- 0.0117

- 0.0026

- 0.0089

0.50

- 0.0010 0.0061 0.0137

- 0.0084

- 0.0132

- 0.0137

- 0.0091

- 0.0115

1.00

- 0.0003 0.0150 0.0154

- 0.0021

- 0.0075

- 0.007 6

- 0.0022

- 0.0074

Table 20. Like table 19, except for 12

- km VISTAS domain.Draft

47

Total stats bias abserr r2 ia rmse

Temperature (K) 0.02

- 0.14

- 0.019

- 0.006

- 0.1609

Mixing Ratio (g/kg) 0.15

- 0.02

- 0.030

- 0.010

- 0.0399

Rel. Humidity (%)

- 0.65

- 0.48

- 0.032

- 0.021

- 0.2737

Wind Speed (m/s) 0.10

- 0.12

- 0.073

- 0.018

- 0.1302

Clouds (%) 0.04 0.91 0.011 0.005 0.9705

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.841

- 0.24

- 0.0488

- 0.0678

- 0.0828

Pcp Threshold (in) Acc Bias FAR P OD CSI ETS TSS HSS

0.01

- 0.0254 0.0395 0.0400

- 0.0073

- 0.0257

- 0.0413

- 0.0487

- 0.0499

0.05

- 0.0254

- 0.0682 0.0421

- 0.0083

- 0.0276

- 0.0377

- 0.0411

- 0.0475

0.10

- 0.0229

- 0.0816 0.0399

- 0.0125

- 0.0276

- 0.0342

- 0.0382

- 0.0449

0.25

- 0.0107

- 0.0272 0.0284

- 0.0350

- 0.0250

- 0.0263

- 0.0426

- 0.0373

0.50 0.0007 0.1146 0.0144

- 0.0505

- 0.0176

- 0.0169

- 0.0471

- 0.0261

1.00 0.0023 0.1937

- 0.0100

- 0.0116 0.0019 0.0023

- 0.0092 0.0040

Table 21. Episode 2 composite statistical comparison of sensitivity multi_blkd r with base case px_acm2 for the US

portion of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.22

- 0.02

- 0.019 0.002

- 0.0627

Mixing Ratio (g/kg)

- 0.23 0.05

- 0.009

- 0.002 0.0662

Rel. Humidity (%)

- 1.53 0.68 0.06 0 0.014 1.1438

Wind Speed (m/s)

- 0.11

- 0.23

- 0.108

- 0.044

- 0.2627

Clouds (%) 4.25 0.65

- 0.007

- 0.004 0.8315

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.872

- 0.25

- 0.0930

- 0.1375

- 0.1640

Pcp Threshold (in) Acc Bias FAR POD C SI ETS TSS HSS

0.01

- 0.0136

- 0.0485

- 0.0018

- 0.0381

- 0.0276

- 0.0261

- 0.0280

- 0.0276

0.05

- 0.0051

- 0.0353

- 0.0016

- 0.0291

- 0.0162

- 0.0142

- 0.0204

- 0.0156

0.10

- 0.0035 0.0398 0.0015

- 0.0260

- 0.0139

- 0.0125

- 0.0203

- 0.0145

0.25

- 0.0005 0.0463 0.0024

- 0 .0245

- 0.0101

- 0.0089

- 0.0196

- 0.0114

0.50 0.0020 0.0806 0.0027

- 0.0294

- 0.0089

- 0.0078

- 0.0242

- 0.0113

1.00 0.0059 0.2326

- 0.0171

- 0.0067 0.0065 0.0074

- 0.0005 0.0128

Table 22. Like table 21, except for 12

- km VISTAS domain.Draft

48

Total stats bias abserr r2 ia rmse

Temperature (K) 0.16

- 0.03

- 0.011

- 0.001

- 0.0470

Mixing Ratio (g/kg)

- 0.12 0.01

- 0.016

- 0.008 0.0215

Rel. Humidity (%)

- 0.31

- 0.16

- 0.026

- 0.015

- 0.0298

Wind Speed (m/s) 0.10

- 0.13

- 0.081

- 0.017

- 0.1462

Clouds (%)

- 2.60 0.38 0.0 01

- 0.002 0.2603

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 1.000

- 0.37

- 0.0601

- 0.0617

- 0.0861

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0193 0.0011 0.0256

- 0.0200

- 0.0250

- 0.0310

- 0.0386

- 0.0391

0.05

- 0.0119

- 0.0047 0.0197

- 0.0240

- 0.0213

- 0.0230

- 0.0308

- 0.0299

0.10

- 0.0080 0.0283 0.0177

- 0.0335

- 0.0220

- 0.0220

- 0.0348

- 0.0296

0.25 0.0034 0.1115 0.0018

- 0.0394

- 0.0115

- 0.0095

- 0.0310

- 0.0138

0.50 0.0079 0.2509

- 0.0122

- 0.0374

- 0.0009 0.0008

- 0.0274 0.0 011

1.00 0.0052 0.5231

- 0.0052

- 0.0403

- 0.0036

- 0.0028

- 0.0345

- 0.0050

Table 23. Episode 3 composite statistical comparison of sensitivity multi_blkdr with base case px_acm2 for the US

portion of the 36

- km domain is shown.

Total stats bias abserr r 2 ia rmse

Temperature (K)

- 0.14 0.07

- 0.009 0.007 0.0185

Mixing Ratio (g/kg)

- 0.40

- 0.02

- 0.004

- 0.006

- 0.0280

Rel. Humidity (%)

- 0.41

- 0.12 0.014 0.008

- 0.0297

Wind Speed (m/s) 0.02

- 0.22

- 0.119

- 0.050

- 0.2555

Clouds (%) 2.84 0.87 0.013 0. 006 1.2969

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.528 0.09

- 0.0884

- 0.0628

- 0.1066

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0069

- 0.1108

- 0.0247

- 0.0624

- 0.0319

- 0.0115

- 0.0155

- 0.0148

0.05 0.0041

- 0.100 3

- 0.0205

- 0.0445

- 0.0159

- 0.0040

- 0.0137

- 0.0052

0.10 0.0012

- 0.0871

- 0.0066

- 0.0415

- 0.0174

- 0.0108

- 0.0239

- 0.0145

0.25 0.0001 0.0839 0.0063

- 0.0417

- 0.0187

- 0.0157

- 0.0333

- 0.0223

0.50 0.0033 0.1390 0.0062

- 0.0492

- 0.0176

- 0.0153

- 0.0412

- 0.0232

1.00 0.0033 0.2018

- 0.0040

- 0.0250

- 0.0033

- 0.0026

- 0.0210

- 0.0045

Table 24. Like table 23, except for 12

- km VISTAS domain.Draft

49

Total stats bias abserr r2 ia rmse

Temperature (K)

- 1.30

- 0.90

- 0.046

- 0.030

- 1.1408

Mixing Ratio (g/kg)

- 0.06

- 0.08

- 0 .022

- 0.007

- 0.0803

Rel. Humidity (%)

- 10.88

- 4.71

- 0.077

- 0.125

- 5.2197

Wind Speed (m/s)

- 0.06 0.03 0.026 0.028 0.0442

Clouds (%) 1.19

- 0.98

- 0.016

- 0.011

- 0.2897

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.668 0.35 0.0371 0. 0318 0.0487

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0434

- 0.2155

- 0.0990 0.0351

- 0.0465

- 0.0732

- 0.0418

- 0.0747

0.05

- 0.0015

- 0.0123

- 0.0063 0.0013

- 0.0036

- 0.0043

- 0.0008

- 0.0039

0.10 0.0003 0.0009 0.0015 0.0011 0.0017 0.0017 0.0013 0.0015

0.25

- 0.0001 0.0093 0.0016

- 0.0050

- 0.0019

- 0.0018

- 0.0047

- 0.0016

0.50

- 0.0002

- 0.0047

- 0.0015

- 0.0056

- 0.0041

- 0.0042

- 0.0056

- 0.0036

1.00

- 0.0001

- 0.0142

- 0.0051

- 0.0132

- 0.0095

- 0.0096

- 0.0132

- 0.0098

Table 25. Episode 1 composi te statistical comparison of sensitivity noah_eta

- my with base case px_acm2 for the US

portion of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 1.08

- 0.56

- 0.012

- 0.018

- 0.7253

Mixing Ratio (g/kg)

- 0.07

- 0.06

- 0.0 10

- 0.003

- 0.0629

Rel. Humidity (%)

- 8.65

- 2.57

- 0.041

- 0.092

- 3.3834

Wind Speed (m/s) 0.29 0.09 0.050 0.042 0.1122

Clouds (%)

- 0.05 0.00 0.001 0.001

- 0.0017

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.568

- 0.15 0.0837 0.0833 0.1179

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0759

- 0.3018

- 0.1553 0.0315

- 0.1011

- 0.1512

- 0.0944

- 0.1393

0.05

- 0.0080

- 0.0418

- 0.0277 0.0018

- 0.0196

- 0.0244

- 0.0087

- 0.0195

0.10

- 0.0023

- 0.0065

- 0.0088

- 0.0040

- 0.0090

- 0.010 3

- 0.0060

- 0.0079

0.25

- 0.0009 0.0020

- 0.0058

- 0.0039

- 0.0067

- 0.0071

- 0.0045

- 0.0054

0.50

- 0.0007 0.0135

- 0.0115

- 0.0010

- 0.0073

- 0.0077

- 0.0017

- 0.0064

1.00

- 0.0002 0.0539

- 0.0179 0.0221 0.0065 0.0063 0.0217 0.0061

Table 26. Like table 25, excep t for 12

- km VISTAS domain.Draft

50

Total stats bias abserr r2 ia rmse

Temperature (K)

- 1.15

- 0.71

- 0.043

- 0.022

- 0.8789

Mixing Ratio (g/kg) 0.00 0.05 0.009 0.003 0.0895

Rel. Humidity (%)

- 5.67

- 0.74 0.032

- 0.008

- 0.5429

Wind Speed (m/s)

- 0.24 0.00 0 .026 0.002

- 0.0108

Clouds (%)

- 0.54 0.45

- 0.006

- 0.008 0.2731

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 1.138 0.31 0.0335 0.0337 0.0475

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0176 0.1064

- 0.0459 0.0344

- 0. 0036

- 0.0269

- 0.0293

- 0.0321

0.05

- 0.0149

- 0.1029

- 0.0288 0.0265

- 0.0046

- 0.0146

- 0.0062

- 0.0180

0.10

- 0.0181

- 0.1142

- 0.0304 0.0150

- 0.0128

- 0.0197

- 0.0122

- 0.0255

0.25

- 0.0085

- 0.0389

- 0.0208

- 0.0193

- 0.0169

- 0.0182

- 0.0264

- 0.0256

0.50 0.0045 0. 1814

- 0.0016

- 0.0465

- 0.0104

- 0.0091

- 0.0393

- 0.0139

1.00 0.0040 0.3590 0.0130

- 0.0311

- 0.0014

- 0.0006

- 0.0268

- 0.0010

Table 27. Episode 2 composite statistical comparison of sensitivity noah_eta

- my with base case px_acm2 for the US

portion of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.81

- 0.45

- 0.025

- 0.014

- 0.6594

Mixing Ratio (g/kg) 0.04 0.05

- 0.001 0.003 0.0776

Rel. Humidity (%)

- 1.75 1.04 0.105 0.029 1.7577

Wind Speed (m/s) 0.29 0.09 0.023 0.01 9 0.0956

Clouds (%) 4.90 0.36

- 0.013

- 0.007 0.4730

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 1.641 0.26 0.0688 0.0883 0.1115

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0024 0.0915

- 0.0481 0.0853 0.0290 0.0056 0.0073 0.0059

0.05

- 0.0126

- 0.1823

- 0.0444 0.0737 0.0113

- 0.0074 0.0121

- 0.0081

0.10

- 0.0169

- 0.1763

- 0.0389 0.0592 0.0027

- 0.0113 0.0109

- 0.0131

0.25

- 0.0111

- 0.0996

- 0.0201 0.0189

- 0.0054

- 0.0098 0.0012

- 0.0125

0.50 0.0045 0.1230 0.0012

- 0.0375

- 0.0091

- 0.0072

- 0.0288

- 0.0105

1.00 0.0101 0.4790 0.0112

- 0.0519

- 0.0057

- 0.0035

- 0.0401

- 0.0060

Table 28. Like table 27, except for 12

- km VISTAS domain.Draft

51

Total stats bias abserr r2 ia rmse

Temperature (K)

- 1.08

- 0.65

- 0.052

- 0.022

- 0.7857

Mixi ng Ratio (g/kg) 0.24 0.07 0.013 0.006 0.1111

Rel. Humidity (%)

- 5.41

- 0.81 0.018

- 0.011

- 0.8111

Wind Speed (m/s)

- 0.23

- 0.02 0.032 0.000

- 0.0192

Clouds (%)

- 3.24 0.25

- 0.004

- 0.005 0.0162

Wind stats bias abserr uerr verr uverr

Wind (deg, m

/s) 1.051 0.49 0.0283 0.0293 0.0407

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0211 0.1203

- 0.0560 0.0376

- 0.0031

- 0.0305

- 0.0353

- 0.0385

0.05

- 0.0115

- 0.0834

- 0.0240 0.0222

- 0.0018

- 0.0099

- 0.0033

- 0.0128

0.10

- 0.0108

- 0.0680

- 0 .0201 0.0081

- 0.0076

- 0.0118

- 0.0076

- 0.0157

0.25

- 0.0009 0.0351

- 0.0070

- 0.0215

- 0.0093

- 0.0088

- 0.0200

- 0.0128

0.50 0.0058 0.2353

- 0.0001

- 0.0527

- 0.0102

- 0.0087

- 0.0441

- 0.0137

1.00 0.0061 0.5801 0.0139

- 0.0332 0.0012 0.0022

- 0.0267 0.0038

Tabl e 29. Episode 3 composite statistical comparison of sensitivity noah_eta

- my with base case px_acm2 for

the US portion of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.52

- 0.27

- 0.052

- 0.005

- 0.4031

Mixing Ratio (g/kg)

- 0.12 0.00

- 0.003 0.000

- 0.0028

Rel. Humidity (%)

- 1.43

- 0.02 0.022 0.006

- 0.1437

Wind Speed (m/s)

- 0.03 0.03 0.033 0.014 0.0470

Clouds (%) 0.01 1.11 0.020 0.010 1.3794

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.358 0. 67 0.0526 0.0891 0.1007

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0031 0.0764

- 0.0417 0.1071 0.0419 0.0064 0.0095 0.0082

0.05

- 0.0133

- 0.1067

- 0.0382 0.0940 0.0256 0.0017 0.0218 0.0022

0.10

- 0.0267

- 0.1942

- 0.0475 0.0579 0.0005

- 0.0175

- 0.0013

- 0.0237

0.25

- 0.0168

- 0.0750

- 0.0384

- 0.0136

- 0.0223

- 0.0264

- 0.0309

- 0.0380

0.50 0.0068 0.1725 0.0080

- 0.0444

- 0.0108

- 0.0077

- 0.0331

- 0.0115

1.00 0.0093 0.5063 0.0305

- 0.0444 0.0012 0.0034

- 0.0339 0.0056

Table 30. Like table 29, e xcept for 12

- km VISTAS domain.Draft

52 

## 5 Auxiliary Results

The results from the five main sensitivity runs indicate that the px_acm2 configuration is overall the

preferred option for annual modeling. Still there are numerous model options that are not tested in t he above

configurations. Accordingly we have conducted a variety of auxiliary sensitivity tests with limited analyses.

These include:

• px_acm3 : Like px_acm2, except with soil moisture/temperature initialization for P

- X soil

layer 2 from EDAS soil layer 3 (40

- 100 cm).

• px_acm4 : Like px_acm2, except it uses the SFCFDDA file from RAWINS and

MMINPUT files processed using RAWINS data rather than REGRID data.

• px_acm5 : Like px_acm2, except it uses Reisner 2 microphysics rather than Reisner 1.

• px_acm6 : Like px_ acm2, except analysis nudging strengths aloft for wind/temps are

increased from 2.5E

- 4, 1.0E

- 4 (36/12 km) to 3.0E

- 4, 3.0E

- 4.

• px_acm7 : Like px_acm2, except the SFCFDDA files from RAWINS are used instead of

those from LITTLE_R.

• px_acm8 : Like px_acm2, excep t the MMINPUT files are processed using data from

LITTLE_R rather than from REGRID.

• px_acm9 : Like px_acm3, except the initialization for soil temperature/moisture is based

on a 25% weighting of EDAS soil layer 1 (0

- 10 cm) values and a 75%

weighting from E

DAS soil layer 3 (40

- 100 cm). Code supplied by Aijun Xiu;

executed for episode 3 only.

• px_acm_n : Like px_acm2, except surface analysis nudging is turned on for temp/moisture

at a strength of 2.5E

- 4, 1.0E

- 4 (36, 12 km temps) and 1.0E

- 4, 1.0E

- 4 (36, 12

km moisture).

• px_acm_n2 : Like px_acm_n, except surface analysis nudging for moisture ss reduced to

1.0E

- 5, 1.0E

- 5 (36, 12 km). Run only for episode 2.

• s0px_acm2 : Like px_acm2, except snow cover effects are not considered (IFSNOW=0) as

opposed to IFSNOW=1 in px_acm2. Run only for episode 1.

• s2px_acm2 : Like px_acm2, except the simple snow model is employed (IFSNOW=2) as

opposed to IFSNOW=1 in px_acm2. Run only for episode 1.

• n0_px_acm2 : Like px_acm2, except soil moisture nudging is turned off.

For the most

part the analyses of these runs will be statistical and will follow the statistical table format

detailed in the prior section. The goal is to quickly determine whether the option being tested proves beneficial.

The analyses are limited to the wintertime episode 1 and the longer of the two summertime episodes, episode 2.

Recall that the px_acm2 run, while performing the best of the original five sensitivities, failed to meet the

statistical “benchmark” for temperature for episode 1. The px_acm3 configurati on is an attempt to rectify this

bias by initializing the model from the single most appropriate EDAS soil layer (40

- 100 cm). This run included

an error correction pass that reduced the effect of “missing” values in the EDAS data set. Tables 31

- 32 show the

results of this sensitivity for episode 1 for both grid resolutions modeled. Note the vast improvement in

temperature bias in both tables. Most of the other variables are relatively unaffected.Draft

53

Total stats bias abserr r2 ia rmse

Temperature (K) 0.65 0.17

- 0.002 0.003 0.2029

Mixing Ratio (g/kg) 0.05 0.00

- 0.002 0.000

- 0.0035

Rel. Humidity (%) 2.53 0.18 0.025 0.015 0.1032

Wind Speed (m/s)

- 0.03 0.02 0.010 0.009 0.0192

Clouds (%)

- 0.02

- 0.01 0.001 0.000 0.0497

Wind stats bias abserr uer r verr uverr

Wind (deg, m/s)

- 0.643

- 0.02 0.0072 0.0026 0.0070

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0028

- 0.0257

- 0.0101 0.0082

- 0.0014

- 0.0038 0.0007

- 0.0037

0.05

- 0.0016

- 0.0193

- 0.0075 0.0047

- 0.0026

- 0.0034 0.0019

- 0.00 31

0.10

- 0.0012

- 0.0252

- 0.0086 0.0069

- 0.0023

- 0.0030 0.0048

- 0.0026

0.25

- 0.0007

- 0.0199

- 0.0092 0.0039

- 0.0038

- 0.0041 0.0031

- 0.0035

0.50

- 0.0002

- 0.0251

- 0.0083 0.0086 0.0000

- 0.0001 0.0082

- 0.0001

1.00 0.0000 0.0152

- 0.0014 0.0081 0.0039 0.00 38 0.0081 0.0039

Table 31. Episode 1 composite statistical comparison of sensitivity px_acm3 with base case px_acm2 for the US portion

of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) 0.96 0.13

- 0.004 0.002 0.1753

Mixing Ratio (g/kg) 0.03 0.00 0.003 0.001 0.0160

Rel. Humidity (%) 3.05 0.39 0.043 0.025 0.2853

Wind Speed (m/s)

- 0.09 0.00 0.022 0.011 0.0014

Clouds (%) 0.19

- 0.09

- 0.002

- 0.001

- 0.0258

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.913 0.13 0.0089

- 0.0115

- 0.0025

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0028

- 0.0285

- 0.0119 0.0100

- 0.0021

- 0.0049 0.0013

- 0.0041

0.05

- 0.0013

- 0.0182

- 0.0076 0.0059

- 0.0019

- 0.0029 0.0027

- 0.0023

0.10

- 0.0013

- 0.024 2

- 0.0105 0.0079

- 0.0024

- 0.0034 0.0049

- 0.0026

0.25

- 0.0009 0.0187

- 0.0113 0.0043

- 0.0044

- 0.0050 0.0029

- 0.0038

0.50

- 0.0004 0.0186

- 0.0093 0.0049

- 0.0022

- 0.0026 0.0043

- 0.0021

1.00

- 0.0002 0.0146

- 0.0105 0.0016

- 0.0032

- 0.0032 0.0015

- 0.0031

T able 32. Like table 31, except for 12

- km VISTAS domain.Draft

54

Unfortunately the improvements noted in the wintertime episode do not translate to the summer episode

(tables 33

- 34). In fact the results indicate a degraded bias result, with the temperatures more s ignificantly cold

biased for both resolutions modeled. The remaining variables tend to be relatively unaffected, generally

showing a very slight degradation in performance. If this sensitivity had produced superior results for all

episodes, we would recomm end further efforts to possibly improve the initialization of the P

- X module in

MM5. Given the ad hoc nature of this sensitivity test that only yields mixed

- bag results at best, we do not

recommend applying this approach for the annual modeling.

An alterna tive approach to using LITTLE_R in the MM5 preprocessing stream is to use RAWINS.

Functionally they serve the same purpose, which is to use observations to “improve” the first

- guess and analysis

fields that drive MM5. NCAR recommends that users employ LITT LE_R, which is a newer, more flexible

program, but also somewhat more difficult to implement. (Unfortunately when this project started the

documentation for LITTLE_R was less than stellar. The updated documentation indicates that upper air data

“probably s hould” be added to the sfc_obs files; we plan to do just that for the annual modeling, but these tests

will not have that “benefit”.) Prior modeling results found little difference between using these programs and

not using them at all. To use P

- X in the r ecommended manner (i.e. with indirect soil nudging) requires that the

SFCFDDA file be available, which necessitates running one of those objective analysis preprocessors. The px

acm2 configuration used LITTLE_R only for that purpose. The px_acm4 configurat ion is actually a two

- fold

sensitivity in that it tests the effects of 1) running RAWINS rather than LITTLE_R to generate the SFCFDDA

files, and 2) using objective analysis (RAWINS, in this case) output as input to INTERPF, as opposed to using

REGRID outpu

t. Tables 35

- 36 show the results for the wintertime episode. In general we see improvement for

most of the variables, especially for the VISTAS 12

- km domain. Precipitation results are very slightly degraded,

but not significantly so. The summertime results for this sensitivity are shown in tables 37

- 38, and they indicate

no clear overall improvement/degradation.

The Reisner 2 microphysics package is sophisticated and computationally expensive, but is believed to

incorporate the appropriate science necessary to properly model precipitation over a variety of conditions.

Tables 39

- 40 show the statistical differences for the wintertime episode. Interestingly enough this px_acm5

sensitivity yields improved temperature and cloud coverage performance, but the preci pitation skill scores show

a very slight degradation. Summertime performance for this sensitivity (tables 41

- 42) show generally degraded

performance. These results do not justify the increased computational/storage cost of implementing Reisner 2

microphysi

cs.

Sensitivity test px_acm6 is designed to show the effects of nudging strength. The very slight increase in

nudging strength at 36

- km is not expected to have a large effect on the results, and the episode 1 statistical

difference table (table 43) bears t his out. The 12

- km results (table 44) are a little more interesting since the

nudging strength aloft is increased more dramatically. The idea is that above the planetary boundary layer

synoptic conditions prevail, thus making analysis nudging at a reasonab ly strong level appropriate. In the

boundary layer itself the wind nudging is kept at a relatively weak level. We do note an improvement in the

temperature and precipitation metrics, but a slight degradation in mixing ratio and winds. The summer case

expec tedly reveals little change at 36

- km resolution (table 45), while the 12

- km results (table 46) show

decreased skill for almost all variables, with temperature and winds being the notable exceptions. Overall it is

hard to justify this configuration as an im provement over the base case.Draft

55

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.62

- 0.23

- 0.018

- 0.007

- 0.2704

Mixing Ratio (g/kg)

- 0.25

- 0.03

- 0.006

- 0.004

- 0.0304

Rel. Humidity (%)

- 0.09

- 0.18

- 0.009

- 0.003

- 0.2014

Wind Speed (m/s) 0. 00

- 0.01

- 0.008

- 0.005

- 0.0114

Clouds (%) 0.15 0.06 0.001 0.001

- 0.0039

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.147 0.07

- 0.0061

- 0.0036

- 0.0068

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0068

- 0.0401 0.00 33

- 0.0273

- 0.0168

- 0.0126

- 0.0159

- 0.0149

0.05 0.0005 0.0599 0.0060

- 0.0292

- 0.0110

- 0.0069

- 0.0160

- 0.0085

0.10 0.0043 0.0794 0.0079

- 0.0297

- 0.0071

- 0.0034

- 0.0162

- 0.0044

0.25 0.0075 0.1285 0.0101

- 0.0305

- 0.0022 0.0003

- 0.0184 0.0004

0.50 0.00 61 0.1842 0.0082

- 0.0326

- 0.0025

- 0.0010

- 0.0245

- 0.0016

1.00 0.0031 0.2807 0.0104

- 0.0232

- 0.0004 0.0001

- 0.0198 0.0003

Table 33. Episode 2 composite statistical comparison of sensitivity px_acm3 with base case px_acm2 for the US portion

of the 36

- km

domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.37

- 0.05

- 0.006

- 0.001

- 0.0514

Mixing Ratio (g/kg)

- 0.08 0.01 0.001 0.001 0.0144

Rel. Humidity (%) 0.60

- 0.11

- 0.005

- 0.002

- 0.0932

Wind Speed (m/s) 0.01 0.00

- 0.011

- 0 .006

- 0.0085

Clouds (%)

- 0.25 0.09 0.003 0.003 0.0358

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.273

- 0.07

- 0.0054

- 0.0047

- 0.0071

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0050

- 0.0409 0.0094

- 0.0255

- 0.01 41

- 0.0099

- 0.0107

- 0.0103

0.05 0.0019

- 0.0455 0.0138

- 0.0246

- 0.0068

- 0.0014

- 0.0076

- 0.0015

0.10 0.0050 0.0672 0.0159

- 0.0252

- 0.0039 0.0015

- 0.0077 0.0017

0.25 0.0080 0.0981 0.0162

- 0.0267

- 0.0009 0.0031

- 0.0110 0.0040

0.50 0.0071 0.1388 0.0097

- 0.0312

- 0.0031

- 0.0008

- 0.0203

- 0.0012

1.00 0.0051 0.2605

- 0.0003

- 0.0361

- 0.0067

- 0.0056

- 0.0299

- 0.0096

Table 34. Like table 33, except for 12

- km VISTAS domain.Draft

56

Total stats bias abserr r2 ia rmse

Temperature (K) 0.08 0.07 0.004 0.002 0.0950

Mixing Ratio (g/kg)

- 0.02 0.01 0.003 0.000 0.0093

Rel. Humidity (%)

- 0.18 0.09 0.009 0.003 0.1488

Wind Speed (m/s) 0.02

- 0.03

- 0.024

- 0.010

- 0.0397

Clouds (%)

- 0.22

- 0.15

- 0.006

- 0.003

- 0.2251

Wind stats bias abserr uerr verr uverr

Wind ( deg, m/s)

- 0.255

- 0.62

- 0.0339

- 0.0300

- 0.0452

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0033

- 0.0153

- 0.0087 0.0022

- 0.0041

- 0.0062

- 0.0035

- 0.0061

0.05

- 0.0010

- 0.0074

- 0.0042 0.0004

- 0.0026

- 0.0031

- 0.0009

- 0.0028

0.10

- 0.001 0

- 0.0073

- 0.0050

- 0.0009

- 0.0039

- 0.0044

- 0.0018

- 0.0038

0.25

- 0.0001 0.0203 0.0032

- 0.0110

- 0.0043

- 0.0042

- 0.0104

- 0.0036

0.50 0.0000

- 0.0346 0.0079

- 0.0170

- 0.0055

- 0.0054

- 0.0165

- 0.0048

1.00 0.0001

- 0.0264 0.0130

- 0.0051 0.0028 0.0028

- 0.0049 0 .0028

Table 35. Episode 1 composite statistical comparison of sensitivity px_acm4 with base case px_acm2 for the US portion

of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.05 0.05 0.009 0.002 0.0709

Mixing R atio (g/kg) 0.03 0.01 0.002 0.001 0.0147

Rel. Humidity (%) 0.12 0.11 0.008 0.003 0.1409

Wind Speed (m/s)

- 0.01 0.00 0.003 0.001 0.0005

Clouds (%) 0.34 0.25 0.006 0.003 0.2262

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.039 0. 04

- 0.0010 0.0036 0.0019

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0035

- 0.0029

- 0.0061

- 0.0040

- 0.0071

- 0.0092

- 0.0072

- 0.0077

0.05

- 0.0002 0.0202 0.0049

- 0.0106

- 0.0034

- 0.0032

- 0.0081

- 0.0025

0.10

- 0.0019 0.0205

- 0.0001

- 0.01 62

- 0.0110

- 0.0117

- 0.0155

- 0.0091

0.25

- 0.0018

- 0.0322 0.0004

- 0.0259

- 0.0186

- 0.0192

- 0.0253

- 0.0147

0.50 0.0001

- 0.0513 0.0145

- 0.0250

- 0.0083

- 0.0081

- 0.0237

- 0.0068

1.00

- 0.0004 0.0103

- 0.0209

- 0.0097

- 0.0145

- 0.0146

- 0.0098

- 0.0143

Table 36. Like table 35, except for 12

- km VISTAS domain.Draft

57

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.10 0.01 0.005 0.001 0.0265

Mixing Ratio (g/kg) 0.02 0.01 0.003 0.001 0.0200

Rel. Humidity (%) 0.18 0.22 0.016 0.005 0.3339

Wind Speed (m/s)

- 0.01

- 0.03

- 0.030

- 0.016

- 0.0438

Clouds (%) 0.05

- 0.13

- 0.004

- 0.003

- 0.1747

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.087

- 0.91

- 0.0344

- 0.0341

- 0.0484

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0027

- 0.00 96 0.0060

- 0.0020 0.0018 0.0044 0.0048 0.0051

0.05 0.0030 0.0180 0.0063

- 0.0040 0.0016 0.0036 0.0021 0.0043

0.10 0.0002 0.0281 0.0000

- 0.0137

- 0.0052

- 0.0041

- 0.0098

- 0.0053

0.25 0.0033 0.0683 0.0031

- 0.0193

- 0.0031

- 0.0019

- 0.0132

- 0.0026

0.50 0.0 018 0.0780

- 0.0014

- 0.0213

- 0.0050

- 0.0044

- 0.0182

- 0.0068

1.00 0.0012 0.0465 0.0179 0.0211 0.0118 0.0118 0.0219 0.0202

Table 37. Episode 2 composite statistical comparison of sensitivity px_acm4 with base case px_acm2 for the US portion

of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.18 0.01 0.003 0.002 0.0145

Mixing Ratio (g/kg)

- 0.06 0.02 0.004 0.002 0.0246

Rel. Humidity (%) 0.37 0.14 0.009 0.003 0.2255

Wind Speed (m/s) 0.01 0.00

- 0.008

- 0.005

- 0. 0036

Clouds (%) 0.50 0.01

- 0.001

- 0.001 0.0901

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.021

- 0.16

- 0.0117

- 0.0020

- 0.0094

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0002

- 0.0212 0.0081

- 0.0104

- 0.0033 0.000 2 0.0000 0.0003

0.05 0.0014

- 0.0212 0.0079

- 0.0131

- 0.0031

- 0.0001

- 0.0034 0.0000

0.10 0.0010 0.0379 0.0057

- 0.0174

- 0.0055

- 0.0030

- 0.0088

- 0.0035

0.25 0.0060 0.0729 0.0120

- 0.0196

- 0.0005 0.0025

- 0.0079 0.0033

0.50 0.0076 0.1079 0.0156

- 0.0131 0. 0048 0.0067

- 0.0034 0.0096

1.00 0.0050 0.1819 0.0174 0.0015 0.0082 0.0090 0.0066 0.0154

Table 38. Like table 37, except for 12

- km VISTAS domain.Draft

58

Total stats bias abserr r2 ia rmse

Temperature (K) 0.12 0.07 0.003 0.002 0.0972

Mixing Ratio (g/kg

)

- 0.01 0.00 0.000 0.000 0.0004

Rel. Humidity (%)

- 0.01

- 0.06

- 0.001 0.000

- 0.0488

Wind Speed (m/s) 0.00 0.01 0.001 0.001 0.0018

Clouds (%) 2.28 0.95 0.024 0.017 0.6766

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.046

- 0.01 0. 0017 0.0008 0.0018

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0031

- 0.0799 0.0283

- 0.0408

- 0.0094

- 0.0030

- 0.0190

- 0.0029

0.05 0.0033 0.0860 0.0247

- 0.0329

- 0.0025 0.0001

- 0.0228 0.0001

0.10 0.0018 0.0700 0.0190

- 0.0265

- 0.0017

- 0 .0006

- 0.0213

- 0.0005

0.25 0.0007 0.0505 0.0169

- 0.0186

- 0.0002 0.0002

- 0.0168 0.0001

0.50

- 0.0003

- 0.0442 0.0046

- 0.0269

- 0.0133

- 0.0132

- 0.0264

- 0.0117

1.00

- 0.0001

- 0.0325

- 0.0015

- 0.0213

- 0.0126

- 0.0126

- 0.0212

- 0.0129

Table 39. Episode 1 comp osite statistical comparison of sensitivity px_acm5 with base case px_acm2 for the US portion

of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) 0.07 0.05 0.003 0.001 0.0578

Mixing Ratio (g/kg)

- 0.01 0.00 0.000 0.000

- 0.0024

Rel. Humidity (%) 0.01

- 0.02 0.006 0.002

- 0.0119

Wind Speed (m/s) 0.00 0.00 0.001 0.000 0.0018

Clouds (%)

- 1.42 0.26 0.011 0.005 0.1818

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.055

- 0.02 0.0006 0.0017 0.0016

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0059

- 0.0605 0.0509

- 0.0538

- 0.0044 0.0049

- 0.0200 0.0040

0.05 0.0037 0.0531 0.0305

- 0.0292 0.0019 0.0053

- 0.0169 0.0041

0.10 0.0014 0.0130 0.0232

- 0.0271

- 0.0025

- 0.0010

- 0.0204

- 0.0008

0.2 5

- 0.0006

- 0.0400 0.0103

- 0.0232

- 0.0104

- 0.0104

- 0.0215

- 0.0079

0.50 0.0001

- 0.0454 0.0133

- 0.0217

- 0.0067

- 0.0065

- 0.0205

- 0.0055

1.00 0.0001

- 0.0020 0.0067 0.0038 0.0053 0.0053 0.0040 0.0052

Table 40. Like table 39, except for 12

- km VISTAS domain

.Draft

59

Total stats bias abserr r2 ia rmse

Temperature (K) 0.07

- 0.03

- 0.002

- 0.001

- 0.0285

Mixing Ratio (g/kg) 0.01

- 0.02

- 0.004

- 0.002

- 0.0248

Rel. Humidity (%)

- 0.20

- 0.12

- 0.009

- 0.003

- 0.1517

Wind Speed (m/s)

- 0.02 0.00

- 0.001

- 0.002

- 0.0034

Clouds (%)

- 0.30

- 0.11

- 0.001 0.000

- 0.2593

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.122

- 0.02 0.0000

- 0.0005

- 0.0003

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0066

- 0.0324 0.0013

- 0.0232

- 0.0150

- 0.0121

- 0 .0151

- 0.0143

0.05

- 0.0023 0.0426 0.0000

- 0.0252

- 0.0122

- 0.0099

- 0.0177

- 0.0122

0.10 0.0006 0.0525 0.0006

- 0.0250

- 0.0092

- 0.0072

- 0.0175

- 0.0092

0.25 0.0046 0.1108 0.0029

- 0.0344

- 0.0072

- 0.0052

- 0.0249

- 0.0074

0.50 0.0060 0.1887 0.0071

- 0.0354

- 0.0037

- 0.0022

- 0.0272

- 0.0034

1.00 0.0024 0.2219 0.0059

- 0.0210

- 0.0017

- 0.0013

- 0.0184

- 0.0021

Table 41. Episode 2 composite statistical comparison of sensitivity px_acm5 with base case px_acm2 for the US portion

of the 36

- km domain is shown.

Tot al stats bias abserr r2 ia rmse

Temperature (K) 0.04 0.00 0.001

- 0.001

- 0.0038

Mixing Ratio (g/kg) 0.00

- 0.02

- 0.003

- 0.001

- 0.0145

Rel. Humidity (%)

- 0.21

- 0.06

- 0.006

- 0.003

- 0.0767

Wind Speed (m/s) 0.01 0.00

- 0.004

- 0.001

- 0.0015

Clouds (%)

- 0.90

- 0.13 0.001 0.001

- 0.3080

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s)

- 0.189

- 0.06

- 0.0011

- 0.0002

- 0.0008

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0087

- 0.0369 0.0034

- 0.0273

- 0.0186

- 0.0169

- 0.0180

- 0 .0177

0.05

- 0.0034

- 0.0169 0.0002

- 0.0176

- 0.0102

- 0.0093

- 0.0130

- 0.0101

0.10

- 0.0022 0.0294

- 0.0004

- 0.0185

- 0.0095

- 0.0084

- 0.0140

- 0.0097

0.25 0.0020 0.0616 0.0029

- 0.0252

- 0.0075

- 0.0055

- 0.0172

- 0.0070

0.50 0.0073 0.1220 0.0128

- 0.0214 0.0011 0.0032

- 0.0112 0.0046

1.00 0.0038 0.1704 0.0055

- 0.0149

- 0.0001 0.0006

- 0.0106 0.0011

Table 42. Like table 41, except for 12

- km VISTAS domain.Draft

60

Total stats bias abserr r2 ia rmse

Temperature (K) 0.01 0.00 0.000 0.000 0.0053

Mixing Ratio (g/kg

) 0.00 0.00 0.000 0.000

- 0.0009

Rel. Humidity (%) 0.07 0.07 0.003 0.002 0.0799

Wind Speed (m/s)

- 0.02 0.00

- 0.002

- 0.002

- 0.0041

Clouds (%)

- 0.03 0.02 0.001 0.000 0.0167

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.145

- 0.04

- 0 .0006

- 0.0011

- 0.0011

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0001 0.0034 0.0010

- 0.0014

- 0.0003 0.0000

- 0.0006 0.0000

0.05 0.0003 0.0041 0.0013

- 0.0012 0.0002 0.0004

- 0.0007 0.0003

0.10 0.0004 0.0025 0.0023 0.0009 0.0021 0.002 1 0.0013 0.0019

0.25 0.0002 0.0000 0.0018 0.0019 0.0023 0.0024 0.0020 0.0019

0.50

- 0.0001

- 0.0105

- 0.0025 0.0046 0.0012 0.0011 0.0045 0.0010

1.00 0.0000 0.0031

- 0.0021 0.0000

- 0.0009

- 0.0010 0.0000

- 0.0010

Table 43. Episode 1 composite statistical comparison of sensitivity px_acm6 with base case px_acm2 for the US portion

of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K) 0.16 0.11 0.008 0.004 0.1259

Mixing Ratio (g/kg)

- 0.03

- 0.03

- 0.010

- 0.003

- 0.0530

Re l. Humidity (%) 0.33 0.38 0.023 0.011 0.4391

Wind Speed (m/s) 0.11

- 0.01

- 0.026

- 0.017

- 0.0069

Clouds (%) 0.28

- 0.19

- 0.006

- 0.003

- 0.1165

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 1.201

- 0.22

- 0.0119 0.0096

- 0.0012

Pcp Thre shold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01 0.0051 0.0104 0.0108 0.0028 0.0097 0.0132 0.0090 0.0108

0.05 0.0041 0.0060 0.0110 0.0070 0.0128 0.0151 0.0104 0.0118

0.10 0.0041

- 0.0031 0.0119 0.0149 0.0187 0.0208 0.0170 0.0158

0.25 0.0012 0.0277

- 0 .0022 0.0204 0.0132 0.0136 0.0197 0.0100

0.50 0.0006 0.0197

- 0.0140 0.0483 0.0218 0.0216 0.0466 0.0177

1.00

- 0.0001 0.0324

- 0.0090 0.0149 0.0058 0.0057 0.0148 0.0056

Table 44. Like table 43, except for 12

- km VISTAS domain.Draft

61

Total stats bias abserr r 2 ia rmse

Temperature (K) 0.00 0.00 0.001 0.001 0.0059

Mixing Ratio (g/kg)

- 0.01

- 0.01

- 0.002

- 0.001

- 0.0132

Rel. Humidity (%)

- 0.06 0.01 0.001 0.001 0.0190

Wind Speed (m/s) 0.00 0.00

- 0.001

- 0.001

- 0.0023

Clouds (%) 0.01

- 0.14

- 0.003

- 0.00 3

- 0.2124

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.026

- 0.12

- 0.0029

- 0.0027

- 0.0039

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0005 0.0045

- 0.0019 0.0017 0.0002

- 0.0007

- 0.0008

- 0.0008

0.05

- 0.0023

- 0.0094

- 0.0042 0.0009

- 0.0019

- 0.0031

- 0.0028

- 0.0038

0.10

- 0.0031

- 0.0116

- 0.0057

- 0.0014

- 0.0038

- 0.0048

- 0.0050

- 0.0061

0.25

- 0.0018

- 0.0141

- 0.0041

- 0.0014

- 0.0027

- 0.0031

- 0.0033

- 0.0043

0.50

- 0.0015

- 0.0280

- 0.0046

- 0.0011

- 0.0028

- 0.0030

- 0.0027

- 0.0 046

1.00

- 0.0005

- 0.0485

- 0.0017 0.0037

- 0.0001

- 0.0002 0.0030

- 0.0003

Table 45. Episode 2 composite statistical comparison of sensitivity px_acm6 with base case px_acm2 for the US portion

of the 36

- km domain is shown.

Total stats bias abserr r2 ia rmse

Temperature (K)

- 0.07 0.04 0.007 0.004 0.0450

Mixing Ratio (g/kg)

- 0.53

- 0.26

- 0.081

- 0.034

- 0.3480

Rel. Humidity (%)

- 2.23

- 0.75

- 0.035

- 0.020

- 1.0033

Wind Speed (m/s) 0.13 0.01

- 0.015

- 0.004 0.0137

Clouds (%)

- 2.24

- 2.06

- 0.039

- 0.0 25

- 2.9210

Wind stats bias abserr uerr verr uverr

Wind (deg, m/s) 0.618

- 1.08

- 0.0026 0.0070 0.0034

Pcp Threshold (in) Acc Bias FAR POD CSI ETS TSS HSS

0.01

- 0.0258 0.0287

- 0.0392

- 0.0118

- 0.0306

- 0.0478

- 0.0512

- 0.0515

0.05

- 0.0290

- 0.0430

- 0.0461

- 0.0185

- 0.0374

- 0.0508

- 0.0534

- 0.0577

0.10

- 0.0311

- 0.0461

- 0.0518

- 0.0304

- 0.0443

- 0.0548

- 0.0618

- 0.0659

0.25

- 0.0251

- 0.0470

- 0.0517

- 0.0462

- 0.0445

- 0.0494

- 0.0666

- 0.0658

0.50

- 0.0170

- 0.0812

- 0.0497

- 0.0518

- 0.0387

- 0.0409

- 0.0653

- 0 .0610

1.00

- 0.0069

- 0.2206

- 0.0253

- 0.0186

- 0.0152

- 0.0160

- 0.0252

- 0.0280

Table 46. Like table 45, except for 12

- km VISTAS domain.Draft

62

Recall from the discussion of sensitivity px_acm4 that a slight improvement was found by using RAWINS

output to drive b oth INTERPF and MM5. Since that sensitivity involved changing two variables at once, we do

not know which change (or both) led to the improvement. Sensitivities px_acm7 and px_acm8 are designed to

answer that question. In px_acm7 we change only the SFCFDDA file that MM5 reads in, using the RAWINS

output rather than LITTLE_R output. Tables 47

- 50 show the results for both grids and for both episodes. Very

slight performance degradation is seen in all tables. Tables 51

- 54 show the results for px_acm8, a sensit ivity in

which LITTLE_R outputs are used as inputs both to MM5 and INTERPF. While the wintertime tables show

very slight performance degradation for precipitation, the remaining variables show universal improvement.

Similar improvements are seen in the sum mertime cases with no performance degradation in the precipitation

statistics. This configuration appears to be an improvement over the base case and should be incorporated into

the annual modeling protocol.

The MM5 ( http://www.mmm.ucar.edu/mm5/documents/MM5_tut_Web_notes/MM5/mm5.html )

documentation states that the IFSNOW option applies when the LSM is not used. So this flag was not deemed

to be important for P

- X applicati ons. Of course when MM5 documentation refers to the LSM it normally means

the NOAH LSM, not the P

- X LSM. So we performed a couple of wintertime only sensitivities to determine the

effect the IFSNOW flag may have on a P

- X run. Table 55 shows the results for the 36

- km domain if the

IFSNOW flag is set to 0. Note the bias improvement of 0.07 o C for temperature for this sensitivity s0px_acm2.

Perhaps snow effects are being double

- counted in P

- X if this flag is turned on? The other variables are relatively

unaffe cted, though it is interesting to note the almost universal slight degradation of precipitation skill. Since

snow is a relatively rare occurrence in the VISTAS domain, little change is seen at 12

- km resolution for the

VISTAS domain, as illustrated in table 56.

Tables 57

- 58 show the statistical difference tables for the s2px_acm2 configuration. This sensitivity turns

the simple snow model on. These runs are very cold

- biased, even for the 12

- km VISTAS domain. Clearly this

configuration is not recommended for annual modeling.

Tables 59

- 60 show the shorter summertime episode (episode 3) results for the px_acm9 sensitivity. The idea

for this test is to see if an integration of EDAS soil temperature/moisture might yield similar wintertime benefits

that we saw in the px_acm3 sensitivity, while not showing the degraded summertime performance that prior

sensitivity showed. Admittedly more work could be done along these lines, but the results for this px_acm9

sensitivity show the summertime performance degradation we had hoped would not exist. This configuration is

not recommended for annual modeling.

The next few sensitivity runs quantify the effects of nudging temperature/moisture near the surface. One

advantage of the P

- X module is that it allows indirect soil mois ture nudging to be applied, thereby in essence

“nudging” the lowest layers of the atmosphere without the instabilities that might result from directly applying

surface analysis nudging of temperature/moisture. To quantify the effects this indirect soil nud ging has on the

results, we turned it off in sensitivity n0px_acm2 and applied the model only for episode 2 at 36

- km resolution

(table 61). Not surprisingly we see general performance degradation, though the character of the simulation is

not changed drama tically. For example, the temperature error metric still shows better performance than what

we saw in the noah_mrf, multi_blkdr, and noah_eta

- my runs (tables 15, 21, and 27, respectively).