![](media/image1.png){width="1.75in" height="1.4465277777777779in"}

Initialization of the NCEP Eta/NAM Model DST with Uncoupled NLDASE Land
Surface States

Brian A. Cosgrove and Charles Alonge

SAIC and Hydrological Sciences Branch

NASA Goddard Space Flight Center, Greenbelt, Maryland

Performed in cooperation with NOAA NCEP EMC

**September 30, 2005**

**Acknowledgments**

This work was supported by the joint NASA/NOAA GAPP program through
funding from the NASA Terrestrial Hydrology Program, and was conducted
at the Goddard Space Flight Center in Greenbelt, Maryland. The resources
of our benchmarking partner, NOAA NCEP, were instrumental to the
formulation, conduction, and reporting of this research. In particular,
the authors would like to recognize the contributions of Kenneth
Mitchell, Michael Ek, Matthew Pyle, Keith Brill, Eric Rogers, Ying Lin,
Hui-Ya Chuang, and Mary Hart.

**Contact Information**

[Brian A. Cosgrove, NLDASE Principle Investigator]{.underline}

SAIC / NASA Goddard Space Flight Center

Hydrological Sciences Branch

NASA GSFC Mail Code 614.3

Greenbelt, MD 20771

Phone: 301-286-5084

Fax: 301-286-8624

Email: Brian.Cosgrove@gsfc.nasa.gov

[Charles Alonge]{.underline}

SAIC / NASA Goddard Space Flight Center

Hydrological Sciences Branch

NASA GSFC Mail Code 614.3

Greenbelt, MD 20771

Phone: 301-286-8272

Fax: 301-286-8624

Email: calonge@hsb.nasa.gov

**Table of Contents**

[Executive Summary 1](#executive-summary)

[Evaluation 3](#evaluation)

[1. Background and Description of Problem
3](#background-and-description-of-problem)

[1.1 LDAS Program History 3](#ldas-program-history)

[1.2 NCEP Eta Decision Support Tool 5](#ncep-eta-decision-support-tool)

[1.3 Linkages to NASA and Partner Agency Goals
6](#linkages-to-nasa-and-partner-agency-goals)

[1.4 Linkages to External Programs 7](#linkages-to-external-programs)

[2. Description of Current Partnering Activities
8](#description-of-current-partnering-activities)

[3. Overview of NCEP Eta Model DST 9](#overview-of-ncep-eta-model-dst)

[3.1 Role of the Eta Model within NCEP
9](#role-of-the-eta-model-within-ncep)

[3.2 Detailed Description of the NCEP Eta Model
10](#detailed-description-of-the-ncep-eta-model)

[4. Consideration of NASA Inputs to Eta Model DST
15](#consideration-of-nasa-inputs-to-eta-model-dst)

[4.1 Potential NASA Inputs 15](#potential-nasa-inputs)

[4.2 Current Use of NASA Data Sets in Eta DST
15](#current-use-of-nasa-data-sets-in-eta-dst)

[4.3 NASA Inputs Selected For Use 16](#nasa-inputs-selected-for-use)

[5. Gaps in Meeting Eta Model Input Needs
18](#gaps-in-meeting-eta-model-input-needs)

[5.1 Forcing Data Limitations 18](#forcing-data-limitations)

[5.2 Data Assimilation Gaps 18](#data-assimilation-gaps)

[Verification and Validation 20](#verification-and-validation)

[6. Design and Implementation of NLDASE Eta DST Initialization System
20](#design-and-implementation-of-nldase-eta-dst-initialization-system)

[6.1 Design Philosophy 20](#design-philosophy)

[6.2 Processing of Input Data Sets 20](#processing-of-input-data-sets)

[6.3 Construction of NLDASE offline modeling system
23](#construction-of-nldase-offline-modeling-system)

[6.4 Configuration of the Eta Model at NASA GSFC
25](#configuration-of-the-eta-model-at-nasa-gsfc)

[6.5 Offline NLDASE Retrospective Simulations and Transfer of Land
Surface States to the Eta Model
26](#offline-nldase-retrospective-simulations-and-transfer-of-land-surface-states-to-the-eta-model)

[7. Description of Verification and Validation Methods and Results
30](#description-of-verification-and-validation-methods-and-results)

[7.1 LIS / Noah LSM / MODIS V&V 30](#lis-noah-lsm-modis-vv)

[7.2 NCEP Eta Model V&V 32](#ncep-eta-model-vv)

[7.3 Eta Model Current Performance and Intrinsic V&V
35](#eta-model-current-performance-and-intrinsic-vv)

[8. Verification and Validation Gaps
38](#verification-and-validation-gaps)

[Benchmark 40](#benchmark)

[9. Overview of Benchmark Design 40](#overview-of-benchmark-design)

[10. Overview of Benchmark Metrics 42](#overview-of-benchmark-metrics)

[10.1 NCEP FVS Regional Benchmark Metric Overview
42](#ncep-fvs-regional-benchmark-metric-overview)

[10.2 Site Specific Benchmark Metrics
45](#site-specific-benchmark-metrics)

[11. Analysis of Regional Benchmark Findings
46](#analysis-of-regional-benchmark-findings)

[11.1 Surface Verification (2m Temperature, 2m Relative Humidity, 10m
Wind Speed)
46](#surface-verification-2m-temperature-2m-relative-humidity-10m-wind-speed)

[11.1.1 2m Temperature Bias 47](#m-temperature-bias)

[11.1.2 2m Temperature RMSE 48](#m-temperature-rmse)

[11.1.3 2m Relative Humidity Bias 49](#m-relative-humidity-bias)

[11.1.4 2m Relative Humidity RMSE 50](#m-relative-humidity-rmse)

[11.1.5 10m Wind Speed Bias 51](#m-wind-speed-bias)

[11.1.6 10m Wind Speed RMSE 52](#m-wind-speed-rmse)

[11.1.7 Surface Verification Summary 53](#surface-verification-summary)

[11.2 Upper Air Verification (500mb Height, 850mb Temperature, 300mb
Temperature, 700mb Relative Humidity, 250mb Wind Speed)
53](#upper-air-verification-500mb-height-850mb-temperature-300mb-temperature-700mb-relative-humidity-250mb-wind-speed)

[11.2.1 500mb Height Bias 54](#mb-height-bias)

[11.2.2 500mb Height RMSE 54](#mb-height-rmse)

[11.2.3 850mb Temperature Bias 55](#mb-temperature-bias)

[11.2.4 850mb Temperature RMSE 55](#mb-temperature-rmse)

[11.2.5 300mb Temperature Bias 55](#mb-temperature-bias-1)

[11.2.6 300mb Temperature RMSE 56](#mb-temperature-rmse-1)

[11.2.7 700mb Relative Humidity Bias 57](#mb-relative-humidity-bias)

[11.2.8 700mb Relative Humidity RMSE 58](#mb-relative-humidity-rmse)

[11.2.9 250mb Wind Speed Bias 58](#mb-wind-speed-bias)

[11.2.10 250mb Wind Speed RMSE 58](#mb-wind-speed-rmse)

[11.2.11 Upper Air Verification Summary
59](#upper-air-verification-summary)

[11.3 Daily Precipitation Verification (0-24 hour forecast, 24-48 hour
forecast, 0-84 hour forecast)
60](#daily-precipitation-verification-0-24-hour-forecast-24-48-hour-forecast-0-84-hour-forecast)

[11.3.1 0-24 hour Bias 61](#hour-bias)

[11.3.2 24-48 hour Bias 61](#hour-bias-1)

[11.3.3 0-84 hour Bias 62](#hour-bias-2)

[11.3.4 0-24 hour Equitable Threat Score
62](#hour-equitable-threat-score)

[11.3.5 24-48 hour Equitable Threat Score
63](#hour-equitable-threat-score-1)

[11.3.6 0-84 hour Equitable Threat Score
63](#hour-equitable-threat-score-2)

[11.3.7 0-24 hour Probability of Detection
64](#hour-probability-of-detection)

[11.3.8 24-48 hour Probability of Detection
64](#hour-probability-of-detection-1)

[11.3.9 0-84 hour Probability of Detection
64](#hour-probability-of-detection-2)

[11.3.10 0-24 hour False Alarm Ratio 65](#hour-false-alarm-ratio)

[11.3.11 24-48 hour False Alarm Ratio 66](#hour-false-alarm-ratio-1)

[11.3.12 0-84 hour False Alarm Ratio 66](#hour-false-alarm-ratio-2)

[11.3.13 Precipitation Verification Summary
67](#precipitation-verification-summary)

[12. Analysis of Site-specific Benchmark Findings
67](#analysis-of-site-specific-benchmark-findings)

[12.1 Surface Temperature, Relative Humidity, and Mean Sea Level
Pressure Comparisons
67](#surface-temperature-relative-humidity-and-mean-sea-level-pressure-comparisons)

[12.2 Individual Eta Precipitation Forecasts
72](#individual-eta-precipitation-forecasts)

[12.3 Surface Radiation Forecast Evaluation
77](#surface-radiation-forecast-evaluation)

[13. Benchmarking Gap and Lessons Learned
81](#benchmarking-gap-and-lessons-learned)

[14. Summary, Conclusions, and Recommendations
81](#summary-conclusions-and-recommendations)

[15. References 85](#references)

[Acronyms 94](#acronyms)

#  {#section .unnumbered}

#   {#section-1 .unnumbered}

# Executive Summary

The NASA Applications Program seeks to transfer NASA data, models, and
knowledge into the hands of end-users through forming links with partner
agencies and associated decision support tools (DSTs). This report
details an effort to further this goal by strengthening the existing
partnership between NASA and NOAA and using NASA resources to improve
NOAA weather forecasts. NASA resources utilized in this effort included
the Land Information System (LIS) and MODIS snow cover data, while
NOAA's contribution consisted of the NOAA NCEP Eta model DST (recently
renamed the NAM model). Using the proven uncoupled modeling approach
implemented in the North American Land Data Assimilation System Arakawa
E (NLDASE) project, LIS was used to generate land surface states which
were then used to initialize NCEP's Eta model in place of the standard
operational initial conditions. The concept upon which this research is
based is the following---that improved Eta model initial land surface
conditions will lead to improved Eta model meteorological forecasts.

This theory was rigorously tested through a series of retrospective Eta
model simulations. Twenty control simulations and 60 experimental
simulations were conducted over the May 1-10, 2003 time period.
Forecasts of surface and upper air meteorological fields, as well as
precipitation, were validated against observations using NCEP's Forecast
Verification System (FVS) and site-specific comparisons with several
in-situ monitoring networks. Several conclusions emerged from this
research and are detailed below:

**[Regional FVS Analysis For All Forecasts]{.underline}**

**Surface Forecasts**

-   The use of NLDASE conditions to initialize the Eta model greatly
    improved forecasts of relative humidity and temperature overall.

-   The impact on wind speed forecasts was mixed.

-   The use of MODIS snow cover data generally improved surface
    forecasts.

**Upper Air Meteorology**

-   The impact on upper air forecasts was more mixed than it was on
    surface forecasts.

-   300mb temperature was the only field which consistently improved
    overall.

-   The RMSE of forecasts was more often reduced than was the bias.

-   MODIS snow cover generally improved forecasts; however this benefit
    was often overshadowed by the unfavorable impact of the uncoupled
    initialization process.

**Daily Precipitation**

-   The impact on precipitation forecasts was mixed and generally small.

-   Equitable threat score and probability of detection scores slightly
    improved, indicating improvements in precipitation placement, and in
    the fraction of time the Eta model issued a non-zero precipitation
    forecast given the occurrence of an observed precipitation event.

-   Use of MODIS snow cover data led to small improvements in bias, but
    had only mixed impacts on ETS, POD, and FAR statistics.

**[Site Specific Analysis For 12Z May 3^rd^, 2003 Eta
Forecasts]{.underline}**

-   In general, the largest site-specific differences in forecasts
    occurred between the NLDASE simulations as a whole and the control
    forecast.

**Surface Forecasts**

-   MODIS data strongly impacted forecasts over Canada.

-   The largest impact was on the amplitude of 2m temperature and
    relative humidity.

-   In some cases, the timing of dry-lines and fronts was impacted
    (positively and negatively) by the use of NLDASE land surface
    states.

-   Mean sea level pressure was only slightly impacted by NLDASE
    initialization.

**Precipitation Forecasts**

-   Short term forecasts were impacted less than long lead (\> 48hr)
    forecasts.

-   Precipitation distribution was impacted less than timing and
    intensity.

-   NLDASE-based forecasts generally improved upon the timing, but not
    the intensity, of control run precipitation forecasts.

**Radiation Flux Forecasts**

-   Downwelling shortwave radiation was poorly forecast in all
    simulations during cloudy conditions.

-   Upwelling and downwelling longwave radiation fluxes were well
    forecast in all simulations.

-   Additional validation is needed due to inopportune locations of
    SURFRAD sites.

**[Recommendation]{.underline}**

-   The NLDASE uncoupled initialization approach is very promising, but
    needs further development before being considered for operational
    implementation within NCEP. Given the strong links to NCEP and the
    care that was taken to ensure compatibility with NCEP operations, it
    is expected that they will eventually adopt some of the techniques
    and/or data sets utilized to positive effect in this study.

The pages which follow detail the results of the benchmark experiments
which focused on uncoupled NLDASE initialization procedure, as well as
the evaluation, verification, and validation activities that were
completed in association with this work. Although this report is divided
into three main sections---evaluation, verification and validation, and
benchmark---it is formulated as a continuous report, progressing from
project inception and formulation, to benchmark results and
recommendations.

# Evaluation

## 1. Background and Description of Problem

The land surface serves as the interface between the atmosphere and the
sub-surface, regulating flows of water, energy, and momentum. These
fluxes are manifested through changes in weather and climate, which
impact the public across all temporal and spatial scales. Day-to-day
weather may greatly affect the transportation sector, while longer term
climate trends may affect sections of the public and government agencies
involved in agricultural and water resource management. Impacts may be
felt only locally during a severe weather outbreak, or across a whole
continent during an extended drought. It is this key role in regulating
surface water and energy fluxes which makes the accurate initialization
of land surface conditions in fully-coupled Numerical Weather Prediction
(NWP) models critical for short term to seasonal meteorological and
hydrological prediction (Shukla and Mintz 1982; Dirmeyer 1997). Land
surface states vary in their temporal characteristics. While surface
soil moisture only exhibits persistence on short terms scales,
subsurface moisture and temperature stores exhibit persistence on
seasonal-to-interannual time scales. Snow, which significantly modifies
surface-atmosphere interaction through modification of surface albedo
and melt processes, also exhibits persistence across a wide range of
time scales. Together with external forcing and internal land surface
dynamics, these levels of persistence have important implications for
the prediction of day-to-day weather, as well as climatic and hydrologic
extremes (Delworth and Manabe 1988, 1989; Dirmeyer and Shukla 1993,
Koster and Suarez 1995; Beljaars et al. 1996).

Unfortunately, these stores of water and energy are often poorly
initialized in the coupled weather models which the public and
government agencies depend upon to manage resources, respond to
disasters, and plan daily activities. Because soil moisture,
temperature, and snow are integrated states, errors in land surface
forcing, model physics and parameterization accumulate in the land
surface stores of fully coupled land surface models (LSMs) often used to
initialize NWP models, which leads to incorrect surface water and energy
partitioning and inaccurate weather predictions. Land Data Assimilation
Systems (LDAS), consisting of uncoupled LSMs forced by observations and
unaffected by the biases that can accumulate in internally cycled
systems, feature more accurate land surface states. Accuracy can be
further improved with the assimilation of quantities such as soil
moisture, surface temperature and snow depth and coverage, which acts to
constrain unrealistic storages arising from errors in LSM physics or
parameterizations. Use of such LDAS land surface states as initial
conditions for NWP forecast models should increase the weather forecast
accuracy of such models.

### 1.1 LDAS Program History

For the past seven years, researchers at the NASA Goddard Space Flight
Center (GSFC) have been developing an LDAS under the auspices of the
collaborative North American Land Data Assimilation System (NLDAS)
project (Mitchell et al. 2003). Including partners from NOAA NCEP, NOAA
OHD, NOAA CPC, NOAA NESDIS, Princeton University, Rutgers University,
the University of Washington, and the University of Maryland, this
project has produced high-quality retrospective and real time land
surface fields over a 1/8^th^ degree CONUS domain. NLDAS produces some
of the best fields of soil moisture and temperature that are currently
available in near-real time and makes them available via ftp and the
project's web-based Real-time Image Generator. The project seeks to
further the development of improved land surface models, the
determination of observation needs and error criteria, and the
development and implementation of new data assimilation techniques.
Participants utilize a broad range of land surface models that include
the Mosaic, CLM2, Catchment, Noah, VIC and Sacramento LSMs. Forcing data
for this modeling effort includes GOES, Doppler radar, rain gauge, Eta
Data Assimilation System (EDAS), and Eta based data sets (Cosgrove et
al. 2003).

Validation efforts focused on NLDAS land surface states and fluxes have
bourn out the high quality nature of NLDAS output (Robock et al. 2003),
and cleared the way for the use of LDAS conditions in the NWP
initialization process. In response to this, NASA GSFC and NOAA NCEP
initiated a follow-on to the NLDAS project, named the NLDAS Arawkawa
E-grid (NLDASE) project. Centered on a primary mission of Eta NWP model
initialization, NLDASE operates on the same 12km grid used by NCEP's Eta
mesoscale model (Rogers et al. 1996), the weather agency's main regional
weather prediction model. By using the same LSM (Noah), grid, and
parameter data sets that the Eta model's land surface component uses,
interpolation issues are avoided and initial conditions can be directly
passed into the Eta model. The NLDASE project seeks to improve
understanding of the interaction between the land surface and the
atmosphere from short- to medium-length time scales, to create accurate,
high resolution land surface data sets useful for future research and
applications, to improve the initialization of the Eta model in a way
that may be adapted to additional NWP models in the future, to improve
forecast accuracy, and to continue NASA GSFC's efforts in the
collaborative NLDAS project. The modeling system, depicted in Figure 1,
is designed to accept a variety of NASA models, data sets, and parameter
sets, and is based on the NASA Land Information System (LIS)
infrastructure (Kumar et al. 2004). These capabilities make NLDASE an
ideal means to leverage NASA products, experience, and data to produce
improved initial conditions for use in NWP model initialization. In
particular, *NLDASE provides a strong framework in which to conduct the
Eta model initialization research which forms the core of this NASA
evaluation, verification and validation, and benchmarking report.*

### ![](media/image2.wmf){width="4.866141732283465in" height="2.941732283464567in"}1.2 NCEP Eta Decision Support Tool

NCEP's Eta model is not a typical Decision Support Tool (DST). Unlike
many DSTs, it is not highly focused on a single mission. It does not
respond to narrow categories of disasters or events, is not limited to
certain geographical locations within the United States, and does not
issue suggested action guidelines. But in many ways it is a prototypical
decision support tool. The Eta model performs the basic DST mission of
integrating data, and outputting a product (in this case a weather
forecast) that can be used to aid decision making processes. And it does
so for an extremely wide range of public and private interests,
impacting decision makers in agricultural, economic, transportation,
military, and resource management areas. Its surface forecasts of
rainfall, snowfall, temperature, wind, and humidity, and upper air
forecasts of temperature, wind, and humidity are used directly by
hundreds of forecasters each day, and indirectly by millions of public
and private end users. Operating on a 12km resolution North American
domain, it is the primary model used by the National Weather Service to
produce weather forecasts for the United States. The model is
hydrostatic and features 60 vertical levels described by a hybrid sigma
level coordinate system. It is initialized by the Eta Data Assimilation
System (EDAS) that assimilates wind, precipitation, radiance, sea
surface temperature and snow cover data, is coupled to the Noah LSM, and
produces forecasts out to 84 hours. As NCEP's main regional model, the
Eta model is used by many of NASA's partner agencies and so provides an
ideal target for the infusion of NASA data and modeling techniques.
Improvements in forecasts made by this DST will benefit these agencies,
as well as a broad spectrum of public and private interests.

### 1.3 Linkages to NASA and Partner Agency Goals

The NLDASE concept of NWP model land surface initialization meshes well
with the strategic goals of the NASA application program, and with the
goals of NOAA, one of NASA's partner agencies. The NASA application
program has, at its core, a mission to transfer NASA data, models, and
techniques to end users through infusion into partner agency decision
support tools. NASA has a wealth of experience in remote sensing,
modeling, and data assimilation, and the effective transfer of such
capabilities into the end user community will benefit decision making,
economic vitality, and environmental stewardship. Toward this end, NASA
has selected 12 applications of national priority that can directly
benefit from NASA's goal of improving weather, climate, and natural
hazard predictions (Figure 2). Ranging from agricultural efficiency to
water management, each of these areas represents an opportunity for the
infusion of NASA information and data into critical DST systems.

![](media/image3.png){width="4.5152777777777775in"
height="3.3645833333333335in"}

**Figure 2.** Twelve NASA applications of national priority. Improved
output from the Eta DST will benefit multiple areas. Source: NASA Earth
Science Applications Plan.

As many of these DST systems require weather and/or land surface
information as input, the improved Eta DST forecasts which emerge from
the benchmarking research in this report have the potential to benefit
DSTs in each of these 12 application areas. Strong candidates include
the Advanced Weather Interactive Processing System (AWIPS), the Advanced
Hydrologic Prediction Service (AHPS), the Invasive Species Forecasting
System (ISFS), the Malaria Monitoring and Surveillance (MMS) system, the
National Air Space -- Aviation Weather Research Program (NAS-AWRP), and
the Agricultural Water Resources and Decision Support Tool (AWARDS).
Many of the details which follow were drawn from NASA's Applied Sciences
web page,
<http://science.hq.nasa.gov/earth-sun/applications/index.html>, and the
NASA Stennis Applied Sciences Directorate web page
http://www.esa.ssc.nasa.gov/m2m/.

The Advanced Weather Interactive Processing System is a computer-based
DST which integrates hydrological, satellite, meteorological, and radar
data into a single display
(http://www.crh.noaa.gov/lmk/soo/awips/awips_overview.htm). It enables
forecasters to interactively display and analyze this wide variety of
data in an efficient way, and so issue accurate forecasts and warnings
in a timely fashion. The system is installed at all of the Weather
Forecast Offices (WFOs) and Regional Forecast Centers (RFCs), and
several national weather centers. Since Eta model output is a
significant source of data for AWIPS, any increases in the accuracy Eta
forecasts of precipitation, temperature, wind, or humidity will directly
benefit AWIPS users, and through them, the public which depends on the
watches and warnings issued by NWS forecasters. Forecasters in these
same locations also use the hydrological forecasting DSTs that are part
of AHPS. With a mission centered on providing improved forecasts of
river flow, floods, and droughts, AHPS forecasts depend, in part, on NWP
forecasts of precipitation, temperature, humidity, snowpack, and soil
moisture. Improved Eta model forecasts of these quantities will directly
impact the flood and drought forecasts issued by hydrologists.
Improvements in these same quantities will benefit AWARDS, which depends
on such input to issue estimates of crop water consumption, crop
suitability assessments, and irrigation requirements.

The transportation sector will also benefit from improved Eta model
forecasts, via increased accuracy of output from the NAS-AWRP DST.
Better precipitation and upper air forecasts will allow NAS-AWRP to
produce superior routing, turbulence, and fuel load guidance, and will
allow better aviation-related forecasts to be produced. Similarly
positive effects will be experienced by end-users tasked with monitoring
malaria outbreaks with MMS. Better characterization of soil moisture
levels, soil temperature, precipitation, and surface temperature will
further MMS vector habitat identification, and will aid in the study of
malaria outbreaks.

### 1.4 Linkages to External Programs 

Beyond aiding DSTs within NASA's national application areas, the DST
benchmarking research contained in this report will benefit national
programs outside of NASA as well. One such program which will be able to
make use of the NLDASE soil moisture states and Eta model forecasts is
the interagency National Integrated Drought Information System (NIDIS).
NIDIS seeks to coordinate and encourage drought related research and
monitoring systems. With partners including NOAA, USGS, USDOD, and the
USDA, NIDIS seeks to "develop the leadership and partnerships to ensure
successful implementation of an integrated national drought monitoring
and forecasting system; create a drought 'early warning system' capable
of providing accurate, timely and integrated information on drought
conditions at the relevant spatial scale to facilitate proactive
decisions aimed at minimizing the economic, social and ecosystem losses
associated with drought". Soil moisture and snow states from the NLDASE
system have the potential to serve as the backbone of a drought monitor,
while improved Eta model precipitation forecasts would benefit near-term
planning.

A second partner agency science program furthered by this research is
the joint NOAA-NASA Global Energy and Water Cycle Experiment (GEWEX)
America's Prediction Project (GAPP). This program seeks to predict
changes in water resources through use of uncoupled and coupled
modeling, and data assimilation techniques. As stated in the GAPP
science implementation plan, a main focus of the project is on the
effective transition of research to operations in which results from
hydrologic prediction studies will be transferred into end-user DSTs for
use in water resource management. Another aim of the program is to
investigate the role of land surface memory in atmospheric
predictability, which is also central to this benchmarking effort.
Although GAPP's focus is on seasonal prediction, many of the techniques
and findings which emerge from this project will be directly
transferable to longer-term seasonal research.

The program goals of the interagency United States Weather Research
Program (USWRP) will also benefit from research findings which emerge
from this study. With a multitude of partners including the Department
of Transportation, the Navy, the USDA, DOE, AMS, NOAA, NSF, NASA, and
the EPA, USWRP seeks to investigate means of improving model
precipitation forecasts. Since land surface states greatly influence
precipitation, initialization of the Eta model with NLDASE land surface
states aligns directly with this goal.

Beyond supporting specific interagency programs, NLDASE benchmarking
activities support the overall missions of NOAA NCEP and NOAA OHD. Each
of these organizations has, at its root, a goal to provide accurate
forecasts to the public. NOAA NCEP is constantly striving to improve
atmospheric model forecasts, while NOAA OHD is working toward
improvements in hydrological forecasts. The benchmarking activities
described in this report have the potential to benefit both of these
missions. By utilizing the same version of the atmospheric NWP model and
Noah LSM used operationally by NCEP, results from NLDASE initialization
experiments will be directly transferable to NCEP operations. Similarly,
NOAA OHD forces many of its flood and river runoff models with model
output from NOAA NCEP. If Eta model forecasts of precipitation,
snowpack, and temperature can be improved, NOAA OHD hydrologic
prediction models will directly benefit. In both cases, the public will
ultimately benefit from the infusion of NASA technology into NOAA
operations.

## 2. Description of Current Partnering Activities

The NLDASE benchmarking activities detailed in this report rest on a
strong foundation of cooperation between NASA and NOAA established in
the NLDAS project. For the past seven years, NASA GSFC and NOAA
NCEP/OHD/NESDIS have been collaborating on the NLDAS project to produce
innovative model and data products that have become a vital research
community resource. NASA GSFC has handled preparation of data, Mosaic
LSM simulations, data dissemination, and forcing data generation, while
NOAA has handled Noah and Sacramento LSM simulations, data
dissemination, and research-to-operations related issues. The NLDAS
effort has spawned numerous projects including GLDAS, LIS, and NLDASE.
NLDASE continues the NASA GSFC and NOAA NCEP partnership, and can be
considered a follow-on project to NLDAS. It seeks to use the uncoupled
land surface modeling approach to generate initial conditions for NWP
models---a goal the NLDAS project has always supported.

NLDASE collaboration has been solid, and NOAA NCEP has been extensively
consulted at all stages of the evaluation, verification and validation,
and benchmarking processes (Figure 3) which form the core of this
project. NCEP has guided the choice of NWP model, LSM, grid domain,
experiment setup, and method of analysis. This close cooperation has
ensured that results which emerge from this benchmarking effort will be
directly applicable to NOAA NCEP operations, and will be well-received
and fully-considered by the operations management team. In this way,
NOAA NCEP is both a partner and an end-user. Significantly, NOAA NCEP
recently implemented an upgrade to the Eta modeling system which
demonstrates their confidence in the LDAS approach. Rather than force
the land surface component of their Eta Data Assimilation System with
NWP-model derived rainfall, they are now forcing it with
observation-based precipitation data (K. Mitchell, personal
communication). This style of offline forcing is at the heart of the
LDAS modeling philosophy, and has been extensively tested in successful
NLDAS validation exercises.

## ![](media/image4.wmf){width="5.995275590551181in" height="2.443700787401575in"}3. Overview of NCEP Eta Model DST

### 3.1 Role of the Eta Model within NCEP

One of the main goals of the National Centers for Environmental
Prediction is to be the leader in providing timely and accurate
information to warn the American people of meteorological events such as
tornadoes, hurricanes, winter storms, an active fire weather season,
developing drought conditions, and flooding potential (NCEP Biennial
Review 2003/2004). More specifically NCEP provides both the public and
private sectors with guidance on national and global weather, water,
climate, and space weather, and issues warnings and analyses both
experimentally and operationally. For over a half-century NCEP has
focused on improving their services through science, particularly in the
realm of numerical modeling. In doing so, NCEP has fostered the
development of numerous atmospheric numerical models that provide
decision makers and operational forecasters with valuable information in
the form of forecast guidance. Examples of such operational models are
the GFS (Global Forecast System) and the RUC (Rapid Update Cycle).
Currently, one of the most widely used operational numerical weather
forecasting models in the United States is NCEP's 12km mesoscale Eta
model---recently renamed the North American Mesoscale (NAM) model. The
Eta model is currently run 4 times daily at NCEP and is used
operationally in 7 countries outside of the United States (Brazil,
Greece, Italy, South Africa, Tunisia, Turkey, Serbia, and Montenegro).
The direct end users of the Eta model forecasts are operational weather
forecasters situated throughout the NCEP centers (e.g., the Storm
Prediction Center (SPC) in Norman, Oklahoma), the National Weather
Service, and a large number of private forecasting firms throughout the
United States. Output from the Eta model is harnessed by NWS forecasters
to issue local weather forecasts, severe weather and flash flood
watches, and terminal aerodrome forecasts (TAFs) for local aviation
activities. A large majority of local NWS forecast offices throughout
the nation implement a workstation version of the Eta model for creating
higher resolution forecast guidance, and researching model performance
to identify and correct local forecast problems within the model.

NCEP also produces higher resolution regional Eta (equipped with
additional non-hydrostatic dynamics) forecasts to support fire weather
(8 km domains) and Homeland Security Activities (4 km domains). For
example, a National Weather Service Incident Meteorologist (IMET) could
travel to a region where a severe wildfire is occurring, assemble a
mobile weather center, use it to access Eta model forecast guidance,
surface and upper air observations, and satellite imagery, and then
brief local fire managers. Decisions would then be made on specific
actions to take to reduce the impact of wildfires. Recently an agreement
between the Environmental Protection Agency (EPA) and NOAA led to the
creation of a national air quality forecasting (AQF) system using the 12
km Eta model and the EPA's Community Multi-scale Air Quality (CMAQ)
modeling system (Otte et. al 2005). The AQF system currently provides
predictions of ground-level mixing ratios of chemicals such as ozone and
nitrous oxide, thus allowing state and local agencies to use that data
in issuing local air quality forecasts. Currently this system only
covers the eastern United States, but will be extended over the entire
country in the near future.

The impacts of the decisions made using forecast guidance from the Eta
model are numerous and far reaching. Arguably, the most important impact
of accurate forecast guidance is the preservation of human life. For
example, is it not always feasible to minimize the structural damage
from severe weather outbreaks, however with ample warning and swift
response, the loss of life can be minimized. A reduction in the casualty
rate during extreme weather events can in part be attributed to good
weather forecasts. Accurate numerical guidance has a significant role in
increasing warning time on severe heat waves, tornado outbreaks, and
wildfire spread, thus allowing decision makers at the federal, state,
and local government levels able to deploy resources more efficiently
and ultimately prevent casualties.

### 3.2 Detailed Description of the NCEP Eta Model

As previously mentioned, the Eta model is run four times daily (4
cycles) within NCEP at 0000 Z, 0600 Z, 1200 Z, and 1800 Z and provides
numerical forecast guidance out to 84 hours. The model runs on a cluster
of IBM Power4 servers and model output is then distributed throughout
AWIPS and the NOAAPORT broadcast system. It is made available to the
public through various NOAA ftp and web sites. The model runs over a
large domain that encompasses all of North and Central America (Figure
4).

The Eta model first became operational in 1993 when it replaced the
Limited-area Fine Mesh (LFM) Model at NCEP. It was selected because it
demonstrated skill in multiple areas including the prediction of
precipitation, intricate surface pressure patterns, cold-air damming,
and east coast cyclone tracks (Black 1994). The Eta model received its
name from the vertical coordinate system used within the model. Many
numerical weather prediction models use a terrain-following vertical
coordinate system and the most commonly used of these is the sigma
coordinate (Phillips 1957). However, this vertical coordinate system
can, at times, generate large errors when computing the pressure
gradient force and horizontal advection/diffusion in steeply sloped
terrain. Mesinger (1984) formulated the eta coordinate to help
ameliorate these errors. The eta coordinate is defined as follows:

###### ![](media/image5.png){width="6.090944881889763in" height="4.409055118110237in"}η = ((p-p~t~)/(p~s~-p~t~)) \* F where, F = (p~rf~(z~s~)-p~t~ )/(p~rf~(0) --p~t~) (1) 

In this notation p is pressure and subscripts rf, s, and t refer
respectively to reference pressure, the model surface, and the model top
(p~t~ = 25 hPa). The height, z, is the geometric height. The sigma
coordinate appears as the F=1 case of the eta coordinate. The reference
pressure used in the Eta model is p~rf~(z)= p~ref~(0)(( T~o~ -- Γz)/
T~o~ )^β^, where p~ref~(0) = 1013.25mb, T~o~ = 288K, Γ= 6.5 °K/
km, β=(RΓ)/g, g = 9.8 ms^-2^, and R=287.04 J K^-1^kg^-1^. Thus, the eta
coordinate terrain assumes a step-like appearance which minimizes errors
associated with steeply sloping coordinate surfaces while maintaining
the simplified lower boundary condition of a terrain following vertical
coordinate.

The Eta model is executed on a semi-staggered Arakawa E grid (Arakawa
and Lamb 1977) (Figure 5). The variables at mass (H) points are
pressure, temperature, and specific humidity, while the variables at
velocity (V) points are the zonal and meridional wind components. As
implemented in the Eta model, the E grid is a 12km resolution rotated
latitude-longitude grid centered at 50N and 111W. This grid is based on
a standard latitude longitude grid in which the equator has been shifted
to 50N, and the prime meridian has been shifted to 111W through a series
of rotations. This causes each row of the E grid to lie along a line of
constant rotated latitude, and each column to lie along a line of
constant rotated longitude. In the operational Eta model the shortest
distance between grid points of the same type is approximately 12 km
(line segment d in Figure 5). The large box in Figure 5 delimits the
extent of the computational domain. The area outside of this box serves
as a blending area over which boundary conditions are slowly introduced
into the computational domain. These boundaries are one way interactive
and forecast conditions are passed in from the global NCEP GFS
forecast.![](media/image6.png){width="4.885416666666667in"
height="2.625in"}

The Eta model makes use of step-mountain topography. In this method of
representing topography, changes in elevation are represented in
discrete steps, and every step is centered on a mass point with a
corresponding velocity point at each vertex. This arrangement is
represented by the diamond shaped box in Figure 5.

The operational Eta model features 60 vertical layers, with layer
thickness varying by height, and with the finest resolution near sea
level and around 200 hPa. The top of each topographical step described
above is located at one of the interfaces between the Eta model's
vertical layers (Figure 6). Temperature, specific humidity, and wind
components are evaluated at the midpoint of each Eta layer. The
fundamental time step within the operational Eta model is 30 seconds,
while the horizontal advection time step is 60 seconds. The computation
of vertical turbulent exchange occurs at the interfaces between each
model level and follows the methods described by Mellor and Yamada (1974
and 1982). The model implements a Paulson scheme for the surface layer
over land (Paulson 1970) and uses a viscous sub-layer over the oceans
(Zilitinkevitch, 1970). Horizontal advection in the Eta model employs a
strict nonlinear energy cascade control (Janjic 1984). Large scale
precipitation and deep/shallow convection are parameterized through the
Betts-Miller-Janjic scheme (1994). Radiation is updated according to the
NCEP version of the GFDL radiation scheme which features the interactive
and random overlap of clouds (Lacis and Hansen 1974; Fels and
Schwarztkopf 1975).

The land surface within the Eta model is represented by the Noah land
surface model (Ek et al. 2003). The Noah LSM simulates soil moisture
(both liquid and frozen), soil temperature, skin temperature, snowpack
water equivalent, snowpack density, canopy water content, and the
traditional energy and water flux terms of the surface energy and water
balance. It is invoked by the Eta model every 5 minutes to simulate
upwelling shortwave and longwave radiation values, and sensible and
latent fluxes from the land surface.
![](media/image7.png){width="3.2604166666666665in"
height="2.1145833333333335in"}

The Eta model is initialized using output from the Eta data assimilation
system (EDAS)---a retrospective implementation of the Eta model that
assimilates observations from multiple sources. The EDAS starts with
initial conditions valid 12 hours prior to the Eta forecast start time.
It then assimilates multiple atmospheric observations into the Eta model
as it integrates forward in time, stopping when it reaches the initial
hour of the Eta forecast cycle. NCEP receives the majority of its
observational data through the Global Telecommunications System (GTS),
the National Environmental Satellite, Data, and Information Service
(NESDIS), and the [NOAA/Radar Operations Center
(ROC)](http://www.roc.noaa.gov/) [Open Radar Product Generator
(ORPG)](http://205.156.54.206/pub/im/ptnr88d.pdf) data stream (Keyser
2005). After the data has been stored locally, NCEP then processes the
observational data into a special monolithic BUFR file known as
PREPBUFR. EDAS then gathers the observational input from NCEP PREPBUFR
files which store the raw atmospheric observations, quality control
information, first guess (forecast values), and final quality controlled
observations. A large amount of data is ingested into the PREPBUFR
files. The observations include: global operational rawinsonde data,
Aircraft Weather Reports (AIRREP), Pilot Reports (PIREP), Aircraft
Communication Addressing and Reporting System (ACARS), Aircraft to
Satellite Data Relay (ASDAR) temperature data, dropsonde data from
reconnaissance aircraft (when available), Defense Meteorological
Satellite Program (DMSP) Special Sensor Microwave Imager (SSM/I) total
precipitable water values and surface wind values, Geostationary
Operational Environmental Satellites (GOES) 10/12 precipitable water
values, cloud top drift (winds), cloud top pressure, and clear sky
radiances, surface meteorological airport reports (METAR), synoptic
automated surface observation system (ASOS) observations over land and
ship/bouy/c-man data over water, NOAA operational atmospheric profiler
data, NEXRAD Vertical Azimuth Display (VAD) (levels 2/2.5/3) winds, and
METEOSAT-7 cloud top drift (winds). Some data that is not included in
the PREPBUFR files is also assimilated. The NESDIS 1B satellite radiance
data is stored as binary and passed directly into the assimilation
system. This data is acquired from the Revised TIROS Vertical Sounder
(RTOVS) and Advanced TIROS Vertical Sounder (ATOVS instruments), which
are onboard the NOAA-14, NOAA-15, and NOAA-16 satellites.

In addition to assimilating a vast amount of data, the current
operational EDAS uses the Stage II Radar product (Baldwin and Mitchell
1997) both to force the Noah LSM, and to adjust latent heating profiles
in the Eta model. Noah soil moisture is then corrected every 24 hours
using the Climate Prediction Center's (CPC) daily precipitation analysis
(Higgins 2000). This is a significant change from the way in which the
operational EDAS was configured during the period of the retrospective
simulations conducted in this benchmark experiment. Specifically, in
2003, The CPC product was not used at all within the EDAS, and the Stage
II product was only used to correct latent heating profiles, thus
creating the potential for large biases to accumulate in Noah LSM soil
moisture.

A further difference between the current EDAS and the version in use
during the 2003 retrospective period centers on precipitation type. In
July 2003, the Noah LSM began receiving the precipitation type
(frozen/unfrozen) directly from the Eta model's microphysics and no
longer computed it from the current surface temperature in the EDAS.
Data assimilation procedures were also upgraded with the addition of
GOES cloud top pressure, and the NEXRAD radial wind data (level 2.5). A
more detailed overview of the Eta/EDAS system can be found in Rogers et
al. (2001) and current updates to this system are posted at the
following URL: <http://www.emc.ncep.noaa.gov/mmb/research/eta.log.html>.

Although widely used and highly successful, NCEP has already implemented
the final changes to the Eta/EDAS system and is planning to transition
to a new regional modeling system to provide numerical forecast guidance
over North America in March 2006 (Rogers et al. 2005). The Eta will be
replaced by the Weather Research and Forecast Non-hydrostatic Mesoscale
Model (WRF-NMM, Janjic et al. 2001; Janjic et al. 2005; Black et al.
2005). The new implementation of the WRF-NMM will be executed on a 10 km
Arakawa-E grid with 60 sigma-pressure hybrid levels instead of
step-mountain eta levels. Additionally, the WRF-NMM system will replace
the Eta 3DVar analysis of the EDAS with a Grid-point Statistical
Interpolation (GSI; Wu et al. 2002) analysis. Initial testing of the
WRF-NMM system has shown it to comparable to Eta model in terms of
objective forecast verification statistics; however, the WRF-NMM
implementation of non-hydrostatic dynamics and improvements in
radiation, turbulence, and convective parameterizations allow for better
identification of particular modes of convection and its initiation.
This will assist forecasters in delineating where severe weather may
occur, and in determining what potential impacts it may have (e.g. hail,
straight line wind damage, tornadoes, and/or lightning) (Weiss et al.
2004).

## 4. Consideration of NASA Inputs to Eta Model DST

### 4.1 Potential NASA Inputs

The Eta model requires a wide variety of parameter data sets and
boundary conditions in order to run over its North American domain.
Given NASA's extensive collection of remote sensing products and
modeling capabilities, there exist many potential NASA inputs to the
DST. Likely candidates for infusion into the DST include MODIS snow
cover (Hall et al. 2002), MODIS vegetation classes (Friedl et al. 2002),
MODIS land surface temperature (LST) (Wan 1999), TRMM precipitation
(Huffman et al. 2005), AMSR-E soil moisture (Njoku et al. 2003) and SWE
(Kelly et al. 2003), and Shuttle Radar Topography Mission (SRTM) (Werner
2001) data. Some of these inputs, such as MODIS-based vegetation class
data and SRTM data, could be directly inserted into the DST with minimal
processing as new parameter sets. However, other inputs, such as the
remaining MODIS, TRMM and AMSR-E data sets, would first need to be
processed by the uncoupled NLDASE system before entering the DST
indirectly through NLDASE-supplied initial land surface conditions. In
that respect the uncoupled NLDASE modeling system can be considered to
be a NASA input to the Eta DST as well.

The Hydrological Sciences Branch at NASA GSFC has extensive experience
using the global MODIS snow cover, vegetation classes, and LST data
mentioned above. Snow cover and LST data are available twice per day,
while vegetation class type is a static product. MODIS snow cover and
LST are available at a 0.05 degree resolution, and vegetation classes
are available at a 1km resolution. Land surface temperature and snow
cover are Noah LSM model states, while vegetation classes are model
parameters. As such, MODIS vegetation classes could potentially be
directly swapped for the values used in standard NCEP operations, while
snow cover and land surface temperature would have to be assimilated
into the NLDASE system before passing indirectly into the Eta model in
the form of NLDASE initial conditions. Similarly, TRMM quantified
precipitation amount data, as a meteorological forcing variable, would
face a comparable path through the NLDASE system before entering the Eta
model. TRMM is global in nature, features a 0.25 degree resolution, and
is available every three hours. AMSR-E, another NASA satellite
instrument (Lobl 2001) onboard the Aqua satellite, provides data which
can be processed into daily surface soil moisture values at a 25km
spatial resolution. As a land surface state, it too would need to be
assimilated into NLDASE rather than directly inserted into the Eta
model. This is not the case for SRTM elevation data, which could be used
by the Eta model with only limited pre-processing. Recorded by the Space
Shuttle in 2000, this global topography data set features a resolution
of 30 meters over the United States, and a resolution of 90 meters
elsewhere.

### 4.2 Current Use of NASA Data Sets in Eta DST

While all of these data sets have the potential to directly or
indirectly benefit Eta model forecast accuracy, only one of them is
currently being used in conjunction with Eta modeling operations. This
data set is the MODIS snow cover product mentioned above, and is only
being used in the summertime over polar regions. A second data set, not
in the list above, is also being used by NCEP. Developed at NASA GISS in
the 1980's (Mathews et al. 1983), this data set is a one-degree global
snow-free surface albedo database. However, this limited use
notwithstanding, NOAA is increasingly seeking to use NASA remotely
sensed data, and is considering the expanded use of MODIS-based snow
products in the near future (J. Eylander, personal communication). NOAA
has also recently ported the LIS software over to its operational
platforms for use as the land surface component of its coupled GFS NWP
model. As such, the current climate is extremely favorable for the
integration of NASA models and data products into the NCEP operational
environment.

### 4.3 NASA Inputs Selected For Use

Given HSB computing limitations and the six month time limitation
imposed on this benchmarking effort, it was impossible to use all of the
promising remotely sensed data fields described above. Instead, it was
decided that the benchmarking effort would focus on three NASA
contributions to the Eta model DST; the LIS modeling system, the
uncoupled NLDASE modeling approach, and the MODIS snow cover product.

The approach taken in this benchmarking effort is to improve Eta model
forecasts through the use of NASA uncoupled modeling techniques and
NASA-supplied land surface initial conditions. As an efficient
integrator of land surface data, and a solid infrastructure for the
uncoupled NLDASE modeling approach, the NASA LIS system is central to
this effort. Through its use of parameter data sets, forcing files, and
assimilation techniques, LIS is able to make use of not only MODIS data,
but the other potential NASA data contributions described above as well.
As many of the aforementioned data sets cannot be directly inserted into
the Eta model, LIS fills the vital role of an intermediary between NASA
data and the end-user DST, providing the DST with land surface
conditions that reflect the influence of the NASA data sets. LIS can be
viewed as the foundation of this benchmarking effort, and will be able
to support future assimilation activities that take place as follow-on
projects to this current effort.

LIS builds upon the capabilities of the ¼-degree GLDAS (Rodell et al.
2004) and the 1/8^th^ degree NLDAS (Mitchell et al. 2004) to model
energy and water states such as snow depth, soil temperature, and soil
moisture, and fluxes such as evaporation, transpiration, and runoff, at
1-km and finer spatial resolutions, and at one-hour and finer temporal
resolutions. The 1-km capability of LIS allows it to take advantage of
the latest EOS-era observations, such as MODIS leaf area index, snow
cover and surface temperature, at their full resolution. LIS features a
high-performance and flexible design, provides infrastructure for data
integration and assimilation, and can utilize an ensemble of land
surface models for execution over user-specified regional or global
domains. The LIS framework is designed using advanced software
engineering principles to enable the reuse and community sharing of
modeling tools, data resources, and assimilation algorithms. LIS
provides generic, model-independent support for high performance
computing, resource management, data handling, inter-language support
and other functions. The LIS software is designed within an
object-oriented framework, with explicit abstract interfaces defined for
customization and extension to different applications. These extensible
interfaces allow the incorporation of new domains, land surface models,
land surface parameters, meteorological input schemes and data
assimilation algorithms into LIS. As these interfaces are designed to
remain independent from specific models and algorithms, the
component-style specification of the system allows rapid prototyping and
development of applications. In order to mimic NCEP operations and
ensure the smooth transfer of land surface conditions from LIS to the
NCEP Eta model, all LIS simulations utilized the version of the Noah LSM
(2.3.1) which was used operationally by NCEP during the 2003 benchmark
period.

As described by Mitchell et al. (2004), the Noah LSM (Chen et al. 1996;
Koren et al. 1999; Ek et al. 2003) is targeted for moderate complexity
and computational efficiency for operational NWP and climate models.
Thus it omits tiling and uses a single layer snowpack, plus a linearized
(non-iterative) solution to the surface energy balance. Originating from
the LSM of Pan and Mahrt (1987), Noah benefits from improvements arising
from year-round assessment in the NCEP Eta model over North America by
NCEP and collaborators (Ek et al. 2003). The Noah version used in this
benchmarking experiment was implemented in the NCEP Eta/EDAS suite in
February of 2002, and includes four layers of fixed thickness, of which
the first three (non-forest) or four (forest) span the root zone. A
similar version of Noah was executed in NCEP's 24-year Regional
Reanalysis. Berbery et al. (2003) examined the large-scale hydrology of
the coupled Eta/Noah model over the Mississippi Basin for the period
June 1995 to May 2002. Versions of the Noah model are currently coupled
to real-time weather forecasting models such as the NCEP Eta model, the
NCEP GFS model, the NCAR/Pennsylvania State University MM5 and the
experimental Weather Research and Forecasting (WRF) model.

Complementing the NLDASE LIS-based offline modeling approach is the
assimilation of NASA MODIS snow cover data. Snow is an important
component of the earth's global energy and water cycle, as a result of
its high albedo, low thermal conductivity, and considerable spatial and
temporal variability (Hall, 1988). The presence or lack of snow greatly
impacts the radiation balance as well as sensible and latent heat fluxes
(Liston et al. 1999). The HSB has extensive experience using this MODIS
product, and has already developed and tested a rule-based MODIS snow
assimilation algorithm within LIS (Rodell and Houser 2004). Given these
facts, and the schedule pressures of the six month time constraint
placed on this benchmarking effort, it was decided that of all of the
remote sensing products mentioned above, the MODIS snow cover product
had the highest likelihood of successful incorporation into this
benchmarking effort, and the highest likelihood of positively impacting
LIS simulations of initial land surface conditions.

The MODIS instrument on board the Terra and Aqua satellites has proven
to be an extremely capable tool, with 44 high-resolution MODIS data
products now available. MODIS records observations at wavelengths from
0.405 and 14.385 µm, and at resolutions ranging from 250m to 1000m.
Instruments are located aboard both the Aqua and Terra satellites,
enabling twice-daily updates of data sets. The MODIS data used in this
benchmarking effort was MOD10C1, a 0.05 degree resolution MODIS
climate-modeling grid-level-3 product derived from daily 500m MODIS
observations (Hall et al. 2002). This product specifies the fraction of
each grid cell that was observed to be cloud covered or snow-covered. It
also supplies a "confidence index" that consists of the fraction of the
grid cell in which the land surface was visible and not obscured by
clouds or darkness at the time of satellite overpass. These snow cover
and confidence values are used by the snow assimilation algorithm within
LIS to remove snow cover, or add a thin layer of snow as needed, based
on the rule-based approach of Rodell and Houser (2004). The MODIS snow
cover product has been validated against in-situ observations and
compared against other satellite-based snow cover products by Maurer et
al. (2003) and Bitner et al. (2002). In both cases, the MODIS product
performed extremely well.

## 5. Gaps in Meeting Eta Model Input Needs

### 5.1 Forcing Data Limitations

The overarching goal of the NLDASE effort is to provide accurate NLDASE
land surface states for the initialization of the Eta model and the
improvement of Eta model forecasts; however, there are several issues
which may hinder the system's overall efficiency. Foremost among these
is the limited availability of real-time forcing data. The NLDAS
observed precipitation data (Cosgrove et al. 2003), which is a blend of
the NCEP stage II hourly 4km gauge/radar national precipitation product
(Baldwin and Mitchell 1997), the 0.25 degree CPC daily rain gauge data
(Higgins 2000), and EDAS precipitation, can be delayed up to 24 hours in
real time due to the daily release of the gauge data shortly after 1200
Z. Without the daily gauge data, the NLDAS product relies upon 24 hour
EDAS precipitation totals, which can decrease the accuracy of the
precipitation data set. Along similar lines, the Climate Prediction
Center's (CPC) satellite/gauge Merged Analysis of Precipitation (CMAP)
product (Xie and Arkin 1997), due to its underlying pentad nature,
features a temporal lag of up to 6 days. This would complicate its use
in a real-time NLDASE system. Data lags of one day affect the UMD hourly
0.5 degree GOES-based surface solar insolation analysis (Pinker et al
1999), the Air Force Weather Agency (AFWA) Agricultural Meteorological
System (AGRMET) global dataset of shortwave and longwave radiation data
(48km), and the NOAA CPC Morphing Technique (CMORPH) precipitation data
set (Joyce et al. 2004), which is currently used in the Eta-model based
Regional Climate Data Assimilation System (Mesinger et al. 2004).

Additional V&V gaps center on data availability within the NLDASE
forcing data sets used to force the uncoupled Noah simulations in this
benchmarking experiment. The NLDASE forcing dataset was designed to be
highly flexible, allowing for use of multiple observational data
sources. However, there are instances when some of the observational
data sources are missing. GLDAS precipitation only became available in
January of 2001, and CMORPH wasn't available until December 2002;
however, the NLDAS precipitation product is available throughout the
entire forcing data set. Similarly, AGRMET downward shortwave radiation
data wasn't available until March of 2001, although the UMD radiation
product is available throughout the benchmarking period with the
exception of July 2003 though August 2003 due to sensor calibration
problems (R. Pinker, personal communication).

### 5.2 Data Assimilation Gaps

Other V&V gaps in providing initial conditions to the Eta model are
related to data assimilation within LIS. Soil moisture and land surface
temperature data were not assimilated in the retrospective Noah LSM
simulations of the benchmark, as such assimilation tools are still being
tested and validated within LIS. By contrast, snow cover data was
assimilated into the Noah LSM. Unfortunately, only data from the Terra
Satellite was available, and the swath nature of this product along with
MODIS's inability to "see" through clouds hinders this products ability
to be used in a real-time mode due to a lack of consistent global
coverage. Assimilation of AMSR-E soil moisture and land surface
temperature (LST) data into the LIS architecture has recently been
implemented in experimental mode using direct insertion and Extended and
Ensemble Kalman filtering; however these methods are currently
undergoing thorough testing for suitability in offline modeling studies
(X. Zhan, personal communication). Soil moisture information is
contained within the AMSR-E C-band (6.9GHz) and X-band (10.7 GHz)
products (Njorku et al. 2003). However, the radio frequency interference
(RFI) discovered within the C-band product (Li et al. 2004), in
conjunction with the coarse 50km resolution of the data, has limited the
product's current usability. Although not included in the current NLDASE
retrospective simulations, it is anticipated that as the data
assimilation capabilities within LIS mature and ongoing AMSR-E X-band
validation efforts progress, these capabilities will be implemented in
the NLDASE retrospective system.

# Verification and Validation

## 6. Design and Implementation of NLDASE Eta DST Initialization System

### 6.1 Design Philosophy

Significant progress has been made by collaborative research teams over
the last few years to develop operational continental (North American
and European) and global LDAS frameworks. Through these projects it has
been demonstrated that the transfer of land information between models
of different types or resolutions is not trivial, and may in fact not be
possible. Therefore, to properly test the initialization of the coupled
Eta/Noah model using NLDASE land surface states, the uncoupled NLDASE
system used in the benchmark process was developed to be as compatible
as possible to the coupled system. In particular, it was designed to
operate on the same native Arakawa E grid as the Eta model, therefore
allowing a direct transfer of NLDASE land surface states to the Eta
model for initialization purposes. This eliminated any sources of error
associated with interpolation between grids. Compatibility was further
strengthened through the use of the standard set of operational Noah LSM
land surface parameters, and through the configuration of the Eta model
used in the benchmarking studies in such a way as to closely match the
operational version run at NCEP in the 2003 benchmarking period. These
choices made it possible to isolate the impacts of NLDASE forcing on the
resulting land surface states used to initialize the Eta model, and
aided in the resulting analysis of Eta forecasts.

### 6.2 Processing of Input Data Sets

As mentioned above, in order to isolate the effect of NLDASE forcing
data on resulting model guidance, the NLDASE retrospective simulations
used the same land surface parameter data as was used by the Noah LSM in
the operational Eta model. The land/sea mask, topography, and albedo
data were extracted directly from the data sets that were used in the
operational Eta model in 2003. The 1° x 1° global vegetation type data
set depicted in Figure 7 (Dorman and Sellers 1989) was used, as was the
Zobler (1986) nine-class 1° x 1° global FAO soil class data set. These
data sets have recently been upgraded in the operational Eta model, and
the USGS 27-category vegetation type data set (Anderson 1976), 1 km Penn
State STATSGO soils data set, and 5 minute ARS FAO soils data set
(Reynolds et al. 1999) have replaced the coarser data sets above.

![](media/image8.wmf){width="5.995138888888889in"
height="5.7027777777777775in"}The goal of this benchmark study is to
improve Eta model forecasts through improved initialization of land
surface states. These states, in turn, depend upon meteorological
forcing data. With this in mind, a flexible, high quality, hourly
forcing dataset was created for use in the retrospective NLDASE
benchmark simulations. The following data sets are included in the
NLDASE forcing data set which spans the period of October 1, 2000 to
August 31, 2003:

1.  EDAS/Eta meteorological fields. Three-hourly wind speed, air
    temperature, pressure, and humidity from the EDAS (or Eta model when
    unavailable) serve as the forcing backbone in the NLDASE forcing
    data set. Standard 2/10 m level data and lowest Eta model level data
    are provided for the aforementioned fields. In addition, the height
    of the lowest Eta model level is provided in addition to the
    Eta-calculated surface exchange coefficient for heat and moisture
    (C~h~). EDAS/Eta data features a resolution of 22km before October
    27^th^, 2001, and a resolution of 12km thereafter. It is often
    difficult for NWP models to provide accurate precipitation and
    radiation information. Therefore, the following datasets act to
    replace the EDAS radiation and precipitation fields when available.

2.  NLDAS Observed Precipitation data. This hourly data is based on a
    merging of the NCEP Stage II hourly 4-km gauge/radar national
    precipitation data (Baldwin and Mitchell 1997) with 0.25 degree CPC
    daily rain gauge data (Higgins et al. 2000), and is valid over the
    continental United States.

3.  GLDAS Precipitation data. Similar to the NLDAS observed
    precipitation data, this six-hourly data set based on a merging of 6
    hourly precipitation data from GDAS (Derber et al. 1991) and pentad
    data from the CPC blended satellite/gauge Merged Analysis of
    Precipitation (CMAP) product (Xie and Arkin 1997).

4.  CPC Morphing Technique (CMORPH) data (Joyce et al. 2004). This
    half-hourly data set is based on precipitation estimates from
    low-orbiting satellites (microwave) and propagates those spatial
    features using infrared data from geostationary satellite data.
    Currently data is gathered from passive microwave sensors aboard the
    DMSP 13, 14 & 15 (SSM/I), the NOAA-15, 16 & 17 (AMSU-B) and the TRMM
    (TMI) spacecraft.

5.  UMD hourly 0.5 degree GOES-based surface solar insolation analysis.
    This product is valid over the lower 48 states as well as southern
    Canada and Northern Mexico (Pinker et al. 1999) and contains hourly
    estimates of downwelling shortwave radiation, photosynthetically
    active radiation (PAR), and surface brightness temperature.

6.  GLDAS observed radiation data. NLDASE forcing incorporates the Air
    Force Weather Agency's (AFWA) Agricultural Meteorological System
    (AGRMET) global data set of shortwave and longwave data at 48km
    resolution that uses the algorithms of Shapiro (1987) to calculate
    surface downwelling shortwave radiation from AFWA Real Time
    Nephanalysis (RTNEPH) 3-hourly cloud maps (Hamill et al. 1992), and
    the AFWA daily snow depth (SNODEP) maps (Kopp and Kiess 1996).

7.  Interactive Multisensor Snow and Ice Mapping System (IMS) (Ramsay
    1998). This data provides daily snow cover at 24-km resolution and
    snow depth at 48-km resolution over the entire Northern Hemisphere.
    It is based on AVHRR visible imagery, geostationary satellite
    imagery (GOES, METEOSAT, and GMS), USAF snow analyses, and surface
    observations.

A bilinear interpolation procedure was used to interpolate all
non-precipitation (and pre-11-2001 EDAS) data sets to the native Eta
model 12km Arakawa-E grid. Precipitation data sets were interpolated
using a budget bilinear method which conserves area average
precipitation values when interpolating to higher resolution grids
(Figure 8). Once interpolated to the Arakawa-E grid, the data was then
temporally interpolated to the hourly resolution of the NLDASE forcing
data set. Winds were interpolated by magnitude and direction and
shortwave radiation data were interpolated using a solar zenith
angle-based process, which more accurately depicts the diurnal solar
cycle. In the case of precipitation, a block temporal interpolation
method was used in which the precipitation rate was assumed to be
constant between observational data points. All other fields were
subjected to linear temporal interpolation algorithms. Elevation
correction of the 22km EDAS data was accomplished through interpolation
of the 22km Eta topography field to 12km, and its subsequent use in
reference height calculations.

### ![](media/image9.wmf){width="5.994488188976378in" height="7.25in"}6.3 Construction of NLDASE offline modeling system

The heart of the NLDASE system is the NASA/GSFC Land Information System
detailed above (Kumar et al. 2004). Although LIS's flexible and powerful
nature made it an ideal choice for running the offline retrospective
NLDASE simulations, several important modifications needed to be made
for it to fully support the retrospective NLDASE modeling simulations
and provide compatible land surface state output to the NCEP Eta model.

Most importantly, LIS was modified to run on the Arakawa-E grid at the
same resolution and over the same domain as the operational 12km Eta
model. Routines were both added and modified to read in the operational
parameter data and latitude/longitude values of the individual grid
points. Currently, the operational Eta model is coupled to version 2.7.1
of the Noah LSM. However, during the 2003 period of the retrospective
benchmarking simulations, the Eta model was coupled to an older version
of the Noah LSM---version 2.3.1. Thus, to ensure transferability of land
surface states, this older version of the Noah LSM was obtained from
NCEP and coded into the LIS architecture. In addition to retrofitting an
older version of the Noah LSM into the LIS code, several routines had to
be written to handle the production and use of the multiple data sets
within the NLDASE forcing data set.

![](media/image10.png){width="6.0in" height="4.667322834645669in"}
Several test runs were performed using Noah 2.3.1 within the LIS
infrastructure to ensure the correct performance of the system. Short
retrospective runs on the order of one week were performed using the
standard 2/10m EDAS meteorological forcing fields and an operational
Noah LSM restart file. The Noah LSM output was then compared to
operational EDAS output from the same time period. As shown in Figure 9,
large differences appeared between the EDAS output and output from
NLDASE runs forced only with EDAS data. Investigation into these
differences revealed that in NCEP operations, the Noah LSM is forced
using the lowest level model data instead of 2/10 m data, which is a
product of the Eta post-processing software. It was also discovered that
the operational Eta model uses specified roughness lengths that are
topographically defined when calculating its surface exchange
coefficients, and does not use vegetation-based parameters as is
commonly done in uncoupled implementations (K. Mitchell, personal
communication).

Taking these differences into account, the LIS code was modified to use
the surface roughness field specified in the Eta model, and the lowest
model level EDAS forcing fields. When the test simulations were rerun
using these alterations, differences greatly decreased. However, since
the operational EDAS is a fully coupled system which invokes the Noah
LSM every 5 minutes, differences will always emerge versus a purely
offline system which makes use temporally interpolated 3-hourly EDAS
data. Minimizing the differences between the offline NLDASE simulations
(EDAS forcing only) and EDAS in terms of driving the Noah LSM further
solidified the compatibility of NLDASE land surface states and the Eta
model.

The final alteration to LIS within the NLDASE system was the inclusion
of MODIS snow cover assimilation. MODIS snow cover data was first
interpolated to the native Eta grid, and then LIS was altered to update
snow cover during the three year retrospective simulations according the
rule based approach of Rodell and Houser (2004). This was done to
improve Noah LSM land surface states. The Noah LSM used within the NLDAS
effort was shown to have high snowmelt and sublimation biases, and
melted off snowpack too quickly (Sheffield et. al 2003; Pan et al.
2003). The snow cover assimilation addressed this by removing and adding
snow cover as needed. In so doing, it maintained the proper land surface
albedo, thus directly impacting the simulated Noah LSM surface fluxes
and states.

### 6.4 Configuration of the Eta Model at NASA GSFC

Originally the NLDASE effort sought to run the NCEP Eta model over
reduced domain areas that were less computationally demanding than the
full North American domain executed at NCEP. However, NASA supercomputer
time became available, thus making full domain simulations possible
within a reasonable amount of time. The operational Eta and EDAS system
is highly customized for use on the IBM supercomputers used within NCEP.
But, as mentioned previously, NCEP maintains a workstation version of
the Eta model for distribution to the NWS forecast offices and
individual researchers---and it is this software which was utilized in
this benchmark experiment.

The workstation Eta source code is largely identical to that of the
operational Eta model, with the main differences appearing in the
methods of model initialization and flexibility in domain selection
(Pyle 2002). The workstation Eta also includes the option to run using a
vertical sigma coordinate instead of the operational eta coordinate, as
well a choice of Kain-Fritsch (Kain and Fritsch 1990, 1993) or
Betts-Miller-Janjic (Betts 1986; Betts and Miller 1986; Janjic 1994)
cumulus parameterization schemes. In order to make the local
installation of the workstation Eta as close as possible to the
operational implementation at NCEP, the workstation Eta was configured
to run using the Betts-Miller-Janjic cumulus parameterization scheme and
the eta vertical coordinate system. The workstation Eta model is
distributed with an older version of the Noah LSM---version 2.2. Thus,
it was necessary to update the model to include Noah version 2.3.1 in
order to be completely compatible with NLDASE output and directly
comparable with the version run operationally in the 2003 benchmark
period. The rest of the workstation Eta model was also configured as
closely as possible to the operational implementation of the model.
Alterations included changing the horizontal resolution to 12 km and
vertical resolution to 60 levels, extending the domain to match the
operational Eta domain, and increasing the frequency of calls to the
surface physics (Noah LSM), radiation, and advection schemes. Finally
the land-sea mask was also changed slightly to match the operational Eta
land-sea mask and the NLDASE land-sea mask. In its basic configuration,
the workstation Eta uses operational Eta NCEP restart files (as opposed
to NLDASE data) and boundary conditions from the Global forecast system.
As a test of workstation Eta functionality, retrospective period output
from the local workstation Eta model was compared with output from the
operational Eta model. Forecasts were found to be nearly identical. All
references to Eta forecasts throughout the benchmarking section of this
report refer to forecasts produced by the workstation Eta model executed
at NASA GSFC.

The workstation Eta model was installed on the NASA GSFC Halem
supercomputer---a 348 node HP/Compaq AlphaServer SC45 featuring four
processors and approximately 2 GB of memory per node. Several
performance benchmarking tests were conducted to assess the tradeoff
between system wait times (batch-queuing system) and Eta model runtime.
Based on these results, the local installation of the Eta model was
configured to run over 84 processors (21 nodes), with 12 processors (3
nodes) dedicated to handling the large amount of input/output required
by the Eta model (hourly restart files are \~2.5 GB each). This
configuration proved to provide the best tradeoff between system wait
time and model run time, and allowed for the execution and post
processing of full-length 84 hour forecasts in approximately 9 hours.

### 6.5 Offline NLDASE Retrospective Simulations and Transfer of Land Surface States to the Eta Model

As previously stated, the NLDASE system was constructed to produce high
quality land surface states for use in initializing the NCEP Eta model.
The creation of these land surface states involved performing multiple
three year retrospective simulations using MODIS snow cover assimilation
and the highest quality forcing components from the NLDASE forcing
dataset. The length of the retrospective simulations was chosen
primarily to allow the Noah LSM to adjust or spin-up to the NLDASE
forcing. Models that are not spun-up can produce severely biased land
surface simulations which can lead to questionable land surface model
output. Beyond the length of the simulations, the choice of LSM is also
important. As previously mentioned, the use of the Noah LSM allowed for
the most compatible transfer of land surface states into the NCEP Eta
model. If other LSMs had been utilized, vigorous testing of the land
surface states and their transferability to the Noah LSM within the Eta
model would have been required---something the six month span of this
benchmarking effort did not allow.

The first of the three uncoupled LIS simulations (LIS1) was executed
without MODIS snow cover assimilation. When available, the UMD
downwelling shortwave radiation (GOES) product was used in place of the
EDAS value over snow-free areas. When the IMS product indicated the
presence of snow cover, AGRMET radiation was used instead (Figure 10).
This procedure was implemented because the GOES radiation product
historically exhibits a low bias in downwelling shortwave radiation over
regions with snow cover (Pinker et al. 2003). If the GOES product was
unavailable, the AGRMET product was used, and if both observed radiation
products were absent from the NLDASE forcing, EDAS values were used.

![](media/image11.wmf){width="6.016141732283464in" height="4.5in"} An
overlay procedure was also implemented for precipitation. The NLDAS
precipitation product was used in place of the other NLDASE
precipitation data fields over the CONUS. Over the remainder of the
domain, the GLDAS precipitation product was used preferentially over the
CMORPH due the GLDAS product's more accurate depiction of total
precipitation amounts (J. Gottschalk, personal communication). When the
GLDAS precipitation was unavailable, the CMORPH data was used, and when
both observed precipitation products were unavailable the default EDAS
precipitation was used to drive the Noah LSM simulation. The use of
CMORPH precipitation in place of GLDAS precipitation is a decision which
needs to be further explored, as the GLDAS precipitation is 6 hourly and
therefore may exhibit difficulty in depicting the highly variable
temporal nature of convective rainfall in the southern portions of the
NLDASE domain. While it is important to examine the benefits of using
different combinations of observed forcing products, these studies are
outside of the domain of this benchmark report, and will be the focus of
future studies.

The second and third retrospective simulations (LIS2 and LIS3,
respectively) involved all the aspects of the aforementioned LIS1
retrospective run, but included the assimilation of daily MODIS snow
cover as well. Snow cover was continually updated throughout both of
these simulations according to the rule based approach of Rodell and
Houser (2004). At each grid point at 10:30 A.M. local time, the MODIS
snow cover was compared to the modeled snow-water equivalent variable
from the Noah LSM. If the Noah LSM had zero snow depth, but MODIS snow
cover was greater than 40%, then a small layer of snow was added in the
LSM. In the retrospective simulations 5mm (LIS2) and 10mm (LIS3)
equivalent heights of water were chosen as the amount of snow to add. If
Noah had non-zero snow depth but the MODIS snow cover for that
particular grid point was less than 10%, then the Noah LSM snow water
equivalent was set to zero. If the Noah and MODIS values were in
agreement or the MODIS snow cover was between 10% and 40%, the Noah
snow-water equivalent value was left unchanged. The difference between a
Noah LSM simulation that included MODIS snow cover assimilation and a
simulation that did not include it is shown in Figure 11. Operationally,
the Noah LSM is updated with IMS snow cover and snow depth data daily
within EDAS through a direct insertion technique which can introduce
assimilation fluxes of water into the soil column (Cosgrove and Houser
2002). The MODIS snow cover assimilation procedure described above was
formulated with the aim of minimizing this type of flux, while still
affecting the albedo (Rodell and Houser 2004).

All three retrospective simulations, were initialized using land surface
states from the operational EDAS system on 00 Z October 1, 2000 and run
through August 2003 with the optimal set of NLDASE forcing described
above. Each retrospective simulation outputs hourly land surface states
and fluxes, thereby allowing for detailed validation and verification.
These states and fluxes are output in a gridded binary format (GRIB)
(Stackpole 1994) to conserve storage space. In addition to the standard
land surface states and fluxes output in the original NLDAS
retrospective simulations (Mitchell et al. 2004), LIS output was
modified to include the surface exchange coefficients for heat/moisture
and momentum to allow for further investigation into offline LSM forcing
methodology.

The NCEP Eta model operational restart files are stored in big-endian
binary format and contain over 300 atmospheric variables, in addition to
land surface states associated with the coupled Noah LSM. In order to
initialize the workstation Eta model with NLDASE land surface states,
the original land surface states had to be extracted from the
operational restart file and replaced by output from NLDASE. Eighteen
fields in total had to be replaced, and consist of the following:
Surface exchange coefficient for heat/moisture, surface exchange
coefficient for momentum, total soil moisture (4 levels), liquid soil
moisture content (4 levels), soil temperature (4 levels), canopy
moisture content, snow water equivalent, snow depth, and surface skin
temperature. Once these fields were replaced, the newly modified restart
files were uploaded to the Halem supercomputer at NASA GSFC for
subsequent use in the retrospective benchmark Eta model forecasts
illustrated in the benchmarking section of this report.

![](media/image12.wmf){width="4.5783464566929135in"
height="6.929921259842519in"}

## 7. Description of Verification and Validation Methods and Results

### 7.1 LIS / Noah LSM / MODIS V&V

Each part of NASA's contribution to this benchmarking effort has been
well-validated and verified. The LIS software has been extensively
tested throughout all phases of its development cycle. As it is based on
the original NASA LDAS model driver code, and is meant to replace and
improve upon this modeling system with a more flexible framework, it has
been rigorously tested to ensure that it can successfully reproduce
traditional ¼ degree GLDAS simulations. The code has been tested at
various other resolutions, including 1km, and has also been tested
across a wide variety of computer systems, including Intel Linux PCs,
Apple Servers, Dec-Alpha super computers, and SGI computer systems.
Recognizing its potential, NCEP ported LIS over to their operational
systems to serve as the land surface component of their Global Forecast
System coupled NWP model. This porting required additional comprehensive
testing, which the LIS software successfully passed. LIS is currently
participating in the Coordinated Enhanced Observing Period (CEOP) model
intercomparison and validation project, as well as the Global Soil
Wetness Project 2 (GSWP2) model intercomparison project. LIS is
well-accepted in the research community, and is being used in additional
projects involving NOAA OHD and the Air Force. The strong modeling
capabilities and innovative approach that LIS brings to these partners,
as well as to NASA research teams, was recently recognized by NASA
through awarding LIS the NASA Software of the Year Award.

The Noah LSM serves as the land surface component in both the Eta NWP
model, and in the LIS modeling infrastructure as configured for this
benchmarking study. As NCEP's operational land surface model, Noah is
continually tested and validated, and has a rich and progressive history
of upgrades. In particular, Noah participated in both the Project for
Intercomparison of Land surface Parameterizations (PILPS), the Global
Soil Wetness Project (Dirmeyer et al. 1999), and is currently
participating in GSWP2 and the CEOP validation project. The Noah land
surface model has been implemented and tested extensively in both
offline and coupled modes, and is used in both global and regional
applications. Updates to its physics, parameterizations, and data sets
are continually implemented by the NCEP land surface research team, with
over 16 upgrades taking place since the release of Noah version 1.0 in
1999. As mentioned above, one of the most important functions of the
Noah LSM component of LIS is to produce initial land surface conditions
that can be directly transferred to the Noah LSM component of the Eta
model. Much of this set of initial conditions was the same selection of
land surface variables that has been thoroughly validated over the
course of the NLDAS project.

NLDAS Noah LSM validation was spearheaded by Rutgers University, and
made use of 72 Oklahoma Mesonet observation sites and 24 ARM/CART
measurement sites (Robock et al. 2003). The ARM/CART stations were used
to validate Noah LSM skin temperature and net radiation, while the
Mesonet stations were used to validate Noah soil moisture and soil
temperature fields. Robock et al. report that of the four NLDAS LSMs
tested, the Noah LSM overall most closely matched the observations. Skin
temperature errors were under 4°C, and were reduced even further when
the Zilintikevich parameter was decreased, which effectively increased
the aerodynamic conductance parameter. Net radiation was
well-reproduced, and simulated soil temperature values generally fall
within 1°C of 5cm-deep Mesonet observations, and 2°C of 25cm-deep
observations, which Robock et al. (2003) characterize as "quite good".
Similarly, Noah was able to reproduce top 40cm soil moisture layer
values well, featuring a 7% high bias, and was able to reproduce the
soil layer anomalies in this layer extremely well (Figure 12). Solid
reproduction of soil moisture anomalies by the Noah LSM was further
substantiated in the GSWP project by Entin et al. (1999) and Chen and
Mitchell (1999). The high quality of Noah land surface state simulations
lends solid support to the concept of using the Noah LSM in an uncoupled
NLDASE configuration to produce high-quality initial land surface
conditions for the Eta NWP model.
![](media/image13.wmf){width="5.889583333333333in"
height="3.622916666666667in"}

The second NASA contribution to this Eta DST benchmarking experiment,
the MODIS snow cover product, has also been extensively validated. As
mentioned above, this product has compared favorably against both NOAA
NESDIS and National Operational Hydrologic Remote Sensing Center
(NOHRSC) snow cover products. MODIS 500m snow cover maps have been
compared by Bitner et al. (2002) to NESDIS 5km snow maps, and to NOHRSC
1km snow maps, and were found to be in good agreement. Further
validation was performed by Maurer et al. (2003), who intercompared the
MODIS 500m product, the 1km NOHRSC product, and ground-based
observations. The MODIS product performed well in this study,
misinterpreting the existence of snow 4%-5% less often than did the
NOHRSC product.

### 7.2 NCEP Eta Model V&V

The main goal of the NLDASE effort is to provide initial land surface
states to the Eta model which do not feature the biases that can
accumulate in land surface states of fully-coupled models. Accurate
initialization of land surface states in these models is critical for
short term to seasonal meteorological and hydrological prediction
because of their controls on water and energy fluxes between the surface
and atmosphere over a variety of timescales (Shukla and Mintz 1982;
Dirmeyer 1997). Much care has been taken to design and develop an
uncoupled NLDASE system that is as similar as possible to the
operational coupled system in order to properly test the initialization
procedure. However, it is possible that even output from the same type
of LSM may need to be scaled during the initialization process if such
LSMs have been subject to different forcing or data assimilation
techniques (R. Koster, personal communication). Due to the time
constraints of this benchmarking effort, statistically scaled land
surface states were not used in initializing the Eta model; however, a
fourth uncoupled retrospective NLDASE simulation involving the use
EDAS-only forcing has been completed and will be used in the near future
to investigate scaling issues.

As a first approach, the direct transfer of uncoupled NLDASE land
surface states to the Eta model was performed. Several key differences
between the NLDASE generated land surface states and the internally
cycled operational EDAS states exist and can have large impacts on Eta
model forecasts. Figure 13 shows the difference in root zone soil
moisture states from the three retrospective NLDASE simulations compared
with values from the EDAS System.

The EDAS soil moisture was significantly wetter than the uncoupled
NLDASE LIS1 soil moisture throughout much of Canada and the high
elevation regions within the continental United States. The NLDASE runs
which included MODIS snow cover assimilation featured reduced soil
moisture differences in these regions. This was particularly evident in
the LIS3 simulation which utilized a 10mm snow water equivalent
threshold. Initial investigations have indicated the main causes of the
remaining differences to be the use of the GLDAS precipitation product
in the uncoupled modeling approach, and the differences in assimilation
methodology. This is supported by Cosgrove and Houser (2002), who found
that snow assimilation via direct insertion can potentially add
significant amounts of unrealistic soil moisture to land surface states,
particularly in the Rocky Mountains and in Eastern Canada.

Differences also arise from NLDASE's use of GLDAS precipitation forcing
in Canada and South America. In South America, particular over the Andes
Mountains region, EDAS precipitation is tightly linked to topography.
The model exhibits a tendency to simulate large amounts of precipitation
throughout the region with the exception of highly elevated areas. This
is consistent with observed local convective activity and mountain
uplift (Figueroa and Nobre 1990; Velazco and Fritsch 1987; Marengo
1995). The GLDAS precipitation product, by contrast, is coarser, and
thus the precipitation is less localized and more evenly distributed
throughout this region. However, this alone does not guarantee that the
EDAS precipitation is more accurate, and an investigation into the
forcing precipitation distribution is warranted as both model and
satellite derived precipitation datasets exhibit large variability
throughout this region (Goncalves et al. 2005, submitted manuscript).
Differences in soil moisture also exist over the United States. With the
exception of the Rocky Mountains and portions of the northern Great
Plains and the Northeast, soil moisture across the much of the United
States was noticeably wetter in the offline NLDASE simulations and is
likely an artifact of the NLDAS precipitation product.

Like soil moisture, soil temperature varied significantly between the
EDAS and uncoupled NLDASE simulations, with the largest differences
appearing in the topmost soil moisture layer. Figure 14 shows the
difference between EDAS and NLDASE 0-10 cm soil temperatures on 00 Z May
1, 2003. Overall, 0-10 cm soil temperatures in the uncoupled NLDASE
simulations were generally warmer throughout much of the North American
domain than those produced by EDAS. LIS1 and EDAS featured the greatest
differences, particularly over the Rocky Mountains and extreme Western
Canada. The inclusion of MODIS snow cover assimilation reduced the
differences in soil temperature throughout these regions, however
significant differences still remain. Differences in soil temperature
correlate well spatially with differences in root zone soil moisture
within the Andes mountain region, and soil temperatures were generally
cooler throughout the southern United States in the uncoupled
simulations. ![](media/image14.png){width="5.84375in" height="4.875in"}

Due to the inclusion of observationally-based forcing data within the
NLDASE simulations and the overall uncoupled modeling approach, NLDASE
does not feature the accumulated biases which can be caused by the
internal cycling of the coupled EDAS system. As previously mentioned,
the NLDAS observed forcing fields and land surface states have undergone
extensive validation and have generally been deemed superior over those
included in the EDAS system. It can thus be expected that the
retrospective land surface states produced by NLDASE, a system which is
modeled after the uncoupled NLDAS approach, are of higher quality than
those produced by
EDAS.![](media/image15.png){width="5.8902777777777775in"
height="4.875in"}

The NLDASE modified Eta restart files used to initialize the workstation
Eta model contain all of the same initial atmospheric conditions as the
operational restart files, and are stored in the same big-endian FORTRAN
IEEE binary file format. The transfer of land surface states from the
NLDASE GRIB files takes approximately one minute to complete on a
standard Linux PC---a quick transfer which is ideal for use in real-time
operational environments. The principle difference between the
operational and NLDASE restart files is the inclusion of NLDASE land
surface states. This form of initialization is transparent to the end
user as the new conditions are written into the modified restart file in
a format which is identical that of the operational conditions. All of
the differences between NLDASE and operational restart files exist in
the land surface states and thus any impacts on the resulting numerical
forecast guidance produced by the Eta model can be directly attributable
to the inclusion of NLDASE land surface states.

### 7.3 Eta Model Current Performance and Intrinsic V&V

Each of the NASA model and data contributions were chosen with the goal
of producing initial land surface conditions for the Eta model which
lead to more accurate Eta weather forecasts than are currently produced
operationally by NCEP. This is a formidable challenge, as the intrinsic
assets that the operational Eta model already makes use of (EDAS initial
land surface states, boundary data sets, assimilation schemes, model
physics, and parameterizations) combine to produce very accurate
forecasts that are constantly improving. As seen in Figure 15, the
average annual equitable threat score for the Eta model's 48 hour
precipitation forecast has increased from approximately 0.34 to 0.43
over the course of nine years. This increase gives a 48 hour
precipitation forecast made in 2002 the same skill level as a 24 hour
precipitation forecast made in 1998. The Eta model also produces very
good surface and upper air forecasts. For example, over the Lower
Mississippi Valley, one of NCEP's official forecast verification
regions, two-meter temperature biases averaged less than 1.4°C for both
12 and 24 hour forecasts over the past month, and RMSE values average
less than 3.0°C for the same time period (Figure 16). Similarly, the 24
hour forecasts of 500mb height and 700mb relative humidity made over the
past month featured biases of only 1m and 0.19% respectively. NCEP
verification web pages feature numerous other examples of the skillful
and increasingly accurate nature of Eta model forecasts.

Unfortunately, given the current operational Eta simulations available,
it is impossible to determine how much of the Eta model's skill is due
to the land surface conditions, versus how much is due to certain
parameter data sets or boundary condition data. Nonetheless, numerous
studies have demonstrated the large impact that initial land surface
conditions have on subsequent NWP model simulations (Delworth and Manabe
1988, 1989; Dirmeyer and Shukla 1993, Koster and Suarez 1995; Beljaars
et al. 1996). Given this, it is expected that the strengths of the
uncoupled NLDASE modeling approach and assimilation of the MODIS snow
cover product will result in high-quality land surface fields that will
have a positive impact on Eta model forecasts. This prediction is
explored in the benchmarking report which follows in later sections.

![](media/image16.png){width="5.99251968503937in"
height="4.630708661417323in"}

![](media/image17.wmf){width="4.918897637795276in"
height="7.698031496062992in"}

## 

## 8. Verification and Validation Gaps

Prior to the start of the benchmarking section, it must be recognized
that the verification and validation activities performed as part of
this study revealed two main data-related verification and validation
gaps. The first of these gaps centers on the lack of adequate in-situ
land surface validation data. Several high quality in-situ land surface
observation networks do exist, monitoring surface energy fluxes, surface
meteorological variables such as radiation, temperature, wind, and
humidity, and sub-surface quantities such as soil moisture and soil
temperature. However, these networks are incapable of providing the
large-scale data-dense measurements needed for domain-wide verification.
This verification is an important part of the overall verification and
validation process, as it aids in gauging the accuracy of Eta model land
surface predictions, the accuracy of operational Eta model initial
conditions, and the accuracy of uncoupled NLDASE initial conditions
which form the foundation of this experiment. Regional networks like
ARM/CART and the Oklahoma Mesonet feature adequate density, but only
limited regional coverage. The Soil Climate Analysis Network (SCAN)
network, by contrast, features national coverage, but low site density
(Figure 17). Together, these networks can supply a rough measure of Noah
LSM performance over the CONUS, and an excellent measure of performance
over the Midwest United States. Unfortunately, these networks cannot
provide any measure of the accuracy of Noah LSM land surface output over
the remainder of the Eta model domain, including Canada, Central
America, and South America. Future in-situ networks and upcoming
satellite missions such as Hydros may assist in the validation of land
surface states and fluxes, but at present the lack of data remains a
shortcoming in the verification and validation
process.![](media/image18.png){width="5.746527777777778in"
height="3.178472222222222in"}

A second data-related verification and validation gap focuses on the
scope of the V&V process in this report. Ideally, all model-related data
sets should be verified and validated in a V&V exercise. In this
experiment, this would include all Eta model and Noah LSM parameter data
sets, all meteorological forcing data sets, and all model state data
sets that are assimilated into either the Eta model or the Noah LSM.
Unfortunately, given the time constraint for this benchmark report, this
extensive verification and validation process proved to be impossible to
complete. However, many of the data sets used in this experiment have
been extensively validated in prior NLDAS research or are currently
being used in NCEP operations, and the uncoupled modeling concept upon
which this research is based is strongly supported by work within the
NLDAS and GLDAS projects. Thus, although an extensive V&V survey is
included in future work plans related to this research, it can be safely
stated that the modeling and data building blocks which form the current
experiment are well established, compatible pieces which together form
an extremely capable modeling system.

# Benchmark

## 9. Overview of Benchmark Design

The significant amount of time it takes to run the Eta model and
post-process the output did not allow for an extensive interannual
benchmarking period. In addition, the six month time constraint of the
benchmarking project limited the number of retrospective forecasts that
could be executed for this report. Therefore, a ten-day study period was
selected to benchmark the performance of the Eta model as initialized
with NLDASE land surface states. The selection of the particular ten day
period was important because a poor choice could have led to biased
results. For example, if a ten day period had been chosen which
contained little or no precipitation throughout much of the model
domain, improvement in the precipitation forecasts would not have been
gauged accurately due to a small sample size of events. A cold season
benchmarking period would have been useful to gauge the impact of NLDASE
snow cover and snow depth on the forecasts; however evapotranspiration
from the land surface is generally dormant during such periods. Thus, in
order to promote the inclusion of a more active land surface, yet at the
same time enable an examination of the impacts of MODIS snow
assimilation, a convectively active Spring season benchmarking period
was deemed optimal. Previous studies have shown that the land surface
can play a critical role in dictating convective initiation and
intensity (Findell and Eltahir 2003; Clark and Arritt 1995).
Additionally, small changes in the planetary boundary layer moisture of
1g/kg can mean the difference between no convection and intense
convection (Crook 1996). A more active land surface and its associated
increases in evaporation can account for such a difference in boundary
layer moisture. It was also hypothesized that the NLDASE land surface
states would have the largest impact in a convectively active period and
an improvement in numerical forecast guidance is arguably most valuable
at such times so that more accurate severe weather watches and warnings
can be produced.

In May 2003 there was a massive convective outbreak across the central
and eastern United States. In total there were 361 verified tornadoes
during the period of May 3 to May 11, 2003 (Figure 18). Of the 361 total
tornadoes, 65 were rated F2 or higher on the Fujita Scale (Fujita 1971)
with 7 of them rated at or above the F4 level. From May 3^rd^ to May
11^th^ 2003 there were a total of 41 fatalities, with 38 taking place on
May 4^th^ 2003 when 81 tornadoes were recorded. Significant tornado
outbreaks such as this require the existence of ample atmospheric
instability and significant wind shear over a large geographic
region---conditions which can be supplied in the Spring by the movement
of low pressure systems eastward from the Rocky Mountains. Throughout
the outbreak, large scale instability was present throughout much of the
central and eastern United States, caused primarily by a persistent
strong southerly low level jet and southwesterly flow aloft (Hamill et
al. 2005). The persistence of the aforementioned patterns was a key
component in contributing to the longevity of the outbreak. Hamill et
al. (2005) also noted the lack of northerly winds from Canada; because
of this, no strong cold fronts were able to push southward and eastward
to dislodge the aforementioned unstable air mass and thus end the
convective outbreak. They also noted the presence of strong shear over
Oklahoma, central and eastern Kansas, Missouri, and western parts of
Illinois, Kentucky, and Tennessee---areas which featured the largest
numbers of tornados in the outbreak. Overall, the SPC and National
Weather Service Forecast offices (Hamill et al. 2005) forecast this
event well, relying in part on forecast guidance from the Eta model.
Tornado watches on May 4^th^, 2003, the most active day, were issued
with an average lead time of over 2 hours, and weather forecast office
warnings were issued with an average lead time of 19 minutes (Department
of Commerce 2004).

![](media/image19.wmf){width="5.995138888888889in"
height="4.405555555555556in"}The May 2003 event was selected due to the
persistence of the convective activity throughout this period, thereby
providing the potential for large differences in the forecasts to emerge
from the use of NLDASE land surface states in the Eta model. It was also
selected due to the fact there existed a significant amount of snow
cover throughout eastern Canada and portions of the intermountain
western United States, which allowed for assessment on the use of MODIS
snow cover assimilation and its impacts on the Eta model forecasts.

A total of 80 workstation Eta model forecasts were executed over the 10
day benchmarking period from May 1^st^ to May 10^th^, 2003, and their
resulting 84-hour forecasts encompassed the period from May 1^st^
through May 14^th^. Twenty forecasts were run using each of four sets of
initial conditions. These four sets include land surface states from the
operational EDAS system (the control run), and the LIS1, LIS2, and LIS3
retrospective offline land surface simulations. These workstation Eta
model simulations are hereafter referred to as the control, LIS1, LIS2,
and LIS3 simulations. Forecasts were initialized at both 00 and 12 Z
from May 1^st^ to May 10^th^ 2003 and run out to 84 hours in order to
mimic the production of operational Eta model forecast guidance during
this period. All 80 forecasts were then compared with observations. In
particular, the LIS1, LIS2, and LIS3 runs were evaluated against the
control run in order to gauge forecast improvement/degradation from the
use of NLDASE uncoupled land surface states.

## 10. Overview of Benchmark Metrics

The focus of this benchmarking effort is to investigate the impact on
Eta model forecasts of using uncoupled NLDASE land surface states as Eta
model initial land surface conditions, and to transition such knowledge
into the NCEP modeling community. Given the important role that the land
surface plays in regulating energy and water fluxes, it can be expected
that changes in initial Eta model land surface states may influence not
only forecasts of surface meteorology such as temperature, humidity and
wind, but precipitation and upper air meteorology as well. These impacts
may be felt over localized areas, or may affect much larger areas. With
this in mind, the benchmark system had to satisfy the following three
important criteria: 1) Capable of site-specific, regional, and national
verification, 2) Capable of validating multiple meteorological aspects
of Eta model forecasts, 3) Has high level of acceptance and familiarity
in NCEP operations to support smooth transfer of benchmark results to
partner agency. Given these conditions, NCEP's Forecast Verification
System (FVS) was chosen as the centerpiece of the regional and national
benchmarking effort, while a separate suite of site-specific
benchmarking metrics was utilized to provide complimentary local
analyses.

### 10.1 NCEP FVS Regional Benchmark Metric Overview 

Used operationally by NCEP to validate their global and regional NWP
models, the FVS consists of two components: 1) A system which generates
mean regional values of the variables being validated, and 2) A
computation and visualization package which transforms the numerical
output from the first element into summary statistical plots (Brill
1999). FVS computes regional statistics over the 19 validation regions
depicted in Figure 19. The system operates by first bilinearly
interpolating gridded model data to the locations of the actual
observations. FVS can accept observations from a wide range of sources,
but for the purposes of this study it was configured to use only
conventional upper air data and surface measurements valid at 2m and 10m
heights. Over each validation region, these observations, 'o', are then
paired with model forecast values, 'f', to compute the following:
mean(f), mean(o), mean(f\*o), mean(f\*f), mean(o\*o), and number of
pairs. For u and v wind component fields, the pairs are used to compute
a separate set of quantities: mean(uf), mean(vf), mean(uo), mean(vo),
mean(uf\*uo+vf\*fo), mean(uf\*uf+vf\*vf), mean(uo\*uo+vo\*vo), and
number of pairs. Through multiplication with the data count, these means
produce a collection of partial sums which can then be used to compute
over 20 different verification statistics.

Additional skill scores can be computed by verifying precipitation
fields against the 1/8^th^ degree CPC PRISM-corrected daily gauge data
set. These skill scores depend upon the storage of a different set of
observation- and forecast-based values. These include the total number
of verification grid points (T), the fraction (F) of grid points which
contain a precipitation forecast above a selected threshold, the
fraction (O) of grid points which contain a precipitation observation
above a selected threshold, and the fraction (H) of grid points which
contain both a forecast and an observed precipitation value above a
selected threshold. Typical precipitation thresholds used in NCEP
precipitation verification operations are 0.01, 0.10, 0.25, 0.50, 0.75,
1.00, 1.50, 2.00, and 3.00 inches.

Guided by a set of user-defined parameters, the second part of the FVS
searches through archives of the data values described above to produce
custom statistical plots and summary values. The software is capable of
producing plots over the 19 validation regions, as well as plots of
statistics covering any combination of regions. While plot axes vary,
the x-axis generally represents the variation in forecast hour or
precipitation threshold, while the y-axis generally represents the
actual statistical measure computed by the FVS. The system is also
capable of averaging all forecast hours together to create a single
summary value, which is useful for summarizing the overall behavior of
the forecast field being verified.

![](media/image20.wmf){width="5.097244094488189in"
height="3.703543307086614in"}Although the flexibility of the FVS makes
it a powerful research tool, it can also lead to an overload of data
given the large number of possible statistical analyses. Given this
flexibility, it was essential to select a small number of statistics to
be used as benchmark metrics in this research. The number of statistical
analyses had to be manageable, yet needed to remain large enough to
ensure a comprehensive evaluation of Eta model output. Following NCEP
operational practices, it was determined that the bias and root mean
squared error (RMSE) statistics would be computed for surface
meteorological output (humidity, wind, temperature) and upper air output
(850mb temperature, 300mb temperature, 700mb humidity, 500mb height, and
250mb winds). Similarly, a subset of FVS statistics consisting of the
false alarm ratio (FAR), the equitable threat score (ETS), the
probability of detection (POD), and the bias was selected for
application to precipitation forecasts. These statistics provide a broad
yet manageable analysis of Eta model forecasts, and, given their
resemblance to the selections used in operational verification
procedures at NCEP, will be efficiently transferred into, and
interpreted by, the operational NCEP community.

In the Forecast Verification System, the bias of a non-precipitation
field is a measure of the difference between the mean forecast value and
the mean observed value, and can be written as:

-   Bias=Mean(f)-Mean(o) **(2)**

where f is the forecast value and o is the observed value (Brill 2005).
Bias is a basic measure of the error contained in the forecast field,
and specifies whether the variable of interest has been overforecast or
under forecast. As this measure is a simple difference between fields,
the bias value maintains the original units of meteorological fields
being examined.

Complementing the bias statistic in this benchmarking report is the root
mean squared error statistic. RMSE is the positive square root of the
mean squared error (MSE), and like the bias statistic, maintains the
units of the original meteorological fields being analyzed. RMSE can be
represented as:

-   RMSE=$\sqrt{\frac{1}{M}\sum_{m = 1}^{M}\left( f_{m} - o_{m} \right)^{2}}$
    **(3)**

where M is the total number of forecast and observation pairs, f is the
forecast value, and o is the observed value (Wilks 1995).

When benchmarking precipitation forecasts, the bias statistic is joined
by the FAR, POD, and ETS statistical measures. Bias is computed
differently when verifying precipitation forecasts, than when analyzing
forecasts of temperature, wind, height, or humidity. According to
Staudenmaier (1996), precipitation forecast bias can be computed as:

-   $\text{Bias} = \frac{F}{O}$ **(4)**

where

-   F = fraction of grid points which contain a precipitation forecast
    above a selected threshold

-   O = the fraction of grid points which contain a precipitation
    observation above a selected threshold

Unbiased forecasts have a bias value of one, values less than one
indicate precipitation events were forecast less often than observed,
and values greater than one indicate an overforecasting of precipitation
events. While bias is a useful measure for determining the
overforecasting or underforecasting of precipitation events, its value
does not reflect the correct or incorrect placement of precipitation
patterns (Staudenmaier 1996).

The false alarm ratio represents the proportion of forecast
precipitation events which fail to occur. It can be computed as:

-   $\text{False Alarm Ratio} = 1\text{.}0 - \frac{H}{F}$ **(5)**

-   H = the fraction of grid points which contain both a forecast and an
    observed precipitation value above a selected threshold

A FAR value of zero represents a perfect score with no false alarms,
while a score of one represents the worst possible score (Stephenson
2000). The FAR is sensitive to false alarms (model forecasts
precipitation but it is not observed) and hits (model forecasts
precipitation and it is observed), but not to missed events (model does
not forecast precipitation but it is observed).

The third precipitation forecast evaluation measure used operationally
by NCEP and implemented in this benchmarking study is the probability of
detection. The POD represents the fraction of instances in which a
precipitation event was both forecast and observed. It is written as:

-   $\text{Probability of Detection} = \frac{H}{O}$ **(6)**

A perfect POD score is one, while the worst POD score is zero
(Stephenson 2000; Wilks 1995). POD scores are sensitive to hits and to
missed events, but not to false alarms.

The final precipitation score used in this research study is the
equitable threat score. According to Schaefer (1990), the ETS can be
written as:

-   $\text{Equitable Threat Score} = \frac{\left( \text{H-CH} \right)}{\left( F + \text{O-H-CH}) \right)}$
    **(7)**

-   CH = $F \times \frac{O}{T}$, the expected number of correct
    forecasts due to chance

-   T = the number of grid boxes inside the verification region

The ETS is a solid estimate of overall precipitation forecast skill
(Staudenmaier 1996), and can range from a perfect score of one, to a
small negative number. A value of zero implies that the model has no
forecast skill (i.e., same as forecast based only on chance), while a
negative value implies that the model is performing worse than if
precipitation forecasts were based only on chance (Brewster et al.
2003). The ETS seeks to eliminate the weakness inherent in a standard
threat score in which a score can be improved by increasing hits at the
expense of an increase in the bias (Rogers et al 1996). Accordingly, ETS
values will decrease if the model forecasts precipitation in an
incorrect location, or if precipitation is observed but not forecast in
a particular location (Staudenmaier 1996).

Application of the bias, RMSE, POD, ETS, and FAR statistical metrics to
the four Eta model simulations which form the basis of this experiment
led to a comprehensive analysis of the impact of NLDASE land surface
states on Eta model forecasts, and is detailed in the sections which
follow.

### 10.2 Site Specific Benchmark Metrics

The aforementioned FVS benchmark metrics are frequently used by NCEP to
evaluate Eta model performance, and provide copious amounts of useful
information as to where weaknesses are present in numerical forecast
guidance. However, these benchmarks are regional in nature and cannot
depict the true impact that the forecast improvement/degradation may
have on a single location. For example, the 2m relative humidity bias
may be 2% too high in the Midwest verification region in the LIS1
forecasts, and 4% too high in the control Eta forecasts. This is not a
large difference in the regional bias, but at a specific location, the
biases in relative humidity may be larger and may have significant
ramifications on local forecast guidance. A selection of surface station
time series are presented to highlight the impacts that NLDASE
initialization has on site-specific ETA forecasts of temperature,
relative humidity, and mean sea level pressure forecasts. Precipitation
timing, placement, and intensity errors will also be presented
graphically to illuminate some of the differences that may not be
depicted in the aforementioned traditional regional FVS skill scores.
Finally, the NCEP FVS system does not evaluate surface irradiative
fluxes; therefore, time series of surface downwelling/upwelling
shortwave and longwave radiation forecasts and observations are
presented to evaluate the impact NLDASE land surface states have on
these fields.

Site specific validation focused on four 84 hour Eta forecasts (control,
LIS1, LIS2, and LIS3) that were initialized at 12 Z on May 3^rd^, 2003.
Hourly surface temperature, humidity, and mean sea level pressure
station data were extracted from the same PREPBUFR files used in
creating the regional FVS statistics. Surface fields from the Eta
forecasts were bilinearly interpolated to station locations which
allowed for a direct comparison of all four forecasts to each another
and to the surface station observations. Data from the Surface Radiation
Budget Network (SURFRAD) were used to evaluate the surface radiation
fluxes against Eta data bilinearly interpolated to the SURFRAD sites.
For precipitation forecasts, NCEP Stage II hourly precipitation data
were compared to hourly precipitation output from the Eta forecasts to
illustrate improvements/degradations. Ideally the NEXRAD Level III
reflectivity and one hour precipitation data should also have been
included for a more complete depiction of precipitation coverage;
however, large gaps in the dataset at the National Climatic Data Center
(NCDC) prevented it from being used in the analysis (W. Brown, personal
communication).

## 11. Analysis of Regional Benchmark Findings

As discussed above, alterations in the Eta model's initial land surface
states were expected to affect not only Eta model forecasts of surface
conditions, but also upper air conditions and precipitation. This
prediction was borne out in the extensive benchmarking efforts which
relied on the metrics discussed in the previous section. For clarity,
discussion of regional benchmarking results is divided into three main
sections: 1) surface verification (2m temperature, 2m humidity, 10m wind
speed), 2) upper air verification (850mb temperature, 300mb temperature,
700mb humidity, 500mb height, and 250mb wind speed), and 3)
precipitation verification.

Although raw numerical verification scores were computed for each of
these fields, the following discussion focuses on the percent
improvement (versus the control run) of each of these scores as opposed
to the raw scores themselves---a more intuitive measure of the effect of
the initialization strategy being examined.

### 11.1 Surface Verification (2m Temperature, 2m Relative Humidity, 10m Wind Speed)

The percent improvement values summarize the performance of the Eta
model over all of the 84 hour forecast periods covering May 1^st^
through May 14^th^, 2003. More specifically, raw verification values
from each three hour forecast period from 0 hours out to 84 hours were
averaged together to form an overall verification score. This averaging
process was necessitated by the overwhelming amount of verification
information. Furthermore, due to space limitations, only selected
examples of FVS output (i.e., from the 00Z LIS2 run) can be shown in the
figures below. Results from Eta simulations initialized at 00Z were
analyzed separately from those of Eta runs initialized at 12Z in order
to preserve the diurnal signatures of forecast performance.

#### 11.1.1 2m Temperature Bias

On average, use of NLDASE land surface states for Eta model initial
conditions led to a strong improvement in Eta model forecasts of 2m
temperature. 00Z Eta bias over the eastern half of the domain improved
by 13.7% through the use of uncoupled initial conditions in LIS1, and
was improved even more (17.5% and 21.1%) when MODIS snow cover was
utilized through the LIS2 and LIS3 simulations. Smaller but mostly
desirable impacts were seen in the 12Z Eta forecast runs as well. Bias
was improved by 2.7% in the LIS2 runs and by 7.2% in the LIS3 runs, but
actually worsened on average by 0.3% in the basic LIS1 simulations. Like
the eastern half of the domain, the western half benefited overall from
uncoupled initial conditions, with improvements of 10.9%, 13.4%, and
16.1% in the 00Z LIS1, LIS2, and LIS3, forecast runs, and 6.6%, 8.6%,
and 12.1% in the 12Z LIS1, LIS2, and LIS3 Eta simulations.

As illustrated in these results, the strongest improvements were seen in
the LIS3 simulation which utilized initial conditions incorporating
MODIS snow cover (Figure 20). The effect of this snow cover is most
evident over the Southern Mountain region, where use of MODIS snow
information led to a further improvement over the non-MODIS LIS1 run by
11.4% in the 00Z runs, and by 17.3% in the 12Z simulations. Other
mountainous regions which featured Spring snow cover benefited as well,
with the Northern Mountain region switching from a 4.1% degradation in
bias in the 00Z LIS1 simulations, to a 1.8% improvement in bias in the
00Z LIS3 simulations.

Overall, use of NLDASE initial conditions greatly improved 2m
temperature bias in the southern half of the CONUS. However, bias values
degraded in Canada, Mexico, and the Gulf of Mexico region by at least
250%, and smaller degradations in bias were noted in the northern half
of the CONUS, especially in the LIS1 and LIS2 simulations. Notably, the
12Z simulations featured a large improvement in bias over the Northwest
Coast region, while the 00z simulations featured a large degradation in
bias over this same region. The opposite was true for the SWD region.
![](media/image21.wmf){width="4.627083333333333in"
height="2.509027777777778in"}

#### 11.1.2 2m Temperature RMSE

As was the case with 2m temperature bias, the use of NLDASE initial
surface conditions to initialize the Eta model led to improvements in
both East region and West region 2m temperature RMSE values. Although
the improvements were smaller than those seen in the bias results
described above, they were more uniform, with positive results across
the board. RMSE improved in the eastern half the domain by approximately
2.3% in all three runs, while western RMSE improved by 1.7%, 2.0%, and
2.4% in the 00Z LIS1, LIS2, and LIS3 runs respectively (Figure 21). From
this it can be deduced that the addition of MODIS snow cover into the
initialization process had only regional impacts, in contrast to the
continental-scale affects that emerged in the bias discussion above.
Another contrast between the bias and RMSE results is that there are
only small differences between the 00Z and 12Z Eta simulations. Because
the improvements and degradations in RMSE from using NLDASE initial
conditions are roughly one order of magnitude less than the parallel
changes in bias, this result might be expected. However, the differences
between 00Z and 12Z RMSE results are even smaller than this would
support. As such, it can be stated that the mechanism by which RMSE is
improved in the LIS1, LIS2, and LIS3 simulations is less sensitive to
the diurnal cycle than is the mechanism by which bias is degraded.

Region-by-region comparison of results reveals a pattern of improvement
and degradation that differs slightly from that seen in the bias
analysis. Rather than a general partitioning of the domain into northern
and southern halves, the results form an east-west partitioning of the
CONUS. RMSE is favorably impacted in the eastern half of the country as
well as the western quarter of the country, with improvements in 00Z
LIS3 RMSE ranging from 0.3% in the Northwest Coast region, to 8.6% in
the Lower Mississippi Valley region (Figure 21). Degradations in 00Z
RMSE are seen in the remainder of the area throughout Mexico, the
Southwest Desert, the Northern Plains region, and eastern Canada, with
LIS3 degradations ranging from 1.4% in Mexico to 3.5% in the Northern
Plains region. ![](media/image22.wmf){width="4.627083333333333in"
height="2.5659722222222223in"}

MODIS snow cover appears to once again have had a favorable impact, with
LIS1 RMSE values from the 00Z Eta simulations degrading by 0.7% versus
the control run over the Southern Mountain region, but with LIS2 and
LIS3 values improving by 0.3% and 0.5%. This demonstrates that although
the use of the basic uncoupled approach can sometimes have an
unfavorable impact on Eta forecasts, the addition of MODIS data into the
creation of initial conditions has the potential to reverse this result
in some regions.

#### 11.1.3 2m Relative Humidity Bias

Continuing the trends discussed in the preceding two sections, use of
NLDASE land surface states to initialize Eta model forecasts leads, on
average, to large improvements in the bias of 2m relative humidity
forecasts. The eastern half of the CONUS experiences an improvement in
bias of 56.6%, 68.3% and 78.2% in the 00Z LIS1, LIS2, and LIS3 runs
respectively, while improvements of 12.9%, 13.6%, and 14.9% are present
in over the western half of the CONUS in these same simulations.
Although East and West region bias is favorably impacted in both the 00Z
and 12Z simulations, it is worth noting that western bias improved more
in the 12Z simulations than in the 00Z simulations, while the reverse is
true of the bias over the eastern half of the domain. An example of 12Z
bias values is given in Figure 22.

The assimilation of MODIS snow cover once again has a favorable impact
on Eta model forecasts, with the LIS3 simulation featuring the lowest
overall bias values of the three LIS simulations or the control run. The
area which benefited most from MODIS snow assimilation was the Midwest
region. In this area the 00Z LIS1 and LIS2 runs featured degradations in
bias over the control run of 475.2% and 204.9% respectively, while the
LIS3 run featured a degradation in bias of only 0.7%. Thus, although the
net impact of NLDASE initialization remained undesirable, assimilation
of MODIS snow cover pushed the LIS2 and LIS3 simulations back in the
desired direction. This favorable influence extended over the entire
domain, and impacted even snow-free areas as was the case with 2m
temperature bias. From this behavior it can be inferred that the air
masses which feature lower humidity bias and benefit from initialization
with MODIS snow data maintain their low humidity bias as they are
advected into warmer regions which lack snow cover.

Regionally, the western third and south-central section of the CONUS
benefit most from NLDASE initialization. In these areas, 2m relative
humidity bias is improved by anywhere from 6.9% over the Northern
Mountain region in the 00Z Eta forecast runs, to 77.3% over the
northwest coast region in the 12Z forecast runs (Figure 22). Areas in
which forecast bias is worsened include western Canada, the Gulf Coast
region, and the Atlantic seaboard. In particular, bias values degraded
by up to 26,187% in the 12Z Eta runs over the Southeast Coast region.
Although on the face of it this result sounds extremely bad, it is more
of a mathematical artifact than a consequential result. The control run
featured a very low relative humidity bias of 0.6%, with the LIS1, LIS2,
and LIS3 runs featuring low relatively humidity biases of approximately
2.4%. Both of these values are well below the average humidity bias
featured across the rest of the domain. Thus, it can be seen that the
large percent degradation in bias versus the control run can be traced
more to the extremely good bias values in the control run, as opposed to
extremely poor bias values in the LIS simulations.

#### 11.1.4 2m Relative Humidity RMSE

An examination of 2m relative humidity RMSE values from the control,
LIS1, LIS2, and LIS3 Eta simulations reveals both similarities and
differences compared to the bias values detailed in the previous
section. Once again, use of NLDASE initial conditions led to
improvements in East and West region values for both 00Z and 12Z runs
across all LIS simulations. 12Z simulation relative humidity RMSE
improved by 5.6%, 5.9%, and 6.3% over the western half of the CONUS in
the LIS1, LIS2, and LIS3 simulations, while it improved by 5.3%, 5.4%,
and 5.4% over the eastern half of the CONUS in those same runs (Figure
23). However, while the consistent improvement in RMSE is similar in
basic nature to the improvement in bias, its magnitude is much less.
This finding is consistent with that detailed in the temperature
analysis above.

Similarities and differences continue to manifest themselves in a
regional analysis as well. Whereas poor bias performance was noted along
the Gulf, Southeast and Northeast Coast regions, these same areas now
feature RMSE improvements in the 12Z simulations of 11.8%, 0.8%, and
1.4% respectively (Figure 23). Western Canada and the Midwest region now
also exhibit an improvement, with RMSE values improving by 2.8% and
1.6%. Interpreted qualitatively, those areas where RMSE was favorably
impacted but bias worsened, feature Eta forecasts of humidity that are
less variable than those of the control run, but which are also
consistently too high or too low more often than the control run. RMSE
values in the remaining regions bear similarities to bias values of the
previous section. NLDASE initialization caused Eta model forecasts of
relative humidity to worsen in RMSE in Mexico and the Northern Plains
region, while the rest of the CONUS featured improvements in RMSE of up
to 18.1%.

![](media/image23.wmf){width="0.6in"
height="0.12992125984251968in"}![](media/image24.wmf){width="0.6in"
height="0.12992125984251968in"} MODIS snow cover, although improving
humidity forecasts in several regions (i.e., up to 1.6% versus control
run in southern mountain region), is once again limited to regional
effects. Regions such as the Southern and Northern Mountains, and
Western Canada benefit from this assimilation, while snow-free regions
are not impacted in either direction. Thus although it appears that
weather systems advected out of snow-covered regions maintain their
improved bias characteristics, they lose any improvement in RMSE that
they may have had.

#### 11.1.5 10m Wind Speed Bias

Whereas Eta forecasts of relative humidity and temperature clearly
benefited from initialization with NLDASE land surface states, Eta
forecasts of wind speed presented a much more mixed message. Overall,
bias in the wind speed field improved over the eastern half of the
CONUS. The largest improvements were seen in the LIS1 simulation, where
00Z simulation bias improved by 4.7%, and bias in the 12Z runs improved
by 4.2%. These values are followed by lesser, although still sizable
bias improvements in the LIS3 and LIS2 simulations. However, the West
region featured degradations in wind speed bias in both the 00Z and 12Z
runs of the LIS1, LIS2, and LIS3 simulations. This contrasts with the
temperature and humidity bias results discussed above, both in terms of
the strictly unfavorable impact on the western region, and in terms of
the fact that the LIS3 simulation did not produce the best forecasts. In
fact, the 7.7% degradation in the West region wind speed bias of the 00Z
simulations makes MODIS-based LIS3 the worst performing simulation
(Figure 24). ![](media/image25.wmf){width="4.627083333333333in"
height="2.4819444444444443in"}

A regional investigation of wind speed bias shows that use of NLDASE
initial land surface states leads to wind speed bias degradations in
almost all western regions, and to bias improvements in all eastern
regions. The only consistent exception is the northwest coast region,
which feature improvements in wind bias of up to 21.3% in the 00Z
simulations (Figure 24). Examination of results outside of the CONUS
continues to set the wind speed forecasts apart from the forecasts of
humidity and temperature. Here, the Eastern Canada, Western Canada, and
Mexico regions all feature improvements in bias, which was not the case
with temperature and relative humidity.

Differences continue in an analysis of the LIS2 and LIS3 MODIS-based
simulations, which illustrates for the first time that assimilation of
MODIS snow cover has a generally unfavorable effect on forecast
accuracy---in this case causing wind speed bias to worsen. Although
there do exist some exceptions (i.e., Southern Mountain region bias
improves by 2.6% in LIS1), much of the snow-covered CONUS features
degraded wind speed forecasts.

This regional assessment of forecast improvement and degradation is
different in overall character from the temperature and humidity
analyses above, but clear conclusions can still be drawn---while MODIS
assimilation appears to have a slight unfavorable impact on wind speed
bias, initialization using NLDASE conditions consistently improves bias
in wind speed forecasts over the eastern CONUS.

####  11.1.6 10m Wind Speed RMSE 

On average, the RMSE of wind speed forecasts improves slightly with use
of NLDASE initial conditions. The eastern and western halves of the
CONUS each feature improvements of up to 1.0% across both 00Z and 12Z
simulations (Figure 25). There is very little variation across either
simulation or cycle time, implying that while use of NLDASE initial
conditions does improve wind speed RMSE in general, the effects are
independent of diurnal influences, and the use of MODIS provides little
if any benefit on the continental scale. Illustrating this, the eastern
and western RMSE values of the LIS3 MODIS-based simulation are actually
the same or worse than the RMSE values in the non-MODIS LIS1 simulation.
![](media/image26.wmf){width="4.627083333333333in"
height="2.5659722222222223in"}

Regionally, initialization with NLDASE conditions improves wind speed
RMSE values along the west coast, over the central portion of the CONUS,
along the gulf coast, and through the Appalachian region. By contrast,
it tends to worsen RMSE levels over western regions such as Mexico, the
Southern Mountains, and the Northern Plains, and over northern regions
such as Canada and the Northeast Coast. The incorporation of MODIS snow
data into Eta initial conditions has little impact on RMSE. All changes
are small in magnitude, with the mountainous western regions benefiting
by about 0.1%, and surrounding regions staying constant or slightly
degrading. LIS2 slightly outperforms LIS3, implying that the thinner
snowpack of the LIS2 simulation leads to better forecasts of wind speed
(from an RMSE perspective) than does the thicker snowpack of the LIS3
simulation.

#### 11.1.7 Surface Verification Summary

As detailed in the surface verification discussions above, Eta forecasts
of 2m temperature, 10m relative humidity, and 2m wind speed were all
impacted by use of NLDASE-based initial conditions. Although impact
varied by region, Table 1 shows that the East and West regions of the
CONUS overwhelmingly benefited, with very large improvements in bias,
and sizeable improvements in RMSE. Several notable conclusions can be
drawn from this data:

-   The use of NLDASE conditions to initialize the Eta model led to
    greatly improved forecasts of humidity and temperature overall.

-   The largest improvements were seen in relative humidity forecasts,
    while the smallest improvements (and some declines in accuracy) were
    seen in the forecasts of wind speed.

-   Use of initial conditions based on assimilated MODIS snow cover data
    generally improved forecasts.

-   In general, the use of the 10mm SWE layer in the MODIS assimilation
    scheme led to better Eta forecasts than did the use of the 5mm SWE
    layer.

-   The addition of MODIS data into the initial conditions had a
    generally continental-scale effect on bias scores, influencing even
    snow-free regions. However, RMSE scores, in general, were only
    impacted on smaller regional scales.

-   Diurnal influences altered the impact of initialization with NLDASE
    conditions, but not in a clear-cut, constant fashion. Improvements
    in 00Z simulations were not consistently more or less than those
    seen in the 12Z
    forecasts.![](media/image27.wmf){width="5.999305555555556in"
    height="1.8909722222222223in"}

### 11.2 Upper Air Verification (500mb Height, 850mb Temperature, 300mb Temperature, 700mb Relative Humidity, 250mb Wind Speed)

As with the surface verification analysis above, the percent improvement
values presented in this section summarize the performance of the Eta
model over all of the 84 hour forecast periods covering May 1^st^
through May 14^th^, 2003. Raw verification values from each three hour
forecast period from 0 hours out to 84 hours were averaged together to
form an overall verification score, and results from Eta simulations
initialized at 00Z were analyzed separately from those of Eta runs
initialized at 12Z in order to preserve the diurnal signatures of
forecast performance. Due to the scarcity of upper air observations,
verification was performed by averaging all CONUS data together, and
statistics were not broken down by region. This procedure follows NCEP's
operational verification procedure. Furthermore, due to space
limitations, only selected examples of FVS output (i.e., 12 hour
forecasts from the 00Z runs) can be shown in the figures below.

#### 11.2.1 500mb Height Bias

Initialization of the Eta model with NLDASE conditions had a mixed
impact on Eta forecasts of 500mb height. Although the overall bias
improved in the 12Z LIS1 and LIS2 simulations by 15.1% and 9.4%
respectively, it worsened in the remaining model runs by 1.4% to 12.4%.
The LIS1 simulation featured smaller degradations of bias values than
the LIS2 and LIS3 simulations, indicating that the addition of MODIS
data had an undesirable impact on 500mb height forecasts. In particular,
the LIS3 simulation featured the largest bias degradations---a finding
which contrasts sharply with the surface verification results.

![](media/image28.wmf){width="5.992913385826772in"
height="3.87992125984252in"} An hour-by-hour examination of bias scores
reveals only two clear trends. The first is that initialization with
NLDASE conditions tended to improve bias in near-term (12 hour)
forecasts, an example of which is shown in Figure 26. The second trend
is that the 12Z simulations outperformed the 00Z simulations for most
forecast lead times.

#### 11.2.2 500mb Height RMSE

An analysis of 500mb height RMSE presents a similarly mixed image.
Overall, RMSE levels tended to slightly improve in the LIS1 and LIS2
simulations, and to worsen in the LIS3 simulations. In all three cases,
changes were small, with improvements of less than 1.1%, and a worsening
of RMSE values of less than 2.8%. As was the case with bias, the LIS3
simulation featured the worst forecasts, indicating that use of the 10mm
MODIS SWE layer was the least desirable option. LIS2 and LIS1 performed
similarly, providing little guidance as to the use of the 5mm MODIS snow
option.

Short term forecasts again benefited from NLDASE initial conditions,
with 24 and 36 hour forecasts improving in the 00Z LIS1 and LIS2
simulations, and 12, 24, and 36 hour forecasts improving in the 12Z LIS1
and LIS2 simulations. In a switch from the bias results, the 00Z
simulations featured better overall RMSE values than the 12Z
simulations.

#### 11.2.3 850mb Temperature Bias

The impact of NLDASE initial conditions on 850mb temperature bias is
clearer cut, and is generally unfavorable. Bias levels worsened by 90.3%
to 216.5% in the 00Z LIS1, LIS2, and LIS3 simulations and by 84.0% to
146.5% in the 12Z simulations. The few favorable impacts which exist are
located in the 12 and 24 hour forecast categories. Here, the 00Z LIS1,
LIS2 and LIS3 runs experienced scattered improvements of up to 22.2%,
while the 12Z LIS1 simulations experienced bias improvements of up to
11.8%. However, these improvements are greatly outnumbered by the cases
where bias worsened.

Use of MODIS data appears to help improve forecast accuracy, as the
non-MODIS LIS1 simulations feature the worst bias, followed by the LIS2
and LIS3 simulations. This is illustrated for 84 hour forecasts in
Figure 27. However, the favorable influence of the MODIS data is not
large enough to counteract the large unfavorable impact of the basic
uncoupled initialization approach. This statement holds true for both
00Z and 12Z simulations as both cycles performed poorly. Although the
12Z simulations performed slightly better than the 00Z simulations on
average, the 00Z simulations featured better short term temperature
forecast bias scores.

#### 11.2.4 850mb Temperature RMSE

850mb temperature RMSE behaves entirely differently than 850mb
temperature bias. LIS1, LIS2, and LIS3 simulations consistently featured
slightly improved RMSE values on the order of 1% to 3%. The only major
exceptions were the 12 hour 00Z simulations, and the 84 hour 12Z
simulations, which featured degradations of RMSE of between 0.06% and
3.5%. This result highlights the major difference between the 00Z and
12Z simulations---short term forecasts turned out to be the major
weakness in the 00Z simulations, while long term forecasts were the
major weakness in the 12Z simulations.

Use of MODIS snow cover data acted to improve the 850mb temperature RMSE
throughout the forecast period. On average, LIS2 featured the best bias
scores in the 12Z simulations, while LIS3 featured the best bias scores
in the 00Z simulations.

#### 11.2.5 300mb Temperature Bias

As was the case with 850mb temperature RMSE values, 300mb bias values
were generally improved through use of NLDASE initial conditions. In
particular, bias values improved by 2.6%, 2.6%, and 2.4% on average in
the 00Z LIS1, LIS2, and LIS3 simulations, and by 0.1% in the 12Z LIS2
simulation. An example of this behavior for the 84 hour forecast lead
time can be seen in Figure 27. The exceptions to this favorable result
are the 12Z LIS1 and LIS3 simulations which featured degradations in
bias of 0.1% and 0.7% respectively.

An hour-by-hour analysis highlights the consistent improvements of bias
in the 00Z simulations, with only the 24 hour forecasts exhibiting
worsened bias. This same examination yields different results when
focusing on the 12Z simulations. Here, the results are much more mixed,
with degradations in bias in the beginning and ending forecast periods,
and bias improvements in the mid-term 24-48 hour forecast periods.

![](media/image29.wmf){width="5.992913385826772in"
height="3.8791338582677164in"} Results are also mixed in terms of the
benefits of MODIS snow assimilation. Averaged over all forecast lead
times, MODIS snow cover did not improve 00Z simulation temperature bias
levels, with LIS2 and LIS3 underperforming LIS1. However, there exist
several forecast lead times, including 12, 48, 72, and 84 hours, in
which LIS2 or LIS3 featured better bias levels than did LIS1. Results
are slightly more consistent in the 12Z simulations, where LIS2 featured
an improvement in overall bias, while LIS1 and LIS3 featured
degradations in bias levels. Based on these results, it is difficult to
determine if the assimilation of MODIS data had a significant favorable
or unfavorable impact on bias levels.

#### 11.2.6 300mb Temperature RMSE

300mb temperature RMSE values, like bias values above, were improved in
the 00Z simulations through use of NLDASE conditions. Although these
improvements were small in magnitude, they extended throughout most of
the forecast periods, with only the 72 and 84 hour lead times
experiencing RMSE degradations. Results were similar in the 12Z
simulations, where improvements in RMSE ranged from 0.1% to 1.4%, and
degradations ranged from 0.6% to 4.6%. Here again, short and mid-term
forecasts improved, while long-lead forecasts degraded. Although results
varied hour-by-hour, the 00Z simulations benefited more on average from
NLDASE initialization than did the 12Z simulations.

Use of MODIS snow cover proved to have an unfavorable effect on
forecasts, with the largest improvements in RMSE over the control run
occurring in LIS1---the simulation which did not make use of MODIS data.
LIS2 and LIS3 had smaller overall improvements, with the 12Z LIS3
simulations even featuring a worsening in RMSE compared to the control
run.

#### 11.2.7 700mb Relative Humidity Bias

![](media/image30.wmf){width="5.992913385826772in"
height="3.8771653543307085in"} Bias levels of 700mb relative humidity
forecasts were greatly degraded through NLDASE initialization. Forecasts
were degraded across multiple forecast lead times, and across both 12Z
and 00Z simulations. Bias worsened by 37.8%, 35.3%, and 36.5% in the 00Z
LIS1, LIS2, and LIS3 runs, and by 35.2%, 32.9%, and 41.6% in their 12Z
counterparts. An hour-by-hour analysis shows that bias tended to worsen
as forecast lead time increased, with 12 hour forecasts only featuring
bias degradations between 4.2% and 8.0%. An example of bias at the 60
hour lead time is shown in Figure 28.

The addition of MODIS snow cover information into the forecast process
of the LIS2 simulations led to a slight improvement in overall bias
versus the LIS1 simulations. Of note, this same improvement was not seen
in the LIS3 simulation, where the 10mm SWE layer was used. LIS2 results
notwithstanding, the small changes which resulted from use of MODIS data
could not overcome the very large unfavorable impact which arose from
the use of uncoupled initial conditions.

#### 11.2.8 700mb Relative Humidity RMSE

An analysis of 700mb relative humidity RMSE presents a much different
picture than did the preceding analysis of bias. Results are mixed, with
all three 00Z simulations worsening in RMSE, and all three 12Z
simulations improvements in RMSE. Changes are relatively small, with
degradations of RMSE of less than 0.4% and improvements in RMSE of less
than 1.0%. As has been the case with many of the upper air variables
discussed so far, improvements in 00Z and 12Z simulation RMSE generally
occurred at the shorter lead times of 12 and 24 hours. Improvements in
RMSE also occurred at the long-lead times of 60-84 hours in the 12Z
simulations. These results highlight the strong impact of diurnal
influences on the forecasts of humidity.

The impact of MODIS data on humidity RMSE was minimal, but mostly
unfavorable. Both 00Z and 12Z LIS2 simulations underperformed the
non-MODIS LIS1 simulations, as did the 12Z LIS3 simulation.

#### 11.2.9 250mb Wind Speed Bias

As with relative humidity, wind speed bias was unfavorably impacted
through use of NLDASE initial conditions. Across all forecast hours,
bias levels worsened by 1.8%, 1.1%, and 0.5% in the 00Z LIS1, LIS2, and
LIS3 simulations, and by 5.3%, 4.9%, and 3.2% in their 12Z counterparts,
indicating a strong diurnal influence on wind speed bias levels. An
example of bias in the 00Z simulations at the 36 hour lead time is given
in Figure 29. Scattered improvements in bias did occur, but were far
outnumbered by degradations in bias. Even so, it is worth noting that
the 12 hour lead time forecasts of the 00Z LIS simulations each featured
bias improvements. This continues the trend noted in previous upper air
discussions of forecast improvement at short lead times.

Use of MODIS data had a favorable impact on wind speed RMSE, with LIS2
and LIS3 featuring better bias values than LIS1. In particular, the 10mm
value used in LIS3 led to better bias values than did the 5mm value used
in LIS2. Nonetheless, neither application of MODIS data was able to
completely counteract the large unfavorable impact of the uncoupled
initialization procedure.

#### 11.2.10 250mb Wind Speed RMSE

In general, wind speed RMSE worsened when initial conditions from NLDAS
were used by the Eta model. Slight degradations in RMSE of 0.09%, 0.06%,
and 0.1% characterize the LIS1, LIS2 and LIS3 00Z simulations, while
slightly larger degradations in RMSE of 0.5%, 0.4%, and 2.0%
characterized the 12Z simulations. Beneficial impacts were mostly
limited to the short lead forecasts, especially in the 12Z simulations.
In all cases, forecasts with lead times of longer than 60 hours featured
worse levels of RMSE than did the control run.

![](media/image31.wmf){width="5.992913385826772in"
height="3.87992125984252in"} Although use of MODIS data led to
improvements in LIS2 forecasts over the results seen in LIS1, it led to
a degrading of forecasts in LIS3 when compared to the same LIS1
baseline. Though these results suggest that the 5mm SWE thickness value
led to better forecasts than the 10mm SWE value, it is also true that
neither implementation of MODIS snow assimilation was able to improve
upon the control run forecasts.

#### 11.2.11 Upper Air Verification Summary

The verification analyses above highlighted the mixed impacts that
NLDASE initial conditions have on Eta model upper air forecasts of 500mb
Height, 850mb Temperature, 300mb Temperature, 700mb Relative Humidity,
and 250mb Wind Speed. As summarized by Table 2, these impacts varied by
variable and by forecast lead time, with short lead times generally
benefiting from uncoupled initialization, and long lead times generally
degrading from this type of initialization. Several significant
conclusions can be drawn from this data:

![](media/image32.wmf){width="5.047916666666667in"
height="3.1743055555555557in"}

Table 2. Percent improvement in bias and RMSE of LIS1 (L1), LIS2 (L2),
and LIS3(L3) runs over control simulation for 850mb and 300mb
temperature (T850, T300), 700mb relative humidity (RH700), 250mb wind
speed (W250), and 500mb height (Z500). Warm colors indicate
improvements, while cool colors indicate degradations.

-   The impacts on upper air forecasts of using NLDASE states to
    initialize the Eta model are more mixed than was the case with
    surface forecasts.

-   300mb temperature is the only upper air element examined which
    consistently benefits from uncoupled initialization

-   RMSE improves more often in relation to the control run than does
    bias, although the changes in RMSE are very small.

-   The worst impacts on bias are focused on the 850mb temperature and
    700mb relative humidity fields, while the worst impacts on RMSE are
    manifested in the 250mb wind speed fields.

-   Simulations utilizing MODIS snow information (LIS2, LIS3) performed
    better than the simulation lacking MODIS snow information (LIS1);
    however this benefit was often overshadowed by the large detrimental
    impact of the uncoupled initialization approach, and LIS1, LIS2, and
    LIS3 often underperformed the control simulation.

-   Overall, the 5mm MODIS SWE depth option held little or no advantage
    over the 10mm SWE depth option---LIS2 performed best in 7 out of 20
    cases, while LIS3 performed best in 6 out of 20 cases.

-   Diurnal influences impacted forecast results, however no discernable
    pattern emerged from the analysis, with 00Z simulations
    outperforming 12Z simulations in some cases, but not others.

### 11.3 Daily Precipitation Verification (0-24 hour forecast, 24-48 hour forecast, 0-84 hour forecast)

As with the surface and upper air verification analyses above, the
percent improvement values presented in this section summarize the
performance of the Eta model forecasts covering the period from May
1^st^ through May 14^th^, 2003. Following NCEP's operational procedures,
verification was performed against the 1/8^th^ degree CPC gauge
precipitation product, valid each day over a 24 hour period extending
from 12Z to 12Z. As the verification data set is valid from 12Z to 12Z,
only certain temporal subsets of Eta forecasts could be validated. In
particular, raw verification values were computed for the 0-24 hour and
24-48 hour forecasts of each 12Z Eta forecast. These values were
averaged together to form overall short lead 0-24 hour forecast scores,
and medium lead 24-48 hour forecast scores. A similar procedure was
followed to compute overall scores for the entire 0-84 hour forecast
period. Unlike the short and medium lead forecast scores, this last
score included precipitation forecasts from the 00Z runs as well as the
12Z runs. Scores detailed below include the false alarm ratio, equitable
threat score, bias, and probability of detection.

#### 11.3.1 0-24 hour Bias

Focusing on short lead time precipitation forecasts, use of NLDASE
initial conditions led to improvements in precipitation bias over the
western half of the CONUS of 27.4%, 24.5%, and 24.5% in the LIS1, LIS2,
and LIS3 simulations, but lead to bias degradations of 7.8%, 6.9%, and
8.2% in those same runs over the eastern half of the CONUS. Forecast
improvements were located mainly over the inland regions of the domain,
while the largest degradations were located along the western and
eastern coastal regions. The exception to this rule was the lower
Mississippi Valley region which consistently experienced a degradation
in bias of approximately 15%.

On average, use of MODIS snow cover information worsened the bias in
precipitation forecasts over the East and West regions. At the same
time, a more detailed regional investigation reveals that use of the
MODIS product slightly improved the bias of precipitation forecasts over
several of the snow-covered regions. A prime example of this result
occurred in the southern mountain region, where the non-MODIS LIS1
simulations featured an 8.4% improvement in bias, while the LIS2 and
LIS3 simulations featured improvements of 10.5% and 9.0%. This same
investigation showed that these effects were continental in scale, with
even snow-free regions showing improvements or degradations in bias
after the addition of MODIS information into the simulation.

#### 11.3.2 24-48 hour Bias

In a reversal from the short lead forecast results, bias in medium lead
forecasts improved in the East region, but generally degraded in the
West region. In addition, these changes were much smaller in magnitude
than those seen in the 0-24 hour forecast verification period and
remained less than 6.8% in all cases. Although not without exception,
precipitation forecasts tended to improve over the mountainous west, the
south-central CONUS, and the Northeast Coast region. Use of MODIS snow
cover led to the best East and West region bias scores, with LIS3
outperforming LIS2 and LIS1. However a more detailed examination reveals
mixed results, with some regions faring better in LIS1 than in LIS2 or
LIS3. Effects were once again continental in scale.

#### 

#### 11.3.3 0-84 hour Bias

Continuing the trends seen in the medium lead forecast analysis, bias
levels for the entire 84 hour forecast period degraded over the West
region with the use of NLDASE initial conditions, and improved over the
East region. This behavior is illustrated for the LIS3 simulations in
Figure 30. In particular, the West region saw degradations in bias of
217.6%, 123.5%, and 52.9% in the LIS1, LIS2, and LIS3 simulations, while
the East region saw improvements in bias of 4.0%, 4.0%, and 5.8% in
those same runs. Improvements in bias mainly occurred over the regions
in the southern half of the CONUS, with the largest improvements in bias
occurring over the southern plains area. Here, improvements ranged up to
77.3%, with LIS3 outperforming LIS1 and LIS2 (Figure 30). As this region
was largely snow-free, it demonstrates that benefits from MODIS data
were again large-scale in nature, with improvements in bias persisting
as air masses were advected from snow-covered regions to snow-free
regions. This process led to overall improvements in bias in LIS2 and
LIS3 as compared to the LIS1 across the East and West regions; however,
the impact was too small in the West region to overcome the large
degrading influence of the uncoupled initialization approach.
![](media/image33.wmf){width="4.627083333333333in"
height="2.4819444444444443in"}

#### 11.3.4 0-24 hour Equitable Threat Score

NLDASE initial conditions had a more uniformly favorable impact on short
lead precipitation ETS values than they had on short lead bias scores.
East and West region ETS values were slightly better than the control
run across the LIS1, LIS2, and LIS3 simulations, with the East region
improving more than the West region. Degraded ETS scores were limited to
the Northeast and Southeast Coast regions, the Southern Mountain region,
the Northern Plains region, and the Midwest region. Improved ETS scores
occurred in the remainder of the study domain, with the Southwest Desert
improving by over 40% compared to the control simulation. Aside from
this one large positive impact, all changes to ETS values were quite
small, averaging less than 1%.

Use of MODIS snow cover data had a small, mixed impact on short lead
forecast results. The ETS of the East and West regions changed by less
than 0.2% when MODIS data was used, with the East region improving
slightly, and the West region degrading slightly in LIS2 as compared to
LIS1. East and West region values in LIS3 remained unchanged when
compared to LIS1. An analysis of regions supports the same overall
conclusion---that MODIS data had no substantial impact on short lead
precipitation ETS scores.

#### 11.3.5 24-48 hour Equitable Threat Score

Although of similar magnitude to the 0-24 hour East and West region ETS
values discussed above, the ETS values of the medium lead forecasts
presented a mixed picture for these two regions. NLDASE initial
conditions improved West region equitable threat scores by 0.6% to 0.7%,
but degraded East region scores by up to 0.2%. A more detailed regional
analysis reveals that degraded scores were clustered along the Northeast
and Northwest Coast regions, and through the middle third of the CONUS.
Improvements were present in the remaining areas, with a large cluster
located in the mountainous west. The impact of MODIS snow information
was mixed as well, with some regions benefiting, and others degrading.
Overall impact was small once again, with the East and West regions
changing by less than 0.2%, and smaller regions changing by less than
0.5%. Overall, MODIS had no substantial large-scale impact on ETS
values.

#### 11.3.6 0-84 hour Equitable Threat Score

Impacts of NLDASE initialization on the equitable threat score remain
small when the entire 84 hours of each forecast period are analyzed. ETS
values improved in the East region by 0.1% in both LIS1 and LIS2, and
stayed constant in LIS3 as seen in Figure 31. The West region
experienced an improvement of 0.2% in LIS1, and degradations of 1.8% and
1.9% in LIS2 and LIS3. An investigation of regions provides further
evidence that the ETS remains largely unaffected by use of NLDASE
conditions. Small improvements of less than 0.7% are located throughout
the western regions, the lower Mississippi valley region, the
Appalachian region, and the Southeast Coast region. Degradations are
small as well, with the exception of a 4.4% worsening in ETS which
occurs in LIS2 over the Northern Plains region. Although once again very
small, the impacts of MODIS snow data are more unfavorable in nature
than they were in the short term or medium term forecasts discussed
above. ![](media/image34.wmf){width="4.627083333333333in"
height="2.4819444444444443in"}

#### 11.3.7 0-24 hour Probability of Detection

As was the case with ETS, use of NLDASE conditions within the ETA model
has little impact on short term probability of detection scores. East
region scores remain either unchanged as in LIS2 and LIS3, or degrade by
0.2% as in LIS1. West region scores improved, but only by 1% or less. A
further similarity lies in the regional distribution of POD improvements
and degradations. Regions in the western CONUS experienced improvements
of up to 5.3%, with smaller improvements occurring in the Southern
Plains, Gulf Coast, and Appalachian regions. POD values worsened in the
remaining regions, including the Midwest region, the Northern Plains
region, and those regions located along the Eastern seaboard. MODIS snow
cover data had a small and mixed impact on POD values, but generally
improved scores over the mountainous regions when the 10mm SWE value was
used. Impacts extended away from snow-covered areas, indicating the
persistent nature of the MODIS-engendered alterations in the
precipitation fields.

#### 11.3.8 24-48 hour Probability of Detection

Probability of detection values in the medium lead precipitation
forecasts changed in a very similar fashion as compared to the short
lead POD values. Improvement and degradation magnitudes were universally
small, although slightly larger than the values seen in the short term
fields. In particular, the West region improved by 2.4%, 2.0%, and 2.0%
in LIS1, LIS2, and LIS3. Degradations of 0.2% were noted over the East
region in LIS1 and LIS2, while an improvement of 0.3% was noted in this
same region in LIS3. Regional distribution of POD alterations is
somewhat similar to that seen in the short term analysis, with
improvements occurring in many of the western and central regions of the
CONUS, and degradations occurring in the Gulf Coast, Midwest, and
Northeast Coast regions. While the overall impact of MODIS snow data was
slight and mixed, persistent improvement in POD values occurred over the
snow-covered Southern Mountain region, as well as the snow-free Lower
Mississippi Valley and Appalachian regions, once again indicating the
small, but continental-scale impacts of MODIS data.

#### 11.3.9 0-84 hour Probability of Detection

Although still small in magnitude, POD changes over all forecast lead
times were more uniformly positive than those seen in either the short
term or medium term forecasts. Improvements of 0.7%, 0.7%, and 0.8%
occurred over the West region in LIS1, LIS2, and LIS3, while a smaller
improvement of 0.2% occurred over the East region in those same
simulations. The distribution of improvements and degradations of POD
scores is similar to that noted in the previous discussion, with
degradations occurring in general over the Gulf Coast and Northeast
Coast regions. This arrangement is illustrated for the LIS3 simulations
in Figure 32. The assimilation of MODIS data into the forecast process
had a small, mixed impact on POD values. Once again, some regions
featured improved POD scores as a result of using MODIS data, while
other regions worsened. The snow-covered southern mountain region, as
before, benefited consistently from use of MODIS data, with the best POD
values arising from use of the 10mm SWE
value.![](media/image35.wmf){width="4.627083333333333in"
height="2.5659722222222223in"}

#### 11.3.10 0-24 hour False Alarm Ratio

NLDASE initial conditions impacted short term FAR scores more than they
impacted the POD scores discussed above, improving false alarm ratios in
the East region and degrading values in the West region. Specifically,
FAR values worsened by 11.9%, 0.8%, and 0.9% in LIS1, LIS2, and LIS3
over the West region, and improved by 2.5%, 2.5%, and 2.3% over the East
region. Results for the LIS3 simulations are shown in Figure 33.
Examining the impact among regions, improvements were located in areas
located along the west coast, as well as in the southeastern quadrant of
the CONUS. FAR values degraded in the Northeast Coast, Southern
Mountains, and Plains regions. These changes were sometimes large in
magnitude, with regional FAR values worsening by up to 14.2%, and
improving by up to 15.4%.

Overall, MODIS snow cover data had a limited, but net favorable impact
on short term FAR scores. The impact varied by region though, with a
notable unfavorable impact over the Southern Mountain region. A large
favorable impact occurred over the Northern Plains region where LIS2
improved over LIS1 by 13.6%; however, this was not enough to counteract
the degrading influence of the uncoupled initialization approach, and
LIS2 still underperformed the control simulation. Another area of
positive impact occurred in the lower Mississippi Valley region, where
LIS3 improved over LIS1 by 1.6%. Here, LIS3 FAR scores were 15.4% better
than the control simulation.

#### 11.3.11 24-48 hour False Alarm Ratio![](media/image36.wmf){width="4.627083333333333in" height="2.5659722222222223in"}

Although NLDASE impacts on 24-48 hour FAR scores are smaller than on
0-24 hour FAR scores, the impacts are more uniformly unfavorable. East
region FAR scores degrade by 0.8%, 0.5%, and 0.3% in LIS1, LIS2, and
LIS3, while West region scores degrade by 0.2% and 0.3% in LIS2 and
LIS3, and improve by 0.2% in LIS1. Values vary more widely among
regions, with an improvement of up to 8.7% across the southeastern
quadrant of the CONUS, the Southwest Desert region, and the Northern
Plains region. Degradations of up to 20.6% occur across the remainder of
the regions, with the worst results occurring over the Southern Mountain
region. The impacts of MODIS data were generally small, with exception
of the Southern Mountain region where LIS3 FAR scores were 4.2% worse
than LIS1 scores. These impacts were continental, but well-mixed in
nature, leading to the absence of a clear overall MODIS benefit.

#### 11.3.12 0-84 hour False Alarm Ratio

As was the case in the 0-24 hour FAR analysis, an analysis of 0-84 hour
FAR values shows that the West region was unfavorably affected by NLDASE
initial conditions in LIS1, LIS2 and LIS3, while the East region
benefited from NLDASE conditions in these same simulations. Improvements
and degradations in FAR scores were well-mixed throughout the CONUS
regions, with a maximum improvement of 2.1% in the Lower Mississippi
Valley region in LIS2 and LIS3, and a maximum degradation of 11.1% in
the Southern Mountain region in LIS3. The largest degradations in FAR
were clustered in southern regions including the Southern Mountain, Gulf
Coast, and Southern Plains regions. MODIS data had an undesirable impact
on FAR scores in the 0-84 hour forecast period, with almost all regions
experiencing degraded FAR scores. The most notable exception was the
Lower Mississippi Valley region which where FAR values improved by 0.4%
in LIS2 and LIS3 over LIS1.

####  11.3.13 Precipitation Verification Summary

The bias, POD, ETS, and FAR verification analyses above highlighted the
mixed impacts that NLDASE initial conditions had on Eta model
precipitation forecasts. As summarized by Table 3, these impacts were
generally minor in nature with the exception of changes to the bias
score. Several significant conclusions can be drawn from this data:

-   The impacts on precipitation forecasts of using NLDASE states to
    initialize the Eta model were more mixed and generally smaller than
    was the case with surface forecasts.

-   Impacts in regions were often much larger than CONUS-wide impacts.

-   ETS (considered the best overall measure of precipitation forecast
    skill) and POD values benefited most often from NLDASE initial
    conditions, although improvements were very small. These changes
    indicate slight improvements both in precipitation placement, and in
    the fraction of time the Eta model issued a non-zero precipitation
    forecast given the occurrence of an observed precipitation event.

-   NLDASE initial conditions led to improvements in bias over the
    western CONUS in short term forecasts, but to a worsening in bias in
    long term forecasts. The reverse was true for the eastern CONUS.

-   Although usually very minor, the influence of MODIS snow cover data
    was continental in nature, affecting even snow-free regions.

-   Utilization of MODIS snow cover data in LIS2 and LIS3 led to small
    improvements in bias over the non-MODIS LIS1 simulations, but had
    only mixed impacts on ETS, POD, and FAR statistics.

## 12. Analysis of Site-specific Benchmark Findings

### 12.1 Surface Temperature, Relative Humidity, and Mean Sea Level Pressure Comparisons

The regional benchmarks of the retrospective Eta forecasts indicated
that the surface temperature and relative humidity fields were most
sensitive to the use of uncoupled NLDASE land surface states. Selected
verification sites throughout this section will show both forecast
improvement and deterioration in these variables from four retrospective
Eta model forecasts (control, LIS1, LIS2, and LIS3) initialized at 12 Z
on May 3^rd^, 2003 and run out to 84 hours (ending 00 Z on May 7^th^,
2003).

A significant number of sites showed good agreement between all four
forecasts, with some sites leaving little room for improvement with
respect to the observations. An example of such a site is depicted in
Figure 34. Four Eta retrospective forecasts are compared against the
observations at Rawlings Municipal Airport in Wyoming. This site is
located approximately 200 km west-northwest of Cheyenne, Wyoming. The 2m
temperature and mean sea level pressure forecasts validated
exceptionally well against the observations throughout the forecast
period. Two meter relative humidity values also verified well with the
observations at this location. Only after 00 Z on May 6^th^ did the
relative humidity forecasts begin to diverge from one another with the
control forecast verifying the best in this period. All four mean sea
level pressure forecasts performed well out to 36 hours. After 36 hours
into the forecast period, all four forecasts deteriorated, over
predicting the mean sea level pressure by as much as 8
hPa.![](media/image37.png){width="6.489583333333333in"
height="5.014583333333333in"}

The four forecasts were not always in such good agreement with one
another. Figure 35 shows a great deal of variation between the four
forecasts at Great Trout Lake, a site in northwestern Ontario, Canada.
This was consistent with the regional verification statistics which
showed that MODIS snow assimilation had the largest impacts throughout
much of Canada. The LIS1, LIS2, and LIS3 forecasts performed better than
the control forecast in terms of relative humidity and temperature for a
large portion of the forecast, while the control run verified the best
with respect to mean sea level pressure. Only in the last 24 hours of
the forecast did the control run perform noticeably better with respect
to temperature and relative humidity.

The Great Trout Lake, Ontario site exhibited a large amount of
variability between all four forecasts. This was generally not the case
for most verification sites. Most verification sites exhibited the
largest differences between the control forecast and the three NLDASE
forecasts. Figure 36 evaluates the four Eta retrospective forecasts in
the small town of Burnet, Texas, located approximately 70 km northwest
of Austin, Texas. Small differences exist between the three NLDASE
forecasts. The control run differs largely from the NLDASE runs,
particularly with respect to 2m relative humidity forecasts. There was a
general improvement in mean sea level pressure forecasts in the NLDASE
initialized runs; however it was very small (on the order of a few hPa).
Throughout most of the forecast, the NLDASE runs verified much better
than the control run with respect to temperature. The same was true for
the relative humidity forecast values. The observations indicated that a
dryline passed through this site and then retrograded westward between
12 Z on May 5^th^ and 00 Z on May 6^th^. The NLDASE runs more
efficiently capture the intensity and timing of the initial dryline
passage through this particular site and several others during this time
period. However, all four forecasts failed to retrograde the dryline
westward during this period.
![](media/image38.png){width="6.489583333333333in"
height="5.014583333333333in"}

The Burnet, Texas, and Great Trout Lake, Ontario, sites both showed
examples of general forecast improvement for the 2m temperature fields
and relative humidity fields in the NLDASE initialized runs. However,
this was not always the case. Figure 37 evaluates the four Eta
retrospective forecasts at Gilliam Airport in northern Manitoba, Canada.
Surface temperatures in the NLDASE runs were consistently worse than
those forecast by the control run. LIS1 produced the worst temperature
and mean sea level pressure forecast. Relative humidity at this site
showed general improvement with LIS2 and LIS3 performing the best. LIS1
severely underestimated the 2m relative humidity throughout the entire
forecast period, while the control run severely overestimated this same
variable. Therefore, it appears the MODIS snow assimilation benefited
the NLDASE forecasts at this location. It is possible that the NLDASE
and EDAS forcing data have dry and wet biases, respectively, and the
MODIS snow cover assimilation reduced the dry bias in the NLDASE
forcing. However, this theory warrants further investigation.
![](media/image39.png){width="6.489583333333333in"
height="5.014583333333333in"}

As discussed previously, the NLDASE land surface states showed general
improvement in the surface temperature and relative humidity forecasts
in most of the verification regions. A majority of the sites examined in
this study featured behavior similar to the verification presented in
Figure 34, with only very small differences existing between the NLDASE
initialized forecasts and the control forecast. When large differences
did exist, such differences were most often between the NLDASE runs as a
whole and the control forecast. Only in specific regions (e.g. Eastern
and Western Canada) did the NLDASE forecasts differ greatly from one
another. ![](media/image40.png){width="6.489583333333333in"
height="5.014583333333333in"}

Many surface verification sites showed that the inclusion of NLDASE land
surface states had the largest impact on the amplitude of the 2m
temperature and humidity forecasts. In general, a select minority of
verification sites showed the NLDASE forecast to have impacts on the
timing of surface features such as fronts and dry-lines. Figure 35
showed how NLDASE land surface states improved both temperature and
relative humidity forecasts throughout a large majority of the forecast
period. Figure 36 showed that the NLDASE land surface states had a
positive impact on the timing and intensity of a passing dry line in
central Texas, and the relative humidity and temperature fields as a
whole, thus improving the accuracy of the numerical forecast guidance.
Overall, mean sea level pressure tended to be very similar between all
forecasts demonstrating that the offline land surface states did not
have much of an impact on this field in particular. The large
differences in temperature and relative humidity illustrated in the
aforementioned verification sites highlight both forecast improvement
and degradation.

As previously mentioned, the regional verification statistics are robust
and account for a very large number verification sites. However, many of
the verification sites show little differences between all four
forecasts, and these small magnitude differences can overwhelm the
larger impacts at a smaller number of sites due to the area averaging
done in the FVS system. As such, differences in bias and RMSE were often
small for many of the FVS verification regions. The site specific
verification efforts have shown that the surface temperature and
humidity forecasts can be quite different at times. These large
differences illustrate both strongly positive and strongly negative
impacts on the forecasts that are sometimes averaged out in the regional
verification statistics.

### 12.2 Individual Eta Precipitation Forecasts

The regional FVS precipitation verification provided a great deal of
information on precipitation bias, probability of detection, false alarm
ratio, and equitable threat scores. This verification system gauged
model forecast performance in 24 hour blocks of accumulated
precipitation, and missed many details associated with short lived
precipitation events. While precipitation differences are large when
evaluating the forecast in this manner (Figure 38), details of the
timing and intensity of individual precipitation events are lost due to
individual events being summed together into 24 hour bins of
precipitation. This section will highlight two individual events within
a single forecast where timing, placement, and intensity were crucial to
the accuracy, or lack thereof, of the precipitation forecasts. The
forecasts examined are the same as those presented in the previous
section.

Figure 39 highlights the Stage II radar/gauge hourly precipitation
product on 12 Z May 4^th^, and hourly precipitation forecasts valid at
the same time for the control and LIS1 simulations. Also depicted is the
difference between the two Eta forecasts (LIS1 -- control). There exists
a significant difference in precipitation between the two forecasts in
central Missouri. This is associated with a small shift in both timing
and intensity between the two forecasts. The LIS1 forecast placed the
maximum precipitation in Missouri further westward and also yielded
higher precipitation amounts than the control simulation. Figure 40 is a
time series of observed versus modeled rainfall at St. Louis, Missouri
and shows that control run handled the overall timing of the
precipitation better than the NLDASE runs. However the NLDASE runs
better predicted the magnitude and timing of the maximum rainfall values
at this particular location. As seen in the site-specific surface
temperature and humidity forecasts, the NLDASE runs exhibited a tendency
to be similar to one another in the precipitation forecasts (Figure 40).
Overall, this event was well forecasted by both the control and NLDASE
initialized forecasts, but the NLDASE runs more accurately depicted the
timing and amount of the heavier precipitation
amounts.![](media/image41.wmf){width="4.501388888888889in"
height="5.583333333333333in"}

The preceding forecast example was located within the first 30 hours of
the forecast, and aside from the previously mentioned event, much of the
rest of the NLDASE and control run precipitation forecasts were very
similar (e.g. small differences in top panel of Fig 38). The NLDASE runs
and the control run began to diverge more as the forecast progressed
beyond hour 30. Figure 41 highlights the Stage II radar/gauge hourly
precipitation product on 06 Z May 4^th^, and hourly precipitation
forecasts valid at the same time for the control and LIS1 simulations.
The difference between the two Eta forecasts (LIS1 -- control) is also
illustrated. The general pattern of precipitation between the two
forecasts looks to be highly similar, but several key differences exist.
The control run produced larger amounts of precipitation in both
southern Illinois and southwestern Arkansas. Both forecasts missed the
precipitation that occurred on the border of Oklahoma and Arkansas by
placing the precipitation too far eastward. However, the control run
does have larger precipitation values in this region. Thus the control
guidance produced more useful information.
![](media/image42.png){width="6.5in" height="4.875in"}

The converse was true with the heavy precipitation that fell over Lake
Superior and Southern Michigan. Both forecasts placed the larger
precipitation amounts too far
![](media/image43.png){width="5.9834645669291335in"
height="4.487401574803149in"}![](media/image44.png){width="6.24015748031496in"
height="2.808267716535433in"}westward, but in this case the LIS1
forecast indicated heavier rainfall was possible in this region. In
Figure 41, the heavy precipitation in southwestern Illinois appears to
be well predicted by the control run and underestimated by the LIS1 run.
Figure 42 provides a more detailed examination, and contains a time
series of forecast and observed rainfall at Mattoon, Illinois located
approximately 100km southeast of Springfield, IL. The overall timing of
the largest precipitation amounts was better handled by the NLDASE
forecasts at this site, but the control forecast did a better job of
predicting the total precipitation amounts for the event. Again, this
event was generally well forecast by all of the retrospective
simulations. However, the larger precipitation amounts forecast by the
control run, although not forecast at the correct time, were
advantageous because they provided useful information regarding the
potential for significant rainfall in these locations.

In general, the NLDASE forecasts were very similar to one another and at
times differed greatly from the control precipitation forecasts. This
was particularly true later in the forecasts as the NLDASE simulations
diverged further from the control forecast. The two precipitation
examples presented here highlighted the many differences that can occur
not only at smaller temporal scales, but within FVS verification regions
as well. These significant differences in precipitation cannot be
demonstrated within the regional precipitation forecast statistics
presented earlier. The regional statistics group forecasts into 24 hour
blocks and average them over several forecasts. FVS analysis results in
very robust evaluations of long term average model skill, but the two
examples above show the large differences that can occur between
individual forecasts.

The positive and negative impacts illustrated above show that individual
forecast details must be considered when referencing different sources
of forecast guidance. For example, the Eta forecasts initialized with
NLDASE land surface states showed an overall decrease in precipitation
forecast skill throughout the Midwest in the regional precipitation
verification statistics; however, when the individual precipitation
event that occurred in St. Louis on May 4^th^, 2003 (Figure 40) was
analyzed as discussed above, the 24-36 hour NLDASE forecasts proved to
be superior to the control run forecasts.
![](media/image45.png){width="6.489583333333333in"
height="2.8881944444444443in"}

### 12.3 Surface Radiation Forecast Evaluation

The NCEP forecast verification system was designed to validate surface
and upper air meteorological fields and does not evaluate surface
radiation fluxes. The same four sample forecasts used in the prior two
sections were validated against downwelling/upwelling shortwave and
longwave radiation data from the SURFRAD network (Figure 43) to assess
the impact NLDASE land surface states have on Eta model forecasts of
these fields. This is not intended to be an exhaustive evaluation of Eta
model surface radiation fluxes, and serves the purpose of highlighting
some of the impacts that uncoupled land surface state initialization can
have on these forecast
fields.![](media/image46.png){width="5.729166666666667in"
height="3.7916666666666665in"}

Large differences in surface radiation fluxes emerge from the use of
NLDASE land surface states in the Eta model. Figure 44 shows the
differences between the control run and the three NLDASE runs at
forecast hour 30, valid at 18 Z on May 4^th^, 2003 over the continental
United States. Large differences exist in many areas, although
differences between all four forecasts were small at most of the SURFRAD
observation locations (circles) during this time.

Figure 45 shows a time series of downwelling/upwelling shortwave and
longwave radiation fluxes at the Desert Rock, Nevada SURFRAD site.
Differences between the NLDASE forecasts and the control forecasts
proved to be very small. A one hour temporal shift is depicted in the
surface downwelling and upwelling shortwave radiation fluxes and is due
to the Eta model post processor's use of an incorrect solar zenith angle
for model output (B. Ferrier, personal communication). Ignoring the
temporal shift, both the upward and downward shortwave radiation fluxes
corroborated well with the observations, with the exception of the first
12 hours of the forecast. Upwelling longwave radiation fields verified
particularly well against the observations in contrast to downwelling
longwave radiation fluxes. Overall, all four forecasts performed
reasonably well at this location, with the largest differences occurring
within the first 12 hours of the forecast.
![](media/image47.png){width="6.5in" height="4.875in"}

Figure 46 shows a time series of downwelling/upwelling shortwave and
longwave radiation fluxes at the Table Mountain, Colorado, SURFRAD site.
Again, differences between the control forecast and NLDASE forecasts
were generally small but noticeably larger, when compared to those
observed at the Desert Rock, Nevada site. All four Eta forecasts
displayed large errors throughout the forecast period in all of the
surface radiation fields. Downwelling shortwave radiation was severely
underestimated throughout a significant portion of the forecast.
Conversely, downwelling longwave radiation was overestimated by all of
the Eta model forecasts throughout a large majority of the forecast
period. Errors in upwelling longwave and shortwave radiation were not as
severe as the downwelling fluxes. Overall, the Eta model did a generally
poor job at simulating surface radiation fluxes at this site during the
forecast period.

As previously mentioned, the preceding discussion was not intended to be
a thorough evaluation of downwelling shortwave radiation fluxes as only
four forecasts were evaluated within a single period. However, the
analyses highlighted several important points that bear summarizing.
Overall, the NLDASE initialized forecasts were very similar to the
control forecasts throughout all of the verification sites. In general,
all four forecasts performed poorly in the simulation of downwelling
shortwave radiation, with errors at times exceeding 500 Wm^-2^. However,
the forecasts performed well under clear sky conditions. All four
forecasts performed reasonably well in the simulation of upwelling and
downwelling longwave radiation fluxes. Large differences occurred
between the control forecasts and the NLDASE forecasts across much of
the domain, but the location of the SURFRAD validation sites failed to
coincide with these differences. Therefore, more surface radiation sites
need to be examined to fully gauge the impact of the NLDASE offline land
surface states, particularly in regions where MODIS snow cover
assimilation may have had a large impact. The sites examined herein
indicated that the uncoupled land surface states did not have a large
impact on the surface radiation fluxes during this
period.![](media/image48.png){width="6.489583333333333in"
height="5.014583333333333in"}

![](media/image49.png){width="6.489583333333333in"
height="5.014583333333333in"}

## 13. Benchmarking Gap and Lessons Learned

The largest gap of this benchmarking exercise takes the form of the
sub-optimal forecast sample size. Due to time and computing constraints,
only a single ten day test period was chosen. Two 84-hour long Eta
simulations were executed each day, leading to a total of 20 control
run, 20 LIS1, 20 LIS2, and 20 LIS3 simulations. Although this number was
large enough to support meaningful and statistically sound verification
efforts (M. Ek, personal communication), a larger number of simulations
would have been more desirable. With the sample size as small as it was,
single weather events may have had a large impact on the benchmarking
process. A longer evaluation period of one month would improve the
general applicability of the results, leading to benchmark results
applicable across a wider variety of weather patterns and conditions.

Along similar lines, the applicability of the benchmark results would
also be improved by extending the analysis into another season. Synoptic
weather patterns differ greatly in Fall, Winter, Spring, and Summer, and
the May time period chosen, although combining elements of Winter (snow
cover) and Spring (severe weather), was not representative of much of
the rest of the year. The land surface is much less active in the cold
season than in the warm season, and validation activities in a month
such as February could very well lead to different benchmark results.

These gaps were recognized as unavoidable obstacles during the
formulation of this benchmarking project, but all possible measures were
taken to minimize the impact of these issues. As previously mentioned,
NCEP was extensively consulted to ensure that the length of the test
period and the number of Eta forecasts met their analysis standards. In
addition, the timing of the test case was chosen to balance the need for
snow-covered areas to test the impact of MODIS data, with the need for a
time period with the type of strong land-atmosphere interactions that
characterize the warm season. Care was taken to choose a challenging
forecasting time period, featuring a large-scale severe weather
outbreak. In this type of weather regime, even modest increases in
forecast skill may have far reaching impacts in areas of public safety
and resource management.

##  14. Summary, Conclusions, and Recommendations

Divided into twelve application areas of national priority, NASA's
applications research program seeks to combine the strengths of NASA
models and data, with the capabilities of partner agency decision
support tools. In this particular study, the authors examined the
effectiveness of linking NOAA NCEP's mesoscale Eta model DST with NASA's
award-winning LIS modeling infrastructure and the NASA MODIS snow cover
product within the NLDASE system. This system combined the atmospheric
modeling strengths of NOAA with the remote sensing and uncoupled land
surface modeling capabilities of NASA, and sought to produce improved
Eta model forecasts of surface fields, upper air fields, and
precipitation. This approach is based on a foundation of work which has
been completed as part of the multi-institution NLDAS and NLDASE
projects, and results from this study will be easily transferable into
the NCEP operational community due to the solid NASA-NOAA LDAS ties.
Although the Eta model is not traditionally considered a decision
support tool, it admirably fills that role, serving as a vital
information source for decision makers in areas ranging from water
management to homeland security. Its forecasts are disseminated to
thousands of people each day, and improvements in Eta output would
benefit the general public as well as those responsible for public
safety, policy decisions, and resource management.

At the center of this benchmarking exercise is the initialization of the
Eta model DST with uncoupled NLDASE land surface states. To investigate
whether or not the use of such initial conditions can improve Eta model
forecasts, two 84-hour Eta model simulations were executed each day over
the May 1^st^ through May 10^th^ 2003 time period. Four sets of Eta
model runs were conducted in this fashion, each using different types of
restart files that included: 1) NCEP operational restart files to
establish a baseline control run, 2) LIS1 restart files to test the
basic effects of uncoupled NLDASE initial conditions, 3) LIS2 restart
files to test the effect of MODIS snow assimilation using a 5mm SWE
value, and 4) LIS3 restart files to test the effect of MODIS snow
assimilation using a 10mm SWE value. Following NOAA operational
practices, NCEP's FVS was used to verify the resulting forecasts against
observations. Bias and RMSE values were computed for 2m temperature, 2m
relative humidity, 10m wind speed, 850mb temperature, 300mb temperature,
700mb relative humidity, 500mb height, and 250mb wind speed. Bias, ETS,
FAR, and POD statistics were computed for the purposes of precipitation
verification.

Overall, initialization of Eta land surface states with NLDASE output
had a mixed impact on forecasts. Surface fields including 2m temperature
and 2m humidity greatly benefited from the uncoupled initialization
process, while the upper air and precipitation fields featured a mix of
desirable and undesirable impacts. Results for each verification
category are detailed below:

**[Regional FVS Analysis For All Forecasts]{.underline}**

-   Regional impacts were often much larger than CONUS-wide impacts.

**Surface Meteorology (2m Temperature, 2m Relative Humidity, 10m Wind
Speed)**

-   The use of NLDASE conditions to initialize the Eta model led to
    greatly improved forecasts of humidity and temperature overall.

-   The impact on wind speed bias and RMSE was mixed.

-   Use of initial conditions based on assimilated MODIS snow cover data
    generally improved forecasts, especially when 10mm SWE value was
    used.

-   MODIS data had a generally continental-scale effect on bias, and a
    regional effect on RMSE.

-   Diurnal influences altered the impact of initialization with NLDASE
    conditions, but not in a clear-cut, constant fashion.

**Upper Air Meteorology (850mb Temperature, 300mb Temperature, 700mb
Humidity, 500mb Height, and 250mb Wind Speed)**

-   The impact of NLDASE initial conditions on upper air forecasts was
    more mixed than it was on surface forecasts.

-   300mb temperature was the only field which consistently improved,
    while 850mb temperature and 700mb humidity bias, and 250mb wind RMSE
    fared the worst.

-   RMSE was more often reduced than was bias, although the changes in
    RMSE were very small (less than 2.5%).

-   MODIS snow cover generally improved forecasts; however, this benefit
    was often overshadowed by the unfavorable impact of the uncoupled
    initialization process.

-   Overall, the 5mm MODIS SWE configuration performed comparably to the
    10mm configuration.

-   Diurnal influences impacted forecast results, but not in a
    discernable pattern.

**Daily Precipitation**

-   The impacts on precipitation forecasts of using NLDASE states to
    initialize the Eta model were mixed and generally very minor.

-   ETS (considered the best overall measure of precipitation forecast
    skill) and POD values benefited most from NLDASE initial conditions,
    although improvements never exceeded 2.4%. These changes indicate
    slight improvements in precipitation placement, and in the fraction
    of time the Eta model issued a non-zero precipitation forecast given
    the occurrence of observed precipitation.

-   Short term forecast bias improved over the western CONUS, but
    worsened in long term forecasts. The reverse was true for the
    eastern CONUS.

-   Use of MODIS snow cover led to small magnitude continental-scale
    improvements in bias, but had mixed impacts on ETS, POD, and FAR
    statistics.

**[Site Specific Analysis For 12Z May 3^rd^, 2003 Eta
Forecasts]{.underline}**

-   In general, the largest differences in forecasts occurred between
    the NLDASE simulations as a whole and the control forecast.

-   Site-specific analysis revealed larger impacts than did regional FVS
    analysis.

**Near-surface Temperature, Relative Humidity, and Mean Sea Level
Pressure**

-   MODIS data strongly impacted forecasts over a few specific regions
    including Eastern and Western Canada.

-   In general, the largest impact was on 2m temperature and relative
    humidity amplitude.

-   In some cases, the timing of dry-lines and fronts was impacted (both
    positively and negatively) by the use of NLDASE land surface states.

-   Mean sea level pressure was only slightly impacted by NLDASE
    initialization.

**Precipitation**

-   Short term forecasts were impacted less than long lead (\> 48hr)
    forecasts.

-   In general, precipitation distribution was impacted less than
    precipitation timing and intensity.

-   NLDASE-based forecasts generally improved upon the timing, but not
    the intensity, of control run precipitation.

**Site Specific Surface Radiation Flux Evaluation**

-   In general, downwelling shortwave radiation was poorly forecast in
    all simulations during cloudy conditions, with errors at times
    exceeding 500 Wm^-2^.

-   Upwelling and downwelling longwave radiation fluxes were well
    forecast in all simulations.

-   Inopportune location of SURFRAD sites failed to capture the large
    differences in control run and LIS1, LIS2, and LIS3 radiation
    forecasts present across domain.

-   More surface radiation sites need to be examined to fully gauge the
    impact of the NLDASE offline land surface states on Eta model
    forecasts of radiation.

With these results in mind, the uncoupled initialization approach is
promising, but needs further development before being considered for
operational implementation within NCEP. Surface forecasts in particular
are greatly improved, but the mixture of impacts in the upper air and
precipitation forecasts leads to the need for additional research and
benchmarking activities. Such activities will be based on the current
NASA-NOAA benchmark partnership, and will expand upon the current
research and modeling infrastructure.

Near-term research must address four main issues: 1) The need to
benchmark Eta simulations in a non-Spring month, 2) Initialization using
NLDASE conditions derived from additional permutations of forcing and
data assimilation procedures, 3) The need to benchmark Eta simulations
over a longer study period of one month, and 4) The need to determine
how best to apply this initialization system to the WRF model---the Eta
model's successor. Each of these research areas will be explored in
collaboration with our NOAA NCEP partners. Particular emphasis will be
placed on the use of the CMORPH precipitation product, on the use of
high resolution MODIS-based vegetation class and LAI data sets, and on
the assimilation of MODIS surface temperature and AMSR-E SWE and soil
moisture. Once these research needs are addressed, results will be
transferred into NCEP modeling operations to be applied as appropriate
based on the outcome of the benchmarking activities.

Initial results from this benchmarking study are promising, and it is
expected that NOAA NCEP will eventually adopt some of the modeling
techniques and data sets utilized to positive effect in this study.

## 15. References

Anderson, James R., and Coauthors, 1976: A Land Use and Land Cover
Classification System for Use with Remote Sensor Data. *USGS*
*Professional Paper #964*: 1976

Arakawa, A., and V. R. Lamb, 1977, Computational design of the basic
dynamical processes of the UCLA general circulation model, Methods in
Computational Physics, **17**, Academic Press, New York, 173-265.

Bahill A. T., and B. Gissing, 1998: Re-evaluating systems engineering
concepts using systems thinking. IEEE Transaction on Systems, Man and
Cybernetics, Part C: Applications and Reviews, 28(4), 516-527.

Baldwin, M., and K. Mitchell, 1997, The NCEP hourly multi-sensor U.S.
precipitation analysis for operations and GCIP research, Preprints, 13th
AMS Conf. on Hydrology, Long Beach, CA, 54-55.

Beljaars, A. C.M., P. Viterbo, M. J. Miller and A. K. Betts, 1996, The
anomalous rainfall over the USA during July, 1993: Sensitivity to land
surface parameterization and soil moisture anomalies, *Mon. Wea. Rev.*,
**124**, 362-383.

Berbery E. H., Y. Luo, K. E. Mitchell, A. K. Betts, 2003: Eta model
estimated land surface processes and the hydrologic cycle of the
Mississippi basin, *J. Geophys. Res*., **108** (D22), 8852,
doi:10.1029/2002JD003192.

Betts, A.K., 1986: A new convective adjustment scheme. Part I:
Observational and theoretical basis. *Quart. J. Roy. Meteor. Soc.*,
**112**, 677-691.

Betts, A.K. and M.J. Miller, 1986: A new convective adjustment scheme.
Part II: Single column tests using GATE wave, BOMEX, ATEX, and Arctic
air-mass data sets. *Quart. J. Roy. Meteor. Soc.*, **112**, 693-709.

Bitner, D., T. Carroll, D. Cline, and P. Romanov, 2002: An assessment of
the differences between three satellite snow cover mapping techniques.
*Hydrol. Processes,* **16,** 3723--3733.

Black, T., D. Deaven, and G. DiMego, 1993: The step-mountain eta
coordinate model: 80-km Early version and objective verifications. *NWS
Technical Procedures Bulletin*, **412**, NOAA/NWS, 31 pp.

\_\_, 1994: NMC NOTES, The New NMC Mesoscale Eta Model: Description and
Forecast Examples. *Wea. Forecasting*, **9**, 265-278.

\_\_, M. Pyle, and H.Y. Chuang: The operational WRF NMM and NCEP. 21st
Conference on Weather Analysis and Forecasting/17th Conference on
Numerical Weather Prediction, Washington, DC.

Brewster, K. A., 2003: Phase-Correcting Data Assimilation and
Application to Storm-Scale Numerical Weather Prediction. Part II:
Application to a Severe Storm Outbreak, *Mon. Wea. Rev.*, **131**,
493-507.

Brill, K. F., 1999: Model verification system at NCEP. Available from
<http://wwwt.emc.ncep.noaa.gov/mmb/papers/brill/fvs.txt>, accessed on
September 27^th^, 2005.

Brill, K. F., 2005: FVS online help file version 2005.08.

Chen, F., K. Mitchell, J. Schaake, Y. Xue, H.-L. Pan, V. Koren, Q. Y.
Duan, M. Ek, and A. Betts, 1996: Modeling of land-surface evaporation by
four schemes and comparison with FIFE observations, *J. Geophys. Res*.,
**101**, 7251-- 7268.

Chen, F., and K. Mitchell, 1999: Using the GEWEX/ISLSCP forcing data to
simulate global soil moisture fields and hydrological cycle for
1987--1988, *J. Meteorol. Soc. Jpn.*, **77**, 167-- 182.

Chuang, H. and G. Manikin, 2001: The NCEP Meso Eta Model Post Processor:
A Documentation, NCEP Office Note No. 438, NOAA/NWS, 52 pp.

Clark, C. A., R. W. Arritt 1995: Numerical Simulations of the Effect of
Soil Moisture and Vegetation Cover on the Development of Deep
Convection. *Journal of Applied Meteorology*, **34**, 2029--2045.

Cosgrove, B. A., and Coauthors 2003: Real-time and retrospective forcing
in the North American Land Data Assimilation System (NLDAS) project, *J.
Geophys. Res.*, **108**(D22), 8842, doi:10.1029/2002JD003118.

Cosgrove, B. A., and Coauthors 2003, Land surface model spin-up behavior
in the North American Land Data Assimilation System (NLDAS), *J.
Geophys. Res.*, **108**(D22), 8845, doi:10.1029/2002JD003316.

Cosgrove, B. A., Houser, P. R., 2002: The Effect of Errors in Snow on
Land Surface Modeling, Preprints, 16th Conference on Hydrology, pp
J136-J137.

Crook, N. A., 1996: Sensitivity of Moist Convection Forced by Boundary
Layer Processes to Low-Level Thermodynamic Fields. *Mon. Wea. Rev.*,
**124**, 1767--1785.

Delworth, T. L. and S. Manabe, 1988: The influence of potential
evaporation on the variabilities of simulated soil wetness and climate,
*J. Climate*, **1**, 523-547.

Delworth, T. L. and S. Manabe, 1989: The influence of soil wetness on
near-surface atmospheric variability, *J. Climate*, **2**, 1447-1462.

Department of Commerce, 2004: Service assessment: Record tornado
outbreaks of May 4--10, 2003. 58 pp. \[Available from National Weather
Service, 1325 East-West Highway, W/OS52, Silver Spring, Maryland
20910-3283, or online at
[www.nws.noaa.gov/](http://www.nws.noaa.gov/)om/assessments/record_may.pdf.\]

Derber, J. C., D. F. Parrish, and S. J. Lord, 1991: The new global
operational analysis system at the National Meteorological Center. *Wea.
Forecasting*, **6**, 538--547.

Dirmeyer, P. A., and J. Shukla, 1993: Observational and modeling studies
of the influence of soil moisture anomalies on Atmospheric Circulation
(Review), NATO ASI Ser. Vol. 16, Springer-Verlag, 23pp.

\_\_, 1997, Can seasonal climate simulation be improved by the GSWP soil
wetness climatology? 77th Annual AMS Meeting, 13th Conference on
Hydrology, February 2-7, 1997, pp 169-171.

\_\_, A. J. Dolman, and N. Sato, 1999: The Global Soil Wetness Project:
A pilot project for global land surface modeling and validation. *Bull.
Amer. Meteor. Soc.*, **80**, 851-878.

Dorman, J.L., and Sellers, P.J., 1989. A Global climatology of albedo,
roughness length and stomatal resistance for atmospheric general
circulation models as represented by the simple biosphere model (SiB).
*Journal of Applied Meteorology*, **28**, 833-855.

Ek, M. B., K. E. Mitchell, Y. Lin, E. Rogers, P. Grunmann, V. Koren, G.
Gayno, and J. D. Tarpley, 2003, Implementation of Noah land-surface
model advances in the NCEP operational mesoscale Eta model. *J. Geophys.
Res.*, **108**(D22), 8851, doi:10.1029/2002JD003296.

Entin, J., and Coauthors, 1999: Evaluation of Global SoilWetness Project
soil moisture simulations, *J. Meteorol. Soc. Jpn.*, **77**, 183--198.

Figueroa, S. N and Nobre, C. A., 1990: Precipitation distribution over
Central and Western Tropical South América. *Climanálise*, **5**, 35-46.

Findell, K. L., E. A. B. Eltahir 2003: Atmospheric Controls on Soil
Moisture--Boundary Layer Interactions. Part I: Framework Development.
*J. Hydrometeor.*, 4, 552--569.

Friedl, M.A., D. K. McIver, J. C. F. Hodges, X. Y. Zhang, D. Muchoney,
A. H. Strahler, C. E. Woodcock, S. Gopal, A. Schneider, A. Cooper, A.
Baccini,F. Gao, C. Schaaf 2002: Global land cover mapping from MODIS:
algorithms and early results, *Remote Sensing of Environment*, **83**
(1-2), 287-302.

Fujita, T. T., 1971: Proposed characterization of tornadoes and
hurricanes by area and intensity. University of Chicago SMRP Research
Paper 91, 42 pp. \[Available from Wind Engineering Research Center,
Texas Tech University, P.O. Box 41023, Lubbock, TX 79409-1023.\]

Goncalves, L. G. and Coauthors: Evaluation of model-derived and remotely
sensed precipitation products for the continental South America. *J.
Geophys. Res.*, Submitted.

Fels, Stephen B. and M. D. Schwarzkopf, 1975: The simplified exchange
approximation: A new method for radiative transfer calculations. *J.
Atmos. Sci.*, **32**, 1475-1488.

Hall, D.K., 1988 :Assessment of polar climate change using satellite
technology. *Rev. of Geophys*., **26**, 26-39.

Hall, D. K., G. A. Riggs, V. V. Salomonson, N. E. DiGirolamo, and K. J.
Bayr, 2002: MODIS snow-cover products. *Remote Sens.Environ.*, **83**,
181--194.

Hamill, T. M., R. P. d'Entremont, and J. T. Bunting, 1992: A description
of the Air Force real-time nephanalysis model. *Wea. Forecasting,*
**7,** 288--306.

Hamill, T.M. and coauthors, 2005: The May 2003 Extended Tornado
Outbreak. *Bull. of the Amer. Met. Soc.*, **86**, 531--542.

Higgins, R.W., W. Shi, E. Yarosh, and R. Joyce, 2000, Improved US
Precipitation Quality Control System and Analysis, NCEP/Climate
Prediction Center Atlas No. 7.

Huffman, G.J., R.F. Adler, S. Curtis, D.T. Bolvin, and E.J. Nelkin,
2005: Global Rainfall Analyses at Monthly and 3-Hr Time Scales. Chapter
4 of Measuring Precipitation from Space: EURAINSTAT and the Future, V.
Levizzani, P. Bauer, and J. Turk, Ed., Kluwer Academic Pub. B.V.,
Dordrecht, The Netherlands, accepted.

Janjic Z., 1984: Nonlinear advection schemes and energy cascade on
semistaggered grids. *Mon. Wea. Rev.*, **112**, 1234-1245.

\_\_, 1994: The step-mountain Eta coordinate model: Further developments
of the convection, viscous sublayer, and turbulence closer schemes.
*Mon. Wea. Rev.*, **122**, 927-945.

\_\_, J. Gerrity, and S. Nickovic, 2001: An alternative approach to
nonhydrostatic modeling. *Mon. Wea. Rev.*, **129**, 1164-1178.

\_\_, T. Black, M. Pyle, H.Y. Chuang, E. Rogers, and G. DiMego, 2005:
The NCEP WRF model core. 21^st^ Conference on Weather Analysis and
Forecasting/17^th^ Conference on Numerical Weather Prediction,
Washington, DC.

Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A
method that produces global precipitation estimates from passive
microwave and infrared data at high spatial and temporal resolution. *J.
Hydrometeor.*, **5**, 487-503.

Kain, J. S., and Fritsch J. M., 1990: A one-dimensional
entraining/detraining plume model and its application in convective
parameterization. J. Atmos. Sci., **47**, 2784--2802.

Kain, J. S., and J. M. Fritsch, 1993: Convective parameterization for
mesoscale models: The Kain--Fritsch scheme. *The Representation of
Cumulus Convection in Numerical Models, Meteor. Monogr.,* No. 24, Amer.
Meteor. Soc., 165--170.

Kelly, R.E., A.T.C. Chang, L. Tsang, and J.L. Foster. 2003. A prototype
AMSR-E global snow area and snow depth algorithm. *IEEE Transactions on
Geoscience and Remote Sensing* **41**(2): 230-242.

Keyser D. 2005: - PREPBUFR Processing at NCEP. Available at
<http://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/document.htm>,
accessed on September 27^th^, 2005.

Kopp, T. J., and R. B. Kiess, 1996: The Air Force Global Weather Central
cloud analysis model. Preprints, *15th Conf. on Weather Analysis and
Forecasting,* Norfolk, VA, Amer. Meteor. Soc., 220--222.

Koren, V., J. Schaake, K. Mitchell, Q. Y. Duan, F. Chen, and J. M.
Baker, 1999: A parameterization of snowpack and frozen ground intended
for NCEP weather and climate models. *J. Geophys. Res.,* **104,** 19
569--585.

Koster, R. D. and M. J. Suarez, 1995: Relative contributions of land and
ocean processes to precipitation variability, *J. Geophys. Res*.,
**100**, 1375-1390.

Kumar, S. V., and Coauthors (2004), LIS - An Interoperable Framework for
High Resolution Land Surface Modeling. Accepted in Environmental
Modeling and Software.

Lacis, A. A. J. E. Hansen, 1974: A parameterization for the absorption
of solar radiation in the earth\'s atmosphere. *J. Atmos. Sci.*, **31**,
118-133.

Li, L., E. Njoku, E. Im, P. Chang, K. St. Germain, 2004: A preliminary
survey of radio-frequency interference over the U.S. in Aqua AMSR-E
Data. *IEEE Transactions on Geoscience and Remote Sensing*, accepted.

Liston, G. E., R. A. Pielke Sr., and E. M. Greene, 1999: Improving
first-order snow- related deficiencies in a regional climate model. *J.
Geophys. Res.,* **104,** 19 559--19 567.

Lobl, E. 2001. Joint Advanced Microwave Scanning Radiometer (AMSR)
Science Team meeting. *Earth Observer* **13**(3): 3-9.

Marengo, J., 1995: Interannual variability of deep convection in the
tropical South American sector as deduced from ISCCP C2 data. *Int. J.
Climatol.*, **15**, 995-1010.

Matthews, E., 1983: Vegetation, Land-Use and Seasonal Albedo Data Sets:
Documentation of Archived Data Tape. NASA Technical Memoradum #86107.

Maurer, E. P., J. D. Rhoads, R. O. Dubayah, and D. P. Lettenmaier, 2003:
Evaluation of the snow-covered area data product from MODIS. *Hydrol.
Processes,* **17,** 59--71

McKellip, R., B. Beckley, C. Birkett, S. Blonski, B. Doom, B. Grant, L.
Estep, R. Moore, K. Morris, K. Ross, G. Terrie, Z. Zanoni, 2004: PECAD's
global reservoir and lake monitor: a NASA systems engineering report,
V1.0. 63 pages, available at <http://aiwg.gsfc.nasa.gov/dss.html>,
accessed on September 27^th^, 2005.

Mellor, G.L., and T. Yamada, 1974: A hierarchy of turbulence closure
models for planetary boundary layers. *J. Atmos. Sci.*, **31**,
1791-1806.

\_\_, and\_\_, 1982: Development of a turbulence closure model for
geophysical fluid problems. *Rev. Geophys. Space Phys.*, **20**,
851-875.

Mesinger F., 1984: A blocking technique for representation of mountains
in atmospheric models. *Riv. Meteor. Aeronautica*, **44**,195-202.

\_\_, and Coauthors 2004, North American regional reanalysis. Submitted
to *Bull. Amer. Meteor. Soc.*

Mitchell, K. E., et al. (2004), The multi-institution North American
Land Data Assimilation System (NLDAS): Utilizing multiple GCIP products
and partners in a continental distributed hydrological modeling system,
*J. Geophys. Res.*, **109**, D07S90, doi:10.1029/2003JD003823.

Njoku, E.G., T.J. Jackson, V. Lakshmi, T.K. Chan & S.V. Nghiem, 2003:
Soil moisture retrieval from AMSR_E. *IEEE Trans. Geosci. Remote Sens.*,
**41**(2), 215-229.

Otte, T. L., and Coauthors: Linking the eta model with the community
multiscale air quality (CMAQ) modeling system to build a national air
quality forecasting system. *Wea. Forecasting*, **20**, 367 -- 383.

Pan, H.-L. and L. Mahrt, 1987: Interaction between soil hydrology and
boundary-layer development. *Bound.-Layer Meteorol.*, **38**, 185-202.

Pan, M. and Coauthors, 2003: Snow process modeling in the North American
Land Data Assimilation System (NLDAS): 2. Evaluation of model simulated
snow water equivalent, *J. Geophys. Res.*, **108**(D22), 8850,
doi:10.1029/2003JD003994.

Paulson, C. A., 1970: The mathematical representation of wind speed and
temperature profiles in the unstable surface layer. *J. Appl. Meterol.*,
**9**, 857-861

Phillips, N.A., 1957: A coordinate system having some special advantages
for numerical forecasting. *J. Meteor.*, **14**, 297-300.

Pinker, R. T., and Coauthors, 2003: Surface radiation budgets in support
of the GEWEX Continental-Scale International Project (GCIP) and the
GEWEX Americas Prediction Project (GAPP), including the North American
Land Data Assimilation System (NLDAS) project, *J. Geophys. Res.*,
**108**(D22), 8844, doi:10.1029/2002JD003301.

Pyle, M. Workstation Eta FAQ,  Online. Internet. NOAA/NCEP. June 2002.
Availble from
[http://sgi](http://sgi/)62.wwb.noaa.gov:8080/wrkstn_eta/faq.html,
accessed on September 27th, 2005.

Ramsay, B.H. 1998: The interactive multisensor snow and ice mapping
system. *Hydrol. Process.*, **12**, 1537-1546.

Reynolds, C. A., T. J. Jackson, and W. J. Rawls, 1999, Estimating
available water content by linking the FAO Soil Map of the World with
global soil profile databases and pedo-transfer functions, American
Geophysical Union, Fall Meeting, Eos Trans. AGU, 80.

Robock, A, and Coauthors, 2003: Evaluation of the North American Land
Data Assimilation System over the Southern Great Plains during the warm
season. *J. Geophys. Res.*, **108** (D22), 8846,
doi:10.1029/2002JD003245.

Rodell, M., and Coauthors, 2004: The Global Land Data Assimilation
System. *Bull. Amer. Meteor. Soc.,* **85,** 381--394.

Rodell M., P. R. Houser 2004: Updating a Land Surface Model with
MODIS-Derived Snow Cover. *J. Hydrometeor.*, **5**, 1064-1075.

Rogers, E. R., T. L. Black, D. G. Deaven, G. J. DiMego, Q. Zhao, M.
Baldwin, N. W. Junker, and Y. Lin, 1996: Changes to the operational
"early" Eta analysis/forecast system at the National Centers for
Environmental Prediction. *Wea. Forecasting.*, **11**, 391 - 401.

\_, E., T. Black, W. Collins, G. Manikin, F. Mesinger, D. Parrish and G.
DiMego, 2000: Changes to the NCEP Meso Eta Analysis and Forecast System:
Assimilation of satellite radiances and increase in resolution. *NWS
Technical Procedures Bulletin*, **473**, NOAA/NWS, 85 pp.

\_\_, M. Ek, Y. Lin, K. Mitchell, D. Parrish, and G. DiMego, 2001:
Changes to the NCEP Meso Eta analysis and Forecast system: Assimilation
of observed precipitation, upgrades to land-surface physics, modified
3DVAR analysis. *NWS Technical Procedures Bulletin*. **473**, NOAA/NWS,
14 pp.

\_\_, Ek M., B. S. Ferrier, G. Gayno, Y. Lin, K. Mitchell, M. Pondeca,
M. Pyle, V. C. K. Wong, and W.-S. Wu., 2005: The NCEP North American
Mesoscale Modeling System: Final Eta model/analysis changes and
preliminary experiments using the WRF-NMM. 21^st^ Conference on Weather
Analysis and Forecasting/17^th^ Conference on Numerical Weather
Prediction, Washington, DC.

Schaefer, J. T., 1990: The critical success index as an indicator of
warning skill. *Wea. Forecasting,* **5,** 570--575.

Shapiro, R., 1987: A simple model for the calculation of the flux of
direct and diffuse solar radiation through the atmosphere. Air Force
Geophysics Lab, AFGLTR-87-0200, Hanscom AFB, MA, 40 pp.

Sheffield, J., and Coauthors, 2003: Snow process modeling in the North
American Land Data Assimilation System (NLDAS): 1. Evaluation of
model-simulated snow cover extent, J. Geophys. Res., 108(D22), 8849,
doi:10.1029/2002JD003274.

Shukla, J. and Y. Mintz, 1982, Influence of land-surface evaporation on
the Earth's climate, *Science*, **215**, 1498-1501.

Stackpole, J. D., A guide to GRIB: The World Meteorological Organization
form for the storage of weather product information and the exchange of
weather product messages in gridded binary form, 1st ed., Off. Note 388,
Natl. Meteorol. Cent. Natl. Weather Serv. Natl. Oceanic and Atmos.
Admin., Camp Springs, Md., Feb. 1994.

Staudenmaier, M., 1996: Precipitation verification statistics from the
NCEP Operational Model Suite, *National Weather Service Western Regional
Technical Attachment*, **96-28**, 11 pp.

Stephenson, David B. 2000: Use of the \"Odds Ratio\" for Diagnosing
Forecast Skill. *Wea. and Forecasting*, **15**, 221-232.

Ucellini L., and Coauthors 2005: NCEP Biennial Review. Available from
[http://www.ncep.noaa.gov](http://www.ncep.noaa.gov/), accessed on
September 27^th^, 2005.

Velazco, I., and J. M. Fritsch, 1987: Mesoscale convective complexes in
the Americas. *J. Geophys. Res*., **92**, 9591-9613.

Xie, P., and P. A. Arkin, 1997: Global Precipitation: A 17-year monthly
analysis based on gauge observations, satellite estimates, and numerical
model outputs. *Bull. Amer. Meteor. Soc.*, **78**, 2539-2558.

Wan Z., 1999: MODIS Land Surface Temperature Algorithm Theoretical Basis
Document. Available from
<http://modis.gsfc.nasa.gov/data/atbd/atbd_mod11.pdf>, accessed on
September 29, 2005.

Weiss, S.J., J. S. Kain, J. J. Levit, M. E. Baldwin, and D.R. Bright:
Examinations of the WRF model for the prediction of severe convective
weather: the SPC/NSSL spring program 2004. Preprints, 22nd Conference on
Severe Local Storms, Hyannis, MA, Amer. Meteor. Soc.

Werner, M., 2001, Shuttle Radar Topography Mission (SRTM), Mission
overview, J. Telecom. (Frequenz), **55**, 75-79.

Wilks, D.S., 1995. *Statistical Methods in the Atmospheric Sciences: an
Introduction*, International Geophysics Series, **Vol. 59**, Academic
Press, 464 pp.

Wu, W.S., R. James Purser, and D. Parrish, 2002: Three dimensional
variational analysis with spatially inhomogeneous covariances. *Mon.
Wea. Rev.*, **130**, 2905-2916.

Ziltinkevitch, S.S., 1970: Dynamics of the planetary boundary layer.
*Gidrometeorologicheskoe Izdatelystvo*, Leningrad, 292 pp. (in Russian).

Zobler, L., 1986, A world soil file for global climate modeling, NASA
Technical Memorandum #87802.

## Acronyms 

  ------------- ----------------------------------------------------------
  **Acronym**   **Meaning**

  ACARS         Aircraft Communication Addressing and Reporting System

  AFWA          Air Force Weather Agency

  AGRMET        AFWA Agricultural Meteorological System

  AHPS          Advanced Hydrologic Prediction Service

  AIRREP        Aircraft Weather Report

  AMS           American Meteorological Society

  AMSR-E        Advanced Microwave Scanning Radiometer - EOS

  APL           Appalachian Verification Region

  AQF           Air Quality Forecast

  ARM/CART      Atmospheric Radiation Measurement Program / Cloud and
                Radiation Test bed

  ASDAR         Aircraft to Satellite Data Relay

  ASOS          Automated Surface Observation System

  ATOVS         Advanced TIROS Vertical Sounder

  AWARDS        Agricultural Water Resources and Decision Support Tool

  AWIPS         Advanced Weather Interactive Processing System

  BUFR          Binary Universal Form for the Representation of
                meteorological data

  CEOP          Coordinated Enhanced Observing Period

  CLM           Common Land Model

  CMAP          CPC satellite/gauge Merged Analysis of Precipitation
                Product+B71

  CMAQ          Community Multi-scale Air Quality modeling system

  CMORPH        CPC Morphing Technique

  CONUS         Continental United States

  CPC           Climate Prediction Center (NCEP)

  DMSP          Defense Meteorological Satellite Program

  DOE           Department of Energy

  DST           Decision Support Tool

  ECA           Eastern Canada Verification Region

  EDAS          Eta Data Assimilation System

  EOS           Earth Observing System

  EPA           Environmental Protection Agency

  ETS           Equitable Threat Score

  FAR           False Alarm Ratio

  FVS           Forecast Verification System

  GAPP          GEWEX Americas Prediction Project

  GEWEX         Global Energy and Water Cycle Experiment

  GFS           Global Forecast System

  GLDAS         Global Land Data Assimilation System

  GMC           Gulf of Mexico Coast Verification Region

  GOES          Geostationary Operational Environmental Satellite

  GRB           Great Basin Verification Region

  GRIB          Gridded Binary

  GSFC          Goddard Space Flight Center

  GSI           Grid-point Statistical Interpolation

  GSWP          Global Soil Wetness Project

  GTC           Global Telecommunications System

  HP            Hewlett Packard

  IBM           International Business Machines

  ISFS          National Environmental Satellite Data and Information
                Service

  LAI           Leaf Area Index

  LDAS          Land Data Assimilation System

  LFM           Limited Fine Mesh Model

  LIS           Land Information System

  LIS1          Retrospective NLDASE land surface simulation (no MODIS
                snow cover assimilation) and Eta model runs using initial
                conditions from this NLDASE simulation

  LIS2          Retrospective NLDASE land surface simulation (5mm MODIS
                snow cover assimilation) and Eta model runs using initial
                conditions from this NLDASE simulation

  LIS3          Retrospective NLDASE land surface simulation (10mm MODIS
                snow cover assimilation) and Eta model runs using initial
                conditions from this NLDASE simulation

  LMV           Lower Mississippi Valley Verification Region

  LSM           Land Surface Model

  LST           Land Surface Temperature

  MDW           Midwest Verification Region

  METEOSAT      Meteorological Satellite

  MEX           Mexico Verification Region

  MMS           Malaria Monitoring and Surveillance System

  MODIS         Moderate Resolution Imaging Spectroradiometer

  NAM           North American Mesoscale

  NAS AWRP      National Air Space Aviation Weather Research Program

  NASA          National Aeronautics and Space Administration

  NCDC          National Climatic Data Center

  NCEP          National Center for Environmental Prediction

  NEC           Northeast Coast Verification Region

  NESDIS        National Environmental Satellite Data and Information
                Service

  NEXRAD        Next Generation Radar

  NIDIS         National Integrated Drought Information System

  NLDAS         North American Land Data Assimilation System

  NMM           Non-hydrostatic Mesoscale Model

  NMT           Northern Mountains Verification Region

  NOAA          National Oceanic and Atmospheric Administration

  NOAAPORT      NOAA Data Portal

  NOHRSC        National Operation Hydrologic Remote Sensing Center

  NPL           Northern Plains Verification Region

  NSF           National Science Foundation

  NWC           Northwest Coast Verification Region

  NWP           Numerical Weather Prediction

  NWS           National Weather Service

  OHD           Office of Hydrologic Development

  ORPC          Open Radar Product Generator

  PILPS         Project for Intercomparison of Land Surface
                Parameterizations

  PIREP         Pilot Reports

  POD           Probability of Detection

  PREPBUFR      Preparatory BUFR

  PRISM         Parameter-Elevation Regressions on Independent Slopes
                Model

  RCDAS         Regional Climate Data Assimilation System

  RFC           River Forecast Center

  RMSE          Root Mean Square Error

  ROC           Radar Operations Center

  RTNEPH        Real-time Nephanalysis

  RTOVS         Revised TIROS Vertical Sounder

  RUC           Rapid Update Cycle

  SCAN          Soil Climate Analysis Network

  SEC           Southeast Coast Verification Region

  SMT           Southern Mountain Verification Region

  SNODEP        Snow Depth Analysis Model

  SPL           Southern Plains Verification Region

  SRTM          Shuttle Radar Topography Mission

  SSM/I         Special Sensor Microwave Imager

  SURFRAD       Surface Radiation Budget Network

  SWC           Southwest Coast Verification Region

  SWD           Southwest Desert Verification Region

  SWE           Snow Water Equivalent

  TAF           Terminal Aerodrome Forecast

  TIROS         Television Infrared Observation Satellite Program

  TRMM          Tropical Rainfall Measuring Mission

  TMI           TRMM Microwave Imager

  UMD           University of Maryland

  USDA          United States Department of Agriculture

  USDOD         United States Department of Defense

  USWRP         United States Weather Research Program

  VAD           Vertical Azimuth Display

  VIC           Variable Infiltration Capacity Model

  WCA           Western Canada Verification Region

  WFO           Weather Forecast Office

  WRF           Weather Research and Forecast Model
  ------------- ----------------------------------------------------------