NASA/CR-2014-218439

Range-specific High-Resolution Mesoscale 

Model Setup: Data Assimilation

Leela R. Watson

ENSCO, Inc., Cocoa Beach, Florida

NASA Applied Meteorology Unit, Kennedy Space Center, Florida

September 2014
NASA STI Program ... in Profile

Since its founding, NASA has been dedicated 

to the advancement of aeronautics and space 

science. The NASA scientific and technical 

information (STI) program plays a key part in 

helping NASA maintain this important role.

The NASA STI program operates under the 

auspices of the Agency Chief Information Officer. 

It collects, organizes, provides for archiving, and 

disseminates NASA’s STI. The NASA STI 

program provides access to the NTRS Registered

and its public interface, the NASA Technical 

Reports Server, thus providing one of the largest 

collections of aeronautical and space science STI 

in the world. Results are published in both nonNASA channels and by NASA in the NASA STI 

Report Series, which includes the following report 

types:

x TECHNICAL PUBLICATION. Reports of 

completed research or a major significant 

phase of research that present the results of 

NASA Programs and include extensive data 

or theoretical analysis. Includes compilations of significant scientific and technical 

data and information deemed to be of 

continuing reference value. NASA counterpart of peer-reviewed formal professional 

papers but has less stringent limitations on 

manuscript length and extent of graphic 

presentations.

x TECHNICAL MEMORANDUM. 

Scientific and technical findings that are 

preliminary or of specialized interest, 

e.g., quick release reports, working 

papers, and bibliographies that contain 

minimal annotation. Does not contain 

extensive analysis.

x CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees.

x CONFERENCE PUBLICATION. 

Collected papers from scientific and 

technical conferences, symposia, seminars, 

or other meetings sponsored or 

co-sponsored by NASA.

x SPECIAL PUBLICATION. Scientific, 

technical, or historical information from 

NASA programs, projects, and missions, 

often concerned with subjects having 

substantial public interest.

x TECHNICAL TRANSLATION. 

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission.

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and 

feeds, providing information desk and personal 

search support, and enabling data exchange 

services.

For more information about the NASA STI 

program, see the following:

x Access the NASA STI program home page 

at http://www.sti.nasa.gov

x E-mail your question to help@sti.nasa.gov

x Phone the NASA STI Information Desk at 

757-864-9658

x Write to:

NASA STI Information Desk

Mail Stop 148

NASA Langley Research Center

Hampton, VA 23681-2199
NASA/CR-2014-218439

Range-specific High-Resolution Mesoscale 

Model Setup: Data Assimilation

Leela R. Watson

ENSCO, Inc., Cocoa Beach, Florida

NASA Applied Meteorology Unit, Kennedy Space Center, Florida

National Aeronautics and

Space Administration

Kennedy Space Center

Kennedy Space Center, FL 32899-0001

September 2014
Acknowledgments 

The author thanks Mr. Brad Zavodsky of NASA’s Short-term Prediction Research and 

Transition Center for providing the modeling scripts, data, and technical support throughout this 

project.

Available from:

NASA Center for AeroSpace Information

7115 Standard Drive

Hanover, MD 21076-1320

443-757-5802

This report is also available in electronic form at

http://science.ksc.nasa.gov/amu
Executive Summary

Mesoscale weather conditions can have an adverse effect on space launch, landing, and 

ground processing at the Eastern Range (ER) in Florida and Wallops Flight Facility (WFF) in 

Virginia. During summer, land-water interactions across Kennedy Space Center (KSC) and Cape 

Canaveral Air Force Station (CCAFS) lead to sea and river breeze front formation, which can 

spawn deep convection that can hinder operations and endanger personnel and resources. Many 

other weak locally driven low-level boundaries and their interactions with the sea breeze front and 

each other can also initiate deep convection in the KSC/CCAFS area. All these subtle weak 

boundary interactions often make forecasting of operationally important weather very difficult at 

KSC/CCAFS during the summer. These convective processes often build quickly, last a short 

time (60 minutes or less), and occur over small distances, all of which also pose a significant 

challenge to the local forecasters. Surface winds during the transition seasons (spring and fall) 

pose the most difficulties for the forecasters at WFF. They also encounter problems forecasting 

convective activity and temperature during those seasons.

Global and national scale models cannot properly resolve important local-scale weather 

features at each location due to their horizontal resolutions being much too coarse. Therefore, a 

properly tuned model at a high resolution is needed to provide improved forecasting capability. 

To accomplish this, the ER and WFF supported the Applied Meteorology Unit (AMU) task to 

perform a number of sensitivity tests in order to determine the best model configuration for 

operational use at each of the ranges to best predict winds, precipitation, and temperature. That 

task was completed in January 2013. This task is a continuation of that work and will provide a 

recommended local data assimilation (DA) and numerical forecast model design optimized for the 

ER and WFF to support space launch activities and for local weather challenges at both ranges.

The AMU examined different model configurations by running the DA on varying grid 

resolutions and nesting configurations to determine the impact on model skill. Data assimilation 

is an important component in producing quality numerical weather prediction forecasts. The DA

software chosen for this task was the Gridpoint Statistical Interpolation (GSI) system developed 

by the National Centers for Environmental Prediction. The NASA Short-term Prediction Research 

and Transition Center provided a set of Perl scripts that run both the GSI DA system and the 

Weather Research and Forecast (WRF) model in a compact system. The objective of the scripts 

is to provide an easy-to-use interface for users to be able to execute GSI and WRF. 

The AMU assessed model skill by using an objective statistical analysis that included the

mean error, root mean square error, and Pearson correlation coefficient. Precipitation forecasts 

were evaluated using the Method for Object-Based Diagnostic Evaluation, a technique developed 

at the National Center for Atmospheric Research. Results indicate that for both the ER and WFF, 

a triple nest configuration that had a 9-km outer, 3-km middle, and 1-km inner nest was the optimal 

model configuration for both ranges. However, a double-nest configuration (2-km outer and 0.67-

km inner nest) performed the best in predicting precipitation over the ER. Being able to predict 

summertime convection over the ER is operationally important and, for this reason, the AMU 

believes the double-nest configuration can be used as an alternative optimal configuration over 

the ER. 

Future work will provide real-time output of the GSI/WRF cycled runs on the AMU’s Advanced 

Weather Interactive Processing System II workstations. The AMU will also explore running the 

system in a rapid-refresh mode and will explore ensemble modeling to further improve model 

output. 

1
Table of Contents

Executive Summary ................................................................................................................... 1

Table of Contents....................................................................................................................... 2

List of Figures ............................................................................................................................ 3

List of Tables ............................................................................................................................. 4

1 Introduction......................................................................................................................... 5

1.1 Phase I Range Modeling Work..................................................................................... 5

1.2 Report Format and Outline........................................................................................... 6

2 Data and Model Configuration ............................................................................................ 7

2.1 Data Assimilation Software .......................................................................................... 7

2.2 Observational Data Ingest............................................................................................ 7

2.3 Model Configuration and Test Cases ........................................................................... 7

2.4 NASA SPoRT Perl Scripts ..........................................................................................11

3 Model Forecast Validation..................................................................................................13

3.1 Observational Data .....................................................................................................13

3.2 MET Software.............................................................................................................15

3.3 ER Results..................................................................................................................17

3.4 WFF Results...............................................................................................................22

4 Conclusions and Future Work............................................................................................27

References ...............................................................................................................................28

2
List of Figures

Figure 1. Map of the ER showing the single 1-km (D01) model domain boundary (1 

dom). ........................................................................................................................ 9

Figure 2. Map of the ER showing the nested 2-km outer (D01) and 0.67-km inner (D02) 

model domain boundaries (2 doms).......................................................................... 9

Figure 3. Map of the triple nest configuration showing the 9-km outer (D01), 3-km 

middle (D02 and D04), and 1-km inner (D03 and D05) model domain 

boundaries over the ER and WFF (5 doms).............................................................10

Figure 4. Map of WFF showing the nested 4-km outer (D01) and 1.33-km inner (D02) 

model domain boundaries (2 doms).........................................................................10

Figure 5. Schematic showing the 12-hour pre-cycling that occurs in the GSI/WRF 

system. Text in green is the model initialization time, solid red arrows are the

pre-cycle forecast, and red dashed arrows are the full model forecast. This 

figure is a recreation of a NASA SPoRT produced schematic..................................12

Figure 6. Map of a sampling of the mesonet and METAR weather station locations over 

the ER and the surrounding areas from MADIS. ......................................................13

Figure 7. Map of a sampling of the mesonet and METAR weather station locations over 

WFF and the surrounding areas from MADIS. .........................................................14

Figure 8. Chart of the average hourly ME for the 12-hour forecast over the entire POR 

for a) wind speed, b) wind direction, c) temperature, d) dewpoint temperature, 

and e) MSLP from the three GSI/WRF configurations at the ER..............................18

Figure 9. Chart of the average hourly RMSE for the 12-hour forecast over the entire 

POR for a) wind speed, b) wind direction, c) temperature, d) dewpoint 

temperature, and e) MSLP from the three GSI/WRF configurations at the ER.........19

Figure 10. Chart of the average hourly PCC for the 12-hour forecast over the entire POR 

for a) wind speed, b) temperature, c) dewpoint temperature, and d) MSLP 

from the three GSI/WRF configurations at the ER....................................................20

Figure 11. Chart of the average hourly a) centroid distance (km), b) area ratio, and c) 

total interest value from the three GSI/WRF configurations for each hour of 

the 12-hour forecast over the entire POR at the ER.................................................21

Figure 12. Chart of the average hourly ME for the 12-hour forecast over the entire POR 

for a) wind speed, b) wind direction, c) temperature, d) dewpoint temperature, 

and e) MSLP from the two GSI/WRF configurations at WFF....................................23

Figure 13. Chart of the average hourly RMSE for the 12-hour forecast over the entire 

POR for a) wind speed, b) wind direction, c) temperature, d) dewpoint 

temperature, and e) MSLP from the two GSI/WRF configurations at WFF...............24

Figure 14. Chart of the average hourly PCC for the 12-hour forecast over the entire POR 

for a) wind speed, b) temperature, c) dewpoint temperature, and d) MSLP 

from the two GSI/WRF configurations at WFF. ........................................................25

Figure 15. Chart of the average hourly a) centroid distance (km), b) area ratio, and c) 

total interest value from the two GSI/WRF configurations for each hour of the 

12-hour forecast over the entire POR at WFF..........................................................26

3
List of Tables

Table 1. List of conventional and satellite observations that can be assimilated into GSI. .......... 8

4
1 Introduction

Mesoscale weather conditions can have an adverse effect on space launch, landing, and 

ground processing at the Eastern Range (ER) in Florida and Wallops Flight Facility (WFF) in 

Virginia. During summer, land-sea interactions across Kennedy Space Center (KSC) and Cape

Canaveral Air Force Station (CCAFS) lead to sea breeze front formation, which can spawn deep 

convection that can hinder operations and endanger personnel and resources. Many other weak 

locally driven low-level boundaries and their interactions with the sea breeze front and each other 

can also initiate deep convection in the KSC/CCAFS area. Some of these other boundaries 

include the Indian River breeze front, Banana River breeze front, outflows from previous 

convection, horizontal convective rolls, convergence lines from other inland bodies of water such 

as Lake Okeechobee, the trailing convergence line from convergence of sea breeze fronts due to 

the shape of Cape Canaveral, frictional convergence lines from the islands in the Bahamas, 

convergence lines from soil moisture differences, convergence lines from cloud shading, and 

others. All these subtle weak boundary interactions often make forecasting of operationally 

important weather very difficult at KSC/CCAFS during the convective season (May-Oct). These

convective processes often build quickly, last a short time (60 minutes or less), and occur over 

small distances, all of which also poses a significant challenge to the local forecasters who are 

responsible for issuing weather advisories, watches, and warnings. Surface winds during the 

transition seasons of spring and fall pose the most difficulties for the forecasters at WFF. They 

also encounter problems forecasting convective activity and temperature during those seasons. 

Therefore, accurate mesoscale model forecasts are needed to aid in their decision making.

Both the ER and WFF would benefit greatly from high-resolution mesoscale model output to 

better forecast a variety of unique weather phenomena. Global and national scale models cannot 

properly resolve important local-scale weather features at each location due to their horizontal 

resolutions being much too coarse. Therefore, a properly tuned model at a high resolution is 

needed to provide improved capability. This task is a multi-year effort in which the Applied 

Meteorology Unit (AMU) will tune the Weather Research and Forecasting (WRF) model 

individually for each range. The goal of the first year, the results of which are in this report, was 

to tune the WRF model based on the best model resolution and run time while using reasonable 

computing capabilities. To accomplish this, the ER and WFF supported the tasking of the AMU to 

perform a number of sensitivity tests in order to determine the best model configuration for 

operational use at each of the ranges to best predict winds, precipitation, and temperature

(Watson 2013). This task is a continuation of that work and will provide a recommended local data 

assimilation (DA) and numerical forecast model design optimized for the ER and WFF to support 

space launch activities. The model will be optimized for local weather challenges at both ranges.

1.1 Phase I Range Modeling Work

In Phase I of this task, the ER and WFF supported the tasking of the AMU to perform a number 

of sensitivity tests in order to determine the best model configuration for operational use at each 

of the ranges to best predict winds, precipitation, and temperature. The goal of the first phase, 

the results of which are in Watson (2013), was to tune the WRF model based on the best model 

resolution and run time while using reasonable computing capabilities.

The AMU compared model forecasts for both the ER and WFF using different WRF model 

dynamical cores, grid configurations, and physical schemes to determine the impact on model 

skill. Different model configurations were tested by varying the dynamical core, grid spacing, 

domain size, and forecast length. This enabled the AMU to determine the optimal configuration 

that allowed for the largest domain size and highest resolution to capture unique weather 

phenomena with the shortest wall-clock run time. Once the configurations were chosen, the AMU 

varied the model physics to determine which produced the best forecasts. The WRF model 

5
forecasts were validated using simple statistics that compared locally available observational 

surface and upper-air data to the forecast data. The objective statistical analysis included the 

model bias, mean error (ME), and the root mean square error (RMSE). Precipitation forecasts 

were compared to nationally available rainfall data using the Method for Object-Based Diagnostic 

Evaluation (MODE), a technique developed at the National Center for Atmospheric Research

(NCAR).

The AMU ran test cases in the warm and cool seasons at the ER and for the spring and fall 

seasons at WFF. For both the ER and WFF, the Advanced Research WRF (ARW) core 

outperformed the Non-hydrostatic Mesoscale Model core. Results for the ER indicate that the Lin 

microphysical scheme and the Yonsei University (YSU) planetary boundary layer (PBL) scheme 

is the optimal model configuration for the ER. It consistently produced the best surface and upper 

air forecasts, while performing fairly well for the precipitation forecasts. Both the Ferrier and Lin 

microphysical schemes in combination with the YSU PBL scheme performed well for WFF in the 

spring and fall seasons.

1.2 Report Format and Outline

This report presents the findings from a study of local DA and numerical forecast model design 

optimized for the ER and WFF to support space launch and for predicting unique weather 

phenomena at the ER and WFF. This analysis examined different model configurations by running 

the DA on varying grid resolutions and nesting configurations to determine the impact on model 

skill. The AMU assessed model skill by using an objective statistical analysis that included the 

ME, RMSE, and Pearson correlation coefficient (PCC). Precipitation forecasts were evaluated 

using MODE. Section 2 describes the DA software, the observational data used in the study, the 

model configurations, and the NASA Short-term Prediction Research and Transition Center 

(SPoRT) supplied Perl scripts used to run the forecast system. Section 3 describes the forecast 

validation technique and the results for both the ER and WFF. Section 4 contains the conclusions 

and future work.

6
2 Data and Model Configuration

The important aspects of this study were choice of DA software, the observational data ingest, 

period of record (POR), the DA/model configurations, and the DA/modeling system Perl scripts

supplied by SPoRT.

2.1 Data Assimilation Software

DA is an important component in producing quality numerical weather prediction forecasts. 

DA methods are used to create a best estimate of the state of the atmosphere at the forecast 

initial time using information from observations and a background model forecast. This is 

accomplished through the minimization of an objective function that measures the weighted 

distance of the analysis from the observations and the background model. The weights assigned 

to each term are based on the error characteristics of the observations and the background model 

(Kleist et al. 2009). This creates an initial state that more closely resembles the state of the 

atmosphere at the model initialization time. 

The DA software chosen for this task was the Gridpoint Statistical Interpolation (GSI) system 

developed by the National Centers for Environmental Prediction (NCEP). The GSI system is a 

three-dimensional variational DA system that is a freely available, community system designed to 

be flexible, state-of-the-art, and can run efficiently on various parallel computing platforms. It can 

be applied to both regional and global applications. 

2.2 Observational Data Ingest

GSI can ingest large quantities of atmospheric observations and has developed capabilities 

for data thinning, quality controlling, and satellite radiance bias correction (Wang 2010). Table 1

lists some of the conventional and satellite radiance/brightness temperature observations that

GSI can assimilate. 

Most of these observations are complex and many of them need to be reformatted into Binary 

Universal Form for the Representation of meteorological data (BUFR) format or quality controlled 

before being used by GSI. NCEP produces quality-controlled data in BUFR format, called

PrepBUFR files that can be used directly in GSI. These freely available files were used for this 

task. 

2.3 Model Configuration and Test Cases

The AMU determined the main model configuration from the first phase of the Range Modeling 

task. The ARW core was used with the Lin microphysical scheme and the YSU PBL scheme for

both the ER and WFF. Although the AMU previous work found the optimal horizontal grid spacing 

for the ER was a 2-km outer and 0.67-km inner domain and a 4-km outer and 1.33-km inner 

domain for WFF, the AMU conducted additional testing on the optimal horizontal grid spacing

once the DA was added. Specifically, different configurations were run to determine the impact 

on the forecasts of running the DA over differing grid resolutions. For the ER, the AMU ran three 

configurations:

x Single domain: 1-km domain in which the DA was run (referred to as ‘1 dom’, Figure 1),

x Nested domain: 2-km outer and 0.67-km inner domain in which the DA was run on the 

outer 2-km domain (referred to as ‘2 doms’, Figure 2),

x Triple nested domain: 9-km outer, 3-km middle, and 1-km inner domain on which the DA 

was run on the 9-km outer domain (referred to as ‘5 doms’, Figure 3).

7
For WFF, the AMU ran two configurations:

x Nested domain: 4-km outer and 1.33-km inner domain in which the DA was run on the 

outer 4-km domain (referred to as ‘2 doms’, Figure 4),

x Triple nested domain: 9-km outer, 3-km middle, and 1-km inner domain on which the DA 

was run on the 9-km outer domain (referred to as ‘5 doms’, Figure 3).

All other parameters were the same for each model run. 

Table 1. List of conventional and satellite observations that can be assimilated into GSI.

Conventional observations

x Radiosondes

x Conventional aircraft 

reports

x MODIS IR and water vapor 

winds

x Surface land observations

x Doppler radial velocities

x SBUV ozone profiles, MLS 

(including NRT) ozone, and 

OMI total ozone 

x Wind profilers: US, JMA 

x Dropsondes 

x GEOS hourly IR and coud 

top wind 

x GPS Radio occultation 

refractivity and bending 

angle profiles

x Pibal winds 

x ASDAR aircraft reports 

x GMS, JMA, and 

METEOSAT cloud drift IR 

and visible winds

x SSM/I wind speeds 

x VAD (NEXRAD) winds 

x SST 

x Doppler wind Lidar data

x Tail Dopppler Radar radial 

velocity and superobservation 

x METAR cloud observations

x SSM/I and TRMM TMI 

precipitation estimates

x Synthetic tropical cyclone 

winds 

x MDCARS aircraft reports 

x EUMETSAT and GOES 

water vapor cloud top winds

x QuikScat, ASCAT and 

OSCAT wind speed and 

direction 

x GPS precipitable water 

estimates 

x Tropical storm VITAL 

x Radar radial wind and 

reflectivity Mosaic 

x PM2.5 

x Surface ship and buoy 

observation

Satellite radiance/brightness temperature observations

x SBUV: n17, n18, n19

x AIRS:aqua

x MHS: metop-a, metop-b, 

n18, n19

x AMSRE: aqua

x GOME: metop-a, metop-b,

x ATMS: NPP

x HIRS: metop-a, metop-b, 

n17, n19

x SSMI: f14, f15 

x SNDR: g11, g12, g13 

x OMI: aura 

x SEVIRI: m08, m09, m10

x GOES_IMG: g11, g12 

x AMSU-A: metop-a, metopb, n15, n18, n19, aqua 

x AMSU-B: metop-b, n17 

x SSMIS: f16 

x IASI: metop-a, metop-b

8
Figure 1. Map of the ER showing the single 1-km (D01) model domain 

boundary (1 dom).

Figure 2. Map of the ER showing the nested 2-km outer (D01) and 

0.67-km inner (D02) model domain boundaries (2 doms).

9
Figure 3. Map of the triple nest configuration showing the 9-km outer 

(D01), 3-km middle (D02 and D04), and 1-km inner (D03 and D05) 

model domain boundaries over the ER and WFF (5 doms).

Figure 4. Map of WFF showing the nested 4-km 

outer (D01) and 1.33-km inner (D02) model domain 

boundaries (2 doms).

10
The AMU ran each DA/model simulation at the specified horizontal resolution with 35 

irregularly spaced, vertical sigma levels up to 50 mb. Each run was initiated four times per day at 

0000, 0600, 1200, and 1800 UTC and integrated 12 hours using the 13-km Rapid Refresh (RAP) 

model for boundary conditions and as the background model first-guess field, Land Information 

System (LIS) data from SPoRT for land surface data, and sea surface temperature (SST) data 

from both NCEP’s Real-time Global SSTs and the SPoRT 2-km SST composites. Initial conditions 

were created using a cycled GSI/WRF approach that is described in Section 2.4. The POR for 

the test cases was from 1200 UTC 27 August 2013 to 0600 UTC 10 November 2013.

2.4 NASA SPoRT Perl Scripts

SPoRT provided a set of Perl scripts that run both the GSI DA system and the WRF model in 

a compact system. The objective of the scripts is to provide an easy-to-use interface for users to 

be able to execute GSI and WRF. The SPoRT scripts are simply a wrapper to support running of 

the software systems, which must be downloaded and installed separately by the user. 

The GSI/WRF scripts use a cycled GSI system similar to the operational North American 

Mesoscale (NAM) model. The scripts run a 12-hour pre-cycle in which data are assimilated from 

12 hours prior up to the model initialization time. This is done due to the time latency of the satellite 

data. Satellite data are not available instantaneously as it takes time to receive and process. If 

the pre-cycling did not occur, there would be very little influence on model output from the satellite 

observations, which have been shown to have the largest positive impact on most forecast 

systems. Once the pre-cycling is complete, a 12-hour forecast is run. 

A schematic of the GSI/WRF pre-cycling system is shown in Figure 5. Model initialization 

times are shown in green, pre-cycle times are shown in blue, and forecasts are shown in red. As 

stated above, the model is run four times per day. For a model initialization time of 0000 UTC

(t00Z in Figure 5), the GSI/WRF scripts are started between 0000 and 0100 UTC. The scripts 

begin their pre-cycling 12 hours prior to initialization time, in this case at 1200 UTC of the previous 

day. Initial conditions for the WRF forecast are first created by processing the RAP data as the 

background model, the LIS land surface data, and NCEP and SPoRT SST data using the WRF 

preprocessing system. Once a background grid has been established, observational data are 

assimilated into the background grid using the GSI system. After the observations have been 

assimilated, a 3-hour WRF forecast is run (red arrow between tm12 and tm09 for t00Z). The precycle begins again using data from nine hours prior to the initialization time (tm09, 1500 UTC of 

the previous day), however, the 3-hour WRF forecast is now used as the background model for 

the next forecast. This process continues every three hours until the initialization time is reached. 

At 0000 UTC after the background grid is created using the 3-hour WRF forecast and the 

observational data is assimilated, a 12-hour WRF forecast is run (red dashed line). The process 

repeats itself every six hours.

11
Figure 5. Schematic showing the 12-hour pre-cycling that occurs in the GSI/WRF 

system. Text in green is the model initialization time, solid red arrows are the precycle forecast, and red dashed arrows are the full model forecast. This figure is a 

recreation of a NASA SPoRT produced schematic.

12
3 Model Forecast Validation

The AMU validated the GSI/WRF model forecasts using statistics that compared locally 

available surface observations to the forecast data. Precipitation forecasts were compared to 

nationally available rainfall data using a technique developed at NCAR.

3.1 Observational Data

In order to verify the GSI/WRF model performance, surface weather observations of 

temperature, dewpoint, wind speed and direction, and atmospheric pressure were required.

NCEP’s Meteorological Assimilation Data Ingest System (MADIS) and Stage IV precipitation data 

were used for the observational datasets.

MADIS was developed by NOAA’s Earth System Research Laboratory Global Systems 

Division to collect, integrate, quality control, and distribute observations from a multitude of 

organizations. MADIS includes various meteorological surface datasets (such as Meteorological 

Aerodrome Report (METAR), maritime, and mesonet data, among others), profiler network data, 

aircraft data, radiosondes, satellite data, hydrological surface data, radiometer, and snow data. 

Figure 6. Map of a sampling of the mesonet and METAR weather 

station locations over the ER and the surrounding areas from MADIS. 

13
For this task, the AMU downloaded METAR and mesonet data to validate the GSI/WRF 

forecasts. METAR is the international standard code format for hourly surface weather 

observations and typically contains data for the temperature, dew point, wind speed and direction, 

precipitation, cloud cover and heights, visibility, and barometric pressure. Mesonet data is a 

network of automated weather stations designed to observe mesoscale meteorological 

phenomena, such as dry lines, squall lines, and sea breezes, and report conditions in time 

intervals anywhere from 1 to 15 minutes. The locations of some of the METAR and mesonet 

stations over the ER and WFF are shown in Figure 6 and Figure 7, respectively.

Figure 7. Map of a sampling of the mesonet and METAR weather 

station locations over WFF and the surrounding areas from MADIS. 

14
To verify precipitation, the AMU compared hourly forecast rainfall accumulation to the NCEP 

Stage-IV analysis (http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage4/). This analysis 

combines radar data and rain gauge reports to produce hourly rainfall accumulation on a 4-km

grid. It is a manually quality-controlled continental U.S. mosaic from the regional 1-hour

precipitation analyses produced by 12 National Weather Service River Forecast Centers (Lin and 

Mitchell 2005). 

3.2 MET Software

For the objective analysis, the AMU compared observed wind speed, temperature, dewpoint 

temperature, mean sea-level pressure, and accumulated precipitation observations to the 

forecast variables using the latest version of the Model Evaluation Tools (MET) software. This 

software was developed by the NCAR Developmental Testbed Center. It is a state-of-the-art suite 

of verification tools that uses output from the WRF model to compute standard verification scores 

comparing gridded model data to point or gridded observations, and uses spatial verification 

methods comparing gridded model data to gridded observations using object-based 

decomposition procedures. For this task, two MET tools were chosen to validate the GSI/WRF 

forecasts: the Point-Stat tool and the MODE tool.

3.2.1 Point-Stat Tool

The MET Point-Stat tool was used to verify the surface forecasts. This tool provides 

verification statistics for forecasts at observation points, in this case, at the METAR and mesonet 

point observations. The Point-Stat tool matches gridded forecasts to point observation locations 

using a user-specified interpolation approach and computes the verification statistics. 

Many output statistics are available within the Point-Stat tool. This study looked at three of 

those statistics for wind speed, temperature (T), dewpoint temperature (Td), and mean sea-level 

pressure (MSLP): the ME, the RMSE, and the PCC; and two statistics for wind direction: the ME

and the RMSE. The statistics compared all mesonet and METAR observations available to the 

corresponding locations in the model forecast output at a 1-hour interval.

The mean error, ME, is a measure of the overall bias of the model parameter being compared. 

A perfect forecast has ME = 0. It is defined as:

¦  

 

n

i

i oi f

n

ME

1

1

where:

n = number of forecast and observation pairs over the forecast period,

fi = WRF forecast of T, Td, MSLP, wind speed, or wind direction, and

oi = observed T, Td, MSLP, wind speed, or wind direction.

The model RMSE was calculated to measure the magnitude of the error. It is useful in 

determining whether the forecasts produced large errors, as it gives relatively high weight to large 

errors. It is calculated using the following equations:

1 = ܧܵܯ

݊෍(݂௜ െ ݋(௜ଶ

௡

௜ୀଵ

ܧܵܯξ = ܧܵܯܴ

where n, fi, and oi are defined as above.

15
The PCC, r, measures the strength of the linear association between the forecast and 

observed parameters. It is defined as:

  

¦ ¦    

¦

 

 

 

 

 n

i

n

i

i i

n

i

i i

f f o o

f f o o

r

1 1

2 2

1

where n, fi, and oi are defined as above and:

f = average forecast over time and space of T, Td, MSLP, wind speed, or wind direction, and

o = average observed over time and space of T, Td, MSLP, wind speed, or wind direction.

The PCC can range between -1 and 1 where 1 indicates a perfect correlation, -1 indicates a 

perfect negative correlation, and 0 indicates no correlation between the forecast and 

observations. For example, the PCC for wind speed measures whether large values of forecast 

wind speed tend to be associated with large values of observed wind speed (positive correlation), 

whether small values of forecast wind speed tend to be associated with large values of observed 

wind speed or vice versa (negative correlation), or whether values of both variables are unrelated 

(correlation near 0). 

3.2.2 MODE

To verify precipitation, the AMU compared the forecast hourly rainfall accumulation to the 

observed rainfall over the same time period. MODE was used to determine the skill of each model 

configuration. MODE is an object-based verification system that compares gridded observations 

to gridded forecasts. It resolves and compares objects, such as areas of accumulated rainfall, in 

both the forecast and observed fields. The objects are described geometrically and then the 

attributes of the objects can be compared (Davis et al. 2006). MODE outputs statistics that 

describe the correlation between the objects and allows the user to identify forecast strengths or 

weaknesses. Details about how objects are identified and characterized can be found in Davis et 

al. (2006). For this report, the objects of interest are areas of accumulated rainfall. Therefore, 

references to objects are references to areas of resolved accumulated rainfall throughout the 

forecast period.

Once the objects have been identified, their various properties are evaluated and compared. 

The object attributes examined by the AMU in this task included the centroid distance, area ratio, 

and total interest value. The centroid distance is the vector difference between the centroids of 

the forecast and observed objects. It describes the location bias in the forecasts. The smaller the 

distance between the centroids, the better the forecast. The area ratio compares the area, or 

number of grid squares, the forecast object occupies compared to the observed object. An area 

ratio of 1 is considered a perfect correlation. Interest value is defined as the differences in 

particular attributes between the forecast and observed objects. Interest values of 0 indicate no

interest, or a poor forecast, while a value of 1 indicates high interest, or a good forecast. The total 

interest value is a weighted sum of specific interest values and is used as an overall indicator of 

the quality of the precipitation forecast. Total interest value is large when forecast and observed 

objects are well correlated (are roughly the same size and are close to each other) and is small 

when they are not well correlated. 

16
3.3 ER Results

The AMU validated model performance for the ER using forecasts from the grids described in 

section 2.3, where 2 doms references the nested 2-km outer and 0.67-km inner domain (Figure 

2), 1 dom references the single 1-km domain (Figure 1), and 5 doms references the triple nested 

9-km outer, 3-km middle, and 1-km inner domain over the ER (Figure 3).

3.3.1 Surface Forecasts

The AMU validated the GSI/WRF forecasts with the local METAR and mesonet data. Figure 

8 shows the ME for wind speed, wind direction, temperature, dewpoint temperature, and MSLP

from the three GSI/WRF configurations averaged over each hour of each 12-hour forecast for the 

entire POR at the ER. 

Overall, the ME results indicate that the triple-nest configuration performed the best of the 

three configurations, followed by the nested domain, and then the single domain. The triple-nest 

configuration had the lowest ME for wind speed, wind direction, dewpoint temperature, and MSLP

while the nested domain had the lowest ME for temperature. The single domain had the highest 

ME for wind speed, temperature, and dewpoint temperature. 

The average hourly ME results for wind speed indicated that all three domain configurations 

over-predicted the wind speed throughout the forecasts. The ME for wind speed for the single 

domain was approximately 2 m/s higher than for either nested domain during the entire forecast 

period with an average hourly ME of approximately 5 m/s. All three domains followed the same 

trend of slightly increasing ME throughout the forecasts for wind direction. The single domain had 

a consistent warm bias throughout the forecasts as indicated by the average hourly ME for 

temperature with a maximum ME of approximately 0.7 K. Both nested domains exhibited a cool 

bias throughout the forecasts, except for the 0-hour forecast. All domain configurations exhibited 

a cool dewpoint temperature bias, with the single domain showing the largest cool bias in the 2

to 2.3 K range. Results from the average hourly ME for MSLP show that all three configurations 

followed the same trend of an initial negative, or low pressure, bias followed by a slight positive, 

or high pressure, bias.

17
Figure 8. Chart of the average hourly ME for the 12-hour forecast over the entire POR for a) 

wind speed, b) wind direction, c) temperature, d) dewpoint temperature, and e) MSLP from the 

three GSI/WRF configurations at the ER.

As with ME, the RMSE results (Figure 9) indicate that the triple-nest configuration performed 

the best of the three configurations, followed by the nested domain, and then the single domain.

The triple-nest configuration had the lowest RMSE for wind speed, wind direction, and dewpoint

temperature while the nested domain had the lowest RMSE for temperature and MSLP. The 

single domain had the highest RMSE for all variables except wind direction.

a

c d

e

b

18
The general trends for the average hourly RMSE for wind speed and direction are nearly the 

same as the ME. The RMSE for temperature and dewpoint temperature indicate that the single 

domain still performed the worst. Average hourly RMSE for MSLP results are similar to the ME 

results, with all three domain configurations showing nearly the same trends in RMSE throughout 

the forecast. 

Figure 9. Chart of the average hourly RMSE for the 12-hour forecast over the entire POR for a) 

wind speed, b) wind direction, c) temperature, d) dewpoint temperature, and e) MSLP from the 

three GSI/WRF configurations at the ER.

a

e

c d

b

19
The PCC can indicate whether the model configurations captured the overall trend of the 

observed variables. Using wind speed as an example, it answers the question of whether the 

model winds fluctuated positively and negatively with the same magnitude as the observed winds. 

The closer the PCC is to 1, the better the model was able to capture these trends. When 

comparing these particular forecast vs. observed variables, only positive coefficients indicate any 

value in the model forecasts.

The results from the PCC (Figure 10) calculation indicate that the triple-nested configuration 

again performed the best of the three configurations followed by the single domain and then the 

nested domain. The triple-nest configuration had the highest PCC for wind speed, temperature, 

and MSLP while the single domain had the highest PCC for dewpoint temperature. 

All results show a positive correlation, except for the nested domain during forecast hours 8-

9 for MSLP indicating that the fluctuations in each of the variables was captured by the model. 

However, overall the values for PCC were low. This may be due to the difficulty in capturing 

mesoscale phenomena in the summertime over the ER.

Figure 10. Chart of the average hourly PCC for the 12-hour forecast over the entire POR for a) 

wind speed, b) temperature, c) dewpoint temperature, and d) MSLP from the three GSI/WRF 

configurations at the ER.

c d

a b

20
3.3.2 Precipitation Forecasts

The AMU compared precipitation forecasts from all three model configurations to determine 

performance differences. One-hour forecast accumulated rainfall for each hour of each of the 12-

hour forecasts was compared to the one-hour accumulation of observed rain using the NCEP

Stage-IV analysis data for each day during the POR. The POR summary statistics for each hour 

of the forecast of centroid distance, area ratio, and total interest value from the MODE software 

are shown in Figure 11. The centroid distance (km) results indicate that the nested domain

precipitation matched the location of the observed precipitation most closely throughout the 

forecasts, followed by the triple-nest configuration and then the single domain. The area ratio for 

the nested domain indicates that the forecast most closely matched the areal coverage of 

observed precipitation, with the triple-nest and single domain performing slightly worse and 

similarly, respectively. Interest value functions near 0.9 for the nested domain indicates that 

overall the forecast correlated the best with the observed precipitation, followed by the triple-nest 

configuration and then single domain. Overall, the nested domain outperformed both triple-nest 

and single domain configurations.

Figure 11. Chart of the average hourly a) centroid distance (km), b) area ratio, and c) total interest 

value from the three GSI/WRF configurations for each hour of the 12-hour forecast over the entire 

POR at the ER.

c

a b

21
3.4 WFF Results

The AMU validated model performance for WFF using forecasts from the grids described in 

section 2.3, where 2 doms references the nested 4-km outer and 1.33km inner domain (Figure 4)

and 5 doms references the triple-nested 9-km outer, 3-km middle, and 1-km inner domain over 

WFF (Figure 3).

3.4.1 Surface Forecasts

The AMU validated the GSI/WRF forecasts with the local METAR and mesonet data. Figure 

12 shows the ME for wind speed, wind direction, temperature, dewpoint temperature, and MSLP

from the two GSI/WRF configurations averaged over each hour of each 12-hour forecast for the 

entire POR at WFF.

Overall, the ME results indicate that the triple-nest configuration performed better than the 

nested domain. The triple-nest configuration had the highest ME for wind speed, wind direction, 

temperature, and MSLP while the nested domain had the highest ME for dewpoint temperature. 

Both model configurations over-predicted the wind speed throughout the forecast with a large 

increase in ME after the first forecast hour. However, the error for the triple-nest configuration was 

nearly 1.5 m/s less than that for the nested domain throughout the forecast. The same large 

increase after the first forecast hour in ME for wind direction was present in the nested domain

with slightly decreasing values of ME after forecast hour 3, while the triple-nest configuration

showed gradual increasing ME error values throughout the forecast period. The triple-nest 

configuration had lower ME values than the nested domain with differences in ME of 

approximately 15 to 30 degrees. ME values for temperature were nearly the same for both 

configurations during the first five forecast hours, after which the triple-nest configuration 

outperformed the nested domain with ME values of nearly 0. The triple-nest configuration

consistently forecasted dewpoint temperature values that were too cool, while the nested domain 

was too cool during the first 5 hours and then too warm after that. Both configurations followed 

the same ME trend for MSLP with pressures that were initially too low and then too high in the 

latter part of the forecasts. 

22
Figure 12. Chart of the average hourly ME for the 12-hour forecast over the entire POR for a) 

wind speed, b) wind direction, c) temperature, d) dewpoint temperature, and e) MSLP from the 

two GSI/WRF configurations at WFF.

Similar to the ME, the RMSE results (Figure 13) indicate that the triple-nest configuration 

performed the best for all variables. The RMSE wind speed and direction charts are nearly 

identical to the ME error charts indicating that there were not any large outliers for forecasted wind 

speed or direction. The triple-nest configuration RMSE for temperature and dewpoint temperature 

was roughly 1.25 K lower than that for the nested domain throughout the forecasts. Except for 

e

d

a b

c

23
forecast hours 2 and 3, the triple-nest configuration outperformed the nested domain for MSLP 

with differences of approximately 60 Pa lower in the latter half of the forecasts.

Figure 13. Chart of the average hourly RMSE for the 12-hour forecast over the entire POR for a) 

wind speed, b) wind direction, c) temperature, d) dewpoint temperature, and e) MSLP from the 

two GSI/WRF configurations at WFF.

e

d

a b

c

24
The results from the PCC calculation (Figure 14) indicate that the triple-nest configuration 

again outperformed the nested domain for wind speed, temperature, dewpoint temperature, and 

MSLP. The PCC for the triple-nest configuration for wind speed and temperature was 

approximately 0.05 to 0.1 higher than the nested domain throughout the forecast. The PCC for 

dewpoint temperature was 0.2 to 0.3 higher for the triple-nest configuration and was up to 0.1 

higher through forecast hours 4 to 11 than the nested domain for MSLP. It is interesting to note 

that the highest PCC values for WFF were nearly double that of the ER. During the summer and 

fall, different systems drive the local weather at each range. There are more synoptic systems 

that influence the weather over the WFF during summer and fall while in the summer, the weather 

over the ER is locally driven by mesoscale phenomena. In general, models will perform better 

when the local weather is driven by larger-scale synoptic systems. Therefore, it is expected that 

the GSI/WRF system performed better for WFF than for the ER during the POR. 

Figure 14. Chart of the average hourly PCC for the 12-hour forecast over the entire POR for a) 

wind speed, b) temperature, c) dewpoint temperature, and d) MSLP from the two GSI/WRF 

configurations at WFF.

d

a b

c

25
3.4.2 Precipitation Forecasts

The AMU compared precipitation forecasts from both model configurations to determine 

performance differences. One-hour forecast accumulated rainfall for each hour of each of the 12-

hour forecasts was compared to the one-hour accumulation of observed rain using the NCEP

Stage-IV analysis data for each day during the POR. The POR summary statistics for each hour 

of the forecast of centroid distance, area ratio, and interest function from the MODE software are 

shown in Figure 15. The centroid distance (km) results indicate that forecasts from both 

configurations were very similar in their ability to match the location of the observed precipitation 

throughout the forecasts. However, the triple-nest configuration did slightly better in predicting the 

precipitation location. The area ratio results indicate that accumulated precipitation for the triplenest configuration matched the areal coverage of observed precipitation more closely, although 

the results were comparable. Total interest values were very similar for both configurations with 

values near 0.8. Overall, the triple nest configuration very slightly outperformed the nested

domain.

Figure 15. Chart of the average hourly a) centroid distance (km), b) area ratio, and c) total interest 

value from the two GSI/WRF configurations for each hour of the 12-hour forecast over the entire 

POR at WFF.

a b

c

26
4 Conclusions and Future Work

This report summarizes the findings from the AMU task to determine a recommended local 

DA and numerical forecast model design optimized for the ER and WFF to support space launch 

activities and for local weather forecasting challenges at each range. The AMU ran the GSI/WRF 

model system over part of the summer and fall seasons for each range while varying grid 

resolutions on which the DA was run and varying the nesting configurations to determine the 

impact on model skill. In general for both the ER and WFF, the triple-nest configuration

outperformed the other configurations. However, although the results for the ER indicate that the 

triple-nest configuration performed best for most variables as evidenced by the ME, RMSE, and 

PCC, the nested configuration did the best in predicting precipitation for the ER. Summertime 

convection over the ER is an important meteorological variable to predict and, for this reason, it 

is the AMU’s recommendation to use either the nested or triple-nest configuration as the optimal 

model configuration for the ER. The triple-nest configuration performed the best for nearly all 

variables at WFF. Wind, temperature, and convective activity forecasts during the fall and spring 

seasons pose the most difficulties for forecasters at WFF. Therefore, it is the AMU’s opinion that 

the triple-nested domain is the optimal model configuration for WFF. 

The AMU has been tasked with a follow-on modeling effort to:

x Port GSI/WRF code from the AMU developmental cluster to the AMU real-time cluster 

and to run every three hours,

x Display real-time output of the GSI/WRF cycled runs on the AMU’s AWIPS II 

workstations,

x Work with SPoRT to determine if the GSI/WRF system can be run in a rapid-refresh 

mode. If so, determine the time-frame to set up the rapid-refresh system and 

implement if possible within the time-frame of this task, and

x Explore ensemble modeling using the WRF model and determine the level of effort 

and time-frame to set up an ensemble modeling system.

This follow-on task will benefit operations by providing high spatial and time resolution forecasts 

from a model tuned specifically for their location. It will also continue to improve the local model 

by incorporating techniques that have been shown to enhance model performance. 

27
References

Davis, C., B. Brown, and R. Bullock, 2006: Object-based verification of precipitation forecasts. 

Part I: Methods and application to mesoscale rain areas. Mon. Wea. Rev., 134, 1772–1784.

Kleist, D. T., D. F. Parrish, J. C. Derber, R. Treadon, W. Wu, S. Lord, 2009: Introduction of the 

GSI into the NCEP Global Data Assimilation System. Wea. Forecasting, 24, 1691–1705.

Lin, Y. and K. E. Mitchell, 2005: The NCEP Stage II/IV hourly precipitation analyses: development 

and applications. Preprints, 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor. Soc.,

P1.2. [Available online at ams.confex.com/ams/Annual2005/techprogram/paper_83847.htm.]

Wang, Xuguang, 2010: Incorporating ensemble covariance in the Gridpoint Statistical 

Interpolation variational minimization: A mathematical framework. Mon. Wea. Rev., 138,

2990–2995.

Watson, L., 2013: Range-specific High-resolution Mesoscale Model Setup. NASA Contractor 

Report CR-2013-217911, Kennedy Space Center, FL, 41 pp. [Available from ENSCO, Inc., 

1980 N. Atlantic Ave., Suite 830, Cocoa Beach, FL, 32931 and online at 

http://science.ksc.nasa.gov/amu/final-reports/range-specific-hi-res-model-setup.pdf.]

28
List of Acronyms

45 WS 45th Weather Squadron

AMU Applied Meteorology Unit

ARW Advanced Research WRF

AWIPS Advanced Weather Interactive 

Processing System

CCAFS Cape Canaveral Air Force 

Station

DA Data Assimilation

ER Eastern Range

GSI Gridpoint Statistical 

Interpolation

KSC Kennedy Space Center

LIS Land Information System

MADIS Meteorological Assimilation 

Data Ingest System

ME Mean Error

MET Model Evaluation Tools

METAR Meteorological Aerodrome 

Report

MODE Method for Object-Based 

Diagnostic Evaluation

MSLP Mean Sea-Level Pressure

NAM North American Mesoscale

NCAR National Center for 

Atmospheric Research

NCEP National Centers for 

Environmental Prediction

NMM Non-hydrostatic Mesoscale 

Model

PBL Planetary Boundary Layer 

PCC Pearson Correlation Coefficient

POR Period of Record

RMSE Root Mean Square Error

RAP Rapid Refresh

SPoRT Short-term Prediction 

Research and Transition 

Center

SST Sea Surface Temperature

WFF Wallops Flight Facility

WRF Weather Research and 

Forecasting

YSU Yonsei University

29
NOTICE

Mention of a copyrighted, trademarked or proprietary product, service, or document does not 

constitute endorsement thereof by the author, ENSCO Inc., the AMU, the National Aeronautics 

and Space Administration, or the United States Government. Any such mention is solely for the 

purpose of fully informing the reader of the resources used to conduct the work reported herein.

30