NASA/CR—2014–218474 

 

Real-time Kennedy Space Center and Cape 

Canaveral Air Force Station High-resolution 

Model Implementation and Verification 

Jaclyn A. Shafer 

ENSCO, Inc., Cocoa Beach, Florida 

NASA Applied Meteorology Unit, Kennedy Space Center, Florida 

Leela R. Watson 

ENSCO, Inc., Cocoa Beach, Florida 

NASA Applied Meteorology Unit, Kennedy Space Center, Florida 

January 2015
NASA STI Program ... in Profile 

Since its founding, NASA has been dedicated 

to the advancement of aeronautics and space 

science. The NASA scientific and technical 

information (STI) program plays a key part in 

helping NASA maintain this important role. 

The NASA STI program operates under the 

auspices of the Agency Chief Information Officer. 

It collects, organizes, provides for archiving, and 

disseminates NASA’s STI. The NASA STI 

program provides access to the NTRS Registered 

and its public interface, the NASA Technical 

Reports Server, thus providing one of the largest 

collections of aeronautical and space science STI 

in the world. Results are published in both nonNASA channels and by NASA in the NASA STI 

Report Series, which includes the following report 

types: 

x TECHNICAL PUBLICATION. Reports of 

completed research or a major significant 

phase of research that present the results of 

NASA Programs and include extensive data 

or theoretical analysis. Includes compilations of significant scientific and technical 

data and information deemed to be of 

continuing reference value. NASA counterpart of peer-reviewed formal professional 

papers but has less stringent limitations on 

manuscript length and extent of graphic 

presentations. 

x TECHNICAL MEMORANDUM. 

Scientific and technical findings that are 

preliminary or of specialized interest, 

e.g., quick release reports, working 

papers, and bibliographies that contain 

minimal annotation. Does not contain 

extensive analysis. 

x CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees. 

x CONFERENCE PUBLICATION. 

Collected papers from scientific and 

technical conferences, symposia, seminars, 

or other meetings sponsored or 

co-sponsored by NASA. 

x SPECIAL PUBLICATION. Scientific, 

technical, or historical information from 

NASA programs, projects, and missions, 

often concerned with subjects having 

substantial public interest. 

x TECHNICAL TRANSLATION. 

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission.

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and 

feeds, providing information desk and personal 

search support, and enabling data exchange 

services. 

For more information about the NASA STI 

program, see the following: 

x Access the NASA STI program home page 

at http://www.sti.nasa.gov 

x E-mail your question to help@sti.nasa.gov 

x Phone the NASA STI Information Desk at 

757-864-9658 

x Write to: 

NASA STI Information Desk 

Mail Stop 148 

NASA Langley Research Center 

Hampton, VA 23681-2199 

This page is required and contains approved text that cannot be changed. 
NASA/CR-2014-218474 

Real-time Kennedy Space Center and Cape 

Canaveral Air Force Station High-resolution 

Model Implementation and Verification 

Jaclyn A. Shafer 

ENSCO, Inc., Cocoa Beach, Florida 

NASA Applied Meteorology Unit, Kennedy Space Center, Florida 

Leela R. Watson 

ENSCO, Inc., Cocoa Beach, Florida 

NASA Applied Meteorology Unit, Kennedy Space Center, Florida 

National Aeronautics and 

Space Administration

Kennedy Space Center 

Kennedy Space Center, FL 32899-0001 

January 2015
Acknowledgments 

The authors would like to thank Mr. Kevin McGrath and Dr. Geoffrey Stano of the Short-term 

Prediction Research and Transition Center and Mr. Erik Magnuson of ENSCO, Inc. for lending 

their time and expertise with the Advanced Weather Interactive Processing System II in order to 

get the real-time model output available in AWIPS II. Dr. Bill Bauman and Ms. Winnie Crawford 

also provided valuable feedback and assistance with completing this task. 

Available from: 

NASA Center for AeroSpace Information 

7115 Standard Drive 

Hanover, MD 21076-1320 

443-757-5802 

This report is also available in electronic form at 

http://science.ksc.nasa.gov/amu
3 

Executive Summary 

Customer: NASA’s Launch Services Program (LSP), Ground Systems Development and 

Operations (GSDO), and Space Launch System (SLS) programs 

NASA’s LSP, GSDO, SLS and other programs at Kennedy Space Center (KSC) and Cape 

Canaveral Air Force Station (CCAFS) use the daily and weekly weather forecasts issued by the 

45th Weather Squadron (45 WS) as decision tools for their day-to-day and launch operations on 

the Eastern Range (ER). For example, to determine if they need to limit activities such as vehicle 

transport to the launch pad, protect people, structures or exposed launch vehicles given a threat 

of severe weather, or reschedule other critical operations. The 45 WS uses numerical weather 

prediction models as a guide for these weather forecasts, particularly the Air Force Weather 

Agency (AFWA) 1.67 km Weather Research and Forecasting (WRF) model. 

Considering the 45 WS forecasters’ and Launch Weather Officers’ (LWO) extensive use of 

the AFWA model, the 45 WS proposed a task at the September 2013 Applied Meteorology Unit 

(AMU) Tasking Meeting requesting the AMU verify this model. Due to the lack of archived model 

data available from AFWA, verification is not yet possible. Instead, the AMU proposed to 

implement and verify the performance of an ER version of the AMU high-resolution WRF 

Environmental Modeling System (EMS) model (Watson 2013) in real-time. The tasking group 

agreed to this proposal; therefore the AMU implemented the WRF-EMS model on the second of 

two NASA AMU modeling clusters. 

The model was set up with a triple-nested grid configuration over KSC/CCAFS based on 

previous AMU work (Watson 2013). The outer domain (D01) has 12-km grid spacing, the middle 

domain (D02) has 4-km grid spacing, and the inner domain (D03) has 1.33-km grid spacing. The 

model runs a 12-hr forecast every hour, D01 and D02 domain outputs are available once an hour 

and D03 is every 15 minutes during the forecast period. 

The AMU assessed the WRF-EMS 1.33-km domain model performance for the 2014 warm 

season (May–September). Verification statistics were computed using the Model Evaluation 

Tools, which compared the model forecasts to observations. The observational datasets included 

Meteorological Assimilation Data Ingest System surface observations and Stage IV gridded 

precipitation data from the National Centers for Environmental Prediction. The mean error values 

were close to 0 and the root mean square error values were less than 1.8 for mean sea-level 

pressure (mb), temperature (K), dewpoint temperature (K), and wind speed (ms-1), all very small 

differences between the forecast and observations considering the normal magnitudes of the 

parameters. The precipitation forecast verification results showed consistent under-forecasting of 

the precipitation object size. This could be an artifact of calculating the statistics for each hour 

rather than for the entire 12-hour period. The AMU will continue to generate verification statistics 

for the 1.33-km WRF-EMS domain as data become available in future cool and warm seasons. 

More data will produce more robust statistics and reveal a more accurate assessment of model 

performance. 

In addition to verifying the model’s performance, the AMU also made the output available in 

the Advanced Weather Interactive Processing System II (AWIPS II). This allows the 45 WS and 

AMU staff to customize the model output display on the AMU and Range Weather Operations 

AWIPS II client computers and conduct real-time subjective analyses. In the future, the AMU will 

implement an updated version of the WRF-EMS model that incorporates local data assimilation. 

This model will also run in real-time and be made available in AWIPS II. 
4

Table of Contents 

Executive Summary ...................................................................................................................... 3 

Table of Contents .......................................................................................................................... 4 

List of Figures ............................................................................................................................... 5 

1 Introduction ............................................................................................................................ 7 

2 Model Installation and Configuration ..................................................................................... 8 

3 Model Forecast Verification ................................................................................................. 10 

3.1 Observational Data ....................................................................................................... 10 

3.1.1 MADIS ................................................................................................................... 10 

3.1.2 Stage IV ................................................................................................................. 11 

3.2 Verification Software ..................................................................................................... 11 

3.2.1 Point-Stat Tool ....................................................................................................... 12 

3.2.2 MODE Tool ............................................................................................................ 12

3.3 Warm Season Verification Results ............................................................................... 12 

3.3.1 Surface Parameters .............................................................................................. 12 

3.3.2 Precipitation ........................................................................................................... 15 

3.3.3 Summary of Results .............................................................................................. 19 

4 WRF-EMS Output into AWIPS II ......................................................................................... 20 

4.1 CAVE Examples ........................................................................................................... 20 

4.2 Procedures to Display WRF-EMS Output .................................................................... 22 

5 Summary and Future Work .................................................................................................. 25 

References .................................................................................................................................. 26 

List of Acronyms ......................................................................................................................... 27
5 

List of Figures 

Figure 1. Map of the eastern United States showing the boundaries of each domain. The outer 

domain (blue rectangle, D01) has 12-km grid spacing, the middle domain (green 

rectangle, D02) has 4-km grid spacing, and the inner domain (yellow rectangle, D03) 

has 1.33-km grid spacing. The AMU calculated verification statistics for the inner-most 

domain, D03. ............................................................................................................... 9 

Figure 2. Map of the mesonet (blue squares) and METAR (blue squares with red circles) 

weather station locations from MADIS. The AMU used the data from these sites to 

calculate verification statistics for the inner-most WRF-EMS domain. ...................... 11 

Figure 3. The ME for each parameter versus forecast hour. Surface pressure is in mb (blue 

dots), temperature in K (green dots), dewpoint temperature in K (red dots) and wind 

speed in ms-1 (purple dots)........................................................................................ 13 

Figure 4. The ME for wind direction in degrees versus forecast hour. ..................................... 14 

Figure 5. Same as Figure 3 but for RMSE. .............................................................................. 15 

Figure 6. Same as Figure 4 but for RMSE. .............................................................................. 15 

Figure 7. Illustration of the technique used in the MODE tool to define precipitation objects: a) 

raw gridded precipitation data, b) smoothed data, c) result from application of 

thresholds, d) final field of objects used in verification statistics (from Brown et al. 2007, 

Figure 2). ................................................................................................................... 16 

Figure 8. Centroid distance versus model forecast hour for the warm season model verification. 

Centroid distances are in number of grid boxes. ....................................................... 17 

Figure 9. Area ratio versus model forecast hour for the warm season model verification. ....... 18 

Figure 10. Interest value versus model forecast hour for the warm season model verification.

 .............................................................................................................................. 18 

Figure 11. CAVE screen shot of the AMU WRF-EMS 12-km frontogenesis output valid at 1900 

UTC on 10 July 2014. The warm colors are frontogenesis and the cool colors are 

frontolysis. ............................................................................................................. 20 

Figure 12. Same as Figure 11 but for the 4-km domain. ........................................................ 21 

Figure 13. Same as Figure 11 but for the 1.33-km domain. ................................................... 21 

Figure 14. CAVE window highlighting how the user selects the Volume Browser to display 

model data. ............................................................................................................ 22 

Figure 15. Volume Browser window in CAVE. The AMU WRF-EMS is selected by clicking the 

Volume drop-down menu under Sources. ............................................................. 23 

Figure 16. A list of available models from the Volume drop-down showing the three AMU WRFEMS domains at the top of the list. ........................................................................ 23 

Figure 17. Volume Browser showing the selection of AMU WRF-EMS parameters and 

associated available products. .............................................................................. 24 

Figure 18. AMU WRF-EMS 12-km forecast surface wind barbs (green). ............................... 24 

Figure 19. Zoomed in image of the product details shown in Figure 18. The red box outlines 

the product name, the blue box is the date and model run time (Z), the purple box 

is the forecast hour and the green box is the valid date and time. ........................ 25 
6

List of Tables 

Table 1. List of statistics available in the MODE Tool the AMU used to verify the model. ................. 16

 
7 

1 Introduction 

NASA’s Launch Services Program, Ground Systems Development and Operations, Space 

Launch System and other programs at Kennedy Space Center (KSC) and Cape Canaveral Air 

Force Station (CCAFS) use the daily and weekly weather forecasts issued by the 45th Weather 

Squadron (45 WS) as decision tools for their day-to-day and launch operations on the Eastern 

Range (ER). Examples include determining if they need to limit activities such as vehicle transport 

to the launch pad, protect people, structures or exposed launch vehicles given a threat of severe 

weather, or reschedule other critical operations. The 45 WS uses numerical weather prediction 

models as a guide for these weather forecasts, particularly the Air Force Weather Agency (AFWA) 

1.67 km Weather Research and Forecasting (WRF) model. 

Considering the 45 WS forecasters’ and Launch Weather Officers’ (LWO) extensive use of 

the AFWA model, the 45 WS proposed a task at the September 2013 Applied Meteorology Unit 

(AMU) Tasking Meeting requesting the AMU verify this model. Due to the lack of archived model 

data available from AFWA, verification is not yet possible. Instead, the AMU proposed to 

implement and verify the performance of an ER version of the high-resolution WRF Environmental 

Modeling System (EMS) model configured by the AMU (Watson 2013) in real time. Implementing 

a real-time version of the ER WRF-EMS would generate a larger database of model output than 

in the previous AMU task for determining model performance, and allows the AMU more control 

over and access to the model output archive. 

The tasking group agreed to this proposal; therefore the AMU implemented the WRF-EMS 

model on the second of two NASA AMU modeling clusters. The AMU also calculated verification 

statistics to determine model performance compared to observational data. Finally, the AMU 

made the model output available on the AMU Advanced Weather Interactive Processing System 

II (AWIPS II) servers, which allows the 45 WS and AMU staff to customize the model output 

display on the AMU and Range Weather Operations (RWO) AWIPS II client computers and 

conduct real-time subjective analyses. 
8

2 Model Installation and Configuration 

The first part of this task was to set up the model to run in real time. Once the NASA AMU 

modeling cluster was configured for AMU use, the AMU installed and configured WRF-EMS, and 

completed a few test runs. After confirming WRF-EMS was running properly, the AMU set up a 

triple-nested grid configuration over KSC/CCAFS based on a previous AMU task (Watson 2013). 

The results of that work showed the best configuration for the ER used the Advanced Research 

WRF core with the Lin microphysical scheme and the Yonsei University planetary boundary layer 

scheme. Once the model setup was configured and tested, the AMU automated WRF-EMS to run 

a 12-hr forecast every hour. The model is initialized every hour using the National Centers for 

Environmental Prediction’s (NCEP) 13-km Rapid Refresh model for boundary and initial 

conditions, Short-term Prediction Research and Transition Center (SPoRT) Land Information 

System land surface data, and SPoRT sea surface temperature data. 

Figure 1 shows the boundaries of the three domains included in this task. D01 is the outer 

domain with 12-km grid spacing, D02 is the middle domain with 4-km grid spacing, and D03 is 

the inner domain with 1.33-km grid spacing. D01 and D02 domain output is available once an 

hour and D03 is available every 15 minutes during the 12-hr forecast period. The boundaries of 

D01, which include much of the eastern United States, were selected should this work grow to 

include Wallops Flight Facility in the future. D03 is centered over the ER and is the domain for 

which the AMU calculated verification statistics to determine model performance. 
9 

Figure 1. Map of the eastern United States showing the boundaries of 

each domain. The outer domain (cyan rectangle, D01) has 12-km grid 

spacing, the middle domain (green rectangle, D02) has 4-km grid spacing, 

and the inner domain (yellow rectangle, D03) has 1.33-km grid spacing. The 

AMU calculated verification statistics for the inner domain, D03. 
10

3 Model Forecast Verification 

The AMU assessed model performance by computing statistics that compared the model 

forecasts to observations. The mean error (ME) and root mean square error (RMSE) were 

calculated for the surface parameter forecasts, and precipitation forecasts were compared to 

nationally available rainfall data using a technique developed at the National Center for 

Atmospheric Research (NCAR). 

3.1 Observational Data 

In order to determine the WRF-EMS model performance, the AMU required surface weather 

observations of temperature (T), dewpoint temperature (Td), wind speed and direction, and mean 

sea-level pressure (MSLP) as well as rainfall data. Based on the previous AMU model verification 

work (Watson 2013), the AMU used the NCEP Meteorological Assimilation Data Ingest System 

(MADIS) and Stage IV precipitation data for the observational datasets. 

3.1.1 MADIS 

The AMU coordinated with NCEP and set up a data connection to the NASA AMU cluster. 

With this connection the AMU automatically receives MADIS observational data using the local 

data manager software in real time. MADIS includes multiple data types including METAR and 

mesonet files. METAR is the international standard code format for hourly surface weather 

observations. Mesonet refers to a network of automated weather stations designed to observe 

mesoscale meteorological phenomena and report conditions in time intervals anywhere from 1 to 

15 minutes. These data were used to verify hourly 2-m T (K), Td (K), 10-m wind speed (ms-1) and 

direction (degrees), and MSLP (mb). Figure 2 shows the locations of the METAR and mesonet 

weather stations (https://madis-data.noaa.gov/sfc_display/) that were used to verify the 

performance of the WRF-EMS inner domain. 
11 

Figure 2. Map of the mesonet (blue squares) and METAR 

(blue squares with red circles) weather station locations from 

MADIS. The AMU used the data from these sites to calculate 

verification statistics for the inner WRF-EMS domain. 

3.1.2 Stage IV 

In addition to the MADIS data, the AMU used gridded NCEP Stage IV precipitation data 

(http://www.emc.ncep.noaa.gov/mmb/ylin/pcpanl/stage4/) to verify the hourly WRF-EMS 

precipitation forecasts. The Stage IV data combines radar data and rain gauge reports to produce 

hourly rainfall accumulation on a 4-km grid. It is a manually quality-controlled mosaic from the 

regional 1-hr precipitation analyses produced by 12 National Weather Service Forecast Centers 

(Lin and Mitchell 2005). 

3.2 Verification Software 

The AMU calculated verification statistics to determine the WRF-EMS model performance 

using the Model Evaluation Tools (MET) software. MET was developed by the NCAR 

Developmental Testbed Center through the support of AFWA and the National Oceanic and 

Atmospheric Administration. It was designed to be a highly configurable, state-of-the-art suite of 

verification tools (MET User’s Guide 2013). The AMU used two of the statistical verification tools 

available in MET for this task: the Point-Stat tool and the Method for Object-Based Diagnostic 

Evaluation (MODE) tool. 
12

3.2.1 Point-Stat Tool 

The Point-Stat tool was used to compute traditional verification scores for the hourly surface 

forecasts including 2-m T (K), Td (K), 10-m wind speed (ms-1) and direction (degree), and MSLP 

(mb). It compares the WRF-EMS forecast to the corresponding MADIS point observations. This 

tool was run on each hourly forecast and consolidated to determine the overall model 

performance. 

3.2.2 MODE Tool 

The MODE tool applies an object-based verification technique in comparing a gridded forecast 

to a gridded analysis. MODE was used to compare the WRF-EMS precipitation accumulation 

forecasts to the NCEP Stage IV observations. 

In order to use MODE for verification, the timing and grid spacing of the forecast must match 

observational data. Since the WRF-EMS produces precipitation forecasts every 15 minutes on a 

1.33-km grid and the Stage IV data is available hourly on a 4-km grid, the AMU had to reformat 

the WRF-EMS output to match the observations. Each of the 15-minute WRF-EMS files were 

converted from 1.33-km to a 4-km grid and then combined to create a 1-hr 4-km WRF-EMS 

precipitation accumulation forecast file. The MODE tool ingested these new files and compared 

them to the Stage IV data for verification. This process was repeated for every forecast hour. 

3.3 Warm Season Verification Results 

The AMU calculated verification statistics to determine the WRF-EMS 1.33-km domain model 

performance for the 2014 warm season (May–September) using the tools described in section 

3.2. 

3.3.1 Surface Parameters 

The Point-Stat tool was used to compute the ME and RMSE for the hourly 2-m MSLP, T, Td, 

and 10-m wind speed and direction. The ME is the overall bias of the model parameter during the 

period of interest, calculated by subtracting the observation from the forecast and averaging the 

differences. It determines whether there is a positive, negative, or no bias in the model forecast 

for any parameter. ME values range from negative infinity to infinity with a perfect score equal to 

0. 

¦  

 

n

i

i oi f

n

ME

1

1

where: 

n = number of model output times and/or vertical levels over the forecast period,

fi = WRF forecast of MSLP, T, Td, wind speed, or wind direction, and 

oi = observed MSLP, T, Td, wind speed, or wind direction. 

The RMSE isolates the magnitude of the model error as ME can mask the variance in the 

differences between the forecast and observations. An ME close to 0 could be the result of 

averaging larger negative and positive differences. The RMSE equation squares the differences, 

calculates an average of the squared differences, then calculates the square root of the average 

value. As with ME, smaller RMSE values indicate better model performance. It ranges from 0 to 

infinity with a perfect score equal to 0. 
13 

ඩͳ ൌ ܧܵܯܴ

݊෍ሺ݂௜ െ ݋௜ሻଶ

௡

௜ୀଵ

where n, fi, and oi are defined as above. 

Figure 3 shows the ME for MSLP, T, Td, and wind speed versus model forecast hour for the 

2014 warm season. The ME curve for T is close to 0 at all forecast hours, indicating a good 

forecast for this parameter. The ME values for wind speed and Td are within ±0.4 with wind speed 

showing a small positive bias and Td showing a slightly negative, or dry, bias. The values for 

MSLP are all negative ranging from -1.2 to -1.4. Considering the magnitude of MSLP values on 

the order of 1000 mb, these biases are very small in comparison. Figure 4 is the same but for 

wind direction. These values increase with forecast hour from just above 25 to 35 degrees. 

Figure 3. The ME for each parameter versus forecast hour. Surface pressure is 

in mb (blue dots), temperature in K (green dots), dewpoint temperature in K (red 

dots) and wind speed in ms-1 (purple dots). 
14

Figure 4. The ME for wind direction in degrees versus forecast hour. 

FiguresFigure 5 andFigure 6 are the same as FiguresFigure 3 andFigure 4 respectively, but 

for RMSE. The RMSE values for MSLP, T, Td and wind speed slightly increase with forecast hour 

but remain between 1 and 1.8 (Figure 5). These values are close to 0 which confirms the 

magnitude of the model error for these parameters is very small. A similar trend exists for wind 

direction but the RMSE values range from about 40 to 50 degrees. These values could be 

because of light and variable winds that are common at night over KSC/CCAFS during the warm 

season. When winds are light there tends to be a large variance in the wind direction that would 

be difficult for the model to forecast. In the future, the AMU will stratify the warm season results 

diurnally to determine how the time of day influences the model wind direction forecasts. 
15 

Figure 5. Same as Figure 3 but for RMSE. 

Figure 6. Same as Figure 4 but for RMSE. 

3.3.2 Precipitation 

The MODE tool was used to verify the model precipitation forecasts. This tool applies an 

object-based verification technique to compare a gridded forecast to a gridded analysis. The 

technique for defining objects in MODE is illustrated in Figure 7. Figure 7a is an example of raw 

gridded data. MODE uses two processes to convert raw gridded precipitation values into 

precipitation objects. The first step is to smooth the data (Figure 7b) and the second is to create 

a convolved field (Figure 7c). Once these steps are complete, MODE has defined final 

precipitation objects (Figure 7d) that are used in the verification statistics. A more detailed 

explanation about this process and MODE is in the MET User’s Guide, chapter 8 (2013). Table 1 

shows the statistics the AMU used from MODE for the model verification. 
16

Figure 7. Illustration of the technique used in the 

MODE tool to define precipitation objects: a) raw 

gridded precipitation data, b) smoothed data, c) 

convolved field, d) final field of objects used in 

verification statistics (from the MET users guide, 

Figure 8-1). 

Table 1. List of statistics available in the MODE tool the AMU used to verify the model. 

Statistic Name Description 

Centroid Distance Distance between two object centroids (in grid units) 

Area Ratio Ratio of the areas of two objects defined as the lesser of the forecast 

area divided by the observation area or its reciprocal (unitless) 

Interest Total interest value computed for a pair of simple objects (unitless) 
17 

The centroid distance is the distance between the centers of two objects: the observed 

precipitation object and the corresponding forecast precipitation object. A perfect forecast would 

have a centroid distance equal to 0. Figure 8 shows the centroid distance versus model forecast 

hour for the warm season verification. As expected, there is a general increase in distance with 

time although it remains between 35 and 38 grid boxes. The distance in km is the centroid 

distance in grid boxes multiplied by the domain resolution of 1.33-km, resulting in distances 

between 46.55 and 50.54 km. These location differences may be present because warm season 

convection remains one of the most poorly forecast meteorological parameters, in part due to 

dynamic and thermodynamic features that occur on the mesoscale (Watson 2007). Note that 

MODE does not calculate statistics for the initialization time (Forecast Hour = 0) since the model 

takes time to spin-up the precipitation forecasts. 

Figure 8. Centroid distance versus model forecast hour for the warm season 

model verification. Centroid distances are in number of grid boxes. 

The area ratio compares the size of the forecast objects to the observation objects (forecast 

area divided by observation area). A perfect forecast would have an area ratio equal to 1. A value 

greater than 1 indicates the model is forecasting larger objects than observed, and a value less 

than one indicates the model is forecasting smaller objects than observed. Figure 9 shows the 

area ratio versus model forecast hour remaining steady at about 0.35. This means the model is 

consistently under-forecasting the size of the precipitation objects. 
18

Figure 9. Area ratio versus model forecast hour for the warm season model 

verification. 

Finally, the interest value compares the differences in attributes between the forecast and 

observed objects, including the centroid distance and area ratio, and gives an indication of the 

overall quality of the model precipitation forecasts. It ranges from 0 to 1 with a perfect score equal 

to 1. More detail about these statistics can be found in Brown et al. 2007. Figure 10 shows the 

interest versus model forecast hour. These values consistently remain just above 0.6 regardless 

of model forecast hour. 

Figure 10. Interest value versus model forecast hour for the warm season model 

verification. 
19 

3.3.3 Summary of Results 

The ME values were close to 0 and the RMSE values were less than 1.8 for MSLP, T, Td, and 

wind speed, all very small differences between the forecast and observations considering the 

normal magnitudes of the parameters. The RMSE for wind direction was between 40 and 50 

degrees which could be because of the light and variable winds that are common at night over 

KSC/CCAFS during the warm season. In the future, the AMU will stratify the results diurnally to 

determine how the time of day influences the model wind direction forecasts. The precipitation 

forecast verification results showed consistent under-forecasting of the precipitation object size. 

This could be an artifact of calculating the statistics for each hour rather than for the entire 12-

hour period. For example, if the timing of the model-forecast precipitation occurs within the hour 

before or after the observed precipitation, then the forecast will not match the observation. The 

model may also correctly forecast the location of the precipitation but the statistics could still show 

poor model performance due to timing. The AMU will continue to generate verification statistics 

for the 1.33-km WRF-EMS domain as data become available in future cool and warm seasons. 

More data will produce more robust statistics and reveal a more accurate assessment of model 

performance. 

 
20

4 WRF-EMS Output into AWIPS II 

In addition to verifying the model’s performance, the AMU also made the output available in 

AWIPS II. This allows the 45 WS and AMU staff to customize the model output display on the 

AMU and RWO AWIPS II client computers and conduct real-time subjective analyses. The AMU 

was able to customize the WRF-EMS runs and ingest the three WRF-EMS domains separately 

in order to display them individually in AWIPS II via the Common AWIPS Visualization 

Environment (CAVE). 

4.1 CAVE Examples 

Example CAVE screen shots of the 12-, 4- and 1.33-km AMU WRF-EMS model frontogenesis 

forecast product are shown in Figures Figure 11, Figure 12 andFigure 13, respectively. The AMU 

wrote Perl scripts that automate the ingest process and update the model in AWIPS II every hour. 

Figure 11. CAVE screen shot of the AMU WRF-EMS 12-km frontogenesis output valid at 

1900 UTC on 10 July 2014. The warm colors are frontogenesis and the cool colors are 

frontolysis. 
21 

Figure 12. Same as Figure 11 but for the 4-km domain. 

Figure 13. Same as Figure 11 but for the 1.33-km domain. 
22

4.2 Procedures to Display WRF-EMS Output 

Like all other model data available in AWIPS II, the AMU WRF-EMS is accessed via the CAVE 

Volume Browser. To open the Volume Browser, the user must select Volume and then Browser 

in the CAVE window (Figure 14). This will open the Volume Browser window. To access the AMU 

WRF-EMS forecasts, click the Volume drop-down list under Sources as indicated in Figure 15. A 

partial list of the model forecasts available via the Volume drop-down menu is shown in Figure 

16. The three AMU WRF-EMS domains are at the top of the list. The user must then select the 

desired Field and Plane, for example Wind and Surface respectively, which will list the available 

products under Product Selection List (Figure 17). The product is loaded by either double clicking 

the item in the Product Selection List, or clicking the Load button. 

Figure 14. CAVE window highlighting how the user selects the Volume Browser to display 

model data. 
23 

Figure 15. Volume Browser window in CAVE. The AMU WRF-EMS is selected by clicking the 

Volume drop-down menu under Sources. 

Figure 16. A partial 

list of available 

models from the 

Volume drop-down 

showing the three 

AMU WRF-EMS 

domains at the top 

of the list. 
24

Figure 17. Volume Browser showing the selection of AMU WRF-EMS parameters and 

associated available products. 

Figure 18 shows the products in the main CAVE window based on the selections highlighted 

in Figure 17. The surface wind barbs within the 12-km, outer-domain are shown in green. The 

bottom right corner of the window shows the loaded product details. Figure 19 is a zoomed in 

image of the details. The red box outlines the product name, in this example it is the AMU WRFEMS 12-km surface winds. Next, the blue box shows the date and time (Z) the model was run. 

The third box (purple) is the forecast hour (0–12 hours). Finally, the green box shows the product’s 

valid date and time. 

Figure 18. AMU WRF-EMS 12-km forecast surface wind barbs (green). 
25 

Figure 19. Zoomed in image of the product details shown in Figure 18. The red box outlines the 

product name, the blue box is the date and model run time (Z), the purple box is the forecast 

hour and the green box is the valid date and time. 

5 Summary and Future Work 

Given the extensive use of the AFWA 1.67 WRF model in operations, the 45 WS proposed a 

task at the September 2013 AMU Tasking Meeting requesting the AMU verify this model. Since 

there is a lack of archived model data available from AFWA, verification is not yet possible. 

Therefore, the AMU proposed to implement and verify the performance of an ER version of the 

AMU high-resolution WRF-EMS model in real-time. The tasking group agreed to this proposal 

and therefore the AMU implemented the WRF-EMS model on the second of two NASA AMU 

modeling clusters to begin producing model output for the verification. The model was set up with 

a triple-nested grid configuration over KSC/CCAFS based on a previous AMU task (Watson 

2013). 

The AMU assessed the WRF-EMS 1.33-km model performance for the 2014 warm season. 

Verification statistics were computed using the MET software, which compared the model 

forecasts to observations. The observational datasets included MADIS surface observations and 

Stage IV gridded precipitation data from NCEP. The ME values were close to 0 and the RMSE 

values were less than 1.8 for MSLP, T, Td, and wind speed, all very small differences between 

the forecast and observations considering the normal magnitudes of the parameters. The RMSE 

for wind direction was between 40 and 50 degrees which could be because of the light and 

variable winds that are common at night over KSC/CCAFS during the warm season. In the future, 

the AMU will stratify the results diurnally to determine how the time of day influences the model 

wind direction forecasts. The precipitation forecast verification results showed consistent underforecasting of the precipitation object size. This could be an artifact of calculating the statistics for 

each hour rather than for the entire 12-hour period. The AMU will continue to generate verification 

statistics for the 1.33-km WRF-EMS domain as data become available in future cool and warm 

seasons. As a potential future task, the AMU could conduct a more in-depth literature review and 

recalculate verification statistics once a larger database has been acquired. More data will 

produce more robust statistics and reveal a more accurate assessment of model performance. 

In addition to the verification, the AMU also made the output available in AWIPS II. This allows 

the 45 WS and AMU staff to customize the model output display on the AMU and RWO AWIPS 

II client computers and conduct real-time subjective analyses. The LWOs and Range Weather 

Forecasters have found the AMU-WRF model's performance quite useful over the entire summer 

months during 2014. It has frequently been the preferred model to help accurately identify 

complex, very small scale boundary interactions. Of particular note, its rapid hourly update 

capability allows for constant correction and fine-tuning which further aids the LWOs during 

sensitive ground and launch operations that require specific timing accuracy. In the future, the 

AMU will implement an updated version of the WRF-EMS model that incorporates local data 

assimilation based on a recently completed AMU task (Watson 2014).This model will also run in 

real-time and be made available in AWIPS II. 

 
26

References 

Brown, G. B., R. Bullock, J. H. Gotway, D. Ahijevych, C. Davis, E. Gilleland, and L. Holland, 2007: 

Application of the MODE object-based verification for the evaluation of model precipitation 

fields. Preprints, 22nd Conf. on Weather Analysis and Forecasting/18th Conf. on Numerical 

Weather Prediction, Park City, UT, Amer. Meteor. Soc., 10.2A. [Available online at 

https://ams.confex.com/ams/pdfpapers/124856.pdf] 

Developmental Testbed Center, 2013: Model Evaluation Tools Version 4.1 User’s Guide 4.1. 

[Available online at 

http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v4.1.pdf] 

Lin, Y., and K. E. Mitchell, 2005: The NCEP Stage II/IV hourly precipitation analyses: development 

and applications. Preprints, 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor. Soc., 

1.2. 

[Available online at https://ams.confex.com/ams/Annual2005/techprogram/paper_83847.htm] 

Watson, L., 2007: Weather Research and Forecasting Model Sensitivity Comparisons for Warm 

Season Convective Initiation. NASA Contractor Report CR-2007-214734, Kennedy Space 

Center, FL, 41 pp. [Available from ENSCO, Inc., 1980 N. Atlantic Ave., Suite 830, Cocoa 

Beach, FL 32931 and online at http://science.ksc.nasa.gov/amu/final-reports/wrf-sensitivitystudy.pdf.] 

Watson, L., 2013: Range-specific High-resolution Mesoscale Model Setup. NASA Contractor 

Report CR-2013-217911, Kennedy Space Center, FL, 41 pp. [Available from ENSCO, Inc., 

1980 N. Atlantic Ave., Suite 830, Cocoa Beach, FL 32931 and online at 

http://science.ksc.nasa.gov/amu/final-reports/range-specific-hi-res-model-setup.pdf.] 

Watson, L., 2014: Range-specific High-resolution Mesoscale Model Setup: Data Assimilation. 

NASA Contractor Report CR-2014-218439, Kennedy Space Center, FL, 34 pp. [Available 

from ENSCO, Inc., 1980 N. Atlantic Ave., Suite 830, Cocoa Beach, FL 32931 and online at 

http://science.ksc.nasa.gov/amu/final-reports/range-specific-hi-res-DA-model-setup.pdf] 
27 

List of Acronyms

45 WS 45th Weather Squadron 

AFWA Air Force Weather Agency 

AMU Applied Meteorology Unit 

AWIPS Advanced Weather Interactive Processing System 

CAVE Common AWIPS Visualization Environment 

CCAFS Cape Canaveral Air Force Station 

EMS Environmental Modeling System 

ER Eastern Range 

GSDO Ground Systems Development and Operations 

KSC Kennedy Space Center 

LSP Launch Services Program 

LWO Launch Weather Officer 

MADIS Meteorological Assimilation Data Ingest System 

ME Mean Error 

MET Model Evaluation Tools 

MODE Method for Object-Based Diagnostic Evaluation 

MSLP Mean Sea Level Pressure 

NCAR National Center for Atmospheric Research 

NCEP National Centers for Environmental Prediction 

RAP Rapid Refresh 

RH Relative Humidity 

RMSE Root Mean Square Error 

RWO Range Weather Operations 

SLS Space Launch System 

SPoRT Short-term Prediction Research and Transition Center 

T Temperature 

Td Dewpoint Temperature 

WDIR Wind Direction 

WRF Weather Research and Forecasting 

 
28

NOTICE 

Mention of a copyrighted, trademarked or proprietary product, service, or document does not 

constitute endorsement thereof by the author, ENSCO Inc., the AMU, the National Aeronautics 

and Space Administration, or the United States Government. Any such mention is solely for the 

purpose of fully informing the reader of the resources used to conduct the work reported herein.