# Presentation: 760282

## Operational Deployment of Datasets

**ATS 690**
**Jason Burks**
**Holly Allen**

## Introduction

**The objective is to show the steps taken to migrate a dataset from a research facility to a weather service operational environment. **
**Knowledge gained will be beneficial to both the NWS and NASA SPoRT group. **

## Steps Needed

- Identifying Dataset for Operational Use
- Display and Data Type Implementation Considerations
- Setup of Data Ingest
- Data Reliability
- Testing of Data
- Training for Use of Dataset
- Use and Evaluation of the Dataset
- Application of the Process to Multiple Offices

## Identify Dataset for Operational Use

- Finding Datasets
- Dropped in lap
- Have a Problem? Find a solution with a new dataset
- File Size is Considered
- How much bandwidth will it require?
- Bandwidth creates ingest limitations
- EX: ADAS datasets were pared down because of their size.
- How much data can AWIPS hold?

## Display and Data Type Implementation Considerations

- How to Display dataset in AWIPS?
- What kind of data is it?
- Categorized in terms of how the data will be displayed
- Text, Point (obs, profiler), Satellite, RADAR, Gridded (model, LMA)
- Does the data make sense? If so,...
- Create a color curve, contouring intervals, temporal resolution, projection information, etc.
- Add data to menus and volume browser.
- Becomes an experimental dataset.
- If proven needed, then determine how to get the data.

## Setup of Data Ingest

- How do we get the data?
- FTP - push the data to a FTP site and go get it.
- LDM - places data in queue. Goes to whoever requests it.
- From the LDAD:

**AWIPS**

**LDAD**

**Firewall**

**Regional HQ**

**/Data/Incoming Directory**

**Moved into a local file, where it is stored. Purged off AWIPS. **

**A script runs from AWIPS to get files from the incoming directory on LDAD. **

## Data Reliability

- Once the data is ingested and displayable, is it reliable?
- Questions to ask in order to determine if its ready for the next step:
- Is it timely?
- Does the data look realistic?
- High quantity of erroneous data?
- Is data consistently available?

## Testing of Data

- It is hard to overcome a ‘tainted mindset’ with forecasters when they are given a bad dataset.
- Datasets should be tested and quality controlled before they are viewed by forecasters.
- Normally, for a dataset to be implemented, an advocate for that dataset is needed.
- The advocate is not a defined person, but usually someone who takes an interest in the new data, and encourages others to look at and use it.

## Training for Use of Dataset

- Normally is done by the Advocate or the Researcher that supplied the dataset.
- Researchers might not know how the dataset will be applied operationally, so contact with the forecaster(s) is necessary.
- Contact with the Researcher is maintained in order to provide information about data outages, server problems, etc.

## Use and Evaluation of the Dataset

- Qualitative
- Immediate feedback (i.e., surveys)
- Useful in identifying case studies
- Quantitative
- Case study analysis.
- Shows impact of data:
- Was lead time or false alarm ratio improved?
- In order to bring a new dataset into full operational use, there must be hard facts and numbers to back up the utility of the dataset.
- Not: “Oh, they thought it was great.”

## Application of the Process to Multiple Offices

- Color Curves
- Better way to transfer between offices.
- Recommend for future AWIPS builds an export/import option.
- Compatibility issues between different servers.
- Sites with upgrades or builds vs. those not.
- Allocate data key for each product
- Details for each product is kept under that key.
- A way to apply data without getting messed up during each build process.
- The extent of the dataset; if it is too large, can make it difficult to send.
- After dataset is provided to new offices, information on how to use/modify/troubleshoot data must also be provided.
- Written documentation. Ex: If this occurs, then check here, etc.
- The process for implementation at the new office begins with training, whether it is done by someone in-house, someone from the previous office, or the initial research team.

## Summary

- Identifying Dataset for Operational Use
- Display and Data Type Implementation Considerations
- Setup of Data Ingest
- Data Reliability
- Testing of Data
- Training for Use of Dataset
- Use and Evaluation of the Dataset
- Application of the Process to Multiple Offices
- These steps should be done in order to provide a successful transfer from the research facility to operations.