# Presentation: 199622

## Intersite CoordinationPerformance and Enhancements

- Mark Mathewson

## Overview

- Items to help reduce processing requirements.
- Configuration Items
- Forecaster Methodology
- What is planned for future revisions?
- RPP ISC Focus Group

- ISC Performance and Enhancements

## ISC Internals

**ISC Internals**

- ISC Performance and Enhancements

## Spatial Mosaic

**Spatial Mosaic**

- ISC Performance and Enhancements

## Temporal Mosaic

**Temporal Mosaic**

- ISC Performance and Enhancements

## ISC Processing

- Consumes a lot of resources
- Shouldn’t be on the lx1 box
- Px processor plans
- Volume of traffic determines resources needed.
- Forecaster can control traffic volume
- Focal Points can also regulate volume through configuration

- ISC Performance and Enhancements

## Performance Characteristics

- Number of Grids
- Hourly grids, vs. longer grids
- Number of weather elements
- Number of Gridpoints per Grid
- 5km resolution – typically 25k points
- 2.5 km resolution – typically 100k points
- AWIPS H/W, S/W sized for 25k points

- ISC Performance and Enhancements

## Performance Characteristics

- Three Processing Steps:
- Creation of netCDF file for transmission
- MHS Sending over WAN
- Receive processing of netCDF file, remapping projection, and mosaic operations.
- Full set of grids – 100 million gridpoints, handled multiple times. Can cause delays of 20 minutes.

- ISC Performance and Enhancements

## Methods to send ISC Data

- Capabilities:
- Choice of SITE
- Choice of time range
- Choice of weather element
- Choice of OFFICIAL or FCST database

- ISC Performance and Enhancements

## Methods to send ISC Data

- Dialogs:
- Send Intersite Grids dialog
- Publish dialog
- Save dialog
- MB3 popup menu over grid block in GM
- Send on Auto Save (IFPS13)

- ISC Performance and Enhancements

## Send ISC Grids Dialog

- ISC Performance and Enhancements

## Publish Dialog

- ISC Performance and Enhancements

## Save Dialog

- ISC Performance and Enhancements

## Are you duplicating ISC sends?

- Options have been added per field request.
- Options used together can cause duplicate grids to be sent and received.
- Forecast Methodology important in determining the right method to use.
- Future s/w changes can alleviate problems.

- ISC Performance and Enhancements

## Sending Guidelines

- Don’t simply send all of your grids all the time. For example,
- Send when save
- Send Intersite Coordination Grids dialog
- Publish dialog
- Triplicate grid sends don’t help performance.

- ISC Performance and Enhancements

## Sending Guidelines

- Send grids as you do your work – basically a few at a time.
- Sending when Save is a good idea
- Auto Save and Sending
- Configure your system to:
- Preset the dialogs to only include certain times or elements

- ISC Performance and Enhancements

## Sending Guidelines

- Send single grids to single site when working out a coordination issue
- Quick delivery/processing of grid - < 1 minute

- ISC Performance and Enhancements

## Configurability (hidden)

- Server-Based (IFPS13)
- MAX_ISC_GRIDS_PER_TRANSACTION
- to optimize use of WAN
- ISC_NEVER_RECEIVE_PARMS
- to toss incoming data based on weather element name
- GFE-Based (IFPS12)
- ISC_neverSendParms
- List of weather element names to NEVER transmit
- ISC_sendLimitBeforeCurrentTime, ISC_sendLimitAfterCurrentTime
- Limit transmissions based on current time (0-7days)

- ISC Performance and Enhancements

## Configurability (GUI)

- SendISCTimes
- List of time periods on SendISC and ISC Discrepancy Dialogs
- SendISCGridsWhenSave, SendISCGridsWhenAutoSave, SendISCGridsWhenPublish
- Controls default button position on dialog
- Controls auto save automatic sends

- ISC Performance and Enhancements

## Configurability (GUI)

- ISC_Sites
- List of sites to which ISC data may be sent
- SendISCGridDialogInitialWEGroup
- Pre-selected set of weather element on SendISC Dialog

- ISC Performance and Enhancements

## ISC Enhancements

- ISC Performance and Enhancements

## ISC Focus Group Purpose

- Help define ISC capabilities
- Suggest changes to ISC
- Help prioritize changes
- Test prototype software with changes prior to AWIPS deployment
- First meeting was last Thursday.
- Ideas presented here are under discussion.

- ISC Performance and Enhancements

## Discrepancy Algorithm

- Highlights Grid Manager if discrepancy thresholds are exceeded
- All it takes is ONE grid point to exceed the threshold to highlight the grid in the grid manager.

- ISC Performance and Enhancements

## Discrepancy Algorithm

- Color code Grid Manager
- RED – YELLOW – GREEN
- Based on # points that exceed threshold
- Based on mean absoloute error
- Based on ???
- Add Discrepancy Tools for
- Vector Direction, Wx, and Discrete Fields

- ISC Performance and Enhancements

## Difference Grid

- Show_ISC_Grid smart tool

- Original Fcst/ISC Grids

- 83

- 56

- Fcst – ISC grid = TDiff

- ISC Performance and Enhancements

## Difference Grid

- All points outside the calculated area are set to 0.
- Could expand that with the ISC-Fcst algorithm
- All points are set to 0, if the threshold is not exceeded anywhere
- Even if there is some discrepancies.

- ISC Performance and Enhancements

## Multiple ISC Databases

- Two ISC Databases – Fcst and Official
- Purpose:
- Separate out the official ISC grids from the Fcst ISC grids
- Good for product generation (to get official data only)
- Issues:
- Performance

- ISC

- ISC Official

- ISC Grid

- ISC Performance and Enhancements

## Show ISC Mode

- Current problem with QPF/SnowAmt
- If your Fcst grid is 12hr long, and you have ISC grids that are only a few hours long, you will be comparing QPFs for different time periods.
- Possible solution:
- Virtual weather elements (hidden), but will sum up the data to compare “apples with apples”.

- ISC Performance and Enhancements

## Show ISC Mode

- ISC Performance and Enhancements

## Show ISC Mode

- Fcst Grid

**0.70”**

***Comparing “apples to apples”***

- ISC QPF

- ISC Grid

- ISC Grid

- ISC Grid

**0.20”**

**0.30”**

**0.40”**

- You fcst 0.70”,
- So do your surrounding
- WFOs.

- ISC Performance and Enhancements

## Virtual ISC Grids

- Transparent to user
- History Update Times?
- What to do, use the oldest of the times, or the newest? What about gaps?
- Do we lose other features?
- Original ISC data is always available

- ISC QPF

- ISC Grid

- ISC Grid

- Virtual ISC Grid

- ISC Grid

**0301z, Eta, mark**

**0613z, Create Scratch, Tom**

**0542z, interpolated, Tracy**

**0301z, 0542z, or 613z?, who edited it?, original source?**

- ISC Performance and Enhancements

## Show ISC Mode – virtual elements

- Applies to accumulative weather elements
- SnowAmt, QPF
- Applies to “maximum” or “minimum-based” weather elements
- MaxT, MinT, PoP

- ISC Performance and Enhancements

## ISC Aging in the database

- ISC grids are never aged.
- If a site last transmitted a week ago, then those grids are still present
- “Aging” is different from “Purging”
- Should we eliminate all grids from a site when we don’t receive data for xxx hours?

- ISC Performance and Enhancements

## ISC Data Request

- ISC data is broadcast only
- Do we need a request facility, that could look at another sites’ database and get their current state?

- ISC Performance and Enhancements

## Remove some of the send options

- Do we have too many options available for sending ISC grids?
- For Example: Can we remove the capability of sending to an individual site?
- If so, then bookkeeping gets easier for other possible enhancements.

- ISC Performance and Enhancements

## Other Possibilities...

- Automatic running of discrepancy tools as ISC data arrives
- More informative messages in status bar on arrival of ISC data
- Auto loading of ISC weather elements when needed (e.g., when running tools)

- ISC Performance and Enhancements

## Summary

- Performance is currently a big issue.
- H/W solutions
- Forecaster methodology
- S/W performance improvements
- S/W Changes
- Functionality changes are on-the-way
- RPP ISC Focus Group will determine these.

- ISC Performance and Enhancements