**Precipitation Post-Analysis**

**Operations Guide**

December 21, 2005

AWIPS Build OB6

Office of Hydrologic Development (OHD)

Hydrologic Software Engineering Branch (HSEB)

**TABLE OF CONTENTS**

1.  **Introduction**

2.  **Application Control**

    1.  **Application Defaults Tokens**

    2.  **Post Analysis Start Script\
        **

3.  **File Directories**

    1.  **Post Analysis Executables Directory**

    2.  **Post Analysis Data Input Directories**

    3.  **Post Analysis Data Output Directories\
        **

4.  **Database Tables**

    1.  **Input Tables**

    2.  **Output Tables\
        **

5.  **Log File\
    **

6.  **Application Operation**

    1.  **Choose date/hour Window**

    2.  **Main Window**

        1.  **Summed Hourly MPE Field**

        2.  **Gage Only Field**

        3.  **Legend**

    3.  **Control Menu**

**6.3.1 Merged Data**

4.  **Options Menu**

    1.  **Zooming**

    2.  **Gage Table**

    3.  **Remove Ap Option**

5.  **Overlay Menu**

6.  **Help Menu**

7.  **Popup Displays**

    1.  **Locator Popup**

    2.  **7x7 Summed Hourly MPE Field Popup**

**Appendix A. Gage Only Field Generation Algorithm**

**Appendix B. Merged Field Generation Algorithm**

**1.0 Introduction**

The Post Analysis application analyzes precipitation data in a "post
event" mode. It allows previously computed gridded hourly Quantitative
Precipitation Estimates (QPE) to be compared with gage data that may
arrive many hours after the hourly QPE grids were initially generated.
By comparing the summation of the hourly grids into durations and time
periods which match 6- and 24-hour gage data, both the multi-hour gage
data and the gridded hourly grids can be quality controlled and adjusted
as appropriate.

To assist in the comparisons, Post Analysis generates and displays a
summed hourly Multi-sensor Precipitation Estimator (MPE) precipitation
field based on the hourly best estimate QPE fields. The summation
duration can be for 6 or 24 hours and is chosen by the user. Post
Analysis also generates and displays a gage-only field based on 6- hour
or 24-hour gage data. It has options to merge these two fields and to
\"time distribute\" gage precipitation not found in the summed hourly
MPE field back onto the individual hourly fields. The adjusted
individual hourly MPE fields can be stored either separately from the
existing best estimate grids or they can overwrite the existing best
estimate grids.

Post Analysis has the option to overlay the gage identifiers or gage
values of reporting gages on the summed hourly MPE field and gage only
field displays. A popup window showing a 7x7 grid of the summed hourly
MPE field centered on the location of the gage is available when the
gage identifiers or gage values are displayed.

Post Analysis also has the capability to display and edit gage values
through a gage table window. If edits to the gage values are made, the
\"Recompute gage-only\" option may be selected to recompute the gage
only field, update the display and update the database with the new gage
value.

Post Analysis has the capability to display assorted supporting
information. Location information can be displayed using a locator
popup. This popup displays the HRAP coordinates, latitude/longitude,
basin name, county name, summed hourly MPE field value and gage only
field value at the location of the cursor. This display is generated by
right mouse clicking on a point in any display (summed hourly MPE
display, gage only display or merged field display). Post Analysis has
the capability to display geographic overlays such as states, counties,
cities/towns, rivers and basin boundaries. The Geographic overlays
appear on both the summed hourly MPE field display and gage only field
display.

This document provides information on Post Analysis operations. It
begins by discussing the application controls, and the input and output
data accessed. The user interface of the application is described, with
sample screen images supporting the discussion. The appendices detail
the algorithm for computing the gage only field and the merged field.

**2.0 Application Control**

Certain key operational characteristics of Post Analysis are controlled
by application default "tokens" and the start script used to initiate
the application

**2.1 Application Defaults Tokens**

The tokens read by Post Analysis are listed below, with a brief
description and default value listed.

\$(db_name) - name of the IHFS database to read from. (e.g. hd_ob6tir)

\$(geo_data) -- base directory containing geographic overlay data. (e.g.
/awips/hydroapps/geo_data)

\$(st3_rfc) - site identifier. (e.g. marfc)

\$(rfcwide_help_dir) -- directory which contains the help files
/awips/hydroapps/precip_proc/local/data/app/mpe/help/

\$(rfcwide_xmrg_dir) -- directory containing the hourly QPE files. Files
are read from this directory and, if the user selects to overwrite them,
the grids will be written to this directory.
/awips/hydroapps/precip_proc/local/data/mpe/qpe

\$(mpe_post_output) - directory where updated hourly QPE files are
written to if the \"Save/Separate\" option is chosen.
/awips/hydroapps/precip_proc/local/data/app/postanalysis

\$(rfcwide_logs_dir) -- directory where the post_analysis.log file is
written. /awips/hydroapps/precip_proc/local/data/log/mpe_fieldgen

\$(st3_date_form) -- date format. mdY (default) or Ymd for name of QPE
files

\$(rfcwide_prism_dir) -- directory containing the PRISM data (files are
named as PRISM_mm, where mm is the month number).
/awips/hydroapps/precip_proc/local/data/app/mpe/prism .

**2.2 Post Analysis Start Script**

The run_post_analysis script executes the Post Analysis process. It is
located in /awips/hydroapps/precip_proc/bin.

This script "sources" the file /awips/hydroapps/set_hydro_env to set up
environment variables for the application. The script generates a log
file named post_analysis.log which contains detailed information related
to the execution of the process.

The script executes the post_analysis.LX executable with the following
optional command line arguments:

post_analysis.LX \[-d\] \[-t\<date time\>\]

The -d option prints detailed debug-style information to the log. The -t
option requires a specified date/hour in GMT time using the YYYYMMDDHH
format. Using this option causes the specified date/hour to be the first
date/hour in the "Choose date/hour" window.

If the "-d" and "-t" options are not specified, then the default command
in the run_post_analysis script will print out only the general, less
verbose, information and will use the current GMT top-of-the-hour time
as the chosen date/hour.

**3.0 File Directories**

The Post Analysis program uses the following directories for execution
control scripts, and to read, write data files and log files. The
referenced tokens were presented in the previous section.

**3.1 Post Analysis Executables Directory**

The executable files post_analysis.LX and run_post_analysis are to be
placed in this directory.

/awips/hydroapps/precip_proc/bin

**3.2 Post Analysis Data Input Directories**

Geographic Data Directory

\- Defined as \$(geo_data)/\$(st3_rfc)/ascii

\- Post Analysis reads coordinate data from file coord\_\$(st3_rfc).dat
in this directory.

Overlay Data Directory

\- Defined as \$(geo_data)/\$(st3_rfc)/binary

\- Post Analysis reads overlay data from this directory

Hourly QPE Field Directory

\- Defined as \$(rfcwide_xmrg_dir)

\- Post Analysis reads the hourly QPE fields from this directory. Sample
filenames are xmrgmmddyyyhhz

Help File Directory

\- Defined as \$(rfcwide_help_dir)

\- Post Analysis reads the help information from files in this
directory. This information is displayed by selecting the \"Help\"
option in the main window. This directory contains topics on main
display, date selection, gage table, editing gage data, single site.

PRISM Data Directory

\- Defined in \$(rfcwide_prism_dir)

\- Post Aanalysis reads PRISM files for the appropriate month.

Resource File Directory

\- resource file S3Post_res is located at
/awips/hydroapps/precip_proc/bin. It defines the geometry of the main
window, fonts and some colors used by the GUI.

**3.3 Post Analysis Data Output Directories**

Hourly QPE Field Directory

\- Defined as \$(rfcwide_xmrg_dir)

\- directory written to if the "Save/Overwrite" option is chosen

Post Analysis QPE Field Directory

\- Defined as \$(mpe_post_output)

\- directory written to if the "Save/Separate" option is chosen.

Post Analysis Log Directory

\- Defined as \$(rfcwide_logs_dir)

\- directory in which post_analysis.log is stored

**4.0 Database Tables**

Post Analysis accesses the following tables from the IHFS database.

**4.1 Input Tables**

CurPP

\- Read the 6 hour or 24 hour total precipitation data.

RadarLoc

\- Get the number of radars and the location of each radar within the
required geographic area.

ColorValue

\- Get color and value for the legend. If not found, a default hard
coded set of colors/values is used.

S3PostAnalPrefs

\- Read the initial setting (on or off) of various overlays such as
state, city, river, basin and radar overlay. Read value for
num_hours_wind which is the number of hours allowed to be displayed by
the "Choose date/hour" window.

S3PostAnalParams

\- Read values for gg_min_gage_val, gg_min_dist, gg_weighting,
kernel_est_scale and rhat used by gage only field generation and the
merge field generation (see Appendices A and B)

**4.2 Output Tables**

CurPP

\- Edited gage values are written to this table. Editing the gage value
in the gage table, followed by selecting \"Recompute_GageOnly\" submenu,
updates the value field in the CurPP table with the edited data for the
particular location. If \"M\" or \"m\" (missing) is entered in the
edited column, it updates the value in the CurPP table with -9999.0.

**5.0 Log File**

Below is a sample log file without the debug option.

\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\--

Invoking /awips/hydroapps/precip_proc/bin/post_analysis.LX at Thu Nov 22
17:29:15 UTC 2005

Current system: lx1-nhdr

Current database: hd_ob6rhax

Welcome to Post Analysis

version number = OB6

no record for user in S3PostAnalPrefs table

minimum gage value (used by gage only analysis)=5.000000

minimum distance (used by gage only analysis)=15

weight parameter (used by gage only analysis)=2

weight scale parameter (used by merge)=1.000000

rhat (used by merge)=0.950000

default overlay colors used

The selected date/hour is 2005-11-22 12:00:00

13 radars defined

HRAP coordinates of radar locations:

AKQ x,y = 985.381531 501.475037

BGM x,y = 937.646484 633.660339

BUF x,y = 881.883179 626.422791

CCX x,y = 918.437073 585.472351

DIX x,y = 994.056030 597.797180

DOX x,y = 990.654419 561.330811

ENX x,y = 965.490906 661.157654

FCX x,y = 921.454041 471.010559

LWX x,y = 951.416626 544.559753

OKX x,y = 1008.037476 634.798279

PBZ x,y = 883.020569 556.854553

RAX x,y = 972.787292 454.673065

RLX x,y = 878.904907 490.249756

total gage number within the required area and time interval is 176

default precip levels,colors used

default precip levels,colors used

Completed post_analysis at Thu Nov 22 17:30:14 UTC 2005

**6.0 Application Operation**

**6.1 Choose date/hour Window**

The first window displayed by Post Analysis shows a table of allowable
ending date/hours that may be selected for processing. The 6- or 24-
hour mode is selected by clicking in the radio button diamond next to
the appropriate value. The default selection is 24 hour. A sample window
is shown in Figure 1.

By default, the first date/hour listed in the table is the current hour
in GMT. This can be changed by specifying the date/hour as an argument
on the command line. In this manner, Post Analysis can be run for
historical cases.

To select a date/hour, move the cursor to the desired row and click with
the LEFT mouse button.

![](media/image1.png){width="4.25in" height="3.310416666666667in"}

**Figure 1 Post Analysis Choose date/hour Display**

This row will be highlighted and the selection may be changed by
clicking on another row.

Three buttons appear at the bottom of this window - \"OK\", \"Cancel\",
and \"Help\". The \"OK\" button initiates Post Analysis processing for
the selected date and hour. The \"Cancel\" button will exit Post
Analysis without doing anything. Choosing the \"Help\" button pops up a
window showing help text.

Many COOP observers stations report data around 12z with a duration of
24 hours. Selecting a date with hour = 12z and a 24 hour processing
duration will often provide the most data rich case. METAR and/or other
data may be available with 6 hour durations at hours 0z, 6z, 12z and
18z. Choosing other hours will generally result in no gage data since no
data matches the specified end time and duration.

**6.2 Main Window**

An example of the main window for MARFC is shown in Figure 2. The main
window consists of three parts: two geographic window displays and a
legend bar. The upper window displays the summed hourly MPE field and
the lower window displays the gage only field.

**6.2.1 Summed Hourly MPE Field**

This window displays a grid created by adding up the precipitation
values for each grid bin from each previously saved hourly best estimate
QPE file. The summation duration is 6 or 24-hour depending on the user's
selection in the "Choose date/hour" window. If there is an error reading
any of the QPE files, an error message will be written to the
post_analysis.log.

It should be noted that although MPE generates many precipitation fields
(RMOSAIC, BMOSAIC, LMOSAIC, etc), Post Analysis reads only the field
designated as the best estimate QPE field.

Post Analysis expects the best estimate gridded fields to be stored in
the xmrg file format.

**6.2.2 Gage Only Field**

This window displays a gage only analysis grid based on the point precip
data values. The point data is read from CurPP table. Only data with
physical element = "PP" area are used. All gage values for gages located
within the geographic area which have an observed time and duration that
match the specified ending time and duration (6 hr or 24 hr) from the
"Choose date/hour" window are considered. Note that the Post Analysis
gage only field does not consider precip values with physical element =
"PC".

Appendix A details the algorithm for calculating the gage only field.

**6.2.3 Legend**

The legend shows the accumulation intervals in inches. The first color
in the legend signifies missing data. The second color signifies a value
of zero. The third color is for the interval greater than zero and less
than the first non-zero value. All colors except the last are for values
within the specified interval (greater than or equal to lower value and
less than higher value). The last color corresponds to all values
greater than the lower value.

![](media/image2.png){width="5.997916666666667in"
height="8.060416666666667in"}

**Figure 2 Summed Hourly MPE field and Gage Only Field**

**6.3 Control Menu**

The Post Analysis "Control" menu contains two options: "Merge Data" and
"Quit". The Quit option simply terminates the application.

**6.3.1 Merged Data**

Selecting this option results in the display of a grid which is created
by merging the summed hourly best estimate field with the gage only
field. This merged field can be used to generate new best estimate
fields or to update previously generated best estimate fields using a
time distribution method. A sample merged field is displayed as Figure
3. Appendix B describes the algorithm for calculating the merged field
and for updating previously generated fields.

The merge data window has a menubar with two options: Control and
Overlays. The "Overlays" option allows various overlays to be displayed
on the window. The Control menu option has three sub-options: Quit and
SaveSave/separate, Quit and SaveSave/overwite and Close Window.

Selecting "Save/separate" causes the new time distributed hourly MPE
fields generated from merged field to be saved in a separate directory.
Selecting "Save/overwrite" causes the new time distributed hourly MPE
fields to overwrite the previous best estimate fields.

Note if there is at least one error occurring when reading the best
estimate files to generate the merged field, no individual hourly MPE
fields will be generated and saved since not enough accurate data can be
used to calculate the time distributed hourly MPE fields.

.

![](media/image3.png){width="5.997916666666667in" height="4.88125in"}

**Figure 3 Post Analysis Merged Field Display**

**6.4 Options Menu**

The "Options" menu includes the following options: "Zoom", "Gage Table"
and "Remove AP" submenus. Each is explained below.

**6.4.1 Zooming**

To activate the Zoom option, the user must first define the region to
zoom. This is

done as follows:

1)  Press and hold the LEFT mouse button at starting point of the zoomed
    region in either display (this point will become the top left point
    of the zoom region).

2)  Drag the cursor until the desired area is enclosed within the
    rectangle

3)  Release the button

4)  Click on the \"Zoom\" option under \"Options\"

The rectangle can be defined on either display. Both displays are
redrawn at the same zoom level. Selecting the \"Zoom\" option again will
return the displays to the default unzoomed mode.

An example of a zoomed window with precipitation gages display is shown
in Figure 4.

Certain features are available only in zoom mode. The "Gage Identifiers"
and "Gage Values" under the "Overlay" menu and the 7x7 popup can be
activated only in zoomed mode.

**6.4.2 Gage Table**

The gage table option presents a table with the following columns:

1)  Gage identifier

2)  Gage Value (units = inches)

3)  Closest Summed Hourly Best Estimate value of 9 nearest neighbor bins
    to the gage value

4)  Lowest Summed Hourly Best Estimate value of 9 nearest neighbor bins
    to gage location

5)  Highest Summed Hourly Best Estimate value of 9 nearest neighbor bins
    to gage location

6)  Edit Column

Gage values are for gages with duration chosen in the "Choose date/hour"
window. The gage table for the sample case is shown in Figure 5.

The gage table allows the user to enter new gage values for particular
gages, and then update the value in the CurPP database table and
redisplay the gage only field by selecting "Recompute Gage Only". This
option recomputes the gage only field with the new gage value(s) and
redisplays it. Zoomed displays are not refreshed. Therefore, the display
must be \"unzoomed\" to see the result.

The information in the gage table window can be ordered by 4 sort
options:

1)  Gage value - display values in descending order

2)  Gage ID - display values alphabetically by gage identifier

3)  Ratio -- display values in descending order. The value is the ratio
    of gage value to the best summed hourly MPE value (if gage value \>
    best summed hourly MPE value), or the ratio of best summed hourly
    MPE value to the gage value (if gage value \< best summed hourly MPE
    value)

4)  Difference - difference between the gage value and the closest gage
    value.

The sort option is for display purposes only. The edit gage value option
is available with all sorting options.

**6.4.3 Remove Ap option**

This option replaces areas of the summed hourly MPE field deemed to be
bad with missing values. To remove an area, click with the left mouse
button and hold to create a rectangle as for the zoom option and then
choose the \"Remove Ap\" option. This will cause the summed hourly MPE
field to be recalculated with missing values substituted in the area
removed. The merged field will replace these missing bins with the
appropriate values from the gage only field.

More than one area may be removed. However, after each area is selected,
the \"Remove Ap\" option must be chosen. This option is not available
from zoomed mode.

**6.5 Overlays Menu**

Geographic data such as states, counties, cities/towns, river, and basin
boundaries can be overlayed (if data is available) on the summed hourly
MPE field display, gage only field display and merged field display
whether in zoomed mode or not. Any already displayed overlay can be
toggled off by re-selecting the overlay.

Note that the \"Gage Identifiers\" and "Gage Values" options under the
\"Overlays\" menu pulldown are activated only in zoom mode and only if
precipitation gage reports are available for the chosen duration and
ending hour. When selected, this option displays the locations of all
reporting precipitation gages by gage identifier. The actual location of
the gage is at the lower left hand corner of the text. This overlay
cannot be toggled on and off. It can be removed by toggling another
overlay option**.**

**6.6 Help Menu**

Post Analysis supports the display of help popup windows, including an
overview of the application and other topics include editing the bias
table and editing radar coverage information.

**6.7 Popup Displays**

Two other types of popup windows for displaying information are
available and are discussed in the following sections.

**6.7.1 Locator Popup**

Post Analysis has a capability to display a locator popup window by
pressing and holding the right mouse button. This locator popup displays
the HRAP coordinates, latitude/longitude, county name, MAP basin name,
summed hourly MPE precipitation value and the gage only precipitation
value at the location of the cursor.

The locator popup can be activated from summed hourly MPE field display,
gage only field display, merged field display, and from displays in
zoomed mode.

**6.7.2 7x7 Summed Hourly MPE Field Popup**

While the displays are in zoomed mode, and when "Gage Identifiers" or
"Gage Values" are activated, Post Analysis can display a 7x7 grid popup
by clicking the left mouse button anywhere in a zoomed window. This
window shows the values of the 7x7 nearest neighbor summed hourly grid
bins centered on the gage closest to the cursor location. The gage is
chosen automatically and may be inside or outside of the zoomed window.
The gage ID is given on the left column along with the 6 or 24-hour gage
precipitation value.

![](media/image4.png){width="6.034722222222222in"
height="7.728472222222222in"}

**Figure 4 Post Analysis Main Window (Zoom Mode)**

![](media/image5.png){width="5.998611111111111in"
height="3.7743055555555554in"}

**Figure 5 Post Analysis Gage Table (sorted by gage value)**

**Appendix A. Gage Only Field Generation Algorithm**

The following parameters are read from the S3PostAnalParams table:

> MINVAL = minimum precipitation value threshold
>
> MINDIST = minimum distance value threshold
>
> IWIND = distance weighting factor

Define the following variables:

PRECIP = the Summed Hourly QPE field

> DISTANCE = the distance from point i, j to gage ig
>
> gage_value(ig) = the gage value of gage ig
>
> gageonly_value(i,j) = the gage only field value at location (i,j)
>
> allzero = TRUE or FALSE
>
> lowgage = TRUE or FALSE
>
> dist(ig) = distance between point i,j and gage ig
>
> WEIGHT(ig) = weighting factor from gage ig.
>
> DENWT = variable used in calculating WEIGHT(i,j)
>
> weighted_value(i,j)= sum of weighted gage values on point(i,j). It
> considers all reporting
>
> gages' weights. Also weighted by the PRISM value.

Calculate allzero:

If (all 8 surrounding bins from to bin i,j have PRECIP = 0)

allzero = TRUE

else

allzero = FALSE

Calculate lowgage:

If (allzero = TRUE and any DISTANCE \< MINDIST and gage_value(ig) \>
MINVAL)

lowgage = TRUE

else

lowgage = FALSE

Calculate the gage only value at each bin i,j :

If (lowgage = TRUE or allzero = FALSE)

gageonly_value(i,j) = weighted_value(i,j) (see below)

> else
>
> gageonly_value(i,j) = 0.0

Note that the "allzero" parameter above "shapes" the gage-only field
according to the Summed Hourly QPE field. The gage-only field for a bin
is set to 0.0 if all 8 surrounding bins from the Summed Hourly QPE field
are 0.0. This algorithm for calculating the gage-only field was
originally used by the StageII application. It has been replaced by a
different algorithm in MPE.

Weighted Value Computation Scheme:

a\) Calculate dist(ig)

b\) For all gages with dist(ig) \< MINDIST, compute

DENWT = DENWT + exp(-dist(ig) / IWIND)

c\) If (no gages have dist(ig) \< MINDIST) then

WEIGHT(ig) = 0.0

else

WEIGHT(ig) = exp(-DISTANCE / IWIND) / DENWT

d)Calculate weighted_value(i,j)

weighted_value(i,j) = sum of weighted gage values from all reporting
gages, where each weighted gage value is also weighted by the PRISM
normals:

weighted_value(i,j)= WEIGHT(ig)\*gage_value(i,j)\* \[xmptim(i,j) /
xmptim(at gage)\]

> where xmptim = PRISM value (historical mean accumulation)

**Appendix B. Merged Field Generation Algorithm**

Use the following constants:

> rhat = sample lag-1 spatial correlation of precipitation estimates;
> currently set to 0.95.; (read from S3PostAnalParams table)

misbin(i,j) = MISBIN data value for HRAP bin i, j .

Define the following variables:

gg(i,j) = the gage only field at bin i,j

> shs(i,j) = the summed hourly value at bin i,j; considered missing if
> \< 0.0

merge(i,j) = the merged field value at bin i,j

obw = observed weight for i,j

num = number of hours chosen for Post Analysis calculation (= 6 or 24)

precip1(i,j) = the hourly MPE field at bin i, j

a\) For each HRAP bin i,j, compute merge field value:

if (gg(i,j)=0 and shs(i,j)=0) then

merge(i,j) = 0

else if (misbin(i,j) = 0) then

merge(i,j) = gg(i,j)

else if (number of reporting gages = 0) then

merge(i,j) = shs(i,j)

else

calculate obw (see below)

merge(i,j) = obw\*gg(i,j) + (1-obw)\*shs(i,j)

> The observed weight is calculated as follows:
>
> sc = scale constant, currently set to 1.0
>
> lnrhat = ln(rhat) = ln(0.95) = -0.0513
>
> dnear = distance in HRAP bins from i,j to closest gage
>
> obw = exp(sc\*lnrhat\*dnear)

b\) Compute ratio value:

if (shs(i,j) \> 0.0) then

then ratio(i,j) = merge(i,j) / shs(i,j)

else

ratio(i,j) = -1\*merge(i,j)

c\) Perform time distribution:

if (ratio(i,j) \< 0 ) then

precip1(i,j) = -ratio(i,j) / num

else

precip1(i,j) = precip1(i,j)\*ratio(i,j)