**IflowsMonitor (IMON) version 3.0**

**Released February, 2006**

**Ron Holmes (ITO State College, PA)**

**814-231-2435**

**Overview**

IMON 3.0 is a Java program to monitor Iflows Precipitation gauges and
Stream Gauges. It alerts forecasters if any of the gauges reach close to
Flash Flood Guidance category for a particular county or any Stream
Gauge reaches some critical percentage of flood stage. IMON is composed
of separate programs: a Decoder and a GUI. The Decoder exists outside of
AWIPS and is used to download Iflows web pages and calculate various
alerting criteria. The GUI runs inside of AWIPS so forecasters can
monitor Iflows gauges and be alerted when a particular gauge reaches
criteria. LDAD also needs to be set up to recognize the files brought
across the firewall into AWIPS. The program will handle up to 1000 rain
gauges and 200 stream gauges. Below is a screen shot showing the
monitoring and alert feature.

![](media/image1.png){width="6.0in" height="4.459722222222222in"}

![](media/image2.png){width="6.0in" height="3.629861111111111in"}

![](media/image3.png){width="6.0in" height="4.5in"}

In all instructions the application directory is the directory where you
untar the various .tar.gz files.

**Download IMON files from the ER FTP Server**

The LAD does not support large files so you must go the ER FTP server to
get the distribution. You will be downloading 2 additional .tar.gz
files: Decoder.tar.gz and GUI.tar.gz and some LDAD related files:

1)  ftp to [ftp.werh.noaa.gov](ftp://ftp.werh.noaa.gov/) User is
    anonymous and Password is your email address.

2)  cd /share/ctp/IflowsMonitor

3)  get the following files:

> Decoder.tar.gz
>
> GUI.tar.gz
>
> LDADinfo.txt

The Decoder.tar.gz file is for the machine external to Awips and the
GUI.tar.gz file is the program for inside Awips. Move the Decoder.tar.gz
file to your external machine and the GUI.tar.gz file inside of AWIPS.
Your GUI.tar.gz will also contain some LDAD-related files to transfer
the Decoded files from outside AWIPS through the firewall.

**PROGRAM UPDATES**

If there are updates to the program it is recommended that you untar any
of the .gz files you downloaded from ER FTP Server into a scratch
directory and **NOT** your operational application directory. You don't
want to overwrite any configuration files you've already spent time
editing. Typically, if there are no major changes, all you would have to
do is replace the compiled .jar file in your application directory with
the new .jar file in the scratch directory. But there may be
changes/additions to some of the configuration files too. You may have
to compare some of the files to see what's changed or simply email the
author.

**IFLOWS WEB SITE AND SRH TEXT DATABASE WEB SITE**

To find out which Iflows Rain and Stream gauges for IMON to monitor and
to download the Iflows Map images for your CWA visit:
[**http://afws.net/**](http://afws.net/)

The FFG files you wish to download to compare Iflows gauges with are
located at:
[**http://www.srh.noaa.gov/data/XXX**/](http://www.srh.noaa.gov/data/XXX/)
where XXX is CWA that has the FFG files.

**Setup:**

**External to AWIPS -- the Decoder**

In order to run the IMON Decoder it must run on a machine external to
AWIPS that has ssh connectivity to your LDAD. The decoder program uses
**WGET** so it can retrieve the Iflows web pages from the Internet.
Unfortunately since LDAD is still an HP system it does not have WGET so
you cannot run the decoder on LDAD. Also you will need to set up the ssh
public/private keys in order for your external machine to SCP the
decoded files to LDAD. See the end of this document to learn how to do
this.

1)  Download the Decoder.tar.gz file from the ERH FTP server in the
    /share/ctp/IflowsMonitor directory and untar this in your
    application directory. Ex: tar --xzvf Decoder.tar.gz

2)  Edit getIflows.sh This is used to download the Iflows web pages for
    the states encompassing your CWA and download the FFG from the
    Southern Region text database web site. There is a configuration
    section at the top.

    a.  Change the ***appdir*** variable to the path of your application
        directory. Make sure the path ends with a slash (/)

    b.  Change the state variables (***state1*** and ***state2***) for
        those states you wish to download Iflows web pages for. Up to 2
        states are supported but you can always change the code further
        down to add more.

    c.  Change the FFG variables (***ffg1,etc***) and the base URL
        variables (***ffg_url1,etc***) to point to the particular FFG
        products you wish to use to compare county FFG values with the
        Iflows gauges. Ultimately you want to construct a path to the
        FFG files on the SRH text database web site that cover the
        states you configured in step c.

    d.  If you are going to monitor and alarm for Stream Gauges set up
        the portion of the URL in variables ***sg1,sg2,etc*** that point
        to the Iflows web pages which contain the Stream Gauge
        Information. Typically these will end in a .HTM extension.
        Ultimately we are trying to construct a path that points to that
        URL. Again see the Iflows web site for these URL's in your
        state.

3)  Edit countylist.txt

    a.  This is the main configuration file that describes which Iflows
        gauges to monitor and alarm for. Visit the Iflows web site link
        (above) to get the gauge ID number for each county in your CWA.
        Currently the decoder does not decode the River Stage Data, it
        just does the Current Rainfall Data. Each CWA listing starts
        with a header listing\
        \
        WFO,\<CWA\>,\<FLAG\>,\<THRESHOLD\>\
        \
        where \<CWA\> is a CWA that you wish to monitor. The \<FLAG\>
        (YES or NO) determines which CWA you wish to be alerted for when
        Iflows gauges reach FFG criteria. **For your CWA the \<FLAG\>
        should be YES. Set the other CWA's to NO.** The \<THRESHOLD\>
        flag **is only used for your CWA** and is the percentage used to
        calculate your FFG alarm criteria. This could be 70,85,90
        percent\...whatever you want. You will not put a threshold flag
        in for the other CWA's. Also, the order of the CWA's in the
        left-hand expandable tree is determined by the order of each CWA
        block in this file so if you would like to see your CWA at the
        top of the expandable tree put your CWA listing of gauges at the
        top of this file.

> Example: Your CWA: WFO,XXX,YES,90
>
> Other CWAs: WFO,YYY,NO

b.  Following the CWA header is a listing of Iflows gauges in the CWA
    which consists of the \<COUNTY NAME\>,\<UGC CODE\>=\<LIST OF
    GAUGES\> format where \<LIST OF GAUGES\> is a list of the Iflows
    gauges in a particular county, in comma delimited form. Up to 1000
    gauges are supported. This \<COUNTY NAME\> should be in the same
    form (case and spelling) as that listed in the FFG guidance you are
    downloading. If the FFG you are using breaks a county up like:\
    N. Centre and S. Centre then there should be a line describing that
    break-up and a list of gauges that pertain to that section of the
    county. For example:

> N. Centre,PAZ018=2887,2852
>
> S. Centre,PAZ019=2871,2881
>
> The UGC code is needed because sometimes counties in a CWA in
> different states next to each other can have the same names.

4)  Edit streamgauge.conf. This is a list of stream gauges that are
    contained in the Stream Gauge HTM files you are downloading.

    a.  Each line is a CSV list

> **GAUGE**\_**NAME,ID,COUNTY,STATE,THRESHOLD,FLOOD_STAGE**
>
> The **THRESHOLD** is some percentage of the Flood Stage you wish to be
> alarmed for. This could be set to anything (and different) for each
> individual gauge. The **FLOOD_STAGE** is determined by you. If a
> stream gauge does not have a flood stage set it to some big number,
> like 100, so that an alarm will never trip.

5)  Set up a cron on your external machine to run getIflows.sh every 15
    minutes since the data updates every 15 minutes on the Iflows web
    site. EX:

> **15 \* \* \* \* /home/ron/IflowsDecoder/getIflows.sh \>
> /home/ron/IflowsDecoder/getIflows.log**

6)  You'll need to generate your Public/Private ssh keys on the external
    machine and give your Public key to LDAD so that these files can be
    transferred to LDAD without a password. See the end of this document
    on how to set this up.

7)  Run the getIflows.sh script by hand to see if it downloads the
    various web pages and decodes them properly. If successful, you
    should see your FFG\* files and 4 CSV files that the GUI needs to
    monitor and alarm for:

    a.  iflows_readings.txt

    b.  iflows_alarmdata.txt

    c.  strguage_readings.txt

    d.  strgauge_alarmdata.txt

**Setup - AWIPS LDAD**

Setup LDADInfo.txt to recognize the files that your external machine
downloads and scp's them into /data/Incoming. Upon receipt of these
files a preProcessIflows.pl (included) script will run which then copies
them across the firewall into your application directory on AWIPS.
Follow the steps below:

1)  As user **ldad** modify /data/fxa/LDAD/data/LDADinfo.txt on DS1 with
    the following lines below to let LDAD know what to do with the files
    you are SCP'ing into /data/Incoming from the external machine that
    runs the decoder. Most likely you will only need to change the FFG
    filenames to match those you are downloading from the SRH text web
    site: **http://www.srh.noaa.gov/data/XXX**/. Each time the decoder
    runs on the external machine these files will be SCP'd to
    /data/Incoming and then the preProcessIflowsAlarm.pl file will copy
    them into your application directory inside of AWIPS. The .txt
    filename extension is not needed. Ex: iflows_readings.txt and
    iflows_alarmdata.txt get created by the decoder and scp'd to
    /data/Incoming but you don't need the .txt extension in the
    LDADInfo.txt file.

> \### example lines to add to LDADinfo.txt on DS1 (under
> data/fxa/LDAD/data)
>
> \### to get various IflowsMonitor files from LS1 to DS1
>
> \### make sure you stop/start LDAD after you make changes to
> LDADInfo.txt
>
> \###
>
> iflows_readings \| \| \| \| \| \| preProcessIflowsAlarm.pl \|
>
> iflows_alarmdata \| \| \| \| \| \| preProcessIflowsAlarm.pl \|
>
> strgauge_readings \| \| \| \| \| \| preProcessIflowsAlarm.pl \|
>
> strgauge_alarmdata \| \| \| \| \| \| preProcessIflowsAlarm.pl \|
>
> FFGPA2 \| \| \| \| \| \| preProcessIflowsAlarm.pl \|
>
> FFGPA3 \| \| \| \| \| \| preProcessIflowsAlarm.pl \|

2)  Edit preProcessIflowsAlarm.pl.

    a.  Change the ***\$appdir*** variable at the top to the application
        directory where you will untar the GUI.tar.gz file (this is
        where you will be running the GUI from on the AWIPS side). Place
        the preProcessIflowsAlarm.pl file in /awips/fxa/ldad/bin on DS1.

3)  Stop and Start LDAD on DS1 because you've changed LDADInfo.txt

    a.  /awips/fxa/ldad/bin/stopLDAD.sh

    b.  /awips/fxa/ldad/bin/startLDAD.csh

4\) Run the getIflows.sh script on your external machine again and see
if the files listed in LDADinfo.txt above are making it into your GUI
application in AWIPS. Hopefully they will get SCP'd to your LDAD without
being challenged for a password. If it prompts you for a password then
your ssh keys are not set up properly

**Internal to Awips -- the GUI**

1)  On DS1 first uncompress the GUI.tar.gz file in your application
    directory (any user can set this up), then untar the .tar file. You
    cannot use tar -xzvf all in one shot\....unfortunately we are still
    dealing with an HP system. Ex:

> gunzip GUI.tar.gz
>
> tar --xvf GUI.tar

2)  Copy the countylist.txt file you modified on your external machine
    into this directory. They must be the same.

3)  Edit GUI.cfg

    a.  Set the GUI Refresh to the number of minutes you wish the GUI to
        refresh. Since the Iflows web site updates the data every 15
        minutes the default is 15 minutes. You probably don't need to
        change this.

    b.  Change the Menu=\<MENU LABEL\>,\<FFG FILE NAME\> line where
        \<MENU LABEL\> is the Menu label text you wish the GUI to
        display and \<FFG FILENAME\> is the name of the file that gets
        stored into the application directory passed in from LDAD. In
        operation when you click on the \<MENU LABEL\> that is displayed
        it will show you the contents of the \<FFG FILE NAME\> in a
        separate window.

> Ex: Menu=MARFC FFG,FFGPA2.txt\
> \
> means when the user clicks on the MARFC FFG menu label it will display
> the contents of FFGPA2.txt that gets transferred into AWIPS from LDAD.

c.  Change the Gif Image flag to Yes or No if you downloaded the images
    for each county from the Iflows web site, or not. If you have
    downloaded gif images they should be in the **gifs** directory with
    a filename of \<COUNTY NAME_XX\>.gif where \<COUNTY NAME\> is the
    same county names listed in countylist.txt and \_XX is the 2 letter
    state identifier for that county. If you have a few counties split
    up, ex N. Centre and S. Centre, just leave the N. or S. off of the
    filename).

EX. Centre_PA.gif

d.  Change the Wav Files flag to Yes or No if you've created .wav files
    (you could record your voice, or just play some chime sound) to be
    played on Awips when a particular gauge reaches FFG values for a
    county. You can create these .wav files yourself...just name them as
    in step .c above with a \_XX.wav extension.

> EX. Centre_PA.wav
>
> e\. Change the Stream Gauge flag to Yes or No if you are decoding and
> monitoring Stream Gauges

4)  Edit already_alarmed.txt

> This file describes which counties in your CWA have already tripped
> alarms. The idea is that once a particular gauge causes a county to be
> alarmed, that county will not show another pop-up alarm window. This
> could get annoying with gauge data refreshing every 15 minutes.
> Therefore this file keeps track of which counties have already been
> alarmed. It consists of \<COUNTY NAME\>**:**\<FLAG\> where \<COUNTY
> NAME\> has the same format of your County name described in
> countylist.txt and \<FLAG\> is either YES or NO. Initially you will
> set this to NO for all counties in your CWA. If gauges do trip for a
> county, this file gets updated to YES for that county. In this way
> when forecasters log out and the next shift comes in and starts the
> program they can see which counties have reached FFG criteria by
> choosing the "Show Current Alarms" Menu item. Note: In order for the
> program to determine when a "rain event" ends so that any existing
> alarms can be cleared out in preparation for the next event the
> program will check the 3 hour interval rainfall amounts for all Iflows
> gauges being monitored. If all of those readings are 0.00 then the
> program "thinks" the event is over and will set all counties to NO in
> the already_alarmed.txt file. However, you can manually override this
> feature at any time and choose the Reset Alarms menu item from the
> Admin menu and everything will be re-set to NO for all counties in
> your CWA.

5)  Edit RunIflowsGUI.sh

    a.  Change the variable ***appdir*** to point to your application
        directory. Make sure the path ends with a slash (/)

    b.  Create a menu option on AWIPS that will run this. Edit
        local.conf in your /awips/fxa/data/appLauncher directory.

**Setup**

**Program Testing**

After setting up the external machine, AWIPS, and LDAD test the decoder
on the external system by running it at a shell prompt and noting any
errors. You should see 4 new iflows related files and your FFG files and
hopefully they will get SCP'd to your LDAD without being challenged for
a password. If it prompts you for a password then your ssh keys are not
set up properly. See the ssh notes below on how to set this up.

Once the files are transferred to LDAD they should end up in your GUI
directory you set up. If not check your preProcessIflowsAlarm.pl and
make sure that perl path

(type: "which perl" at a command line to determine this) is correct and
the path to your application directory is correct. Make sure the
permissions on this directory have group (fxalpha) writable because the
user that transfers them over into the GUI directory is **user ldad**
and user ldad belongs to group fxalpha. Also it is important that the
file already_alarmed .txt has group fxalpha writeable permissions
because any user that runs the program must be able to update that file.

**Program Operation**

**Monitoring Stream Gauges**

Verison 3.0 supports monitoring and alarming for Stream Gauges. A
separate window will display showing these gauges. This window can be
hidden at any time and will continue to monitor gauges and pop an alarm
if a gauge reaches criteria. You can redisplay the window at any time by
choosing the Monitor Stream Gauge menu item.

**Sorting Iflows Gauges**

The GUI has a few features that make monitoring Iflows gauges easy. The
main feature is the ability to sort any of the gauges in ascending or
descending mode simply by clicking on the column header, successively.
An arrow will indicate which way it is sorting. For example, this makes
it easy to see which gauges show the highest amounts in your CWA. In
addition you can drill down through the expandable tree on the left side
of the GUI to monitor gauges based on a CWA, a county, or a particular
gauge.

**Receiving Alarms**

**Rain Gauges**

When a particular rain gauge reaches the threshold you set up
(70%,85%,90%) of FFG for the county it will trip an alarm. This alarm
will be a pop-up window showing the information for the gauge along with
a .gif image of the gauge location (if you've downloaded the .gif images
from the Iflows web site). Also a .wav file will play if you've created
one for this county. This pop-up alarm will only display once for a
particular county in automatic mode during program operation. After this
county is flagged as having been already alarmed it will not show a
pop-up window for any other gauge in this county\...even if more gauges
reach FFG criteria in that county. With iflows reading coming in every
15 minutes this could get annoying. So forecasters do not get annoying
pop-up windows every time the GUI refreshes. However, forecasters can
manually choose to view all counties that have been previously alarmed
by clicking on the "Show Current Alarms" menu item. The program will
automatically "reset" this list of counties that have already been
alarmed when all iflows gauges report 0.00 precipitation in the 3 hour
column. This is done so that the program can reset itself for a new
event after an event has ended and there is a window of dry weather. Of
course the forecaster can also manually override this feature at any
time and reset the list by choosing the "Reset Alarm" menu item under
Admin

**Stream Gauges**

This feature works in much the same manner. When a stream gauge reaches
some threshold of the Flood Stage that you've set a pop-up alarm will
display showing you the information. An internal list is kept which lets
the program know this stream gauge has already reached criteria and will
not display again, for that gauge only. However, unlike the rain gauges,
other stream gauges in the same county will trip alarms as they reach
criteria. Also, unlike with the rain gauges, the status of whether the
stream gauge has been tripped is not saved to disk, only in memory, so
that when you close the program and open it again, you will be
re-alerted that those gauges have gone above criteria.

**Displaying FFG Guidance**

Current FFG guidance downloaded can be displayed for comparison using
Show FFG menu item.

**Displaying Locations of Iflows Gauges**

Locations of Iflows gauges can be displayed by clicking on the Maps menu
item. If you've downloaded the .gif images from the Iflows web site for
each county in your CWA and placed them in the **gifs** directory then
you should be able to display them.

Setting up Secure, Password-less communication between your external
machine and LS1 (LDAD)

In order to get the decoded Iflows files from your external machine to
LDAD without passwords you need to generate a pair of public/private ssh
keys. Here's what you do:

1\) On the external machine as the user who will be running the program
via cron, change to your home directory. Type:

ssh-keygen --t rsa

If it then prompts you to enter a passphrase, just hit return.

When it asks you what file you would like to save your key as, just hit
return.

2\) From your home directory cd to the .ssh hidden directory. You will
see a pair of keys that was just generated. The id_rsa file is the
Private key and id_rsa.pub key is the Public key. You want to leave the
Private key here.

3\) Next you will be appending the contents of the Public key you just
generated to user ldad's authorized keys file on LDAD. Rename your newly
generated public key to something like ***your_machine_name.public***.
Then transfer the ***your_machine_name.public*** file over to LS1 to the
/ldad/.ssh hidden directory.

Run:

cat ***your_machine_name.public*** \>\> authorized_keys2

Delete the ***your_machine_name.public*** file when you are done. Your
public encrypted key should now be at the end of whatever was in the
authorized_keys2 file on LS1. This tells LDAD to let your external
machine in via the secure ssh or scp communication standard. For this
program you will be SCP'ing the decoded files to LS1.