# **ISC Routing Test Plans**

## ***Standalone Loopback Testing***

### **Purpose:** 

Initial GFESuite testing with IRT web service in isolation.  Components
tested include Request/Reply, VTEC active table sharing, and ISC traffic
loopback.  The loopback capability is enabled by changing the
TRANSMIT_SCRIPT configuration item to loopback to the iscDataRec script.
This configuration will echo back the same data that is sent.

This test case will exercise all of the changed functionality and
scripts without requiring official AWIPS installations nor the official
central IRT web service.

### **Environment:** 

-   IRT web service installed on the local machine, standalone (i.e.,
    not the operational service)

-   GFESuite installation - single site installation

-   GFESuite configured to use IRT web service, enabling send on save,
    enabling isc request

-   GFESuite configured for loopback through the TRANSMIT_SCRIPTS config
    item.

-   GFESuite configured for VTEC active table sharing.

### Procedure: {#procedure .unnumbered}

1)  Install IRT web service on the same machine as you plan to install
    the GFESuite. Start the web service using the direct "python
    RoutingTableSvc.py" command.

2)  Install GFESuite as any site with the same "mhsid" as the site. Do
    not start the ifpServer. Create a etc/SITE/localConfig.py override
    file containing the following lines to set up local loopback (note a
    version of this localConfig is located in the GFESuite source tree
    under GFESuite/scripts/test/localConfigLB.py):

> from serverConfig import \*\
> import serverConfig\
> serverConfig.ISC_ROUTING_TABLE_ADDRESS =
> "[http://0.0.0.0:8080](http://0.0.0.0:8080/)" *(modify address as
> needed, the 0.0.0.0:8080 works if the IRT web service is on the same
> host as the ifpServer)\
> *serverConfig.REQUEST_ISC = 1\
> serverConfig.SEND_ISC_ON_SAVE = 1\
> serverConfig.TRANSMIT_SCRIPT = GFESUITE_HOME + \'/bin/run/iscDataRec
> MSGID %SUBJECT %ATTACHMENTS\'

3)  To test the proper registration of the ifpServer to the IRT Routing
    Table and the IRT register() call, bring up the ifpServer. Use the
    IRT web interface to verify that server is registered.  Examine the
    GFESuite\'s data/logfiles/YYYYMMDD/ifpServer\* log for registration
    information. You should see an entry similar to:\
    00:42:41.698 IrtAccess.py 197 EVENT: IRT access: okay register t=
    0.064 {\'protocol\': 20070723, \'wfos\':
    \'ABQ,AMA,BOU,CYS,DDC,GJT,GLD,LBF,PUB,RIW\', \'site\': \'PUB\',
    \'server\': \'localhost\', \'dims\': \'145,145\', \'proj\':
    \'Grid211\', \'mhsid\': \'PUB\', \'parms\':
    \'RH,HrsOfSun,WindWaveHgt,FreeWind,DSI,IceAccum,WaveHeight,Swell,Wind20ft,WindChill,Sky,Wx,T,StormTotalSnow,IceCoverage,SnowAmt,Period2,VentRate,Wetflag,WindGust,MinT,Hazards,RHtrend,Td,PoP,TransWind,QPF,Haines,SurfHeight,MixHgt,Swell2,HeatIndex,MarineLayer,MaxT,LAL,MaxRH,Ttrend,SnowLevel,InvBurnOffTemp,Stability,FzLevel,MinRH,Period,CWR,Wind\',
    \'port\': 98000044, \'Bbox\': \'38.0,26.0,9.0,9.0\'}

> 00:42:41.700 IrtAccess.py 168 EVENT: IRT Registration Successful.
> Re-register time: Thu Oct 25 01:42:41 2007

4)  To test the proper unregistration of the ifpServer and the IRT
    unregister() call, shut down the ifpServer.  Use the IRT web
    interface to verify that server has been unregistered.  Examine
    data/logfiles/YYYYMMDD/ifpServer\* log for unregistration
    information. You should see an entry similar to:\
    20:35:50.955 IFPServer.C 319 EVENT: Attempting to stop IRT Access

> 20:35:53.015 IrtAccess.py 197 EVENT: IRT access: okay unregister t=
> 0.023 {\'protocol\': 20070723, \'wfos\':
> \'ABQ,AMA,BOU,CYS,DDC,GJT,GLD,LBF,PUB,RIW\', \'site\': \'PUB\',
> \'server\': \'localhost\', \'mhsid\': \'PUB\', \'port\': 98000044}
>
> 20:35:53.016 IrtAccess.py 98 EVENT: Unregistered from IRT
>
> 20:35:53.016 IFPServer.C 322 EVENT: Finished with IRT Access

5)  This step tests the "Send ISC On Save", the iscExtract and
    iscDataRec script, the ifpnetCDF and iscMosaic programs, and the
    getaddrs() call for the IRT web service. Bring up the ifpServer.
    Bring up the GFE and load both the Fcst and ISC databases for a
    scalar (e.g., T), vector (e.g., Wind), discrete (e.g., Hazards), and
    weather (e.g., Wx) parm.  Create some grids containing various
    values for each of the weather elements (e.g., scalar, vector,
    discrete, weather).  Save them.  Verify that they appear in the ISC
    database within a minute or two by watching the Grid Manager display
    of the ISC database.

6)  This step tests the ISC request/reply feature of the GFE and the
    getservers() call of the IRT. To test this feature, the ISC database
    must be purged. Purge the ISC database using the purgeAllData
    program, with a command similar to "./purgeAllData -d
    BOU_GRID\_\_ISC_00000000_0000", where BOU should be replaced with
    your site identifier.  Verify that there are no grids in the ISC
    database by examining the database through the Grid Manager. Use the
    GFE menu entry Consistency-\>ISC Request/Reply to request ISC data
    from your own server.  The dialog should list your own ifpServer for
    your site. Verify that the missing grids appear in the ISC database
    within a minute or two.   Examine the ISC/Fcst databases and ensure
    all grids are now sync\'d, i.e., same inventory of grids and same
    contents (within your forecast area of responsibility) for the ISC
    and Fcst databases.

7)  VTEC Active Table Sharing uses the ISC Routing information. To test
    the set of requestAT, sendAT, ingestAT scripts, a modification on
    the released sendAT script must be made. If the modification is not
    made the s/w will realize that the requested active table contains
    the same data as the requesting server\'s active table and thus will
    not loopback. Modify the released bin/run/sendAT script with a text
    editor and change FORCE_SEND to True.   Create a
    etc/SITE/localVTECPartners with the following information in it (a
    copy of this file is available in the GFESuite software tree under
    GFESuite/scripts/test/localVTECPartnersLB.py):

> import VTECPartners\
> from VTECPartners import \*\
> VTEC_REMOTE_TABLE_FETCH_TIME = 60\
> VTEC_RESPOND_TO_TABLE_REQUESTS = 1\
> VTEC_TABLE_REQUEST_SITES = \[\'PUB\'\] *use your configured site*

8)  In order to test the VTEC table sharing, the ifpServer needs to be
    restarted. Shortly after the restart the s/w will perform a
    requestAT which will start the sequence of events. After the
    ifpServer is started, examine the ifpServer, requestAT, sendAT,
    iscDataRec, and ingestAT log files located in the
    data/logfiles/YYYYMMDD directory to ensure that a request is made
    (ifpServer and requestAT log files), sent out and looped back,
    request honored, and a reply sent back and looped back (sendAT log
    file), and then ingested with ingestAT (ingestAT log file).\
    \
    Here is an example of the ifpServer log file:\
    \
    15:04:51.047 ifpServer.C 595 EVENT: Starting requestAT process\
    \
    Here is an example of the requestAT log file:\
    \
    15:04:51.529 requestAT 35 EVENT: \*\*\*\*\*\*\*\*\*\*\* requestAT
    \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
    \[\'/scratch/TRUNK/GFESuite/release/bin/run/requestAT\', \'-H\',
    \'localhost\', \'-P\', \'98000044\', \'-L\', \'20070723\', \'-M\',
    \'PUB\', \'-S\', \'PUB\', \'-t\', \'http://0.0.0.0:8080\', \'-x\',
    \'/scratch/TRUNK/GFESuite/release/bin/run/iscDataRec MSGID %SUBJECT
    %ATTACHMENTS\'\]

> 15:04:51.552 Client.C 163 EVENT: Establishing connection to server on
> localhost:98000044 ver=20070723
>
> 15:04:51.554 Client.C 183 EVENT: Connection complete to server on
> localhost
>
> 15:04:51.637 Client.C 205 VERBOSE: Dropping Server Connection
>
> 15:04:51.638 requestAT 123 EVENT: Data: (\'PUB\', \'PUB\', \[\'KBOU\',
> \'KGLD\', \'KRIW\', \'KDDC\', \'KLBF\', \'KABQ\', \'KCYS\', \'KAMA\',
> \'KGJT\'\], {\'KBOU\': 18}, 1193260380.0)
>
> 15:04:51.638 requestAT 140 EVENT: Requesting Server:
> mhs=PUB,host=localhost,port=98000044,proto=20070723,site=PUB
>
> 15:04:51.654 IrtAccess.py 197 EVENT: IRT access: okay getservers t=
> 0.014 {\'wfoids\': \'PUB\'}
>
> 15:04:51.669 requestAT 234 EVENT: Matching Servers:
>
> mhs=PUB,host=localhost,port=98000044,proto=20070723,site=PUB
>
> 15:04:51.669 requestAT 240 EVENT: Chosen Servers:
>
> mhs=PUB,host=localhost,port=98000044,proto=20070723,site=PUB
>
> 15:04:51.674 requestAT 260 EVENT: Transmit:
> /scratch/TRUNK/GFESuite/release/bin/run/iscDataRec MSGID
> GET_ACTIVE_TABLE2
> /scratch/TRUNK/GFESuite/release/products/ATBL/tmp-YpEb9.reqat,/scratch/TRUNK/GFESuite/release/products/ATBL/tmp7ELNLP.xml\
> \
> Here is an example of the sendAT log file:\
> \
> 15:04:52.826 sendAT 45 EVENT: \*\*\*\*\*\*\*\*\*\*\* sendAT
> \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
>
> 15:04:52.827 sendAT 59 EVENT: Cmd: \[\'-s\', \'PUB\', \'-a\', \'PUB\',
> \'-H\', \'localhost\', \'-P\', \'98000044\', \'-L\', \'20070723\',
> \'-M\', \'PUB\', \'-S\', \'PUB\', \'-x\',
> \'/scratch/TRUNK/GFESuite/release/bin/run/iscDataRec MSGID %SUBJECT
> %ATTACHMENTS\', \'-f\', \'KBOU\', \'-f\', \'KGLD\', \'-f\', \'KRIW\',
> \'-f\', \'KDDC\', \'-f\', \'KLBF\', \'-f\', \'KABQ\', \'-f\',
> \'KCYS\', \'-f\', \'KAMA\', \'-f\', \'KGJT\', \'-c\', \"{\'KBOU\':
> 18}\", \'-t\', \'1193260380.0\', \'-v\',
> \'/scratch/TRUNK/GFESuite/release/products/ATBL/tmpqXQYyv.ato\',
> \'-X\',
> \'/scratch/TRUNK/GFESuite/release/products/ATBL/tmp68FFTm.xml\'\]
>
> 15:04:52.828 sendAT 100 EVENT: reqSite= \[\'PUB\'\]
>
> 15:04:52.828 sendAT 101 EVENT: filterSites= \[\'KBOU\', \'KGLD\',
> \'KRIW\', \'KDDC\', \'KLBF\', \'KABQ\', \'KCYS\', \'KAMA\', \'KGJT\'\]
>
> 15:04:52.828 sendAT 102 EVENT: mhsSite= \[\'PUB\'\]
>
> 15:04:52.829 sendAT 103 EVENT: reqCountDict= {\'KBOU\': 18}
>
> 15:04:52.829 sendAT 108 EVENT: reqIssueTime= 1193260380.0 Wed Oct 24
> 21:13:00 2007
>
> 15:04:52.829 sendAT 113 EVENT: MyServer:
> mhs=PUB,host=localhost,port=98000044,proto=20070723,site=PUB
>
> 15:04:52.830 sendAT 124 EVENT: Local Table Length= 18
>
> 15:04:52.831 sendAT 136 EVENT: Site Filtered Table Length= 18
>
> 15:04:52.832 sendAT 143 EVENT: Squeezed Table Length= 18
>
> 15:04:52.832 sendAT 157 EVENT: NewestFound= 1193260380.0 Wed Oct 24
> 21:13:00 2007
>
> 15:04:52.832 sendAT 158 EVENT: IssueTime check. Newer record found=
> False
>
> 15:04:52.833 sendAT 177 EVENT: MissingRec check. Missing record found=
> False
>
> 15:04:52.833 sendAT 178 EVENT: lclCountBySite= {\'KBOU\': 18}
>
> 15:04:52.833 sendAT 179 EVENT: reqCountBySite= {\'KBOU\': 18}
>
> 15:04:52.836 sendAT 201 EVENT: #dataSize: 3991 #gzipSize: 806
>
> 15:04:52.871 sendAT 243 EVENT: Destinations:
>
> mhs=PUB,host=localhost,port=98000044,proto=20070723,site=PUB
>
> 15:04:52.873 sendAT 278 EVENT: Transmit:
> /scratch/TRUNK/GFESuite/release/bin/run/iscDataRec MSGID
> PUT_ACTIVE_TABLE2
> /scratch/TRUNK/GFESuite/release/products/ATBL/tmpqXQYyv.ato.gz,/scratch/TRUNK/GFESuite/release/products/ATBL/tmpqy5iXr.xml
>
> 15:04:53.236 sendAT 303 EVENT: Final: wctime: 0.41 ,cputime: 0.26\
> \
> Here is an example of the ingestAT log file:\
> \
> 15:04:53.573 ingestAT 35 EVENT: \*\*\*\*\*\*\*\*\*\*\*\*\* ingestAT
> \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
>
> 15:04:53.574 ingestAT 44 EVENT: Cmd: \[\'-f\',
> \'/scratch/TRUNK/GFESuite/release/products/ATBL/tmpVZJEqS.ati\',
> \'-X\',
> \'/scratch/TRUNK/GFESuite/release/products/ATBL/tmp87-Hcl.xml\'\]
>
> 15:04:53.582 ingestAT 83 EVENT: Source Server:
> mhs=PUB,host=localhost,port=98000044,proto=20070723,site=PUB
>
> 15:04:53.583 MergeVTEC.py 269 VERBOSE: Filter Sites: \[\'KBOU\',
> \'KGLD\', \'KRIW\', \'KDDC\', \'KLBF\', \'KABQ\', \'KCYS\', \'KAMA\',
> \'KGJT\', \'KWNS\', \'KNHC\', \'KPUB\'\]
>
> 15:04:53.583 MergeVTEC.py 47 EVENT: MergeVTEC Starting
>
> 15:04:53.584 MergeVTEC.py 50 EVENT: Build Date: Wed Oct 24 17:12:26
> 2007 Built By: mark Built On: camper.fsl.noaa.gov (linux2) Version:
> development
>
> 15:04:53.584 MergeVTEC.py 52 EVENT: remoteFN=
> /scratch/TRUNK/GFESuite/release/products/ATBL/tmpVZJEqS.ati localFN=
> /scratch/TRUNK/GFESuite/release/data/vtec/active.tbl sites=
> \[\'KBOU\', \'KGLD\', \'KRIW\', \'KDDC\', \'KLBF\', \'KABQ\',
> \'KCYS\', \'KAMA\', \'KGJT\', \'KWNS\', \'KNHC\', \'KPUB\'\]
>
> 15:04:53.586 MergeVTEC.py 61 EVENT: Remote Table size: 18
>
> 15:04:53.587 VTECTableUtil.py 282 VERBOSE: LOCK granted. Wait: 0.000
> sec.
>
> 15:04:53.589 MergeVTEC.py 68 EVENT: Active Table size: 18
>
> 15:04:53.593 MergeVTEC.py 76 EVENT: Active Table squeezed size: 18
>
> 15:04:53.593 MergeVTEC.py 77 EVENT: Other Table size: 18
>
> 15:04:53.594 MergeVTEC.py 79 EVENT: Other Table squeezed size: 18
>
> 15:04:53.596 MergeVTEC.py 255 VERBOSE: Table Changes: \[\]
>
> 15:04:53.596 MergeVTEC.py 83 EVENT: Updated Active Table size: 18
>
> 15:04:53.597 MergeVTEC.py 86 EVENT: Updated Active Table squeeze size:
> 18
>
> 15:04:53.599 VTECTableUtil.py 307 VERBOSE: LOCK duration: 0.012 sec.
>
> 15:04:53.600 MergeVTEC.py 116 EVENT: MergeVTEC Finished
>
> 15:04:53.600 ingestAT 106 EVENT: Final: wctime: 0.03 ,cputime: 0.30\
> \
> Here is an example of the iscDataRec log file showing two calls to the
> iscDataRec script for the bounce back with the subjects
> GET_ACTIVE_TABLE2 and PUT_ACTIVE_TABLE2:\
> \
> iscDataRec1 6165 15:04:52.053 iscDataRec 65 EVENT: \*\*\* iscDataRec
> \*\*\* \[\'MSGID\', \'GET_ACTIVE_TABLE2\',
> \'/scratch/TRUNK/GFESuite/release/products/ATBL/tmp-YpEb9.reqat,/scratch/TRUNK/GFESuite/release/products/ATBL/tmp7ELNLP.xml\'\]
>
> iscDataRec1 6165 15:04:52.053 iscDataRec 71 EVENT: SUBJECT:
> GET_ACTIVE_TABLE2 MSGID: MSGID
>
> iscDataRec1 6165 15:04:52.054 iscDataRec 75 EVENT: file received:
> /scratch/TRUNK/GFESuite/release/products/ATBL/tmp-YpEb9.reqat 178
>
> iscDataRec1 6165 15:04:52.054 iscDataRec 75 EVENT: file received:
> /scratch/TRUNK/GFESuite/release/products/ATBL/tmp7ELNLP.xml 1320
>
> iscDataRec1 6165 15:04:52.088 iscDataRec 214 EVENT: GET_ACTIVE_TABLE2
> Sent to: localhost port= 98000044 proto= 20070723 connectT= 0.0 xmtT=
> 0.01 size= 1320 filename=
> /scratch/TRUNK/GFESuite/release/products/ATBL/tmp-YpEb9.reqat
>
> iscDataRec1 6175 15:04:53.193 iscDataRec 65 EVENT: \*\*\* iscDataRec
> \*\*\* \[\'MSGID\', \'PUT_ACTIVE_TABLE2\',
> \'/scratch/TRUNK/GFESuite/release/products/ATBL/tmpqXQYyv.ato.gz,/scratch/TRUNK/GFESuite/release/products/ATBL/tmpqy5iXr.xml\'\]
>
> iscDataRec1 6175 15:04:53.194 iscDataRec 71 EVENT: SUBJECT:
> PUT_ACTIVE_TABLE2 MSGID: MSGID
>
> iscDataRec1 6175 15:04:53.194 iscDataRec 75 EVENT: file received:
> /scratch/TRUNK/GFESuite/release/products/ATBL/tmpqXQYyv.ato.gz 806
>
> iscDataRec1 6175 15:04:53.195 iscDataRec 75 EVENT: file received:
> /scratch/TRUNK/GFESuite/release/products/ATBL/tmpqy5iXr.xml 315
>
> iscDataRec1 6175 15:04:53.233 iscDataRec 214 EVENT: PUT_ACTIVE_TABLE2
> Sent to: localhost port= 98000044 proto= 20070723 connectT= 0.01 xmtT=
> 0.0 size= 315 filename=
> /scratch/TRUNK/GFESuite/release/products/ATBL/tmpqXQYyv.ato.gz

9)  Restore the original version of the sendAT script by setting
    FORCE_SEND to False.

10) Remove the etc/SITE/localVTECPartners.\* files. Restart the
    ifpServer and GFE.

11) The ifpServer will periodically reregister itself with the IRT web
    service. Examine the ifpServer log file (search for \'Re-register
    time\') for the re-registration time provided by the IRT web
    service. Wait until that time is past and re-examine the preserved
    log file to ensure that a re-registration occurs. An example of the
    logged re-register time from the ifpServer log is:\
    \
    00:42:41.700 IrtAccess.py 168 EVENT: IRT Registration Successful.
    Re-register time: Thu Oct 25 01:42:41 2007

## ***Multiple ifpServer Testing***

### **Purpose:** 

To test multiple GFESuite installations, with an isolated IRT web
service in isolation, and not using the AWIPS WAN or AWIPS equipment.
Components tested include Request/Reply, VTEC active table sharing, and
ISC traffic. 

### **Environment:**

-   IRT web service, installed on a common host accessible by all of the
    GFESuites.

-   Test mmhs script executed on each one of the machines representing
    an AWIPS host. This script is located in the GFESuite tree as
    GFESuite/scripts/test/mmhs. The link for msg_send must be installed.
    The script is started using mmhs --server=WANID
    \--iscdr=/path/to/iscDataRec"

-   GFESuite installations -- multiple sites on multiple hosts. Each
    host should have more than one GFE installation. The sites are
    chosen to be in a line to make it easy to verify the proper routing
    of ISC data. The "mhsid" represents a set of machines, on which the
    GFESuites for various sites are installed, and is similar to the
    AWIPS mhsid which is common for all hosts at a site.

-   All GFESuites configured to use IRT web service, enabling send on
    save, enabling isc request

-   All GFESuites configured to enable VTEC active table sharing.

    -   All GFESuites configured to use a special transmission script
        via the TRANSMIT_SCRIPTS config item.\
        \
        Recommended Site configuration. Note on the installGFE command,
        the -m switch provides the message handler id; for purposes of
        our internal transfer script (mmhs), we simulate AWIPS nodes
        that are running several ifpServers by using the -m switch
        during the install. All machines must be on the same subnet.
        These particular sites are set up so that ISC traffic from some
        of the machines should never get to other machines since their
        domains are too far away. There are duplicate sites, which are
        used to confirm multiple choices with request/reply. The sites
        were also chosen to ensure that the configuration matches the
        backup site configuration in the VTECPartners. The "Send ISC
        During Test" flag is used to denote that this server is running
        "operationally". If "Send ISC During Test" is set to no, then
        the installation simulates a service backup site that hasn\'t
        taken over yet and no ISC grids should be sent from that
        installation for the tests. The configuration in the table below
        represents 4 WFOs (BOU, GLD, TOP, and EAX) with each WFO having
        2 or 3 ifpServer domains (BOU, PUB, GLD, DDC, TOP, ICT, EAX).

  ------------ ------------ ----------------- --------- --------------------------
  MHS ID       Configured   Host/Port         Send ISC  Installation Directory
  (awips       Site                           During    
  simulated)                                  Test?     

  BOU          BOU          camper/98000050   yes       /scratch/test/BOU

  BOU          PUB          camper/98000051   yes       /scratch/test/PUB

  GLD          GLD          smaug/98000052    yes       /scratch/test/GLD

  GLD          DDC          smuag/98000053    yes       /scratch/test/DDC

  GLD          BOU          smaug98000054     no        /scratch/test/BOU

  TOP          TOP          maia/98000055     yes       /scratch/test/TOP

  TOP          ICT          maia/98000056     yes       /scratch/test/ICT

  EAX          EAX          quint/98000057    yes       /scratch/test/EAX

  EAX          TOP          quint/98000058    no        /scratch/test/TOP

  EAX          ICT          quint/98000059    no        /scratch/test/ICT
  ------------ ------------ ----------------- --------- --------------------------

### **Procedure:**

1)  Install IRT web service on camper (example machine). Start the web
    service using the direct "python RoutingTableSvc.py" command.

2)  Install all of the GFESuites per the above configuration table,
    using different nodes and installation directories as needed. Do not
    start the ifpServers. For each installation, create a
    etc/SITE/localConfig.py override file containing the following lines
    to set up the mmhs (miniature message handling system). (Note a
    version of this localConfig is located in the GFESuite source tree
    under GFESuite/scripts/test/localConfigMMHS.py):\
    \
    from serverConfig import \*\
    import serverConfig\
    serverConfig.ISC_ROUTING_TABLE_ADDRESS =
    "[http://camper.fsl.noaa.gov:8080](http://0.0.0.0:8080/)" *(modify
    address as needed, to point to the IRT web service)\
    *serverConfig.REQUEST_ISC = 1\
    serverConfig.SEND_ISC_ON_SAVE = 1\
    serverConfig.TRANSMIT_SCRIPT = GFESUITE_HOME + \'/bin/run/msg_send
    -s %SUBJECT -a %ADDRESSES -i %WMOID -c 11 -p 0 -e %ATTACHMENTS\'

3)  VTEC Active Table Sharing uses the ISC Routing information. Create a
    etc/SITE/localVTECPartners with the following information in it (a
    copy of this file is available in the GFESuite software tree under
    GFESuite/scripts/test/localVTECPartnersMMHS.py):\
    \
    import VTECPartners\
    from VTECPartners import \*\
    VTEC_REMOTE_TABLE_FETCH_TIME = 60\
    VTEC_RESPOND_TO_TABLE_REQUESTS = 1

4)  Since this test is not using the real AWIPS WAN, we must simulate
    it. The mmhs simulation package allows transmissions from each
    ifpServer to be routed to other ifpServers on the network according
    to their configured "mhsid", which emulates the AWIPS WAN. You will
    need to run the mmhs script once per machine (to simulate the mhs
    nodes). The script will not exit until you abort it, and it must be
    running for the tests to be successful. The mmhs script is located
    in the GFESuite source tree under GFESuite/scripts/test/mmhs. Copy
    the script into each of the installation bin/run directories marked
    in the above configuration table. Create a symbolic link in the
    bin/run directory called msg_send pointing to the mmhs script (using
    ln -s mmhs msg_send). Start the mmhs script on each of the machines.
    The command for each of the four nodes looks like this:

  ------------- -------- -----------------------------------------------------
  MHSid         Host     Command
  (simulated)            

  BOU           camper   cd /scratch/test/BOU/bin/run\
                         mmhs --server=BOU
                         \--iscdr=/scratch/test/BOU/bin/run/iscDataRec"

  GLD           smaug    cd /scratch/test/GLD/bin/run\
                         mmhs --server=GLD
                         \--iscdr=/scratch/test/GLD/bin/run/iscDataRec"

  TOP           maia     cd /scratch/test/TOP/bin/run\
                         mmhs --server=TOP
                         \--iscdr=/scratch/test/TOP/bin/run/iscDataRec"

  EAX           quint    cd /scratch/test/EAX/bin/run\
                         mmhs --server=EAX
                         \--iscdr=/scratch/test/EAX/bin/run/iscDataRec"
  ------------- -------- -----------------------------------------------------

5)  Start the various ifpServers.

6)  To test the proper registration of the ifpServers to the IRT Routing
    Table and the IRT register() call, bring up the ifpServer. Use the
    IRT web interface to verify that all of the servers are registered. 
    Examine each of the GFESuite\'s data/logfiles/YYYYMMDD/ifpServer\*
    log for registration information. You should see an entry similar
    to:\
    00:42:41.698 IrtAccess.py 197 EVENT: IRT access: okay register t=
    0.064 {\'protocol\': 20070723, \'wfos\':
    \'ABQ,AMA,BOU,CYS,DDC,GJT,GLD,LBF,PUB,RIW\', \'site\': \'PUB\',
    \'server\': \'localhost\', \'dims\': \'145,145\', \'proj\':
    \'Grid211\', \'mhsid\': \'PUB\', \'parms\':
    \'RH,HrsOfSun,WindWaveHgt,FreeWind,DSI,IceAccum,WaveHeight,Swell,Wind20ft,WindChill,Sky,Wx,T,StormTotalSnow,IceCoverage,SnowAmt,Period2,VentRate,Wetflag,WindGust,MinT,Hazards,RHtrend,Td,PoP,TransWind,QPF,Haines,SurfHeight,MixHgt,Swell2,HeatIndex,MarineLayer,MaxT,LAL,MaxRH,Ttrend,SnowLevel,InvBurnOffTemp,Stability,FzLevel,MinRH,Period,CWR,Wind\',
    \'port\': 98000044, \'Bbox\': \'38.0,26.0,9.0,9.0\'}\
    \
    00:42:41.700 IrtAccess.py 168 EVENT: IRT Registration Successful.
    Re-register time: Thu Oct 25 01:42:41 2007

7)  To test the proper unregistration of the ifpServers and the IRT
    unregister() call, shut down several of the ifpServers.  Use the IRT
    web interface to verify that those servers has been unregistered. 
    Examine the specific server data/logfiles/YYYYMMDD/ifpServer\* logs
    for unregistration information. You should see an entry similar to:\
    20:35:50.955 IFPServer.C 319 EVENT: Attempting to stop IRT Access\
    20:35:53.015 IrtAccess.py 197 EVENT: IRT access: okay unregister t=
    0.023 {\'protocol\': 20070723, \'wfos\':
    \'ABQ,AMA,BOU,CYS,DDC,GJT,GLD,LBF,PUB,RIW\', \'site\': \'PUB\',
    \'server\': \'localhost\', \'mhsid\': \'PUB\', \'port\': 98000044}\
    20:35:53.016 IrtAccess.py 98 EVENT: Unregistered from IRT\
    20:35:53.016 IFPServer.C 322 EVENT: Finished with IRT Access

8)  This step tests the "Send ISC On Save", the iscExtract and
    iscDataRec script, the ifpnetCDF and iscMosaic programs, and the
    getaddrs() call for the IRT web service. Bring up the ifpServers
    that were previously shut down. Bring up all of the GFEs and load
    both the Fcst and ISC databases for a scalar (e.g., T), vector
    (e.g., Wind), discrete (e.g., Hazards), and weather (e.g., Wx)
    parm.  For only those GFEs listed in the configuration table that
    indicate "Send ISC During Test", create some grids containing
    various values for each of the weather elements (e.g., scalar,
    vector, discrete, weather).  Save them.  Examine all of the
    surrounding site\'s GFEs ISC database (or use \'Show ISC Mode\') to
    verify that the ISC data was received. Also verify that grids sent
    appear in their own ISC database.

9)  This step tests the ISC request/reply feature of the GFE and the
    getservers() call of the IRT. To test this feature, the ISC database
    must be purged. For each ifpServer, purge the ISC database using the
    purgeAllData program, with a command similar to "./purgeAllData -d
    BOU_GRID\_\_ISC_00000000_0000", where BOU should be replaced with
    your site identifier.  For each GFE, verify that there are no grids
    in the ISC database by examining the database through the Grid
    Manager. For each GFE, use the GFE menu entry Consistency-\>ISC
    Request/Reply to request ISC data for domains.  The choice of server
    normally is automatically selected on the real AWIPS since it
    detects the standard configuration of dx4/px3 98000000/98000001.
    There are no standard configurations for this test so the
    pre-selected values may not be correct. Select all of the domains,
    then ensure that the shown servers match what is shown in the
    configuration table above. Change the server selection as necessary
    to ensure that all of the selected servers are listed in the
    configuration table as "Send ISC During Test" with "yes". The dialog
    should also list your own ifpServer for your site -- select it if
    you are listed as "Send ISC During Test" with "yes", otherwise
    select your domain and another server with "yes" in the table.
    Verify that the missing grids appear in the ISC database within a
    minute or two.

10) VTEC Active Table Sharing uses the ISC Routing information. To test
    the set of requestAT, sendAT, ingestAT scripts, shut down one or two
    GFEs for different domains and bring them back up in PRACTICE mode.
    Create a hazard product and transmit it. Shut the GFEs back down and
    bring them back up in operational mode. Start the GHG Monitor. On
    each system where a product was transmitted, physically copy the
    \.../data/vtec/PRACTICE.tbl file over the \.../data/vtec/active.tbl
    file. This will cause the vtec tables to be different on several of
    the hosts. After about a minute, the VTEC sharing will cycle and
    other servers will detect that the vtec tables are out-of-sync.
    Watch the GHG Monitor on the other GFEs, you should see the events
    arrive as the sharing works if the event is within your neighboring
    area. Examine the requestAT, sendAT, and ingestAT log files on
    various systems. They should indicate who the request was made to,
    who the reply was sent to, and finally the ingestAT step where the
    vtec tables being sync\'d.

11) To clean up, shut down the GFEs, the ifpServers, and the mmhs
    handler. Remove the temporary installation.

## ***AWIPS Environment Testing***

### **Purpose:** 

To test multiple GFESuite installations on AWIPS, using an isolated IRT
web service. Components tested include Request/Reply, VTEC active table
sharing, and ISC traffic. 

### Caution:

-   If the configuration of the test is not done carefully, it is
    possible to corrupt operational AWIPS systems by sending ISC and
    VTEC data to these systems. Care must be taken to understand the
    configuration to ensure that no operational sites will be affected
    by this test.

### **Environment:**

-   IRT web service, installed on a common host accessible by all of the
    GFESuites. Recommend this is installed on the actual central server
    machine in the normal location (if there are no operational OB8.3
    sites on the network), or in an alternate location with an alternate
    URL (if there are operational OB8.3 sites on the network).

-   Two AWIPS WFO installations. The installations must represent
    neighboring sites to fully test the functionality. If the primary
    installations do not represent neighboring sites, then only limited
    testing can only occur for the two primary sites. Later in the test
    we bring up service backup and configure it for two other adjacent
    sites. At a minimum you have three adjacent sites while the best
    situation is for four adjacent sites. Note: you can modify the site
    configuration of a GFESuite without redoing a complete AWIPS
    installation by changing the contents of the GFESuite\'s two files:
    \.../etc/BASE/siteConfig.py, \.../bin/run/site.sh, removing all
    databases under \.../data/databases/BASE/GRID, and restarting the
    ifpServer.

-   All GFESuites configured to use IRT web service, enabling send on
    save, enabling isc request

-   All GFESuites configured to enable VTEC active table sharing.

    -   All GFESuites configured to use the standard msg_send AWIPS
        script via the TRANSMIT_SCRIPTS config item, which is the
        default configuration.

### Procedure:

1)  Ensure the IRT web service is installed and enabled. It is best to
    have it configured through the apache web server, as in normal
    operations. You might want to use an isolated version if there are
    OB8.3 sites already using the IRT web service.

2)  Ensure the AWIPS installation is properly done on two different
    nodes and that the primary ifpServers are running and the svcbu
    ifpServers are not running. For the two primary ifpServers, create a
    etc/SITE/localConfig.py override file containing the following
    lines:\
    \
    from serverConfig import \*\
    import serverConfig\
    serverConfig.ISC_ROUTING_TABLE_ADDRESS =
    "[http://0.0.0.0:8080](http://0.0.0.0:8080/)" *(modify address as
    needed, to point to the IRT web service)\
    *serverConfig.REQUEST_ISC = 1\
    serverConfig.SEND_ISC_ON_SAVE = 1

3)  VTEC Active Table Sharing uses the ISC Routing information. Create a
    etc/SITE/localVTECPartners for each of the primary ifpServers with
    the following information in it:\
    \
    import VTECPartners\
    from VTECPartners import \*\
    VTEC_REMOTE_TABLE_FETCH_TIME = 60\
    VTEC_RESPOND_TO_TABLE_REQUESTS = 1\
    VTEC_TABLE_REQUEST_SITES = \[\'xxx\',\'yyy\',\'zzz\'\] *(modify
    addresses as needed to represent other sites/domains represented by
    your test cluster, but not your site)*

4)  On each of the primary ifpServers, replace the active.tbl (VTEC
    table) with the empty.tbl (Empty VTEC table) in order to purge all
    records. The command is:\
    \
    cd /awips/GFESuite/primary/data/vtec\
    cp empty.tbl active.tbl

5)  Restart the primary ifpServers on both AWIPS hosts.

6)  Use the IRT web interface to verify that the two primary servers are
    registered.  Examine each of the GFESuite\'s
    /awips/GFESuite/primary/data/logfiles/YYYYMMDD/ifpServer\* log for
    registration information. You should see an entry similar to:\
    00:42:41.698 IrtAccess.py 197 EVENT: IRT access: okay register t=
    0.064 {\'protocol\': 20070723, \'wfos\':
    \'ABQ,AMA,BOU,CYS,DDC,GJT,GLD,LBF,PUB,RIW\', \'site\': \'PUB\',
    \'server\': \'localhost\', \'dims\': \'145,145\', \'proj\':
    \'Grid211\', \'mhsid\': \'PUB\', \'parms\':
    \'RH,HrsOfSun,WindWaveHgt,FreeWind,DSI,IceAccum,WaveHeight,Swell,Wind20ft,WindChill,Sky,Wx,T,StormTotalSnow,IceCoverage,SnowAmt,Period2,VentRate,Wetflag,WindGust,MinT,Hazards,RHtrend,Td,PoP,TransWind,QPF,Haines,SurfHeight,MixHgt,Swell2,HeatIndex,MarineLayer,MaxT,LAL,MaxRH,Ttrend,SnowLevel,InvBurnOffTemp,Stability,FzLevel,MinRH,Period,CWR,Wind\',
    \'port\': 98000044, \'Bbox\': \'38.0,26.0,9.0,9.0\'}\
    \
    00:42:41.700 IrtAccess.py 168 EVENT: IRT Registration Successful.
    Re-register time: Thu Oct 25 01:42:41 2007

7)  This step tests the "Send ISC On Save", the iscExtract and
    iscDataRec script, the ifpnetCDF and iscMosaic programs, and the
    getaddrs() call for the IRT web service. Bring up a GFE for each of
    the AWIPS installations and load both the Fcst and ISC databases for
    a scalar (e.g., T), vector (e.g., Wind), discrete (e.g., Hazards),
    and weather (e.g., Wx) parm.  Create some grids containing various
    values for each of the weather elements (e.g., scalar, vector,
    discrete, weather).  Save them.  Examine the GFEs ISC database to
    verify that the ISC data was looped back and appear in their own ISC
    database. Unless the primary AWIPS installations are configured for
    adjacent sites, you will only see your own grids come back into the
    GFE. If the primary installations are configured for adjacent sites,
    then you will see your transmitted grids appearing in the other
    site\'s ifpServer/GFE.

8)  This step tests the ISC request/reply feature of the GFE and the
    getservers() call of the IRT. To test this feature, the ISC database
    must be purged. For each primary ifpServer, purge the ISC database
    using the purgeAllData program (located in /awips/GFESuite/primary),
    with a command similar to "./purgeAllData -d
    BOU_GRID\_\_ISC_00000000_0000", where BOU should be replaced with
    your site identifier.  For each GFE, verify that there are no grids
    in the ISC database by examining the database through the Grid
    Manager. For each GFE, use the GFE menu entry Consistency-\>ISC
    Request/Reply to request ISC data for domains.  The choice of server
    normally is automatically selected on the real AWIPS since it
    detects the standard configuration of dx4/px3 98000000/98000001. In
    the isolated non-operational test, there will be few servers listed.
    Select all of the possible domains and default servers listed. The
    dialog will also list your own ifpServer for your site. Verify that
    the missing grids appear in the ISC database within a minute or two.

9)  VTEC Active Table Sharing uses the ISC Routing information. To test
    the set of requestAT, sendAT, ingestAT scripts, examine the
    requestAT and ingestAT log files. The ifpServer restart in an
    earlier step should already have triggered the requestAT and
    ingestAT mechansism. Start the GHG Monitor, which may show you the
    events currently synchronized. Synchronization may not have occurred
    since the ifpServer will attempt to find an active ifpServer
    representing the site listed in VTECPartners.py -- and there likely
    are no sites running the current software in the field matching that
    site identifier.

10) Determine the site identifier for one of the active primary
    ifpServers. Start service backup for that svcbu ifpServer for an
    adjacent site. Start service backup for the other svcbu ifpServer
    for another adjacent site. Now there should be three ifpServers that
    represent three adjacent sites. \[Note: it is likely that the fourth
    ifpServer is not adjacent.\]

11) For the two service backup ifpServers, shut them down, add these
    additional lines to any etc/SITE/localConfig.py that is found:\
    \
    import serverConfig\
    from serverConfig import \*\
    serverConfig.ISC_ROUTING_TABLE_ADDRESS =
    "[http://0.0.0.0:8080](http://0.0.0.0:8080/)" *(modify address as
    needed, to point to the IRT web service)\
    *serverConfig.REQUEST_ISC = 1\
    serverConfig.SEND_ISC_ON_SAVE = 1

12) VTEC Active Table Sharing uses the ISC Routing information. For the
    two service backup ifpServers, add these additional lines to any
    etc/SITE/localVTECPartners.py that is found, or add:\
    \
    import VTECPartners\
    from VTECPartners import \*\
    VTEC_REMOTE_TABLE_FETCH_TIME = 60\
    VTEC_RESPOND_TO_TABLE_REQUESTS = 1\
    VTEC_TABLE_REQUEST_SITES = \[\'xxx\',\'yyy\',\'zzz\'\] *(modify
    addresses as needed to represent other sites/domains represented by
    your test cluster, but not your site)*

13) Restart the svcbu ifpServers on the two AWIPS installations.

14) Start the svcbu GFEs on the two AWIPS installations.

15) Use Consistency-\>ISC/RequestReply on the two svcbu GFEs and select
    all domains and all appropriate ifpServers. Verify that ISC data
    appears from the other sites that have sent the data previously.

16) Create some grids and save them on the two svcbu GFEs. Verify that
    they arrive on the primary GFEs if the sites are configured for
    adjacent domains.

17) Create some grids on the primary GFEs and save them. Verify that
    they arrive on the svcbu GFEs if the sites are configured for
    adjacent domains.

18) To test the VTEC active table sharing, bring up a GFE in practice
    mode on any of the GFEs. Create a hazard product and transmit it.
    Manually copy the practice VTEC table (\.../data/vtec/PRACTICE.tbl)
    into the active table slot (\.../data/vtec/active.tbl file). This
    simulates the creation of a product, the transmission of it, and the
    reception of it as the VTEC table becomes updated. On other GFEs,
    bring up the GHG Monitor and verify that the event has been
    synchronized. Synchronization is configured for this test to occur
    every 60 seconds.

19) To test the proper unregistration of the ifpServers and the IRT
    unregister() call, exit service backup on both AWIPS installations. 
    Use the IRT web interface to verify that those servers has been
    unregistered.  Examine the specific server
    data/logfiles/YYYYMMDD/ifpServer\* logs for unregistration
    information. You should see an entry similar to:\
    20:35:50.955 IFPServer.C 319 EVENT: Attempting to stop IRT Access\
    20:35:53.015 IrtAccess.py 197 EVENT: IRT access: okay unregister t=
    0.023 {\'protocol\': 20070723, \'wfos\':
    \'ABQ,AMA,BOU,CYS,DDC,GJT,GLD,LBF,PUB,RIW\', \'site\': \'PUB\',
    \'server\': \'localhost\', \'mhsid\': \'PUB\', \'port\': 98000044}\
    20:35:53.016 IrtAccess.py 98 EVENT: Unregistered from IRT\
    20:35:53.016 IFPServer.C 322 EVENT: Finished with IRT Access