***IVP Software Suite***

***Test Procedure***

By

Henry Herr

Office of Hydrologic Development

National Weather Service

***Table Of Contents***

[General Setup Procedure 3](#__RefHeading___Toc162687944)

[Test 1: Testing Vfyruninfo Editor 4](#__RefHeading___Toc162687945)

[Test 2: Executing the Automated Test 8](#__RefHeading___Toc162687946)

[Test 3: Testing IVP GUI 10](#__RefHeading___Toc162687947)

[**A.** **Test Plan item 1** 10](#__RefHeading___Toc162687948)

[**B.** **Test Plan item 2** 10](#__RefHeading___Toc162687949)

[**C.** **Test Plan item 3** 12](#__RefHeading___Toc162687950)

[**D.** **Test Plan item 4** 12](#__RefHeading___Toc162687951)

[**E.** **Test Plan item 5** 16](#__RefHeading___Toc162687952)

[**F.** **Test Plan item 6** 17](#__RefHeading___Toc162687953)

[**G.** **Test Plan item 7** 17](#__RefHeading___Toc162687954)

[**H.** **Test Plan item 8** 20](#__RefHeading___Toc162687955)

[**I.** **Test Plan item 9** 21](#__RefHeading___Toc162687956)

[**J.** **Test Plan item 10** 22](#__RefHeading___Toc162687957)

[**K.** **Test Plan item 11** 24](#__RefHeading___Toc162687958)

[**L.** **Test Plan item 12** 24](#__RefHeading___Toc162687959)

[**M.** **Test Plan item 13** 26](#__RefHeading___Toc162687960)

[**N.** **Test Plan item 14** 27](#__RefHeading___Toc162687961)

[**O.** **Test Plan item 15** 29](#__RefHeading___Toc162687962)

[**P.** **Test Plan item 16** 30](#__RefHeading___Toc162687963)

[Test 4: Testing IVP Batch Builder 32](#__RefHeading___Toc162687964)

***Verification Software Test***

[]{#__RefHeading___Toc162687944 .anchor}***General Setup Procedure***

Identify the following:

-   \<test database name\>: The name of the test data base for the
    tested build. Note that this database must be installed and
    populated with data prior to running this test.

-   \<database host\>: The name of the machine that hosts the test data
    base.

-   \<test files directory\>: The directory where the verification
    regression test files are located. It contains subdirectories
    doc/test/input, doc/test/output, doc/test/templates,
    doc/test/screenshots, and scripts. The regression test script is in
    \<test files directory\>/scripts. The other directories contain
    testing files.

Open up a session on the archive machine (for OHD, this is ax2-nhdr).
Type in the following commands at the prompt for that session:

> export apps_dir=/awips/hydroapps/lx
>
> export adb_name=\<test database name\>
>
> export PGHOST=\<database host\>
>
> export PGUSER=pguser
>
> export PGI=/fs/opt/pgi
>
> export rax_pghost=\<database host\>
>
> export pguser=pguser

For OHD development, the \<test database name\> is "vfytest_ob8" and the
\<machine hosting test databse\> is "ax2".

Make sure the following directory exists, in order for the automated
regression test to work:

\$(get_apps_defaults vsys_dir)/input/test

Also, do the following to ensure that the correct input, output, and
files directories are used:

> export vsys_input=
>
> \$(get_apps_defaults vsys_dir)/input
>
> export vsys_output=\$(get_apps_defaults vsys_dir)/output
>
> export vsys_files=\$(get_apps_defaults vsys_dir)/files
>
> export vsys_scripts=\$(get_apps_defaults verify_scripts)

Finally, make sure that the apps-defaults token vsys_scripts now points
to the directory containing the version of the verification software
scripts that are to be tested, and that token util_rls points to the
directory containing the version of the verification software jar files
that are to be tested. Default settings for both tokens are usually
correct.

***Verification Software Test***

[]{#__RefHeading___Toc162687945 .anchor}***Test 1: Testing Vfyruninfo
Editor***

**0. Setup**

Be sure to run the General Setup Procedure prior to executing this test.

**A. Test of Vfyruninfo Editor**

Do the following within the same session as the SETUP:

1.  Action: Delete all rows from the vfyruninfo table by executing this
    command from the command line:

psql -d \<test database name\> -c \"delete from vfyruninfo;\"

Result: The vfyruninfo table will be emptied from the archive database.

2.  Action: Type in the following:

> cd \$(get_apps_defaults vsys_scripts)
>
> ivpruninfo
>
> *NOTE: If this is being run at OHD on ax-nhdr, you must include the
> --r option in the ivpruninfo script ("ivpruninfo --r").*
>
> Result: A window titled "IVPRuninfo Table Editor, v. ob#.# ##/##/##"
> will open up. This window contains two tables: a top table labeled
> "Ingest Filter Table Contents" and a bottom table labeled "Included in
> the Vfyruninfo Table".

3.  Action: From the top table, hold down \<ctrl\> and click on the rows
    for location (lid) "CLKW2", pe "HG", dur "I", extremum "Z" and fcst
    ts "FE" and "FF".

> Result: Two rows of the top table will be selected.

4.  Action: Click on the **Choose** button.

> Result: The two selected rows of the top table will be removed from
> the top table, and will be added, with a green background, to the
> bottom table.

5.  Action: Select the row in the top table for the location (lid)
    "ELRP1". Click on the **Create** button.

> Result: A window title \"Vfyruninfo Record Editor\" will open. It will
> contain 7 text fields. By default, the **Duration** field will be set
> to \"I\", the **Extremum** field will be set to \"Z\", and the
> **Response Time** field will be set to \"MEDIUM\". All other fields
> will be empty.

6.  Action: Enter the following text into each text field, and click
    **OK** when finished:

    -   **Location**: "ELRP1, JHNP1, MTXV2, OLNN6, RGLN4"

    -   **PE**: "HG"

    -   **Duration**: "I"

    -   **Extremum**: "Z\\\\

    -   **Forecast TS**: "FE, FF"

    -   **Sensor Pref List**: \<nothing; leave it blank\>

    -   **Response Time**: "MEDIUM" (this text field should be "MEDIUM"
        by default.

> Result: Rows generated by creating every combination of location, pe,
> duration, extremum, and type source given above will be created and
> inserted into the bottom table. If the created rows in the bottom
> table exist in the top table, they will be removed from the top table.
> There should be 12 rows in now in the bottom table, two for each of
> the locations above, and 2 for location CLKW2.

7.  Action: Select both rows from the bottom table associated with the
    location of CLKW2 and click the **Sensor Prefs** button.

> Result: A window titled \"Set Sensor Preferences\" will open.

8.  Action: In the text field type in the values \"RA,RB\" and click
    **OK**.

> Result: The two selected rows in the bottom table will now have a
> sensor pref column value of \"RA,RB\".

9.  Action: Select the four rows from the bottom table for locations
    CLKW2 and ELRP1, and click on the **Response** button.

> Result: A window titled \"Set Response Time\" will open.

10. Action: From the pull-down choice box, select \"FAST\" and click
    **OK**.

> Result: The four selected rows in the bottom table will now have a
> response column value of \"FAST\".

11. Action: Select both rows from the bottom table associated with the
    location OLNN6 and click the **Response** button.

12. Action: From the pull-down choice, select \"SLOW\" and click **OK**.

> Result: The two selected rows in the bottom table will now have a
> response column value of \"SLOW\".

13. Action: Click **Save**.

> Result: A dialog titled \"Confirm Save\" will open up. It will state
> that the current contents of the vfyruninfo table will be overwritten.

14. Action: Click **Yes**.

> Result: All the rows in the bottom table will now have gray/white
> backgrounds and no rows will be selected.

15. Action: Select the rows for ELRP1 from the bottom table and click
    **Unchoose**.

> Result: The two rows associated with ELRP1 in the bottom table will be
> moved to the top table and will have red/pink backgrounds.

16. Action: Click **Load**.

> Result: The two rows in the top table for location ELRP1 will be
> removed and will be added to the bottom table. All rows for both
> tables will have gray/white backgrounds. The bottom table will be
> exactly the same as it was before performing Step 15.

17. Action: Click **Actions \>\> Exit**.

> Result: The program will exit.

This completes the test of the Vfyruninfo Editor.

**Test Verification**

To verify that the vfyruninfo table has changed, do the following in
Session 2:

(Verify that the psql session is still running)

i.  Type in the following and then press \<Enter\>:

> psql --d \<test database name\> --c "select \* from vfyruninfo;"

ii. There should be 14 rows in the table, as shown:

lid \| pe1 \| pe2 \| dur \| extremum \| fcstts1 \| fcstts2 \| sents1 \|
sents2 \| senrk \| resptime

\-\-\-\-\-\--+\-\-\-\--+\-\-\-\--+\-\-\-\--+\-\-\-\-\-\-\-\-\--+\-\-\-\-\-\-\-\--+\-\-\-\-\-\-\-\--+\-\-\-\-\-\-\--+\-\-\-\-\-\-\--+\-\-\-\-\-\--+\-\-\-\-\-\-\-\-\--

CLKW2 \| H \| G \| I \| Z \| F \| E \| R \| A \| 1 \| FAST

CLKW2 \| H \| G \| I \| Z \| F \| E \| R \| B \| 2 \| FAST

CLKW2 \| H \| G \| I \| Z \| F \| F \| R \| A \| 1 \| FAST

CLKW2 \| H \| G \| I \| Z \| F \| F \| R \| B \| 2 \| FAST

ELRP1 \| H \| G \| I \| Z \| F \| E \| \_ \| \_ \| 1 \| FAST

ELRP1 \| H \| G \| I \| Z \| F \| F \| \_ \| \_ \| 1 \| FAST

JHNP1 \| H \| G \| I \| Z \| F \| E \| \_ \| \_ \| 1 \| MEDIUM

JHNP1 \| H \| G \| I \| Z \| F \| F \| \_ \| \_ \| 1 \| MEDIUM

MTXV2 \| H \| G \| I \| Z \| F \| E \| \_ \| \_ \| 1 \| MEDIUM

MTXV2 \| H \| G \| I \| Z \| F \| F \| \_ \| \_ \| 1 \| MEDIUM

OLNN6 \| H \| G \| I \| Z \| F \| E \| \_ \| \_ \| 1 \| SLOW

OLNN6 \| H \| G \| I \| Z \| F \| F \| \_ \| \_ \| 1 \| SLOW

RGLN4 \| H \| G \| I \| Z \| F \| E \| \_ \| \_ \| 1 \| MEDIUM

RGLN4 \| H \| G \| I \| Z \| F \| F \| \_ \| \_ \| 1 \| MEDIUM

***Verification Software Test***

[]{#__RefHeading___Toc162687946 .anchor}***Test 2: Executing the
Automated Test***

**0. Setup**

Be sure to run the General Setup Procedure prior to executing this test.

**A. Regression Test of Verification Software**

To perform the regression test, testing the IVP Batch Program and
ingestpairs program, login in to a session on the archive machine (for
OHD, this is ax2-nhdr) and do the following:

1.  Action: Type in the following:

> cd \<test files directory\>/scripts
>
> vfy_testit --b \<test files directory\> all
>
> The \<test directory\> is the directory where the test tar package was
> unpacked or the location of the root directory of the checked-out
> subversion repository.
>
> The first line of output to the screen will be as follows:
>
> PLEASE CHECK THESE ENVIRONMENT SETTINGS FOR CORRECTNESS:
>
> Examine the output printed to the screen and make sure all of the
> settings are correct. If so, type in 'y' and press \<ENTER\>.
>
> Result: Text should immediately be printed in stdout. The first line
> should be
>
> ==== EXECUTING TEST OF VERIFICATION SOFTWARE ====
>
> This test script makes many database queries and calls to ivpbatch and
> ingestpairs. As each call is made, a line or two will be printed to
> the screen describing what is being done and, if applicable, the
> command line being executed.

This completes the automated regression test of the IVP Batch Program
and Ingest Pairs program.

**Test Verification**

To verify the results of the run, view the all_differences.txt file
generated by the test script. The full path name of this file is
provided at the bottom of the screen output from the test script. Some
differences are acceptable. These include:

-   The number of rows deleted when deleting rows of the vfyruninfo or
    vfypairs tables.

-   The following files will be listed under the heading "BASELINE FILES
    NOT FOUND IN \< output directory\>":

> \<test files directory\>/output/ivpbatch.imagedata001.dat
>
> \<test files directory\>/output/ivpbatch.imagedata001b.dat
>
> \<test files directory\>/output/ivpbatch.imagedata013.dat
>
> These files are not generated as part of the automated test, but are
> needed for the manual GUI test (the next test).

-   File name differences within the output from the executions due to
    the user being different. For example, the location of batch input
    files depends upon the username of the user.

***Verification Software Test***

[]{#__RefHeading___Toc162687947 .anchor}***Test 3: Testing IVP GUI***

**0. Setup**

Be sure to run the General Setup Procedure prior to executing this test.

In order to have all the needed files in place for this manual test, you
must either run the automated test (Test 2) prior to running this test,
or do the following:

> cd \$(get_apps_defaults vsys_scripts)
>
> vfy_testit --b \<test files directory\> setup

In either case, do the following export to ensure you are using the
correct input file:

> export vsys_input=
>
> \$(get_apps_defaults vsys_dir)/input/test/\$(whoami)

A.  []{#__RefHeading___Toc162687948 .anchor}**Test Plan item 1**

**Make sure the IVP starts.**

1.  Action: Type in the following:

> go vsys_scripts
>
> ivp
>
> Result: The Verification Group Manager window pictured in
> ivp.screenshot001.png will pop up on the screen. If it does not, then
> the test fails.

B.  []{#__RefHeading___Toc162687949 .anchor}**Test Plan item 2**

> **Verify that each field can be edited via the Verification Group
> Manager window using the tools provided.**

1.  Action: Click on **Load From Batch**.

> Result: The file browser window pictured in ivp.screenshot002.png will
> pop up on the screen. Note that the file contents of the initial
> directory may not be the same.

2.  Action: Select the file ivpbatch.graphics.input001.bat and click
    **Open**.

> Result: The Verification Group Manager window will now look like that
> pictured in ivp.screenshot003.png.

3.  Action: Click on the **Fixed** button next to the Start Time text
    field.

> Results: The Choose Date/Time window pictured in ivp.screenshot004.png
> will pop up.

4.  Action: Select the date/time \"April 6, 1999 23:59:59 EST\" by doing
    the following: (1) in the year choice box, select 1999; (2) in the
    month choice box, select \"April\"; (3) dick on day 6; (4) specify
    hour 23 by using the spinner up arrow; (5) type in minute 59 and
    second 59 in the text fields displaying minutes and seconds; (6) in
    the time zone choice box, select "EST"; (7) click **OK**.

> Result: The Start Time text field will reflect the new date.

5.  Action: Click on the **Rel** button next to the End Time text field.

> Results: The Choose Relative Date/Time window pictured in
> ivp.screenshot005.png will pop up.

6.  Action Select the relative date/tim \"\* + 1 weeks 1 days 12 hours\"
    by doing the following: (1) click on the \"+\" radio button; (2) in
    place of \"dd\" in the text field type in \"1\"; (3) click on
    **\>\>**; (4) from the choice box currently display \"weeks\" select
    \"days\"; (5) click on **\>\>**; (6) in the text field type in
    \"12\"; (7) from the choice box select \"hours\"; (8) click on
    **\>\>**; (9) click on **OK**.

> Result: The End Time text field will reflect the new date.

7.  Action: Using the spinner, text field, and choice box next to the
    label \"Time Step:\", specify the time step \"2 weeks\".

8.  Action: Using the spinners, text fields, and choice boxes, specify
    the lead time start to be \"0 days\", lead time end to be \"120
    hours\", and the lead time step to be \"NONE\" (select \"NONE\" from
    the box; the text field will be ignored).

9.  Action: Using the spinner, text fields, and choice boxes, specify
    the issuance time start to be "0 hours", issuance time end to be "24
    hours" and the lead time step to be "12 hours".

10. Action: Click on **Edit List** next to the Fcst Type Sources text
    field.

> Result: The Select FCST_TS Values window pictured in
> ivp.screenshot006.png will pop up.

11. Action: Select \"ALL\" from the list and click **OK**.

> Result: The Fcst Type Sources text field will reflect the new value of
> \"ALL\".

12. Action: Select the River Response to include slow and medium rivers
    by doing the following: (1) click on the \"ALL\" radio button (all
    radio buttons should now be enabled); (2) click on the \"SLOW\"
    radio button; (3) click on the \"MEDIUM\" radio button\".

> Result: The \"SLOW\" and \"MEDIUM\" radio buttons for the River
> Response will now be selected.

13. Action: Select the River Response to be all, as before, by clicking
    on the \"ALL\" radio button.

> Result: The \"SLOW\", \"MEDIUM\", and \"FAST\" radio buttons will now
> be deselected and disabled (i.e. will not respond to user clicks),
> with only the \"ALL\" radio button selected and enabled.

14. Action: Repeat steps B.1 and B.2 in order to load the batch file
    again.

> Result: The Verification Group Manager window should look like that
> pictured in ivp.screenshot003.png.

15. Action: Set the Time Step to be \"2 weeks\", the End lead time to be
    \"5 days\", and the Forecast Type Sources to be "FE,FF,FR".

> Result: The Verification Group Manager window should now look like
> that pictured in ivp.screenshot007.png.

C.  []{#__RefHeading___Toc162687950 .anchor}**Test Plan item 3**

> **Verify that the Verification Location Manager window can be opened
> from the Verification Group Manager window.**

1.  Action: Click on **Edit Locations**.

> Result: The window pictured in ivp.screenshot008.png will pop up.
> There are two tables in the window: a top table labeled "Available
> Locations" and a bottom table labeled "Locations Chosen for Display".
> Additionally, there is a filter panel at the top of the window by
> several text fields and **Select** buttons.

D.  []{#__RefHeading___Toc162687951 .anchor}**Test Plan item 4**

**Verify that all the features of the Verification Location Manager
function properly.**

1.  Action: In the top table, press the left mouse button on the row for
    the location \"ABR\" with pe of "TA", dur of "I", ext of "N", and ts
    of "FB". Drag the mouse down to the row for \"ACMP1\" with pe of
    "TA", dur of "I", ext of "N", and type source (ts) of "FF". Release
    the mouse button.

> Result: The first 7 rows of the table will be selected. A selected row
> is highlighted in blue.

2.  Action: Press \<ctrl\> and click on the row for the location \"ABR\"
    with pe of "TA", dur of "I", ext of "X", and type source of "FB".

> Result: That row only will be deselected.

3.  Action: Click on the first row. Scroll down to the bottom of the
    table. Press \<shift\> and click on the last row.

> Result: All rows will be selected.

4.  Action: Click on **Select None**, just above the top table.

> Result: All rows will be unselected.

5.  Action: Click on the **Select** button next to the PE text field in
    the filter panel.

> Result: A Select PE Values window will pop up. It will look just like
> the Select FCST_TS Values window, but the list will include \"ALL\",
> \"HG\", \"HP\", \"HT\", \"PP\", \"QT\", and \"TA\".

6.  Action: Select \"HG\" and click **OK**.

> Result: The top table will now only display rows with a pe of "HG".
> All rows will be deselected.

7.  Action: Click on the **Select** button next to the DUR text field.

> Result: A Select DUR Values window will pop up. It will look just like
> the Select FCST_TS Values window, but the list will include \"ALL\",
> \"D\", \"I\", \"Q\".

8.  Action: Select "I" and click **OK**.

> Result: The top table will only display rows with a pe of "HG" and dur
> of "I".

9.  Action: In the Ext text field, change the text to \"Q\" and press
    \<ENTER\>.

> Result: No rows will be displayed in the top table, because no rows
> have a pe of "HG", dur of "I" and extremum of "Q".

10. Action: In the EXT text field, change the text to \"Z\" and press
    \<ENTER\>.

> Result: The top table will only display rows with a pe of \"HG\", dur
> of \"I\", and ext of \"Z\".

11. Action: Select the first 9 rows from the top table (those rows for
    locations ALNP1, BAIN6, and BEAP1). Click on **Edit Obs Cat**.

> Result: The Observed Category Editor window pictured in
> ivp.screenshot009.png will pop up.

12. Action: Specify the categories: \"MIN,1.0\*ModFS,MAX\" by doing the
    following: (1) click on **Add MIN**; (2) click on **Add MAX**; (3)
    enter the value 1 into the Scalar text field; (4) select the
    \"ModFS\" choice from the choice box currently displaying
    \"FS\"; (5) click on the **\>\>** button next to the choice box; (6)
    click on **OK**.

> Result: A window titled \"Category Editor Error!\" will pop up saying
> that it is unable to process the categories for at least one row (of
> the table). The rows will still be selected in the Verification
> Location Manager window.

13. Action: Click on **OK** in the error window.

> Result: The window will close.

14. Action: Click on **Close** in the Observed Category Editor.

> Result: The window will close.

15. Action: Click on the header for the column \"fs/f\".

> Result: The rows will now be sorted by the value in column \"fs/f\".

16. Action: Click on the **Select** button next to the Critical Values
    text field.

> Result: A Select Missing Values window will pop up. It will look just
> like the Select FCST_TS Values window, but the list will include
> \"Any\", \"Non-missing Values Only\".

17. Action: Click on \"Non-missing Values Only\" and click **OK**.

> Result: The rows with a pe of \"HG\", dur of \"I\", extremum of \"Z\",
> and with no critical stage values (columns with a gray background) of
> -999.0 will be selected.

18. Action: Click on **Select All**, just above the top table.

19. Action: Press and hold the \<ctrl\> key, and click on the first 6
    selected rows.

> Result: The six rows will be unselected. They are rows for locations
> CWCM2 and DAWM2.

20. Action: Click on **Edit Obs Cat**. In the Observed Category Editor,
    click on **OK**.

> Result: The Observed Category Editor window will close and all
> selected rows will remain selected. The Observed Category column in
> the table for all selected rows will now be \"MIN,1.0\*ModFS,MAX\".

21. Action: Click on **Edit Obs Cat**. In the Observed Category Editor
    window, select the row \"1.0\*ModFS\" in the list to the right and
    click on **Delete**.

> Result: The row will be removed from the list.

22. Click on **OK**.

> Result: The Observed Category Editor window will close and all
> selected rows will remain selected. The Observed Category column in
> the table for all selected rows will now be \"MIN, MAX\".

23. Action: Click on **Edit Fcst Cat**.

> Result: The Forecast Category Editor window that looks similar to that
> pictured in ivp.screenshot009.png will pop up. The only difference
> will be that the title of the window is Forecast Category Editor.

24. Action: Specify the categories: \"MIN,4,8,MAX\" by doing the
    following: (1) click on **Add MIN**; (2) click on **Add MAX**; (3)
    enter the value 8 in the Absolute text field; (4) click on **\>\>**
    next to the text field; (5) edit the value to be 4 in the Absolute
    text field; (6) click on **\>\>** next to the text field (NOTE that
    the 4 is inserted before the 8 in the list to the right); (7) click
    on **OK**.

> Result: The Forecast Category Editow window will close. The Forecast
> Category column in the table for all selected rows will now be
> \"MIN,4,8,MAX\".

25. Action: Click on **Unchoose All** in the Verification Location
    Manager window.

> Result: All of the locations (rows) will be \"unchosen\". Rows that
> are "chosen" are displayed in the bottom table with a red and pink
> background. These rows correspond to the verification locations that
> are chosen to be analyzed and shown in the IVP Data Display. Rows that
> are unchosen are displayed in the top table with a gray and white
> background.

26. Action: Click on **Reset Filter**.

> Result: All of the field in the filter panel will now display their
> initial values, and all available rows will be displayed in the top
> table.

27. Action: Click on the location column header in the top panel.

> Result: The row of the top table will now be sorted by location in
> ascending order.

28. Action: In the bottom table, select the rows for location OLNN6 and
    type sources "FE" and "FF", and click on **Choose**.

> Result: The row should now be chosen for display, and no row will be
> selected in the table.

29. Action: Select the rows for location OLNN6 in the bottom table and
    click on **Unchoose**.

> Result: The rows for OLNN6 will now be unchosen, and no row will be
> selected in the table.

30. Action: Click on **Select All** above the top table and click on
    **Choose**.

> Result: All rows should now be chosen, and no row will be selected in
> the table.

31. Action: Click on table column header \"location\" in the bottom
    table.

> Result: The rows in the table will now be sorted alphabetically by the
> location column in ascending order. The row for location "ABR", pe
> "TA", dur "I", ext "N", and type source "FB" will be first.

32. Action: Click on **Unchoose All**, and manually select in the top
    table (while holding the \<ctrl\> key) the rows for locations CLKW2,
    ELRP1, JHNP1, MTXV2, OLNN6, and RGLN4, and type sources "FE", "FF",
    and "FR". In each case, the pe is \"HG\", dur is \"I\", and ext is
    \"Z\". Click on **Choose**.

> Result: The 18 rows corresponding to the locations and type sources
> will be chosen.

33. Action: Sort the bottom table by location (ascending) and response
    time (decesending).

> Result: The row for location OLNN6 and type source "FE" will be first.

E.  []{#__RefHeading___Toc162687952 .anchor}**Test Plan item 5**

> **Verify that the IVP Data Display can be opened from the Verification
> Group Manager window.**

1.  Action: Click on **Create Display** in the Verification Group
    Manager.

> Result: A progress window titled "Loading all IVP Data Display data"
> pictured in ivp.screenshot010a.png will pop-up. It will contain text
> providing the analysis interval, total pairs loaded so far, and the
> location and physical element currently being loaded. After a delay of
> up to a few minutes (usually much less), the progress window will
> close and the IVP Data Display window pictured in
> ivp.screenshot010.png will pop up.

F.  []{#__RefHeading___Toc162687953 .anchor}**Test Plan item 6**

> **Verify that the IVP Data Display interacts correctly with the
> Verification Group Manager and Verification Location Manager.**

1.  Action: In the Verification Group Manager, change the End lead time
    to 3 days and move the mouse over top of the IVP Data Display.

> Result: A window titled \"Underlying Pairs To Display Have Changed!\"
> will pop up, saying that since the Verification Group Manager settings
> have been changed, the IVP Data Display must be recreated.

2.  Action: Click on **OK** in the window.

> Result: The message window and the IVP Data Display window will close.

3.  Action: Change the End lead time in the Verification Group Manager
    back to 5 days and click on **Create Display** in the Verification
    Group Manager.

> Result: The same IVP Data Display window as before will open. Note
> that when initially opened this time, the progress window may not
> paint until after it has been updated, which usually happens after a
> couple minutes.

4.  Action: In the IVP Data Display, select the row for location \"OLNN6
    (HGIFFZZ)" from the Locations list.

> Result: The IVP Data Display will change to look like that one
> pictured in ivp.screenshot011.png.

G.  []{#__RefHeading___Toc162687954 .anchor}**Test Plan item 7**

**Verify that the menu options of the IVP Data Display work.**

1.  Action: Click on **Actions \>\> Show Group Mgr**.

> Result: The Verification Group Manager window should be visible and
> made into the active window.

2.  Action: **Actions \>\> Show Location Mgr** in the IVP Data Display
    window.

> Result: The Verification Location Manager window should be visible and
> made into the active window.

3.  Action: Click on **Actions \>\> Save Image File**.

> Result: A file browser window much like that one pictured in
> ivp.screenshot002.png should pop up on the screen. The only difference
> should be the initial directory and the **Open** button becoming
> **Save**.

4.  Action: Specify a file name for the image to save and click on
    **Save**. Make sure the extension is .jpeg or .jpg.

> Result: The file browser window will close and a message dialog titled
> \"Image Save Successful!\" will pop up giving the name of the file
> that has been created. Check that the image file matches the chart
> displayed in the IVP Data Display.

5.  Action: Click on **OK** in the Image Save Successful window.

> Result: The window will close.

6.  Action: Click on **Actions \>\> Save Image File**. When the file
    browser window pops up, specify a name for the image file, and make
    sure the extension is .png. Click on **Save**.

> Result: The file browser window will close and a message dialog titled
> \"Image Save Successful!\" will pop up giving the name of the file
> that has been created, which be exactly as you specified. Check that
> the image file matches the chart displayed in the IVP Data Display.

7.  Action: Click on **OK** in the Image Save Successful window.

> Result: The window will close.

8.  Action: Click on **Options \>\> Show line y = x**.

> Result: An orange line will be displayed on the chart that is the line
> for which the forecast and observation equal.

9.  Action: Click on **Options \>\> Hide line y = x**.

> Result: The orange line will be removed.

10. Action: In the IVP Data Display, press and hold the mouse button at
    approximately the point (2, 6), drag the pointer to approximately
    (4, 2), and release the button.

> Result: A rectangle will be drawn with a corner at the point where you
> pressed the mouse button and the opposite corner at the point where
> you released the mouse button.

11. Action: Click on **Options \>\> Zoom In**.

> Result: After a brief delay, the chart will be redrawn to show only
> what is in the rectangle you selected in the previous step. Note that
> the OLNN6 points should still be displayed in blue.

12. Action: Click on **Options \>\> Zoom Out**.

> Result: The chart will look as it did before clicking on the \"Zoom
> In\" button in the previous step.

13. Action: Click on **Actions \>\> Create a New Display**.

> Result: A new Verification Group Manager window will be opened.

14. Action: In the new Verification Group Manager window, open up the
    batch file ivpbatch.graphics.input001.bat and click on **Create
    Display**.

> Result: After up to a few minutes, a new IVP Data Display should open.
> In the window\'s title should be the text \"Data Source #1\".

15. Action: In the new IVP Data Display window, click **Actions \>\>
    Close**. Click on the **X** in the upper right corner of the new
    Verification Group Manager window.

> Result: The new windows opened up in steps G.13 and G.14 will be
> closed.

16. Action: In the IVP Data Display window, click **Views \>\> Time
    Series View**.

> Result: A progress window will pop-up titled "Loading Observed Data
> for Time Series Views". After the observations are loaded, the window
> will close and the IVP Data Display will change to the window pictured
> in ivp.screenshot011a.png.

17. Action: From the Issuance Times list, select the first 9 lines
    (2003-08-27 through 2003-09-04). Select 'RZ' in the Observed Type
    Sources panel. Click on **Update Plot**.

> Result: The IVP Data Display will change to the window pictured in
> ivp.screenshot011b.png.

18. Action: In the legend of the plot, click on the legend entry for
    series "2003-08-30 16:34:00".

> Result: The IVP Data Display will change to the window pictured in
> ivp.screenshot011c.png.

H.  []{#__RefHeading___Toc162687955 .anchor}**Test Plan item 8**

> **Verify that the Verification Pairs Data Viewer and Verification
> Pairs Data Editor windows can be opened up from the IVP Data
> Display.**

1.  Action: Click on **Actions \>\> Show Pairs Data Viewer**.

> Result: The Verification Pairs Data Viewer window pictured in
> ivp.screenshot012.png will open up.

2.  Action: In the IVP Data Display, draw a rectangle on the chart (as
    in Step G.10) around the 5 largest points on the plot (y-axis value
    is approximately 9.5 ft). Be sure to select the 5 largest points!

> Result: In the Verification Pairs Data Viewer, rows specifying data
> that is NOT in the region specified by the rectangle will be displayed
> with a white background. All rows within the region will have the same
> color background as before. In this case, there will be only one row
> in the region.

3.  Action: In the Verification Pairs Data Viewer, click on **Save to
    File**. A file browser will open up. Specify a file name for the
    output file and click on **Save**.

> Result: A file by the name you specified will be created. It should
> look like that one given in ivpbatch.imagedata001.dat, except that the
> columns will be reordered. When the Pairs File Save Message window
> opens up, click on **OK** to close it.

4.  Action: Click on **Close** in the Verification Pairs Data Viewer.

> Result: The window will close.

5.  Action: Click on **Actions \>\> Show Pairs Data Editor**.

> Result: The Verification Pairs Data Editor window pictured in
> ivp.screenshot012a.png will open up.

6.  Action: Select one row from the **Selected Data** table and click on
    **Mark as Bad**.

> Result: For that row in the **Selected Data** table, the "in db"
> column will be set to 'N' with a red background and the quality column
> will be set to 'BAD' with a red background. You will need to deselect
> the row to see the change. The plot in the IVP Data Display will
> change: the point corresponding to the modified pair will be drawn as
> a hollow point (unfilled circle) and will no longer be connected to
> the rest of the time series via a line.

7.  Action: Select one row from the **Selected Data** table, click on
    the **Edit Fcst** checkbox, and, in the IVP Data Display, click
    within the plot somewhere.

> Result: In the **Selected Data** table, the forecast value will change
> to reflect the spot where the click happened and the row's fcst cell
> will have a red background. The "in db" column entry will be set to
> 'N' and have a red background. Also, the plot in the IVP Data Display
> will updated to reflect the new forecast value and crosshairs will be
> drawn crossing at the point corresponding to the selected row.

8.  Action: In the IVP Data Display, click on **Views \>\> Scatter Plot
    View**. In the Verification Pairs Data Editor, select one row from
    the **Selected Data** table, click on both the **Edit Obs** and
    **Edit Fcst** checkboxes (make then both checked), and in the IVP
    Data Display, click within the plot somewhere.

> Result: In the **Selected Data** table, the observed and forecast
> values will change to reflect the spot where the click happened. Also,
> the plot in the IVP Data Display will update to reflect the new
> forecast value. Note that brown crosshairs are drawn in the IVP Data
> Display crossing at the point corresponding to the selected row.

9.  Action: In the Verification Pairs Data Editor, click on **Select New
    Pairs**.

> Result: Both the **Edit Obs** and **Edit Fcst** checkboxes will be
> unchecked.

10. Action: In the IVP Data Display, draw a new rectangle to select
    different points.

> Result: The **Selected Data** table will update to show the newly
> selected points. Any pairs that were changed in the preceding steps
> will be moved to the **Changed and Unsaved Data** table.

11. Action: Select every row within the **Changed and Unsaved Data**
    table and click on **Recover**.

> Result: All of the changed data will revert to their original values.
> The data plotted within the IVP Data Display will change to be the
> same as that pictured in ivp.screenshot011.png.

12. Action: In the Verification Pairs Data Editor, click on **Update
    Plot and Close**.

> Result: The window will close.

I.  []{#__RefHeading___Toc162687956 .anchor}**Test Plan item 9**

> **Verify that the Plot Definition Manager window can be opened up from
> the IVP Data Display window.**

1.  Action: Click on **Actions \>\> Show Plot Mgr**.

> Result: The Plot Definition Manager window pictured in
> ivp.screenshot013.png will pop up.

J.  []{#__RefHeading___Toc162687957 .anchor}**Test Plan item 10**

**Verify that the options in the Plot Definition Manager can be set
correctly.**

1.  Action: Click on **Load From Batch** and select the
    ivpbatch.graphics.input001.bat. Click on **Open**.

> Result: The Plot Definition Manager window should change to that in
> ivp.screenshot.test014.png.

2.  Action: Click on the Select Primary Statistics **Choose Stat**
    button.

> Result: The IVP Statistic Chooser Manager window pictured in
> ivp.screenshot015.png will pop up.

3.  Action: Click on the checkbox for \"Error Statistics\".

> Result: All checkboxes currently checked should become unchecked. All
> checkboxes under the Error Statistics heading should become checked.

4.  Action: Click on the checkbox next to \"Mean Error (ME)\".

> Result: The checkbox for \"Mean Error (ME)\" should become unchecked,
> and the \"Error Statistics\" checkbox should become unchecked.

5.  Action: Repeat Step J.4.

> Result: Both checkboxes unchecked in Step J.4 should now become
> checked, again.

6.  Action: Click on the checkbox for \"Maximum (MAXIMUM)\" under the
    Quantiles and Extremes heading, click on the \"Sample Size\"
    checkbox, click on the \"Leadtime Statistics\" checkbox, and finally
    click on the "Errors Statistics" checkbox.

> Result: As each click is done, the old checked checkboxes will become
> unchecked and the new ones will become checked.

7.  Action: Click on the checkboxes for "CDF Plot (CDFPLOT)", "PDF Plot
    (PDFPLOT)", and "ROC Plot (ROCPLOT)", in that order.

> Result: As each click is done, the old checked checkboxes will become
> unchecked and the new one will become checked.

8.  Action: Click **Cancel**.

> Result: The window will close and the Plot Definition Manager will not
> change.

9.  Action: Click on the Select Primary Statistics **Choose Stat**
    button.

> Result: The IVP Statistic Chooser Manager window pictured in
> ivp.screenshot015.png will pop up.

10. Action: Click on the "Error Statistics" checkbox and click **OK**.

> Result: The window will close, and the Select Primary Statistics text
> field will change to \"RMSE,MAXERR,MAE,ME\".

11. Action: Select the NUM SAMPLES statistic for the secondary
    statistics by doing the following: (1) click on the Select Secondary
    Statistics **Choose Stat** button; (2) click on the \"Statistic
    Sample Size (NUM SAMPLES)\" checkbox in the IVP Statistic Chooser
    Manager; (3) Click on **OK**.

> Result: The window will close and the Select Secondary Statistics text
> field will change to \"NUM SAMPLES\".

12. Action: From the Select Primary Plot Type choice box, select
    \"SCATTER\", and from the Select Secondary Plot Type choice box,
    select "BAR".

> Result: The choice boxes will reflect the selection.

13. Action: From the Select X-Axis Variable choice box, select
    \"Location\".

> Result: The choice box will reflect the selection and the Select
> Forecast Category choice box will become enabled (i.e. no longer
> grayed out).

14. Action: Verify that the selections available in the Select Forecast
    Category choice box include \"Do Not Use\", \"All Categories
    Combined\", \"Category 1\", \"Category 2\" and \"Category 3\". From
    the choice box select \"Do Not Use\".

> Result: The Select Forecast Category choice box will reflect the
> choice and become disabled, and the Select Observed Category choice
> box will become enabled with a selection of \"Use Only Category\".

15. Action: Verify that the selections available in the Select Observed
    Category choice box include "Do Not Use" and "Use Only Category".
    From the Select Observed Category choice box select \"Do Not Use\".

> Result: The Select Observed Category choice box will reflect the
> choice and become disabled, and the Select Forecast Category choice
> box will become enabled with a selection of \"All Categories
> Combined\".

16. Action: From the Select Comparison Variable choice box, select
    Location.

> Result: A dialog titled \"Plot Definition Manager Warning!\" will
> pop-up. It will explain that the comparison variable and x-axis
> variable must be different.

17. Action: Click on **OK** in the dialog.

> Result: The dialog will close and the comparison variable will still
> be set to \"None\".

K.  []{#__RefHeading___Toc162687958 .anchor}**Test Plan item 11**

> **Verify that the IVP Statistic Display window can be opened from the
> Verification Plot Definition manager, and that it displays an
> appropriate chart based on user settings.**

1.  Action: Click on **Create Plot**.

> Result: The window pictured in ivp.screenshot016.png will pop up.

L.  []{#__RefHeading___Toc162687959 .anchor}**Test Plan item 12**

**Verify that the menu options in the IVP Statistic Display work.**

1.  Action: Click on **Actions \>\> Save Image File**. When the file
    browser opens up, select an appropriate file name for a png file
    (include a .png extension) and click **Save**.

> Result: An image file is created with the name you specified. When the
> Image Save Successful window opens up, click on **OK** to close it.
> Verify that the image file matches the chart displayed in the IVP
> Statistic Display window.

2.  Action: Under the Actions menu, click on **Actions \>\> Save Batch
    File**.

> Result: The IVP Batch File Creation Wizard window pictured in
> ivp.screenshot017.png will pop up.

3.  Action: Click on **Next**.

> Result: The IVP Batch File Creation Wizard window change to that
> pictured in ivp.screenshot017a.png.

4.  Action: Click on **Edit Locations**.

> Result: A Verification Location Manager will pop-up.

5.  Action: Unchoose all locations with lid 'RGLN4' and click on **OK**.

> Result: The locations with lid 'RGLN4' will not longer be in the
> Locations panel.

6.  Action: Click on **Edit Locations**, choose all locations with lid
    'RGLN4'. Sort the list by location (ascending) and then response
    (descending), so that the rows for OLNN6 are first. Click on **OK**.

> Result: 'RGLN4' locations will now be in the Locations panel.

7.  Action: Click on **Next**.

> Result: The IVP Batch File Creation Wizard window change to that
> pictured in ivp.screenshot017b.png.

8.  Action: Click on **Add One Per Location ID**.

> Result: The Set Location Restrictions for Added Groups window,
> pictured in ivp.screenshot017c.png will pop-up.

9.  Action: Click on **Cancel**.

> Result: The window will close and the IVP Batch File Creation Wizard
> window will be unaffected.

10. Action: Click on **Next**.

> Result: The IVP Batch File Creation Wizard window change to that
> pictured in ivp.screenshot017d.png.

11. Action: Click on **Select** next to the Image File To Create text
    field.

> Result: A file browser will pop up.

12. Action: Type in the file name "testimage.png". Click on **Open**.

> Result: The browser will close and the Image File To Create text field
> will contain the complete path name of the image file you selected.

13. Action: In the Image File To Create text field, remove all of the
    directory text up to the file name, leaving only \"testimage.png\"
    in the text field.

14. Action: In the Data File To Create \[optional\] text field, type in
    the file name \"testdata.dat\".

15. Action: Click on the **Create New Template** radio button.

> Result: The Template File Name text field and **Select** button will
> become enabled, allowing them to be used.

16. Action: In the \"Template File Name\" text field, type in the file
    name \"testtemplate.txt\".

17. Action: Click on **Select** next to the \"Batch File Name\" text
    field.

> Result: A file browser will pop up.

18. Action: Type in the file name \"test.bat\" and click on **Save**.

> Result: The browser will close and the Batch File Name text field will
> contain the complete path name of the batch file you selected.

19. Action: Click on **Finish**.

> Result: The IVP Batch File Creation Wizard window will close. A window
> titled \"Template File Save Message\" will pop up and will give the
> complete path name of the batch file that was created. Verify that the
> template file was created (in the \$(vsys_files)/\$LOGNAME/templates
> directory) and exactly matches \"ivp.createdtemplate001.txt\" in that
> same directory.

20. Action: Click on **OK** in the save message window.

> Result: The message window will close and a new message window titled
> "Batch File Save Message" will pop up. Verify that the batch file was
> created (in the \$(vsys_input) directory) and exactly matches
> \"ivp.createdbatch001.bat\" in the same directory.

21. Action: Click on **OK**.

> Result: The message window and the IVP Batch File Creation Wizard will
> both close.

M.  []{#__RefHeading___Toc162687960 .anchor}**Test Plan item 13**

> **Verify that the IVP Statistics Data Viewer can be opened up from the
> IVP Statistic Display.**

1.  Action: Under the Actions menu, click on the **Actions \>\> Show
    Data** menu item.

> Result: The IVP Statistics Data Viewer window pictured in
> ivp.screenshot018.png will pop up.

2.  Action: Click on **Save to File**. A file browser will open up.
    Specify a file name for the output file and click **Open**.

> Result: A file by the name you specified will be created. It should
> look like that one given in ivpbatch.imagedata001b.dat. When the Stat
> File Save Message window opens up, click on **OK** to close it.

3.  Action: Click on **Close**.

> Result: The IVP Statistics Data Viewer will close.

N.  []{#__RefHeading___Toc162687961 .anchor}**Test Plan item 14**

> **Verify that the Chart Properties Manager window can be used to edit
> properties of the chart displayed in either the IVP Data Display or
> IVP Statistic Display.**

1.  Action: Click on **Actions \>\> Edit Chart Properties**.

> Result: The Chart Property Manager window pictured in
> ivp.screenshot019.png will pop up.

2.  Action: Click on each of the tabs available under the Chart Titles
    heading, and then under the Chart Others heading.

> Result: There are a total of 12 possible appearances for the Chart
> Property Manager depending on what tabbed panel is currently active.
> They are as follows:
>
> **Chart Title (Chart Titles)** - ivp.screenshot019.png
>
> **Y-Axis 1 Title (Chart Titles)** - ivp.screenshot020.png
>
> **Y-Axis 2 Title (Chart Titles)** - ivp.screenshot021.png
>
> **X-Axis Title (Chart Titles)** - ivp.screenshot022.png
>
> **Legend Title (Chart Titles)** - ivp.screenshot023.png
>
> **Legend (Chart Titles)** - ivp.screenshot024.png
>
> **Tick Mark Font (Chart Titles)** - ivp.screenshot025.png
>
> **Chart Dimension (Chart Others**) - ivp.screenshot026.png
>
> **Primary Y-Axis (Chart Others)** - ivp.screenshot027.png
>
> **Secondary Y-Axis (Chart Others)** - ivp.screenshot028.png
>
> **X-Axis (Chart Others)** - ivp.screenshot029.png
>
> **X-Axis Annotation (Chart Others)** - ivp.screenshot030.png
>
> When the tab specified is clicked on, the Chart Property Manager will
> appear as shown in the corresponding screen-shot.

3.  Action: Click on the **Chart Dimension (Chart Others**) tab.

> Result: The display will change to that pictured in
> ivp.screenshot026.png.

4.  Action: Change the \"Chart Height\" value to be 600. This can be
    done by either typing in the text field and pressing \<Enter\>, or
    by shifting the horizontal slider to the right.

> Result: The slider and text field will change and, the **Default**
> radio button will become deselected.

5.  Action: Click on the **Secondary Y-Axis (Chart Others)** tab.

> Result: The display will change to that pictured in
> ivp.screenshot028.png.

6.  Action: Click on the Upper Value **Reset** button.

> Result: The Reset Slider window pictured in ivp.screenshot031.png will
> pop up.

7.  Action: Change the Major Tick Spacing text field to \"400\" and
    click on **OK**.

> Result: The Reset Slider window will close, and the Secondary Y-Axis
> tab panel of the Chart Property Manager will reflect the new settings
> of 400 for the major tick spacing. The **Default** radio button will
> remain selected.

8.  Action: Click on the **X-Axis Annotation (Chart Others)** tab.

> Result: The display will change to that pictured in
> ivp.screenshot030.png.

9.  Action: Click on the **90** radio button.

> Result: The **0** radio button will become unselected, and the **90**
> radio button will become selected. The **Default** radio button will
> become unselected.

10. Action: Click on the **Y-Axis 1 Title (Chart Titles)** tab.

> Result: The display will change to that pictured in
> ivp.screenshot020.png.

11. Action: Select \"Courier\" from the font choice box (currently
    displaying Serif). Select 12 for the size, and click on Italics.

> Result: The **Default (Text & Font)** radio button will be unselected.

12. Action: Click on **Text Color**.

> Result: The Choose Color window pictured in ivp.screenshot032.png will
> pop up.

13. Action: Click on a red color from the colors grid, and then click on
    the **RGB** tab. In the RGB panel, make sure the Green and Blue
    components are 0. Click on **OK**.

> Result: The Choose Color window will close and the label shown in the
> Y-Axis 1 Title panel text field will turn red.

14. Action: Type in the string "Here I am" in the text area.

15. Action: Click on the **Default (Text Only)** radio button.

> Result: The text area will change back to its original value prior to
> the previous step, but the font will remain the same (Courier, 12,
> bold-italics, red).

16. Action: In the Chart Property Manager, click on **OK**.

> Result: The Chart Property Manager will close, and the IVP Statistic
> Display window will change to that pictured in ivp.screenshot033.png.

17. Action: In the IVP Statistic Display window, click on **Actions \>\>
    Close**.

> Result: The IVP Statistic Display window will close.

O.  []{#__RefHeading___Toc162687962 .anchor}**Test Plan item 15**

> **Create many different graphics by changing the settings in the Plot
> Definition Manager and verify that they are created correctly.**

1.  Action: In the Plot Definition Manager window, click on the **x** in
    the upper right corner to close the window.

> Result: The window will close.

2.  Action: In the IVP Data Display window, click on **Actions \>\>
    Close**.

> Result: The window will close.

3.  Action: In the Verification Group Manager Load the batch file
    ivpbatch.graphics.input002.bat. Click on **Create Display**.

> Result: A progress window displaying the progress of loading the data
> will pop-up. After a delay of up to a few minutes (usually much less),
> all data will be loaded and the progress window will close, after
> which an IVP Data Display window will pop up.

4.  Action: Click on **Actions \>\> Show Plot Mgr** in the IVP Data
    Display.

> Result: The Plot Definition Manager window will pop-up.

5.  Action: In the Plot Definition Manager, load the batch file
    ivpbatch.graphics.input002.bat and click on **Create Plot**.

> Result: An IVP Statistics Display window will be opened that displays
> a chart that looks like the image in the batch output file
> ivpbatch.image002.png.

6.  Action: Click on **Actions \>\> Show Data** to open the IVP
    Statistics Data Viewer. Click **Save to File** and create a file.

> Result: The file created should look like ivpbatch.imagedata002.dat.

7.  Action: In the IVP Statistic Display window, click on **Actions \>\>
    Close**.

> Result: The IVP Statistic Display window will close.

8.  Action: Repeat steps O.1 -- O.7, but replace
    ivpbatch.graphics.input002.bat with the following files:

-   ivpbatch.graphics.input003.bat

-   ivpbatch.graphics.input004.bat

-   ivpbatch.graphics.input005.bat

-   ivpbatch.graphics.input006.bat

-   ivpbatch.graphics.input007.bat

-   ivpbatch.graphics.input008.bat

-   ivpbatch.graphics.input009.bat

-   ivpbatch.graphics.input010.bat

-   ivpbatch.graphics.input011.bat

-   ivpbatch.graphics.input012.bat

-   ivpbatch.graphics.input013.bat

> Note the number following the "input" in the file names. This will be
> referred to as \<###\> within the Results, below.
>
> Result: The chart displayed in step O.5 should look like
> ivpbatch.image\<###\>.png, and the file created in step O.6 should
> look like ivpbatch.imagedata\<###\>.dat. The one exception is for when
> \<###\> = 010 (ivpbatch.graphics.input010.bat). In this case, the
> chart will look like ivpbatch.image010.CRZA3.png and the file created
> will look like ivpbatch.imagedata010.CRZA3.dat.

P.  []{#__RefHeading___Toc162687963 .anchor}**Test Plan item 16**

**Verify that the windows can close down.**

1.  Action: Click on **x** in the upper right corner of the Plot
    Definition Manager window.

> Result: The window will close down.

2.  Action: Click on **x** in the upper right corner of the Verification
    Location Manager and Verification Group Manager.

> Result: The windows will close down.

3.  Action: In the IVP Data Display, click on **Actions \>\> Exit**.

> Result: The program will exit.

***Verification Software Test***

[]{#__RefHeading___Toc162687964 .anchor}***Test 4: Testing IVP Batch
Builder***

**0. Setup**

Be sure to run the General Setup Procedure prior to executing this test.

In order to have all the needed files in place for this manual test, you
must either run the automated test (Test 2) prior to running this test,
or do the following:

> cd \$(get_apps_defaults vsys_scripts)
>
> vfy_testit --b \<test files directory\> setup

In either case, do the following export to ensure you are using the
correct input file:

> export vsys_input=
>
> \$(get_apps_defaults vsys_dir)/input/test/\$(whoami)

**A. Test of the IVP Batch Builder**

For the following test steps, all screenshot images (i.e.
ivp.screenshot\*) are located in the directory \<test files
directory\>/screenshots. All other image and data (.dat) files can be
found in the directory \<test files directory\>/output.

1.  Action: Type in the following:

> go vsys_scripts
>
> ivpbatchb
>
> Result: The Verification Group Manager window pictured in
> ivpbb.screenshot001.png will pop up on the screen. If it does not,
> then the test fails.

2.  Action: Click on each of the tabs in turn: **Pairing**, **Pairing
    2**, **Location**, **Timing**, **Group 1**, **Group 2**,
    **Graphics**, and **Actions**.

> Result: As each tab is clicked, the **Command Panel** on the left side
> of the window will change. The new screen shots will be as follows:

**Pairing 1** \-- ivpbb.screenshot001.png

> **Pairing 2** \-- ivpbb.screenshot002.png
>
> **Location** \-- ivpbb.screenshot003.png
>
> **Timing** \-- ivpbb.screenshot004.png
>
> **Group** **1** \-- ivpbb.screenshot005.png
>
> **Group** **2** \-- ivpbb.screenshot006.png
>
> **Graphics** \-- ivpbb.screenshot007.png
>
> **Actions** \-- ivpbb.screenshot008.png

3.  Action: Click on **File \>\> Open**.

> Result: A file browser will pop up.

4.  Action: Select the file ivpbatch.graphics.input002.bat and click
    **Open**.

> Result: The **Command List** on the right side of the window will
> reflect the contents of the selected batch file.

5.  Action: Click on line 25 of the command list and click on **Commands
    \>\> Comment Out Command**.

> Result: Line 24, which is for command END_TIME, will be commented out;
> i.e. a \'#\' will be inserted at the beginning.

6.  Action: Click on the **Timing** tab, click the checkbox for \"Set
    END_TIME To:\", and click on **Fixed** for the END_TIME.

> Result: The checkbox for END_TIME will be checked and a window titled
> \"Choose Date/Time\" will pop up.

7.  Action: Specify the date/time \"2005-02-28 23:59:59 GMT\" and click
    **OK**.

> Result: The window will close and the \"Set END_TIME to:\" text field
> will update with the specified date.

8.  Action: Select line 24 of the command list and click on **Insert**.

> Result: The command \'END_TIME = \"2005-02-28 23:59:59 GMT\"\' will be
> inserted at line 24.

9.  Action: Click on line 27, which is for the command LEADTIME_START
    and click **Edit**.

> Result: A window titled \"Editing LEADTIME_START Command\" will open
> up. A panel within the window will be the same as the LEADTIME_START
> panel on the **Timing** command panel.

10. Action: Select 0 days as the LEADTIME_START and click **OK**.

> Result: The window will close and line 27 will change to be
> \'LEADTIME_START = 0days.

11. Action: Click on line 28, which is for the command LEADTIME_END and
    click **Edit**.

> Result: A window titled \"Editing LEADTIME_END Command\" will open up.
> A panel within the window will be the same as the LEADTIME_END panel
> on the **Timing** command panel.

12. Action: Select 72 hours as the LEADTIME_END and click **OK**.

> Result: The window will close and line 15 will change to be
> \'LEADTIME_END = 72hours\'.

13. Action: Click on line 41 and click **Edit**.

> Result: A window titled \"Editing PRIMARY_STATS Command\" will pop up.
> It will show a PRIMARY_STATS value of \"POD,HFAR,TFAR,CSI\".

14. Action: Within this window, click **Edit**.

> Result: A window titled \"IVP Statistic Chooser Manager...\" will
> popup. This window is also part of the IVP GUI. The statistics RMSE,
> Maximum Error, Mean Abs Error, and Mean Error will be checked. Also,
> the **Error Statistics** check box will be checked.

15. Action: Click on **Mean Error (ME)**.

> Result: The **Mean Error (ME)** checkbox will be unchecked, and the
> **Error Statistics** checkbox will be unchecked.

16. Action: Click on **OK**.

> Result: The window will close and the editing window text field will
> change to reflect that ME has been unselected.

17. Action: Click on **OK**.

> Result: The window will close and line 41 will change to
> \"PRIMARY_STATS = RMSE,MAXERR,MAE\".

18. Action: Click on Line 51, corresponding to command GRAPH_TEMPLATE,
    and click **Delete**.

> Result: Line 25 will be removed from the **Command List**.

19. Action: Click on **File \>\> Save As**.

> Result: A file browser will open.

20. Action: Specify the file ivpbatchb.created001.bat and click
    **Open**.

> Result: The file name given above the command list will change to be
> the specified file name.

21. Action: Click **Run**.

> Result: A window titled \"IVPBatchBuilder: Confirm Run\" will open.

22. Action: Verify that the execution command is correct and click
    **OK**.

> Result: The window will close and output will be generated to the
> background terminal. Once the run is completed (after as much as a
> minute or two), a window titled Batch Output Viewer will open. It will
> have two files in it: ivpbatch.image002.png and
> ivpbatch.imagedata002.dat.

23. Action: Click on the first file (the png file) and click **View**.

> Result: A window with a title of the file name will be opened. It will
> look like that in ivpbb.screenshot009.png.

24. Action: Click on the \'**x**\' in the upper right corner to close
    the window.

25. Action: Click on the second file (the dat file) and click **View**.

> Result: The window pictured in ivpbb.screenshot010.jpg will open up.

26. Action: Click **Okay**.

> Result: The window will close.

27. Action: Click **Close**.

> Result: The Batch Output Viewer window will close.

28. Action: Click **File \>\> Exit**.

> Result: The program will shut down.