**TestMaster^™^ Evaluation Report**

(Based on the Pilot Program)

Prepared by:

Thomas Huang

Integration and Test

Image Processing System

Science Data Processing Systems - Section 388

Jet Propulsion Laboratory

4800 Oak Grove Drive

Pasadena, CA 91109

and

Teradyne, Inc.

Software and Systems Test

44 Simon Street

Nashua, NH 03060

October 16, 1997

**Table of Contents**

Applicable Documents [iv](#applicable-documents)

Jet Propulsion Laboratory's Documents.
[iv](#jet-propulsion-laboratorys-documents.)

Vendor Documents: [iv](#vendor-documents)

Introduction [1](#introduction)

Goals [1](#goals)

Background [2](#background)

Pilot Program Approach [2](#pilot-program-approach)

Current Testing Process: [3](#current-testing-process)

XVD Software [5](#__RefHeading___Toc400199537)

Black Box Test Techniques [5](#black-box-test-techniques)

TestMaster Description [6](#testmaster-description)

TestMaster and the Test Process [6](#testmaster-and-the-test-process)

TestMaster Tool [7](#testmaster-tool)

XRunner Test Harness [9](#xrunner-test-harness)

VERITAS ViSTA TEST Code Coverage Tool
[10](#veritas-vista-test-code-coverage-tool)

Pilot Generated Test Models [11](#pilot-generated-test-models)

XVD Modeling and Test Generation [12](#xvd-modeling-and-test-generation)

Mars Pathfinder (MPF) Subsystem Modeling and Test Generation
[16](#mars-pathfinder-mpf-subsystem-modeling-and-test-generation)

Reusable Submodels [18](#reusable-submodels)

Pilot Task Evaluation [21](#pilot-task-evaluation)

Test Quality Comparison [22](#test-quality-comparison)

Comparison Summary [23](#comparison-summary)

System Modeling Estimate [23](#system-modeling-estimate)

Basis for the Estimation [23](#basis-for-the-estimation)

Labor Hours Analysis [24](#labor-hours-analysis)

Code Coverage [27](#code-coverage)

TestMaster Tool Observations and Results
[27](#testmaster-tool-observations-and-results)

Conclusions: [29](#conclusions)

Glossary: [30](#glossary)

List of Figures

Figure 1, Current Integration and Testing Approach to Software Testing
[4](#__RefHeading___Toc400120116)

Figure 2, Functional Testing with XRunner and VERITAS ViSTA TEST
[6](#__RefHeading___Toc400120117)

Figure 3, Functional Testing with VICAR and VERITAS ViSTA TEST 6

Figure 4, TestMaster Elements 7

Figure 5, Model and Programmable Elements 8

Figure 6, Mercury Interactive\'s XRunner^®^ 10

Figure 7, ViSTA Utilities model 11

Figure 8, Top-level XVD model 13

Figure 9, File menu submodel 13

Figure 10, File Open submodel 14

Figure 11, Edit menu submodel 15

Figure 12, Preference submodel 15

Figure 13, Display preference submodel 16

Figure 14, Top-Level System Test model for MPF 16

Figure 15, MPFNAV function model 17

Figure 16, MCAULEY function model 17

Figure 17, MPFMOS function model 18

Figure 18, Test model used to demonstrate the functionality of Combo\_
19

Figure 19, Documentation for submodel Combo\_ 19

Figure 20, Implementation of Combo\_ 20

Figure 21, Combo\_ integrated into MPFMOS application model 20

Figure 22, Partial MPF image processing subsystem specification model 22

Figure 23, Total Time Spent during Pilot Program and the breakdowns 25

## Applicable Documents

### Jet Propulsion Laboratory's Documents.

#### SPECIFICATIONS:

+----------------+------------------------------------+----------------+
| Identification | Title                              | Revision/Date  |
+----------------+------------------------------------+----------------+
| Jet Propulsion | **XVD User's Guide**               | *              |
| Laboratory,    |                                    | *15-Sep-1997** |
| Science Data   |                                    |                |
| Processing     |                                    |                |
| System         |                                    |                |
+----------------+------------------------------------+----------------+
| Jet Propulsion | **VICAR User's Guide**             | **Rev B.       |
| Laboratory,    |                                    | 14-Oct-1994**  |
| Science Data   | **Version 3**                      |                |
| Processing     |                                    |                |
| System         |                                    |                |
+----------------+------------------------------------+----------------+

## 

### Vendor Documents:

  ---------------- ------------------------------------- ----------------
  Identification   Title                                 Revision/Date

  Teradyne         Using TestMaster^^                   **Ver. 1.80**
  Software &                                             
  System Test                                            

  Teradyne         Introduction to Modeling With         **Rev.
  Software &       TestMaster^^                         Jul-1997**
  System Test                                            

  ISBN             Black Box Testing Techniques for      **1995**
  0-471-12094-4    Functional Testing of Software and    
                   Systems                               

  Mercury          XRunner^^ User's Guide               **Ver. 4.0**
  Interactive                                            

  Mercury          TSL Reference Guide                   **Ver. 4.0**
  Interactive                                            

  VERITAS^^       User's Guide for the VERITAS^^       **Ver. 2.0.3**
  Software         ViSTA^^ Code Coverage Tools          
  Corporation                                            
  ---------------- ------------------------------------- ----------------

TestMaster is a trademark of Teradyne, Inc., 44 Simon Street, Nashua,
New Hampshire 03060

XRunner is a registered trademark of Mercury Interactive Corporation,
470 Potrero Avenue, Sunnyvale, CA 94086

ViSTA is a registered trademark of VERITAS Software Corporation, 1600
Plymouth Street, Mountain View, CA 94043

# Introduction

This is an evaluation of the TestMaster automated test generator
software developed by Teradyne Software and Systems Test Inc. This
evaluation is based on a 30-day pilot program conducted by both the
Integration and Test team of the Science Data Processing System at Jet
Propulsion Laboratory and Teradyne Software and Systems Test Inc. The
object of this pilot program was to evaluate TestMaster's ability to
provide test generation for the science data processing environment at
JPL. This environment includes science data interpretation and image
data enhancement applications.

TestMaster is a software package that enables test engineers to visually
model their applications into state machine-based models. Based on the
state machine model, TestMaster will automatically generate test scripts
that can be used to test the application-under-test manually or
automatically.

The application-under-test (AUT) for this pilot program was XVD. A few
of the Mars Pathfinder image processing applications were also selected
for system-level testing and also as standard VICAR (Video Image
Communication and Retrieval) applications. The specification for the XVD
can be found on the World Wide Web at

> *http://www-mipl.jpl.nasa.gov/GUIHelp_1.4/Xvd/Xvd.html.*

An automated execution and verification system developed by Mercury
Interactive called XRunner^®^ was used to automatically execute and
verify the TestMaster-generated TSL (Test Script Language) scripts for
testing XVD. For the VICAR applications, the TCL (Transportable
Applications Executive Command Language) scripts can be verified by
executing the generated PDF (Proc Definition File) under VICAR
environment. A code coverage tool developed by VERITAS^®^ Software
Corporation called ViSTA^^ Code Coverage was used to measure the code
coverage on some of the applications that were implemented with C
language. Currently, VERITAS Software Corporation has been converted to
CenterLine and its ViSTA Code Coverage Tool has been converted to QC
Coverage.

# Goals

The primary goal of this pilot program was to evaluate the suitability
of TestMaster software within the current testing process and the
quality of tests it generates for the science data process environment.

The pilot program's primary tasks involved designing, generating, and
executing tests using TestMaster's model-based technique. The tests
would be constructed to execute under automated test execution systems
(XRunner and/or VICAR). The JPL test engineer would construct TestMaster
models using the available specifications for XVD and selected VICAR
applications.

At the conclusion of this pilot program, we would evaluate the
following:

-   The effort needed to create preliminary tests.

-   The effort needed to generate test suites with varying test
    coverage.

-   The relationship between FRD (Functional Requirement Document)
    requirements and model elements.

-   The quality of manually generated test suites vs. the test suites
    generated by TestMaster.

-   The effort required to update existing models to respond to software
    changes.

# Background

The Integration and Test team of the Science Data Processing System at
JPL is currently testing the XVD Software and VICAR applications using
manually written and executed test procedures. XVD is a general-purpose
graphic viewer program that is used to display data files mainly in
VICAR image format. It is a Graphical User Interface (GUI) application
that is driven by user interactions. It has many options and user
preferences that can be configured by the user interactively.

Although the pilot used XRunner as test harness, many of the science
data processing applications are command-line interface oriented. These
applications are all operating under VICAR. VICAR is a set of computer
programs and procedures designed to facilitate the acquisition,
processing and handling of digital image data. It is based on the
Transportable Applications Executive (TAE) that was developed for the
NASA Goddard Space Flight Center. User interfaces with VICAR by issuing
a series of TAE Command Language (TCL) statements. One of the goals of
the pilot was to determine the successfulness of modeling VICAR
command-line applications.

# Pilot Program Approach

The Integration and Test engineer assigned to conduct the pilot program
would approach the task as follows:

1.  Become familiar with black box testing techniques.

2.  Become familiar with TestMaster.

3.  Become familiar with the XRunner as a test harness.

4.  Construct the XVD and VICAR application software models.

5.  Generate test scripts for XVD functions to execute automatically on
    the XRunner test harness.

6.  Generate Proc Definition Files (PDF, file contains TCL commands for
    command-line interface applications) to execute automatically under
    VICAR.

7.  Execute the TestMaster-generated TSL scripts on the XRunner test
    harness.

8.  Execute the TestMaster generated PDF under VICAR and measure
    coverage when possible.

9.  Compare the effort to design, generate and execute tests using
    automated testing techniques with that of the current manual testing
    process.

10. Compare the quality and coverage of the tests generated by each
    method.

Finally, Integration and Testing and Teradyne would jointly author a
report on the results of the 30-day pilot program.

Teradyne provided a four-day classroom-training course on the use of
TestMaster. The course included lecture, laboratory exercises, and
take-home reference materials for the students to keep. The course also
provided "hands-on" experience with the tool under instructor
supervision.

# Current Testing Process:

The current Integration and Test process block diagram is shown below.
The testing process conducted by the Integration and Test team can be
divided into three phases. (See Figure 1)

![](media/image1.wmf)

[]{#__RefHeading___Toc400120116 .anchor}Figure 1, Current Integration
and Testing Approach to Software Testing

**Phase I (Preliminary Test)**

This is the initial approach phase. In this phase, the test engineer
performs initial checking of the delivered applications for each
subsystem. The test engineer verifies the build status of the software
and identifies the changes that were made. If the application is new to
the engineers, then the learning process also occurs in this stage.

**Phase II (Module Level Test)**

In this phase, the test engineer executes the delivered unit tests and
generates new test cases to verify the correctness of the module. The
test cases are generated manually. The validity and coverage of the
tests generated depend on the engineer's knowledge of the software and
his or her testing experience. The amount of test effort is also
constrained by the given testing schedule.

**Phase III (End-to-End, System Level Test)**

In this phase, the test engineer's focus is on the integration among
applications. Since JPL's science data processing software are driven by
data, different methods of data generation and manipulation can affect
the sequence of the processing operation that has direct affect on the
end products (i.e. images and/or scientific data).

**TestMaster Insertion into Process**

It is the Integration and Test team's believe that TestMaster will
improve the current testing process by reducing the time spend on Phase
II and direct engineer's focus toward the end-to-end system level
testing. Since models built by TestMaster can easily be tailored to
generate tests that can quickly verify the changes to the software, the
time spend on Phase II can be greatly reduced. By importing models built
for Phase II and integrating them together as subsystem, the test
engineer will be able to generate models for Phase III with very little
effort.

[]{#__RefHeading___Toc400199537 .anchor}XVD Software

XVD is a Graphic User Interface image viewer. It is mainly designed to
view VICAR image files and allows the user to perform various image
enhancements. XVD is an application saturated with configurable user
preferences and image manipulation options. It provides the user with
information on each pixel of the image and a histogram on color
distribution. The primary functions of XVD are its zoom capability and
its rich stretch algorithm functions. Both of these functions enable
scientists to surface any hidden information within the digital image.

# Black Box Test Techniques

The TestMaster tool presupposes a Black-box testing strategy.

Testing strategies are either structural or behavioral in nature.
Structural testing is also called "glass-box" or "white box" testing.
Behavioral or functional testing is called "black-box" testing.

Testers perform Glass-box testing by having complete access to the
system-under-test's design. In the case of a software program, this
approach allows the tester to ensure that every source code statement
and every branch of a conditional check executes correctly.

Black-box testing, is based upon the functional, performance, and
interface requirements of the system-under-test. Knowledge of the
internal workings of the system is not needed and is even avoided. This
approach encourages the tester to concentrate on ensuring all of the
requirements of the system are met. Tests are constructed without being
influenced by the details of the system implementation.

Often, a hybrid test strategy occurs, with unit level testing performed
using the "white-box" approach and higher system level testing performed
using the "black-box" approach.

Teradyne provided *[Black-Box Testing: Techniques for Functional Testing
of Software and Systems]{.underline}* by Boris Beizer, John Wiley &
Sons, 1995 as an introduction to black-box testing as part of the course
material during TestMaster training.

# TestMaster Description

TestMaster is a software tool developed by Teradyne Software & Systems
Test. It is an automatic test script generator designed to test software
and hardware systems using the black-box approach to testing.

## TestMaster and the Test Process

TestMaster creates test scripts for automated or manual test execution.
It does not address test execution and validation. Test execution and
validation are performed by the XRunner and VICAR execution environments
for this pilot program. The execution and validation aspects of the
testing process still require a significant amount of manual effort and
training. The effort is expended primarily in executing the appropriate
test steps and validating that the system has responded correctly to the
stimulus. This is a repetitive process. Executing the steps and
interpreting the system response also require training and experience
with the system.

TestMaster focuses on Test Generation, which is prior to test execution.
The figure below shows a Block Diagram of the test process used in the
pilot. (See Figure 2 and Figure 3)

![](media/image2.wmf)

######## 

######## Figure 2, Functional Testing with XRunner and VERITAS ViSTA TEST

![](media/image3.wmf)

Figure 3, Functional Testing with VICAR and VERITAS ViSTA TEST

## TestMaster Tool

The tool is possible by the development of fourth generation, modern
visually programmed testing tools that provides both a human and machine
readable model of the system-under-test. Using this class of tool, the
test engineers are encouraged to:

-   Operate at a higher level of abstraction (only the system's behavior
    is relevant).

-   Focus on the test goals and strategy.

-   Delegate the generation of the actual test scripts to the automated
    tool.

TestMaster runs on a variety of UNIX-based workstations. It is comprised
of three major elements: a graphical editing tool (Model Editor), a test
program generator (Path Generator) and a Model Debugger. (See Figure 4)

![](media/image4.wmf)

Figure 4, TestMaster Elements

TestMaster is both a tool and a methodology. The software tool supports
the Test Engineer's application of the methodology. The methodology
first requires the construction of a behavioral model of the
system-under-test. Testers construct this model using TestMaster's
graphical editing tools while referring to the specification (in this
case the XVD specification and VICAR software specifications) of the
system-under-test. When a valid model of the system exists, TestMaster's
path generator executes to generate a suite of test scripts in the
language of the target-automated tester (i.e. test harness).

TestMaster models are hierarchies of Extended Finite State Machines
(EFSM). An ESFM is an extension of the classic state machine notation
used in engineering design. Like classic state machines, ESFMs are
constructed from States, Transitions and Events. In addition, they
include several extensions for the behavioral modeling environment. The
use of EFSMs solves the basic problems with using classic state machines
for model-based engineering problem solving.

The first of these problems is *state explosion.* State explosion is the
tendency for models of complex systems to become very large. Large
models contain a lot of states and transitions. This makes them
unmanageable (i.e. difficult to understand and edit). EFSMs solve this
problem by embedding historical context into the models through an EFSM
feature called a Predicate.

In addition, hierarchies of state machines allow the engineer to create
state machines within state machines. This technique of layering allows
construction of complex models that are built-up from simple state
machine models.

A Predicate is a Boolean condition that must be true in order for a
state transition to be a legal path between states in the behavioral
model. The predicate checks the context of a model, unlike a traditional
state machine that has no historical context. Predicate information is
typically an integral part of the model specification for the
system-under-test. Predicates are a component of EFSM transitions. For
example, an image file must be open before any kind of zooming and
stretching can be performed.

A second problem with state machine-based model test generation is it
tends to generate far too many tests for practical use. Large numbers of
trivial tests and unwanted permutations of previous tests can result
that are uneconomical to execute on even a highly automated test
execution environment. TestMaster overcomes this problem with the EFSM
Constraint feature.

A constraint is a control condition embedded in a state transition to
guide the path generator. A constraint is not a predicate, because it is
*not* part of the system-under-test's specification. Constraints provide
the model builder with a convenient way to limit the number of tests
being generated for execution on the test harness or performed manually.
Using constraints, tests that are not interesting to the tester can be
"filtered" out of the test suite. Constraints are a component of EFSM
transitions. (See Figure 5)

![](media/image5.wmf)

Figure 5, Model and Programmable Elements

TestMaster produces test scripts by embedding manual or automated script
code statements into the model. These statements or "snippets" are
embedded into the model's transitions. The model builder embeds the
statement or command required for the test harness to drive the
system-under-test from the current state to the next state or verify
correctness of a subsequent action. Because the model is behavioral, the
paths though the model's logic will contain the valid executable test
statements needed to drive the system-under-test under all circumstances
and scenarios. When model construction is finished, the model serves as
an input to TestMaster's Test Generator. The Test Generator concatenates
the statements in the model's transitions into executable test scripts.

The Path Generator is a software engine that generates a suite of test
scripts in the test language choice (automated or manual). The engine
can generate a range of test suites for the system-under-test varying in
degrees of test coverage. For example, the test engineer may select
transition cover (basic degree of coverage) or full cover (most
comprehensive degree of coverage) on a model by model basis.

When the Path Generator is set to transition cover it will find the
minimum number of tests required to make sure that each modeled
transition and path is traversed at least once. This is equivalent to
all transitions in the behavioral model being included in the test suite
in at least one test. In contrast, full cover generates tests for all
transitions, to all states, in all contexts - a set of tests that
represent every possible path and permutation of paths through the
model.

A range of options exists between transition and full coverage. The test
engineer can control the Path Generator to select the range of test
coverage desired. The range between transition and full cover is
referred to as hybrid coverage. In addition, "filter" capabilities exist
to only cause the generation of tests that meet user defined criteria.
For example, a test engineer may elect to create a test suite containing
only tests exercising newly added functions to the system-under-test.

These features ensure the user can generate tailored test suites. A
tailored suite would have the maximum economic value within the
available test execution time and resource budget. The practical result
is that the user can generate tests for specific purposes, such as:
regression testing, overnight build testing, bug detection, etc..

The test engineer does not have to wait to complete a model of the
system-under-test before generating useful test scripts. Partially
complete models can generate useful tests. In this regard, TestMaster
supports an incremental and continuous improvement process of model
building.

# XRunner Test Harness

XRunner is an automated testing tool for X Windows applications. It
offers user the capability to create adaptable and reusable test scripts
that can verify the functionality of the application-under-test. It
enables the user to create tests by recording how the user interacts
with the application. Every point and click applied to the GUI
application is recorded in a test script that is in C-like Test Script
Language (TSL). The tester can als enhance the generated test script by
manual programming with TSL. (See Figure 6)

XRunner operates in two modes for test recording:

[Context Sensitive mode]{.underline} records user actions and
interactions in terms of GUI objects. The physical location of the
object on screen is not important. The TSL script generated will
describe the object selected and action performed. In order for XRunner
to operate under this mode, the tester must first generate a GUI Map
through XRunner's Learn feature. Since the GUI Map is stored separately
from the test scripts, changes to the application's user interface only
requires the GUI Map to be updated. The Integration and Testing team was
unable to configure XRunner to operated properly under the Context
Sensitive mode, therefore automated verification of the test scripts
generated from pilot was not performed.

[Analog Sensitive mode]{.underline} records every mouse clicks, keyboard
inputs, and the exact coordinates traveled by the mouse. The mode is
generally being used where the exact mouse coordinates are important to
the application-under-test, such as drawing applications. Since
different workstations may configure their display devices differently,
the TSL script generated under this mode may not be portable.

![](media/image6.jpeg){width="5.995833333333334in"
height="3.651388888888889in"}

Figure 6, Mercury Interactive's XRunner.

# VERITAS ViSTA TEST Code Coverage Tool

ViSTA TEST is a code coverage measurement tool. Any C or C++ application
can use this tool to determine what code fragments has and has not been
exercised. This enables the software developer and test engineer to
release software with greater confidence. Code that has been specially
compiled with the coverage compiler will report each item as it is
exercised by the test suite. Reports generated by ViSTA TEST will report
the measurements of functional call entry/exit coverage, basic block
coverage, code segment coverage, test coverage partitioning, path
coverage, error-seeding, interface coverage, simulation, and exception
coverage (for C++). Complexity of the software is the base for
measurement of coverage. ViSTA TEST supports several static metrics for
measuring Cyclomatic Complexity, Essential Complexity, Halstead's
Software Science, and CK Metrics. (See Figure 7)

![](media/image7.wmf)

Figure 7, ViSTA Utilities model

# Pilot Generated Test Models

In order to appreciate the modeling strategy applied during the pilot,
it is useful to revisit the nature of unit level testing as compared to
systems level testing. A unit typically encompasses a single system
function. Developer typically performs unit test before the unit is
being delivered. It is also being performed by test engineer during the
first two phases of software testing process. In addition, new system
functions are added at the unit level, binding unit test to regression
testing. It is advisable to test exhaustively at the unit level. This
can be a practical strategy because when the unit is considered in
isolation, the number of tests required for an exhaustive test, while
large, is still manageable.

On the other hand, it is rarely possible to even consider exhaustive
testing at the system or subsystem level (Phase III). The reason is that
the (sub)system is made up of numerous "units". The number of tests
required for exhaustive system coverage grows exponentially to numbers
that will exceed the ability of any automated tester or test engineer to
apply in a reasonable amount of time. Since system test combines
together many system functions to form a test scenario, the test
objective is to verify that the units interact properly in the system as
a whole, but not to exhaustively test each unit that makes up the
system.

The model strategy used for this pilot was based on the desire to have
"dual-use" models: one use being unit testing to test individual system
functions and the other system testing to test related functions. The
unit testing required building a model that could generate all possible
test cases in order to support the potential for exhaustive test
coverage of a single function. This same model, however, had to meet the
needs of system testing: the other dual use requirement. During the
course of test generation for system testing it is necessary that this
same model provide a more limited number of test paths. Limiting the
number of scenario paths is important for the reasons discussed above.

In the pilot, XVD is viewed as a "mega-unit" that is composed large
number of complex units. Therefore, the pilot approached XVD as a unit
not a system. System level testing was demonstrated through the modeling
of several Mars Pathfinder (MPF) applications. The modules selected for
the demonstration were MPFNAV, MCAULEY, and MPFMOS. The modeling style
used to model the System level testing was achieved through four steps.

1.  Identify the top-level software calling sequences and requirements.

2.  View each unit as a submodel of the system.

3.  Design and construct each submodel with respect to the system model.
    Unit testing was also being performed during this step.

4.  Integrate the submodels to produce the complete system model
    designed from the first step. As mentioned earlier, the number of
    test generated for system level coverage may grow exponentially.
    Therefore, fine-tuning of the system model is necessary.

## XVD Modeling and Test Generation 

The modeling for XVD employed a hierarchical modeling strategy, where
the submodel was defined by each encapsulating window of the
application. Since XVD is a GUI application, modeling of user events is
necessary to produce the correct TSL script. This has greatly increased
the complexity of the model. The pilot has produced a partially complete
model of XVD. It successfully generated test cases for the File_Menu and
Edit_Menu. Stubs were used to simulate other yet to be defined
submodels.

The top-level view of the application contains four primary objects.
These are the items on the menu bar. When there is no image being
displayed, the only interaction a user can generate through the items on
the menu bar. The Setup and Cleanup submodels are defined for initiation
and termination of XRunner and XVD. (See Figure 8)

![](media/image8.jpeg){width="6.0in" height="3.6in"}

Figure 8, Top-level XVD model

The File_Menu submodel contains only two selections: Open and Exit.
However, user can generate several events under this menu. The pull-down
menu has Tear-Off capability and it can be driven by accelerated
keyboard entries. The pilot has fine-turned the user-generated events by
placing emphasis on the mouse clicks. (See Figure 9)

![](media/image9.jpeg){width="5.998611111111111in"
height="2.3208333333333333in"}

Figure 9, File menu submodel

The Open_Menu submodel is defined by a standard Motif file-selection
Widget. The objects in this window can be grouped into three submodels:
Set_Filter, Set_Directory, and Set_File. The Filter selection box allows
the user to filter only the type of file he or she wishes to see in the
file-selection box (e.g. \*.img, earth.\*). The Directory selection box
allows the user to select a directory through mouse clicks. The File
selection box lists out all available files within a given directory
with respect to the input filter. Since this is a GUI application,
user-triggered events can be in any combination. A good example of this
is the ability to change the directory when the file cannot be located
in the current directory. Various transitions were added to model the
possible user-triggered events. Fine-tuning of the model is required to
avoid infinite loops. Each of the submodels defines the complex user
inputs events generated through mouse and keyboard interactions. (See
Figure 10)

![](media/image10.jpeg){width="5.996527777777778in"
height="4.357638888888889in"}

Figure 10, File open submodel

The submodel Edit_Menu defines the three primary objects where user
interactions would occur. Like the File_Menu submodel, users can
generate several events under this menu other than just using the mouse
and pointing to one of the three primary objects. The pull-down menu can
be tear-off and triggered by accelerated keyboard entries. Fine-tuning
of the submodel is required to properly simulate the common user
interactions. The Preferences selection allows the user to interactively
configure the appearance of XVD. Data Range allows the user to set the
color distribution of the image. Undo is to reverse the previous action.
(See Figure 11)

![](media/image11.jpeg){width="5.99375in" height="3.0506944444444444in"}

Figure 11, Edit menu submodel

The Preferences submodel defines all possible user events in selecting
the preferences of the application. The main focus of this submodel is
the Display_Preferences, which defines the possible user selection on
the appearance of the XVD application window. To fine-tune this
submodel, a COMBINATION function is invoked to make sure all possible
combinations of user events will be covered. (See Figure 12 and Figure
13)

![](media/image12.jpeg){width="5.997222222222222in" height="3.6in"}

Figure 12, Preferences submodel

![](media/image13.jpeg){width="5.998611111111111in"
height="3.7895833333333333in"}

Figure 13, Display preference submodel

## Mars Pathfinder (MPF) Subsystem Modeling and Test Generation 

Modeling of Mars Pathfinder image-processing subsystem is performed by
following the four-step process described earlier. The modules involved
in this subsystem include MPFNAV, MCAULEY, and MPFMOS. To derive the
top-model system model, the model engineer needs to first determine the
calling sequence for the modules, and define the relations between the
modules. (See Figure 14)

![](media/image14.jpeg){width="5.997222222222222in"
height="2.188888888888889in"}

Figure 14, Top-Level System Test model for MPF.

Models where derived by grouping of program parameters. Instead of
treating them as parameters, the model engineer treated the submodels as
processing entities. Since these modules were all designed for MPF image
processing, they shared some common input parameters. Therefore, many of
the submodels can be reused among them. One of the observations through
the pilot program was reusability of common models. If we treat
submodels as subroutines, we can achieve reusability of submodels by
adapting the same disciplines as software development. (See Figure 15,
Figure 16, and Figure 17)

![](media/image15.jpeg){width="5.99375in" height="3.4625in"}

Figure 15, MPFNAV functional model

![](media/image16.jpeg){width="5.991666666666666in"
height="3.1243055555555554in"}

Figure 16, MCAULEY functional model

![](media/image17.jpeg){width="5.992361111111111in"
height="3.5131944444444443in"}

Figure 17, MPFMOS function model

# Reusable Submodels

One of the major discoveries by the model engineer through modeling
VICAR command-line applications was the concept of reusable submodels.
As mentioned earlier, there are common parameters among image processing
applications. Re-modeling of identical submodels between applications
can by quite time consuming and inefficient. The model engineer
developed generic submodels that can be easily configured and interfaced
into application models.

An example of reusable submodels was the generic model to generate
values for parameters such as SIZE, SLAREA, and SSAREA. The SIZE
parameter expects a list of four arguments: starting line, starting
sample, number of line, and number of sample. The parameter SLAREA
(starting line area) and SSAREA (starting sample area) are lists that
expect a variable number of arguments. The size of the list depends on
the number of input images. These parameters are different, but they
have their similarities. They all have the same format that is a list of
numbers separated by commas and enclosed within parentheses.

The packaged generic submodels derived by the pilot have similar
components as an image processing subroutine's .COM file. It has a
top-level test model to demonstrate the functionality of the generic
submodel. (See Figure 18)

![](media/image18.jpeg){width="5.995138888888889in" height="2.8875in"}

Figure 18, Test model used to demonstrate the functionality of Combo\_

Each submodel has help documentation that describes the interfaces and
purpose of the submodel. (See Figure 19)

![](media/image19.jpeg){width="3.2895833333333333in"
height="3.8847222222222224in"}

Figure 19, Documentation for submodel Combo\_

The Combo\_ submodel was used to generate combinations of numbers for
parameters SIZE, SLAREA, and SSAREA. It expects two arguments:
combo_maxValue and combo_maxIterate. When integrating this submodel into
the application model, the engineer assigns a maximum sample or line
value to combo_maxValue and assigns the number of images to
combo_maxIterate. For the SIZE parameter, the engineer assigns 4 to
combo_maxIterate. Submodels within an application or system model have
unique naming requirements. The underscore in the submodel name
"Combo\_" is used to avoid name collision with other existing submodels
within the application or system model. (See Figure 20)

![](media/image20.jpeg){width="5.996527777777778in"
height="3.004166666666667in"}

Figure 20, Implementation of Combo\_

The submodel Combo\_ was integrated into the MPF application models to
assist in generation of SLAREA and SSAREA. It was also used to generate
MLLQ and NORMAL with minor modifications in the number sequence to be
generated. In the application model for MPFMOS, the subroutine was
renamed to mpfmos_SLAREA and mpfmos_SSAREA to reflect the function of
the submodel. (See Figure 21)

![](media/image21.jpeg){width="6.0in" height="1.8541666666666667in"}

Figure 21, Combo\_ integrated into MPFMOS application model.

The submodels can be stored as a TAR file in a common access area in
Integration and Test, rather than as a packaged .COM file.

# Pilot Task Evaluation

Initially, the stated goals for evaluating TestMaster included a
complete model of the XVD and demonstrate the testing sequence under
XRunner. This was too ambitious a goal for a 30-day pilot program. It is
a time consuming and complex task to model every user event for XVD due
to the complexity of the application. The biggest obstacle was not
learning TestMaster, it was correctly configuring XRunner to operate
properly under the current science data processing environment. However,
the generation of the MPF System Test model was a success.

Due to the limitation of XRunner has placed over this pilot program,
only a partial XVD model was generated. It was decided that proper
modeling of the File_Menu and Edit_Menu would be enough to prove the
suitability of TestMaster for generating TSL test scripts. The model
that resulted establishes the infrastructure for testing the entire
software, while at the moment only testing selective objects. The
remaining objects can be added-on incrementally to make the model a
fully functioning representation of XVD. The TSL scripts generated by
TestMaster were required to execute under the Context Sensitive mode of
XRunner, which is currently not operational. Another limitation of
XRunner has placed over this pilot program was its failure to learn the
GUI window objects, which is required to generate the GUI Map file for
Context Sensitive execution. However, the correctness of the model can
be verified by hand to ensure the proper generation sequence.

The modeling of MPF Subsystem model was a success. The three selected
candidates were properly modeled and were all tested with Transition
Coverage. Several software failures were discovered during the execution
of the generated test scripts. Constraining the models was necessary to
sort-out redundant test cases and improves test generation time. The
efforts required to integrate the submodels were very minimal.

During our modeling activity, we discovered the specification for the
software did not provide enough detail to accurately model some
functions and objects of the software. It became necessary to consult
with the software systems engineer to clarify specific functions before
they could be modeled. This observation suggests that the specification
alone may not always provide an unambiguous requirement specification
for the subsystem developers.

Since TestMaster uses EFSM's to model the system for testing, it is
possible that in addition to generating "black box" tests, it could also
be used to actually define the software requirements specifications.
This would have the added benefit of reducing the number of system
defects caused by incomplete and/or conflicting specifications. The
TestMaster models could also be used in other contexts that require
specifications. (See Figure 22)

![](media/image22.jpeg){width="5.997916666666667in"
height="3.8368055555555554in"}

Figure 22, Partial MPF image processing subsystem specification model.

The layered modeling approach, as discussed in the previous section, is
a compromise between the conflicting goals of a totally exhaustive test
and a carefully controlled scenario for validating a subsystem. As noted
below in the TestMaster tools observations section, if the tool had the
ability to associate different constraint sets with the same ESFM
diagrams, it would further reduce the effort needed to generate models
to perform both scenario and exhaustive testing.

The use of a model-based testing strategy allows the test engineer to
easily create new scenarios without being knowledgeable of the automated
test language and the underlying tester implementation. The model
becomes the knowledge base for this information. However, adding or
changing functionality of a model requires that the test engineer be
knowledgeable of the underlying tester's capabilities.

The tool is capable of generating thousands of tests for validating the
subsystem under test. The challenging part of TestMaster is in applying
constraints to the models. Constraints are needed to reduce the number
of tests to as few as possible giving the largest amount of coverage,
and being executable within the given time and budget limitations.

## Test Quality Comparison 

A review of the tests generated by TestMaster models shows that it
produced comparable coverage to the current, manually created tests. The
pilot model generated much more tests.

At the time of the pilot program, there was no complete subsystem test
for the Mars Pathfinder imaging applications. The number of tests
generated manually was limited and was not sufficient. The quality of
the tests generated by TestMaster for the subsystem test revealed the
limitations of manual test generation. One of the advantages of using
TestMaster is the reduction in manual labor for the test engineer. PDF
and TSL scripts were generated and can be executed under test harness
without any manual modifications. The coverage of the tests generated
was desirable.

The TestMaster test suite detected a few failures in the MPF
system-under-test. The specifications for MPFNAV, MCAULEY, and MPFMOS
stated the applications could handle images taken by all cameras.
However, these applications fail to handle any of the images taken by
the Pathfinder Rover. A great portion of the code was not covered due to
these failures, because there are three cameras onboard of the Rover.
There are software packages developed specifically for the Rover images.
If these applications were designed only to handle Lander's images, then
the code fragments need to be removed and the specifications of the
applications need to be modified. Other problems found involved the
SLAREA and SSAREA parameters. Different combinations of valid size
values can trigger Abnormal Process Termination of MCAULEY.

In addition, the TestMaster model produced 140+ tests in Hybrid, and
5000+ tests in Full coverage schemes for each of the MPF applications.
Hybrid and Full Coverage tests are not part of this comparison.

### Comparison Summary

There were huge increases in test coverage on the tests generated with
TestMaster compared to tests generated manually.

The TestMaster tests contained more test steps per test than the
existing suite. The automatically generated tests included some trivial
test steps, which were determined to be outside of the normal use
pattern for the function. The Teradyne engineer says that a more
rigorous constraining of the MPF models would eliminate test steps
outside of the normal use pattern. However, these steps are allowed by
the specification and are relevant to the function's test.

Finally, the tests generated with TestMaster covered a wider range of
executions. The time spent on model construction can vary from a few
hours to one day of test engineer's time for a typical VICAR
application. Tests generated with Transition Coverage were sufficient
and adequately exercise all functionality of a given application.

# System Modeling Estimate

In order to make a fair estimate as to the time required to complete the
modeling process for XVD and MPF subsystem, it is necessary to take a
closer look at where the time was spent on the pilot program.

## Basis for the Estimation

The amount of time it takes to complete the modeling process depends on
many different factors. A list of the most important factors is shown
below.

-   Knowledge of the test harness.

-   Resource availability.

-   Model reusability.

-   Knowledge of the system under test by the model engineer.

-   Completeness of the Specification.

-   Function/Feature complexity.

-   Skill of the modeling engineer(s).

-   Learning curve.

One of the limiting factors for the pilot program was the configuration
of the test harness. Since XRunner's Learn feature was not operational
under the Science Data Processing environment, the model engineer was
unable to identify the correct logical names to many of the window
objects to XVD. Even though the flow of events generated by TestMaster
has correctly simulated most of the user interactions, the logical names
embedded in the output TSL script need to be updated after XRunner has
been properly configured.

Modeling software with EFSMs through specifications is a new paradigm to
the Integration and Test team. Just as there is more than one way to
implement a given algorithm, there is more than one way to model a piece
of software with EFSMs. Disciplines in modeling must be adept to improve
the reusability and maintenance of the model. Models generated by the
model engineer were being modified numerous times, because of the
engineer constantly learning new ways to improve the efficiency and
maintainability of the system models.

In order for the models to effectively reflect the behavior of the
application-under-test, the modeling engineer must have working
knowledge of the application-under-test. The Integration and Test team
has long history of testing GUI systems and VICAR software. Therefore,
the model engineer was able to begin the pilot with only learning
TestMaster and learning to configure XRunner. It was obvious to the
modeling engineer that lack of documentation of the system-under-test
will present real challenges to the modeling process. The consistency of
the documentation is also a huge factor in the correctness of the final
model product.

The learning curve was not a huge factor, due to the model engineer's
experience with the systems-under-test. Learning to use TestMaster was
achieved through the in-class session. Learning to configure XRunner
presented a real challenge to the pilot, which was not resolved at the
end of the pilot program. Learning to model with EFSMs was a new
experience to the Integration and Test team. The engineer's modeling
skills were improved significantly at the end of the pilot program.

## Labor Hours Analysis

The hours spent on the pilot program are shown in the figure below:

![](media/image23.wmf)

Figure 23, Total Time Spent during Pilot Program and the breakdowns.

## TestMaster vs. Manual Test Generation

It should be obvious that once the model is built, the time for test
generation is minimal. Adapting to TestMaster will require a paradigm
shift within the Integration and Testing team, because modeling with
EFSM is a new concept to the team. Therefore, it was the model
engineer's belief that there will not be a significant improvement in
test time during the initial implementation of TestMaster. The time it
takes learning TestMaster, modeling with ESFM, and developing subsystem
models may even slow down the testing process during initial
implementation. However, the time for future test generation will
improve substantially after the models have been built. Since most
software maintenance will only involve adding new capabilities or
correction of documented failures, modifications to the existing models
will require very little effort. A 5-year projection was made at the end
of the pilot program to compare the time difference after implementing
TestMaster.

+--------+-------------------+-------------------+-----------------+---+
|        |                   | **Test with Test  | **Traditional   |   |
|        |                   | Master**          | Way**           |   |
|        |                   |                   |                 |   |
|        |                   | **(Hour)**        | **(Hours)**     |   |
+--------+-------------------+-------------------+-----------------+---+
|        | System Function   | **8**             | **8**           |   |
|        | Analysis          |                   |                 |   |
+--------+-------------------+-------------------+-----------------+---+
|        | Develop Model     | **16**            | **0**           |   |
|        | Strategy          |                   |                 |   |
+--------+-------------------+-------------------+-----------------+---+
|        | Modeling          | **96**            | **0**           |   |
+--------+-------------------+-------------------+-----------------+---+
|        | Model Debug and   | **80**            | **370**         |   |
|        | Test Generation   |                   |                 |   |
+--------+-------------------+-------------------+-----------------+---+
|        | Test Execution    | **28**            | **50**          |   |
+--------+-------------------+-------------------+-----------------+---+
|        | **Total Time for  | **228**           | **428**         |   |
|        | initial           |                   |                 |   |
|        | implementation**  |                   |                 |   |
+--------+-------------------+-------------------+-----------------+---+
|        | Additional        | Two               | Two             |   |
|        | maintenance and   | deliveries/year   | deliveries/year |   |
|        | new case          |                   |                 |   |
|        | generation over 5 | It will take 10   | It will take 50 |   |
|        | years             | hours to learn    | hours to learn  |   |
|        |                   | about the new     | about the new   |   |
|        |                   | changes and       | changes and     |   |
|        |                   | modify existing   | manually        |   |
|        |                   | model.            | generate test   |   |
|        |                   |                   | cases.          |   |
|        |                   | A total of 20     |                 |   |
|        |                   | hours             | A total of 100  |   |
|        |                   |                   | hours           |   |
+--------+-------------------+-------------------+-----------------+---+
|        | In 5 years        | **100**           | **500**         |   |
+--------+-------------------+-------------------+-----------------+---+
|        | **Total**         | **328**           | **928**         |   |
+--------+-------------------+-------------------+-----------------+---+

## Cases Generated

There were two sets of cases generated for each model, Transition and
Full Coverage. However, only the Transition Coverage test cases were
executed during this pilot program. The main focus of the model engineer
was not just to increase number of test cases for the
Application-Under-Test, but to review the quality of the test cases to
ensure all parameters or user-triggered-events were covered.

  --------- ------------------------ ----------------- ----------------- --
            **Application/System     **Transition      **Full Coverage** 
            Under Test**             Coverage**                          

            XVD                      22                1687              

            MPFNAV                   9                 5000+             

            MCAULEY                  23                5000+             

            MPFMOS                   13                5000+             

            MPF system test          21                5000+             
  --------- ------------------------ ----------------- ----------------- --

# Code Coverage

One of the goals of the pilot was to measure the code coverage of the
tests generated by TestMaster. Some of the generated Transition Coverage
test cases were measured against the traditional unit tests. While 100%
code coverage is always ideal for test engineers, a long history of
revisions and maintenance to the testing applications, and existing
problems of the applications have placed limits on the code coverage
outputs.

As mentioned earlier, several software failures were discovered in some
of the MPF software. These failures have prevented coverage on some code
fragments of the applications. Another limitation was "dead codes". A
long history of code modifications, such as changes in calling sequences
in existing subroutines, have prevented some code fragments to be
covered. Changes to the application's PDF can generate obsolete
parameters in the applications. Removing a parameter from the
application's PDF, but failure to remove the code fragments that are
triggered by the parameter from the application source can do this.

Code coverage was measured with ViSTA TEST coverage tool. Each set of
tests for a given application was measured through its Call Coverage,
Exit Coverage, Implicit Exit Coverage, Basic Block Coverage, and Segment
Coverage. There was an average of 30% increase in code coverage for
Transition Coverage. Full coverage test cases were also generated, but
they were not executed. The limitations on available resources (machine,
disk space, and time) have prevented the pilot to measure the coverage
on Full Coverage tests.

# TestMaster Tool Observations and Results

The pilot program was executed using TestMaster version 1.80. Our
overall impression of the tool is that it is suitable for the Science
Data Processing environment. The quality of the generated tests is
directly proportional to the quality of models constructed. The modeling
time varies with respect to the application under test. It is also
dependent on the goal of the model engineer. If the model was
constructed to be "disposable", that is, no model reuse was intended,
then modeling can be done in a matter of a few hours. If reuse and
future expansions are expected, then more detail planning and design are
required. In the case of system-level testing, the model engineer has
taken the second approach. It is the model engineer's belief that
software-engineering concepts should be applied in all model
constructions.

The model engineer provided Teradyne with several suggestions for
improvements on TestMaster. Some of the suggestions have already been
implemented in the next release of the software. These suggestions
include:

**Reference to Global Variable:**

This was a bug discovered during the pilot. A global variable was
referenced in a submodel, which was used as a loop-control-variable. The
submodel was not able to retrieve the global variable, and the submodel
entered an infinite loop. The Teradyne engineer has submitted a bug
report to the TestMaster development. The changes should be reflected in
the next point-release.

**Submodel Renaming:**

When a submodel is renamed through right mouse click on the submodel
icon, TestMaster creates a new submodel with the new name and discards
all contents of the old submodel. The work-around to this problem was to
rename the submodel through the main application window. The Teradyne
engineer has submitted a change request to TestMaster development. The
changes should be reflected in the next release.

**Debug and Replay Capabilities:**

While there is a debugger in the current version of TestMaster, it has
provided little assistance in locating problems within the model. For
complex models, it is very easy to introduce contradicting conditions
and infinite loops to the model. TestMaster offers a replay feature that
can replay any generated path, but these paths are "bug free" paths
which offers no assistance in locating the infinite loop problems.

Example of the problem:

A complex model was constructed and the engineer began to execute the
Path Generator. The Path Generator generated *n* cases and it entered an
infinite loop when trying to generate the *n*+1 path. If *n* is a small
number and the model is relatively simple, the debugger may be used to
replay the generation process step-by-step to locate the problem.
However, it would be a very time consuming process when *n* is large
and/or the model is very complex. The replay feature can only replay any
of the *n* paths that were generated, but the engineer's interest is on
the *n*+1 path.

It is the model engineer's strong recommendation to Teradyne to
introduce a Replay UpTo feature to the Debugger. In the above example,
the engineer can tell the Debugger to run the Path Generator to generate
up to the *n* paths (i.e. the Path Generator will have all the
preconditions up to the *n*^th^ path in memory) and the real debugging
process can begin at *n*+1 path.

**User Interface:**

To start a TestMaster session, the user must supply a model name at the
command-line. TestMaster does not have an Open feature in its File menu.
It is the model engineer's recommendation to the TestMaster development
to implement an Open feature to the software.

# Conclusions:

**The effort needed to generate software models:**

Modeling software with EFSM is a task with several requirements. The
model engineer must have working knowledge of the application-under-test
and/or the system-under-test. The ability to reflect the behavior of the
application-under-test with EFSM is required. Boris Beizer's book on
Block-Box Testing supplied by Teradyne was a wonderful reference on
modeling with EFSM. As in software engineering, future reuse is a key
factor in model construction. When a model is intended to be reused and
integrating to future system tests, a higher level of abstraction is
required to ensure ease of integration and configuration. Once the model
is constructed, the effort required for future integration and
configuration will be minimal. Of course, this is dependent upon the
quality of model constructed.

**The effort needed to generate different levels of coverage:**

This requires very little effort to the engineer. TestMaster has
features that can generate various levels of tests. Model constraining
is required to generate an effective set of tests, therefore model
constraining also affect the number of output tests.

**The effort required to change generated scripts:**

For standard command-line VICAR applications, TestMaster can be
configured to generate a complete PDF without any retrofitting. For TSL
scripts, a LEX script was provided by Teradyne to extract generated
cases into small files. In both cases, the effort required to change
scripts was minimal.

**Overall Observation:**

TestMaster can provide higher quality tests over the current manual test
generation process. It is a paradigm shift to the Integration and Test
team by modeling software with EFSMs. Test generation with EFSMs has
definite advantage over current test generation process, because it can
generate all possible combination and permutation of tests.

The concept of reuse can greatly reduce the modeling time, since there
are many common parameters among image processing applications.
Construction of application model can be easily achieved by integrating
existing generic submodels together. Another advantage TestMaster offers
is the verification of software specification. The pilot has already
found several inconsistencies in the system-under-test through modeling.

# Glossary:

**Black-box testing** - A testing method in which the internal working
of the system is not know or even avoided. Testing's solely based upon
stimulus and response.

**Constraint** - A user-defined control embedded into a TestMaster model
that guides the path generator. It is not a part of the system
specification, only a control of the tests being generated.

**Domain Knowledge** - An understanding of the system under test. A
knowledge of how the features and functions of the system are supposed
to behave.

**Executable test script** - A full test script with a test sequence
from beginning to end, not just a small portion or step in the test
sequence.

**Exhaustive testing** - Testing every possible combination and
permutation of the model of the system under test. Testing every
possible input to verify every possible output.

**Extended Finite State Machine (EFSM)** - A graphical representation of
all of the different possible states a system can be in and how it gets
from one state to another. Extended finite state machines differ from
standard state machines by the use of predicates and constraints.

**Full Coverage** - Taking every possible path through a TestMaster
model.

**Function** - A single element of the system under test. A small part
of the system that has a specific purpose.

**Functional Model** - A model that represents only one function of the
specification.

**Functional Test** - A test to prove basic functionality of a specific
function or feature having a very narrow range.

**Hybrid Coverage** - A user defined level of test coverage between full
and transition coverage. In TestMaster, this is achieved by setting some
submodels to full and others to transition cover.

**Mode Model** - A model that represents the working environment of the
system under test.

**Negative case testing** - When invalid input data is applied to see if
the system will properly "trap" the error and handle it.

**Positive case testing** - When valid input is applied to verify a
positive or predetermined outcome.

**Predicate** - A condition that must be met in order to take a
transition to another state in a EFSM. This is usually a condition that
is defined in the specifications or a physical condition of the system
under test that is not defined in the code.

**Requirement** - A single statement that the system must obey.

**Snippet** - A piece of embedded code in a TestMaster model which will
execute the step need to take the transition it is embedded in.

**System-Under-Test (SUT)** - the hardware or software application which
is the object of testing.

**TAE (Transportable Applications Executive)** - It is the base of VICAR
that was developed for the NASA Goddard Space Flight Center.

**Test execution** - Performing the test or tests on the system. This
could be done manually or automatically.

**Test Generation** - Developing or generating the tests that need to be
executed on the system under test.

**Transition Coverage** - The tests generated by taking several paths to
cover each transition in the model at lest once.

**TSL (Test Script Language)** -- A scripting language used by XRunner
to describe all mouse and keyboard interactions. A test engineer can use
this language to customize specialized test cases.

**ViSTA TEST** - A code coverage measurement tool developed by VERITAS
Software Corporation. Any C or C++ application can use this tool to
determine what code fragments have and have not been covered.

**VICAR (Video Image Communication and Retrieval)** - A set of computer
programs and procedures designed to facilitate the acquisition,
processing, and handling of digital image data.

**XRunner** - An automated testing tool for X Windows applications
developed by Mercury Interactive. It offers user the capability to
create adaptable and reusable test scripts that can verify the
functionality of the application-under-test.

**XVD** - A Graphic User Interface image viewer. It is mainly designed
to view VICAR image files and allows user to perform various image
enhancements.