# Document Change history

  ---------------- ------------------ -----------------------------------
  Version Number   Date               Description

  Draft - 1        11/12/2006         Patrick -- template draft

  Draft - 2        12/1/2006          Patrick -- added some detail

  Draft -3         3/28/2007          Robert Morrell: WFU staff added

                                      

                                      

                                      

                                      

                                      

                                      

                                      

                                      
  ---------------- ------------------ -----------------------------------

**\
**

***Cancer Central Clinical Patient Registry(C3PR)***

**Test Plan**

**Version 2.0**

**TABLE OF CONTENTS**

1\. Document Change history [i](#document-change-history)

2\. Introduction [1](#introduction)

2.1 Scope [1](#scope)

2.1.1 Identification [1](#identification)

2.1.2 Document Overview [1](#document-overview)

2.2 Resources [2](#resources)

2.3 Referenced documents [2](#referenced-documents)

3\. Software Test Strategy [4](#software-test-strategy)

3.1 Objectives [4](#objectives)

3.2 APPROACH [4](#approach)

3.2.1 Unit Testing [5](#unit-testing)

3.2.2 Integration Testing [6](#integration-testing)

3.2.3 User Acceptance Testing (UAT) [6](#user-acceptance-testing-uat)

3.3 Description of Functionality [7](#description-of-functionality)

3.4 Specific Exclusions [7](#specific-exclusions)

3.5 Dependencies & Assumptions [7](#dependencies-assumptions)

3.6 General Criteria for Success [7](#general-criteria-for-success)

3.6.1 Readiness Criteria [7](#readiness-criteria)

3.6.2 Pass/Fail Criteria [8](#passfail-criteria)

3.6.3 Completion Criteria [8](#completion-criteria)

3.6.4 Acceptance Criteria [8](#acceptance-criteria)

4\. Software Test environment [9](#software-test-environment)

4.1 DUKE [9](#duke)

4.1.1 Software Items [9](#software-items)

4.1.2 Hardware and Firmware Items [9](#hardware-and-firmware-items)

4.1.3 Other Materials [10](#other-materials)

4.1.4 Participating Organizations [10](#participating-organizations)

5\. Test Schedules [11](#test-schedules)

5.1 Time Frames for Testing [11](#time-frames-for-testing)

6\. Risks [12](#risks)

Appendix A -- Acronym List [13](#__RefHeading___Toc152728838)

Appendix B -- Test report [14](#__RefHeading___Toc152728839)

# Introduction

This Test Plan prescribes the scope, approach, resources, and schedule
of the testing activities. It identifies the items being tested,
features to be tested, testing tasks to be performed, personnel
responsible for each task, and risks associated with this plan.

## Scope

This document provides instruction and strategy for incorporating
Software Testing practices and procedures into the C3PR project. This
document demonstrates the application of testing on this software. C3PR
Release 1 provides a highly configurable participant registry for
managing clinical study registration at a single site. However, it
cannot easily manage cross-site studies, and it can be very difficult to
install, configure, and manage. The primary goal of C3PR Release 2 is to
build upon Release 1 to make it more generally useful to the cancer
research community. A scoping goal will be for C3PR to manage
participant registration to multi-site studies. This involves the
integration of C3PR instances at different centers. Additionally,
integration beyond C3PR will be needed to perform institution-level and
PI-level eligibility checks. The current registration model is somewhat
limited, allowing for simplistic randomization based upon manual
assignment of participants to arms. Furthermore, protocol management in
C3PR will be greatly enhanced. Though C3PR is not intended as a central
protocol repository, protocol information is critical to the
registration process. Protocol information will be expanded to meet the
use cases of multiple eligibility check-points, multi-institutional
studies, and differing randomization schemes. C3PR Release 1 provided
registration from a participant-centric standpoint. This workflow will
be enhanced to provide registration from a protocol-centric view, and
reporting will be extended to provide this information to external
systems to generate complex reports.

The scope of testing on the project is limited to the requirements
identified in the project's Software Requirements Specification (SRS).
The project has been broken up into three phases (Elaboration,
Construction, and Transition) with one month iterations in each.
Requirements for separate functional areas are determined at the start
of each iteration. The impact of this on the Test Plan is that the
specifics of the test plan and the test data will be determined as
requirements are included in the SRS.

See the C3PR V2 Vision and Scope Document for more detailed information
on project scope.

### Identification

C3PR V2

### Document Overview

This Test Plan defines the strategies necessary to accomplish the
testing activities associated with C3PR. We expect that it will be
further developed as the development project proceeds. Testing
procedural instructions are included in the Standard Operating
Procedures (SOPs).

The remaining Test Plan sections are organized as follows:

-   **Section 3: Software Test Strategy**: Describes the overall
    approach and techniques to be used during testing.

-   **Section 4. Software Test Environment**: Describes the intended
    testing environment.

-   **Section 5. Test Schedules**: Contains or references the schedules
    for conducting testing.

-   **Section 6. Risks**: Identifies the risks associated with the Test
    Plan.

-   **Appendix A. Acronym List:** Defines the acronyms used on the
    project.

## Resources

-   Patrick McConnell, Duke: Technical Lead, Test Manager

-   Ram Chilukuri: Lead Developer

-   Manav Kher: Developer

-   Kruttik Aggarwal: Developer

-   Priyatam Mudivarti: Developer

-   Ramakrishna Gundala: Developer

-   Vijaya Chadaram: Domain Expert

-   Robert Morrell: Lead Elaborator

-   Steven Cheng: Analyst

-   Kimberly Hill Livengood: Domain Expert

-   Lisa Dixon: Domain Expert

-   Sharon Elcombe: Lead Elaborator

-   Diane Parkin: Domain Expert

-   Marin Shanley: Domain Expert

-   Patty Spears: Lead Elaborator of patient advocates

See the C3PR V2 Communication Plan for more details on stakeholders and
their roles.

## Referenced documents

For additional project specific information, refer to the following
documents:

-   Vision and scope document

    https://GForge.nci.nih.gov/plugins/scmcvs/cvsweb.php/c3prv2/documentation/communication/c3prv2_vision_scope.doc?cvsroot=c3prv2

-   Communication Plan

    https://GForge.nci.nih.gov/plugins/scmcvs/cvsweb.php/c3prv2/documentation/communication/c3prv2_communication_plan.doc?cvsroot=c3prv2

-   Test reports

    https://GForge.nci.nih.gov/plugins/scmcvs/cvsweb.php/c3prv2/documentation/testing/reports/?cvsroot=c3prv2

```{=html}
<!-- -->
```
-   Use case document

> https://GForge.nci.nih.gov/plugins/scmcvs/cvsweb.php/c3prv2/documentation/use_cases/c3prv2_use_cases.doc?cvsroot=c3prv2

-   SRS

    https://GForge.nci.nih.gov/plugins/scmcvs/cvsweb.php/c3prv2/documentation/requirements/c3pv2_srs.doc?cvsroot=c3prv2

-   Project plan

    https://GForge.nci.nih.gov/plugins/scmcvs/cvsweb.php/c3prv2/documentation/planning/c3prv2_project_plan.doc?cvsroot=c3prv2

-   Risk matrix

    https://GForge.nci.nih.gov/plugins/scmcvs/cvsweb.php/c3prv2/documentation/risks/?cvsroot=c3prv2

# Software Test Strategy

## Objectives

C3PR 2.0 will result in a production system that is fully functional
with respect to the requirements listed in section 2.3. The overall
object of this test plan is to provide unit, integration, and quality
assurance testing for the whole of the C3PR delivered software. Unit
testing is done during code development to verify correct function of
source code modules, and to perform regression tests when code is
refactored. Integration tests verify that the modules function together
when combined in the production system. User acceptance testing verifies
that software requirements and business value have been achieved.

## APPROACH

The testing approach is to convert the use cases described in the C3PR
V2 use case document into a number of automated unit and integration
tests to ensure the software conforms to the requirements. The following
proposes the approach for testing C3PR 2.0:

-   Create a clear, complete set of test cases from the use case
    documents and review it with all stakeholders.

-   Throughout the project, maintain the Requirements Traceability
    Matrix so that any stakeholders or tester has a concise record of
    what tests are run for each requirement.

-   All test cases will be command line accessible to take advantage of
    continuous integration testing thru the use of ANT for all testing
    phases.

Some of the practices that the Duke team will adopt are:

-   Derive test cases/unit tests from updated functional specifications
    of all relevant use cases. Unit tests and testing scenarios will be
    constructed in parallel with core development efforts, while
    validating the specifications against the relevant use cases. The
    use of diagrammatic representations of use cases in the form of
    task-based flow-charts, state diagrams, or UML sequence diagrams may
    facilitate creation of test cases and monitoring outcomes.

-   Teaming testers with developers to provide close coordination,
    feedback, and understanding of specific modules for testing
    purposes.

-   Ongoing peer-review of design and code as a team based form of
    software inspection. Each developer will review and run code written
    by another developer on a regular basis (acting as QA inspectors in
    turns), along with joint code review to gain consensus on best
    practices and common problems.

```{=html}
<!-- -->
```
-   Automated test execution using Ant and unit testing to support rapid
    testing, capturing issues earlier in the development lifecycle, and
    providing detailed logs of frequent test results (through nightly
    builds). The automated test environment will be carefully setup to
    ensure accurate and consistent testing outcomes.

-   Regression testing ensures that the changes made in the current
    software do not affect the functionality of the existing software.
    Regression testing can be performed either by hand or by an
    automated process. The regression testing will be achieved by using
    a nightly build.

-   Continuous Testing uses excess cycles on a developer's workstation
    to continuously run regression tests in the background, providing
    rapid feedback about test failures as source code is edited. It
    reduces the time and energy required to keep code well-tested, and
    prevents regression errors from persisting uncaught for long periods
    of time

-   Integration and System testing tests multiple software components
    that have each received prior and separate unit testing. Both the
    communication between components and APIs, as well as overall
    system-wide performance testing should be conducted.

-   Usability Testing to ensure that the overall user experience is
    intuitive, while all interfaces and features both appear consistent
    and function as expected. Comprehensive usability testing will be
    conducted with potential users (non-developers) with realistic
    scenarios and the results will be documented for all developers to
    review.

-   Bug-tracking and resolution will be managed by regularly posting all
    bugs and performance reports encountered in GForge, with follow-up
    resolution and pending issues clearly indicated for all developers
    and QA testing personnel to review.

### Unit Testing

During system development, each developer performs unit testing of his
or her code before it is finished. Unit testing is implemented against
the smallest non trivial testable element (units) of the software and
involves testing the internal structure, such as logic and data flow,
and the unit\'s functional and observable behaviors. The centrepiece of
the C3PR unit testing strategy will be the JUnit unit-testing framework
and will be augmented by HTTPUnit, Schemamule, and DBUnit. Examples of
unit testing for C3PR 2.0 software are as follows:

-   Design and develop the interface for a non-trivial JAVA class.

-   Write test case using JUnit testing all methods in the interface.

-   As the class is developed the test case is run to ensure the class
    is working properly

Complex test cases will be implemented with the Haste unit-testing
framework. This will allow the gluing of test cases into a "story" that
represents the overall unit being tested.

CruiseControl will run unit tests each time the repository is revised
and will archive test reports. Developers will be able to run unit tests
as needed to verify correct function of their code, the results of these
ad-hoc unit tests will not be saved.

### Integration Testing

Integration testing is a form of "white-box testing" where this testing
method makes sure that the correct outputs are produced when a set of
inputs are introduced to the software. In this case, the software
internals are visible and closely examined by the tester. The business
logic of the system, including grid services, will be tested. The set of
unit tests will be implemented with Ant, JUnit, Haste, and performance
profiling tools. The following sections describe how each of the
components will be tested.

#### Grid Services

Grid Services will be tested using the caGrid system testing
infrastructure, which provides facilities to dynamically build, deploy,
invoke, and tear down grid services. Business logic will be tested by
providing contrived inputs to the services and comparing the outputs to
known values.

####  ESB

The Enterprise Service Bus will be tested using the Haste framework.
Messages will be sent to the ESB and appropriate routing and
construction of output messages will be validated.

#### GUI

The web interface will be tested using HTTPUnit, which mimics the
functionality of a web browser. User scenarios will be constructed, data
input into web forms, and the output validated.

### User Acceptance Testing (UAT)

Acceptance Level Testing represents the final test action prior to
deploying any version of C3PR. Adopters will perform Acceptance Level
testing using one or more releases of C3PR to verify its readiness and
usability as defined in the use-case(s) and supporting documents.

Subject matter experts will test the end-user application (web
interface) and determine its acceptability from a usability standpoint.
Each use case and requirement will be translated into at least one test
case. The focus of these test cases will be on final verification of the
required business function and flow of the system, emulating real-world
usage of the system. To facilitate the UAT, we plan to engage the
subject matter experts throughout the software development lifecycle,
especially during the use case collection and prototyping sessions. We
also plan to provide access to the interim builds of the system to the
subject matter experts so that they can gain familiarity and provide
valuable feedback for increasing the usability of the system. The lead
architect (Patrick McConnell) and lead developer (Ram Chilukuri) will
closely work with subject matter experts during the UAT.

User acceptance testing will be performed by the following individuals:

-   Vijaya Chadaram: domain expert

-   Robert Morrell: Lead Elaborator

-   Steven Cheng: Analyst

-   Kimberly Hill Livengood: Domain Expert

-   Lisa Dixon: Domain Expert

-   Sharon Elcombe: lead elaborator

-   Diane Parkin: domain expert

-   Marin Shanley: domain expert

-   Patty Spears: lead elaborator of patient advocates

## Description of Functionality

See the Use Case Document and the SRS (section 2.3 contains references)

## Specific Exclusions

Not Applicable

## Dependencies & Assumptions

**Java programming language**

C3PR V2 is developed in the Java programming language. The Java 5 SDK is
being used for development. Integration tests and other tools and
utilities will be written in ruby, groovy, or other appropriate
languages that are useful in the testing environment. These languages
provide some features that are not available in Java.

**Application Server**

The C3PR V2 implementation requires a Java application server. Apache
Tomcat and the Globus container will be used for development and
testing. Currently, only Tomcat 5.1.28 is supported.

**Relational database**

The backend database targets both MySQL and Oracle relational databases.
Unit tests will be run against both target databases.

**Web browser**

User acceptance testing and integration testing will target the Internet
Explorer 6.x web browser.

## General Criteria for Success

Criteria for overall success are 100% success of all automated unit
tests and most tests are satisfactory successful of the manual tests.
Focus in phase I will be on automated testing, and focus in phase II
will be on manual user acceptance testing and performance testing.

### Readiness Criteria

Tests will be ready to be written when the following criteria have been
met:

-   Use cases are complete

-   Use cases are translated into executable tests

-   APIs are available for individual modules

Tests will be ready to be run when

-   Source code for individual modules is available and runnable

-   The tests are written

-   Dependent services are deployed

### Pass/Fail Criteria

The follow criteria will be employed for determining the success of
individual tests:

-   *Appropriate data returned*: equality comparison of results to
    locally cached data

-   *Performance*: documentation of performance in time and subjective
    determination that performance is acceptable for the complexity of
    the operation

### Completion Criteria

The criteria for completion of the testing procedures is that the system
produces the output desired by the user within expected performance
requirements. Testing is considered completed when:

-   The assigned test scripts have been executed.

-   Defects and discrepancies are documented, resolved, verified, or
    designated as future changes.

### Acceptance Criteria

For user acceptance testing, a range of bug severities will be employed
such that a severity can be assigned to the success of each test case.
For example, a tester could assign acceptable, acceptable with issues,
unacceptable. For unit, system, and integration testing, acceptance is
determined by the automated test completing successfully.

When testing is complete, the software is acceptable when the test
manager and project manager determine that existing unresolved issues
are documented and within subjective tolerance. Both user acceptance
testing and automated system/integration/unit tests will be taken into
consideration.

# Software Test environment

This section describes the software test environment at each intended
test site.

##  DUKE

*The Test Environment*: The Test Environment is a stable area for
independent system and integration testing by the Test Team. This area
consists of objects as they are completed by Developers and meet the
requirements for promotion. This environment ensures that objects are
tested with the latest stable version of other objects that may also be
under development. The test environment is initially populated with the
latest operational application and then updated with new changed objects
from the development environment.

*The Acceptance Testing Environment:* The acceptance-testing environment
provides a near-production environment for the client acceptance
testing. The release is delivered by the SCM group and managed by the
client.

### Software Items

Java 1.5.x: used to run the Java programs that make up the tests

Ant 1.6.x: used to run automated tests in batch

JUnit 3.x/.4.x: used to implemented specific stateless test cases for
automated unit testing

Haste 1.0.x: used to implement specific stateful test cases for
automated unit testing

Microsoft Word: used to document testing activities

CVS: used to version test results

CruiseControl 2.4.1: continuous build and testing framework

### Hardware and Firmware Items

Continuous build machine:

> cagrid1.duhs.duke.edu\
> Windows 2003 Server

Test deployment machine:

> TBD

### Other Materials

None

### Participating Organizations

The testing group consists of the project\'s Test Manager, and the
Tester(s). The groups listed below are responsible for the respective
types of testing:

-   **Unit Testing:** Development team members from SemanticBits will be
    responsible for conducting the unit tests.

-   **Integration Testing**: Development team members from SemanticBits
    will be responsible for conducting the integration tests.

-   **User Acceptance Testing**: The end-user representatives perform
    User Acceptance Testing, which includes Duke and Wake Forrest SMEs.

# Test Schedules

## Time Frames for Testing

The Test Manager will coordinate with the Project Manager and add the
planned testing activities to the master project schedule. Refer to the
project SDP and schedule for additional information.

Unit and Integration Testing will be performed through the lifetime of
the project. Quality Assurance testing will begin in month 3 of the
elaboration phase.

# Risks

See the C3PR V2 risk matrix.

[]{#__RefHeading___Toc152728838 .anchor}Appendix A -- Acronym List

  --------- -------------------------------------------------------------
  Acronym   Description

  CCR       Change control request

  DBA       Database Administrator

  HSTOMP    Health Services Team Organizational Management Portal

  MS        Microsoft

  PAL       Process Asses Library

  PM        Project Manager

  RM        Requirements Manager

  RTM       Requirements Traceability Matrix

  SCM       Software Configuration Management

  SDLC      Software Development Lifecycle

  SE        Software Engineering

  SEPG      Software Engineering Process Group

  SM        Software Manager

  SOP       Standard Operating Procedure

  SPI       Software Process Improvement

  SQA       Software Quality Assurance

  SW        Software

  TM        Test Manager

  VM        Version Manager
  --------- -------------------------------------------------------------

[]{#__RefHeading___Toc152728839 .anchor}Appendix B -- Test report

JUnitReport will be used to generate test reports from JUnit and Haste
driven tests. See the C3PR test report for reports on manual testing.