**caAERS Application Development**

**Test Plan**

**Submitted By:** SemanticBits, LLC

**Date:** April 5, 2007

**Document Version:** 1.03

# Document Change history {#document-change-history .unnumbered}

+---------------+------------------+-----------------------------------+
| Version       | Date             | Description                       |
| Number        |                  |                                   |
+---------------+------------------+-----------------------------------+
| Draft - 1     | 12/18/2006       | Ram -- template draft             |
+---------------+------------------+-----------------------------------+
| 1.01          | 2/9/2007         | Updated to incorporate Reviewer   |
|               |                  | feedback.                         |
+---------------+------------------+-----------------------------------+
| 1.02          | 3/5/2007         | Updated to incorporate 508        |
|               |                  | Compliance Testing                |
+---------------+------------------+-----------------------------------+
| 1.03          | 4/5/2007         | 1.  Updated to incorporate        |
|               |                  |     feedback (added Kim L and     |
|               |                  |     Zhiqiang C. as DE's and       |
|               |                  |     Functional Testers)           |
|               |                  |                                   |
|               |                  | 2.  Added sample 508 Compliance   |
|               |                  |     Testing Scripts.              |
+---------------+------------------+-----------------------------------+
| 1.04          | 5/8/2007         | Updated to include HIPAA          |
|               |                  | references. Section 2.2.5.        |
+---------------+------------------+-----------------------------------+
|               |                  |                                   |
+---------------+------------------+-----------------------------------+
|               |                  |                                   |
+---------------+------------------+-----------------------------------+
|               |                  |                                   |
+---------------+------------------+-----------------------------------+
|               |                  |                                   |
+---------------+------------------+-----------------------------------+
|               |                  |                                   |
+---------------+------------------+-----------------------------------+
|               |                  |                                   |
+---------------+------------------+-----------------------------------+

**TABLE OF CONTENTS**

**Document Change history [ii](#document-change-history)**

1\. Introduction [1](#introduction)

1.1 Scope [1](#scope)

1.1.1 Identification [1](#identification)

1.1.2 Document Overview [1](#document-overview)

1.2 Resources [2](#resources)

1.3 Referenced documents [2](#referenced-documents)

2\. Software Test Strategy [4](#software-test-strategy)

2.1 Objectives [4](#objectives)

2.2 APPROACH [4](#approach)

2.2.1 Unit Testing [5](#unit-testing)

2.2.2 Integration Testing [6](#integration-testing)

2.2.3 User Acceptance Testing (UAT) [6](#user-acceptance-testing-uat)

2.2.4 Section 508 Compliance Testing
[7](#section-508-compliance-testing)

2.2.5 HIPAA Compliance Testing [8](#hipaa-compliance-testing)

2.3 Description of Functionality [11](#description-of-functionality)

2.4 Specific Exclusions [11](#specific-exclusions)

2.5 Dependencies & Assumptions [11](#dependencies-assumptions)

2.6 General Criteria for Success [11](#general-criteria-for-success)

2.6.1 Readiness Criteria [12](#readiness-criteria)

2.6.2 Pass/Fail Criteria [12](#passfail-criteria)

2.6.3 Completion Criteria [12](#completion-criteria)

2.6.4 Acceptance Criteria [12](#acceptance-criteria)

3\. Software Test environment [14](#software-test-environment)

3.1 SemanticBits [14](#semanticbits)

3.1.1 Software Items [14](#software-items)

3.1.2 Hardware and Firmware Items [14](#hardware-and-firmware-items)

3.1.3 Other Materials [15](#other-materials)

3.1.4 Participating Organizations [15](#participating-organizations)

4\. Test Schedules [16](#test-schedules)

4.1 Time Frames for Testing [16](#time-frames-for-testing)

5\. Risks [17](#risks)

Appendix A -- Acronym List [18](#__RefHeading___Toc166501938)

Appendix B -- Test report [19](#__RefHeading___Toc166501939)

Appendix C -- Sample Section 508 Compliance Script
[20](#__RefHeading___Toc166501940)

# Introduction

This Test Plan prescribes the scope, approach, resources, and schedule
of the testing activities. It identifies the items being tested,
features to be tested, testing tasks to be performed, personnel
responsible for each task, and risks associated with this plan.

## Scope

This document provides instruction and strategy for incorporating
Software Testing practices and procedures into the Cancer Adverse Event
Reporting System (caAERS) project. This document demonstrates the
application of testing on this software. The purpose of this project is
to develop and to deploy an adverse event reporting system that is
nationally scalable with a robust architecture to meet the needs of the
caBIG™ Community. The caAERS requirements, data model, and use cases
developed during Release 1 will be the foundation for this effort. The
Developer Team will execute Elaboration, Construction and Transition
Phase activities for this project. The project will be carried out using
the Agile Unified Process Framework, emphasizing continuous integration,
testing, and risk management. Two Adopters, Wake Forest and The Mayo
Clinic have been assigned and funded by caBIG™ and their input and
collaboration throughout this project is critical to the timely release
of functional software. The Developer Team will work closely with these
Adopters, any additional adopters identified by NCICB, and the Adverse
Events Special Interest Group to ensure the deliverables of this project
will meet the needs of the caBIG™ Community.

The scope of testing on the project is limited to the requirements
identified in the project's Software Requirements Specification (SRS).
The project has been broken up into three phases (Elaboration,
Construction, and Transition) with one month iterations in each.
Requirements for separate functional areas are determined at the start
of each iteration. The impact of this on the Test Plan is that the
specifics of the test plan and the test data will be determined as
requirements are included in the SRS.

See the caAERS 1.0 Vision and Scope Document for more detailed
information on project scope.

### Identification

caAERS 1.0

### Document Overview

This Test Plan defines the strategies necessary to accomplish the
testing activities associated with caAERS. We expect that it will be
further developed as the development project proceeds. Testing
procedural instructions are included in the Standard Operating
Procedures (SOPs).

The remaining Test Plan sections are organized as follows:

-   **Section 3: Software Test Strategy**: Describes the overall
    approach and techniques to be used during testing.

-   **Section 4. Software Test Environment**: Describes the intended
    testing environment.

-   **Section 5. Test Schedules**: Contains or references the schedules
    for conducting testing.

-   **Section 6. Risks**: Identifies the risks associated with the Test
    Plan.

-   **Appendix A. Acronym List:** Defines the acronyms used on the
    project.

## Resources

-   Ram Chilukuri: Technical Manager

-   Edmond Mulaire: Project Manager

-   Vinay Kumar: Architect

-   Joshua Phillips: Lead Developer

-   Sujith Thayillthodi: Developer

-   Kulasekaran Sethumadhavan: Developer

-   Sean Whitaker: Analyst

-   Rhett Sutphin: Co-Lead Developer

-   Cal Collins: Functional Lead

-   Krikor Krumlian: Domain Expert

-   Sharon Elcombe: Lead Elaborator

-   Bob Morrell: Lead Elaborator

-   Renee Webb: Domain Expert

-   David Patton: Domain Expert

-   Natasha Ganzenko: Domain Expert

-   Kim Livengood: Domain Expert

-   Zhiqiang Cheng: Analyst

See the caAERS 1.0 Communication Plan for more details on stakeholders
and their roles.

## Referenced documents

For additional project specific information, refer to the following
documents:

-   Vision and scope document

    [http://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Management/caAERS%20Vision%20and%20Scope.doc?root=caaersappdev&view=log](http://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Management/caAERS Vision and Scope.doc?root=caaersappdev&view=log)

-   Communication Plan

    <https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Management/Communications_Plan-caAERS.doc?root=caaersappdev&view=log>

    -   Test reports

        https://gforge.nci.nih.gov/plugins/scmcvs/cvsweb.php/caAERSv2/documentation/testing/reports/?cvsroot=caAERSv2

```{=html}
<!-- -->
```
-   Use case document

> [https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Use%20Cases/caAERS_draft_Use_Case.doc?root=caaersappdev&view=log](https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Use Cases/caAERS_draft_Use_Case.doc?root=caaersappdev&view=log)

-   SRS

> <https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/SRS/caAERS_draft_SRS.doc?root=caaersappdev&view=log>

-   Project plan

> [https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Management/caAERS%20Application%20Development%20PMP.doc?root=caaersappdev&view=log](https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Management/caAERS Application Development PMP.doc?root=caaersappdev&view=log)

-   Risk Matrix

> [https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Management/Risk%20Matrix/?root=caaersappdev](https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Management/Risk Matrix/?root=caaersappdev)

-   Risk management plan

    <https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Management/RiskMgmtPlan_caAERS.doc?root=caaersappdev&view=log>

# Software Test Strategy

## Objectives

caAERS 1.0 will result in a production system that is fully functional
with respect to the requirements listed in section 2.3. The overall
object of this test plan is to provide unit, integration, and quality
assurance testing for the whole of the caAERS delivered software. Unit
testing is done during code development to verify correct function of
source code modules, and to perform regression tests when code is
refactored. Integration tests verify that the modules function together
when combined in the production system. User acceptance testing verifies
that software requirements and business value have been achieved.

## APPROACH

The testing approach is to convert the use cases described in the caAERS
1.0 use case document into a number of automated unit and integration
tests to ensure the software conforms to the requirements. The following
proposes the approach for testing caAERS 1.0:

-   Create a clear, complete set of test cases from the use case
    documents and review it with all stakeholders.

-   Throughout the project, maintain the Requirements Traceability
    Matrix so that any stakeholders or tester has a concise record of
    what tests are run for each requirement.

-   All test cases will be command line accessible to take advantage of
    continuous integration testing thru the use of ANT for all testing
    phases.

Some of the practices that the SemanticBits team will adopt are:

-   Derive test cases/unit tests from updated functional specifications
    of all relevant use cases. Unit tests and testing scenarios will be
    constructed in parallel with core development efforts, while
    validating the specifications against the relevant use cases. The
    use of diagrammatic representations of use cases in the form of
    task-based flow-charts, state diagrams, or UML sequence diagrams may
    facilitate creation of test cases and monitoring outcomes.

-   Teaming testers with developers to provide close coordination,
    feedback, and understanding of specific modules for testing
    purposes.

-   Ongoing peer-review of design and code as a team based form of
    software inspection. Each developer will review and run code written
    by another developer on a regular basis (acting as QA inspectors in
    turns), along with joint code review to gain consensus on best
    practices and common problems.

```{=html}
<!-- -->
```
-   Automated test execution using Ant and unit testing to support rapid
    testing, capturing issues earlier in the development lifecycle, and
    providing detailed logs of frequent test results (through nightly
    builds). The automated test environment will be carefully setup to
    ensure accurate and consistent testing outcomes.

-   Regression testing ensures that the changes made in the current
    software do not affect the functionality of the existing software.
    Regression testing can be performed either by hand or by an
    automated process. The regression testing will be achieved by using
    a nightly build.

-   Continuous Testing uses excess cycles on a developer's workstation
    to continuously run regression tests in the background, providing
    rapid feedback about test failures as source code is edited. It
    reduces the time and energy required to keep code well-tested, and
    prevents regression errors from persisting uncaught for long periods
    of time

-   Integration and System testing tests multiple software components
    that have each received prior and separate unit testing. Both the
    communication between components and APIs, as well as overall
    system-wide performance testing should be conducted.

-   Usability Testing to ensure that the overall user experience is
    intuitive, while all interfaces and features both appear consistent
    and function as expected. Comprehensive usability testing will be
    conducted with potential users (non-developers) with realistic
    scenarios and the results will be documented for all developers to
    review.

-   Bug-tracking and resolution will be managed by regularly posting all
    bugs and performance reports encountered in gForge, with follow-up
    resolution and pending issues clearly indicated for all developers
    and QA testing personnel to review.

### Unit Testing

During system development, each developer performs unit testing of his
or her code before it is finished. Unit testing is implemented against
the smallest non trivial testable element (units) of the software and
involves testing the internal structure, such as logic and data flow,
and the unit\'s functional and observable behaviors. The centrepiece of
the caAERS unit testing strategy will be the JUnit unit-testing
framework and will be augmented by Schemamule, and DBUnit. The Spring
Framework's mock library and EasyMock will be used to create mock
objects to assist in separating unit tests. Examples of unit testing for
caAERS 1.0 software are as follows:

-   Design and develop the interface for a non-trivial Java class.

-   Write test case using JUnit testing all methods in the interface.

-   As the class is developed the test case is run to ensure the class
    is working properly

Complex test cases will be implemented with the Haste unit-testing
framework. This will allow the gluing of test cases into a "story" that
represents the overall unit being tested.

CruiseControl will run unit tests each time the repository is revised
and will archive test reports. Developers will be able to run unit tests
as needed to verify correct function of their code, the results of these
ad-hoc unit tests will not be saved.

### Integration Testing

Integration testing is a form of "white-box testing" where this testing
method makes sure that the correct outputs are produced when a set of
inputs are introduced to the software. In this case, the software
internals are visible and closely examined by the tester. The business
logic of the system, including grid services, will be tested. The set of
unit tests will be implemented with Ant, JUnit, Haste, and performance
profiling tools. The following sections describe how each of the
components will be tested.

#### Grid Services

Grid Services will be tested using the caGrid system testing
infrastructure, which provides facilities to dynamically build, deploy,
invoke, and tear down grid services. Business logic will be tested by
providing contrived inputs to the services and comparing the outputs to
known values.

####  ESB

The Enterprise Service Bus will be tested using the Haste framework.
Messages will be sent to the ESB and appropriate routing and
construction of output messages will be validated.

### User Acceptance Testing (UAT)

Acceptance Level Testing represents the final test action prior to
deploying any version of caAERS. Adopters will perform Acceptance Level
testing using one or more releases of caAERS to verify its readiness and
usability as defined in the use-case(s) and supporting documents.

Subject matter experts will test the end-user application (web
interface) and determine its acceptability from a usability standpoint.
Each use case and requirement will be translated into at least one test
case. The focus of these test cases will be on final verification of the
required business function and flow of the system, emulating real-world
usage of the system. To facilitate the UAT, we plan to engage the
subject matter experts throughout the software development lifecycle,
especially during the use case collection and prototyping sessions. We
also plan to provide access to the interim builds of the system to the
subject matter experts so that they can gain familiarity and provide
valuable feedback for increasing the usability of the system. The
development team will closely work with subject matter experts during
the UAT.

User acceptance testing will be performed by the following individuals:

-   Renee Webb: domain expert

-   David Patton: domain expert

-   Natasha Ganzenko: domain expert

-   Sharon Elcombe: lead elaborator

-   Bob Morrell: lead elaborator

-   Zhiqiang Cheng: analyst

-   Kim Livengood: domain expert

### Section 508 Compliance Testing

Testing will be conducted to ensure that the caAERS application is in
compliance with the following Section 508 accessibility requirements:

> \(a\) A text equivalent for every non-text element shall be provided
> (e.g., via \"alt\", \"longdesc\", or in element content).
>
> \(b\) Equivalent alternatives for any multimedia presentation shall be
> synchronized with the presentation.
>
> \(c\) Web pages shall be designed so that all information conveyed
> with color is also available without color, for example from context
> or markup.
>
> \(d\) Documents shall be organized so they are readable without
> requiring an associated style sheet.
>
> \(e\) Redundant text links shall be provided for each active region of
> a server-side image map.
>
> \(f\) Client-side image maps shall be provided instead of server-side
> image maps except where the regions cannot be defined with an
> available geometric shape.
>
> \(g\) Row and column headers shall be identified for data tables.
>
> \(h\) Markup shall be used to associate data cells and header cells
> for data tables that have two or more logical levels of row or column
> headers.
>
> \(i\) Frames shall be titled with text that facilitates frame
> identification and navigation.
>
> \(j\) Pages shall be designed to avoid causing the screen to flicker
> with a frequency greater than 2 Hz and lower than 55 Hz.
>
> \(k\) A text-only page, with equivalent information or functionality,
> shall be provided to make a web site comply with the provisions of
> this part, when compliance cannot be accomplished in any other way.
> The content of the text-only page shall be updated whenever the
> primary page changes.
>
> \(l\) When pages utilize scripting languages to display content, or to
> create interface elements, the information provided by the script
> shall be identified with functional text that can be read by assistive
> technology.
>
> \(m\) When a web page requires that an applet, plug-in or other
> application be present on the client system to interpret page content,
> the page must provide a link to a plug-in or applet that complies with
> §1194.21(a) through (l).
>
> \(n\) When electronic forms are designed to be completed on-line, the
> form shall allow people using assistive technology to access the
> information, field elements, and functionality required for completion
> and submission of the form, including all directions and cues.
>
> \(o\) A method shall be provided that permits users to skip repetitive
> navigation links.
>
> \(p\) When a timed response is required, the user shall be alerted and
> given sufficient time to indicate more time is required.

Where possible, the caAERS application will be tested using automated
508 Compliance testing software, such as the Cynthia Says test for
Section 508 compliance.

In addition, the functional test scripts that are created for UAT
testing will include, where appropriate, steps to verify and test the
accessibility features of the caAERS application.

A report will be created to show the results of the caAERS 508
compliance testing.

### HIPAA Compliance Testing

HIPAA, the Health Insurance Portability and Accountability Act, requires
healthcare organizations to take added precautions to ensure the
security of their networks and the privacy of patient data. As stated in
the SRS document, caAERS is required to conform to applicable HIPAA
specifications.

There are three broad classes of HIPAA requirements:

**HIPAA Privacy Rule**

The HIPPA Privacy Rule mandates the protection and privacy of all health
information. This rule specifically defines the authorized uses and
disclosures of \"individually-identifiable\" health information.

**HIPAA Transactions and Code Set Rule**

The HIPPA Transaction and Code Set Standard addresses the use of
predefined transaction standards and code sets for communications and
transactions in the health-care industry.

**HIPAA Security Rule**

The HIPAA Security Rule mandates the security of electronic medical
records. Unlike the Privacy Rule, which provides broader protection for
all formats that health information make take, such as print or
electronic information, the Security Rule addresses the technical
aspects of protecting electronic health information. More specifically,
the HIPPA Security standards addresses these aspects of security:

**- Administrative security** -- assignment of security responsibility
to an individual. 

**- Physical security** - required to protect electronic systems,
equipment and data.

**- Technical security** - authentication & encryption used to control
access to data.

While the caAERS application will be designed to support all the
applicable HIPAA requirements, the burden for ensuring that the use of
caAERS is in compliance resides with the adopting institution. That is,
it is the responsibility of the institution to ensure that the necessary
business, technical, and security policies and procedures are put into
effect and enforced and that the necessary physical security safeguards
are in place.

Our testing will be designed to ensure that the technical security
requirements are satisfied by the caAERS application. Test scripts will
be designed to test each of the HIPAA Security Rule items marked for
testing in the table below.

  ----------------------------------- -----------------------------------
  **Description. R=Required.          **Testing Strategy**
  A=Addressable**                     

  **Access Control** (R)-- Include    To be tested.
  mechanism to allow access only to   
  those persons or software programs  
  that are authorized.                

  **Unique User Identification**      To be tested.
  (R)-- Assign a unique name and/or   
  number for tracking user identity.  

  **Emergency Access Procedure**      No specific testing. Because caAERS
  (R)-- Establish procedures for      is a web based application
  obtaining necessary electronic      accessible by browser, an
  protected health information during Administrator should be able to
  an emergency.                       access the application and data
                                      from anywhere at anytime within
                                      SSL.

  **Automatic Logoff** (A)-- Include  To be tested.
  mechanism that terminates an        
  electronic session after a          
  predetermined time of inactivity.   

  **Encryption and Decryption** (A)-- To be tested.
  Include mechanism to encypt and     
  decrypt electronic protected health 
  information.                        

  **Audit Controls** (R)-- Include    To be tested.
  mechanism that records and examines 
  activity in information systems     
  that contain or use electronically  
  protected health information.       

  **Integrity** (R)-- Implement       To be tested.
  mechanism to protect electronic     
  protected health information from   
  improper alteration or destruction. 

  **Authenticate Electronic Protected To be tested.
  Health Information** (A)-- Include  
  mechanism to corroborate that       
  electronic protected information    
  has not been altered or destroyed   
  in an unauthorized manner.          

  **Person or Entity Authentication** To be tested.
  (R)-- Include mechanism to verify   
  that person or entity seeking       
  access to electronic protected      
  health information is the one       
  claimed.                            

  **Transmission Security** (R)--     To be tested.
  Include mechanism to guard against  
  unauthorized access to electronic   
  protected health information that   
  is being transmitted over an        
  electronic communications network.  

  **Integrity Controls** (A)--        To be tested.
  Include mechanism to ensure         
  electronically transmitted          
  electronic protected health         
  information is not improperly       
  modified without detection until    
  disposed of.                        

  **Encryption** (A)-- Include        To be tested.
  mechanism to encrypt electronic     
  protected health information        
  whenever deemed appropriate.        
  ----------------------------------- -----------------------------------

## Description of Functionality

See the following documents.

Use Case Document:

[https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Use%20Cases/caAERS_draft_Use_Case.doc?root=caaersappdev&view=log](https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/Use Cases/caAERS_draft_Use_Case.doc?root=caaersappdev&view=log)

SRS Document:

<https://gforge.nci.nih.gov/plugins/scmsvn/viewcvs.php/docs/SRS/caAERS_draft_SRS.doc?root=caaersappdev&view=log>

## Specific Exclusions

NA

## Dependencies & Assumptions

**Java programming language**

caAERS 1.0 is developed in the Java programming language. The Java 5 SDK
is being used for development. Integration tests and other tools and
utilities will be written in appropriate languages based on the testing
environment.

**Application Server**

The caAERS 1.0 implementation requires a Java application server. Apache
Tomcat and the Globus container will be used for development and
testing.

**Relational database**

The backend database targets both PostgreSQL and Oracle relational
databases. Unit tests will be run against both target databases.

**Web browser**

User acceptance testing and integration testing will target the Internet
Explorer 6.x web browser.

## General Criteria for Success

Criteria for overall success are 100% success of all automated unit
tests and most tests are satisfactory successful of the manual tests.
Focus in phase I will be on automated testing, and focus in phase II
will be on manual user acceptance testing and performance testing.

### Readiness Criteria

Tests will be ready to be written when the following criteria have been
met:

-   Use cases are complete

-   Use cases are translated into executable tests

-   APIs are available for individual modules

Tests will be ready to be run when

-   Source code for individual modules is available and runnable

-   The tests are written

-   Dependent services are deployed

### Pass/Fail Criteria

The follow criteria will be employed for determining the success of
individual tests:

-   *Appropriate data returned*: equality comparison of results to
    locally cached data

-   *Performance*: documentation of performance in time and subjective
    determination that performance is acceptable for the complexity of
    the operation

### Completion Criteria

The criteria for completion of the testing procedures is that the system
produces the output desired by the user within expected performance
requirements. Testing is considered completed when:

-   The assigned test scripts have been executed.

-   Defects and discrepancies are documented, resolved, verified, or
    designated as future changes.

### Acceptance Criteria

For user acceptance testing, a range of bug severities will be employed
such that a severity can be assigned to the success of each test case.
For example, a tester could assign acceptable, acceptable with issues,
unacceptable. For unit, system, and integration testing, acceptance is
determined by the automated test completing successfully.

When testing is complete, the software is acceptable when the test
manager and project manager determine that existing unresolved issues
are documented and within subjective tolerance. Both user acceptance
testing and automated system/integration/unit tests will be taken into
consideration.

# Software Test environment

This section describes the software test environment at each intended
test site.

##  SemanticBits

*The Test Environment*: The Test Environment is a stable area for
independent system and integration testing by the Test Team. This area
consists of objects as they are completed by Developers and meet the
requirements for promotion. This environment ensures that objects are
tested with the latest stable version of other objects that may also be
under development. The test environment is initially populated with the
latest operational application and then updated with new changed objects
from the development environment.

*The Acceptance Testing Environment:* The acceptance-testing environment
provides a near-production environment for the client acceptance
testing. The release is delivered by the SCM group and managed by the
client.

### Software Items

Java 1.5.x: used to run the Java programs that make up the tests

Ant 1.6.x: used to run automated tests in batch

JUnit 3.x/.4.x: used to implemented specific stateless test cases for
automated unit testing

Haste 1.0.x: used to implement specific stateful test cases for
automated unit testing

Microsoft Word: used to document testing activities

CVS: used to version test results

CruiseControl 2.4.1: continuous build and testing framework

### Hardware and Firmware Items

Continuous build machine:

> cagrid1.duhs.duke.edu\
> Windows 2003 Server

Test deployment machine:

> TBD

### Other Materials

None

### Participating Organizations

The testing group consists of the project\'s Test Manager, and the
Tester(s). The groups listed below are responsible for the respective
types of testing:

-   **Unit Testing:** Development team members from SemanticBits,
    Northwestern University and Akaza Research will be responsible for
    conducting the unit tests.

-   **Integration Testing**: Development team members from SemanticBits,
    Northwestern University and Akaza Research will be responsible for
    conducting the integration tests.

-   **User Acceptance Testing**: The end-user representatives perform
    User Acceptance Testing, which includes Northwestern University,
    Mayo Clinic and Wake Forest SMEs.

# Test Schedules

## Time Frames for Testing

The Test Manager will coordinate with the Project Manager and add the
planned testing activities to the master project schedule. Refer to the
project SDP and schedule for additional information.

Unit and Integration Testing will be performed through the lifetime of
the project.

# Risks

See the caAERS 1.0 risk matrix.

[]{#__RefHeading___Toc166501938 .anchor}Appendix A -- Acronym List

  --------- -------------------------------------------------------------
  Acronym   Description

  CCR       Change control request

  DBA       Database Administrator

  HSTOMP    Health Services Team Organizational Management Portal

  MS        Microsoft

  PAL       Process Asses Library

  PM        Project Manager

  RM        Requirements Manager

  RTM       Requirements Traceability Matrix

  SCM       Software Configuration Management

  SDLC      Software Development Lifecycle

  SE        Software Engineering

  SEPG      Software Engineering Process Group

  SM        Software Manager

  SOP       Standard Operating Procedure

  SPI       Software Process Improvement

  SQA       Software Quality Assurance

  SW        Software

  TM        Test Manager

  VM        Version Manager
  --------- -------------------------------------------------------------

[]{#__RefHeading___Toc166501939 .anchor}Appendix B -- Test report

JUnitReport will be used to generate test reports from JUnit and Haste
driven tests. See the caAERS test report for reports on manual testing.

[]{#__RefHeading___Toc166501940 .anchor}Appendix C -- Sample Section 508
Compliance Script

  --------------------- ------------------- --------------------------- ---------------------------
  **Source/Standard**   **Optionality**     **Pass**                    **Fail**

  **Section 508,        **Mandatory**        Every image, Java applet,  A non-text element has no
  §1194.22**                                Flash file, video file,     alt description.
                                            audio file, plug-in, etc.   
                                            has an alt description.     

  **(a)** A text                                                        
  equivalent for every                                                  
  non-text element                                                      
  shall be provided                                                     
  (e.g., via \"alt,\"                                                   
  \"longdesc,\" or in                                                   
  element content).                                                     

  *This standard maps   **Mandatory**        Complex graphics (graphs,  Complex graphics have no
  to WCAG v1.0 Ckpt                         charts, etc.) are           alternative text, or the
  1.1*                                      accompanied by detailed     alternative does not fully
                                            text descriptions.          convey the meaning of the
                                                                        graphic.

                        **Mandatory**        The alt descriptions       Alt descriptions are
                                            succinctly describe the     verbose, vague, misleading,
                                            purpose of the objects,     inaccurate, or redundant to
                                            without being too verbose   the context (e.g. the alt
                                            (for simple objects) or too text is the same as the
                                            vague (for complex          text immediately preceding
                                            objects).                   or following it in the
                                                                        document).

                                                                        

                        **Mandatory**        Alt descriptions for       Alt descriptions for
                                            images used as links are    images used as links are
                                            descriptive of the link     not descriptive of the link
                                            destination.                destination.

                                                                        

                        **Mandatory**        Decorative graphics with   Purely decorative
                                            no other function have      graphics have alt
                                            empty alt descriptions      descriptions that say
                                            (alt= \"\"), but they never "spacer, "decorative
                                            have missing alt            graphic," or other titles
                                            descriptions.               that only increase the time
                                                                        that it takes to listen to
                                                                        a page when using a screen
                                                                        reader.

  **Section 508,        ***The use of        Multimedia files have      Multimedia files do not
  §1194.22**            multimedia          synchronized captions.      have captions, or captions,
                        presentations is                                which are not synchronized.
                        [not]{.underline}                               
                        recommended. If                                 
                        used, then pass                                 
                        criteria must be                                
                        met.***                                         

  **(b)** Equivalent                                                    
  alternatives for any                                                  
  multimedia                                                            
  presentation shall be                                                 
  synchronized with the                                                 
  presentation.                                                         

  *This standard maps                                                   
  to WCAG v1.0 Ckpt                                                     
  1.4*                                                                  

  **Section 508,        **Mandatory**        If color is used to        The use of a color
  §1194.22**                                convey important            monitor is required.
                                            information, an alternative 
                                            indicator is used, such as  
                                            an asterisk (\*) or other   
                                            symbol.                     

  **(c)** Web pages                                                     
  shall be designed so                                                  
  that all information                                                  
  conveyed with color                                                   
  is also available                                                     
  without color, for                                                    
  example from context                                                  
  or markup.                                                            

  *This standard maps                                                   
  to WCAG v1.0 Ckpt                                                     
  2.1*                                                                  

  **Section 508,        **Mandatory**        Style sheets may be used   The document is confusing
  §1194.22**                                for color, indentation and  or information is missing
                                            other presentation effects, when the style sheet is
                                            but the document is still   turned off.
                                            understandable (even if     
                                            less visually appealing)    
                                            when the style sheet is     
                                            turned off.                 

  **(d)** Documents                                                     
  shall be organized so                                                 
  they are readable                                                     
  without requiring an                                                  
  associated style                                                      
  sheet.                                                                

  *This standard maps                                                   
  to WCAG v1.0 Ckpt                                                     
  6.1*                                                                  

  **Section 508,        ***The use of image  Separate text links are    The only way to access
  §1194.22**            maps, particularly  provided outside of the     the links of a server-side
                        server-side image   server-side image map to    image map is through the
                        maps is             access the same content     image map hot spots, which
                        [not]{.underline}   that the image map hot spot usually means that a mouse
                        recommended. If     accesses.                   is required and that the
                        used, then pass                                 links are unavailable to
                        criteria must be                                assistive technologies.
                        met.***                                         

  **(e)** Redundant                                                     
  text links shall be                                                   
  provided for each                                                     
  active region of a                                                    
  server-side image                                                     
  map.                                                                  

  *This standard maps                                                   
  to WCAG v1.0 Ckpt                                                     
  1.2*                                                                  

  **Section 508,        ***The use of image  Standard HTML client-side  Server-side image maps
  §1194.22**            maps, particularly  image maps are used, and    are used when a client-side
                        server-side image   appropriate alt             image map would suffice.
                        maps is             descriptions are provided   
                        [not]{.underline}   for the image as well as    
                        recommended. If     the hot spots.              
                        used, then pass                                 
                        criteria must be                                
                        met.***                                         

  **(f)** Client-side                                                   
  image maps shall be                                                   
  provided instead of                                                   
  server-side image                                                     
  maps except where the                                                 
  regions cannot be                                                     
  defined with an                                                       
  available geometric                                                   
  shape.                                                                

  *This standard maps                                                   
  to WCAG v1.0 Ckpt                                                     
  9.1*                                                                  

  **Section 508,        **Mandatory**        Data tables have the       Data tables have no
  §1194.22**                                column and row headers      header rows or columns.
                                            appropriately identified    
                                            (using the \<TH\> tag).     

  **(g)** Row and                                                       
  column headers shall                                                  
  be identified for                                                     
  data tables.                                                          

  *This standard maps   **Mandatory**        Tables used strictly for   Tables used for layout
  to WCAG v1.0 Ckpt                         layout purposes do NOT have use the header attribute
  5.1*                                      header rows or columns      when there is no true
                                            (i.e., the use of \<TH\>).  header.

                                                                        

                        **Mandatory for all  Tables used for layout     Tables used for layout
                        new applications.** purposes make sense when    purposes DO NOT make sense
                                            read in a linearized manner when read in a linearized
                                            (i.e., left to right and    manner (i.e., left to right
                                            top to bottom).             and top to bottom).

  **Section 508,        **Mandatory**        Table cells are            Columns and rows are not
  §1194.22**                                associated with the         associated with column and
                                            appropriate headers (e.g.   row headers, or they are
                                            with the ID, headers, scope associated incorrectly.
                                            and/or axis HTML            
                                            attributes).                

  **(h)** Markup shall                                                  
  be used to associate                                                  
  data cells and header                                                 
  cells for data tables                                                 
  that have two or more                                                 
  logical levels of row                                                 
  or column headers.                                                    

  *This standard maps                                                   
  to WCAG v1.0 Ckpt                                                     
  5.2*                                                                  

  **Section 508,        ***The use of        Each frame is given a      Frames have no titles, or
  §1194.22**            frames is           title that helps the user   titles that are not
                        [not]{.underline}   understand the frame's      descriptive of the frame's
                        recommended. If     purpose.                    purpose.
                        used, then pass                                 
                        criteria must be                                
                        met.***                                         

  **(i)** Frames shall                                                  
  be titled with text                                                   
  that facilitates                                                      
  frame identification                                                  
  and navigation.                                                       

  *This standard maps                        Each source file, that     Source file is present,
  to WCAG v1.0 Ckpt                         populates the frame, has a  without a title.
  12.1*                                     meaningful title through    
                                            the **\<title\>** tag.      

  **Section 508,        ***The use of        Screen does not have any   Screen has elements that
  §1194.22**            graphics that move, elements that move, flash,  move, flash, rotate, etc.
                        flash, continually  rotate, etc. (e.g.,         (e.g., animated gifs,
                        refresh, rotate, or animated gifs, scrolling    scrolling marques, etc.).
                        move text across a  marques, etc.).             
                        screen is                                       
                        [not]{.underline}                               
                        recommended. If                                 
                        used, then pass                                 
                        criteria must be                                
                        met.***                                         

  **(j)** Pages shall                                                   
  be designed to avoid                                                  
  causing the screen to                                                 
  flicker with a                                                        
  frequency greater                                                     
  than 2 Hz and lower                                                   
  than 55 Hz.                                                           

  *This standard maps                                                   
  to WCAG v1.0 Ckpt                                                     
  7.1*                                                                  

  **Section 508,        ***The use of a      A text-only version is     A text-only version is
  §1194.22**            text-only alternate created only when there is  provided only as an excuse
                        page is             no other way to make the    not to make the \"main\"
                        [not]{.underline}   content accessible, or when version fully accessible.
                        recommended. If     it offers significant       
                        used, then pass     advantages over the "main"  
                        criteria must be    version for certain         
                        met.***             disability types.           

  **(k)** A text-only                        The text-only version is   The text-only version is
  page, with equivalent                     up-to-date with the "main"  not up-to-date with the
  information or                            version.                    "main" version.
  functionality, shall                                                  
  be provided to make a                                                 
  web site comply with                                                  
  the provisions of                                                     
  these standards, when                                                 
  compliance cannot be                                                  
  accomplished in any                                                   
  other way. The                                                        
  content of the                                                        
  text-only page shall                                                  
  be updated whenever                                                   
  the primary page                                                      
  changes.                                                              

  *This standard maps                        The text-only version      The text-only version is
  to WCAG v1.0 Ckpt                         provides the functionality  an unequal, lesser version
  11.4*                                     equivalent to that of the   of the "main" version.
                                            "main" version.             

                                             An alternative is          No alternative is
                                            provided for components     provided for components
                                            (e.g., plug-ins, scripts)   that are not directly
                                            that are not directly       accessible.
                                            accessible.                 

  **Section 508,        **Mandatory**        Only device-independent    Device-dependent events
  §1194.22**                                events have been used for   have been used without
                                            important functionality. If providing a
                                            device-dependent events     keyboard-accessible
                                            have been used, then a      alternative.
                                            keyboard-accessible         
                                            alternative is provided.    

  **(l)** When pages                                                    
  utilize scripting                                                     
  languages to display                                                  
  content, or to create                                                 
  interface elements,                                                   
  the information                                                       
  provided by the                                                       
  script shall be                                                       
  identified with                                                       
  functional text that                                                  
  can be read by                                                        
  assistive technology.                                                 

                                                                        

                                             Fields associated with     Fields can be populated
                                            LOVs, also allow direct     only through the use of
                                            typing of the appropriate   LOVs.
                                            value into the field.       

                                                                        

                                                                        

                                                                        

                                             Alternate means exist, to  No alternate means exist,
                                            access                      to access
                                            functionality/information   functionality/information
                                            that has been provided      that has been provided
                                            dynamically through         dynamically through
                                            javascript at runtime       javascript at runtime.
                                            (e.g., tree menus) for      
                                            gender-specific fields to   
                                            capture disease.            

                                                                        

  **Section 508,        ***The use of an     A link is provided to a    No link is provided to a
  §1194.22**            applet, plug-in, or disability-accessible page  page where the plug-in can
                        other application   where the plug-in can be    be downloaded and/or the
                        is                  downloaded.                 download page is not
                        [not]{.underline}                               disability-accessible.
                        recommended. If                                 
                        used, then pass                                 
                        criteria must be                                
                        met.***                                         

  **(m)** When a web                         All Java applets, scripts  Plug-ins, scripts and
  page requires that an                     and plug-ins (including     other elements are used
  applet, plug-in or                        Acrobat PDF files and       indiscriminately, without
  other application be                      PowerPoint files, etc.) and alternatives for those who
  present on the client                     the content within them are cannot access them.
  system to interpret                       accessible to assistive     
  page content, the                         technologies, or else an    
  page must provide a                       alternative means of        
  link to a plug-in or                      accessing equivalent        
  applet that complies                      content is provided.        
  with §1194.21(a)                                                      
  through (l).                                                          

  **Section 508,        **Recommended**      All form controls have     Form controls have no
  §1194.22**                                text labels adjacent to     labels, or the labels are
                                            them (to indicate a visual  not adjacent to the
                                            association).               controls (to indicate a
                                                                        visual association).

  **(n)** When          **Mandatory**        Form elements have labels  There is no linking of
  electronic forms are                      associated with them in the the form element and its
  designed to be                            markup (i.e., the ID and    label is in the HTML.
  completed on-line,                        For elements).              
  the form shall allow                                                  
  people using                                                          
  assistive technology                                                  
  to access the                                                         
  information, field                                                    
  elements, and                                                         
  functionality                                                         
  required for                                                          
  completion and                                                        
  submission of the                                                     
  form, including all                                                   
  directions and cues.                                                  

                                                                        

  **Section 508,        **Mandatory**        A link is provided to      There is no way to skip
  §1194.22**                                skip over lists of          over lists of links.
                                            navigational menus or other 
                                            lengthy lists of links.     

  **(o)** A method                                                      
  shall be provided                                                     
  that permits users to                                                 
  skip repetitive                                                       
  navigation links.                                                     

  **Section 508,        **Mandatory**        The user has control over  The user is required to
  §1194.22**                                the timing of content       react quickly, within
                                            changes.                    limited time restraints.

  **(p)** When a timed                                                  
  response is required,                                                 
  the user shall be                                                     
  alerted and given                                                     
  sufficient time to                                                    
  indicate more time is                                                 
  required.                                                             
  --------------------- ------------------- --------------------------- ---------------------------