# Comments Submission: Docket 00D-1538

> December 21, 2001\
> Submitted by:

+-------------------------+---------------------+---------------------+
| > Paul Bleicher, M.D.   | Michael Owings\     | Kenneth O'Brien\    |
| > Ph.D.\                | Director, Quality   | Director, Quality   |
| > Chairman, Chief       | and Regulatory      | and Regulatory      |
| > Scientific Officer\   | Affairs-Services    | Affairs-Products    |
| > \                     |                     |                     |
| > Phase Forward         |                     |                     |
| > Incorporated\         |                     |                     |
| > 1440 Main Street\     |                     |                     |
| > Waltham,              |                     |                     |
| > Massachusetts 02451   |                     |                     |
+-------------------------+---------------------+---------------------+

> Phase Forward appreciates the opportunity to review and comment on
> this much needed Guidance on Validation. In general, the Guidance
> directly addresses some important issues for the industry, and
> reflects standard practices and the FDA's experience. However, there
> are some areas that may need additional consideration. Our comments
> reflect our experience, and that of many of our customers, in the
> validation of enterprise software for Internet-based clinical data
> management.

## Specific Comments

### 5.1 System Requirements Specifications

> **Comment:** The phrase "you should obtain evidence that the computer
> system [implements those needs correctly]{.underline} and that they
> are traceable to system design requirements and specifications" may be
> difficult to understand. It is unclear what "implements those needs
> correctly" specifically means.
>
> We propose that item 5.1 should read:

"..you should obtain evidence that the computer system performs specific
operations consistent with the end-use and/or product requirement
specifications, and that these requirements are traceable to the system
requirements and specifications if available".

Additionally, it should be considered that some product design
requirement specifications may be proprietary and confidential, and may
not be available for the end-user to incorporate and trace to end user
requirements and specifications.

### 5.2 Documentation of Validation Activity

**Comment:** Additional clarity is needed regarding requirements for
validation documentation. It is recommended that the
terminology/nomenclature be matched to acceptable software methodology
terminology and definitions (e.g. IEEE STD), Validation and Verification
Report, Testing Summary Report.

**Comment:** The Validation plan discussed in the Guidance in Section
5.2.1 Validation Plan, may also be defined or referred to as "
Verification Plan" or the "Verification and Validation Plan" as
referenced in IEEE standard 1012-1986 when applied to software
validation and testing. The Guidance should be written so that all are
considered acceptable. Similarly, we would apply the same standard to
Section 5.2.2 Validation Procedures, which may also be defined or
referred to as "Verification procedures" or the "Verification and
Validation procedures." Finally, the same consideration would apply to
the Validation report in Section 5.2.3, which may be defined or referred
to as " Verification Report" or the "Verification and Validation
Report".

**Comment:** Traditionally actual test results (pass/fail) would not be
included in the Validation/Verification Report - a discussion or
analysis of the results would be included and followed by a conclusion.
The testing summary report provides additional quantitative results in
detail.

### 5.4 Dynamic Testing

> **Comment:** For **Section 5.4.3 How test results should be
> expressed**, it is recommended that the Guidance include an example of
> test results expressed in "quantified terms" and recommend including
> the phrase " where applicable" vs. "whenever possible" in the
> statement regarding expression of test results.

Not all test results can, or should be expressed in quantified terms. If
a test script is written accurately with a full and detailed description
of the specific activity performed, along with the expected result, a
result expressed as Pass/Fail should suffice. In the event that a
failure is recorded, the result should provide a description of the
failure, reference the bug or defect number, and provide traceability to
resolution of the failure. Additionally, validation of complex systems
often involves tens of thousands of individual tests. Resource
requirements for such tests are very significant; requirement for
quantified description of results would likely lead to significantly
reduced numbers of tests performed in the commercial environment.
Finally, automated validation has the potential to perform exhaustive
validation testing, beyond the capability of manual testing. However,
automated testing results must, by their nature, be expressed as
pass/fail.

It is recommended that the Guidance include examples of test results
expressed in "quantified terms". We recommend replacing the phrase
whenever possible" to the phrase "where applicable" in the statement
regarding expression of test results.

### 6.1.3 Functional Testing of Software

**Comment:** System testing (i.e. platform, operating system, database,
and application) should adequately address testing of commercial
off-the-shelf (COTS) software, assuming the commercial components are
necessary to support the application. For example if formal testing
requires the application to successfully submit data to the database, it
should not be necessary to exhaustively validate Oracle, or Microsoft
NT. The validation plan should cover the functionality of the
application to store, retrieve and display data with integrity and
compliance with 21 CFR 11. It is important that testing is designed to
test functionality of the COTS directly supporting the software
application. If a particular configuration is required in supporting the
application this should be considered and included in the formal testing
of the application and system testing.

Therefore, it is recommended that certain software products that is
considered industry standard infrastructure software should not require
testing by the end-user aside from the system tests.

### 6.2.1 Internet Validation

**Comments:** This section modulates between system, validation issues,
security and technology. It is recommended that items under Internet
Validation be reorganized into separate topics, and addressed
individually for clarity. It is suggested that the section include the
following elements.

> **System Validation:** The system by definition will include the three
> following elements: Source computer, Internet data transmission, and
> the Destination Computer or server.
>
> **Source Computer:** The source computer may be configured in at least
> two different configurations. The approach to validation and the
> extent of validation necessary for the source computer varies widely
> depending on the configuration.
>
> **1) Permanently and temporarily installed proprietary software and/or
> data (including ActiveX, Java, and/or local data others):** This
> configuration is one in which the actual software for data collection
> and storage resides on the local computer used to send
> information/data to the destination computer, either permanently, or
> during the session or transaction. In this scenario, control (physical
> and logical security), change control, and full validation or the
> source computer is of utmost importance. The client computer must be
> kept in a "validated state" at all times, and under change control to
> ensure the integrity of the data and system. In this scenario, the
> validation and control of the source computer is essentially identical
> to that of the destination computer. This would include all elements
> referred to and discussed under the validation, change control
> (configuration management), and equipment installation.
>
> **2) No locally installed software or data:** This scenario is one in
> which the actual software used for data collection includes standard,
> installed software (e.g. a web browser) that has not been modified by
> additional software such as plug-ins, ActiveX components, or other
> local applications. The source computer transfers data to a
> centralized server via a standard web browser. In this scenario, the
> application for recording the data does not reside on the source
> computer. With the initiation of each session the destination computer
> or server can confirm that the source computer is utilizing a
> validated browser type and version. Items discussed under validation,
> change control, and equipment installation would not apply in this
> scenario.
>
> **Internet Data Transfer and Data Security:** Internet data transfer
> via TCP/IP is point-to-point in small packets. Since these packets are
> not stored locally, it should not be necessary to validate the path of
> transmission of these packets, any more than it should be necessary to
> validate the path of bits of data within a computer processor or bus.
>
> SSL should be specifically mentioned as an acceptable methodology for
> securing Internet data transfer. SSL ensures the privacy of message
> transmission and the integrity of data transmission as specified in
> the SSL 3.0 specification
> ([*http://home.netscape.com/eng/ssl3/draft302.txt*](http://home.netscape.com/eng/ssl3/draft302.txt)).
> The integrity of the data is guaranteed through the use of secure hash
> algorithms. Non-repudiation can be managed by a properly managed
> authentication schema, either biometric, or more commonly
> non-biometric (including username/password). It should be noted that
> most current manifestations of digital certificates are entirely
> dependent on authentication, typically username/password. Therefore,
> the extent of non-repudiation, data integrity, and data privacy is no
> better with digital certificates than it is with SSL.

Delivery acknowledgements should not require an alternative route beyond
the Internet. It should be possible to reliably determine, record, and
notify receipt of data into a database through an Internet application,
as part of the functionality of the application itself.

**Destination Computer:** The destination computer must be maintained in
a "validated state" at all times, and under change control to ensure the
integrity of the data and system. This would include all elements
referred to and discussed under the validation, change control
(configuration management), and equipment installation*.* Logical and
physical security measures must be in place to ensure the security,
quality and integrity of the data.

We hope that these comments will be helpful in considering the form and
content of the final Guidance.