Draft Guidance for Industry - Not For Implementation

Guidance for Industry

21 CFR Part 11; Electronic

Records; Electronic Signatures

Validation

Draft Guidance

This guidance document is being distributed for comment purposes only.

Comments and suggestions regarding this draft document should be submitted

within 90 days of publication in the Federal Register of the notice announcing the

availability of the draft guidance. Submit comments to Dockets Management

Branch (HFA-305), Food and Drug Administration, 5630 Fishers Lane, room

1061, Rockville, MD 20852. All comments should be identified with the docket

number 00D-1538.

For questions regarding this draft document contact Paul J. Motise, Office of

Enforcement, Office of Regulatory Affairs, 301-827-0383, e-mail:

pmotise@ora.fda.gov.

U.S. Department of Health and Human Services

Food and Drug Administration

Office of Regulatory Affairs (ORA)

Center for Biologics Evaluation and Research (CBER)

Center for Drug Evaluation and Research (CDER)

Center for Devices and Radiological Health (CDRH)

Center for Food Safety and Applied Nutrition (CFSAN)

Center for Veterinary Medicine (CVM)

August 2001
Draft Guidance for Industry - Not For Implementation

ii

Guidance For Industry

21 CFR Part 11; Electronic Records;

Electronic Signatures

Validation

Additional copies of this draft guidance document are available from the Office of

Enforcement, HFC-200, 5600 Fishers Lane, Rockville, MD 20857; Internet

http://www.fda.gov/ora/compliance_ref/part11.htm

U.S. Department of Health and Human Services

Food and Drug Administration

Office of Regulatory Affairs (ORA)

Center for Biologics Evaluation and Research (CBER)

Center for Drug Evaluation and Research (CDER)

Center for Devices and Radiological Health (CDRH)

Center for Food Safety and Applied Nutrition (CFSAN)

Center for Veterinary Medicine (CVM)

August 2001
Draft Guidance for Industry - Not For Implementation

iii

Guidance For Industry

21 CFR Part 11; Electronic Records; Electronic Signatures

Validation

Table of Contents

1. Purpose .........................................................................................................1

2. Scope ............................................................................................................1

2.1 Applicability ................................................................................................2

2.2 Audience ....................................................................................................3

3. Definitions and Terminology ..........................................................................3

4. Regulatory Requirements; What Does Part 11 Require? ..............................3

5. Key Principles................................................................................................4

5.1 System Requirements Specifications.........................................................4

5.2 Documentation of Validation Activity ..........................................................6

5.2.1 Validation Plan.....................................................................................6

5.2.2 Validation Procedures..........................................................................6

5.2.3 Validation Report ................................................................................6

5.3 Equipment Installation................................................................................7

5.4 Dynamic Testing.........................................................................................7

5.4.1 Key Testing Considerations.................................................................7

5.4.2 Software testing should include: ..........................................................8

5.4.3 How test results should be expressed. ................................................8

5.5 Static Verification Techniques ....................................................................9

5.6 Extent of Validation ....................................................................................9

5.7 Independence of Review..........................................................................10

5.8 Change Control (Configuration Management)..........................................10

6. Special Considerations ................................................................................11

6.1 Commercial, Off-The-Shelf Software........................................................11

6.1.1 End User Requirements Specifications..............................................12

6.1.2 Software Structural Integrity...............................................................12

6.1.3 Functional Testing of Software ..........................................................13

6.2 The Internet..............................................................................................13

6.2.1 Internet Validation..............................................................................13

Appendix A - References .................................................................................15
Draft Guidance for Industry - Not For Implementation

1

Guidance For Industry1

21 CFR Part 11; Electronic Records; Electronic Signatures

Validation

1. Purpose 

The purpose of this draft guidance is to describe the Food and Drug Administration's

(FDA’s) current thinking regarding considerations in meeting the validation requirements of

Part 11 of Title 21 of the Code of Federal Regulations; Electronic Records; Electronic

Signatures. It provides guidance to industry, and is intended to assist persons who are

subject to the rule to comply with the regulation. It may also assist FDA staff who apply

part 11 to persons who are subject to the regulation.

2. Scope

This draft guidance is one of a series of guidances about part 11. We intend to provide

information with respect to FDA's current thinking on acceptable ways of meeting part 11

 

1 This draft guidance was prepared under the aegis of the Office of Enforcement by the FDA Part 11

Compliance Committee. The committee is composed of representatives from each center within the Food

and Drug Administration, the Office of Chief Counsel and the Office of Regulatory Affairs.

This draft guidance, when finalized, will represent the Food and Drug

Administration’s (FDA’s) current thinking on this topic. It does not create or confer

any rights for or on any person and does not operate to bind FDA or the public. An

alternative approach may be used if such approach satisfies the requirements of

applicable statutes and regulations.
Draft Guidance for Industry - Not For Implementation

2

requirements to ensure that electronic records and electronic signatures are trustworthy,

reliable, and compatible with FDA's public health responsibilities.

Electronic record and electronic signature systems consist of both manual procedural

controls and technical controls implemented through computer systems. This draft

guidance focuses on validation of computer systems. It identifies key validation principles

and addresses some frequently asked questions, but it is not intended to cover everything

that computer systems validation should encompass in the context of electronic

record/electronic signature systems. You can read more information about computer

systems validation in the documents listed in Appendix A - References.

2.1 Applicability

This draft guidance applies to electronic records and electronic signatures that persons

create, modify, maintain, archive, retrieve, or transmit under any records or signature

requirement set forth in the Federal Food, Drug, and Cosmetic Act (the Act), the Public

Health Service Act (PHS Act), or any FDA regulation. Any requirements set forth in the

Act, the PHS Act, or any FDA regulation, with the exception of part 11, are referred to in

this document as predicate rules. Most predicate rules are contained in Title 21 of the

Code of Federal Regulations. In general, predicate rules address the research, production,

and control of FDA regulated articles, and fall into several broad categories. Examples of

such categories include, but are not limited to, manufacturing practices, laboratory
Draft Guidance for Industry - Not For Implementation

3

practices, clinical and pre-clinical research, adverse event reporting, product tracking, and

pre and post marketing submissions and reports.

2.2 Audience

We intend this draft guidance to provide useful information and recommendations to:

• Persons subject to part 11;

• Persons responsible for validation of systems used in electronic recordkeeping;

• Persons who develop products or services to enable implementation of part 11

requirements; and,

This draft guidance may also assist FDA staff who apply part 11 to persons subject to the

regulation.

3. Definitions and Terminology 

Unless otherwise specified below, all terms used in this draft guidance are defined in FDA’s

draft guidance document, “Guidance For Industry, 21 CFR Part 11; Electronic Records;

Electronic Signatures, Glossary of Terms,” a document common to the series of guidances

on part 11.

4. Regulatory Requirements; What Does Part 11 Require?

Section 11.10 requires persons to “employ procedures and controls designed to ensure the

authenticity, integrity, and, when appropriate, the confidentiality of electronic records, and

to ensure that the signer cannot readily repudiate the signed record as not genuine.” To
Draft Guidance for Industry - Not For Implementation

4

satisfy this requirement persons must, among other things, employ procedures and controls

that include "[v]alidation of systems to ensure accuracy, reliability, consistent intended

performance, and the ability to discern invalid or altered records."

5. Key Principles

Here are some key principles you should consider when validating electronic recordkeeping

computer systems.

5.1 System Requirements Specifications

Regardless of whether the computer system is developed in-house, developed by a

contractor, or purchased off-the-shelf, establishing documented end user (i.e., a person

regulated by FDA) requirements is extremely important for computer systems validation.

Without first establishing end user needs and intended uses, we believe it is virtually

impossible to confirm that the system can consistently meet them. Once you have

established the end user’s needs and intended uses, you should obtain evidence that the

computer system implements those needs correctly and that they are traceable to system

design requirements and specifications. It is important that your end user requirements

specifications take into account predicate rules, part 11, and other needs unique to your

system that relate to ensuring record authenticity, integrity, signer non-repudiation, and,

when appropriate, confidentiality. For example, as noted above, section 11.10 has a

general requirement that persons who use closed systems to create, modify, maintain, or

transmit electronic records must employ procedures and controls designed to ensure the
Draft Guidance for Industry - Not For Implementation

5

authenticity, integrity, and, when appropriate, the confidentiality of electronic records, and

to ensure that signers cannot readily repudiate signed records as not genuine. In addition,

section 11.30 requires persons who use open systems to employ procedures and controls

identified in section 11.10, as appropriate; persons who use open systems must also

implement special procedures and controls, such as document encryption and use of digital

signature standards, as necessary under the circumstances, to ensure record authenticity,

integrity, and confidentiality.

Other factors not specifically addressed in part 11 may also impact on electronic record

trustworthiness, integrity and system performance. You should consider these factors and

establish appropriate requirements specifications for them, as well. Here are some

examples:

• Scanning processes: where a paper record is scanned to create an electronic

record, scanner resolution, scanning rates, color fidelity, and the type of

hardware interface may impact the accuracy and reliability of the electronic

record as well as system performance.

• Scalability: in a networked environment, system performance may be affected

by the number of workstations and bandwidth demands of file size and types.

• Operating environment: sources of electromagnetic interference, radio

frequency interference, temperature/humidity, and electrical power

fluctuations may affect system performance.
Draft Guidance for Industry - Not For Implementation

6

5.2 Documentation of Validation Activity

We consider thorough documentation to be extremely important to the success of your

validation efforts. Validation documentation should include a validation plan, validation

procedures, and a validation report, and should identify who in management is responsible

for approval of the plan, the procedures and the report.

5.2.1 Validation Plan

The validation plan is a strategic document that should state what is to be done, the scope

of approach, the schedule of validation activities, and tasks to be performed. The plan

should also state who is responsible for performing each validation activity. The plan

should be reviewed and approved by designated management.

5.2.2 Validation Procedures

The validation procedures should include detailed steps for how to conduct the validation.

It should describe the computer system configuration, as well as test methods and

objective acceptance criteria, including expected outcomes. The procedures should be

reviewed and approved by designated management.

5.2.3 Validation Report

The validation report should document detailed results of the validation effort, including test

results. Whenever possible, test results should be expressed in quantified terms rather

than stated as “pass/fail.” The report should be reviewed and approved by designated

management.
Draft Guidance for Industry - Not For Implementation

7

5.3 Equipment Installation 

Prior to testing, you should confirm that all hardware and software are properly installed

and, where necessary, adjusted and calibrated to meet specifications. User manuals,

standard operating procedures, equipment lists, specification sheets, and other

documentation should be readily accessible for reference.

5.4 Dynamic Testing

5.4.1 Key Testing Considerations

• Test conditions: test conditions should include not only “normal” or “expected”

values, but also stress conditions (such as a high number of users accessing a

network at the same time). Test conditions should extend to boundary values,

unexpected data entries, error conditions, reasonableness challenges (e.g.,

empty fields, and date outliers), branches, data flow, and combinations of

inputs.

• Simulation tests: some testing may be performed using simulators, usually

conducted off-line outside of the actual user’s computing environment.

• Live, user-site tests: these tests are performed in the end user’s computing

environment under actual operating conditions. Testing should cover

continuous operations for a sufficient time to allow the system to encounter a

wide spectrum of conditions and events in an effort to detect any latent faults

that are not apparent during normal activities.
Draft Guidance for Industry - Not For Implementation

8

5.4.2 Software testing should include:

• Structural testing: this testing takes into account the internal mechanism

(structure) of a system or component. It is sometimes referred to as “white

box” testing. Structural testing should show that the software creator followed

contemporary quality standards (e.g., consensus standards from national and

international standards development organizations, such as those listed in

Appendix A of this guidance). This testing usually includes inspection (or

walk-throughs) of the program code and development documents.

• Functional testing: this testing involves running the program under known

conditions with defined inputs, and documented outcomes that can be

compared to pre-defined expectations. Functional testing is sometimes called

“black box” testing.

• Program build testing: this testing is performed on units of code (modules),

integrated units of code, and the program as a whole.

5.4.3 How test results should be expressed.

Quantifiable test results should be recorded in quantified rather than qualified (e.g.,

pass/fail) terms. Quantified results allow for subsequent review and independent

evaluation of the test results. 
Draft Guidance for Industry - Not For Implementation

9

5.5 Static Verification Techniques

While dynamic testing is an important part of validation, we believe that by using dynamic

testing alone it would be virtually impossible to fully demonstrate complete and correct

system performance. A conclusion that a system is validated is also supported by

numerous verification steps undertaken throughout the system development. These

include static analyses such as document and code inspections, walk-throughs, and

technical reviews. Where available, knowledge of these activities and their outcomes can

help to focus testing efforts, and help to reduce the amount of system level functional

testing needed at the user site in order to validate that the software meets the user’s needs

and intended uses.

5.6 Extent of Validation

When you determine the appropriate extent of system validation, the factors you should

consider include (but are not limited to) the following:

• The risk that the system poses to product safety, efficacy, and quality; note that

product means the FDA regulated article (food, human or veterinary drug,

biological product, medical device, or radiological product);

• The risk that the system poses to data integrity, authenticity, and confidentiality;

and,

• The system’s complexity; a more complex system might warrant a more

comprehensive validation effort.
Draft Guidance for Industry - Not For Implementation

10

5.7 Independence of Review

It is a quality assurance tenet that objective self-evaluation is difficult. Therefore, where

possible, and especially for higher risk applications, computer system validation should be

performed by persons other than those responsible for building the system. Two

approaches to ensuring an objective review are: (1) Engaging a third party; and, (2) dividing

the work within an organization such that people who review the system (or a portion of the

system) are not the same people who built it.

5.8 Change Control (Configuration Management)

Systems should be in place to control changes and evaluate the extent of revalidation that

the changes would necessitate. The extent of revalidation will depend upon the change’s

nature, scope, and potential impact on a validated system and established operating

conditions. Changes that cause the system to operate outside of previously validated

operating limits would be particularly significant.

Contractor or vendor upgrades or maintenance activities, especially when performed

remotely (i.e., over a network), should be carefully monitored because they can introduce

changes that might otherwise go unnoticed and have an adverse effect on a validated

system. Examples of such activities include installation of circuit boards that might hold

new versions of “firmware” software, addition of new network elements, and software

“upgrades”, “fixes” or “service packs.” It is important that system users be aware of such
Draft Guidance for Industry - Not For Implementation

11

changes to their system. You should arrange for service providers to advise you regarding

the nature of such revisions so you can assess the changes and perform appropriate

revalidation.

We consider regression analysis to be an extremely important tool that should be used to

assess portions of a system that were themselves unchanged but are nonetheless

vulnerable to performance/reliability losses that the changes can cause. For instance, new

software might alter performance of other software on a system (e.g., by putting into place

new device drivers or other code that programs share.) Regression testing should be

performed based on the results of the regression analysis.

6. Special Considerations

6.1 Commercial, Off-The-Shelf Software

Commercial software used in electronic recordkeeping systems subject to part 11 needs to

be validated, just as programs written by end users need to be validated. See 62 Federal

Register 13430 at 13444-13445 (March 20, 1997.) We do not consider commercial

marketing alone to be sufficient proof of a program’s performance suitability. The end user

is responsible for a program’s suitability as used in the regulatory environment. However,

the end user's validation approach for off-the-shelf software is somewhat different from

what the developer does because the source code and development documentation are

not usually available to the end user. End users should validate any program macros and
Draft Guidance for Industry - Not For Implementation

12

other customizations that they prepare. End users should also be able to validate off-theshelf software by performing all of the following:

6.1.1 End User Requirements Specifications

End users should document their requirements specifications relative to part 11

requirements and other factors, as discussed above. The end user’s requirements

specifications may be different from the developer’s specifications. If possible, the end

user should obtain a copy of the developer’s requirements specifications for comparison.

6.1.2 Software Structural Integrity

Where source code is not available for examination, end users should infer the adequacy of

software structural integrity by doing all of the following:

• Conducting research into the program’s use history. This research should

include: (1) Identifying known program limitations; (2) evaluating other end user

experiences; and, (3) identifying known software problems and their resolution;

and

• Evaluating the supplier’s software development activities to determine its

conformance to contemporary standards. The evaluation should preferably be

derived from a reliable audit of the software developer, performed by the end

user’s organization or a trusted and competent third party.
Draft Guidance for Industry - Not For Implementation

13

6.1.3 Functional Testing of Software

End users should conduct functional testing of software that covers all functions of the

program that the end user will use. Testing considerations discussed above should be

applied. When the end user cannot directly review the program source code or

development documentation (e.g., for most commercial off-the-shelf software, and for some

contracted software,) more extensive functional testing might be warranted than when such

documentation is available to the user. More extensive functional testing might also be

warranted where general experience with a program is limited, or the software performance

is highly significant to data/record integrity and authenticity. Note, however, we do not

believe that functional testing alone is sufficient to establish software adequacy.

6.2 The Internet

We recognize the expanding role of the Internet in electronic recordkeeping in the context

of part 11. Vital records, such as clinical data reports or batch release approvals, can be

transmitted from source to destination computing systems by way of the Internet.

6.2.1 Internet Validation

We recognize that the Internet, as computer system, cannot be validated because its

configuration is dynamic. For example, when a record is transmitted from source to

destination computers, various portions (or packets) of the record may travel along different
Draft Guidance for Industry - Not For Implementation

14

paths, a route that neither sender nor recipient can define or know ahead of time. In

addition, entirely different paths might be used for subsequent transfers.

The Internet can nonetheless be a trustworthy and reliable communications pipeline for

electronic records when there are measures in place to ensure the accurate, complete and

timely transfer of data and records from source to destination computing systems.

Validation of both the source and destination computing systems (i.e., both ends of the

Internet communications pipeline) should extend to those measures. We therefore

consider it extremely important that those measures are fully documented as part of the

system requirements specifications, so they can be validated. Examples of such measures

include:

• Use of digital signature technology to verify that electronic records have not

been altered and that the sender’s authenticity is affirmed.

• Delivery acknowledgements such as receipts or separate confirmations

executed apart from the Internet (e.g., via fax or voice telephone lines.)
Draft Guidance for Industry - Not For Implementation

15

Appendix A - References

Much has been written about activities that support computer systems validation. You may

find the following references useful to your validation efforts.

Food and Drug Administration References

Electronic Records; Electronic Signatures Final Rule, 62 Federal Register 13430 (March

20, 1997). 

Glossary of Computerized System and Software Development Terminology, Division of

Field Investigations, Office of Regional Operations, Office of Regulatory Affairs, Food and

Drug Administration, August 1995.

Guidance for Industry: Computerized Systems Used in Clinical Trials, Food and Drug

Administration, April 1999.

Guidance for Industry and for FDA Staff: General Principles of Software Validation, Center

for Devices and Radiological Health, Food and Drug Administration, Draft – June 1997.

Guidance for the Content of Pre-market Submissions for Software Contained in Medical

Devices, Office of Device Evaluation, Center for Devices and Radiological Health, Food

and Drug Administration, May 1998.

Guidance for Industry, FDA Reviewers and Compliance on Off-the-Shelf Software Use in

Medical Devices, Office of Device Evaluation, Center for Devices and Radiological Health,

Food and Drug Administration, September 1999. 

Guideline on General Principles of Process Validation, Center for Drugs and Biologics, &

Center For Devices and Radiological Health, Food and Drug Administration, May 1987.

Reviewer Guidance for a Pre-Market Notification Submission for Blood Establishment

Computer Software, Center for Biologics Evaluation and Research, Food and Drug

Administration, January 1997

Student Manual 1, Course INV545, Computer System Validation, Division of Human

Resource Development, Office of Regulatory Affairs, Food and Drug Administration, 1997. 

Technical Report, Software Development Activities, Division of Field Investigations, Office

of Regional Operations, Office of Regulatory Affairs, Food and Drug Administration, July

1987.
Draft Guidance for Industry - Not For Implementation

16

Other Government References

W. Richards Adrion, Martha A. Branstad, John C. Cherniavsky. NBS Special Publication

500-75, Validation, Verification, and Testing of Computer Software, Center for

Programming Science and Technology, Institute for Computer Sciences and Technology,

National Bureau of Standards, U.S. Department of Commerce, February 1981.

Martha A. Branstad, John C Cherniavsky, W. Richards Adrion, NBS Special Publication

500-56, Validation, Verification, and Testing for the Individual Programmer, Center for

Programming Science and Technology, Institute for Computer Sciences and Technology,

National Bureau of Standards, U.S. Department of Commerce, February 1980.

J.L. Bryant, N.P. Wilburn, Handbook of Software Quality Assurance Techniques

Applicable to the Nuclear Industry, NUREG/CR-4640, U.S. Nuclear Regulatory

Commission, 1987.

H. Hecht, et.al., Verification and Validation Guidelines for High Integrity Systems.

NUREG/CR-6293. Prepared for U.S. Nuclear Regulatory Commission, 1995. 

Patricia B. Powell, Editor. NBS Special Publication 500-98, Planning for Software

Validation, Verification, and Testing, Center for Programming Science and Technology,

Institute for Computer Sciences and Technology, National Bureau of Standards, U.S.

Department of Commerce, November 1982.

Patricia B. Powell, Editor. NBS Special Publication 500-93, Software Validation,

Verification, and Testing Technique and Tool Reference Guide, Center for Programming

Science and Technology, Institute for Computer Sciences and Technology, National

Bureau of Standards, U.S. Department of Commerce, September 1982.

Delores R. Wallace, Roger U. Fujii, NIST Special Publication 500-165, Software

Verification and Validation: Its Role in Computer Assurance and Its Relationship with

Software Project Management Standards, National Computer Systems Laboratory,

National Institute of Standards and Technology, U.S. Department of Commerce,

September 1995.

Delores R. Wallace, et.al. NIST Special Publication 500-234, Reference Information for

the Software Verification and Validation Process. Computer Systems Laboratory, National

Institute of Standards and Technology, U.S. Department of Commerce, March 1996.

Delores R. Wallace, Editor. NIST Special Publication 500-235, Structured Testing: A

Testing Methodology Using the Cyclomatic Complexity Metric. Computer Systems

Laboratory, National Institute of Standards and Technology, U.S. Department of

Commerce, August 1996.
Draft Guidance for Industry - Not For Implementation

17

International and National Consensus Standards

ANSI / ANS-10.4-1987, Guidelines for the Verification and Validation of Scientific and

Engineering Computer Programs for the Nuclear Industry, American National Standards

Institute, 1987.

IEEE Std 1012-1986, Software Verification and Validation Plans, Institute for Electrical and

Electronics Engineers, 1986.

IEEE Standards Collection, Software Engineering, Institute of Electrical and Electronics

Engineers, Inc., 1994. ISBN 1-55937-442-X.

ISO 9000-3:1997, Quality management and quality assurance standards - Part 3:

Guidelines for the application of ISO 9001:1994 to the development, supply, installation and

maintenance of computer software. International Organization for Standardization, 1997.

ISO/IEC 12119:1994, Information technology – Software packages – Quality requirements

and testing, Joint Technical Committee ISO/IEC JTC 1, International Organization for

Standardization and International Electrotechnical Commission, 1994.

ISO/IEC 12207:1995, Information technology – Software life cycle processes, Joint

Technical Committee ISO/IEC JTC 1, Subcommittee SC 7, International Organization for

Standardization and International Electrotechnical Commission, 1995.

ISO/IEC 14598:1999, Information technology – Software product evaluation, Joint

Technical Committee ISO/IEC JTC 1, Subcommittee SC 7, International Organization for

Standardization and International Electrotechnical Commission, 1999.

Software Considerations in Airborne Systems and Equipment Certification. Special

Committee 167 of RTCA. RTCA Inc., Washington, D.C. Tel: 202-833-9339. Document

No. RTCA/DO-178B, December 1992.

Production Process Software References

The Application of the Principles of GLP to Computerized Systems, Environmental

Monograph #116, Organization for Economic Cooperation and Development (OECD),

1995.

George J. Grigonis, Jr., Edward J. Subak, Jr., and Michael Wyrick, “Validation Key

Practices for Computer Systems Used in Regulated Operations,” Pharmaceutical

Technology, June 1997.

Guide to Inspection of Computerized Systems in Drug Processing, Reference Materials

and Training Aids for Investigators, Division of Drug Quality Compliance, Associate Director

for Compliance, Office of Drugs, National Center for Drugs and Biologics, & Division of
Draft Guidance for Industry - Not For Implementation

18

Field Investigations, Associate Director for Field Support, Executive Director of Regional

Operations, Food and Drug Administration, February 1983.

Daniel P. Olivier, “Validating Process Software”, FDA Investigator Course: Medical

Device Process Validation, Food and Drug Administration.

GAMP Guide For Validation of Automated Systems in Pharmaceutical Manufacture,Version

V3.0, Good Automated Manufacturing Practice (GAMP) Forum, March 1998:

Volume 1, Part 1: User Guide 

 Part 2: Supplier Guide

Volume 2: Best Practice for User and Suppliers. 

Technical Report No. 18, Validation of Computer-Related Systems. PDA Committee on

Validation of Computer-Related Systems. PDA Journal of Pharmaceutical Science and

Technology, Volume 49, Number 1, January-February 1995 Supplement.

Validation Compliance Annual 1995, International Validation Forum, Inc.

General Software Quality References

Boris Beizer, Black Box Testing, Techniques for Functional Testing of Software and

Systems, John Wiley & Sons, 1995. ISBN 0-471-12094-4.

Boris Beizer, Software System Testing and Quality Assurance, International Thomson

Computer Press, 1996. ISBN 1-85032-821-8.

Boris Beizer, Software Testing Techniques, Second Edition, Van Nostrand Reinhold, 1990.

ISBN 0-442-20672-0.

Richard Bender, Writing Testable Requirements, Version 1.0, Bender & Associates, Inc.,

Larkspur, CA 94777, 1996.

Silvana Castano, et.al., Database Security, ACM Press, Addison-Wesley Publishing

Company, 1995. ISBN 0-201-59375-0.

Computerized Data Systems for Nonclinical Safety Assessment, Current Concepts and

Quality Assurance, Drug Information Association, Maple Glen, PA, September 1988.

M. S. Deutsch, Software Verification and Validation, Realistic Project Approaches, Prentice

Hall, 1982.

Robert H. Dunn and Richard S. Ullman, TQM for Computer Software, Second Edition,

McGraw-Hill, Inc., 1994. ISBN 0-07-018314-7.
Draft Guidance for Industry - Not For Implementation

19

Elfriede Dustin, Jeff Rashka, and John Paul, Automated Software Testing – Introduction,

Management and Performance, Addison Wesley Longman, Inc., 1999. ISBN 0-201-43287-

0.

Robert G. Ebenau, and Susan H. Strauss, Software Inspection Process, McGraw-Hill,

1994. ISBN 0-07-062166-7.

Richard E. Fairley, Software Engineering Concepts, McGraw-Hill Publishing Company,

1985. ISBN 0-07-019902-7.

Michael A. Friedman and Jeffrey M. Voas, Software Assessment - Reliability, Safety,

Testability, Wiley-Interscience, John Wiley & Sons Inc., 1995. ISBN 0-471-01009-X.

Tom Gilb, Dorothy Graham, Software Inspection, Addison-Wesley Publishing Company,

1993. ISBN 0-201-63181-4.

Robert B. Grady, Practical Software Metrics for Project Management and Process

Improvement, PTR Prentice-Hall Inc., 1992. ISBN 0-13-720384-5.

Janis V. Halvorsen, A Software Requirements Specification Document Model for the

Medical Device Industry, Proceedings IEEE SOUTHEASTCON '93, Banking on

Technology, April 4th -7th, 1993, Charlotte, North Carolina.

Bill Hetzel, The Complete Guide to Software Testing, Second Edition, A Wiley-QED

Publication, John Wiley & Sons, Inc., 1988. ISBN 0-471-56567-9.

Watts S. Humphrey, A Discipline for Software Engineering. Addison-Wesley Longman,

1995. ISBN 0-201-54610-8.

Watts S. Humphrey, Managing the Software Process, Addison-Wesley Publishing

Company, 1989. ISBN 0-201-18095-2. 

Capers Jones, Software Quality, Analysis and Guidelines for Success, International

Thomson Computer Press, 1997. ISBN 1-85032-867-6.

Stephen H. Kan, Metrics and Models in Software Quality Engineering, Addison-Wesley

Publishing Company, 1995. ISBN 0-201-63339-6.

Cem Kaner, Jack Falk, Hung Quoc Nguyen, Testing Computer Software, Second Edition,

Vsn Nostrand Reinhold, 1993. ISBN 0-442-01361-2.

Craig Kaplan, Ralph Clark, Victor Tang, Secrets of Software Quality, 40 Innovations from

IBM, McGraw-Hill, 1995. ISBN 0-07-911795-3.

Edward Kit, Software Testing in the Real World, Addison-Wesley Longman, 1995. ISBN 0-
Draft Guidance for Industry - Not For Implementation

20

201-87756-2.

Alan Kusinitz, “Software Validation”, Current Issues in Medical Device Quality Systems,

Association for the Advancement of Medical Instrumentation, 1997. ISBN 1-57020-075-0.

Michael R. Lyu, Editor, Handbook of Software Reliability Engineering, IEEE Computer

Society Press, McGraw-Hill, 1996. ISBN 0-07-039400-8.

Steven R. Mallory, Software Development and Quality Assurance for the Healthcare

Manufacturing Industries, Interpharm Press,Inc., 1994. ISBN 0-935184-58-9.

Brian Marick, The Craft of Software Testing, Prentice Hall PTR, 1995. ISBN 0-13-177411-

5.

Glenford J. Myers, The Art of Software Testing, John Wiley & Sons, 1979. 

ISBN 0-471-04328-1.

Daniel Olivier, Conducting Software Audits, Auditing Software for Conformance to FDA

Requirements, Computer Application Specialists, San Diego, CA, 1994.

William Perry, Effective Methods for Software Testing, John Wiley & Sons, Inc. 1995.

ISBN 0-471-06097-6.

William E. Perry, Randall W. Rice, Surviving the Top Ten Challenges of Software Testing,

Dorset House Publishing, 1997. ISBN 0-932633-38-2.

Roger S. Pressman, Software Engineering, A Practitioner's Approach, Third Edition,

McGraw-Hill Inc., 1992. ISBN 0-07-050814-3.

Roger S. Pressman, A Manager’s Guide to Software Engineering, McGraw-Hill Inc., 1993

ISBN 0-07-050820-8.

A. P. Sage, J. D. Palmer, Software Systems Engineering, John Wiley & Sons, 1990.

Joc Sanders, Eugene Curran, Software Quality, Addison-Wesley Publishing Co., 1994.

ISBN 0-201-63198-9.

Ken Shumate, Marilyn Keller, Software Specification and Design, A Disciplined Approach

for Real-Time Systems, John Wiley & Sons, 1992. ISBN 0-471-53296-7.

Dennis D. Smith, Designing Maintainable Software, Springer-Verlag, 1999. 

ISBN 0-387-98783-5.

Ian Sommerville, Software Engineering, Third Edition, Addison Wesley Publishing Co.,

1989. ISBN 0-201-17568-1.
Draft Guidance for Industry - Not For Implementation

21

Karl E. Wiegers, Creating a Software Engineering Culture, Dorset House Publishing, 1996.

ISBN 0-932633-33-1.

Karl E. Wiegers, Software Inspection, Improving Quality with Software Inspections,

Software Development, April 1995, pages 55-64.

Karl E. Wiegers, Software Requirements, Microsoft Press, 1999. ISBN 0-7356-0631-5.

DocID ValidationDraft_PostRES.doc

08/29/01