**Comments/Concerns on the Feb. 2003 Draft Guidance for Industry on Part
11, Electronic Records; Electronic Signatures-Scope and Application**

Comment from Darrell Tuomari, 28 APR 03

First of all, I would like to thank the FDA for withdrawing the previous
Part 11 guidance documents that appear to have lead to an overly broad
interpretation of Part 11. It is true that tremendous amount of
resources, effort and money have been spent by industry to try and
attain Part 11-compliance. It is also true that there have been many
presentations on Part 11 made by many different people, including many
consultants giving their \"expert opinions\" on Part 11, that has lead
diverse interpretations and made Part 11-compliance so complex that it
can not realistically be achieved. In the current guidance document on
lines #90-102, the FDA has stated that while they re-examine Part 11,
they are withdrawing the previously reviewed Part 11 guidance documents
on validation, glossary of terms, time stamps and maintenance of
electronic records, and CPG 7153.17, and the guidance on electronic
copies of electronic records. In addition on Lines 104-108, it indicates
that the FDA may revise some provisions of the Part 11 regulations.
Thus, it appears that the FDA is stepping back and making a \"fresh
start\" concerning their interpretation of the Part 11 regulations and
rather than worry about what we have tried to do, instead not try to
focus on what we need to do. I applaud and greatly appreciate this
effort by the FDA to try and help industry to refocus their efforts on
what the agency really needs to have, versus all the things we could do,
or things that would be nice to have. Similarly, I also have tried to
step back and unburden myself from previous interpretations and opinions
of the Part 11 regulations. Therefore, the following comments and
concerns on Part 11 reflect my current thoughts on what I think we
really need to be Part 11 compliant, which highlight the major hurdles
that need to be resolved to make appropriate use of electronic records
for GLP-studies so that the electronic records are equivalent to the
paper records. The goal of both industry and the FDA is to have accurate
reliable records on which to base scientific assessment and regulatory
decisions. As part of my assessment, of how we have tried to generate
reliable accurate study records, I briefly review the progression of
data from paper to electronic records.

**Paper Records and Raw Data.**

When the Good Laboratory Practices (GLP\'s) became effective in June
1979, most of the industry was using paper records. Clinical
observations were originally recorded by hand onto paper; and much of
the laboratory analytical equipment would simply printout the values for
the various assays performed. These original recordings of the
observations/data, were initialed and data by the people generating the
data and were defined in the GLP\'s as the raw data, because they were
necessary for reconstruction and evaluation of the report of the study.
One notable exception is that for histopathology data, the raw data is
not the original recording of an observation, because the pathologist,
frequently refines their diagnosis during the course of a study as new
information becomes available. Therefore, in the Preamble to the GLP\'s
final rule dated Sept. 4, 1987 the histopathology raw data was defined
as the signed and dated report from the pathologist; and stated that the
interim pathologist\'s notes do not need to be retained. Since the Part
11 regulations do not supersede the Predicate Rule, it does not matter
if the interim pathologist\'s notes are recorded on paper or
electronically, they do not have to be retained.

Early use of computers in GLP studies was limited to data manipulation,
not collection. Once the raw data for a study has been completed, the
study data could then be entered into various computer systems that
could be used to manipulate the data: for example to compile an
incidence summary or calculate group means. However, any subsequent data
entry and/or manipulation could always be checked for accuracy by
comparing the computer printouts back to the study raw data. For a GLP
study, the Quality Assurance Unit (QAU) had to verify that a study
report accurately reflected the raw data for a study.

**Hybrid Electronic Records.**

As computer systems became more widely used for initial data capture,
the standard practice was to printout a hard copy of data captured and
initial and date it. Thus, this hard copy printout became the raw data
for the records captured electronically. Any subsequent data changes had
to be tracked by initialed and dated hard copy printouts. In addition,
electronic audit trails became incorporated into the computer systems
that also tracked any data changes. Thus, in addition to the hard copy
printouts, there was an electronic audit trail of what was changed, who
made the change, when, and why the change was made.

**\"Paperless\" Electronic Records with Electronic Signatures.**

The eventual goal of achieving a paperless study was that everything is
captured and manipulated, and retained electronically. There would be no
required hard copy printouts to initial and date. All changes are
tracked by an electronic audit trails tracking: who, what, when, and
why. The Part 11 regulations do not define raw data, because these
regulations require an audit trail that tracks all of the changes made
following any time after the data was first entered. However, as
previously discussed for histopathology data, the electronic audit
trails would not be active until after the pathologist has locked the
database; because the interim notes of the pathologist do not need to be
retained. For all other types of data, the electronic audit trail is
active once the data is captured, and is required for any changes made
to the data.

A key component to the paperless electronic records, is the use of
electronic signatures. The electronic signature needs to be equivalent
to a handwritten signature to the extent that someone can not readily
deny that they did the data entry or data change. There are a variety of
biometric devices that can be used to help ensure that an electronic
signature is authentic. However, the use of electronic signatures is
beyond the scope of my current concerns. What is critical is that we are
able to generate electronic records that are equivalent to paper
records.

**Electronic Records should be equivalent to paper.**

The background information provided in Lines #55-60, state that the FDA
issued Part 11 to provide the criteria for their acceptance of
electronic records (with electronic or handwritten signatures) as
EQUIVALENT to paper records with handwritten signatures; with the intent
to permit the widest possible use of electronic technology. My concerns
are that the electronic records should be equivalent to the paper
records, but many have made interpretations of Part 11 that require the
electronic records to be superior to the paper records.

In the current guidance on Lines 256-261, states that if you have the
ability to search, sort or trend Part 11 records; that copies should
provide the same capability. How is this equivalent to a paper record?
This indicates that Part 11 requires that the electronic record need to
be \"superior\" to paper records, and have capabilities that paper
records do not. Why is this necessary, if the electronic and the paper
record are supposed to be equivalent?

Because of this requirement to maintain program functionality,
tremendous efforts have been made to maintain the ability to manipulate
(i.e. search or sort) the electronic data. Some previous interpretations
of Part 11 have indicated that one needs to maintain ALL of the
functionality of the computer program that was originally used to
capture/record the data, and also maintain all the associated metadata
for those electronic records. This has placed a tremendous burden on
industry by having to maintain old computer hardware and software, and
to maintain the training of staff so that those old computer systems can
still be operated if required by some auditor at some future date. Even
with the best of efforts, where a sponsor has maintained old hardware,
what happens if it breaks and it is impossible to get replacement parts?
Also, the rate of change in some areas can be dramatic. Recently there
have been tremendous advances/changes in analytical chemistry with the
development of LC/MS/MS analytical instruments. The software packages
are rapidly being updated as the instruments advance, and having to
archive and maintain software packages that are no longer supported by
the vendor becomes a monumental task. In addition, it raises major
concerns when one tries to migrate data from one computer system to
another, and some sort of migration failure occurs. Having to maintain
program functionality is a major obstacle to successful data migration
from one computer system to another. In addition, if a migration failure
does occur, it is still uncertain how this problem should be resolved.
Therefore, it is important to ask is it really a Part 11 requirement to
maintain all of the metadata and original computer program functionality
with the archived electronic records of a study?

**Requirement for Metadata and Computer Program Functionality.**

Some people have interpreted Part 11 to indicate that all the metadata
and program functionality have to be maintained with the electronic
records, so that the FDA reviewer could manipulate the data just the
same way as the clinical pathologist or analytical chemist did during
the course of a study. For example, based on a review of the scatter
plots produced by some hematology analyzers, and /or a review of the
peripheral blood smear, the clinical pathologist may decide that it is
necessary to correct the segmented neutrophil count because they saw
numerous band neutrophils were detected on the scatterplot. However, it
is important to note that during the their evaluation of the data, the
clinical pathologist reviews all information available, including the
scatter plots and blood smears, to ensure that the hematology data
output is as accurate as possible. It is also important to note that the
clinical pathologist may have decided to correct the segmented
neutrophil count because numerous band neutrophils were seen on the
blood smears, which were not readily evident based on a review of the
scatterplots. Therefore, even if all the metadata and computer program
functionality were maintained with the electronic records, a reviewer,
would still not be able to do the same evaluation that the clinical
pathologist did during the study. Also, one has to question what is the
value or reanalyzing the data after the study is over.

**Metadata is not needed for Post-Study Data Reanalyzes.**

After a study report is issued, there is no value to be able to reset
the hematology program parameters and redo the analysis. For example, if
one were to reset the cell size gating limits, and then reanalyze the
data and observe some differences in the differential leukocyte count.
What would these new numbers indicate? What value would the new numbers
be since they were not the data on which the study report was based or
audited? If the sponsor is going have to address any concerns raised by
this new analysis of the data, what assurances does the sponsor of a
study have that the person performing the reanalyzes of data was
properly trained in the use of computer program originally used to
capture the data?

Also, as previously stated, electronic records are supposed to be
equivalent to paper records, but the metadata and program functionality
are not maintained by paper records.

Thus, if electronic records were archived to non-electronic media such
as microfiche, the metadata and the program functionality would not be
maintained.

**Non-Electronic Archiving of Records.**

In the current guidance, Lines 275-281, state that the FDA dose not
intend to object if the electronic records are archived in a
non-electronic format such as paper or microfilm. It is obvious that if
the electronic records are archived on paper, that not all of the
metadata will be retained and absolutely no computer program
functionality is retained. Therefore, is it really a Part 11 requirement
that all the metadata and the computer program functionality be
retained? Since paper and electronic records are supposed to be
equivalent and the metadata and program functionality are not retained
in paper records, why would they have to be retained in electronic
records? What is the critical information that needs to be retained in
both paper and electronic records? Guidance Lines #279-281 seem to
emphasize that what is critical is that the content and meaning of the
archived records are preserved.

**Preservation of Record Content and Meaning.**

To maintain the content and meaning of archived records does not require
that the original program functionality or metadata data be maintained.
Laboratory instrumentation has changed dramatically over the last couple
of decades. For example, hematology analyzers used to simply provide a
printout of the hematology data such as hematocrit, hemoglobin, etc.
Current hematology analyzers can now create and store huge metadata
files that contain the electronic information that was used to generate
the hematology data that they used to simply printout. It is not
necessary that this metadata is maintained and archived to maintain the
content and meaning of the data. For example, consider some hematocrit
data. The hematology analyzer indicated that Dog #2 had a hematocrit of
45 on Study Day 10, then the critical information that has to be
maintained and archived is that Dog #2 on Study Day 10 had a hematocrit
of 45. The content and meaning of a hematocrit of 45 does not require
that all the metadata (all of the individual cell volumes that were used
to calculate the hematocrit) be maintained or archived. However, because
of Part 11, some people have now interpreted the metadata to be the raw
data.

**Metadata versus Raw Data.**

As previously stated, Part 11 does not define raw data. Based on my
interpretations of the GLP\'s the raw data would be the record that
initially showed that Dog#2 on Study Day 10 had a hematocrit of 45,
because the original observation being made is that the hematocrit is
45. This interpretation would be the same whether the analysis was done
in 1983 with a hematology analyzer that simply printed out the
hematocrit values, or if it was done in 2003 with a hematology analyzer
that did compile large metadata files, that actually contained all of
the individual cell volumes measured that it used to calculate that the
hematocrit was 45. Also, as discussed previously, if the electronic data
were archived to non-electronic media, the data saved would be that the
hematocrit was 45. Therefore, the critical question becomes, what needs
to be retained/archived so as to preserve the meaning and content of the
study data, as well as what is necessary for the reconstruction and
evaluation of the study report?

**Study Assessment is based on the Study Data, not the Metadata.**

The assessment of a toxicity/safety study and resulting interpretations
and conclusions is based on an assessment of the study data, not the
metadata. The data review requires that the data be in a human readable
form. For hematology data this would include values for each of the
hematology parameters such as the hematocrit, hemoglobin, etc. This data
review does not require access to the metadata or the hematology program
functionality. The data reviewer can perform a variety of manipulations
of the hematology data as part of their toxicological assessment, such
as sort the hematology data, calculate means, or graph the data. Note,
all of this data manipulation could be done using electronic records, or
it could be done manually based on paper records. However, when it comes
to finalization of the study report, the Quality Assurance Unit will
need to be able to audit the accuracy of these data manipulations to
ensure that they accurately reflect the raw data. The QAU does not
require access to the hematology computer functionality or metadata in
order to audit the data in the final study report. What the QAU does
need is that the study data be available in a human readable form so
that they can audit the final study report to ensure that it accurately
reflects the study raw data. Therefore, what needs to be
retained/archived are the records that preserve the meaning and content
of the study data, and which are necessary for the reconstruction and
evaluation of the study report.

**Digital Pathology Images.**

There has been a lot of debate over whether of not one can compress
digital image files because some information may be lost. However, if
even the trained eye can not tell the difference in images from a
compressed file versus an uncompressed file, does it matter if some
digital information was lost? I know that it is possible to compress a
digital file so severely that tremendous amount of information/detail is
lost. However, if the pathologist thinks that the digital image
accurately represents the lesion they are trying to illustrate, does it
matter if the image is from a compressed file or not? As previously
discussed, electronic records are supposed to be equivalent to paper,
not superior to paper. Even with film photography, severe
information/detail can be lost during the printing process. With film
photography, the pathologist chose the print that they thought was most
accurately representative of the lesion they wanted to show. Why should
this selection process by the pathologist change because they were using
a digital camera? If the pathologist thinks that a compressed digital
image record retains the content and meaning of the image that they are
trying to archive, why would this not be an appropriate electronic
record?

It is also important to note that any pathology images contain only
fragmentary information about the lesion illustrated. In gross
pathology, a photo may be taken to illustrate what the pathologist meant
by brown discoloration. Any image can only be of a single plane view and
do not represent the total assessment of lesion that the pathologist
performed by examining the total dimensions of the lesion, with perhaps
many cross sections. In addition, no image can capture all the
variations of color seen in lesion, or any of the variations in texture,
or smells that the pathologist observed. Thus, the gross pathology
images are supplemental/supporting information, and the pathologist\'s
gross written description is the gross pathology raw data. In
histopathology, most images are only a small fraction of the total
tissue evaluated by the pathologist, and also are meant as supplemental
information, taken to illustrate a particular component of a lesion. As
previously discussed, the histopathology raw data is the signed and
dated report from the pathologist. The pathologist will note in the
study records when gross or histopathologic photos are taken. However,
because the pathology photos are supplemental information, it varies
with the study type and sponsor as to whether or not any pathology
photos are actually included in the study report, but usually few
photos, if any are included. Digital photograph has potential that it
could facilitate the inclusion of more pathology images into study
reports.

**Part 11 Goal is Reliable Study Data Electronic Records.**

Both industry and the FDA have the same goal for Part 11 in that we all
want to have accurate, reliable electronic records. In addition,
resources, time and money can best be used to facilitate Part 11
compliance if they are properly focused on the critical areas. In trying
to attain Part 11 compliance, it is important that the electronic
records be equivalent to the paper records, and not require the
electronic records to be superior to the paper records. The FDA guidance
documents need to keep focus on what they really need to have in the
electronic records. We have extensively discussed maintaining metadata
and program functionality. In order to properly focus any future
efforts, the future FDA guidance documents need to clearly address
metadata and program functionality. The content and meaning of the study
data can be archived without retaining all the metadata and program
functionality; because we have done this in the past with our paper
archives.

**Electronic Record versus Electronic Signature.**

In the current regulations, the concepts of electronic records and
electronic signatures are interwoven. If the Part 11 regulations are
revised in the future, I recommend that they be made separate sections
in the regulations. First, concentrate on creating and maintaining
appropriate electronic records. Once that is done, then one can try to
address the use of electronic signatures as a separate topic, on what
would be necessary if one wanted to do a \"paperless\" study.

Also, the preceding comments on Part 11 were from a GLP-perspective.
Some of the difficulty interpreting Part 11 may be because it is being
applied equally to GLP as to GCP and GMP. However, there are different
GXP\'s, because the concerns and issues are different for each area. For
example, far more stringent change control is required to make any
changes to an analytical method for the drug active in a currently
approved product (an GMP-concern) than would be appropriate to apply to
a chemist working on metabolism study trying to identify an unknown (an
GLP-concern). Therefore, on any future revisions of the Part 11
regulations, it may be useful to separate out those requirements which
are primarily GMP-concerns or primarily GLP- or GCP-concerns.