Document ID: EPA-HQ-OEI-2003-0001-0260
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2005-09-20T04:00Z

1
RESOLVE
EPA­
Record
Keeping
Rule
1/
31/
02,
Irving,
TX
Tape
2
[
Side
A]

Joe
Retzer
(
continued
from
Side
B,
tape
one):
 
we
would
handle
that
in
my
world
is
such
and
such.

Pat
McFadden:
I'm
Pat
McFadden
with
Gustafson.
In
a
paper
world,
this
is
easy
because,
I
have
my
paper.
I
write
down
whatever
it
is
I'm
changing.
The
SOPs
are
in
place
to
tell
me
how
to
deal
with
paper,
and
it's
me
and
my
pen.
When
you
jump
into
the
electronic
world,
everything
changes.
So
you're
asking
us
to
generate
records
in
a
form
that
can't
be
altered,
and
you're
asking
us
to
maintain
these
records.

Well,
the
problem
is,
a
lot
of
our
equipment
has
software
on
it
that
is
pre­
CROMERRR
compliant.
Now
I
know
that
if
you
buy
new
software,
there
is
stuff
that
is
compliant
with
[
the
Food
and
Drug
Administration]
Part
11,
however
to
upgrade
everything
would
be
extremely
expensive.
So
our
old
software
has
not
computer­
generated
audit
trails,
and
there
is
no
way,

without
spending
money,
that
we
can
put
that
into
place.
So
that
becomes
a
financial
burden,
to
upgrade
all
the
software
2
into
this
[
Good
Laboratory
Practices]­
compliant
software.
So
we
don't
have
these
computer­
generated
audit
trails.

Then
when
we
archive
things,
we
archive
our
paper.
It
has
no
meta
data,
it
has
no
audit
trail,
and
so
that,
number
nine
we're
also
not
in
compliance
with,
until
we
change
our
software.
So
when
CROMERRR
goes
into
effect,
basically
you're
saying
to
us,
all
our
software
now
has
to
be
changed.
We
have
to
go
out
and
purchase
all
new
software
for
all
this
analytical
instrumentation
that
we
have
so
that
we
will
be
in
compliance.

Is
that
my
understanding?
Is
that
correct?

Joe
Retzer:
That
would
be
correct,
if
CROMERRR
were
published
or
promulgated
today
as
it
were
proposed.

Unidentified
Male
Speaker:
In
looking
at
these,
if
all
these
criteria
do
exist
for
paper,
I'm
not
sure
even
all
the
companies
are
complying
with
all
the
criteria
for
paper
even
today.

Joe
Retzer:
I
don't
think
they
are,
formally.
They're
not
formally
regulatory
criteria
for
paper.
I
was
suggesting
that
it
might
be
good
business
practice,
is
generally
what
I
was
saying.

Unidentified
Male
Speaker:
But
are
they
generally
going
to
be
requirements
for
CROMERRR?
3
Joe
Retzer:
Well
that's
the
question
before
us
today,
is
whether
they
should
be
and
are
there
some
that
are
more
difficult
than
others?

Unidentified
Male
Speaker:
I
mean
my
experience,
and
I
used
to
run
our
emission
fee
billing
program
for
our
agency,

and
when
we
needed
information
from
facilities.
We'd
have
to
go
several
levels
deep
into
departments
to
try
and
get
that
information.
It
wasn't
corporate­
wide.
Only
a
few
people
knew
where
it
was,
and
it
didn't
seem
like
it
would
comply
with
any
of
these.

Even
in
our
own
agency,
the
refinery
people
have
to
submit
reports
to
us
telling
us
how
much
crude
they
outload,
1172
reports,
and
those
go
to
the
refinery
unit
and
nowhere
else,

and
they
keep
those,
and
if
people
need
them,
they've
got
to
know
where
that
information
resides
in
our
agency.
It's
not
corporate­
wide
in
a
scanned
image
system
and
things
of
that
sort,
even
though
we
do
have
an
imaging
system
for
a
lot
of
our
documents.
So
there
are
a
lot
of
documents
that
are
spread
out.
There
are
some
that
are
available
corporate­
wide,
and
it
runs
the
whole
gamut.

Nancy
Freshour:
Nancy
Freshour,
Dow
Chemical
again.
To
answer
on
question
three,
if,
for
example,
in
our
[
Good
Laboratory
Practices]
environment,
if
we
produce
an
accurate
4
and
complete
copy
of
an
e­
record
on
paper,
and
we
maintain
that
currently,
and
do
everything,
sign,
date,
this
is
now
adding
an
extra
requirement,
it's
requiring
us
now
to
have
an
e­
form
retained,
so
this
is
an
added
requirement.
So
just
addressing
that
aspect
of
the
thing.

Joe
Retzer:
I'm
sorry,
so
now
once
you've
put
it
on
paper
and
signed
it,
then
you
say
I
don't
need
this
e­
record
anymore?

Nancy
Freshour:
Well,
if
you've
established
that
it's
an
accurate
presentation
of
that
electronic
[
unintelligible],
the
time/
date
stamp,
the
person
who
generated
it,
there
are
multiple
levels
of
security
to
get
into
the
system,
the
analyst,
for
example,
signs
and
dates
it
and
everything
and
we
have
it
all
as
an
original
signed
record
that
way,
and
you
are
allowed,
currently
under
[
Good
Laboratory
Practices],
to
state
that
the
raw
data
is
the
paper
copy,
now
we
have
to
keep
an
erecord
of
it,
too.

Joe
Retzer:
Right,
so
you're
saying
that
under
current
practices,
right
now
you
get
rid
of
the
e­
record,
or
that
you
could.

Nancy
Freshour:
Theoretically
yes,
you
could.
But
now
this
requires
us
to
keep
it
and
keep
it
for
all
those
other
criteria
­­
all
these
criteria.
5
Now
going
back
to
your
example
of
oh,
we
have
a
forensic
signature
now,
what
if
somebody,
your
enemy
wants
to
submit
something.
What
is
the
risk
of
that?
I
mean
you're
talking
about
general
business
practices.
Well,
anything
that
we
do,

we
look
at
the
risks
and
the
benefits
of
doing
it,
and
also
the
regulations
that
are
required,
of
course,
but
in
this
rule
it's
one­
size­
fits­
all.
There's
no
risk
evaluation
that's
been
done
here
about
each
of
these
types
of
records,
and
there's
an
immense
amount
of
records
under
the
definition
of
e­
record.

And
so
there's
been
no
risk
evaluation.
So
maybe
the
signatures
require
more
but
there's
a
lot
of
personal
information
that's
been
required
for
the
signature,
but
there's
been
no
risk
evaluation.

Terry
Salem:
Terry
Salem
from
the
TNRCC.
I'd
like
to
revisit
a
topic
that
I've
touched
on
in
previous
meetings
in
the
past,
which
is
the
fallacy
of
the
perfect
paper
system.

And
I
guess
in
regard
to
these
criteria,
even
paper
degrades.

Criteria
number
three
cannot
even
be
met
by
paper
forever.

Paper
degrades,
it
becomes
human
unreadable.

With
respect
to
"
cannot
be
altered
without
detection"
in
item
one,
also,
it
may
not
be
 
there
are
crude
examples
where
you
certain
could
detect.
There
are
also
examples
where
it
would
be
virtually
impossible
to
detect.
6
I
also
have
some
concern
about
the
level
to
which
we
will
contort
ourselves
to
assure
that
records
have
not
been
altered,

and
that
goes
back
to
the
whole
question
of
the
risk
analysis
that
needs
to
be
done
with
respect
to
what
we
are
attempting
to
gain
here.
And
we've
chatted
about
this
a
lot
in
previous
meetings,
and
I
think
that
is
really
the
initial
analysis
that
needs
to
be
done
by
EPA,
to
determine
where
you
want
to
gain
benefit
in
areas
where
we
don't
have
to
meet
the
burden
beyond
a
reasonable
doubt
standard.
I
think
that's
where
we
need
to
start.

Trying
to
do
this
in
one
fell
swoop
and
address
all
of
the
arenas
in
40
CFR
where
reporting
is
done
by
paper
and
we
could
have
gains.
If
we
started
in
an
area
where
we
don't
have
to
reach
that
high
level
of
evidentiary
burden,
we
could
start
to
create
systems
that
we
all
felt
comfortable
with
and
that
could
meet
some
semblance
of
these
criteria,
perhaps
not
these
exact
criteria,
and
we
would
all
gain
valuable
working
knowledge
and
experience
and
comfort
level,
and
perhaps
start
to
set
the
court
record
on
evidentiary
burdens,
and
start
to
see
where
courts
really
have
trouble
with
some
of
these
audit
trail
issues,
the
non­
repudiation
issues,
and
those
kinds
of
things,

and
sort
of
start
down
this
path
in
a
way
that
is
risk­
analysis
7
based
instead
of
trying
to
bite
off
this
whole,
big,
huge
piece
of
the
apple
at
once.

Richard
Wiard:
Richard
Wiard,
[
Process
Data
Control
Corp.]
again.
I
guess
I'll
focus
on
this
paragraph
one
and
go
right
to
that
bottom
section
that
says,
"
cannot
be
altered
without
detection."
Now
if
we're
talking
about
electronics,

it's
difficult
to
conceive
of
something
that,
with
the
right
amount
of
technical
experience,
you
cannot
move
the
bits
and
the
bytes
and
make
a
change
somewhere,
at
least
limited.

It
would
appear
to
me
more,
I
guess
in
harmony
with
what
Terry
was
saying
a
second
ago,
we
need
to
say
"
cannot
be
altered
except
with
explicit
approval
by
administrator"
or
something
of
that
nature
that
begins
to
understand
the
reality
of
how
electronic
data
is
really
worked
with
and
who
it's
accessed
by.

For
example,
a
computer
software
program
that
would
protect
and
a
database
that
would
protect
alteration
by
anybody
but
a
system
administrator
or
an
owner
or
somebody,
would
appear
to
me,
to
meet
the
spirit
of
what
number
one
might
be
aiming
toward
and
be
achievable.
The
words
"
cannot
be
altered,"
I
cannot
conceive
how
that
can
be
achieved,
in
a
way.

I
think
that's
been
our
experience
in
nearly
every
electronic
media,
we
used
to
think
a
photograph
as
being
evidence
in
legal
8
matters
where
now
we
know
they
can
be
doctored
where
you
can
almost
not
detect
that
a
photograph
has
been
changed.

Well,
that's
bound
to
have
affected
the
legal
process
tremendously
in
regard
to
photographic
evidence.
I
think
this
is
going
to
have
to
address
those
same
realities
and
get
to
a
point
where
we
say
what
is
a
reasonable
protection
of
that
record
against
accidental
or,
I
guess
you'd
say,
someone
else
trying
to
change
it,
or
do
you
make
duplicate
copies
that
can
be
compared
some
way,
but
I
think
we're
going
to
have
to
come
up
with
some
way
to
interpret
that
"
cannot."
And
I
would
just
suggest
rewording
that
piece
of
it
to
be
more
reasonable.

Robin
Roberts:
What
of
this
notion
of
the
standard
being
too
high?

Greg
Nutt:
I
think
we
have
a
problem
with
scope
here
and
we
also
have
a
problem
with
general
approach.
You
don't
try
and
take
good
practices
for
paper,
as
Joe
described,
and
blindly
try
to
apply
those
to
electronic
systems,
because
electronic
systems
have
different
limitations.

You
start
instead
with
what
are
good
practices
for
maintaining
data
integrity
in
electronic
systems?
And
those
practices
are
well
established
in
the
financial
world
and
can
be
applied
to
electronic
records,
however
we
choose
to
define
that,
for
environmental
records.
Standard
approaches
with
9
respect
to
separation
of
authority,
with
respect
to
audit
trails
and
how
those
audit
trails
are
managed,
and
I
think
that's
where
we
ought
to
start.

And
that
is
addressed
in
CROMERRR
to
a
certain
extent,
but
then
these
more
extreme
requirements
are
put
on
and
I
want
to
note
that
you
left
out
something
on
three,
if
you
look
at
the
proposal,
three
also
requires
that
those
records
be
available
for
off­
site
review
in
electronic
form,
which
is
something
that
gives
me
quite
a
bit
of
grief
as
a
web
architect
and
trying
to
provide
that
information
to
anybody
off
site.

And
I
also
have
some
real
concerns
about
the
enforceability
of
this
rule.
TNRCC
does
not
have
a
legion
of
systems
analysts
to
go
down
to
Houston
and
inspect
these
systems
to
ensure
compliance.
And
to
my
knowledge,
EPA
does
not
either,
so
I
wonder
why
we're
even
making
rules
that
we
can
never
enforce.

Robin
Roberts:
I'm
interested
in
the
extent
to
which
anyone
here
thinks
that
these
might
reflect
good
business
practices,
that
these
are
reasonable.
I
mean
you
seem
to
be
saying
that
these
nine
criteria
are
stringent
even
by
current
good
business
practices,
is
that
so?
Is
there
anyone
who
can
walk
through
all
nine
and
say
how
they
do
or
don't
apply
now?
10
How
do
these
diverge,
at
what
point
in
the
paper
world,
even,

would
you
cover
those?

Mel
Swoboda:
This
is
Mel
Swoboda
again,
and
I
guess
I
can't
really
get
into
the
details
of
some
of
the
computer
things,
but
just
the
observation
as
a
user
in
a
computer
system,
and
Greg
might
be
able.
Greg
touched
on
a
part
of
it,

having
to
manage
a
website
and
having
to
keep
firewalls
up
to
keep
people
out
with
uncontrolled
access.

Many,
many
of
our
systems
and
many
of
our
companies,
and
probably
all
the
companies,
if
you
have
an
IS
group,
and
their
function
is
just
the
security
of
your
systems,
making
sure
that
you
can
get
in,
that
they
know
who
does
what
from
an
internal
standpoint
 
I
can't
get
to
every
record
that
you'd
like
me
to
get
to.
If
you
were
to
ask
me,
I
would
say,
I
don't
have
access
to
it,
and
we've
gotten
to
where
we
do
a
lot
of
those
things.

And
I
think
there
are
 
I
want
to
stress
think
 
I
believe
there
are
some
ISO
general
standards
that
a
lot
of
people
follow
that
deal
with
those
specific
kinds
of
issues.
Now,
do
they
deal
specifically
with
providing
me
with
detailed
audit
trails?
I
don't
know,
some
of
those
people
could
probably
really
tell
you
that
specifically.
But
I
do
know
they
can
track
a
heck
of
a
lot
of
information
around
movements,
whether
11
it
is
in
a
form,
necessarily,
if
I
get
into
one
of
my
systems
that
you
know
I'm
going
to
specifically
be
addressing
a
specific
piece
of
information,
I
don't
know.

The
other
thing,
I
guess,
that
comes
back,
I
looked
at
this
stuff
and
I
started
applying
it,
it
goes
back
to
where
do
you
cut
the
line
off?
Where
do
we
draw
the
line
on
these
trails
that
makes
it
appropriate
to
maybe
do
some
things
like
this?
If
it's
electronic
letters
that
I'm
putting
in,
maybe
it's
appropriate.
If
it's
an
electronic
report
that
I
put
in,

maybe
they're
appropriate.

But
when
we
start
backing
out
of
that,
it
starts
to
be
very,
very
difficult
to
figure
out
where
the
line
is
going
to
be
in
all
this
stuff,
and
Title
V
is
a
classic
one.
I
see
those
guys
over
there,
and
I
think,
thank
God
I'm
in
water
and
I
don't
have
to
deal
with
them.
Because
you
get
down
to
the
point,
in
those
data
systems
and
whatnot,
that
it
gets
extremely
difficult
to
meet
those
requirements.

Robin
Roberts:
Comments
on
criteria
again,
these
nine
criteria,
one
short
of
being
commandments.
Excuse
me,
I
can't
talk
about
religion
in
a
federal
government
setting.

Jim
McFarland:
Jim
McFarland
with
IR4
again.
What
I'm
hearing
here
as
far
as
the
electronic
systems
to
meet
this
particular
criteria
that
CROMERRR
has
proposed,
a
lot
of
the
12
safety
features
that
we're
now
using
are
sort
of
isolating
parts
so
that
no
one
person
has
access
to
everything.
That's
fine
for
a
big
company,
it's
not
that
practical
for
the
small
entities
that
I
deal
with,
typically,
a
one­
or
two­
person
field
researcher
that's
doing
[
Good
Laboratory
Practices]

residue
trials.

Small
laboratories.
That
type
of
safeguard
system
may
not
be
that
doable
in
that
kind
of
situation,
regardless
of
how
much
money
you
spend
on
the
software.
It's
going
to
be
very
likely
that
one
person
will
have
access
to
everything.
So
whatever
safety
devices
you're
trying
to
put
in
to
satisfy
the
legal
or
the
lawyer
aspects
may
not
even
be
there
if
you
go
the
full
nine
yards.

George
Britton:
George
Britton
with
Aventis
CropScience.

I
just
attended
a
meeting
of
the
National
Association
of
Independent
Crop
Consultants
that
met
in
Albuquerque
last
week.

One
of
the
questions
that
came
up
was
dealing
with
electronic
notebooks
and
the
[
Good
Laboratory
Practices]
variety.
And
one
fellow
got
up
and
asked
this
specific
question.
He's
a
one­
man
operation,
his
office
is
in
his
home.
I
am
assistant
manager
for
the
software,
because
he
is
the
guy,
he's
the
only
one
there.
Because
he
is
the
professional
investigator
doing
that,

so
he
would
log
in
as
a
different
user,
and
he
asked
the
13
question,
a
very
simple
question,
"
What
happens
to
me?
If
I'm
out
there
in
the
field,
I
have
a
heart
attack,
I
die,
whatever
happens,
how
is
the
information
brought
forward
to
a
sponsor
organization?"

And
all
these
kinds
of
things
down
there
don't
apply.

It's
basically
up
to
the
integrity
of
the
person.
All
of
the
fraud
cases
that
have
been,
at
least
publicly
prosecuted,

there's
been
IBT,
the
Craven
Labs,
and
the
currently
Infratech
System
that's
being
prosecuted
as
we
speak.
All
those
fraud
issues,
where
data
was
manipulated,
was
on
the
input
side.

It
wasn't
on
the
record­
keeping
side
of
the
house
where
problems
were
occurring,
it
was
on
the
input,
the
calibration
curves,
it
was
manipulating
of
chromatographic
peaks
before
they
were
recorded.
So
all
the
effort
to
prevent
fraud,
any
evidentiary
stake
that
has
been
going
forward,
is
not
addressed
by
any
of
these
nine
criteria.

Yes,
one
would
like
to
keep
them
safe,
by
the
current
practice
yes,
they
need
to
be
generally
accurate,
you're
required
by
law
already
to
be
accurate
and
are
prosecuted
if
you
don't.
What
works
for
a
very
small
outfit
with
one
or
two
people
will
not
be
the
same
solution,
processes
and
procedurewise
that
the
larger
corporations
can
do.
14
As
an
example,
Aventis,
even
before
all
this
came
out,

started
to
look
at
the
possibility
of,
simply
for
the
ease
of
information
management,
looked
at
a
record­
keeping
system,

spent
a
quarter
of
a
million
dollars
just
to
get
the
hardware,

software,
and
licenses
for
a
limited
installation
of
this,
the
idea
being
that
we
are
a
global
company.
We
do
have
records
all
over
the
world.
We
do
make
submissions
all
over
the
world.

How
do
we
have
the
final
signed
reports,
not
the
data
that
backs
them
up,
the
signed
reports,
available
for
our
regulatory
folks
to
combine
and
make
the
necessary
submissions
to
Asia,

Japan,
Europe,
wherever,
so
they'll
have
ready
access
to
the
signed,
certified
documents?

That's
just
for
one
little
case.
I'm
not
addressing
the
raw
data
and
the
other
things.
With
the
records­
management
system,
as
we
begin
to
do
it,
it
does
have
the
features
that
you
can
do
audit
trails
and
the
fact
that
you
can
maintain
versions
and
you
know
who
checked
it
out
and
who
checked
it
in
and
if
there
were
changes
made
between
the
two.

Those
kinds
of
things
will
work
at
a
larger
organization,

but
they
will
not
work
for
the
low
analytical
lab,
the
field
contractor,
those
kinds
of
solutions
won't
work.
There
has
to
be
recognition
that
I
will
do
things
at
certain
levels,
I
cannot
do
them
at
other
levels,
simply
because
of
cost.
15
Robin
Roberts:
I
wonder,
what's
the
experience
of
people
having
corrupt
data
input
through
the
system
from
the
beginning
as
opposed
to
having
it
manipulated
after
it's
been
entered?

Joe
Retzer:
Well,
they
shouldn't
speak
up
if
they've
manipulated
the
data.

Robin
Roberts:
We'll
erase
your
name,
then,
after
input.

Kenna
Study:
This
is
Kenna
Study
with
the
state
of
Florida's
drinking
water
program,
and
we
did
prosecute
labs
for
dry­
labbing.
Typically,
they
didn't
have
data
to
support
it,

and
it
took
a
person
interceding
and
reviewing
the
data
to
figure
that
out.
It
had
nothing
to
do
with
what
was
in
the
database,
other
than
the
fact
that
they
got
the
paper
that
came
in.
They
looked
at
it,
they
became
suspicious
and
did
an
investigation.
It
was
at
the
input
site,
not
the
transfer
of
the
data
or
anything
like
that.

Now
you
could
argue
that,
at
the
time
they
went
to
the
lab
and
they
didn't
have
the
records
to
support
it,
that
maybe
something
of
a
record­
keeping
nature
was
completely
missing
and
played
a
part.
But
I
don't
see
these
criteria
as
being
supportive
for
that
kind
of
an
investigation
at
all.
I
don't
see
the
difference.

I
mean,
it
would
be
similar
to
when
we
get
the
data
in
on
paper
currently
and
it's
got
a
signature
on
it
from
the
lab,
us
16
saying,
"
well,
we
can't
accept
that.
You
need
to
show
up.
I
need
to
see
I.
D.,
and
all
this
other
kind
of
stuff,"
is
more
parallel
to
what
you're
proposing
here.

Joe
Retzer:
I
think,
again,
some
of
what
we
were
doing
here
clearly
reflects
the
compliance
concerns
of
some
EPA
compliance
folks,
and
one
of
the
areas
that
they
have
had
a
number
of
cases
over
the
years
in
are,
for
example,
in
discharge
monitoring
reports,
where
there
has
been
some
things
like
stuff
whited
out
on
the
reports
and
data
manipulated
after
the
initial
laboratory
work
has
been
done.
So
there
are
a
number
of
examples
of
midstream
data
manipulation
as
well
as
the
kind
of
problems
with
data
entry.

Don
Whaley:
This
is
Don
Whaley
with
Navajo
Refining
again.
Not
to
say
that
there's
not
changes
under
the
QA/
QC
process,
there
are
times.
We
have
an
internal
lab.
They
run
GCs.
They
input
data
in
columns
into
a
database,
then
they
convert
that
to
a
spreadsheet
and
you
can
obviously
see
they
entered
a
number
in
the
wrong
column.
That's
part
of
the
QA/
QC
review
you
go
through
before
you
generate
reports.

I
guess
 
as
far
as
 
if
fraud
were
widespread,
there
wouldn't
be
so
many
companies
turning
in
reports
showing
noncompliance
So
I
think,
basically
the
integrity's
out
there.

Not
that
there
aren't
some
fraudulent
cases
going
on
out
there.
17
So
I
don't
think
you
should
punish
the
whole
industry
for
the
few
bad
apples
that
are
out
there.

Joe
Retzer:
Well,
I
guess
the
point,
I'm
not
an
enforcement
person
myself,
but
we
spent
a
lot
of
time
talking
with
them,
taking
sort
of
many
of
your
arguments
to
that
group.

And
I
guess
they
would
say,
even
if
it's
98
percent
or
so,
or
whatever,
99
and
a
half
percent,
whatever
the
actual
compliance
rate
is,
the
problem
is
that
there
are.
In
fact,
people
who
do
send
in
fraudulent
reports
and,
in
fact,
you
need
a
strong
deterrence
from
doing
that
and
one
of
the
ways
of
providing
that
deterrence
is
to
provide
a
regulatory
scheme
that
enables
you
to
enforce
against
them
if
you
catch
them.

And
one
of
the
particular
issues
that
we
haven't
really
talked
about
a
lot
here,
is
the
fact
that
in
some
of
the
cases,

the
fraud
involves
not
just
administrative
cases,
but
criminal
ones.
And
in
a
criminal
case,
you
really
have
to
be
able
to
track
the
fraudulent
activity
back
to
an
individual.

And
that's
part
of
the
reason
for
things
like
an
audit
trial,
to
be
able
to
say,
I
want
to
know
who,
what
individual
at
what
particular
time,
may
this
manipulation,
this
fraudulent
manipulation
of
the
data,
whether
it's
at
the
front
end
or
somewhere
in
the
middle.
18
And
that's
a
whole
sort
of
different
ballgame
than
other
situations.
For
example,
is
anyone
here
involved
in
TRI
reporting?
TR
reporting,
you
send
your
report
in.
TRI
sends
you
back
a
statement
that
says,
"
Here's
the
report
you
sent
us,

take
another
look
at
it,
we
think
you
might
have
had
a
problem
in
this
area,"
or
"
your
piece
here
translates
differently
than
what
you
sent
last
year,
are
you
sure
that's
right?"
And
you
get
sort
of
a
second
chance
to
send
the
data
back
in.

So
there's
a
variety
of
different
kinds
of
reporting
circumstances
that
people
operate
in
their
terms
of
compliance
with
EPA.
But
a
lot
of
the
reason
for
the
thinking
behind
these
criteria,
as
well
as
on
the
reporting
side,
the
criteria
there,
is
that
there
are
criminal
statutes
that
govern
fraud
associated
with
reporting
to
government
agencies,
and
that
means
you
have
to
be
able
to
track
it
back
to
an
individual.

And
on
the
electronic
side,
that's
what's
tough.
On
the
paper
side,
you
at
least
have
a
signature
on
paper
that
people
can
go
look
at
and
see
if
you
can
verify
the
signature.
On
the
electronic
signature,
you
know,
how
can
you
be
sure
it's
not
a
dog,
you
know,
a
pet
sending
you
the
info?
There's
this
famous
New
Yorker
cartoon.

How
do
you
verify
it's
the
individual,
especially
if
you're
talking
about
a
criminal
fraudulent
case
and
you
have
to
19
convince
a
jury
that
this
person
intended
to
send
this
information
fraudulently,
and
that
it
was
that
person
and
not
somebody
else
in
the
office
who
had
access
to
that
computer
who
sent
that
data
in.
And
that's
kind
of
a
lot
what's
behind
the
requirements
you
see
in
this
rule,
both
in
the
record­
keeping
side
and
the
reporting
side.

Mel
Swoboda:
Joe,
this
is
Mel
Swoboda
of
DuPont.
I'm
going
to
take
off
the
DuPont
hat
a
minute
and
put
on
an
old
regulator
hat.
It's
been
years
ago,
but
we
were
dealing
with
paper
and
you
could
always,
I
guess,
even
if
you're
in
this
situation,
if
you've
got
a
problem,
there
are
things
that
show
up
in
it
that
you're
going
to
be
able
to
track
that
identify
you've
got
a
problem.

To
think
at
this
time,
even
if
you've
got
a
paper
signature,
you've
got
a
good
signature,
I
think,
with
some
of
the
things
you've
see
as
far
as
what
you
can
do
with
a
Xerox
machine
or
a
good
computer,
those
days
are
over,
too.
I
mean
the
reality
is,
like
my
dad
used
to
tell
me,
locks
are
to
keep
honest
people
honest.
If
somebody's
crooked,
they're
going
to
do
whatever
it
takes.
Whatever
you
put
that
record
up
there,

they
will
get
through
and
manipulate
it.

Now
the
question
is,
what
is
the
true
value
to
what
we're
trying
to
protect?
Which
is
environment.
And
that's
the
hard
20
question.
And
I
guess
I
have
a
hard
time
seeing
where
we're
doing
anything­­
to
truly
spend
a
lot
of
dollars
on
something
that
has
a
lot
of
value,
from
an
environmental
standpoint;
I
mean
straight
environmental.

And
we're
just
stacking
a
lot
of
things.
If
I
have
to
do
something
like
this,
I'm
going
to
spend
a
lot
of
money
on
it
that
I
could
be
applying
in
other
places
environmentally
where
I
need
to
do
some
work.

Robin
Roberts:
A
few
more
comments
and
then
we
should
break
for
lunch,
about
five
minutes
early,
why
don't
we
make
it
a
one
hour
and
five
minute
lunch
and
return
at
the
scheduled
time
of
1:
15.
So
1:
15,
we'll
see
you
back
here
and
we'll
see
you
promptly
at
1:
15.

What
I'd
like
to
do
now
is
find
out
if
over
lunch,
in
your
discussions,
maybe
you
had
any
further
thoughts
about
the
general
criteria
discussion
that
we
concluded
before
we
left
that
you'd
like
to
say
now,
that
upon
further
reflection
comes
to
the
surface.

And
then
we'll
move
on
to
the
2:
15
to
3:
15
item
there
on
alternative
approaches
to
e­
record
standards,
keeping
in
mind
that
this
is
an
information
exchange
kind
of
an
options
development
conversation
we're
trying
to
have
and
that
any
21
alternatives
that
may
be
identified
by
you
aren't
anything
that
are
being
considered
as
possible
implementation
items.

I
have
one
comment
here
and
then
I'll
come
back
to
you,

sir.

George
Britton:
George
Britton
of
Aventis
CropScience.

In
thinking
about
all
this,
it
seems
to
me
there
are
two
levels
of
risk
analysis
that
need
to
be
done.
One
is,
what
does
all
this
have
to
do
with
a
particular
program
in
terms
of
the
risk
of
the
environment,
what
is
the
cost
benefit
in
there?

The
other
thing
is
recognizing
that
each
of
the
current
programs
require
different
kinds
of
records,
for
example
160
has
60
different
kinds
of
records
required
to
be
kept
in
one
form
or
the
other
and
most
of
those
are
not
submitted
to
the
Agency
under
formal
reporting
purposes.

For
example,
one
of
them
is
the
quality
assurance
master
schedule,
which
is
literally
a
listing
of
studies
and
the
status
ongoing
in
the
facility.
It's
never
submitted
to
the
Agency.
It
is
viewed,
for
their
benefit
when
they
come
in,
to
say
what's
ongoing,
what's
the
status,
what
the
head
counts
are
and
those
kinds
of
issues.

What's
the
real
risk
of
either
fraud
or
anything
else
if
a
particular
version
of
that
was
misplaced,
lost,
or
damaged
in
some
fashion?
I
don't
think
that's
a
very
big
risk
in
the
22
whole
scheme
of
the
environment
of
what's
going
on.
Yes,
we'd
want
to
do
it,
we
could
regenerate
it
from
the
protocols
that
we
are
required
to
keep
within
the
QA/
QC
unit.

The
other
thing,
an
example
of
a
record
that
would
be
in
an
electronic
format
that's
required
under
the
predicate
rule
is,
for
example,
that
the
archive
be
indexed.
Yes,
we've
chosen
to
do
it
electronically,
simply
because
it's
convenient
to
do
it
electronically.
Right
now,
in
our
particular
case,

I'm
tracking
over
3,000
studies.
If
I
had
3,000
notecards,
if
you
asked
me
to
find
all
the
pieces
of
a
particular
study,
it
would
take
me
some
time
to
go
manually
through
it.

If
I
use
an
electronic
database,
I
can
find
it
much
quicker
and
make
it
much
more
available
to
you
when
you
come
in
to
audit.
But
even
with
that,
I
can
still
regenerate
it,
even
from
the
paper,
if
it
were
damaged
to
the
point
it
was
irreversible.

But
in
the
whole
scheme
of
things,
does
that
really
affect
the
environment?
The
answer
is
no.
In
all
these
records,

there
doesn't
seem
to
be
an
acknowledgement
that
there
are
records
like
that
that
have
different
policies,
that
have
different
risks
to
the
environment
or
different
risks,
in
terms
of
good
business,
if
you
lost
it,
fire
damage
or
whatever.
23
There
are
two
issues,
first,
what
it
does
in
terms
of
protecting
what
you
discharge
and
put
out,
protecting
the
environment
and
what's
the
cost
of
doing
it,
and
the
same
thing,
what
does
it
really
do
in
my
normal
practice
if
I
lose
a
particular
record?
Those
are
cases
that
don't
seem
to
be
put
forward
in
this
and
need
to
be
addressed.

Jim
McFarland:
Jim
McFarland
from
IR4.
You'd
asked
earlier
about
holes
that
need
to
be
plugged,
additional
legislation
or
rules
that
need
to
be
made
in
order
to
help
this
process
of
not
only
CROMERRR
but
record
keeping
in
general,
it
sounds
like,
and
electronic
record
keeping
in
particular.

The
discussion
to
me
has
pointed
out
that
there
is
not
direct
correlation
between
electronic
and,
maybe
some,
but
there's
not
a
direct
correlation
between
electronic
records
and
paper
records,
they
are
different
and
there
are
different
considerations.

One
issue
that
I
touched
on
earlier
and
has
been
touched
on
by
other
people
in
the
sense
of,
the
suggestion
was
made
to
draw
a
line
and
say,
from
this
point
forward,
CROMERRR
applies.

At
that
point
the
suggestion
was
the
report
that's
submitted.

The
data
and
the
information
previous
to
that
CROMERRR
wouldn't
apply
to.
Well,
I'm
not
sure
that's
workable.
24
But
one
area
that
I
see,
especially
in
my
realm,
which
is
[
the
Federal
Insecticide,
Fungicide,
and
Rodenticide
Act]
[
Good
Laboratory
Practices],
there
needs
to
be
some
sort
of
line
drawn
in
the
sand,
so
to
speak,
at
the
definition
of
original
data.
We
have
an
interpretation
from
Frances
Leem
at
[
the
EPA
Office
of
Enforcement
and
Compliance
Assistance].
Hers
is
very
much
along
the
lines
of
CROMERRR.
She
believes
an
electronic
signal
is
the
original
data.

And
this
has
been
a
matter
of
some
discussion
and
a
little
bit
of
controversy
since
I
started
with
the
[
Good
Laboratory
Practices]
program
in
'
93.
The
first
meeting
I
actually
went
to
was
an
EPA
meeting
in
Washington,
D.
C.,
where
the
host
of
the
meeting
from
the
EPA
made
the
statement
that
the
original
signal
was
the
raw
data.
Well
it
didn't
fly
very
well.

As
I
know
the
[
Good
Laboratory
Practices]
right
now,
it's
not
specified
either
in
the
guidelines,
the
automated
data
guidelines,
or
in
the
regulation
itself
what
the
original
raw
data
is
in
a
situation
where
the
data
is
being
generated
electronically.
It's
pretty
cut­
and­
dried
in
some
situations,

if
you
record
a
number,
that's
the
original
raw
data.
The
guidelines
say
that
in
the
case
of
a
balance,
say,
that
has
a
printer
attached,
if
you
have
no
direct
input
into
changing
or
modifying
that
data,
the
original
data
is
that
printout.
25
If
you
put
it
into
a
spreadsheet,
that's
a
whole
different
matter.
I
think
this
is
an
area
where
there
needs
to
be
some
additional
input
as
far
as
rules
from
EPA
or
some
government
entity
to
define
what
original
data
is.

Robin
Roberts:
Other
concerns
about
data,
what
is
original
or
raw
data,
or
the
whole
notion
of
two
kinds
of
risk
analyses?

Unidentified
Female
Speaker:
Getting
to
the
contract
lab
issue,
how
do
we
separate
the
data,
how
do
we
get
the
actual
original
data,
it
was
mentioned
before
lunch,
there
are
a
lot
of
proprietary
issues
involved.
To
give
a
very
simple
example
of
what
happens
all
the
time,
a
typical
environmental
contract
laboratory
will
analyze
multiple
clients'
samples
in
the
same
analytical
batch.
And
that's
standard
practice,
it's
just
economically
efficient
for
them
to
do
that.

But
then
they
have
to
separate
that
data
and
give
the
results
to
the
individual
client,
and
they
may
print
it
out.

You
get
the
hard
copy
from
them,
but
in
that
whole
electronic
sequence,
they
can't
give
us
all
that
data,
because
they
can't
give
the
other
client's
data,
proprietary
reasons
of
confidentiality.
So
they
have
to
separate
that
out.
There
would
be
gaps
in
the
electronic
sequence.
I
don't
know
how
those
would
be
handled,
how
we're
going
to
address
those
26
issues,
and
any
time
that
you
separate
things
out
or
go
through
those
other
processing
steps.
There's
always
a
chance
for
error
and
loss,
of
course,
because
it's
basically,
I
guess
you
would
consider
it
like
a
transcription
of
data.
You're
transcribing
data.
So
those
are
some
of
the
other
issues
concerning
the
contract
lab
issue.

Robin
Roberts:
Great,
any
other
comments
or
thoughts
on
this,
or
that
came
to
mind
as
you
were
having
lunch?

If
there
aren't,
we
can
come
to
the
next
section
on
our
agenda,
which
is
about
alternative
approaches
to
electronic
record­
keeping,
and
to
some
degree
we've
spoken
to
this
issue
already,
but
that
shouldn't
stop
us
from
talking
about
it
some
more.

Some
of
the
questions
you
might
want
to
think
about
in
this
area
are,
are
there
recognized
industry
standards
for
maintained
electronic
records
to
assure
integrity
and
reliability?
What
are
good
alternatives
to
audit
trails?

Audit
trails,
that's
a
complicated
thing.
We've
discussed
that
a
lot
of
the
currently
available
software
that's
widely
used
doesn't
have
an
audit
trail
component
and
if
this
would
be
a
difficult
thing
to
build
in.

So
basically
this
is
an
opportunity
to
hear
from
you
about
alternative
technologies
that
might
satisfy
record­
keeping
27
provisions,
not
of
CROMERRR,
although
that
would
be
great,
but
also
within
your
own
corporate
environment
or
government
environment.
How
are
you
dealing
with
these
questions
of
data
integrity
and
data
nonrepudiation?
Are
you
going
to
make
me
work
really
hard
on
this
one?
Well,
are
there,
then
any
comments
from
the
panelists
on
this
one?

Joe
Retzer:
Yes,
I'd
also
just
like
to
hear
if
anybody
has
any
additional
examples
of
current
regulatory
ambiguities
or
problems
in
the
record­
keeping
areas,
generally.

Robin
Roberts:
What
about
Texas?

Unidentified
Male
Speaker:
Well
there's
one
that's
been
well
documented
and
been
very
well
known
for
a
long
period
of
time.
It
gets
into
the
difference
in
part
160
and
169,
books
and
records
for
[
Good
Laboratory
Practices].
The
[
Good
Laboratory
Practices]
does
allow,
that's
part
160,
allows
for
verified
or
certified
copies
of
raw
data,
it
will
substitute
for
the
original
raw
data.

Books
and
records,
169
Part
2K,
specifically
says
that
a
producer
must
keep
the
records
for
as
long
as
the
product
is
registered
and
as
long
as
the
producer
is
in
business.
The
original
record.

This
has
always
been
a
conflict
between
the
[
Good
Laboratory
Practices]
and
books
and
records.
Hopefully,
with
28
the
previous,
about
1998,
where
they
were
going
to
revise
the
[
Good
Laboratory
Practices]
to
be
in
concert
with
the
OLCDs
and
books
and
records,
some
of
these
things
were
addressed
in
that
proposal.
I
don't
remember
the
detail
to
that
particular
thing,
but
that's
a
particular
case
where
one
program
will
allow,
or
part
of
its
program
will
allow,
copies
when
the
verbiage
in
the
other
says
the
originals.

Joe
Retzer:
And
do
you
take
that
to
mean
that
if
the
original
was
electronic,
then
you'd
have
to
keep
the .

Unidentified
Male
Speaker:
Yes,
whatever
the
original
is.

And
by
using
this
extreme
definition
of
everything
that's
been
maintained,
created,
dah,
dah,
dah,
I'm
in
violation,
today
even,
from
that.
Even
before
this
 
books
and
records,
there's
some
ambiguities
within
[
the
Federal
Insecticide,
Fungicide,

and
Rodenticide
Act]
itself,
where
those
two
items,
one
allows
copies,
one
doesn't
say,
it's
not
clarified
what's
meant
in
both
of
those
cases.

That's
just
one
side
of
the
house
where
electronic
records
 
what
we
attempted
to
do,
the
idea
is
to
maintain
number
one
up
here,
the
generation
of
accurate
records,
is
we
choose
to
have
kept,
from
the
electronic
systems,
the
chromatographs
are
signed
and
dated.
The
original
one,
the
first
one
off
the
machine,
is
kept,
regardless.
29
By
instrumental
analysis
the
peaks
may
not
be
exactly
right.
Maybe
the
baseline
windows
are
not
exactly
right.
We
may
reprocess
it
four
or
five
times.
All
the
in­
betweens
we've
chosen
not
to
keep,
but
we've
taken
the
one
that
we
will
use
as
the
basis
of
reporting,
that's
signed
and
dated,
both
the
very
first
one
that
was
printed,
with
all
its
errors,
and
the
one
that
was
accepted,
are
both
kept.
We
keep
them
on
paper
and
the
access
to
that
record
or
records
becomes
narrower
and
narrower
as
you
get
toward
the
reporting
process.

Once
they
get
to
the
archive
point,
basically
it's
the
one
or
two
people
in
our
organization
that
have
access
to
the
archive
who
have
that
information.
We
have
chosen
to
image
all
the
information,
all
that's
kept
in
the
study,
file
all
the
correspondence,
all
invoices .
[
end
side
A]

[
Side
B]

 
the
CDs,
and
we
always
can
go
back
and
check
the
electronic
image
of
the
CD
if
you
really
wanted
to
go.
If
this
is
really
what's
on
there,
you
could
always
go
back
to
the
original
paper
and
actually
check
it
against
it.
You
can
actually
go
back
and,
because
those
CDs,
when
those
scanned
images
are
done,
that's
the
date
on
the
record.
So
you
know
when
it
was
done.
But
once
you
see
the
CDRW
record,
you're
not
able
to
go
back
and
add
it,
or
change
the
record
anywhere.
30
Those
are
some
of
the
kinds
of
things
that
are
going
on
to
address
a
lot
of
this.
While
there's
not
an
audit
trail
per
se,
on
the
paper
you've
got
the
signature
side,
but
as
you
get
from
the
fellow
who
generated
it
to
the
study
director
to
the
QA/
QC
unit.
To
the
archive,
the
access
to
the
records
become
much
more
narrow,
and
basically
once
it's
in
the
archive,
it
never
sees
the
light
of
day
again.

That's
one
reason
for
imaging
it,
because
you
now
have
the
record
available,
the
image
of
it,
that
your
people
can
get
information
from.
But
the
original,
which
we
define
as
a
substitute,
verified
copy
of
the
paper,
is
now
available
for
everybody
else,
it's
away,
safe,
and
secure.
Policies
and
procedures
as
opposed
to
a
rigorous
technology
force
[?]

Robin
Roberts:
How
many
of
you
work
with
regulations
that
have
a
criminal
component
to
them?
And
how
many
civil?
I'm
trying
to
get
at
the
whole
issue
of
risk
analysis.
This
onesize
fits­
all
seems
to
be
geared
towards
regulations
where
you're
trying
to
have
data
that
will
stand
up
in
court
beyond
a
reasonable
doubt,
and
there
is
some
notion
that's
too
heavy
a
burden
or
too
heavy
a
burden
for
some
of
this
data
to
carry,

that
it's
not
worthy
or
up
to
that
kind
of
status.
And
I'm
just
wondering
how
many
of
you
deal
in
programs
where
you
don't
31
have
that
kind
of
burden
and
how
many
of
you
do.
Also
trying
to
figure
out
a
way
to
get
you
to
talk,
but .

Any
other
ideas
about
alternative
approaches
to
e­
record
standards
and
how
it's
being
done?

Nancy
Freshour:
Sorry,
I'm
going
to
digress,
Nancy
Freshour
from
Dow
Chemical
again.
We
talked
about
maintaining
the
data
in
these
systems,
if
you
talk
about
original
data,
my
experience
when,
as
in
the
laboratory
when
we
were
using,
say
Access
Chrome
or
Turbo
Chrome
data
systems,
you're
collecting
metagraphic
files
and
at
some
point,
that
system
is
only
going
to
hold
so
many
files.
You
have
system
managers,
they
do
backup
tapes,
they
save
the
tapes,
they
save
all
that,
but
that's
not
the
original,
the
original
is
in
the
data
system.

But
after
a
while
the
system
can't
hold
it,
it
slows
down,

it
can't
even
collect
data
because
it
has
no
place
to
put
it
or
it's
having
trouble
finding
the
places
to
put
it,
so
you
have
to
delete
the
data
from
that
system,
migrate
it
to
something
else
in
order
to
keep
operating
your
system.

So
I
don't
know
how
you're
going
to
be
able
to
keep
those
original,
electronic
records,
unless
you
fill
up
a
system,
then
implement
another
system,
it's
technically
impossible
to
do
that
and
keep
running.
32
Robin
Roberts:
I
even
have
a
question
about
this
original
data.
Is
there
an
industry
standard
which
says
it's
okay,
it's
generally
recognized
that
you're
deleting
data,
because
progress,
the
idea
of
staying
in
business?

Nancy
Freshour:
Well,
we
print
out
the
chromatograms
because
it's
under
[
Good
Laboratory
Practices].
So
we
have
that
and
we
have
the
tape
backups,
but
we
don't
have
the
original
electronic
files.

Robin
Roberts:
At
what
point
is
it
acceptable,
then,
or
is
there
a
point
at
which
it's
no
longer,
it's
generally
recognized
within
the
industry
that
it's
unreasonable
not
to
have
an
electronic
record
of
data?
Or
can
it
vary
from
process
to
process,
industry
to
industry?

Basically
what
you're
saying
is
that
in
industry
there's
this
reality
that
it
goes
to
paper
and
what
I'm
asking
is,
is
there
a
generally
recognized
point
at
which
it
goes
to
paper
or
is
this
a
management
call
within
all
the
corporations
that
have
to
deal
with
this?

Nancy
Freshour:
I
guess
from
our
experience
it's
business
needs
and
regulatory,
because
if
you're
looking
across
the
business,
not
just
environmental,
Dow
provides
a
variety
of
tools
for
us,
both
software
and
hardware
tools,
but
the
businesses
have
to
make
a
decision
and
the
owners
of
whatever
33
that
is
have
to
make
decisions
on
what
they
need
to
meet
their
business
needs
and
their
regulatory
needs
and
implement
it
that
way.
So
there's
no
quote,
unquote
"
one­
size­
fits­
all"
standard
even
within
our
company
on
how
you're
going
to
apply
it,

because
not
everybody
is
governed
by,
say,
environmental,
some
people
are
governed
by
something
else.
So
I
guess
that's
not
really
an
answer
to
your
question,
but
that's
just
the
reality.

Bill
Yancey:
Bill
Yancey
with
BP.
We
have
several
means
of
collecting
real­
time
data
that
we're
required
to 
In
the
cases
of
our
continuous
emission
monitors,
we
take
six
second,

in
other
words,
every
six
seconds,
a
carbon
monoxide
reading,

and
to
calculate
a
rolling
average.
And
as
you
can
imagine,

after
a
few
months
the
data,
we
tried,
we
filled
up
a
few
file
boxes
and
filled
up
a
warehouse
at
the
end
of
five
years,
but
the
electronic
storage
simply
was
not
able
to
be
sustained.

So
we
kept
the
paper
reports,
and
that's
where
this
warehoused
amounts
of
rolling
averages
of
carbon
monoxide
came
in.
But
in
the
instrumentation
that
we
use
for
environmental
reporting
purposes,
it
goes
all
the
way
to
things
like
flow
meters,
which
we
use
for
waste
water
discharges.
And
most
of
our
flow
meters
read
every
fifteen
seconds,
and
of
course
continuous
pH
analyzers,
those
read
about
every
ten
seconds.
34
Again,
the
electronic
storage
of
all
these
data
bytes
is
just
unpractical
so
we
rely
on
the
printout
for
that
month's
worth
of
data,
and
that's
what
we
submit,
in
some
cases
electronically.
But
the
records
that
are
maintained
electronically
are
very,
very
short­
lived.
Thanks.

Robin
Roberts:
Any
comments?
Well
then
why
don't
we
just
take
a
look
at
this
next
discussion
here
again.
It's
something
that's
come
up,
but
I'm
thinking
that
perhaps
you
may
have
more
comments
about
it
as
well.
What
special
issues
are
raised
by
CROMERRR
criteria
for
long­
term
archiving?
We've
somewhat
covered
that.

Do
you
have
a
question
or
a
comment?

Pat
McFadden:
Pat
McFadden
from
Gustafson.
When
you
look
at
long­
term
archiving
of
electronic
data,
there
are
so
many
issues
to
consider,
that
it
really
is
pretty
overwhelming.

We
currently
have
several
different
systems,
and
recently
we
tried
to
pull
up
some
data
from
1998.
The
system
crashed
when
we
tried
to
pull
it
up.
We
were
going
to
reinstall
the
software,
the
software
from
1998,
at
that
point,
was
still
old.

It
wouldn't
go
onto
a
computer
because
it
had
to
be
a
386
or
less,
it
had
to
run
five­
and­
a­
quarter­
inch
floppies,
so
there
was
no
way
for
us
to
get
this
data
back.
The
electronic
files
are
lost
forever.
35
And
that's
only
three
and
a
half
years
ago.
So
you
think
about,
if
you're
going
to
have
electronic
files,
how
are
you
going
to
migrate
the
data?
If
you
migrate
the
data,
how
are
you
going
to
migrate
the
audit
trails?
How
are
you
going
to
migrate
the
meta
data?
Who's
going
to
be
responsible
for
checking
all
these?

Our
quality
assurance
guy
is
not
a
computer
expert.
That
means
that
we
have
to
hire
somebody
new
to
be
in
charge
of
this
whole
new
process
of
migrating
data
to
new
systems.
Plus,
we
have
to
be
able
to
afford
new
systems
and
afford
new
people,

and
it
becomes
very
complex.

So
we
have
old
data
that
we
can't
look
at,
because
we
can't
bring
it
up
on
a
system
anymore
because
we
didn't
think
to
migrate
it,
because
we
thought
it
would
still
be
good.

This
will
be
full­
time
jobs
for
lots
of
people,
we're
a
small
company,
but
I'm
thinking
you
big
guys
out
there,
you
have
tons
of
data
that's
on,
hopefully,
on
CDs
or
whatever,
and
you're
going
to
have
to
migrate
them
to
your
new
system.
Let's
say
Chem
Station
totally
revamps
how
it
works
and
Agilent
[
Technologies]
comes
out
with
brand
new
software.
So
you're
going
to
have
to
take
your
old
software,
every
single
residue
study
you've
ever
done,
every
single
sample
you've
ever
analyzed,
you're
going
to
have
to
migrate
it
to
the
new
system,
36
so
that
when
that
EPA
auditor
comes
in
to
look
at
your
study.

You
can
bring
that
file
up
electronically,
because,

theoretically,
we're
going
to
have
to
bring
up
our
electronic
files
as
well
as
our
paper
files
to
show
them
we've
been
in
compliance
with
CROMERRR.

We're
not
sure
how
that's
really
going
to
work,
are
they
going
to
bring
their
copy
of
the
software
with
them
on
their
computers
or
how
that's
going
to
work.
We
haven't
even
worried
about
that
problem
yet.

But
the
problem
also
becomes,
then,
do
you
have
to
save
your
systems?
Are
you
going
to
have
these
legacy
systems,
or
you
could
have
a
whole
archive
full
of
computers.
You
know,

here's
my
little
Tandy,
I've
been
saving
it
just
in
case
you
guys
come
in
and
look
at
this.
We've
gone
through
lots
and
lots
of
different
computers,
and
people
keep
changing
things.

Bill
Gates
always
makes
new
products
so
that
he
can
sell
more
things
and
he
can
make
more
money,
which
makes
us
have
to
upgrade
all
the
time,
and
every
time
we
upgrade
our
system,

then
we're
going
to
have
to
validate
the
system
again.
So
we're
not
only
spending
money
on
migrating
data,
we're
revalidating
systems,
we're
doing
verification.
All
of
that
is
very
expensive.
37
If
the
standard
was,
make
the
paper
copy,
put
it
in
the
archive
and
we'll
be
happy
with
that,
then
our
costs
would
be
reduced
and
I
think
that's
something
that
needs
to
be
looked
at,
because
saving
electronic
files
for
the
life
of
a
[
Federal
Insecticide,
Fungicide,
and
Rodenticide
Act]
registration
is
almost
impossible.

Donald
Whaley:
This
is
Don
Whaley
with
Navajo
Refining.

One
point
that
you
brought
up
as
far
as
the
archiving,
and
the
changeover
in
the
technology,
even
the
databases,
we
are
some
of
the
fortunate
ones
that
were
able
to
do
a
114
request
last
year,
about
this
time
of
year,
in
fact,
and
some
of
the
lab
data
that
we
went
to
retrieve,
we
actually
had
to
go
through
efforts
to
load
historical
databases
on,
major
efforts
to
get
that
information.

But
EPA
requested
spreadsheet
format
submittals,
and
at
that
point
in
time
they
didn't
ask
for
any
kind
of
validation,

any
kind
of
auditing,
yet
they
were
willing
to
accept
that
in
the
context
of
enforcement
issues.

Robin
Roberts:
So
that's
past
experience
with
old
data
being
good
enough
in
an
enforcement
case,
and
no
issues
were
raised
about
its
integrity
or
authenticity,
and
the
court
held
it
to
be
nonrepudiatable,
similar
criminal
cases?
38
Don
Whaley:
Ultimately
a
consent
decree
came
out
of
it
although
there
was
never
any
action
taken
as
a
result
of
the
114.
There
were
some
other
issues
that
were
settled
with
the
state
and
then
our
partnership
announcement
that
came
out
in
December.
I
guess
that
would
be
considered
a
civil
action.

Robin
Roberts:
Anyone
been
to
court
with
any
data
here?

Great,
it
seems
like
the
notion
of
having
to
keep
a
small
museum
of
hardware
in
your
corporate
garages
to
read
old
formats
that
Bill
Gates
has
forced
us
to
move
away
from.
Are
there
any
other
concerns
about
how
you
would
deal
with
this
old
data
that
accumulates
in
any
long­
term
archiving?

Evi
Huffer:
I
just
wanted
to
mention
something,
we
keep
coming
back
to
the
conversion
issue
and
being
allowed
to,

particularly
in
the
[
Good
Laboratory
Practices]
world,
to
keep
printed
copies
of
your
raw
data.

EPA
has
been
doing
some
additional
research
in
that
area.

We
have
a
contractor
who
is
working
on
a
study
right
now,
we'll
be
getting
a
copy
of
it
next
week.
It's
looking
at
converting
electronic
records
to
paper
records
and
the
feasibility
for
meeting
CROMERRR
criteria
and
what
software
is
out
there
that's
available,
etc.

We
will
be
making
that
report
available
in
the
docket,
so
check
the
docket
in
a
couple
of
weeks.
We'd
like
to
see
the
39
report
first,
and
get
the
contract
and
clean
it
up,
if
there
are
typos
and
things
like
that
in
there,
but
hopefully,
even
in
the
next
two
weeks
we'll
be
able
to
have
it
out
on
the
web
for
you
and
you'll
be
able
to
find
it
on
our
website,
which
is
in
the
presentation
package.

Mel
Swoboda:
Evi,
one
question.
Mel
Swoboda,
with
DuPont.
This
kind
of
drives
to
a
piece
that
really,
I
know
you
guys
are
not
involved
directly
in,
but
the
hard
look
that
needs
to
happen
on
what
do
we
really
need
to
keep?
I
know
[
the
Resource
Conservation
and
Recovery
Act],
I
understand
just
this
past
week
came
out
with
a
notice
about
looking
at
record
retention.

The
question
then
comes
back,
do
you
really
need
to
know,

the
classic
one
I
always
use,
is
do
you
really
need
to
know
in
1979
on
January
the
31st
at
two
o'clock
in
the
afternoon
what
the
flow
rate
is
on
my
injection
rod?
Is
that
really
an
important
piece
of
information
at
this
point
in
time?
Or
do
you
need
to
know
what
the
annual
volume
that
I
injected,
which
you
then
use
in
your
modeling
for
another
EPA
program
to
model
out
ten
thousand
years
of
where
the
waste
goes?

So
it
goes
back
to
pushing
back
to
the
different
departments
and
saying,
let's
do
a
hard,
hard
look
at
what
am
I
required
to
keep
and
what
is
the
value,
and
even
beyond
that,
40
is
the
data
any
good?
Because
as
we
go
through
systems,
if
I
look
at,
and
I'm
just
going
to
use
water
quality
as
an
example,

if
I
go
back
15
years
and
look
at
the
water
quality
data,

that's
garbage.
For
purposes
of
doing
any
compliance
today,

it's
nowhere
near
the
technology
of
detections
that
we
have
today
and
it's
useless
information
at
this
point.

So
it's
really
one
of
those
questions
that
makes
a
lot
of
the
things
that
we're
talking
about
here
kind
of
moot.
Now
[
the
Federal
Insecticide,
Fungicide,
and
Rodenticide
Act],
I
understand,
is
some
different
things,
but
I
would
still
venture
to
guess
that
there
are
pieces
in
that,
that
the
question
is
still
a
valid
question.

Evi
Huffer:
Good
point.

Nancy
Freshour:
Well,
I
agree
with
the
previous
comment.

This
is
Nancy
Freshour
with
Dow
Chemical
again.
I
did
have
a
question,
Evi,
on
your
comments
about
the
contractor
looking
at
[
Good
Laboratory
Practices]
conversion
of
electronic
records.

Evi
Huffer:
He
wasn't
specifically
looking
at
[
Good
Laboratory
Practices]
conversion.
The
contractor
is
looking
at
the
feasibility
of
meeting
the
CROMERRR
criteria,
when
you
take
an
electronic
record
not
necessarily
one
under
the
[
Good
Laboratory
Practices],
and
convert
it
to
a
paper
record,
and
vice­
versa.
41
Joe
Retzer:
You
mean
the
criteria
in
terms
of
preserving
the
meta
data.

Evi
Huffer:
Preserving
the
meta
data,
the
audit
trail,

that
type
of
thing,
the
date/
time
stamp.

Nancy
Freshour:
Okay,
and
then
what
would
be
available,

and
then
those
would
be
specified
so
that
we
would
kind
of
be
limited
to
what
was .

Joe
Retzer:
No,
we're
just
doing
some
research
on
the
topic.

Evi
Huffer:
This
is
just
some
research
to
see
what's
out
there
right
now,
what
technologies,
what
software
is
available,

what
the
cost
is,
etc.

Nancy
Freshour:
Okay,
because
it's
not
just
[
Good
Laboratory
Practices].
[
Good
Laboratory
Practices]
is
one
of
the
areas,
of
course.
It's
all
recognized
by
everybody
here,

that
that's
where
a
lot
of
conversion
to
paper
occurs,
[
Good
Laboratory
Practices],
[
the
Federal
Insecticide,
Fungicide,
and
Rodenticide
Act],
and
[
unintelligible].
And
in
these
continuous
monitoring
situations,
where
we
have
to
collect
minute­
to­
minute
data,
and
are
required
to
do
that,
because
the
technology
is
there
and
the
Agency
recognizes
that
the
technology
is
there.
So
they
want
us
to
collect
minute­
to­
42
minute
data,
whereas
years
ago
we
couldn't
do
that,
we
don't
necessarily
keep
that
on
paper.

Evi
Huffer:
Right.

Nancy
Freshour:
So
even
if
you
have
a
feasibility
study
and
you
can
convert
that
to
paper,
I
don't
think
we're
probably
going
to
want
to
convert
that
to
paper.

Evi
Huffer:
And
this,
I
think,
goes
back
to
the
point
about
looking
at
the
scope
of
the
rule,
and
our
definition
of
what
an
electronic
record
is.
I'm
not
 
when
we
wrote
this
rule,

I
don't
think
it
was
our
intention
for
these
volumes
of
secondby
second
compliance
data
to
be
included
in
these
electronic
records.

Joe
Retzer:
I
guess
the
question
is,
what
does
the
current
regulation
on
the
books
require
you
to
keep?
And
that
again,
what
we're
trying
to
do
is
say,
what
does
that
require
you
to
keep,
if
the
current
regulation
requires
you
to
keep
that
data,
what
we're
trying
to
address
is
the
question
of
is
it
easier
for
you
to
keep
it
electronically,
if
you
have
to
keep
it
X
amount
of
time,
or
is
it
easier
for
you
to
keep
it
electronically
but
print
it
out.
I
mean
then
print
it
out
after
a
certain
length
of
time.

And
that's
the
issues
we're
trying
to
look
at
here,
is
that
if,
in
fact,
we
don't
put
in
a
provision
that
says
once
43
electronic
you
have
to
keep
it
electronic,
we're
trying
to
do
some
research
on,
basically,
do
existing
approaches
to
printing
out
the
data
preserve
what's
necessary?

Evi
Huffer:
Could
I
just
ask
a
question
here?
Let's
assume
that
CROMERRR
is
out
on
the
table
and
we
haven't
defined
what
an
electronic
record
is,
that
definition
is
not
there.

Would
you
then,
in
reading
the
rule,
would
you
interpret
the
rule
to
apply
to
this
type
of
data,
without
that
definition?

Nancy
Freshour:
Could
you
repeat
the
question
again?
I
don't
understand
what
you're .

Evi
Huffer:
For
your
particular
program,
let's
just
say
we
take
out
the
definition
of
electronic
record.
It's
not
in
the
rule,
it's
not
on
the
table.
And
what
we've
given
you
is,

we
haven't
defined
what
a
record
is,
we're
assuming
that
you
know
what
a
record
is
because
it's
defined
in
the
predicate
regulation.
So
we've
given
you
this
criterion
that
an
electronic
record­
retention
system
has
to
meet.
Would
you
assume,
in
reading
the
regulation
that
this
data
that
you're
talking
about,
this
monitoring
data,
would
fall
under
that
record
retention?
So
it's
more
than
just
the
definition.

Nancy
Freshour:
Nancy
Freshour,
Dow.
Yes,
you
would
have
to
assume
that,
because
anybody
who's
enforcing
it
or
inspecting
against
that
would
assume
the
most
conservative
44
interpretation
of
it,
and
so
yes,
you'd
have
to
take
that
route.

Terry
Salem:
This
is
Terry
Salem
with
TNRCC,
you'd
would
also
have
to
assume
that
because
the
underlying
predicate
regulation,
in
most
instances,
I
think,
requires
that
you
report
that
monitoring
data
in
some
forms.
And
so
if
the
predicate
regulation
is
already
assuming
that
that
data
is,
in
and
of
itself,
a
record.

Evi
Huffer:
And
is
being
kept?

Terry
Salem:
And
is
being
kept,
Title
V
is
a
perfect
example,
deviation
reporting,
every
bit
of
that
second­

bysecond
data
on
that
sim
is
feeding
an
emission
standard
that
you
have
to
report
every
little
blip,
and
then
you
have
to
go
in
and
certify
continuous
or
intermittent
compliance.

And
so
I
think
the
underlying
issue
here
is
more
than
just
how
do
we
keep
our
data,
it's
how
do
we,
and
I'll
say
we
as
environmental
regulators,
want
to
structure
the
picture
of
environmental
compliance,
right?
What
is
it
we
care
about,

what
is
it
we're
trying
to
get
to,
and
that
goes
back
to
the
whole
theme
earlier
of
what
is
it
that
we
really
care
about?

Data­
wise,
from
an
environmental
compliance
perspective,
what
are
you
trying
to
target,
what
are
you
trying
to
attack?
Where
are
the
pressure
points
for
you
as
an
environmental
regulator
45
to
ferret
out
the
information
to
keep
everything
in
line
and
whole.

And
right
now
the
problem
with
doing
this
as
a
broad
swath
approach,
I
think,
is
that
every
one
of
those
predicate
regulations
was
never
created
with
this
in
mind.
They
were
targeted
to
get
the
most
data
possible,
to
put
the
most
technological
constriction
out
there
that
was
possible,
you
know,
get
the
most
data,
get
it
as
timely
as
possible,
get
it
all,
and
from
that
you
would
see
the
whole
picture
of
compliance.

And
so
those
underlying
regulations
are
not
structured
in
a
way
that
makes
it
easy
to
then
back
it
out
and
say,
well
what's
really
the
record
of
importance
from
a
record­
keeping
perspective?

Sue­
Chi
Shen:
This
is
Sue­
Chi
Shen
from
Gustafson.
I'm
in
the
record
[
unintelligible]
and
I
have
another
situation
which
I
want
to
share
with
you.
Because
from
time
to
time,
in
the
regulatory
department,
we
do
interface
with
EPA
before
we
are
ready
to
set
our
[
unintelligible]
trial,
say
for
field
[
unintelligible],
say
you
need
twenty
locations,
but
then,

based
on
the
preregistration
meeting,
based
on
lots
of
rationales.
Yes,
sometimes
the
EPA
says,
say
Dr.
Smith,
the
head
of
her
division,
says
yes,
okay,
based
on
your
rationale,
46
yes,
we
agree
that
you
are
able
to
reduce
your
regular
locations
down
from
twenty
to
ten.
Okay?

Historically
what
we
do
is,
yes,
we
have
a
communication
back
and
forth
by
e­
mail
and
in
the
end
you
will
send
me
a
note
that
says,
Sue,
we
agree,
ten
locations
will
be
enough.
So
what
we
do
is,
we
make
a
hard
copy
and
I
will
send
that
hard
copy
to
Pat,
she
is
in
charge
of
our
[
unintelligible]
trial
 
when
she
finishes
her
ten
location
report.

I
think
it's
a
bit
nice
that
she
actually
attaches
that
communication
agreement
of
Dr.
Whoever
for
the
head
of
her
division,
the
justification
why
we
only
do
ten
instead
of
twenty.
I
think
that's
very
important,
you
know,
when
you
review
packages
years
from
now,
you'd
say
that
is
important.

Okay,
this
kind
of
information,
again,
go
back
to
what
the
gentleman
says,
what
are
required
under
these
electronic
records.
Do
I
have
to
actually,
when
I
negotiate
this
kind
of
agreement
for
the
last
three
months?
To
come
to
the
final
conclusion,
is
it
important,
do
I
have
to
show
all
that,
my
email
blah,
blah,
blah,
under
this
new
requirement?

That's
my
question.
This
is
how
do
you
interface
the
information
communication
log
between
the
records
people
from
headquarters.
It's
important
to
be
included.
This
is
something,
I
think,
that
needs
to
be
defined.
It
has
a
lot
of
47
impact
on
how
our
computer
lady,
over
at
headquarters.
She's
going
to
panic.
She'll
say,
"
Oh,
God,
I've
got
to
worry
about
your
computer
instead
of
worrying
about
Pat's
computer,"
blah,

blah,
blah,
this
is
just
another
situation.

Robin
Roberts:
Any
comments
about
this
problem?
Sure.

Sue­
Chi
Shen:
I
do
have
another
question,
we're
talking
about
the
record­
keeping
requirement,
what
are
EPA
persons
required
to
do
at
this
point?
Dr.
Smith
gave
me
the
okay
two
years
ago,
hey,
I
send
you
an
e­
mail,
so
you
can
get
by
with
ten
locations.
Is
he
required
to
keep
that
communication
under
his
file?
Is
he
required
to
do
that,
or
do
you
rely
on
the
rest
of
us
to
keep
every
piece
of
information,
say
you
told
me
so­
and­
so
two
years
ago.
Is
he
required?
I
can
create
something
from
Dr.
Smith
that
shows
he
told
me
to
do
that.
Are
they
required
to
do
it
over
on
their
side
in
case,

unfortunately
I
lose
my
paperwork,
I
can
go
to
him?
Are
they
required
to
do
that,
that's
my
question.

Joe
Retzer:
I
guess
that
would
really
depend
on
the
standard
operating
procedures
of
that
particular
group,
but
emails
of
the
sort
you
are
talking
about
at
EPA
are
official
records,
so
we
have
to
keep
those.
We
can
print
them
out,

however.
48
Evi
Huffer:
Actually
we
have
to
print
them
out,
we
have
to
print
them
out
at
this
point.

Robin
Roberts:
If
there
are
no
other
questions
or
comments.
Yeah,
we
have
one
here.

Janey
Shih:
Hi,
this
is
Janey
Shih
from
Lockheed
Martin.

We
are
talking
about
this
data
from
a
laboratory,
for
monitoring
data,
kind
of
directly
environmental
monitoring
data.
But
like
for
air
emission
inventory,
a
lot
of
the
data
is
coming
from
our
inventory
system,
another
system
which,
I
think,
was
created
maybe
fifty
years
ago,
which
for
sure
does
not
meet
the
CROMERRR
standard.
In
this
case
what
are
we
going
to
do?
Are
we
going
to
have
to
redo
the
whole
system
for
the
whole
company
in
order
to
be
able
to
comply
with
CROMERRR?

That
would
be
a
question,
absolutely,
that's
what
we
are
really
worried
about.

Robin
Roberts:
Any
response
to
that?

Joe
Retzer:
Well
I'm
not
sure
I
understood
the
question
exactly.
If
you're
saying
that
part
of
your
system
does
not
meet
the
CROMERRR
requirements,
if
we
were
to
go
ahead
with
the
rule
as
it
stands
now,
which
we're
strongly
suggesting
to
you
that
we
are
not,
but
if
we
were
to
do
that,
yes,
you
would
need
to
come
into
compliance
with
those
requirements.
49
Janey
Shih:
So
you
are
talking
about
all
of
the
systems
where
our
data
is
coming
from
has
to
also
meet
the
CROMERRR
standards.

Joe
Retzer:
You're
saying
if
the
data
comes
from
somewhere
else,
like
another
company
or
something
like
that?

Janey
Shih:
No,
our
company's
other
systems.

Joe
Retzer:
Oh,
your
other
data
systems.

Janey
Shih:
Yes,
we
have
many,
many
systems,
we
don't
only
have
one
environmental
data
management
system.

Joe
Retzer:
Again,
it
sort
of
depends
on
what
the
underlying
regulations
define
as
the
record
that
you
have
to
keep,
okay?
So
if
the
air,
water,
FIFRA,
whatever
regulation
defines
that
as
a
record,
for
that
program .

Janey
Shih:
For
example,
equipment
maintenance
records,

that
would
be
kept
in
our
maintenance
system.

Joe
Retzer:
Right,
if
keeping
those
records
is
a
requirement
under
one
of
EPA's
regulations
at
this
point,
as
CROMERRR
is
now
written,
it
would
apply
to
those
records.

Janey
Shih:
Our
IT
people
will
be
really
busy.

Don
Whaley:
This
is
Don
Whaley
with
Navajo
Refining.
One
thing
I
was
curious
about,
if
we
go
to
electronic
reporting,

will
this
lessen
the
phenomenon
of
the
EPA
inspectors
showing
up
and
wanting
to
see
copies
of
all
our
reports?
Because
50
historically,
I
often
wonder
why
we
send
in
reports
at
all,

because
EPA
never
seems
to
have
them.

Evi
Huffer:
That
was
a
comment,
right?

Joe
Retzer:
Well
I
mean,
obviously
one
of
the
advantages
of
electronic
reporting
is
that
the
data
goes
into
EPA's
systems
in
close
to
real
time,
so
then
they
should
have
access
to
it
as
opposed
to
mailing
in
a
piece
of
paper,
and
it
may
or
may
not
get
into
the
database
at
some
point.

Evi
Huffer:
And
particularly
under
the
system
that
we're
envisioning
with
CDX,
we'll
be
archiving
a
copy
of
the
data
as
it
comes
in.

Joe
Retzer:
Right.
But
also
the
vast
majority,
however,

of
direct
reporting
now
is
actually
to
the
states
as
opposed
to
directly
to
EPA.
So
if
you're
talking
about
a
report
on
the
delegated
program,
but
an
EPA
inspector,
you
have
to
realize,

the
data
goes
to
the
state,
gets
processed
into
the
state
system,
and
then
gets
updated
into
the
EPA
system
on
some
periodic
basis,
which
may
be
quarterly
or
semi­
annually,
and
then
it
gets
into
the
EPA
system.
So,
obviously
sometimes
there's
some
lag.

[
Question
from
audience,
not
spoken
into
microphone.]

Joe
Retzer:
Then
they
should
have
it,
yes.
51
Don
Whaley:
These
are
reports,
I
was
thinking,
that
either
go
directly
to
the
regional
office
and
what
I
would
think
would
be
an
enhancement
from
the
electronic
reporting
would
be,
if
EPA
would
take
advantage
of
it,
the
accessibility
of
the
data
so
that
an
inspector
can
pull
whatever
reports
he
needs
prior
to
going
on
site
and
saving
us
from
going
through
and
digging
up
the
data
and
making
additional
copies
at
that
time.

Evi
Huffer:
It
should
certainly
help
to
improve
our
own
internal
business
processes
to
that
effect.

George
Britton:
George
Britton,
Aventis
CropScience.
Why
do
we
even
need
this?
If
we're
submitting
this
information
to
you
electronically,
and
you're
keeping
it
and
archiving
it,
why
should
we
even
bother
to
keep
it?
What
is
the
record
for
having
to
do
it,
other
than
with
any
reasonable
time
for
audit?

Evi
Huffer:
The
records
that
we're
most
concerned
about
under
Subpart
C
are
those
records,
those
documents,
that
are
not
sent
to
EPA
or
to
the
state.
They're
documents
that
are
maintained
solely
by
the
facility,
it's
information
that's
never
sent
to
us.

Joe
Retzer:
But
your
question
is
a
reasonable
one,
if
you
send
us
a
report
or
an
electronic
report,
why
do
you
have
to
keep
a
copy?
That's
a
good
question.
52
George
Britton:
The
other
side
of
that
is,
for
the
long
term,
for
the
information,
there
are
things
you
can
do
to
collect
information,
you
can
convert
it
into
a
storable
medium
that
is
more
stable
than
magnetic
kinds
of
things.
Keeping
functionality
for
a
few
years,
like
three
years,
like
the
IRS
does
for
normal
tax
returns,
unless
you're
under
investigation,

in
which
case
it
goes
to
seven.

Keeping
a
system
available
for
a
three­
year
period
after
you
quit
using
it
as
an
active
system
is
a
doable,
at
least
in
the
[
Good
Laboratory
Practices]
environment,
it
could
be
a
doable
thing.
But
keeping
it
for
the
FIFRA,
forty
years,
is
a
practical
impossibility.
When
was
the
last
time
you
read
Remington,
Ray,
and
UNIVAC
programs?
When
did
you
run
one?

The
FIFRA
community
has
had
projects
registered
since
1950.

So
therein
is
the
problem
with
long­
term
archiving.
If
you
moved
it
to
some
medium,
whether
you
captured
it
electronically
to
an
image
file,
there
are
several
ones
out
there,
then
you
have
a
chance
of
keeping
that
a
little
longer.

But
for
the
functionality
audit,
for
the
long­
term
retention,

it's
a
real
problem.

It
gets
back
to
the
same
idea
with
contract
labs
that
supply
the
sponsors
and
the
environmental
monitoring
data.

Sometimes
the
companies
that
are
contracted
for
the
work
don't
53
even
have,
or
couldn't
even
read
the
information
if
they
had
it.
The
summary
report
would
be
the
important
thing
that
is
kept
long
term,
things
that
you
sign
and
put
your
signature
on.

That's
what
you're
really,
ultimately,
certifying,
whatever
you
would
normally
put
a
signature
on,
the
equivalent,

electronically,
is
what
you
would
want
to
keep.
That's
what
you
would
fall
back
to
in
a
litigation
scenario.

Joe
Retzer:
Well
I
think
you're
raising
kind
of
another
issue
here,
which
is
are
the
current
record­
retention
requirements
appropriate?
What
should
be
kept
in
FIFRA
for
what
length
of
time,
what
should
be
kept
for
a
particular
water
regulation,
what's
the
appropriate
length
of
time?
Now
that's
not
really
an
issue
that
we
tried
to
address
in
CROMERRR.

Again,
we're
just
trying
to
build
on
what
the
underlying
records
are,
or
record­
retention
requirements
are,
that
folks
in
those
programs
have
come
up
with
over
time.

And
it
may
be
that
if
that
part
were
better
defined,
then
the
electronic
record
keeping
part
would
be
a
lot
easier
to
address.

George
Britton:
Agreed.

Kenna
Study:
I
was
also
going
to
ask
a
question
about
when
the
information
is
sent
to
the
EPA,
if
that
absolves
us
from
having
to
keep
it.
And
then
I
realized
it's
already
54
happened
a
number
of
times.
As
a
state
agency,
we
send
up
a
couple
hundred
thousand
record
per
quarter,
basically
from
the
drinking
water
systems,
and
we
do
get
into
back
and
forth
over
what
we
did
or
did
not
do.

I
was
going
to
say,
oh,
good,
we
don't
have
to
keep
it,

but
I
realized
that
we
would
keep
it
in
order
to
prove
that
we
did
give
you
something,
so
it's
probably
a
moot
point.

Joe
Retzer:
We
the
state,
you
mean,
might
conduct
an
inspection?
You're
speaking
on
behalf
of
the
state
of
Florida?

Kenna
Study:
Well,
for
my
program,
I
don't
know
enough
about
all
the
program
areas,
but
we
send
up
a
lot
of
data
every
quarter
to
the
EPA
database
that
comes
to
us
from
the
water
systems,
and
it
would
be
nice
to
say
oh
well
we
sent
you
that,

now
you
go
do
something
with
it
and
we'll
go
do
something
else.

But
I
think
the
reality
is
we'd
keep
it,
because
we
do
have
arguments
about
what
we
did
and
didn't
do,
so
that's
our
evidence,
I
guess.

We
keep
it
 
I
mean,
once
it's
in
the
database,
it's
possible.
I
was
looking
at
some
of
the
things
about
the
audit
trails,
we
have
a
user
time
stamp
associated
with
records,
but
I
can't
tell
you
what
that
person
did
except
in
instances
pertaining
precisely
to
compliance.
Like
if
they
deleted
a
violation,
if
our
user
did
that,
if
somebody
in
the
drinking
55
water
program
did
that,
we'd
have
a
record.
Or
if
somebody
in
the
drinking
water
program
went
into
the
data
system
and
changed
a
result
for,
let's
say
nitrate
has
a
standard
of
ten,

and
they
went
in
there
and
that
record,
which
has
got
maybe
five
pieces
of
meta
data
associated
with
it,
changed
it
from
.1
to
.2,
I
wouldn't
know
that.

I
would
know
the
day
they
went
in
there
and
who
went
in
there,
but
I
wouldn't
know
precisely
which
one
of
those
fields
had
been
altered
and
how.
So
I
wouldn't
know,
specifically,
on
your
number
eight
here,
how
far
you
intend
to
take
that.

Joe
Retzer:
Okay,
well
first
of
all,
it
doesn't
really
apply
to
states.

Kenna
Study:
Okay,
well
now,
now.
I
mean,
not
yet!

Well,
I
see
this
thing
filtering
up.

Joe
Retzer:
Right.
I
guess
the
distinction,
again,
is
the
relationship
between
states
and
EPA
isn't
a
regulatory
one,

it's
one
of
partnership
and
agreement
on
who's
going
to
do
what
in
a
broader
regulatory
scheme.

Kenna
Study:
Then
perhaps
as
a
delegated
program
for
EPA
you
could
let
us
deal
with
the
industry?

Greg
Nutt:
This
is
Greg
Nutt
from
TNRCC
again,
I
just
want
to
address
what
Joe
said
before.
He
talked
about
the
rosy
relationship
between
the
states
and
EPA
on
data
transfer
56
issues,
which
could
be
a
whole
other
day
of
discussion
at
least.

On
the
realisticness
[
sic]
of
a
records­
retention
schedule
in
the
program
regulations,
and
here's
a
little
experience
about
what
we've
gone
through
in
the
TNRCC
trying
to
establish
a
records­
retention
schedule
for
our
internal
records.
And
typically
what
happens
is
that
the
records
management
people
go
to
the
program
people
and
say,
"
How
long
do
we
need
to
retain
this?"
And
the
program
people
say,
"
Forever."
And
the
records
people
go
back,
and
they
say,
"
Okay,
here's
how
much
it's
going
to
cost."
And
then
the
management
in
the
program
areas
say,

"
You
know,
we
could
probably
get
by
with
just
two
or
three
years."

And
so
there's
a
problem
here
with
having
the
program
staff
in
EPA
realistically
address
 
let
me
say
there's
a
potential
to
have
a
problem
with
the
program
people
in
EPA
realistically
addressing
the
records­
retention
schedule,

because
they're
not
tied
to
the
costs
associated
with
that
record
retention.

Joe
Retzer:
Right.
Well,
it's
probably
something
that
deserves
real
attention.
Typically
the
way,
when
EPA's
regulations
are
done,
when
people
think
about
the
compliance
costs
and
they're
analyzed,
they
don't
initially
think
about
57
the
record
keeping
part
as
being
as
high
as
the
technology
part,
for
example,
of
the
compliance,
and
sometimes
probably
those
things
like
record
retention
don't
get
the
same
amount
of
attention
 
except
in
a
rule
like
this
 
as
some
other
parts
of
the
rule
might.

And
many
of
those
rules
go
back
many,
many
years
and
are
really
based
on,
again,
thinking
of
things
in
terms
of
paper
retention,
so
that's
really
where
I
rate
that
potential
issue.

Chris
Marlia:
Chris
Marlia
from
the
South
Coast
again.
I
know
you
mentioned
that
the
states
don't
come
under
this
CROMERRR
program
in
terms
of
record
keeping,
however
the
states
do
come
under
the
provisions
of
having
to
have
a
recordsreceiving
system,
which
is
pretty
onerous.
We're
going
to
have
to
duplicate
what
CDX
does
before
we
report
to
EPA.
I
just
wanted
to
make
that
comment.

Joe
Retzer:
Okay,
let's
save
that
to
the
end
and
we
can
talk
about
that
a
little
bit.
For
those
who
are
interested
in
that,
they
can
stay
and
talk.
I'm
not
sure
everyone
is,
since
we
sort
of
billed
this
as
being
about
records.

Robin
Roberts:
We
might
be
at
that
part
where
we
talk
about
that.

Joe
Retzer:
Yeah,
we
might
be
at
the
end.

Evi
Huffer:
Are
we
going
to
move
on
or?
58
Robin
Roberts:
Oh,
we
have
a
comment
here.

Unidentified
Female
Speaker:
When
it
comes
down
to
the
final
version,
I'd
like
to
see
the
rule
say
something
about
who
keeps
electronic
files
when
something
is
contracted
out.
If,

as
a
sponsor
company,
I'm
required
to
keep
all
electronic
files
for
everyone
I've
contracted
with,
it
could
be
a
lot
of
people.

I
have
twenty
field
cooperators
out
there
growing
field
corn
for
me
who
may
be
generating
electronic
weather
data
or
generating
freezer
data
on
electronic
systems,
or
whatever
other
electronic
data
they
may
generate.
Then
I
have
a
contract
analytical
lab
who's
generating
all
sorts
of
electronic
data,
none
of
which
I've
ever
seen.
At
this
point
we
accept
paper
from
them
as
our
standard.

I
would
like
the
rule
to
define
whether
or
not
I
will
also
be
responsible
for
maintaining
electronic
files
for
this
contract
work.
If
I'm
required
to
maintain
electronic
files,

then
I'll
be
required
to
buy
the
software
to
look
at
them
with.

If
I
contract
out
a
C14
study,
obviously
I
don't
have
that
kind
of
software
since
I'm
not
registered
to
use
radio­
labeled
compounds.
So
I
would
have
to
go
purchase
the
software
that
they
use
to
pull
their
electronic
files
up
on
my
computer.
The
same
with
weather
systems
or
any
other
kind
of
electronic
collection
systems
that
we
don't
have
at
our
facility.
59
If
you
start
thinking
about
that,
there
could
be
a
horrendous
cost
of
being
able
to
look
at
electronic
files
in
our
facility
for
equipment
we
don't
have.
I
don't
want
to
have
to
go
and
buy
software
for
any
kind
of
chromatographic
system
that
we
don't
have
in
our
building.
So
I
think
the
rule
needs
to
define
how
much
electronic
files
we
have
to
keep
from
contracted
people.

Robin
Roberts:
And
that
came
up
earlier
also,
about
the
relationship
that
many
of
you
have
with
contractors
and
what
your
responsibility
is
to
be
able
to
read
the
data
that
they
collect
for
you
ten
years
afterwards.

Mel
Swoboda:
This
is
Mel
Swoboda
again.
One
of
the
issues,
I
guess,
with
long­
term
archiving
that
makes
it
partly
a
problem
too
is
if
you
look
at
a
lot
of
the
regulations,
and
we
mentioned
one
where
you're
monitoring
every
second,
or
you
have
a
record
data
point
every
second,
every
unit
or
something
like
that.

And
I
if
I
go
back
and
look
at
some
of
the
regulations,
if
I
look
at
the
water
regulations,
continuous
monitoring,
well
when
they
wrote
continuous
monitoring,
they
were
thinking
of
a
chart
with
a
pen
on
it
and
that
was
all
they
were
looking
at,

and
not
the
intensity
with
which
a
computer
can
go
out
and
pick
up
data.
60
One
of
the
pieces
that
I
could
see
that
you
could
have
a
lot
of
value
for
is
looking
hard
at,
from
a
historical
archiving,
the
filtering
capability,
or
what
you
do
to
set
some
standards
around
what
kind
of
filtering,
so
that
if
I
have
a
data
point,
or
I
have
a
line
of
information
that
may
move,

maybe
a
tenth
over
a
long
period
of
time,
is
that
really
important?
Should
I
be
picking
up
a
data
point,
every
one
of
those
data
points?
Should
I
be
picking
up
a
data
point
every
hour,
if
there's
no
change
in
those
from
an
archival
standpoint?
So
that
you
really
don't
try
and
get
this
massive
dump
of
data
on
you
and
you
get
something
that
really
shows
what's
happening
over
a
period
of
time.

That,
I
think,
is
something
that
would
be
of
value
for
CROMERRR
to
come
in
with,
with
some
thoughts
and
some
processes
from
the
people
who
actually
do
that
kind
of
storing
of
data,

on
what
the
capabilities
are,
what's
the
best
way
to
do
it,
and
then
set
standards
that
are
reasonable.

Everybody
with
computers
now,
numbers
are
absolute,
if
it
reads
out
to
seven
or
eight
digits,
it's
an
absolute
number.

The
number
may
not
be
good
but
to
one
place.
But
we
keep
looking
at
these
numbers
as
if
they're
the
absolute
Gospel.

There's
something
to
be
said
for
the
old
slide
rules .

[
End
side
B,
end
of
transcript.]
61