0950
 1                      UNITED STATES DISTRICT COURT
                        FOR THE DISTRICT OF COLUMBIA
 2   
 3    ELOUISE PEPION COBELL,       :  Civil Action 96-1285
      et al.                       :
 4                  Plaintiffs     :
                                   :  Washington, D.C.
 5            V.                   :  Wednesday, October 17, 2007
                                   :
 6    DIRK KEMPTHORNE, Secretary   :
      of the Interior, et al.      :
 7                                 :
                    Defendants     :  MORNING SESSION
 8   
                    TRANSCRIPT OF EVIDENTIARY HEARING
 9                                DAY 5
                   BEFORE THE HONORABLE JAMES ROBERTSON
10                      UNITED STATES DISTRICT JUDGE
11   APPEARANCES:
12   For the Plaintiffs:      DENNIS GINGOLD, ESQUIRE
                              LAW OFFICES OF DENNIS GINGOLD
13                            607 14th Street, NW
                              Ninth Floor
14                            Washington, DC 20005
                              (202) 824-1448
15   
                              ELLIOTT H. LEVITAS, ESQUIRE
16                            WILLIAM E. DORRIS, ESQUIRE
                              KILPATRICK STOCKTON, L.L.P.
17                            1100 Peachtree Street
                              Suite 2800
18                            Atlanta, Georgia  30309-4530
                              (404) 815-6450
19   
                              KEITH HARPER, ESQUIRE
20                            JUSTIN GUILDER, ESQUIRE
                              KILPATRICK STOCKTON, L.L.P.
21                            607 14th Street, N.W.
                              Suite 900
22                            Washington, D.C.  20005
                              (202) 585-0053
23   
                              DAVID C. SMITH, ESQUIRE
24                            KILPATRICK STOCKTON, L.L.P.
                              1001 West Fourth Street
25                            Winston-Salem, North Carolina  27101
                              (336)  607-7392
0951
 1   For the Defendants:      ROBERT E. KIRSCHMAN, JR., ESQUIRE
                              JOHN WARSHAWSKY, ESQUIRE
 2                            MICHAEL QUINN, ESQUIRE
                              J. CHRISTOPHER KOHN, ESQUIRE
 3                            U.S. Department of Justice
                              1100 L Street, N.W.
 4                            Washington, D.C.  20005
                              (202) 307-0010
 5   
                              JOHN STEMPLEWICZ, ESQUIRE
 6                            Senior Trial Attorney
 7                            U.S. Department of Justice
 8                            Commercial Litigation Branch
 9                            Civil Division
10                            Ben Franklin Station
11                            P.O. Box 975
12                            Washington, D.C.  20044
13                            (202) 307-1104
14   
15   
16   Court Reporter:          REBECCA STONESTREET
17                            Official Court Reporter
18                            Room 6511, U.S. Courthouse
19                            333 Constitution Avenue, N.W.
20                            Washington, D.C.  20001
21                            (202) 354-3249
22   
23   
24   Proceedings reported by machine shorthand, transcript produced
25   by computer-aided transcription.
0952
 1   
 2   
 3   
 4   
 5   
 6                           C O N T E N T S
 7   
 8   WITNESS                  DIRECT    CROSS     REDIRECT  RECROSS
 9   
10   DR. FRITZ SCHEUREN
11   By Mr. Warshawsky          960       --        --          --
12   By Mr. Gingold             --        1026      --          --
13   
14   
15                            E X H I B I T S
16   
17       NUMBER                                   ADMITTED
18   
19   
20       DEFENSE:
21   
22         6 - 7                                    1026
0953
 1                        P R O C E E D I N G S
 2            THE COURT:  Good morning, everybody.  Mr. Kirschman?
 3            MR. KIRSCHMAN:  Good morning, Your Honor.  Before we
 4   continue with the examination of our witness, I would like to
 5   bring two matters to your attention if I could, please.
 6            The first is a scheduling issue related to the
 7   presentation of our responsive witnesses, those witnesses who
 8   have prepared reports in response to plaintiffs' experts.
 9            This is a matter that, until about 9:30 last night, the
10   parties appeared to be in agreement on.  But there's a
11   divergence now, and because of the need to clarify the schedule,
12   we thought we should bring this to the Court's attention.
13            We have a total of six responsive reports.  Our
14   presentation of most of our responsive witnesses was, we
15   anticipated, going to be after you actually first heard the
16   reports and the testimony they were responding to, so that you
17   would have context for what they were saying, and that they
18   would be responding to what actually was entered into the
19   record.  That includes specifically Caren Dunne,
20   Professor Langbein, and Dr. Hinkins.  We thought -- and we think
21   that is the orderly way to progress, so that you're not hearing
22   testimony from our witnesses that is speculative and based on
23   what they assume they will hear.
24            There's also a question as to whether, for example,
25   plaintiffs -- certain of plaintiffs' experts will testify, or
0954
 1   whether you will favorably entertain motions we have regarding
 2   their expertise.  And there may be things that our witnesses
 3   would testify that plaintiffs would end up not raising once they
 4   call their witnesses.
 5            So for the efficient, orderly presentation of the
 6   issues being addressed, we had assumed that our responsive
 7   witnesses would proceed after you heard what was first said by
 8   plaintiffs.
 9            As of yesterday, plaintiffs in fact had told us
10   regarding Dr. Scheuren, they would oppose the presentation of
11   our responsive witnesses in our case-in-chief.  The issue had
12   come up because we had suggested that Dr. Scheuren should only
13   appear once.  The issue he's responding to, the meta-analysis,
14   is a fairly narrow issue, and we had suggested that he just
15   address that while he's on the stand today.
16            So that was how the issue surfaced.  And we seemed to
17   be in agreement that, for the other responsive witnesses, they
18   would only testify after plaintiffs had a chance to present
19   their case.  That changed, I got a phone call last night about
20   9:30.
21            But obviously the parties need to know -- we feel
22   strongly that it only makes sense that our witnesses actually
23   get to respond to something you've heard.  But resources will
24   have to be diverted if there's now something different afoot.
25            THE COURT:  All right.  Well, like all trial questions,
0955
 1   I find that a little bit abstract.  Maybe I can hear from the
 2   plaintiffs, and hear how that relates to individual witnesses.
 3            MR. KIRSCHMAN:  Your Honor, could I give you one
 4   example, to be more concrete?  And certainly Mr. Harper can add
 5   his.
 6            THE COURT:  Okay.
 7            MR. KIRSCHMAN:  One of plaintiffs' expert witnesses,
 8   for example, is Dwight Duncan, who has challenged the
 9   statistical sampling employed by the Department of the Interior
10   through NORC.  In their pretrial statement, plaintiffs indicated
11   that Dwight Duncan would testify for up to 12 hours on direct.
12            THE COURT:  And I said nobody is going to testify for
13   12 hours.
14            MR. KIRSCHMAN:  We anticipate that's not going to
15   happen.  Nevertheless, we also anticipated that there would be a
16   fair amount of testimony coming from Dwight Duncan on these
17   issues.  And Susan Hinkins, Dr. Susan Hinkins from NORC, has
18   prepared a detailed response to Mr. Duncan's detailed report.
19   So for you to hear Dr. Hinkins give a detailed response
20   before --
21            THE COURT:  I hear that.  I hear that.  Let me hear
22   what the plaintiffs have to say about it.
23            MR. DORRIS:  Your Honor, good morning.
24            THE COURT:  Good morning.
25            MR. DORRIS:  You'll recall some of this came up where
0956
 1   there was a question about whether these experts were responding
 2   experts or rebuttal experts.  And there was prior argument, and
 3   the government took the position that they were responsive
 4   experts and not rebuttal experts.
 5            What Mr. Kirschman says, the issue had not been
 6   discussed between the two parties as to whether or not these
 7   other experts should be testifying or not.  We called him back
 8   at 9:30 last night after he had asked us the question in an
 9   earlier call in the evening.  That's the reason why that call
10   was made, where the parties first discussed this overall issue.
11            Now, with respect to their experts.  For example
12   Mr. Scheuren, I'm convinced, today is going to be talking about
13   Mr. Duncan and what he says.  That happens in every case, where
14   an expert that's called in the whoever-goes-first case is
15   talking about their opinion and why it should be accepted as
16   opposed to the other expert's opinion.
17            And that's exactly what should happen here.  They
18   should be put in a position to put on their entire case.  If
19   that includes experts that they want to call that are going to
20   give testimony that is different than our experts, that's what
21   they ought to do in moving forward.  That's our position,
22   Your Honor.
23            THE COURT:  Well, I don't know how this relates to
24   other witnesses, but as it relates to Duncan and Hinkins,
25   accepting the representation of Mr. Kirschman as accurate and in
0957
 1   good faith that Hinkins' testimony is nothing but a response to
 2   Duncan, then Duncan goes first.  Okay?
 3            Now, I don't know how that rule applies to other
 4   witnesses, but that's my ruling on that one.
 5            MR. KIRSCHMAN:  Your Honor, would you like me to raise
 6   the other two witnesses where this most aptly applies?
 7            THE COURT:  Might as well.  Let's hear about them.
 8            MR. KIRSCHMAN:  Caren Dunne, Your Honor, is a CPA with
 9   Clifton Gunderson.  By the way, all of these witnesses that I'm
10   mentioning we had identified as may-call, largely because of
11   this, depending on what we hear when plaintiffs put on their
12   case.
13            But Caren Dunne was listed as a may-call, and she will
14   be responding to the report prepared by Don Pallais,
15   P-A-L-L-A-I-S, regarding the Accounting Standards Manual and
16   issues related to that.  Her report is detailed and tracks his
17   opinions -- he issues an opinion; she responds in kind, related
18   to the Accounting Standards Manual.  So to hear her response to
19   Mr. Pallais' positions on the use of the ASM just again wouldn't
20   be logical and wouldn't be efficient.
21            Plaintiffs had suggested if that one of our witnesses
22   like Ms. Dunne heard something new, we could call her back a
23   second time.  And certainly, that wouldn't make sense.  So
24   Caren Dunne is another one specifically.
25            A third is Professor Langbein, who has offered a
0958
 1   responsive expert report on the standards governing Trust
 2   accounting.  Professor Langbein is responding to Robert Vaughn,
 3   who provided a report for plaintiffs on the standards of Trust
 4   accounting.
 5            This is particularly notable because Mr. Vaughn cited
 6   his experience as the basis for his opinions on the standards.
 7   We don't know what those experiences are, and until we actually
 8   hear him testify, Professor Langbein will be limited in how he
 9   can respond to that.
10            THE COURT:  Okay.  Counsel, I don't want to spend the
11   morning on this.  I want to think about it a little bit.  We are
12   in some sense, all of us, making this up as we go along, because
13   I don't know that there's ever been a trial quite like this one
14   before.
15            But as I see it, in the broadest outline, the reason
16   the government is going first -- even though it is ultimately
17   the plaintiffs who have the burden of proof in this case on most
18   concepts, the government is going first because the government's
19   case is:  Here's what we're doing.  The plaintiffs are going to
20   say:  That's not good enough.  And the government is going to
21   respond with its responsive witnesses.
22            That suggests to me that the ordinary and logical
23   course of this trial is for the government to put on the rest of
24   its here's-what-we're-doing witnesses; for the plaintiffs to
25   then attack it, as they will be doing with expert witnesses; and
0959
 1   for the government then to respond with their experts.
 2            So it sounds to me -- but I'm not ruling on this, I
 3   just want to think about it.  It sounds to me like Dunne and
 4   Langbein come after the witnesses to whom they're responding.
 5   But let's think about it, and we may talk about it a little bit
 6   more.  We don't have to decide that this morning.  I want to get
 7   a witness on the stand and get moving.
 8            MR. KIRSCHMAN:  Your Honor, I'm sorry.  I have one more
 9   matter to bring to your attention.
10            THE COURT:  Quickly.
11            MR. KIRSCHMAN:  It will be quick, Your Honor.  We're
12   moving orally for leave to present a new defendant's exhibit.
13   It is marked as DX-365.  It is a demonstrative exhibit that has
14   been prepared by Dr. Abe Haspel, with the assistance of
15   contractors.  The document was just completed late yesterday,
16   early evening.  It is a demonstrative that will assist the Court
17   in visualizing what Dr. Haspel will be testifying to regarding
18   the relationship between throughput and its different
19   variations, and the historical accounting work.
20            There is information in this document that is already
21   contained in other documents, other defendants' trial
22   exhibits --
23            THE COURT:  When is Haspel going to testify?
24            MR. KIRSCHMAN:  The order now, Your Honor, that we have
25   provided to the plaintiffs is that Dr. Scheuren will testify
0960
 1   today, he will be followed by Dr. Lasater, and then Dr. Haspel
 2   will testify.
 3            We anticipate therefore he will probably be testifying
 4   tomorrow morning.  That's a rough approximation, depending on
 5   the length of the direct and the cross of the two witnesses who
 6   precede him.
 7            THE COURT:  All right.  Well, he can use the
 8   demonstrative.  Whether it comes into evidence remains to be
 9   seen, but he can use the demonstrative.
10            MR. KIRSCHMAN:  Thank you, Your Honor:
11            THE COURT:  Let's get Dr. Scheuren on the stand,
12   please.
13            MR. WARSHAWSKY:  Good morning, Your Honor.
14            THE COURT:  Good morning, Mr. Warshawsky.
15                    CONTINUED DIRECT EXAMINATION
16   BY MR. WARSHAWSKY:
17   Q.  Good morning, Dr. Scheuren.
18   A.  Hi.
19   Q.  Dr. Scheuren, I went to move into the specific facts
20   underlying your various recommendations in this matter.  But one
21   last point I do want to tie up very quickly.
22            When we were talking yesterday about precision and
23   confidence, or margin of error or assurance levels, whatever,
24   and we talked about sample size, is there a linear relationship
25   between sample size and precision or confidence; in other words,
0961
 1   twice as big a sample means twice as much confidence, twice as
 2   precise?
 3   A.  Unfortunately for clients who have to spend the money, it's
 4   not linear.  It's quadratic.  If you want to narrow the
 5   confidence interval, say shorten it by half, you have to have a
 6   sample that's four times as large.
 7   Q.  And at some point, does increasing the size simply not make
 8   a difference?
 9   A.  In the context of the client's needs for precision, I would
10   say yes, it does.
11   Q.  Referring you again to your expert report, Defendant's
12   Exhibit 6 we have up on the screen here table one on page five.
13   Can you just generally explain what is presented in this table?
14   A.  This is a table that looks at sample size.  If you have a
15   world, hypothesized world of a one percent error rate, and you
16   want a margin of error around that, very narrow margin of error
17   around that -- excuse me.  It's assuming no errors in the
18   observations, and a margin of error of one percent.
19            And there are three columns.  The first column is
20   population size, and the second column is a 95 percent
21   confidence interval, and 99 percent confidence interval.  These
22   are the ones that we've been talking about yesterday.
23            And what you're seeing in the table is, if you have --
24   if you are at a population size of 500, the sample size that you
25   need to achieve this is 224, which is a very large fraction of
0962
 1   that total population.
 2            As you move up the table, or actually down the table,
 3   the sample size grows, but very slowly.  Very slowly.  And the
 4   fraction that it becomes of the population drops dramatically.
 5   That's the point of the table.
 6   Q.  And in fact, for a population 100,000 to 10 million, there's
 7   no change in the sample size required.  Is that correct?
 8   A.  That's correct.  Rounds to that, right.
 9   Q.  Dr. Scheuren, I would like you to refer now to what's been
10   marked, it's in the administrative record, AR-402.  The Bates
11   numbers on that is 00049-6.  I have the cover page on the
12   screen.
13            Now, Dr. Scheuren, would you identify what this
14   document is, please?
15   A.  This is a report that we produced for the Office of
16   Historical Trust Accounting on our sample design plans for the
17   2003.  This would have been the earlier accounting plan.
18   Q.  You're referring to the plan that was submitted to the
19   District Court in 2003?
20   A.  Correct.
21   Q.  And you were one of the authors of this report?
22   A.  Yes, I was.
23   Q.  Referring now to page three of this document, which is on
24   the screen -- yes?
25   A.  I need to have the point size come up a little bit.  That's
0963
 1   why I had a problem yesterday.
 2   Q.  And as I said, Dr. Scheuren, we don't have your notebook of
 3   hard copies in front of you.  If that would assist you, we can
 4   give you that as well.
 5   A.  Let's do this point size.
 6   Q.  Okay.  Is that big enough?
 7   A.  Absolutely.
 8   Q.  Dr. Scheuren, there's a section captioned "General
 9   Approach."  What was the purpose of including this in your
10   report?
11   A.  Well, people like me get in the weeds a little too much.
12   Maybe I did that yesterday, even.  This is something that was to
13   give the senior people in the government access to what our
14   general ideas were.
15   Q.  And the first two bullet points on the screen here propose
16   sampling transactions under 5,000, and propose that all
17   transactions over 5,000 be directly reconciled.  Is that
18   stratification?
19   A.  Yes, that's the -- 5,000 is a boundary, a stratification
20   boundary.
21   Q.  Why did you propose different strata for your sampling
22   model?
23   A.  Well, we had concerns here at the beginning of this about
24   whether we were going to be able to find all the records.  And
25   the larger amounts - okay, 5,000 is a reasonably large amount -
0964
 1   cause considerable concerns if you can't find the records.
 2   Q.  Why is that?
 3   A.  The reason for that is, based on my background, when I was
 4   at the IRS, when you have a situation where you have
 5   controversies of the kind that we are seeing here, if you can't
 6   find the document, you can't -- if you can't support the
 7   document, then it's treated as an error.
 8            And indeed, that is what we have done in our work here.
 9   When we couldn't find the document, when the team of people that
10   did the work, including us, were unable to find the documents in
11   the litigation support work that we did, all of those were
12   treated as errors.  And that was a concern.
13   Q.  And you're referring to the statistical analysis of those
14   items?
15   A.  That's correct.
16   Q.  In fact, if you couldn't find support, does that mean that
17   the transaction was erroneous?
18   A.  No.
19   Q.  Why?
20   A.  But there is no evidence that it is accurate.  And that
21   worst case approach was what I've seen done in other settings;
22   I've mentioned the IRS, and it's what we did here.
23   Q.  Let me ask you something.  Yesterday, I'm not sure if you
24   were in the courtroom when Judge Robertson asked this, when
25   Ms. Herman was testifying about the reconciliation efforts for a
0965
 1   $0.58 item, and the estimate was that it cost several thousand
 2   dollars to reconcile that item, why would you go to the effort
 3   of reconciling a $0.58 item?
 4   A.  Well, you have to believe that the Office of Historical
 5   Trust Accounting was not too pleased with that, when they came
 6   back that it cost so much money.
 7            The reason we did this is because, how do we know it
 8   was $0.58?  What if it was $50,000 but had been mis-keyed as
 9   $0.50?
10            So we did look at the very small items, and an awful
11   lot of very small transactions, especially credits, and we
12   looked at them to see if in fact they were understated.
13            We almost found none of them understated.  I think
14   there were two that were slightly understated.  Maybe they were
15   $0.50 above or something like that, a dollar above.  If I may
16   continue with this, since it is relevant?
17   Q.  Please.
18   A.  When we Went back after we did the first round of the
19   litigation support work, we cut the sample size for the very
20   small items because we weren't learning anything.  We didn't cut
21   it to zero, because there was a chance the ones we hadn't found
22   were different than the ones we had.  But we didn't cut it to
23   zero, but we cut it down a way lot, to save some of this money.
24            I mentioned yesterday that one of the steps, this
25   four-step process, is the redesign step:  You plan it, you carry
0966
 1   it out, you make inferences from it, and then you look at what
 2   you've done and you redesign it.
 3            And that's what we did here, and we reduced the work
 4   and reduced the money.  We weren't learning anything.  We
 5   weren't sure we were right, but we were sure enough that we
 6   could cut the sample size.  And we did that dramatically.
 7   Q.  But how does a $0.58 item get selected in the first place?
 8   A.  Because it's part of the population.  Every item in the
 9   population, every transaction in the population that we were
10   given, had a positive probability of being selected.
11   Q.  Now, in your second bullet point on the screen, you have a
12   sample statement in quotes, "With 99 percent confidence, more
13   than 99 percent of all transactions up to $5,000 are accurate."
14            What does that mean, that kind of a statement?
15   A.  Well, it is a decision -- the client makes this decision,
16   but we offered it to them, that they want to be that accurate.
17   It's a very important case.  It's appropriate, to give respect
18   to all of the parties, that we have a high degree of accuracy.
19   And this was what we proposed.  And --
20   Q.  Go ahead.  I'm sorry.  I didn't mean to cut you off.
21   A.  Remember our confidence interval idea of yesterday.  Okay?
22   We have here an example of the application.  99 percent
23   confidence, that's a confidence interval.  In fact, it's a
24   one-sided confidence interval, because we're only looking at one
25   kind of issue here.
0967
 1            And what this means is that the error rate -- to put it
 2   in a negative way, the error rate is one percent or less.
 3   Q.  What do you mean by one-sided confidence level?
 4   A.  Well, when I drew the picture yesterday, I had a possibility
 5   of being -- of making a mistake on both sides.  That possibility
 6   to make a mistake on both sides always exists, of course, if one
 7   of the sides isn't zero, but the interest here in view of the
 8   case is really a one-sided interest.
 9            You heard me mention that the way we're dealing with
10   errors is to look -- I guess we haven't talked about this yet.
11   We're looking at the issues in terms of the way the plaintiffs
12   would look at them.  Mistakes are made in all directions, in
13   large activities.  This is normal human behavior.  There are
14   only certain kinds of errors are really of concern to the
15   plaintiffs; those in fact which the plaintiffs are
16   disadvantaged.  And so that's what we were focusing on.
17   Q.  Is 99 percent -- let me try that again.
18            99 percent confidence level, is that standard for this
19   type of analysis?
20   A.  Actually, it's extreme.
21   Q.  In what way?
22   A.  In most business settings, I do a lot of work in business
23   settings, a 95 percent standard would have been used.  But
24   because of the nature of this case, the importance of it, I
25   recommended a 99 percent, which means that the sample sizes are
0968
 1   considerably larger as a result of that.
 2   Q.  And why don't we go ahead and refer to page six now of this
 3   exhibit, AR-402.  There's a table at the top which I have on the
 4   screen.  Would you explain what was depicted in that table
 5   within your report?
 6   A.  This is a table, again with three columns.  One of them is a
 7   column which -- the first column which looks at different
 8   classifications, different sizes of transactions; the second one
 9   are the total number of such transactions in the population that
10   we were sampling; and the third is the sample size that at this
11   time we recommended, very large sample size.
12   Q.  Okay.  And then referring back to -- we'll leave the table
13   up here on the screen.  But earlier when we were talking about
14   your bullet points on the first page, you talked about a
15   certainty stratum.  I think that's sometimes used as a term.  Is
16   that correct?
17   A.  Or take-all.  If you look at the 5,000-plus row here, you
18   see that the number of transactions in the population, which is
19   in the second column, and the number in the third column, are
20   the same.  We were taking them all.
21   Q.  That's not a sample, is it?
22   A.  I would argue it's a sample, but you could argue it's not a
23   sample.
24   Q.  Well, you're the expert.
25            In any event --
0969
 1   A.  It's a 100 percent sample.
 2   Q.  Very good, sir.  For the other two strata, though, under 500
 3   and 500 to 5,000, you were proposing sampling for those.  Right?
 4   A.  That's correct.
 5   Q.  And a total of 160,000 from those two strata?
 6   A.  That's correct, as well.
 7   Q.  And just to be clear, that was in the electronic ledger era?
 8   A.  Yes.
 9   Q.  Is that a large sample size for that population?
10   A.  Yes, extremely large.
11   Q.  Why did you recommend such a large sample size?
12   A.  We had carefully examined what people thought was wrong with
13   the system.  The possibility of local problems in some agencies
14   led us to have -- ask for a reasonable sample size at the agency
15   level in order to detect that, and, if it was there, to
16   estimate, estimate that error rate with some precision.
17   Q.  Well, elaborate on that a little bit.  What kinds of things
18   were you looking at when you said you considered what people
19   thought was wrong with the system?
20   A.  Well, there was information that we had seen - and a lot of
21   it is conversational so I don't have documentation for this -
22   that particular problems existed in particular agencies, okay,
23   in some various points during the historical period that we're
24   looking at here.  We're looking at from '85 to 2000.  And that
25   led us to want to seek to know that well enough to be able to
0970
 1   detect it.
 2            And indeed, this isn't the complete sample size here.
 3   We're recommending an adaptive approach.  And in that world, we
 4   were recommending adapting up in those situations where there
 5   was some softness in the data, there were some problems with the
 6   work that had been done historically, that we would actually
 7   increase the sample size even more.
 8   Q.  Well, what kinds of problems were you talking about, though?
 9   A.  We're talking about mistakes in the accounting.  That's the
10   focus of this engagement.
11   Q.  Did you consider the potential for missing data?
12   A.  Indeed, that's one of the factors here.  That dominated our
13   concern about the 5,000-plus, the 100 percent strata.  The
14   missing data problem also was a concern everywhere.  How we
15   would handle the missing data problem was something we hadn't
16   fully faced here at this point.  But we were expecting to
17   encounter a lot of it, and that was why we had such large
18   samples, one of the reasons we had such large samples.
19   Q.  So yesterday when we talked about provisional sample sizes,
20   do you remember that, sir?
21   A.  Yes, I do.
22   Q.  Is that basically what we're talking about?
23   A.  I would consider this a provisional sample size, right.  We
24   had not done -- my apology.  I interrupted you.
25   Q.  No, you didn't.
0971
 1   A.  I have already apologized to the Court Reporter, and she
 2   told me that she would punish me later depending on how often I
 3   did it.
 4            We had not actually got into the work at this point.
 5   We had not been able to do any significant amount of pilot work,
 6   for many reasons.  So this was very much a provisional sample
 7   size.
 8   Q.  So missing data, problems at the agencies, those were
 9   hypotheses?
10   A.  Correct.  Concerns --
11   Q.  And -- go ahead.
12   A.  Concerns about high error rates.  Plaintiffs thought the
13   error rates were low -- excuse me.  The government thought the
14   error rates were low, the plaintiffs thought they were high.  We
15   had to balance those considerations in designing the sample.
16   Q.  Did NORC actually draw sample transactions using this sample
17   design?
18   A.  We did for Alaska.
19   Q.  And describe the process where you drew the sample.
20   A.  In the Alaska case, which was unusual in that Alaska was
21   not -- you know, not very active until very late in the history
22   of the Indian Trust, all of the transactions were available in
23   one place in Alaska.  And we simply scanned all of them.  All of
24   them were scanned.
25            The work was then divided up in equal parts among the
0972
 1   accounting firms - there were four accounting firms - in order
 2   to see if the accounting firms were doing the work the same way.
 3   And the work was never completed because there was a hold on the
 4   work.  But the design was done.
 5   Q.  What do you mean by a hold on the work?
 6   A.  We were not allowed to -- the work could not be finished for
 7   reasons that had to do with, I believe the combination of a
 8   Court concern, opposed by the Court, and also later on a
 9   Congressional concern about funding.
10   Q.  I see.  What population, by the way, were you designing for
11   on this?  In other words, what types of accounts were included
12   in your population?
13   A.  We were focusing on land-based accounts here.  Judgment and
14   per cap are all being done, at least up to this point, on
15   100 percent basis.  A good nature of the population makes that
16   the appropriate thing to do.  But these were land-based
17   accounts.
18   Q.  And was it every account that ever existed in the IIM Trust?
19   A.  Every one that was in the electronic era, called the
20   electronic ledger era, yes.  In Alaska.
21   Q.  Was that as of --
22   A.  Well, in 1985.  Of course, Alaska didn't start until '87,
23   but right.
24   Q.  There's been some discussion here -- and just to be clear,
25   the accounts that you sampled from, were they accounts that
0973
 1   existed as of October 1994?
 2   A.  Yeah, they were on or after October 1994.  That was the
 3   population that the client defined for us to sample from.
 4   Q.  So just to be clear, if an account ceased to exist prior to
 5   October 1994, you didn't sample from it?
 6   A.  It was not in the population eligible for selection.
 7   Q.  Now, you mentioned that there was a hold on the 2003 plan
 8   sampling.  Right?
 9   A.  Yes, I did.
10   Q.  What was the next thing that you did with regard to
11   sampling?
12   A.  Well, we did do some analysis of what we had obtained from
13   Alaska, of course.  The next important activity was the
14   litigation support activity that we engaged in later that year.
15   Q.  And that's sometimes called the LSA project?
16   A.  That's correct.  The acronyms are...
17   Q.  How did you first learn about the LSA project?
18   A.  Well, I think Abe Haspel went and talked to Jeff Zippin and
19   Bert Edwards about it.  And it was something that we were asked
20   to do, and all of us got involved rather readily.  It was in the
21   fall of that year.
22   Q.  And just if you would identify who these individuals were at
23   the time.
24   A.  Abe Haspel is in the role he has now.
25   Q.  And that is?  He hasn't testified yet.
0974
 1   A.  Well, he's an undersecretary, I don't know what quite the
 2   title is, at Interior.  I think Cason -- he reports to Cason,
 3   but he's right at that level, next level down.
 4   Q.  Got you.
 5   A.  And of course Bert Edwards is the director of the Office of
 6   Historical Trust Accounting, and Jeff Zippin is his deputy.
 7   Q.  What did you do next with regard to the LSA?
 8   A.  Well, in October/November we were talking about this, and
 9   then it was something that was going to happen right away.
10   Q.  And I'm sorry, just to interrupt.  What year are you talking
11   about?
12   A.  This is, I think 2003.  Yes, fall of 2003.
13   Q.  Okay.  All right, sir.
14   A.  So we were asked to draw a sample.  We recommended a sample
15   of 1,000 accounts, with an average of 20 transactions per
16   account.  That's a pretty large sample also.  That was in
17   keeping with our earlier thinking, but it was...
18            Our focus was different now.  Our focus was on trying
19   to get an overall understanding of what was happening with the
20   system, not the kind of -- not necessarily the kind of estimates
21   that could be used for an historical accounting.  If I might
22   continue.
23            If the results were very favorable, very low error
24   rates, then they would have a role in the historical accounting.
25   If they were what we expected to see, okay, then much more would
0975
 1   have been needed.
 2   Q.  When you say what you expected to see --
 3   A.  We expected to find a higher error rate than we found.
 4   Q.  I would like to have you now look at, from the
 5   administrative record, AR-438, which we'll have put up on the
 6   screen.  The Bates number is 000040-2.
 7            Dr. Scheuren, what is on this particular document from
 8   the administrative record?
 9   A.  These are the --
10   Q.  Do you need this blown up again?
11   A.  Yes.  Not the first page, but when we get to something else.
12   Don't pick on me because I'm older than you are.
13            These are our results, one of the results reports from
14   that effort.
15   Q.  Okay.  And this is a report dated September 30, 2005?
16   A.  That's correct.
17   Q.  Who prepared this report?
18   A.  Well, I'm one of the authors, I think, and a number of other
19   people were involved.
20   Q.  Who are these other individuals?  Dr. Hinkins, we've heard
21   about.
22   A.  Robin Lee.  She's actually here in the courtroom, and
23   she's -- she and Susan and I have been the central people in
24   this engagement.
25            We have different roles, and you'll learn that as time
0976
 1   goes on.  But when anything is decided, we all end up agreeing.
 2   If one of us disagrees, we end up keep working it out until we
 3   all agree.  That's the basic approach we've taken.
 4            I think it's pretty good.  Robin is very, very
 5   experienced, excellent statistician, ABD.  Patrick is also a
 6   Ph.D.  He's no longer with us; he's gone back to England.
 7   Ali Mushtaq has a master's degree in computing, and is an
 8   amazingly good statistician.  Mike Kwanisai, who is someone I
 9   mentioned yesterday, was a student of Steve Thompson.
10            Steve Thompson is the leading expert in the world on
11   adaptive sampling, and we brought Steve in as a consultant to
12   have him take a look at this problem.  We thought it was a
13   problem that adaptive sampling could apply to.  He not only said
14   yes to that, but he offered us a student who has worked with us
15   since then.
16   Q.  I know we talked about it yesterday, just very, very -- one
17   or two sentences.  Would you remind us what adaptive sampling
18   is?
19   A.  Adaptive sampling is a technique --
20            THE COURT:  I remember.
21            MR. WARSHAWSKY:  Okay, very good.  That's the only
22   audience that matters.
23   BY MR. WARSHAWSKY:
24   Q.  And what was the purpose of this report, sir?
25            THE COURT:  Might not remember your name, but I
0977
 1   remember what he said about adaptive sampling.
 2            MR. WARSHAWSKY:  That's all I care about, Your Honor.
 3   A.  The purpose of this report was to tell the client what the
 4   results were, and to present them in a way that we felt
 5   comfortable represented the enormous work that had been done by
 6   an awful lot of contractors here to get to this point.
 7   BY MR. WARSHAWSKY:
 8   Q.  And this is reporting the LSA results?
 9   A.  That's correct.
10   Q.  Now, let me refer you -- we're going to go to page four of
11   this report, a highlight section again.  And let's just focus on
12   the first two bullet points right now.
13   A.  Okay.
14   Q.  Are you there, Dr. Scheuren?
15            Okay.  Would you explain the point that's being made in
16   the second bullet point about reconciling over 99 percent of the
17   items?
18   A.  Well, despite our pessimism based on the information we had
19   before we did the 2003 plan, and astonishing based on that
20   expectation, an astonishingly successful outcome occurred, and
21   we found virtually all of the records.  And over 99 percent of
22   the randomly selected records were found.
23   Q.  And you referenced all 12 BIA regions being sampled?
24   A.  All regions, yes.
25   Q.  Very good.  Let's go to the third bullet.  I gather this
0978
 1   largely follows the point that you just made?
 2   A.  That's correct.
 3   Q.  What did you mean about the reconciliation -- this is the
 4   wording here, quote, "The reconciliation should put to rest
 5   concerns about the impact that the remaining one percent
 6   unreconciled transactions might have on the results," end quote.
 7   A.  What I meant was that, even had they been wrong, that would
 8   not have changed our results.  In fact, what we did do was we
 9   treated them as if they were wrong in reporting our results.
10   One more moment.
11            That is a model assumption.  I think it's a severe
12   model assumption.  It's probably they're not as bad as that, but
13   we could not -- the accountants could not support them, and so
14   we treated them as incorrect.
15   Q.  Well, let's look at some of the quantification that you put
16   down in these bullet points that follow.  The next bullet point,
17   the fourth bullet point, you refer to a debit difference rate.
18   What are debits?
19   A.  They are disbursements, things that leave the system.
20   Q.  Okay.  And would you summarize what you've concluded here?
21   A.  Well, the debit difference rate that we found in the data
22   was 0.4 percent, okay, which is very low error rate.  Even in a
23   commercial world, that would be arguably a low error rate.
24            We calculate -- because we're talking about a sample,
25   we calculate an assurance rate, assurance level.  That's kind of
0979
 1   a confidence interval that we talked about, a one-sided
 2   confidence interval.  And with that 99 percent assurance level,
 3   we think we can say with 99 percent assurance that the debit
 4   rate in the population is 1.3 percent or less.
 5   Q.  You also refer then to a rate of differences that are
 6   disadvantageous to the account holders -- I'm sorry.  This is
 7   the last sentence.  "Our estimate for the rate of
 8   differences" -- you state that, "Our estimate for the rate of
 9   differences that are disadvantageous to the account holders,
10   with an assurance level of 99 percent, is 0.7 percent."
11   A.  That's correct.
12   Q.  What are you referring to when you talk about
13   disadvantageous?
14   A.  Mistakes can be made in both directions.  From the
15   plaintiffs' side of view, the fact that some plaintiffs received
16   more money, okay, not really of interest.  What's of interest is
17   that some received less than they should have received.  And so
18   we focused only on the disadvantageous portion of this.  The
19   error rate that you see here is 1.3 percent.  About half of
20   that, when you round it, is the .6 percent.
21            Later on in this page, and in some supporting work that
22   is not before the Court, we have tested the hypothesis that the
23   system is, quote, "fair," unquote; that mistakes are made in
24   both directions about at the same rate.  And this is a rate
25   estimate.  These are percentages of transactions in error.  We
0980
 1   then talk a little bit later about dollars in error, but that
 2   comment about fairness applies to both estimates.
 3   Q.  You may have just misspoken, I think, when you referred to
 4   a .6 percent rate.  If you put your glasses on.
 5   A.  Oh, I should have paid -- pointed a sentence up, to
 6   the .7 percent for the 99 percent.  That's correct.
 7   Q.  What was the .6 percent that's referenced in your bullet
 8   point there?
 9   A.  If I narrow the confidence interval from a 99 percent to a
10   95 percent, okay, that's the answer I get for that.
11   Q.  Now, is this an attribute analysis or a variable analysis?
12   A.  This is an attribute analysis.
13   Q.  And why is that?
14   A.  We're looking -- we're counting mistakes.  They happened or
15   they didn't happen.
16   Q.  Well, would you contrast that now with the next bullet
17   point?  Summarize what you're saying in that bullet point,
18   please.
19   A.  Here we're looking at the dollars, the dollars in error for
20   debits, still.  The dollars that were disadvantageous to the
21   account holders.
22   Q.  What did you conclude -- first of all, is that a variable
23   analysis?
24   A.  Yes, it is.
25   Q.  What did you conclude with regard to the debit differences?
0981
 1   A.  The disadvantageous estimates were no more than four
 2   million, okay.  That is, there was an opportunity to make dollar
 3   errors.  And in terms of the whole system over this electronic
 4   era, we estimated it at a high assurance level, 99 percent, that
 5   it's $4 million or less.
 6   Q.  $4 million from 1985 through 2000?
 7   A.  That's correct.  Given -- I want to emphasize, because there
 8   were issues about this that may come up, this is given the
 9   population that we were sampling from.  It's based on the
10   population that was subject to being sampled.
11   Q.  And you also have a conclusion there using this 95 percent
12   level, which you said --
13   A.  That's correct.
14   Q.  -- which is more common in business settings.  What did you
15   find there?
16   A.  That instead of four million, there was two million.
17   Q.  Let's move down to the next bullet point.  You have a
18   comparable finding with regard to credit differences.  What are
19   the credits that you're referring to here?
20   A.  These are receipts that go into the accounts, and there are
21   transfers also included.
22   Q.  And would you summarize your conclusions with regard to
23   that?
24   A.  We found a higher error rate for credits.  1.3 percent was
25   the estimate we had.  At the assurance level of 99 percent,
0982
 1   we've gone through this now before, we can state that the
 2   difference for credits is no more than seven percent.  That's
 3   not a small number, in my opinion.  But still, that's what it
 4   is.  So it was seven percent.
 5            But the disadvantageous -- can I continue with this?
 6   Q.  Please.
 7   A.  The disadvantageous fractions is about half of that.  Again,
 8   we can support that, that the fairness of the system, half of
 9   those credit mistakes were in favor of the government and half
10   were in favor of the plaintiffs.
11   Q.  Is that the four percent that you're referring to here?
12   A.  That's right.
13   Q.  And again, you also have a similar finding for 95 percent.
14   Correct?
15   A.  We're rounding the percentages, and that's why you're
16   getting these -- they're not exactly half.
17   Q.  And in the next bullet point you quantified -- you came up
18   with a variable analysis.  Is that correct?
19   A.  That is correct.
20   Q.  And what did you find for credits?
21   A.  Same conditions.  86 million was the -- disadvantageous to
22   the account holders.
23   Q.  That was at the 99 percent level?
24   A.  That's correct.  And at 95, it's 42 million.
25   Q.  Finally, the last bullet point, the one that appears below
0983
 1   that, I think you may have referred to this earlier, but explain
 2   what that bullet point is about.
 3   A.  It's worth repeating.  One of the considerations we had was,
 4   to the extent -- and I'll say it again:  To the extent to which
 5   the system was fair, I mean, that the error rates are low, or
 6   lower than expected anyway, was a good thing.  But were they all
 7   in one direction, or not?
 8            And they were not.  They were about equally --
 9   statistically, anyway.  A lot of uncertainty here.  But we could
10   not reject -- I'll say it this way a little more formally:  We
11   could not reject the null hypothesis that the underlying
12   processes which generated the errors were the same for both
13   advantageous and disadvantageous.
14   Q.  I would like to refer within this document to Appendix A.
15   This begins on page 22 --
16            THE COURT:  Would you accept "even-handedly erroneous"
17   as a substitute for "fair"?
18            THE WITNESS:  I would accept that, absolutely.  In
19   fact, I think I like it better.  I believe we use the word
20   "fair," though.  But I like even-handedness.  Thank you.  Thank
21   you, Your Honor.
22   BY MR. WARSHAWSKY:
23   Q.  Right.  Refer now to page 22, and we'll blow up the first
24   couple of paragraphs here.  What's presented in Appendix A to
25   this document?
0984
 1   A.  What we're talking about is how we did the sampling here,
 2   for all 12 regions.
 3   Q.  Let's start with the big one here, A.1, the 10-Region group.
 4   What does that refer to?
 5   A.  May I start in a different order?  We had done Alaska
 6   already.  We had had -- we had a good population for Alaska, and
 7   Eastern Region had been done as a result of the 120-day report.
 8   Q.  Okay.  Why don't we -- if you prefer to go in that order,
 9   let's go first to page 26, then.
10   A.  All right.  Excuse me.
11   Q.  It's okay.  Whatever makes it simpler.  A.2, is this the
12   Alaska sample that you were talking about?
13   A.  That's correct.
14   Q.  And describe what you did with the Alaska sample.
15   A.  We had already been sampling in Alaska as a result of the
16   2003 plan, and so we drew a sample of that sample and then
17   proceeded to -- it was then passed to the accountants for their
18   work.
19   Q.  Now, in drawing your sample, did you directly select
20   transactions for Alaska?
21   A.  We went to accounts again.
22   Q.  So you started with accounts.  And what did you do with the
23   accounts?
24   A.  We sampled within the accounts, to choose transactions
25   within the accounts.  We favored it -- since we've already
0985
 1   selected some transactions randomly, if we had already selected
 2   a transaction randomly in the account that we selected randomly,
 3   we used that transaction.  If we had not selected that
 4   transaction in the earlier sampling, we added more transactions
 5   to sample.
 6   Q.  And I believe you indicated you did the Eastern Region
 7   before the 10-Region?
 8   A.  Correct.  In the 120-day report --
 9   Q.  I'm sorry.  Why don't we go first to page 27, then.
10            MR. WARSHAWSKY:  Blow that up more for me, even.
11   BY MR. WARSHAWSKY:
12   Q.  It indicates OHTA's first reconciliation was completed for
13   the Eastern Region in 2003.
14   A.  Correct.
15   Q.  So that preceded the Alaska sample?
16   A.  Yes.
17   Q.  Tell us about the Eastern Region sample, how that was drawn.
18   A.  Going back to the 120-day report, the plan was to do them
19   all, to take a census.  It wasn't a very big population.  That
20   made sense.  But it was very difficult to do.  And it was one of
21   those things that needed to get done before other things that
22   were bigger needed to be done.
23            And because it wasn't happening, the accounting firm
24   that had the responsibility of doing the 100 percent census,
25   which you don't need a statistician for, came to us.  And they
0986
 1   said, "Help us out."  And we said "Sure.  Let's draw a sample.
 2   Let's take what you've already done and treat them as
 3   100 percent, give them a weight of one, a probability of one,
 4   since they were selected with certainty.  And then we'll sample
 5   the rest for you, and then you'll have to find them all."
 6            And so we developed a sample.  In fact, my colleague, I
 7   believe Robin Lee, who's sitting here, is actually the one who
 8   drew this sample.  And we gave them the sample to do, and then
 9   they went out and did their best.  And the results were
10   reasonable, although there were some transactions - I think
11   there were eight - that they did not find.
12   Q.  So you started trying to do 100 percent --
13   A.  They did, yes.
14   Q.  -- and then drew a sample.  Were you still able to perform a
15   statistically valid analysis on the --
16   A.  Because we were keeping track of the probability of
17   selection, we were able to make a inference, an estimate for the
18   Eastern Region.  Even if it were done in this sort of "try this
19   way, then try that way" kind of style, fundamentally -- one more
20   moment.
21            Fundamentally, if you're keeping track of how you do
22   the work and you're systematic about it, even though you
23   encounter challenges along the way - and when you're doing
24   something for the first time, you should expect challenges along
25   the way - and you expect a procedure which you can modify and
0987
 1   still accomplish your goal.
 2            One of the problems in some of the work that had been
 3   done prior to our work was that others had encountered problems
 4   that they didn't anticipate, and they, quote, "ran out of
 5   money," unquote, and --
 6   Q.  I'm sorry --
 7   A.  -- they were unable to finish.
 8   Q.  Let me just interrupt a moment.  You've talked about
 9   Andersen running out of money.
10   A.  I am talking about Andersen, yes.
11   Q.  What project are you talking about there?
12   A.  The work they did on Tribal, and they started for IIM, yes.
13   Q.  And what time frame was this, do you know?
14   A.  About three or four years before we were working on this,
15   yeah.  I've heard the presentations, I've looked at some of
16   their data.  I'm afraid the exact dates are not in my head at
17   the moment.
18   Q.  But this predates your involvement?
19   A.  They do.  I want to make the comment that excellent work was
20   done by them, but they didn't finish it.  Okay?  And that they
21   didn't finish it was really a major problem as a result.
22   Q.  Are you familiar with the House committee report called
23   "Misplaced Trust"?
24   A.  In general terms.
25   Q.  Do you know if that report describes the effort that you're
0988
 1   talking about?
 2   A.  I believe that's probably the case, yes, where it's
 3   described.
 4   Q.  Now, after OHTA did the Eastern Region and the Alaska
 5   Region, you moved on to the 10-Region?
 6   A.  We did the 10 Regions, yes.
 7   Q.  Why don't we pull up page 22 now.  And there were a total of
 8   12 regions.  Is that correct?
 9   A.  Correct.
10   Q.  So this is the rest of the regions?
11   A.  Remaining 10.
12   Q.  And in the second sentence under A.1 it states, quote, "The
13   data used for designing and drawing the sample was provided by
14   FTI Consulting, Inc., an OHTA professional accounting
15   contractor, in December 2003."
16            Why did you draw the sample from FTI data?
17   A.  Well, the way the population is set up did not give us a way
18   to put the units we wanted to observe together automatically.
19   We had to do some work -- Michelle, whom you've heard here, led
20   that effort to bring accounts together, because there can be
21   multiple account numbers for the same account holder, and that
22   had to be addressed so that we would get an unambiguous
23   selection probability at the account holder level.
24            And she went ahead and did that, and did what you call
25   at that point a validation effort.  And she was churning them
0989
 1   out -- after all, we were all pushed pretty hard by this
 2   initiative.  And it happened in the fall.  She was churning them
 3   out agency by agency -- excuse me, region by region, and passing
 4   them to us as she did them.
 5            And we spent our holiday - and she did, too - working
 6   on this, in order to meet a delivery in the first week of
 7   January, which we met.  And it was an astonishing achievement,
 8   even though we didn't celebrate Christmas fully.
 9   Q.  And so the effort that you've just described for
10   Ms. Herman's team, that's what you were referring to in the next
11   sentence by the data validation procedures"?
12   A.  That's correct.
13   Q.  The second paragraph of this section talks about this
14   process again.  And this is near the bottom of that paragraph,
15   where you talk about the sample of transactions under $100,000
16   being selected in two phases.
17   A.  Correct.
18   Q.  First, a sample of accounts, and then a sample of
19   transactions.
20   A.  We sampled 1,000 accounts, and then averaging
21   20 transactions per account.  That was the design.
22   Q.  Let me take a couple of points here.  Why did you use
23   $100,000 as the cutoff?
24   A.  There needed to be an upper cutoff of some kind, to protect
25   against errors that occurred in the large accounts that might be
0990
 1   different than the small accounts, in the large accounts those
 2   100,000 in this definition.
 3            The actual boundary, the strata boundary, was
 4   negotiated with the client.  I think that number comes from
 5   Abe Haspel.
 6   Q.  So for $100,000 and over, what was done?
 7   A.  We were going to take all the transactions, no matter what
 8   accounts they were from.
 9   Q.  And then you sampled under $100,000.  Why did you allocate
10   among the regions?  You referred to 20, I think each region, at
11   least?
12            THE COURT:  Excuse me.  I'm missing something here.
13   100 percent reconciliation of every transaction involving
14   $100,000?
15            THE WITNESS:  Yes, sir.
16            THE COURT:  But what's the difference between that
17   choice and the choice made in an earlier table, which was to run
18   it down, everything of $5,000 or more?
19            THE WITNESS:  May I answer that?
20            THE COURT:  Yes.  The question is to you.
21            THE WITNESS:  The 100 percent strata was changed from
22   5,000 to 100,000.  The belief was, we're not doing an historical
23   accounting at this point.  We didn't know we were doing a
24   historical accounting.  It turns out, I think, we may have
25   accomplished that, to a degree.  But we were not doing it.
0991
 1            And we were trying to take a look at what was happening
 2   in this population, and you have to concern yourself with the
 3   large transactions.
 4            So there does have to be a take-all strata at some
 5   level, and since we were trying to move quickly through this
 6   process, we moved that boundary from 5,000, where there were
 7   73,000 accounts, up to 100,000.
 8            THE COURT:  And that was for this litigation support --
 9            THE WITNESS:  That's correct.
10            THE COURT:  -- project.  Okay.
11   BY MR. WARSHAWSKY:
12   Q.  By the way, there's been some suggestion that litigation
13   support -- the litigation support project was designed with an
14   eye towards coming up with as small a number as possible for
15   exposure.  Is that true?
16   A.  I have no knowledge of that.  That's not the way we do our
17   work.  We took a risk, by having a small sample, that we have a
18   large variance.  But the data are unbiased.  The work is done by
19   a reputable accounting firm, four reputable accounting firms,
20   and checked by a fifth reputable accounting firm.  And I do not
21   believe that anything was done to push it in one direction or
22   another.  It just was what was it.
23            MR. GINGOLD:  Objection, Your Honor.  I think it's
24   beyond the scope of the witness' expertise.  He states in
25   several of his own reports that his firm did not perform the
0992
 1   basic reconciliation work.  And specifically he says, in 514-6,
 2   "It is beyond the scope" of his work and his expertise to
 3   comment on the accounting done.
 4            MR. WARSHAWSKY:  Your Honor, he's simply --
 5            THE COURT:  The objection is sustained insofar as it
 6   supposes that the witness is giving an expert opinion about the
 7   quality of the accounting firms.
 8            MR. GINGOLD:  Thank you, Your Honor.
 9            MR. WARSHAWSKY:  And just to be clear, I wasn't asking
10   for an opinion.
11   BY MR. WARSHAWSKY:
12   Q.  Let me ask you to refer now to the third paragraph --
13            THE COURT:  We don't do that in this classroom, Doctor.
14   He asks the questions.
15            THE WITNESS:  Thank you, judge.
16   BY MR. WARSHAWSKY:
17   Q.  You referred to this a little earlier, and I just wanted to
18   get this elaborated on a little bit.  The first sentence of the
19   third paragraph talks about, "Allocating the sample among
20   10 regions, roughly proportional to the number of accounts in
21   the regions, minimum of 20 accounts per region."  Why did you
22   use that approach?
23   A.  We wanted to achieve representativeness across the
24   geography, and later on across time, and even still later on in
25   the design, across type of transaction.  And this is a starting
0993
 1   point for that effort.
 2   Q.  Okay.
 3            MR. WARSHAWSKY:  Let's go to page 23.  And at the
 4   bottom, the very last few sentences, you can even cut off the
 5   table.
 6   BY MR. WARSHAWSKY:
 7   Q.  The statement reads, quote, "The work was to be divided into
 8   four replicates," end quote.
 9            Now, I didn't know about replicates before this case.
10   What's a replicate?
11   A.  A replicate is a representative sub-sample of the overall
12   sample, such that if you had designed a smaller sample, it would
13   have looked, for design purposes, exactly like the larger
14   sample.
15            In this case we chose four replicates because of our
16   concerns about how hard this would be to do.  After all, we had
17   just seen how difficult Eastern Region was.  And if you get into
18   the details of Eastern Region, that was not a challenging,
19   compared to later activities, a challenging activity.  It was
20   difficult enough.  So we divided into four replicates.
21            And we had a very tight time frame, so we recommended
22   to the client that we start with a single replicate.  And then,
23   if there were resources and time available, other replicates
24   would be added.
25   Q.  Earlier in your testimony you talked about a sample of
0994
 1   20,000 transactions, I believe?
 2   A.  That was the full sample, if we had done all of it.
 3   Q.  Is that divided among the four replicates?
 4   A.  That's correct.  We only did a fourth of it, yes.
 5   Q.  So it's 5,000 transactions per replicate, roughly?
 6   A.  Correct.  But we did -- we did all of the 100 percent cases,
 7   so that made it 6,000.
 8   Q.  Now, if you only ended up doing one of the four replicates,
 9   that's what you said.  Right?
10   A.  Correct.
11   Q.  How does that affect your statistical analysis?  How does it
12   affect the inferences you draw from the single replicate?
13   A.  It makes the margin of error larger.  Since there were four
14   replicates, and since we've already established this quadratic
15   relationship with sample size, we take the square root of four,
16   that's two; so that means our margin of error is twice what it
17   would have been if we had done the whole thousand.
18   Q.  And when you talk about your margin of error, again that
19   refers back to the numbers you talked about in your highlights
20   section?
21   A.  Yes.
22   Q.  Now, let's go up to page -- skip a couple of pages, to
23   page 25.  And just very quickly, this is below table A.5.  Is
24   this what you were just referring to when you said OHTA
25   subsequently made the decision to reconcile all four replicates
0995
 1   of the high-dollar transactions, but only one replicate of the
 2   national sample?
 3   A.  That's correct.
 4   Q.  And the national sample was the 20,000 transactions?
 5   A.  That's correct.
 6   Q.  You seem to be pausing on that response.
 7   A.  The 20,000 was of the sample.  Okay?  The 100 percent strata
 8   was on top of that, so it was actually -- you know, that's why I
 9   paused.
10   Q.  Let's jump back to pages 12 through 14, and I think we can
11   go through this in relatively quick order.  But you've got a
12   section captioned "Debit results."  And that continues, if you
13   want to just show the next two pages.
14            Okay.  You're familiar with pages 12 through 14 of this
15   report.  Correct?
16   A.  Yes, I am.
17   Q.  And what was the purpose of these three pages in your
18   report?
19   A.  They were to describe in summary fashion the way we
20   approached the estimates, and some details around those numbers.
21   Q.  And do these support what you set out in your highlights?
22   A.  They support the highlights.  That's exactly what they're
23   for.
24   Q.  Go to table four on page 13, please.
25            Can you see that, sir?
0996
 1   A.  Yes, I can.
 2   Q.  I'm sorry.  What is depicted in table four?
 3   A.  Let me -- these are the in-scope debit transactions that
 4   were selected for reconciliation.  This is the sample, and there
 5   is the 100 percent strata.
 6            At this stage of the game there were two -- we divided
 7   the 100 percent strata up into two pieces:  Those over half a
 8   million, which are in the first -- in the beginning of this were
 9   all to be obtained; and when we ran into some challenges, those
10   between 100,000 and 500,000, we did not have all of them
11   available at that time.
12            The rest is a national sample; that's the third row.
13   And then the total at the bottom.  And then across the columns,
14   how much throughput there was in the sample, the number of
15   reconciled transactions, the reconciled throughput, and the
16   number of differences that were found.
17   Q.  What's the difference between the sample size and reconciled
18   transactions?
19   A.  Sample size are the cases that we originally selected and
20   passed to the accountants to do their reconciliation.  And the
21   reconciled transactions are the ones that came back to us from
22   the sample.  They didn't find them all, sometimes.
23   Q.  I beg your pardon?
24   A.  They didn't always find them all, or couldn't always
25   reconcile them anyway.
0997
 1   Q.  Let's look at the national sample numbers now, here.  2,372,
 2   is that the total number of transactions that were selected for
 3   debits?
 4   A.  That is correct.
 5   Q.  What does it mean that they reconciled all but nine of them?
 6   A.  Well, it means that they didn't find the supporting
 7   documents that they needed in order to be able to make a
 8   determination as to whether the transaction was correct or was
 9   in error.  That's what it means.
10   Q.  The zero under "number of differences," what does that mean?
11   A.  For the 2,363 transactions which they reconciled, they found
12   no differences.  No errors, in a more casual way.
13   Q.  As a statistician, did you draw any conclusions when you saw
14   they were able to reconcile all but nine out of 2,372?
15   A.  Well, it's an astonishingly good result, frankly.  I've done
16   a lot of other projects in a lot of other places , and this is
17   probably the very best result, in terms of what you've targeted
18   to do and what you actually achieved, that I've ever seen.
19   Q.  And let's go to that section on page 14 near the bottom,
20   where we have some -- I don't know why I ended up highlighting
21   everything here, but what's set out in your "inference for debit
22   transactions" section.  Just generally, what is that?
23   A.  Well, this is a somewhat longer statement of you've already
24   asked me to talk about earlier.  And I'm not sure I need to do
25   more than that, unless you want me to read something
0998
 1   specifically.
 2   Q.  Fair enough.  Let me ask you about the first sentence in the
 3   final paragraph where it says, "The above figures are based on a
 4   very conservative estimate of the effect of the unreconciled
 5   transactions by calculating the estimates assuming that, if
 6   reconciled, these transactions would have differences," end
 7   quote.
 8            What did you mean by that?
 9   A.  Those cases that were not found were treated as if they were
10   errors.  Okay?  And that's a very conservative assumption.
11   Q.  You're talking about the nine transactions, the nine
12   transactions?
13   A.  Yes.
14   Q.  Why did you make that assumption?
15   A.  I had earlier used the analogy with the work we did at IRS.
16   Many of us are from IRS and have done this kind of work at IRS
17   earlier.  Dr. Hinkins and Robin Lee, and now Ed Murrow, who just
18   joined us, they're all from the IRS.  And IRS does an awful lot
19   of samples of these sorts, and so it was appropriate that we
20   look at their practice.  And I felt comfortable with adopting
21   it, and that's what we recommended.
22   Q.  So in your analysis, you assumed all nine transactions were
23   in error?
24   A.  Correct.
25   Q.  In fact, you believe that would have been the case if you
0999
 1   had found support for the nine transactions?
 2   A.  You're asking me to speculate.  Do you want me to speculate?
 3   Q.  Please.
 4   A.  No.
 5   Q.  Why is that?
 6   A.  No, I don't think so.  I have actually --
 7            MR. GINGOLD:  Objection, Your Honor.  This is outright
 8   speculation.
 9            THE COURT:  Sustained.  You're slurring in your older
10   age, Mr. Gingold.
11            MR. GINGOLD:  Just tired.
12            THE WITNESS:  Am I allowed to thank Mr. Gingold?
13   BY MR. WARSHAWSKY:
14   Q.  Let me ask you this:  Based on everything you saw, do you
15   have any reason to believe that all nine of those transactions
16   in fact were erroneously recorded?
17   A.  Let me answer a slightly different question from that, if
18   you would.
19   Q.  No, you're only allowed to answer my questions,
20   Dr. Scheuren.
21   A.  My answer is, I do have no knowledge that they were in
22   error.
23   Q.  Let's look at pages 15 to 17 of this report.  And we don't
24   need to look at any specific section right now.  You're familiar
25   with these pages of your report.  Right?
1000
 1   A.  Yes.  These are parallel to the debit stuff we've just seen.
 2   Q.  And why don't we go to table seven on page 16.  And is that
 3   a similar --
 4   A.  Yes, it is.
 5   Q.  -- table to the one you've described for the debit analysis?
 6   A.  Yes, it is.
 7   Q.  And would you summarize what you found for the national
 8   sample there?
 9   A.  The national sample line, there were 2,128 transactions
10   sampled.  And the sample throughput is the next column.  And the
11   number of reconciled transactions is the column after that,
12   2,117, which means there were 11 that were not reconciled.  And
13   then the throughput that was reconciled, and the number of
14   differences found; here there were 36 differences found.
15   Q.  So in this case, is it fair to say there were some items
16   that were reconciled, but with differences?
17   A.  That's correct.
18   Q.  As opposed to items treated as errors, for your purposes,
19   because of no support?
20   A.  That's correct.
21   Q.  Now, why don't we move down to the bottom of page 16,
22   carrying on to 17.  Again, I think my over-highlighted sections
23   are there.  Let's go to the top of page-17.
24            Again, this section on inferences for credit
25   transactions, is that comparable to what we said earlier to --
1001
 1   A.  Yes, it's a somewhat lengthier version of the highlights
 2   that we talked about awhile ago.
 3   Q.  And again, now, the first sentence in this final paragraph,
 4   quote, "These figures incorporate the most conservative missing
 5   data model for the transactions selected but not reconciled.
 6   The unreconciled transactions have differences," end quote.
 7   A.  That is correct.
 8   Q.  And again, that's your assumption about where an item
 9   couldn't be reconciled:  You treated that as an error?
10   A.  We didn't find the supporting documents to make a decision.
11   The accountants didn't, and so we treated it as an error.
12   Q.  And in the course of your work, did you see anything that
13   caused you to believe that in fact those 11 items were
14   erroneous?
15   A.  I have no knowledge of that.
16   Q.  All right.  Why don't we move on to a document in the
17   administrative record, AR-437.  The Bates number on this is
18   000040-1.
19            Dr. Scheuren, are you familiar with this document?
20   A.  Yes, I am.
21   Q.  What is this document?
22   A.  This is a follow-on of the previous document, where we're
23   looking at the reconciliations remaining high dollars.  In the
24   earlier document there was a distinction between those high
25   dollars that were between 100,000 and 500,000, and those that
1002
 1   were 500,000 and over.  And we had recommended to the client
 2   that they not complete the transactions between 100,000 and
 3   500,000, because they probably wouldn't learn very much from it
 4   and we had already made an estimate.
 5            They decided to complete it, and this is what the
 6   results were when that happened.
 7   Q.  And now let's move to AR-442, Bates number is 000042-2.
 8            Are you familiar with this document, sir?
 9   A.  I am.
10   Q.  And what is this?
11   A.  It's another one of the results.  And here we're talking
12   about how the litigation support led us to the sampling plan
13   that we submitted.
14   Q.  Okay.  And if we move down this document, there are two
15   bulleted items in the first paragraph.
16   A.  Yes.
17   Q.  Do you recognize --
18   A.  Those are the two documents you just asked me to talk about;
19   the first one at length, the second one just by reminding the
20   Court of what it was.
21   Q.  Okay.  So that was AR-438 and 437?  I'm sorry, the
22   September 30, 2005 document --
23   A.  Yes.
24   Q.  -- and the December 31, 2000 -- I'm sorry.  The December 31,
25   2006 document?
1003
 1   A.  Yes.  These are the two that we've just looked at, yeah.
 2   Q.  Very good.
 3            Now, I'm a little more judicious with highlighting
 4   here.  You see I've highlighted a sentence that reads, quote,
 5   "These analyses have caused NORC to rethink the sampling
 6   proposals made in the January 6, 2003 historical accounting
 7   plan, the one that was submitted to the District Court."  Do you
 8   see that, sir?
 9   A.  Yes.  I wrote those words.
10   Q.  I'm sorry?
11   A.  I wrote those words.
12   Q.  Why did you write those words?
13   A.  Because we have rethought it.  One learns something over
14   time, and that's what happened.  I mean, we started out very
15   knowledgeable about our skill sets, but not about the
16   population.  We now know a lot about the population, too.  And
17   we concluded that the original plan, now that we've done the
18   litigation support activity, was much too large, was
19   over-engineered, as the phrase goes, and unnecessarily large.
20   Q.  Okay.  And in what -- what was it specifically that you
21   rethought as a result of your work?
22   A.  Well, most fundamentally was, we could find the documents.
23   We actually -- I mean, we didn't do it.  We did a little of it.
24   My colleague Robin Lee was heavily involved in that.  But most
25   of the work was done by the accountants, by experts on searching
1004
 1   at the various Indian repositories that we worked in.  And they
 2   found the records.  It was astonishing.
 3            The second thing was that the error rate was low
 4   relative to what our expectations were.  Quite low.
 5   Q.  Okay.  We'll go into this in a little bit more detail.  Not
 6   too much, but a little bit more.
 7            Bottom of page two.  First of all, there's a
 8   highlighted statement, which you can see on the screen, "Thus,
 9   the large sample size proposed in the 2003 plan was not
10   necessary."  Your words again?
11   A.  Those are my words, and that's a summary statement of what I
12   just said.  We didn't need that big a sample for the electronic
13   ledger era.
14   Q.  There's a highlighted statement above there.  That was your
15   recommendation as a result of this work?
16   A.  That is correct.
17   Q.  And that recommendation was?
18   A.  That we did not recommend any further reconciliations of the
19   posted transactions.  Can I elaborate why I said that, or do you
20   want to do it later?
21   Q.  Please.  That would have been my next question.  Go ahead.
22   A.  It has to do with how much it costs to do the
23   reconciliations, as to how much exposure or how much reduction
24   in our understanding -- how much better our estimates would be.
25            Let me make something very clear, Your Honor.  When we
1005
 1   increase the sample size, we don't expect to change the point
 2   estimate, the estimate of the error rate.  That's not going to
 3   happen.  What's going to happen is, we're going to shrink the
 4   margin of error around the estimate.  Okay?  And that's the
 5   issue.  So when you shrink the margin of error, of course, you
 6   change some of these assurance levels that we talked about
 7   earlier.
 8            And we looked at this issue.  Let's say we had done a
 9   second replicate.  We had four, and we did a second replicate.
10   The litigation support activity cost well over $20 million.
11   Okay?  Very expensive.  Some of that was for the 100 percent
12   certainty strata, and the rest was for the sample.
13            Well, the certainty strata has already been done.
14   You're not going to spend that money again.  Those cases that
15   were over 100,000, 100,000 or over, we've been calling them the
16   certainty strata.  And then -- but we could have done another
17   replicate, another sample replicate.
18            The order magnitude on cost of that was about what we
19   would have been able to reduce the assurance level by.  So we
20   were not going to gain a great deal because of the fact that
21   this increasing sample sizes doesn't give you a very big result
22   in terms of lowering the assurance levels or lowering the
23   exposure or whatever phrase you want to use.  It's about
24   comparable.
25            So it didn't make sense to us.  You're going to have to
1006
 1   spend real money, okay, and there's always a chance, okay, that
 2   what you're expecting to happen doesn't happen, and you might
 3   end up not even being pleased with what you came up with.
 4            And by then -- may I add one more thing?  By then,
 5   everyone involved was totally exhausted.  I mean, the amount of
 6   effort that people stretched themselves to do was such that at
 7   least you give them a break before you do it.  So that was also
 8   a factor here.
 9   Q.  But you said, if you had done a second replicate, it would
10   have cost about 20 million?
11   A.  Probably a little under 20 million.
12   Q.  And so to complete that sample, as you originally designed
13   it, another $60 million?
14   A.  Close to that, yeah.
15   Q.  Near the top of page two, a little bit further up, the first
16   sentence under "2003 plan versus LSA design."  You've touched on
17   this already a couple of times, but explain what you meant here
18   when you said the 2003 plan indicated an adaptive approach would
19   be followed.
20   A.  Our plan then and our plan now is to base what we end up
21   with what the final sample size is based on what we're learning.
22   And if, as in this case, we thought that the error rates -- if
23   the error rates were higher than we expected them to be, say in
24   a particular agency, we would have increased the sample size in
25   that agency in order to achieve the same overall level that we
1007
 1   expected to achieve.
 2            We have a set of assumptions we're making.  Those
 3   assumptions lead us to a tolerable error.  The client agrees
 4   that that's what they want.  If the assumptions are wrong, and
 5   they're wrong in that the error is bigger than assumed, in order
 6   to still achieve what the client wants, you have to increase the
 7   sample size.  The error is what it is.  You can't change the
 8   error rate, but you can reduce the sampling error around that
 9   number.
10   Q.  Next to last paragraph in this section, right above the
11   highlighted section, "In the LSA project, however," do you see
12   that paragraph?
13   A.  Yes, I do.
14   Q.  That summarizes what you've already said, right, about the
15   record find and the reconciliation rates?
16   A.  Yes.
17   Q.  And that was a basis for your recommendation?
18   A.  Yes, it is.
19   Q.  Now, let's go to the final page, your summary conclusions.
20   You've already talked about your strong recommendations.
21   Explain -- you've got slightly different analysis on the debit
22   transactions and the credit transactions.
23   A.  That's correct.
24   Q.  And would you explain the distinction between your analysis
25   here?
1008
 1   A.  We actually already thought at this point, given the results
 2   of debits, that we had drawn a sample that was too big for
 3   debits already.  We didn't actually need as big a sample as that
 4   to achieve what we thought the client wanted.
 5            Credits are a closer call, a trade-off on costs.  Okay?
 6   Because there was some lumpiness in the credit results, and
 7   there was -- the error rates were higher, slightly higher,
 8   1.3 percent.  Remember, the client expectations were
 9   one percent; that's what we designed it at.  So it was slightly
10   higher, at 1.3 percent.
11            What saved us was the fact that the find rate was so
12   exceedingly good.  We found virtually everything.  Well, the
13   accountants and others found virtually everything.
14   Q.  Now explain, though, what you're talking about in the credit
15   transactions about cost effectiveness.
16   A.  Well, you've just heard that, and I'll just recount it
17   again.  If you had increased the sample size for credits, you
18   would have reduced the margin of error on credits.  The estimate
19   would have been about the same, 1.3 percent, but it has
20   uncertainty so it could have wobbled a little bit.  But you were
21   going to spend a lot of money to do that, and it just -- why not
22   just take -- we asked the client, "Why don't you just take the
23   higher assurance level," okay?  It's reasonable to do that, and
24   that would be fair.
25   Q.  Dr. Scheuren, I would like you to refer now to AR-427.
1009
 1            THE COURT:  How much more direct do you have,
 2   Mr. Warshawsky?
 3            MR. WARSHAWSKY:  I would say a half an hour, at most.
 4            THE COURT:  In that case, let's take a mid-morning
 5   break.
 6            Mr. Smith, could I see you at side-bar, and
 7   Mr. Kirschman?
 8            (Recess taken at 11:02 a.m.)
 9            THE COURT:  All right, sir.  Please continue.
10   BY MR. WARSHAWSKY:
11   Q.  Dr. Scheuren, I'd like to get back to one point that you
12   mentioned before the break.  When we were talking about -- you
13   were explaining reasons not to go ahead and do a second
14   replicate.  Do you remember that?
15   A.  Yes, I do.
16   Q.  One of the reasons you said was you might not get the result
17   you liked.  What did you mean by that?
18   A.  Well, let me -- an ill-chosen word, I might add.  You have a
19   point estimate of 1.3 percent - we'll talk about credits - and
20   you're going to do another sample, you're going to get a bigger
21   sample, but you're still going to have uncertainty in the
22   results.
23            And the point estimate might not be 1.3 when you've
24   added the second replicate.  It could be smaller, could be
25   larger, depending on whether you like a smaller or larger
1010
 1   number.  You might not like that result, and you have spent a
 2   lot of money.
 3            From the government's point of view, what really
 4   matters is not so much the point estimate but the upper bound.
 5   The upper bound is seven percent, as you'll remember.  Okay.
 6   99 percent upper bound.  You would expect, given the larger
 7   sample, that that upper bound would shrink, be smaller, if you
 8   get the same 1.3 estimate.  You can't guarantee the 1.3.  After
 9   all, it's an uncertain world.
10            So does it pay to reduce the upper bound and spend
11   about as much money in order to reduce it, as you would gain if
12   it were smaller?
13            So I didn't think it was worth doing, and I recommended
14   it to them and they've subsequently adopted that.
15   Q.  Dr. Scheuren, I'd like you to refer to -- from the
16   administrative record AR-427, Bates numbers 000052-3.
17            MR. WARSHAWSKY:  We may have to blow this up a little
18   bit.  There we go.
19   BY MR. WARSHAWSKY:
20   Q.  March 15, 2007 memo from you to Bert Edwards.  Do you
21   recognize this document?
22   A.  Yes, I do.
23   Q.  And what is this?
24   A.  It's our proposal to the Office of Historical Trust
25   Accounting.  Okay.
1011
 1   Q.  Okay.  Your proposal with regard to what, I'm sorry?
 2   A.  The 2007 plan.
 3   Q.  And so this ultimately is what led to the plan -- or this is
 4   with regard to the plan that's before the Court now?
 5   A.  That's correct.
 6   Q.  Now, I'd like you to refer, bottom of page two -- first of
 7   all, to put it in context, on the first page, not the cover
 8   memo, but the page one that follows it, you have a section
 9   captioned, electronic ledger era.
10   A.  I'll take your word for it at the moment.
11   Q.  Here, it's blown up a little bit.  Does that help?
12   A.  Yes, I do.
13   Q.  And it's within this section that I want to ask you some
14   questions.  This carries over to page three.
15   A.  I'm not sure -- this has been moving around in front of me,
16   so...
17   Q.  You should have worked with the paper copy.
18            Starting at the bottom of page two.
19            MR. WARSHAWSKY:  Can you get the bottom of page two and
20   that section at the top of page three on one screen?  I don't
21   know if you can do that or not.  Perfect, I think.
22   BY MR. WARSHAWSKY:
23   Q.  You have this paragraph that begins at the bottom of page
24   two where you're talking about, "based on the results of the LSA
25   sample, you conclude only relatively modest follow-up work?"
1012
 1   A.  Correct.
 2   Q.  And you refer to two areas not yet covered.  What did you
 3   mean by the posting tests?
 4   A.  We talked about the -- initially about the fact that we were
 5   only looking at transactions that had been posted.  So our
 6   sample frame population that we're sampling from are posted
 7   transactions.
 8            There is a possibility that there are unposted
 9   transactions that were never posted, and we need to look at
10   that.  That's an important issue.  Okay?
11            And I recommended that we do that.  That's the first of
12   these two on posting.
13   Q.  And what are you recommending be done with regard to any
14   transactions ultimately found to have been unposted?
15   A.  That they would be errors, that there would probably be
16   errors if they were -- should have been posted and were not
17   posted, that would be errors.  And that would be a serious
18   errors.
19   Q.  How would you assess those?
20   A.  Well, we're working on that right now.  I happened to be
21   here at the end of the day yesterday because I was getting ready
22   to testify when Michelle was talking about Holton (sic).
23   Q.  Was it Horton?
24   A.  Horton, yes.  One of the places we've been testing, testing
25   in that case, procedurally how we would do this.  Okay?
1013
 1            MR. WARSHAWSKY:  Just so the record's clear, I saw a
 2   puzzled look on the reporter's face.
 3            THE WITNESS:  Well, I knew I misspoke when I said it,
 4   because I'm not too good at names.
 5   A.  Anyway, that was a procedural test to see how hard it was
 6   going to be to do this.
 7            We went out and did another sample, okay, to get
 8   transactions that had not come to us as posted; by going back to
 9   the land or leases or combinations, we were then exploring those
10   issues in Horton.
11            And subsequently, we've done three more of these
12   activities, three more agencies; more complicated revenue
13   streams, et cetera, et cetera.  Do you want me to mention them?
14   Maybe it doesn't matter.  We're learning how to do this, and of
15   course we can't do this ourselves.  The accountants have to come
16   in.
17            So we found the records, the records are ready to be
18   reconciled.  But this is a different kind of accounting, so I've
19   been told, than the accounting that was done for the
20   transactions.  The Accounting Standards Manual doesn't apply to
21   the kinds of activities that were engaged here.
22            And so the accountants have to come together and
23   developed their procedures.  And I believe in the context of
24   earlier activities, they have to be submitted to the Court.
25   Q.  Now, this is work that still needs to be done --
1014
 1   A.  It's under way.  We've done our part.
 2   Q.  Dr. Scheuren, just so we have a clean record, this is work
 3   that you believe still needs to be done with regard to the
 4   electronic ledger era?
 5   A.  Indeed.  And we are doing pilots of that.
 6   Q.  The second item that you identified in your memo is,
 7   "cleanup tests of transactions discovered later to be in the
 8   population of interest but not included in the original sampling
 9   frame."  What were you referring to there?
10   A.  Here again, Michelle is involved.  She has been talking
11   about this for quite some time.  Those cleanup transactions,
12   those added transactions, quite a few of them, need to be tested
13   just like the other transactions that we've already tested in
14   the LSA.  And we recommended that.  They all should be subjected
15   to the same sort of testing that we've been doing for the LSA.
16   Q.  And that includes sampling?
17   A.  That indeed includes sampling, yes.
18   Q.  Dr. Scheuren, I'd like you to refer now -- let's shift
19   topics a little bit to administrative record 426.  This is Bates
20   number 000052-2.  We're looking at a May 25, 2007 memorandum
21   authored by you.  Correct?
22   A.  Signed by me, yes.  Usually authored by -- as I've already
23   indicated, it's authored by Robin and Susan and I.  If you've
24   looked at these memos, and maybe others have noticed this, some
25   are signed by Susan and some are signed by me.  But it's usually
1015
 1   a joint work product of all of us.  This I wrote more of than
 2   some of the others.
 3   Q.  But you put your initials on it.  Right?
 4   A.  Yes, indeed.
 5   Q.  And what was the purpose of this memo?
 6   A.  It was to give them a strategy for the sampling to be done.
 7   And the focus was shifting the emphasis from making an estimate,
 8   which is what we did in the LSA and what we intended to do in
 9   the 2003 plan, to testing a hypothesis that the paper ledger era
10   would produce an error rate similar to the electronic ledger
11   era.
12   Q.  Now, this proposal is with regard to the paper ledger era.
13   Correct?
14   A.  Correct.
15   Q.  What is meant by the paper ledger era?
16   A.  These are the transactions that were not included in the
17   electronic population, as augmented by the work that Michelle
18   has been doing, they were not included in the electronic
19   population.
20   Q.  And this was not part of the LSA, then?
21   A.  No.
22   Q.  I've highlighted in the subject, "initial hypothesis
23   testing."  What is meant by the term "initial hypothesis
24   testing"?
25   A.  We are trying to develop an approach here where -- we don't
1016
 1   claim we know enough to call it -- the word "initial" is just a
 2   starting point.  Okay?  But we want to look at the hypothesis,
 3   as I've already stipulated, that the error rates for these two
 4   eras are virtually the same, same magnitude.
 5   Q.  Why are you wanting to test that hypothesis?
 6   A.  Because it allows us to draw a smaller sample.  And if we're
 7   wrong, we can increase it later.  But if we're right, we save
 8   the money, or the government saves the money.
 9   Q.  Why not simply approach the paper ledger the same way that
10   you did the electronic ledger era?
11   A.  Well, first of all, we know a lot more than we did when we
12   started.  It would be, I think, wasteful not to use that
13   information.  This is a way, not the only way, this is a way to
14   use that information.
15   Q.  On the second page of this document, you have a section
16   captioned, "recommended paper ledger" --
17            THE COURT:  Just excuse me a minute.
18            This hypothesis that the paper ledger era error rate
19   will be the same or roughly the same as the electronic era,
20   whose hypothesis is that?
21            THE WITNESS:  We are proposing that as a hypothesis to
22   the client.
23            THE COURT:  Okay.  But the real hypothesis is -- well,
24   I suppose you can comment on this rather than answer the
25   question.  The real hypothesis is, let's try to do it for less
1017
 1   money, and by doing it for less money, one hypothesis that flows
 2   from the less money assumption is same error rate, which we
 3   contest.
 4            I mean, it's driven more by cost than by anything else,
 5   is it not, this hypothesis?
 6            THE WITNESS:  Cost and time, yes.  I mean, it takes a
 7   long time to do this.  We didn't carry you through this, but in
 8   the earlier LSA effort, we ran out of money.  We started in
 9   January.  Remember I told you we all worked Christmas?
10            THE COURT:  Right.
11            THE WITNESS:  And we started in January and we ran
12   until somewhere in mid August and we ran out of money.  Had to
13   stop.  Okay?  And that gave us a chance to resample, redesign,
14   and resume at a smaller sample size.  So money mattered, and
15   time mattered, too.
16            THE COURT:  Okay.  I got it.
17   BY MR. WARSHAWSKY:
18   Q.  Well, is cost a valid consideration when you design a
19   sample?
20   A.  It always is in all the work I've ever done, sure.  You're
21   dealing with a client who has to put the money out, and the
22   client wants a result within a certain tolerance band, okay, for
23   the resources that they have available.
24            And you go to the client and you see the client doesn't
25   have enough money, and you say, you don't have enough money to
1018
 1   do this; you've got to change your tolerance level or you've got
 2   to get some more money.  We'll do our best to come up with a
 3   good sample design which uses frugally your resources --
 4            THE COURT:  I don't think we have to quarrel about
 5   that, Mr. Warshawsky.  If there's one proposition that is law of
 6   the case, it is that money is relevant.  So proceed from there.
 7            MR. WARSHAWSKY:  Very good.  Very good.
 8   BY MR. WARSHAWSKY:
 9   Q.  So in your basic sample design, you proposed two stages.
10   Correct?
11   A.  Correct.
12   Q.  And would you describe what those are?
13   A.  These two stages are the same two stages we used for the
14   LSA.  We would take a sample of accounts from a list of all the
15   accounts - the problem here is getting the list - and known or
16   believed to contain transactions, and then we'll sample the
17   transactions.
18   Q.  Now, we can refer to the next page, page three.  And you've
19   already described to some extent what I've highlighted there,
20   the purpose of your hypothesis test.
21            What happens if -- or what do you propose if, in fact,
22   you run the test and the hypothesis is accepted, that, in fact,
23   the difference rate for the paper ledger era is statistically
24   similar to or even smaller than the electronic ledger era?
25   A.  We will have designed the sample size to be sufficient, so
1019
 1   that should that hypothesis be accepted, we'll be finished.
 2   We'll be able to produce results that will be satisfactory for
 3   the client's goals.
 4   Q.  And what happens if that hypothesis is rejected?
 5   A.  We're going to need a larger sample, and we'll adaptively
 6   increase the sample to achieve -- after going back to the
 7   client, the client can change their goals, but if they want to
 8   keep their goals what they are, we're going to have to have a
 9   bigger sample.
10   Q.  Now, Dr. Scheuren, you're aware that there's been some
11   debate in the prehearing pleadings about the role of
12   meta-analysis.  Correct?
13   A.  Yes, I am.
14   Q.  What is meta-analysis?
15   A.  Meta-analysis is a technique, one of many, to combine
16   studies from many sources on similar or related subjects to try
17   to draw together what is known about the subject.  And perhaps
18   in drawing it together, to gain some advantage, some additional
19   information from it.
20            If I might elaborate one more time.  I've been alluded
21   a couple of times to the difference in ages.  I began, and
22   sampling began, in an era of where the world was data poor.
23   Q.  Data poor?
24   A.  Data poor.  There was very little data available.  And the
25   sample that was drawn in those days - this is in the '30's, for
1020
 1   example - were small to begin with.  And then the world became
 2   data rich, and then, of course, there's lots of information that
 3   comes from data.
 4            And then, from the information, how we combine all this
 5   information?  We're now moving to that stage in science.  We're
 6   combining information and using techniques, like meta-analysis,
 7   to do that.  Certainly not the only one, but it's certainly a
 8   valid one.
 9   Q.  Is it an accepted technique in the world of statistics?
10   A.  At least accepted, and our reports cited, a dozen or near a
11   dozen reports which describe it.  It's heavily used, especially
12   in the medical area.
13   Q.  I'm sorry, you said your reports.  What are you referring
14   to?
15   A.  We did two volumes, a summary of the work that we did on the
16   meta-analysis.  And in one of those lines, I guess the first
17   one, we talked about or cited other reports that were done,
18   other approaches in the literature on meta-analysis.
19   Q.  What was the role of meta-analysis in terms of formulating
20   your recommendations for the paper ledger era?
21   A.  We were looking for, I think the phrase actually is a trout
22   in the milk.  We're looking for something that said to us, this
23   hypothesis just doesn't work.  Why even start on this without
24   looking to see if there's a major, major result out there which
25   says that this hypothesis isn't going to be satisfactory.
1021
 1            So we did that and we looked at all of the studies we
 2   could find.
 3            So it's basically a due diligence step, and if we had
 4   not found that, we would not -- if we had found something big
 5   out there, we would not have been making this recommendation as
 6   to the hypothesis.
 7            MR. WARSHAWSKY:  Would you pull up Defendant's
 8   Exhibit 7, page 15, please?  And I would like you to blow up the
 9   last three paragraphs there.
10   BY MR. WARSHAWSKY:
11   Q.  This is from your rebuttal report.  Correct, sir?
12   A.  Yes.
13   Q.  And you quoted a statement from Mr. Duncan's analysis there.
14   Do you see that?
15   A.  I do now.
16   Q.  Okay.  Do you agree with the statement that, "the
17   meta-analysis performed by NORC plays a central role in
18   purportedly strengthening and corroborating DOIs conclusions",
19   and then continues on?
20   A.  Adjectives and adverbs are always dangerous.  The word
21   "central," I do find a problem with that.  It plays a role,
22   among a bunch of other things, in order to plan ahead before we
23   actually undertake the early pilot activities, which we've
24   already done some of those, too, on this 2007 plan.  It plays a
25   role, not a central role.
1022
 1   Q.  Okay, sir.
 2   A.  I would say it's a minor role.
 3   Q.  Go ahead.
 4   A.  It's a minor role given what we found.  If we had found a
 5   problem, a serious problem, it would have been -- it would have
 6   stopped us.  Probably would have stopped us.  But that didn't
 7   happen.
 8   Q.  Aside from the meta-analysis work, what activities, if any,
 9   have been undertaken to develop the design for the paper ledger
10   era?
11   A.  One of the things we did was to look at the work done by
12   Treasury and General Accounting Office, and for the period 1890
13   to 1950, 60-year period, to see whether we were going to find
14   problems in that 60-year period.  And that's one of the reports
15   in our administrative record.  In a very one sentence or one
16   phrase way, we didn't find the kind of problems that would have
17   led us to change our mind.
18            Another thing we did, and we're almost finished now,
19   were their reservation histories.  In the 2003 plan we said --
20   or whoever wrote that plan said that there would be reservation
21   histories produced.  They're almost all finished now.  They're
22   very useful to see if there's anything special going on in a
23   particular area that might draw you to a different conclusion as
24   to the initial - important to emphasize "initial" - hypothesis.
25            I haven't read all of those myself.  I've only read a
1023
 1   few of them.  I haven't seen everything yet.  We're not through
 2   the end of that.  Others will look at those.  But I think we're
 3   going to end up probably where we are here.
 4            We've actually done, as I mentioned, some early pilot
 5   activity, looking for some -- you know, you to have to go get
 6   the ledger, remember?  We don't have these ledgers
 7   electronically.  So we have to find a ledger, find the account,
 8   and then follow the account back historically to see what's
 9   going on.  And we've begun that process.
10            Have we done accounting for that yet?  No, we have not.
11   But we're looking at cost issues here, too, after all.  That
12   bears on this.
13            MR. WARSHAWSKY:  Would you put up Defendant's
14   Exhibit 6, please, page 16?
15   BY MR. WARSHAWSKY:
16   Q.  Dr. Scheuren, this is your expert report, page 16 of it?
17   A.  Yes.
18   Q.  And do you recognize the three bullet points on this page?
19   A.  Yes.
20   Q.  And what is set forth in these bullet points?
21   A.  These are three recommendations that grew out of the work we
22   did.  There are many other recommendations we've made to the
23   client, but these seem to be important enough to bring to this
24   level of attention.
25   Q.  And would you summarize the opinions set forth in your first
1024
 1   bullet point?
 2   A.  We've been talking about this a lot.  In the light of the
 3   baseline error rate that we established in the study, we thought
 4   it was wisest -- the word "wisest" is something that I should
 5   have put in quotes because it's not necessarily -- I'm not an
 6   expert on wisdom.  Wisest, since the unit cost of the accounting
 7   ledger was expected to be even higher, that we would start with
 8   a smaller sample size and focus on that as a way of starting.
 9            We had started with a larger sample size in 2003;
10   turned out to be overengineered, enormously overengineered, in
11   our opinion.  We'll start with a smaller sample size, and if we
12   needed to, we would go up in size.
13   Q.  Would you summarize your second opinion?
14   A.  I recommended that we didn't need to do any more
15   transactions from the same population in the electronic ledger
16   era.  I've already elsewhere - and you've drawn this out of me -
17   recommended that we do additional testing in the electronic
18   records era to make sure that the transactions were complete, or
19   complete enough, and to test the new transactions that were
20   added by the work that Michelle's team has done.
21   Q.  And finally, in your third paragraph you talk about the more
22   conventional assurance level of 95 percent.  Do you see that?
23   A.  Because of the importance of the case, and because of the
24   concerns about potentially higher error rates than was initially
25   thought to be the case, we set the assurance level at
1025
 1   99 percent.  And setting it at 99 percent causes you to have a
 2   much larger sample than if you set it at 95 percent.
 3            It's entirely possible, whether you set it at 95 or 99,
 4   to provide the estimate at the other level.  It's just you're
 5   not using it to design the sample size.  If you set it at 95,
 6   you're going to have a smaller sample at the get-go, and it's a
 7   cost issue.
 8            And given what we found out, I think it's reasonable.
 9   And you could always go back and make the estimate at 99.
10   Q.  And how would you do that?
11   A.  Simple.  Mathematically, do it.  I mean, you go -- a
12   95 percent confidence interval, to increase it to a 99 percent
13   confidence level, you multiply it by a constant that depends on
14   the nature of the sample design you're talking about.  But it
15   would be about 30 percent, 40 percent more.
16            THE COURT:  In other words, you keep the sample size
17   the same, you just report different results, whether it's 95 or
18   99 percent?
19            THE WITNESS:  Exactly.  And it's a design issue at the
20   estimation stage.  At the analysis stage, the client can do
21   either one, a very minor change.
22            MR. WARSHAWSKY:  Thank you, Dr. Scheuren.
23            Your Honor, we would move into admission Defendant's
24   Exhibit 6 and Defendant's Exhibit 7.  That's Dr. Scheuren's
25   expert and rebuttal expert report.
1026
 1            THE COURT:  All right.  I'll receive 6 and 7.
 2            (DEFENDANT EXHIBIT Number 6 and 7 were moved into
 3   evidence.)
 4            THE COURT:  Mr. Gingold?
 5            MR. GINGOLD:  Thank you, Your Honor.  Good morning,
 6   Your Honor.
 7            THE COURT:  Good morning.
 8                          CROSS-EXAMINATION
 9   BY MR. GINGOLD:
10   Q.  Dr. Scheuren, my name is Gingold and I represent plaintiffs.
11            I'd like to discuss with you at the outset some general
12   issues.  You talked about cost as being an important factor at
13   least as your task is assigned.  Is that correct?
14   A.  That's correct.
15   Q.  But is cost something that can affect reliability?
16   A.  Of course it can.
17   Q.  How can it do that?
18   A.  Larger sample and more costly sample would increase
19   reliability.
20   Q.  What about replication of results?
21   A.  You want to -- your first few words were a little blurred
22   there.  Can you repeat?
23   Q.  Replication of results?
24            THE COURT:  You said, "what about replication of
25   results."
1027
 1   A.  You mean if you repeat the results?  What do you mean -- I
 2   mean, we talked about replication in the context of sampling in
 3   which we had four replicates.  Is that the sense in what you
 4   mean?
 5   BY MR. GINGOLD:
 6   Q.  No, I'm asking you generally about reliability.  And with
 7   respect to statistical sampling, isn't the measurement process
 8   itself required to be reliable?
 9   A.  Of course it is.
10   Q.  And isn't replication or reproduction of the results
11   important to determine consistency?
12   A.  Replicability I think is the word you want to use.  And the
13   answer is, of course it is.  And in the samples we're drawing,
14   we are producing replicable -- the results can be replicated.
15   Q.  So your position is the ability -- your belief in the
16   ability to replicate them satisfies the requirement of
17   reliability.  Correct?
18   A.  I didn't say that.
19   Q.  Okay.
20   A.  Replicability is one dimension of reliability.  Reliability
21   has a lot to do with the margin of error you have in mind.
22   Q.  I'm sorry, Doctor --
23   A.  May I?  As I've indicated, that's a client decision, and our
24   role as samplers is to meet the client requirement.
25   Q.  You made it very clear that the client determines the nature
1028
 1   and scope of your engagement.  Correct?
 2   A.  Yes.
 3   Q.  And you operate to the best you can within the scope set
 4   forth by the client.  Correct?
 5   A.  Yes.  But I also stipulated the independent role that I
 6   would have not -- we would have not have done this if we had
 7   not -- if we hadn't the ability to provide an independent role.
 8   Q.  I understand that.  But my question was, you operate within
 9   the scope --
10   A.  Yes, I do.  Resources are resources.
11   Q.  Are you familiar with the Reference Manual on Scientific
12   Evidence?
13   A.  Yes, I am.
14   Q.  Are you familiar with the discussion in the reference manual
15   concerning the accuracy of measurements and reliability,
16   referring to the reproduction of results?
17   A.  I don't have that in my head right now.  I mean, you want to
18   provide it to me on the screen?
19   Q.  No, I'm asking if you're familiar with it.  That's all.
20   A.  I'm generally familiar only.
21   Q.  So it's your understanding that the reference manual talks
22   about the ability to reproduce the results, and not the actual
23   production of the results to ensure reliability of the process.
24   Correct?
25   A.  Why don't we divide that up a little bit more?
1029
 1   Q.  Does the reference manual, based on your recollection,
 2   require the reproduction of results to ensure reliability?
 3   A.  The results should be ones you can repeat.  That's correct.
 4   Q.  It is correct, isn't it?
 5   A.  You should be able to get the same -- but you're obviously
 6   doing it with sampling error and so on.  So you may be -- I'm
 7   not really sure what you're asking here.
 8   Q.  I'm asking whether or not for you to state that your
 9   conclusions are reliable, statistically reliable, that you need
10   to reproduce the results.
11   A.  In an estimated way, the expected value is yes.  Okay.
12   Q.  Again, statistics is all estimates.  Correct?
13   A.  Pretty much.
14   Q.  You're not talking about what has actually occurred, you're
15   dealing with probabilities and estimates.  Correct?
16   A.  That is correct.  But there was an interpretation of what
17   you said that I actually go through and reproduce the actual
18   calculations of every step along the way.  Okay?  That is a form
19   of replication as well that's used typically in an IT setting.
20   And I take it you were talking about a statistical setting only.
21   But both apply here, in fact.
22   Q.  But you are testifying as an expert in statistics, not IT.
23   Correct?
24   A.  Correct.
25   Q.  The questions I'm asking you are with respect to what is
1030
 1   stated as your expertise.
 2   A.  Thank you.
 3   Q.  So my question for you is:  If you chose not to replicate
 4   the results, does that affect reliability as discussed in the
 5   Reference Manual on Scientific Evidence?
 6   A.  Internally, our sample already allows us to replicate the
 7   result.  Okay?  A new sample, a fresh, completely new sample is
 8   not necessary, in my opinion, as an expert, to achieve this
 9   standard of replicability.
10   Q.  So if, in fact, the Reference Manual on Scientific Evidence
11   states that reliability refers to the reproductability of
12   results, that's what you believe is the case, not the need to
13   reproduce the results to prove the consistency.  Correct?
14   A.  My answer is yes.  Yes.
15   Q.  And what we're talking about, whether you can possibly
16   produce it or you have, doesn't go to the accuracy of the
17   results, does it?
18   A.  No.
19   Q.  So you can actually reproduce the results 10,000 times, and
20   it's not stating anything about its accuracy, is it?
21   A.  Depending on how you did it.  I mean, what we're doing in
22   probability sampling is developing methods that allow us to
23   measure the accuracy at the same time we do the work.
24   Q.  Because you're providing consistency.  Correct?
25   A.  That's one of the factors, yes.
1031
 1   Q.  But you're not providing a statement of validity of the
 2   results, are you?
 3   A.  You're moving around in your concepts here, and I suppose
 4   that's your prerogative.  The results of validity are somewhat
 5   different from the issue of replicability.  I have a process
 6   which is going to produce a certain result.  Okay?  Is that
 7   result on target?  Is it valid?  That's a separate issue.
 8   Q.  Is it accurate?
 9   A.  That's -- replicability doesn't mean -- here's the result,
10   okay, that I'm going to get from this process, and here's the
11   true value over here.  Okay?  And if on the average this process
12   hits that true value, even though it doesn't in any case, that's
13   valid.  Okay?  All right?  That's valid.  That's validity.
14   Okay?
15            If it's a good process and you've got a nice narrow
16   margin of error and so forth, it's reliable.
17   Q.  If you stepped on the scale 14 times and produced a weight
18   of 300 pounds, it's consistent, isn't it?
19   A.  That's correct.
20   Q.  And it's statistically reliable, isn't it?
21   A.  Yes.
22   Q.  But it's inaccurate, isn't it?
23   A.  Yes.
24   Q.  Thank you.
25            And you're not determining the validity of the
1032
 1   underlying transactions in your statistical analysis, are you?
 2   A.  I'm not an accountant.
 3   Q.  That's right.  Or you're not a consultant who does the
 4   analysis, whether or not he or she is an accountant.  Correct?
 5   A.  Some of the analysis we did do based on the --
 6   Q.  No, no.  You're not doing the analysis of whether or not
 7   there's an error in the transactions.  Correct?
 8   A.  That's correct, we did not do that.
 9   Q.  So whichever consultants, accountants or otherwise, make
10   that judgment, you're accepting that judgment.  Correct?
11   A.  We did look at the quality assurance samples.  We actually
12   designed the quality assurance samples that were used by Graham
13   Thornton, which is one of the accounting firms they engaged.
14            And we've seen that work, we've actually incorporated
15   the errors that they found - very few errors - but we
16   incorporated those errors in the statement of the uncertainty in
17   these results.
18   Q.  But you're a man of precision, aren't you, with regard to
19   what you do and how you testify.  Correct?
20   A.  Doing my best.
21   Q.  You don't determine what is an error and what isn't an
22   error, do you?
23   A.  No.
24   Q.  The consultants and others, including accountants, make
25   those determinations.  Correct?
1033
 1   A.  That is correct.
 2   Q.  So whatever the error result is, is not determined by you?
 3   A.  We analyze the resulting errors that are determined by
 4   others.  That's what we did.
 5   Q.  Right.  So if a transaction that's not on the system is not
 6   considered an error, whatever your acceptable error rate is
 7   doesn't include that.  Correct?
 8   A.  I've already told you that we are going to be looking for
 9   transactions that should have been on the system and were not on
10   the system, and once we have identified them, we will sample
11   them.
12   Q.  That wasn't my question.  And you responded to
13   Mr. Warshawsky in a direct examination.  You weren't talking to
14   me.
15            And you haven't testified as an expert before, have
16   you, Dr. Scheuren?
17   A.  Not in this setting.  My apology.
18            MR. GINGOLD:  If I may, Your Honor.
19   BY MR. GINGOLD:
20   Q.  Your answers should actually be directed to the Court.  I'm
21   asking questions.
22   A.  Certainly.  I knew that.
23   Q.  That's all I'm doing.
24   A.  All right.  Please resume.
25   Q.  Thank you.
1034
 1            If transactions are not on the system, as, for example,
 2   within what is defined as the electronic records era, they are
 3   not viewed as errors, are they?
 4   A.  It's an odd way of saying this.  My answer is, the
 5   population that we're making an inference about, okay -- within
 6   the population that we're making an inference about, they're not
 7   viewed as errors.
 8   Q.  What is your understanding of the definition of the
 9   electronic records era?
10   A.  The original version of this --
11   Q.  How about the version you're working with.
12   A.  You walked away from the mic.  You want to --
13   Q.  Yeah, yeah, I would like you to --
14   A.  The electronic --
15   Q.  If I may, I'm sorry.
16            What is the definition of the electronic records era as
17   you understand it, and how you are working with it in your
18   statistical analysis?
19   A.  The electronic records era, or ledger era, are all those
20   transactions that were electronically available, okay, or have
21   subsequently been added in this work that you've heard Michelle
22   Herman talk about.  They were not --
23   Q.  I'm sorry.
24   A.  That was not the population about which we made our
25   statements for the LSA.  The population that we made our
1035
 1   statements for the LSA were transactions for account holders
 2   that existed on or after October 1994.  Those are the only ones
 3   we looked at.
 4   Q.  If I may, for precision purposes, because this will help and
 5   probably expedite the cross-examination, so what you're talking
 6   about as you understand the electronic records era are, it's
 7   from a date, a period in time, a date certain.  Correct?  1985
 8   to 2000?
 9   A.  That isn't what I said, sir.
10   Q.  Oh, it is not the case?
11   A.  No, it's not what I said.
12   Q.  Okay.
13   A.  What I said -- may I repeat what I said?
14   Q.  Please do.  Because I didn't understand it.
15   A.  I'll leave that alone.  Let me say it this way --
16   Q.  Please.
17   A.  What I said was all of the transactions that were on the
18   system, okay, electronically, or added by Michelle Herman's
19   team, okay, are in the electronic ledger era.  That was not the
20   population that we sampled.  We sampled a smaller population.
21   Q.  So dates are unimportant in your understanding of the
22   electronic era.  Correct?  1985 to 2000 is irrelevant, based on
23   your testimony.  Correct?  That's not the issue, is it?
24   A.  The dates that -- the records that existed on the system,
25   okay, is what I mean by electronic records era.  The 1985 is
1036
 1   actually a tricky problem, as you know.
 2   Q.  Well, please explain.  Because I'm trying to get an
 3   understanding.  Because much of the information that's been
 4   produced in the administrative record and testimony of other
 5   witnesses have suggested that the electronic record --
 6   electronic ledger era is a period of time from 1985 to 2000.
 7   But please explain why my understanding is incorrect.
 8   A.  It is that.  But it's also those transactions which were --
 9   that were in that time period that were electronically
10   available.  Not all of the transactions in that time period
11   remained electronic.  Okay?  Some of them were written over, as
12   you've heard.  I assume you've heard.
13            But the ones that were available or have been added
14   subsequently by Michelle's team, I would consider the definition
15   of electronic records era, or ledger era.
16   Q.  Why did you say the 1985 period is a tricky time period?
17   A.  Because at that point the process that we're talking about
18   began.  But because of the nature of the system, it didn't begin
19   everywhere at the same time.
20   Q.  So you haven't defined the era.  That's not your definition,
21   is it?
22   A.  No.
23   Q.  Who defined the era?
24   A.  I don't think I know the answer to that directly.  I mean,
25   it's been a working definition that I've heard others use, okay,
1037
 1   and we've been looking at a subset of that population, as you
 2   heard me testify to.
 3   Q.  Is it your understanding --
 4            THE COURT:  This is for both of you.  We don't have
 5   time for this sparring.  I have heard the term "electronic era"
 6   described to be a time period between 1985 and 2000.  The
 7   witness says that's true.  I've also heard him say that some
 8   documents created in that period of time are not available
 9   electronically.  I've also heard him say that the LSA -- or
10   excuse me, that his analysis, maybe the LSA, uses a different
11   concept of electronics era that is for transactions that
12   occurred on or after October 1994.  Is all that correct?
13            THE WITNESS:  Yeah.
14            THE COURT:  Let's move on.
15   BY MR. GINGOLD:
16   Q.  One last clarification.
17            MR. GINGOLD:  Your Honor, there's only one reason I was
18   asking the question.  The electronic era and the IRMS system
19   began in the 1970's, so I was trying to understand why it
20   started in 1985.
21            THE COURT:  If he knows.
22   BY MR. GINGOLD:
23   Q.  So do you know why the electronic era, as defined and as it
24   has been --
25   A.  I don't know.
1038
 1   Q.  And as a result, the pre-1985 transactions on the IRMS
 2   system are part of the paper ledger era.  Is that correct?
 3   A.  That's the definition we're using, yes.
 4   Q.  And you don't know why, do you?
 5   A.  I've answered that.  The answer:  I don't know.
 6   Q.  Okay.  Thank you.
 7            What is your target population for the electronic
 8   records era?
 9   A.  Well, the target population ultimately will be all of the
10   records.
11   Q.  Do you have a population number in mind with regard to
12   transactions?
13   A.  I don't think we're ready -- we're finished yet.  I don't
14   think Michelle is finished yet, so I don't know the number.
15   Q.  Have you heard the term 28 million?
16   A.  I don't think that's the number, but no, I haven't.  I don't
17   know what that means.  There's lots of numbers out there.
18   Q.  Okay.  So with respect to -- you don't have a target
19   population of transactions yet, do you, for the electronic
20   records era?
21   A.  That's correct.  It's coming close, though.
22   Q.  Do you have an understanding of the number of transactions
23   for the paper records era?
24   A.  There have been estimates made.  In fact, I'm one of the
25   people that made those estimates.  And they're subject to a
1039
 1   great deal of uncertainty.
 2   Q.  So is your answer you don't know what the number of
 3   transactions are at this time?
 4   A.  There are -- estimates have been made.  The answer is no, I
 5   don't know.
 6   Q.  Thank you.
 7            Is it true that the further back in time you go, there
 8   is a greater problem with regard to finding records?
 9   A.  I don't know that.
10   Q.  You don't know that?
11   A.  In fact, may I conjecture?
12            THE WITNESS:  Am I allowed to conjecture, Your Honor?
13            THE COURT:  No.  If you don't know, you don't know.
14   BY MR. GINGOLD:
15   Q.  With regard to the determination -- or your hypothesis, I'm
16   sorry, that the paper records era has similar characteristics to
17   the electronic record era transactions, you stated, I believe,
18   that the meta-analysis is a minor part of your hypothesis.
19   Correct?
20   A.  That's correct.
21   Q.  And I believe you stated -- and this is not a direct
22   restatement of what you testified to, but I think you've
23   referenced the GAO settlements package.  Is that correct?
24   A.  GAO Treasury settlement packages, yes.
25   Q.  Right.  Is that a minor part as well?
1040
 1   A.  It actually plays somewhat more of a role, I would say.
 2   It's pretty good work.  We liked what we saw, but it's still a
 3   minor role, yes.  Still minor.
 4   Q.  So the meta-analysis is minor and GAO's settlement packages
 5   are?
 6   A.  Also minor.
 7   Q.  And you then I think stated reservation histories.  Correct?
 8   A.  Correct.
 9   Q.  And is that also minor?
10   A.  Yes, also small.  Yes.
11   Q.  So you identified three particular points that support your
12   hypothesis that are all minor.  What is the major basis of your
13   hypothesis?
14   A.  Of course, those are additive in my -- they are additives.
15   They all add up to -- the word "minor" is -- I would use the
16   word "small."
17            The fourth one I mentioned to you is that we've been
18   doing some early pilot activity, okay, to see whether what we're
19   saying about the paper ledger era is going to turn out to be the
20   case.
21   Q.  And the pilot work is representative, do you believe?
22   A.  No, it's not representative at this point.  We're still
23   learning how to do this.
24   Q.  Okay.  So is this part of the LSA?
25   A.  No.
1041
 1   Q.  So this is independent of the LSA?
 2   A.  It's independent.
 3   Q.  So you've identified four factors which are minor.  They all
 4   add up to be your conclusion.  Is that what that is?
 5   A.  They make it possible for us to support the idea of starting
 6   with a test rather than with a large sample.  We might end up
 7   with a large sample at the end.
 8   Q.  So you've taken samples right now in your pilots to allow
 9   you to reach the conclusion that the hypothesis is correct.  Is
10   that fair?
11   A.  We haven't got that far yet, but we will get that far.
12   Q.  So you haven't done a sample?
13   A.  We're offering the plan.  We haven't completed it.
14   Q.  What I'm asking you is a little different, though.  I'm
15   asking you, your hypothesis, as I understand it, is that the
16   paper ledger era is similar to the electronic ledger era.
17   Correct?
18   A.  That's correct.  In errors, in error rates, yes.
19   Q.  Sufficient for you to be able to apply whatever procedures
20   you've used in the electronic era to the paper era.  Correct?
21   A.  No, that's not what we're doing.  We're starting with a
22   different idea altogether.  The hypothesis testing is not --
23   we're not making estimates as our main focus, we're trying to
24   look at whether -- what we're going to learn is similar in error
25   rate, not in process, to what we did earlier.
1042
 1   Q.  So it has a similar characteristic, and that characteristic
 2   is --
 3   A.  At the end of the day, yes.
 4   Q.  And you haven't taken any samples yet.  Correct?
 5   A.  No.
 6   Q.  So what you've done -- I'm sorry.  Go ahead.
 7   A.  My apology.  Not representative samples of sufficient size
 8   to make us feel comfortable at this point statistically.
 9   Q.  Have you taken samples of any size?
10   A.  Yes, we have.
11   Q.  Where have you done that?
12   A.  We've been looking at them.
13   Q.  Beg your pardon?
14   A.  We've been looking at them.  We went out and got some
15   ledgers.  I didn't do this myself.  We obtained some ledgers and
16   we've examined the information there.
17            We need to bring, I think -- and we're moving away from
18   my expertise here.  We need to bring the accountants together to
19   look at whether the Accounting Standards Manual needs to be
20   modified as you move backward in time to the paper era.  That
21   hasn't happened.
22   Q.  That hasn't happened yet?
23   A.  No.
24   Q.  And whatever you're doing, whether it's the electronic
25   ledger era or the paper ledger era is dependent upon the results
1043
 1   of what the accountants and other consultants do and conclude.
 2   Correct?
 3   A.  Yes.
 4   Q.  And if their work happens to be wrong, then your hypothesis
 5   may need to be changed.  Correct?
 6   A.  The answer is yes.  Although I might add, if I might add,
 7   that there's a lot of checking going on in this process.  It's
 8   not a single accountant, or accounting firm, even, making these
 9   decisions.
10   Q.  No, but the result is the same.  If the accountants and the
11   consultants are correct, that will help support your hypothesis,
12   and if they're not, you have to make changes.  Correct?
13   A.  That's correct.
14   Q.  With regard to the meta-analysis, since it's one of the four
15   you've identified as supporting your hypothesis, you have stated
16   in your rebuttal report and you've stated in other reports -
17   that's you and NORC - that you've reviewed approximately 900
18   studies.  Is that fair to say?
19   A.  The word "reviewed" needs to be modified.  We've discovered,
20   we've identified about 900 studies.  We've reviewed I think 800
21   something.  I don't have that number in my head.
22   Q.  So the references to 900 in the various reports only refer
23   to the fact that you've identified them, not reviewed them.
24   Correct?
25   A.  You have looked at the reports.  You know the answer to that
1044
 1   is yes.
 2   Q.  If you can please answer the question.
 3   A.  My answer is yes.  Yeah, we tabulate exactly what we did in
 4   the reports.
 5   Q.  Does anyone in your firm review -- I mean, did your firm,
 6   either individually or collectively, review all of the reports
 7   that you identified?
 8   A.  No one person looked at all the reports.
 9   Q.  Collectively, did NORC look at all the reports?
10   A.  Remember, some of the reports that we identified as existing
11   were ones we could not find.  They're out of file.  And some of
12   them that we found, after quick looking were simply not
13   applicable.  And the ones that were applicable were reviewed and
14   analyzed, and are in the report.
15   Q.  And did you, yourself, review any of the reports, Doctor?
16   A.  A couple dozen, yeah.
17   Q.  Were they judgmentally selected for review or randomly
18   selected?
19   A.  Some in early going were randomly selected.  Then I began to
20   ask questions, do you have anything like this, anything like
21   this, and people would bring things to me.
22   Q.  Did your firm create a database with the reports and the
23   summaries that were prepared for the reports?
24   A.  The reports were scanned.  Okay?  And a summary and a
25   database was created, a partial database was created.  The work,
1045
 1   as you know because we've had exchanges of correspondence, was
 2   not finished in terms of a complete database.  In one of our
 3   reports, said we were going to do this, but it fell in priority
 4   such that it has not been finished.
 5            MR. GINGOLD:  For the record, the correspondence was
 6   between the Department of Justice and plaintiffs' counsel.
 7   Dr. Scheuren was not involved in it, as far as I know.
 8   BY MR. GINGOLD:
 9   Q.  With respect to the database, it was not then utilized as a
10   searchable database by NORC.  Is that a fair statement?
11   A.  It was used but not as a searchable database, no.
12   Q.  So any suggestions that it was may not be precisely correct.
13   Correct?  Any suggestion that it had been used as a searchable
14   database is not correct.  Is that a fair statement?
15   A.  We didn't say it had.
16   Q.  We can go into that later, but I just want to make sure I
17   understand what the facts are.
18            MR. GINGOLD:  I would like to identify a document, it's
19   called -- it's NORCMA, underline, and the last three digits are
20   231.
21   BY MR. GINGOLD:
22   Q.  Doctor, do you recall seeing this particular -- this is --
23   for purposes of clarity, this is a summary of what appears to
24   have been performed by your firm with regard to one of the
25   documents that was produced as a meta-analysis document.
1046
 1            Have you ever seen this?
 2   A.  I've seen this template before.  I'd have to look carefully
 3   to see if I had seen this specific document.
 4   Q.  The template is a template that was prepared by your firm.
 5   Is that correct?
 6   A.  Correct.
 7            MR. GINGOLD:  And I would like to go to what would be
 8   identified on the Bates number as 234.
 9   BY MR. GINGOLD:
10   Q.  Did you recall seeing this document at all, Doctor?
11   A.  I don't have any recollection of that.
12   Q.  Was it a problem in your firm when you were reviewing these
13   documents that the documents were missing page numbers?
14   A.  We discussed that in our analysis.  And we separated those
15   where there were page numbers missing from the source -- quote,
16   "source document" on original.  They weren't always original.
17   That was the problem from those where there were significant
18   gaps.  In those that there were significant gaps, we made them
19   out of scope.  We couldn't analyze them.  But when there was
20   just a page or two here and there, we were able for our purposes
21   to be able to make a determination.
22   Q.  Do you have an understanding of how many documents were
23   missing pages?
24   A.  I have an understanding, and we have reported how many were
25   sufficiently incomplete that we couldn't use.
1047
 1   Q.  Do you remember --
 2   A.  Those which have a random page here or there missing, I
 3   don't have that estimate.
 4   Q.  But the significant pages that were missing, how many
 5   documents, do you know?
 6   A.  It's in the report.  I don't have that in front of me.
 7   Q.  You also said, I believe, that you had an accountant
 8   review --
 9   A.  We did.
10   Q.  -- the reconciliations.  What is the name of the accountant?
11   A.  Carolyn Haase.
12   Q.  Could you please spell that?
13   A.  H-A-A-S-E.
14   Q.  And what firm is Carolyn Haase with?
15   A.  She had been controller at a small law firm, and we asked
16   her to come to work for us to help us do this.  She wanted to
17   make a transition, she agreed to.  She came with us for eight or
18   nine months, I forgot how long, and then she worked for Deloitte
19   for another year on the historical accounting.
20            And now she is a controller, maybe chief financial
21   officer, I'm not sure what the title is, in another small firm.
22   Q.  And she's a certified public accountant?
23   A.  She is not.  She has a master's degree in accounting.
24   Remember, she didn't do any accounting, all she did was help us
25   categorize these reports.
1048
 1   Q.  Well, I just asked whether or not she was a certified public
 2   accountant.
 3   A.  I've answered that.
 4   Q.  Is it your understanding that it is important that computer
 5   systems have adequate document controls in order to rely on the
 6   data housed in those systems?
 7   A.  I'm not sure what you're going to here.  But computer
 8   systems need to be adequately documented, yes.
 9            MR. GINGOLD:  I would like to call up Bates 5520-1, and
10   I would like to go to a document which is 12, page 12 of the
11   document.
12            THE COURT:  You're calling up documents by Bates
13   numbers.  They're not in the administrative record.  I don't
14   know whether they have exhibit numbers or not.
15            MR. GINGOLD:  The only thing we have is a Bates number
16   out of the administrative record, and it's 5520 --
17            THE COURT:  You're just blowing them by the witness on
18   the witness stand.  They're not going to be exhibits.  If you're
19   going to move them into evidence, you need to put exhibit
20   numbers on them so we can keep track of them.  Are they on your
21   exhibit list?
22            MR. GINGOLD:  Yeah, these are from the administrative
23   record.
24            THE COURT:  Oh, from the administrative record.  All
25   right.
1049
 1            MR. GINGOLD:  That's all we're using, Your Honor.
 2            THE COURT:  That's fine.  Okay.
 3            MR. WARSHAWSKY:  Your Honor, just for clarification, in
 4   some cases, I think, for example, the NORC document that you
 5   referred to earlier, some of those were not, I believe, in the
 6   administrative record.  But the Bates numbers that he's utilized
 7   right now, this one is.  I can certainly give the AR --
 8            THE COURT:  If somebody says Bates number, I'm to
 9   understand that it's in the administrative record?
10            MR. GINGOLD:  It's my understanding that everything we
11   received in the administrative record is Bates stamped.
12            MR. WARSHAWSKY:  And just to be clear, you used a
13   document with a number NORC, I think, earlier?
14            MR. GINGOLD:  That's correct.
15            MR. WARSHAWSKY:  Was that in the administrative record?
16            MR. GINGOLD:  No, it was not.  That was subsequently
17   produced.
18            MR. WARSHAWSKY:  Right.  That's what I wanted to be
19   clear about.  Where you have a Bates number, I can certainly
20   give a cross reference to the administrative record here.  I
21   have that here.  I can look it up.
22            MR. GINGOLD:  Your Honor, what we will do --
23            THE COURT:  Do you think Bates has anything to do with
24   these numbers anymore?
25            MR. GINGOLD:  Your Honor, there were approximately
1050
 1   17,000 pages of documents in the administrative record that were
 2   produced.  When we input the information in the system, it was
 3   input -- it was not segregated other than by Bates number at the
 4   time we received the information.
 5            There have been some subsequent productions, but it was
 6   a significant process.
 7            THE COURT:  Fine.  I was just musing on whether Bates
 8   really deserves to have his name on all those numbers that we're
 9   using.
10            THE WITNESS:  Clearly from a different century.
11            THE COURT:  Really, it was.  And I remember that
12   century.  Go ahead, Mr. Gingold.
13            MR. GINGOLD:  It was only a couple of years ago,
14   Your Honor.
15   BY MR. GINGOLD:
16   Q.  Let's look at page one.  Page one of the document,
17   Dr. Scheuren --
18   A.  How do I know what I'm supposed to have in front of me?
19   Q.  This is page one.
20            Now, do you recall sending this e-mail 10/3/01 to
21   Bert Edwards?  That's October 3, 2001.
22            I would like you to turn --
23   A.  I need a little longer with this, please.  This is a long
24   time ago.  Yes.
25            MR. GINGOLD:  If we can please call up page 12.  I
1051
 1   would like to focus on the last paragraph.
 2   BY MR. GINGOLD:
 3   Q.  Do you see, Doctor, the head note is, "period of Legacy
 4   (1985 to 1998) or existing (1998 to date)."  Do you see that?
 5   A.  Yes, I do.
 6   Q.  And this refers to computerized accounts.  Correct?
 7   A.  Yes.
 8   Q.  I would like you -- let's see, one, two, three, four, five
 9   lines from the bottom.  Did you write this, by the way, Doctor?
10   A.  Yes, I did.
11   Q.  Okay.  And it states as follows:  "To make full use of these
12   computer files for a historical accounting, we would need to
13   check on how the continuing accounting data are being collected
14   now and what is being done to limit data capture and other
15   errors.  How are such errors prevented in earlier legacy
16   systems?  What procedural handbooks and documented system checks
17   exist?  Can a conformance audit be used rather than relying
18   mainly on a search for supporting transaction detail?  Bottom
19   line, here is how tight are, (were), the controls for these
20   computer systems, and hence, how much reliance can be placed on
21   them?" Close, question mark.
22            Do you recall saying that?
23   A.  Yes.  It's actually in the first predesign document that
24   NORC has.  I don't know where this came from, but the final
25   version was in the predesign.
1052
 1   Q.  So that statement is correct, isn't it?
 2   A.  I said that.
 3   Q.  And you believe it was correct then and it's correct now.
 4   Correct?
 5   A.  That's a little bit of an interest to say.  The answer is
 6   basically it's correct, still.
 7   Q.  Thank you.
 8   A.  But the issues there are different now than they were then.
 9   Q.  Can you tell me if it is no longer true that "the bottom
10   line is how tight were the controls for these computer systems,
11   and hence, how many reliance can be placed on them?"  Is that
12   true?
13   A.  What we're talking about here is a contrast between two
14   different approaches to doing this work.  One is conformance
15   audit, which if the controls were tight enough, you wouldn't
16   have to do a reconciliation.
17            And the other is, if you're not sure about the
18   controls, you can do -- you have to do a reconciliation.  Much
19   more expensive process.
20   Q.  So basically what you're still saying here isn't -- remains
21   to be true, isn't it?  What you said here is true today?
22   A.  There were two approaches, and that is true.  There were two
23   approaches.
24   Q.  Thank you.
25            Now, do you, yourself, have any knowledge whether or
1053
 1   not the IRMS system has had sufficient controls?
 2   A.  I do not.
 3   Q.  Do you know if any of the other systems that house the data
 4   had sufficient controls?
 5   A.  Not in the way you're using the phrase, no.
 6   Q.  How about the way you're using the phrase?  We're using your
 7   terms now.
 8   A.  The answer would still be the same.  I do not know.
 9   Q.  Did anyone inform you whether or not they had sufficient
10   controls?
11   A.  No.  We did undertake to do some testing of the systems,
12   okay, that we're looking at to see what kind of results we could
13   have expected from them, but the issue of control is really
14   beyond our scope.
15   Q.  It's beyond your scope?
16   A.  That's correct.
17   Q.  And you're only making conclusions about the consistency of
18   the information, not about the accuracy of the data.  Correct?
19   A.  We're going back to that again?
20   Q.  Well, I'm just trying to understand.
21   A.  My answer is, if I have a transaction that I'm testing with
22   supporting documents, and I find the supporting documents, okay,
23   even though it's very difficult to find them, okay, but I do
24   find them, okay, I'm going to be -- if I'm an accountant --
25   Q.  Doctor, you're not an accountant.  We're talking to you
1054
 1   about your opinions --
 2            MR. WARSHAWSKY:  Objection, Your Honor.  The witness --
 3            THE WITNESS:  You're asking me a question outside of my
 4   scope, then.
 5            THE COURT:  Let him finish his answer, counsel.
 6            THE WITNESS:  May I finish?
 7            THE COURT:  Yes.
 8   A.  And the accountants actually are pulling this information
 9   from a system, okay, and they're making a comparison with a
10   transaction that's on an electronic database, and they end up
11   concluding that the two match, okay, or that they don't match.
12   But they end up concluding something that they believe is true.
13   That's what I relied on when we did our work.
14            If they had a different kind of system, then you could
15   use a different kind of approach, but that kind of approach was
16   not judged suitable.
17   BY MR. GINGOLD:
18   Q.  So the answer is the accountants or consultants are making
19   those conclusions.  Correct?
20   A.  That's correct.
21            MR. GINGOLD:  I'd like to turn to what is identified as
22   Bates number 421.
23   BY MR. GINGOLD:
24   Q.  And I would like to first point out the first page, which is
25   page one.  This is a January 29th, 2007 memorandum from you to
1055
 1   Jeffrey Zippin.
 2   A.  Yes.
 3   Q.  Do you recall this, document?
 4   A.  Yes, I do.
 5            MR. GINGOLD:  I'd like to turn to Bates number two,
 6   footnote one.
 7   BY MR. GINGOLD:
 8   Q.  This is similar to what you testified in response to a
 9   question from Mr. Warshawsky.
10   A.  Yes.
11   Q.  Do you see that?
12   A.  Yes, I do.
13   Q.  It talks about when you were at the IRS and how you viewed
14   taxpayers as whether or not they were trustworthy?
15   A.  Yes.  This is --
16   Q.  And what did you say in this footnote, Doctor?
17   A.  In our IRS background, all instances of unsupported
18   transactions could be disallowed.
19   Q.  And why is that?
20   A.  Pardon me?
21   Q.  Why is that?
22   A.  Because there was no trust in the taxpayers involved.
23   Q.  Does that also apply to systems, if the systems are
24   untrustworthy?
25   A.  That wasn't the context in which I used this.
1056
 1   Q.  I asked you if it also applies to the systems.
 2   A.  I don't have the answer to that question.  That's not what
 3   we were addressing here.
 4   Q.  But you've made -- I think you've made statements in your
 5   opinion, or rebuttal opinion, and also in other reports that
 6   you've filed with the Department of Interior that I think you
 7   found that the systems have been performing, or there's no
 8   evidence that they haven't been performing.  Do you recall
 9   making those statements?
10   A.  I do.
11   Q.  And are you referring in that regard to the paper ledger era
12   when you made those statements?
13   A.  In that particular context, which is I'm referring to the
14   system as a whole, yeah.
15   Q.  So that would include both the paper and the electronic?
16   A.  Yes, sir, yeah.
17            MR. GINGOLD:  I would like to call up an exhibit which
18   is identified as PPX4174.
19   BY MR. GINGOLD:
20   Q.  Doctor, just for clarification, this was not one of the GAO
21   reports that was included in your meta-analysis.  Have you ever
22   seen this before?
23   A.  I'm not sure.  What's the date on it, 1982?
24   Q.  1982, that's correct.  So it is shortly before the defined
25   electronic ledger era.  Do you see that?
1057
 1   A.  Uh-huh.
 2   Q.  Doctor, I would like you to turn to what would be the
 3   page 2-I of this exhibit.  And there's a head note.  It says,
 4   "Causes of the System's Breakdown."  Do you see that?  Sorry.
 5   A.  Thank you.
 6   Q.  And I would like to read the first sentence.  "GAO
 7   identified two basic causes for the breakdown of the accounting
 8   system."
 9            I'm trying to move this a little rapidly, but this
10   deals with the Individual Indian Trust, and also the Tribal
11   Trust systems.
12            MR. GINGOLD:  If the Court would like me to go through
13   them in more detail, I will.
14   BY MR. GINGOLD:
15   Q.  But in any event, were you aware that at least the
16   Controller General in 1982 believed the accounting systems had
17   broken down?
18   A.  I'm aware of the general work done by other organizations
19   who have audited this system over the years.  Okay?  And --
20   Q.  So you're aware there were at least identifiable problems.
21   Correct?
22   A.  Oh, yes.
23   Q.  But you hadn't been told the systems had broken down, were
24   you?
25   A.  I remember that phrase, yeah.
1058
 1   Q.  I would like to turn to page 3-I, second paragraph from the
 2   bottom above the head note.  And I would like to read the first
 3   part of it, the first few, I think, five lines of the paragraph.
 4            "These matters have been pointed out repeatedly over a
 5   number of years by GAO, Interior's inspector general, and
 6   various system study groups.  Bureau managers were well aware of
 7   the problems and acknowledged that the information in the
 8   automated accounting and finance system was unreliable."
 9            Had you been informed that by your client, the
10   defendant?
11            THE COURT:  Mr. Gingold, this is a 25-year old
12   document.  Right?
13            MR. GINGOLD:  That's correct, Your Honor.
14            THE COURT:  I gather this is all in aid of your
15   proposition that if was once unreliable, you can consider it
16   unreliable for all purposes, since you used to work for the IRS.
17   Is that what you're saying?
18            MR. GINGOLD:  Not entirely, Your Honor, no.  It was
19   unreliable at that point in time.  If it has ever become
20   reliable, it should be viewed as reliable from that point
21   forward, but not for the historical data.
22            THE COURT:  So you're focused on old-fashioned
23   pre-electronic era information?
24            MR. GINGOLD:  Your Honor, that was -- your hypothesis
25   with regard to the paper records era, as I understand
1059
 1   Dr. Scheuren, is that the characteristics in the paper era are
 2   similar to the electronic era.
 3            THE COURT:  And you want him to say that was not a
 4   reasonable hypothesis because of all this old stuff from the
 5   GAO.  Right?
 6            MR. GINGOLD:  Well, the question is whether or not his
 7   hypothesis is supportable based on four points that he
 8   identified which support his hypothesis.
 9            THE COURT:  Well, you know, rather than showing him
10   these 25-year old words, seems to me that you ought to stick
11   within his expertise, which is statistics.  I think he's told us
12   where the hypothesis came from.  It came from the dollars.  You
13   try the hypothesis, if it works, fine; if it doesn't work, you
14   look at something else.
15            We keep going back to these old documents, which
16   frankly aren't doing much for me.  Let's break for lunch.
17            (Recess taken at 12:31 p.m.)
18   
19   
20   
21   
22   
23   
24   
25   
1060
 1               CERTIFICATE OF OFFICIAL COURT REPORTER
 2   
 3            I, Rebecca Stonestreet, certify that the foregoing is a
 4   correct transcript from the record of proceedings in the
 5   above-entitled matter.
 6   
 7   
 8   
 9   _______________________________              _________
10   SIGNATURE OF COURT REPORTER                  DATE
11   
12   
13   
14   
15   
16   
17   
18   
19   
20   
21   
22   
23   
24   
25