For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

 OFFICIAL TRANSCRIPT PROCEEDING 

 FEDERAL TRADE COMMISSION 

MATTER NO. P044804 

TITLE FACTA IMPLEMENTATION 

DATE RECORDED: JUNE 30, 2004 

TRANSCRIBED: AUGUST 4, 2004 

PAGES 1 THROUGH 252 

 FTC ROUNDTABLE 

 A REVIEW OF METHODOLOGIES THAT ASSESS ACCURACY AND 

 COMPLETENESS OF CREDIT REPORTS 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

1

 FEDERAL TRADE COMMISSION 

 I N D E X

ROUNDTABLE WORKSHOP: PAGE: 

Session I 7 

Session II 79 

Session III 130 

Session IV 187 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

2

 FEDERAL TRADE COMMISSION 

In the Matter of: ) 

FACTA Implementation ) Matter No. P044804 

 ) 

------------------------------) 

June 30, 2004 

 

The following transcript was produced from a 

live tape provided to For The Record, Inc. on July 27, 

2004. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

3

 P R O C E E D I N G S 

 - - - - -

MR. VANDER NAT: -- mandates from Congress, a 

mandate called Section 319 of the Fair and Accurate 

Credit Transactions Act and it's very brief. It's easy 

to say. There, the sound is finally coming in, okay. 

It's easy to say what the mandate is, it's very 

hard to do. It's a very daunting mandate for us. I'll 

just take a moment to read it to you, although I'm sure 

you've read it before. 

The Commission shall conduct an ongoing study 

of the accuracy and completeness of information contained 

in consumer reports prepared or maintained by consumer 

reporting agencies and methods for improving the accuracy 

and completeness of such information, and that is the 

extent, in essence, of the mandate. I'm sure that a 

number of you have read it and thought, oh, my goodness. 

It is our understanding that Congress 

understands this to be a very substantial undertaking and 

this is undergirded by the fact that it has an 11-year 

time horizon. I don't know of many studies that have an 

11-year time horizon and I wonder how many of us will 

still be involved with this 11 years from now. I 

probably will not be, although I may be here for a chunk 

of those years. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

4

We know that credit reporting issues are very 

complex and that there are strongly held positions and 

that these are hot button items, all right? And so, we 

recognize that we'll solicit a variety of opinions and 

that we need your expertise. I think most of you know 

that our very first report to Congress is due this 

December 1. The mandate has many reports due. The first 

one is due this December 1 or December 4. And it will 

come as no surprise to you that in that first there will 

not be any assessment of accuracy and completeness. 

Basically, in that first report, on Section 319, we will 

simply set forth a plan for a baseline study next year 

and we're hoping that this roundtable here today will 

give us a lot of good input, including critique on how we 

may best undertake such a baseline study. 

In a moment, we're going to start with our 

presentations, but I want to make a few ground rules 

which we will call, aptly, the rules of engagement. I 

think we all heard this phrase before. The rules of 

engagement. And this may be appropriate since credit 

reporting issues are indeed complex and they are hot 

button issues, and people tend to disagree about how 

things ought to be done. 

So, the first thing I want to say to you is, 

please recognize that we're not here to belittle the work 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

5

of anyone else. We recognize that because of the 

complexity, any study is bound to have shortcomings, 

critiques, ours will, too. So, let's understand that 

we're going to behave like professionals and we're going 

to talk civilly and if things get out of order, I have a 

little bell. 

(Ringing bell.)

MR. VANDER NAT: And if you hear the bell, that 

means things are not going quite right here. I thought 

about a gavel, but a gavel is too ponderous. It's too 

heavy-handed. A bell is, I think, just the right touch. 

Have a look at your agenda for a moment. I 

think you all picked one up. The one that you picked up 

at the table this morning is the latest and final, final 

agenda. You know in government work, we always have the 

final, final product. It takes a while to get there. 

So, this is the final, final agenda. It has only small 

variations from the ones you saw before, all right? 

The sessions are similar in this regard, that 

each session has about 35 minutes of group discussion. 

The first session is different in the sense that it's 

longer. The first session runs 95 minutes. We thought 

we'd get a lot done while you are all still fresh and 

awake. And the other sessions have 80 minutes each. But 

no matter what session we have, there are going to be 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

6

about 35 minutes of group discussion. 

Now, the members at the table, people who have 

name tags at the table, will be given first chance to 

speak during any discussion period. We're hoping that we 

will have some time for comments from the general 

audience. We will try mightily to get there, but if the 

discussion is going well and people are having lots of 

good back and forth at the table, we may or may not get 

to the general audience. But if you feel that you had 

some comment to make that we missed that we should have 

taken note of, but we didn't get to you, please submit it 

to us in writing and we will be happy to give it very 

careful consideration, and we realize that there are many 

things that should have been said or could have been said 

today that probably we won't get to. 

During the actual discussion period, I'm going 

to ask members at the table to please raise their hand if 

you want to speak, all right? And then I'll acknowledge 

you and state your name. You don't have to state your 

affiliation for every member of the table. We have a 

master list, we know the affiliation. But if you want to 

say it, it's fine. But in the interest of time, just 

state your name and then go on with your comment, 

question or observation. 

Finally, in terms of our thinking here today, 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

7

the FTC is going to be making a presentation -- a first 

presentation of the morning, and it will deal with the 

direct review of consumer reports by consumers. But I 

want you to know that we have not made up our mind on any 

specific methodology. We're open to the proposition that 

a number of methodologies may have to be used in a 

baseline study. We're just being transparent in our 

first presentation. Transparency is good today. It's 

good for government to be transparent. So, we're letting 

you know what we're thinking about, but these are not our 

last thoughts. These are thoughts in which we invite 

your critique and we know that a number of modifications 

no doubt will come and that an 11-year project will have 

a number of studies associated with them. 

Okay, with that said, I'd like to introduce to 

you our first presenter for today, Mr. Gerry Butters. He 

will present to us some thoughts on the use of consumer 

surveys to measure the accuracy and completeness of 

credit reports. 

SESSION I: METHODOLOGY FOCUSING ON CONSUMER 

 REVIEW OF CREDIT REPORTS

MR. BUTTERS: Thank you, Peter. I would also 

like to thank all the participants here for coming to 

help us to respond to Congress' mandate to measure the 

accuracy and completeness of credit reports. Our primary 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

8

goal for this year is to develop at least one, and 

probably more than one, method to use to measure accuracy 

and completeness. As one of these methods, we are 

investigating the use of consumer surveys. 

In my presentation, I will describe some of our 

thoughts about how we might survey consumers and deal 

with the many problems that a survey approach must 

overcome. We are eager to hear your ideas, also. As 

Peter said, we're not sold on consumer survey as 

necessarily the only or the best approach, but please 

note, we don't have any choice. We have to do some kind 

of a study. We can't tell Congress that just because the 

measurements will have lots of errors and problems, we're 

not going to do it. So, if you've got ideas for a better 

design, even if it takes a different approach, we want to 

hear them. 

So, our study goal is to measure the accuracy 

and completeness of credit reports. What do we mean by 

an inaccuracy? We mean an error of commission, something 

that is there that shouldn't be there that causes the 

consumer's credit score to be altered. By 

incompleteness, we mean an error of omission. Some 

useful piece of information that is generally collected 

is missing and that causes the consumer's credit score to 

be altered. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

9

And what do we mean by credit reports? We mean 

credit reports in the usual sense, the reports that are 

used by lenders, employers, landlords and others to help 

them determine with whom to do business and sometimes, 

also, what prices to charge. We plan to focus on the 

reports of the big three, Equifax, Trans Union and 

Experian. 

How then do we propose to use a consumer survey 

to measure the accuracy and completeness of credit 

reports? We propose to ask a large number of consumers, 

chosen nationally, and as representative as possible of 

the national population, to review their own credit 

reports with the assistance of experts to identify 

possible inaccuracies and incompleteness and to examine 

each disputed item carefully to resolve conflicts in the 

consumers' views in the records of the repositories and 

the furnishers that supply this information. We propose 

to measure the effect of these problems on the consumers' 

credit scores. 

So, going back to the -- using the credit score 

as a measure, why do we want to do it this way? The 

purpose of credit reports is to help lenders determine 

whom to lend to and on what terms. When lenders make 

these decisions, they usually use the consumer's credit 

score as a summary of the most important information in 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

10

the file. Mistakes in the credit report that have no 

effect on the credit score are likely to have no effect 

on the consumer. Mistakes that have large effects on the 

credit score will have a major effect on the consumer. 

Therefore, it makes sense to measure the importance of 

inaccuracies and incompleteness by the magnitude of their 

effects on the consumer's credit score. 

So, that's our rationale. We realize that 

everything is going to be somewhat an oversimplification. 

 There are many different credit scores. There are 

decisions that might be made that aren't determined 

through the credit score, but this is where we're 

starting, and if you have other ideas, we want to hear 

them. 

What are the advantages of the consumer survey 

approach? The consumer is the single greatest expert 

about his or her own credit history. To get comparable 

information from other sources, one might need to contact 

many furnishers and speak to people with no personal 

memory of events. So, it seems logical to start with the 

consumer. 

Second, such a study would fill a gap in the 

existing literature. To the best of our knowledge, 

currently there is no nationally representative study 

using a consumer survey approach that critically examines 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

11

the consumer's recollections and judgments. Now, set 

against these advantages, the consumer survey approach is 

prone to a number of problems, particularly if we were to 

rely uncritically on consumer's review of their own 

files. 

You can follow, incidentally, in your handouts. 

 All the slides are in the handout. 

These problems come from several sources. 

First, since credit reports may be provided in a 

consumer-unfriendly way, they may confuse consumers, 

leading them not to recognize items that really do belong 

to them. Second, consumers may mistakenly challenge 

items because they do not understand their credit 

obligations. I will provide an example shortly. Third, 

some accurate items will be viewed to be inaccurate 

because of lapses in the consumer's memory or failure to 

maintain and consult the appropriate records. Fourth, 

consumer answers to a survey may be biased, either 

intentionally or unintentionally. For example, they may 

do a better job of remembering and confirming favorable 

information in their files than derogatory information. 

Consumer bias is especially likely to be an 

issue with regard to incompleteness. If consumers have a 

mortgage loan in good standing that is not included in 

their report, they will have an incentive to report that. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

12

 We can't be as confident that consumers with unreported 

delinquencies will voluntarily report those. 

So, how can we design a study that will 

minimize these problems? Some of these problems could be 

mitigated through education and coaching. In particular, 

we anticipate the need for a person who is very 

knowledgeable about credit reporting conventions, 

financial institutions and credit law, to work closely 

with a consumer to help sort out some of these problems. 

 

As an example, the first problem, one way a 

credit report can be unfriendly is by listing an account 

under an unfamiliar name. If the consumer does not 

recognize an account with, say, a First National Bank, 

the expert coach might be able to inform the consumer 

that First National handles accounts for some well-known 

retail store, maybe Wal*Mart or Sears and the consumer 

might now recognize that he does have an account with 

this store. 

For an example of the second problem, consider 

a married woman who cosigns a loan with her husband. 

Later, they divorce and her ex-husband agrees to be 

responsible for paying this loan. But if the ex-husband 

fails to make payments, she is still jointly responsible 

and the missed payments still belong on the credit 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

13

report. If she does not understand these legal 

intricacies, she will mistakenly report the item as 

pertaining only to her ex-husband and not to her. An 

expert coach could explain that she was still legally 

responsible and resolve this apparent discrepancy. 

Similarly, an expert coach could help reduce 

problems by probing the consumer's memory and asking the 

consumer to systematically check his records. 

To provide better protection against faulty 

consumer memory and also to protect against consumer 

bias, we also anticipate the need to obtain information 

from the point of view of the furnisher of the 

information that the consumer views as incorrect. This 

might be done in one or more of several different ways. 

One way would be to ask the consumer to try to dispute 

the item directly with the furnisher. A second would be 

to ask the consumer to carry through a formal dispute 

process with the CRAs. A third would be for the coach or 

another member of the research team who might be a 

professional with many preexisting industry contacts to 

contact furnishers. 

We would be especially interested to hear the 

views of any of you today regarding how we might, most 

effectively, obtain the views of furnishers regarding 

items that consumers believe to be in error or to be 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

14

missing. 

In adapting our consumer survey design, we must 

also keep in mind that it needs to work on a nationwide 

scale. For example, nationwide participation would 

appear to rule out in-person interviews, leaving 

telephone contact and mail, supplemented by email when 

available, as the most realistic approach. 

A large nationwide survey would also appear to 

require a fairly large number of consumer coaches if 

we're following this method. That would create several 

more problems. We would need to develop uniform 

selection and training procedures to make sure that these 

coaches adopt a consistent approach. It might be 

difficult to find many coaches with the requisite 

knowledge. This would tend to reduce the average 

expertise level of each coach and require more elaborate 

training procedures. 

There also may be limits in our ability to 

provide adequate training. It may not be possible to 

summarize all the expertise a coach would need into a 

short training course. Finally, with many different 

coaches, we would need to devise consistency checks to 

minimize coach-specific biases and demonstrate the 

reproducibility of the procedures. Similar problems will 

result, also, regarding the resolution of consumer-
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

15

furnisher conflicts over accuracy. 

Because of these problems that are so tricky, 

we plan to conduct one or more pilot studies of the 

consumer survey approach to address the various problems 

that we talked about. We want to know, does expert 

coaching work, how much time will it take per consumer? 

Is the consumer study workable at all? Are there more 

problems that we have not yet identified? How expensive 

will it be to conduct a full scale survey? 

Last May, we invited several parties to submit 

bids to conduct such a pilot study and we have already 

mailed to each presenter today a copy of the scope of 

work for this pilot study. A copy for your later 

reference is appended at the back of the handout you have 

in front of you. It contains a few more details of what 

we propose to try than I have time to present today. 

Assuming now that the pilot is successful, a 

big assumption I grant, and we find that the consumer 

survey approach is promising, we'll still have to address 

a number of sampling issues. First, what is the 

appropriate sampling frame, what is the appropriate 

sampling population? Is it the entire U.S. adult 

population, the U.S. adult population with credit 

history, a sample of credit files obtained from the big 

three CRAs? Should we look at all credit reports 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

16

actually drawn for business purposes? There's a 

distinction here between credit reports drawn for 

business purposes and drawn by consumers just to check 

the accuracy, for example. 

In particular, some consumers are much more 

active in the credit market than others. In our 

analysis, should we give more weight to those consumers 

who are more active or should we give equal weight to 

everyone? It could be argued either way. As an example, 

one might say, suppose an individual never or rarely 

applies for credit, then why should we be concerned about 

the accuracy of a report that he never actually asks for, 

but he only would have asked for if he did apply? And if 

an individual applies for credit many times, then isn't 

there more potential for benefit or harm because errors 

or accuracies would affect more decisions? This would 

weigh in favor of using a sample population of the actual 

reports, not consumers, and that would result in giving 

more weight to consumers who are more active in the 

credit market. 

Or one could argue, instead, that maybe the 

reason why some people rarely apply for credit might be 

because there's an error in their report that resulted in 

their being rejected and maybe they subsequently gave us 

even trying to apply. Each person potentially could be 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

17

just as active in the credit market as any other. This 

argument would weigh in favor of giving equal credit, 

equal weight to all consumers. 

A related issue in terms of what's the right 

sampling population is the same report question. This is 

a question that comes up in another study that we have to 

do for Congress and we don't have any answers here at 

this point. So, I'm only going to say what the problem 

is and say nothing more about it. 

The same report question is, how should a study 

deal with the possibility that a report received by a 

consumer may be different from a report on that same 

consumer provided to a business user as the result of 

differences in the identifying information provided to 

the CRA and the way the CRA's matching algorithms respond 

to these differences? 

Okay, moving on to other sampling issues that 

need to be resolved, there's an issue of sample selection 

bias. We might have under-representation of various 

groups, such as non-English speakers, undocumented 

residents, consumers with poor communication skills. One 

can make arguments that we might either get over or 

under-representation of consumers with credit problems. 

Another issue is sample size. This will depend 

on which results we wish to measure and how much 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

18

precision we desire. If we want precision of plus or 

minus two percentage points for the entire population 

sampled, we would need a sample of about 1,000 consumers. 

 If we want results broken down by sub-populations, we 

might need a sample of several thousand. 

One way to economize on the size of the sample 

would be to use stratified sampling techniques. For 

example, if we want to understand not only the incidence 

of inaccuracies and incompleteness in credit reports but 

also their nature and possible causes, then we should 

over sample reports that are more prone to these 

problems. For example, we might over sample consumers 

whose reports from the big three CRAs have discrepancies 

that result in a big difference in credit scores. 

Alternatively, we might over sample reports with low 

credit scores or with thin files. 

If a principal concern is the prevalence of 

inaccurate information that is derogatory, we would be 

most likely to find these cases in reports with low 

scores. If a principal concern is the effect of 

incomplete information due to leaving out examples of 

credit lines in good standing, then we would tend to find 

these cases in reports with few items, so-called thin 

files. Stratified sampling would also require a 

prescreening of credit files. So, we need to address 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

19

privacy concerns associated with prescreening. 

One possibility would be to ask a credit 

reporting agency or an organization with an existing 

legitimate business need to have access to credit files, 

to prescreen files in their possession and select ones 

that meet our stated criteria. The organization could 

then ask consumers on our behalf whether they would be 

willing to participate in a study. In this way, we would 

never have any access to any private consumer information 

without first having obtained the consent of the 

consumer. 

Okay, now, what do we anticipate would be the 

final results? Well, we can't, of course, give numbers, 

but we can anticipate how we would want to break out our 

results. So, one -- this is very tentative. For each 

item in the credit report that affects the credit score, 

we might classify it in terms of whether it's confirmed 

accurate, confirmed inaccurate for various reasons. It 

might pertain to the wrong person, it might have 

incorrect information. Or we might not be able to 

resolve the accuracy fully and we don't want to force 

things into a category if we're not sure, so we might 

say, of undetermined accuracy because the item might 

belong to a different consumer, but we're not sure, or 

undetermined accuracy because we're unable to resolve a 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

20

conflict in the views of the consumer and the furnisher. 

 

Now, a different category is an item which is 

improperly duplicative of another item, so the item is 

correct, but the duplication would lead to a false 

conclusion that a consumer has more open accounts than is 

true and that could affect the credit score. 

Similarly, if items are taken through the 

formal dispute process, we could classify them in terms 

of the results of that dispute process. It might be 

confirmed accurate because the consumer now agrees it is 

accurate or confirmed inaccurate because the furnisher 

now agrees it is inaccurate, could be presumed inaccurate 

because the furnisher does not respond. But there we 

wouldn't know for sure. Or it might be unresolved 

because the furnisher might confirm the accuracy but the 

consumer may continue to disagree. So, we wouldn't try 

to force these results into a particular box, but we'd 

say exactly what we know and let other people then draw 

their conclusions which side they believe in a dispute. 

Next, we would -- for each credit report, we 

would now classify it according to the number of 

accurate, inaccurate and uncertain items, the number of 

missing items, the effect of inaccurate, missing and 

uncertain items on the credit scores, individually and in 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

21

total. We could do further analysis to identify the 

contributing factors that may have led to inaccuracies 

when this is possible. We don't anticipate being able to 

attribute a single cause to every inaccuracy. Examples 

of causes or contributing factors include errors in 

identifying information given by the furnisher or the 

consumer, errors in files of the furnisher regarding the 

consumer's account, fraud, ID theft. I'm not trying to 

be exhaustive here, just illustrations. 

And, finally, we could do further analysis to 

break down the incidence of errors according to 

demographic factors, such as age, gender, marital status, 

ethnicity, citizenship, et cetera. 

So, that's it. Now, we eagerly await your 

feedback. Is the use of consumer surveys a reasonable 

approach for measuring accuracy and completeness? Are 

there other problems and biases that we have not 

identified? Do you have any suggestions about how to 

counter these problems and biases, including ones that we 

haven't thought of yet? The basic question, is the whole 

approach workable? If not, we need a better approach. 

So, what ideas do you have for a better approach? That's 

it. Thank you. 

MR. VANDER NAT: Thank you, Gerry. Is this mic 

on? Can somebody -- I am told that these mics are on 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

22

unless you push them off. These mics are supposed to be 

working. I was assured that. Would somebody else try 

their mic to see if -- is it on now? 

(Testing of microphones.)

MR. VANDER NAT: Let's have about a 10-minute 

discussion right now on what Gerry has said. The reason 

is this. The next presentation may also run 25 or 30 

minutes and by the time we have another presentation, you 

have a number of different thoughts in your mind. So, 

let's try for about a 10-minute discussion right now and 

then move on to the next presentation. 

Now, about these mics, I am told that they are 

automatically on unless you push them off. You see 

there's a little push button or a push section in the 

center, and if you push that, it's supposed to be off. 

So, don't make the mistake of having it on when you 

didn't want to have it on. We know there are some 

notorious mistakes in that situation. 

So, is there anyone who would like to begin 

with an observation or a question? You're all being so 

shy. Stuart? 

MR. PRATT: Well, let me -- just a couple of 

things, Peter and Gerry both. Thank you for a very 

thorough overview of where you intend to go and, of 

course, I think like everybody else, we're reacting to 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

23

this for the first time today, not having had the 

opportunity to hear some of the granularity that you've 

now tried to put to that framework that you've sent to us 

in advance. 

Just a couple of observations. I guess in part 

what you're looking for is a score migration or a score 

shift analysis of some sort, what does the score look 

like and then how does the score change. I guess in 

talking with some of our members the question is, well, 

when do you get to a material test of that? In other 

words, you might find many scores have a very small 

increment of change and there may be others who can 

contribute to that, but there may also be instances where 

there's a more material change in the score, if you're 

going to go down this road of using scores as a method of 

that. And to expand on that just a little bit, then you 

have to decide, in what context is the score being used 

and what kind of market? Is it an auto finance market? 

Is it a mortgage market? Is it a -- I think there, too, 

then you'll find that the tool, the scoring tool, which 

is a method by which a decision is made, is then used 

differently depending on the market in which that 

application is being made and the type of credit for 

which the consumer is applying and so on. 

So, I realize you didn't try to cover 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

24

everything in your slides, but those are just thoughts, I 

think, in terms of how you approach that part of the 

study. I think if you have no materiality test, then 

yes, many scores might move a little bit depending on -- 

assuming the change has something to do with the score in 

the first place, but the question is by how much and by 

how much of an effect. You probably would need to look 

at more than one lender in the marketplace. 

My second observation, if I could just do this, 

is we've even had consumers who are victims of identity 

theft with police reports who will clean up other items 

of information in their file at the same time with their 

police report, along with the items that were fraudulent. 

 So, I guess I just say that in order to make sure that 

we all understand the -- I'm not sure how you're going to 

find these experts, if you're going to go down this road 

of using experts, but it isn't to say -- this isn't to 

cast dispersions across all of our society, but just 

simply to say that that's a very important -- the credit 

repair effect, the temptation to try to correct a report 

is there whenever there's a file with adverse 

information, even when it's accurate adverse information, 

even if the consumer, in some cases, is a victim. 

We've even done surveys of consumers who have 

purchased credit repair services, and some of the surveys 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

25

came back saying, I thought I would give it a shot. I 

just wanted to see if I could get that bad information 

off my file, even though I knew it was accurate. So, 

there's always that caution flag. 

You've acknowledged that, I think, in your 

slide and we appreciate that because I think that's an 

important component of this thought process as well is to 

make sure -- and then, finally, if you're choosing these 

experts, however you find them -- and it will be 

interesting to see how you find those experts, and I say 

this with all seriousness, I assume you will put them 

through security reviews to make sure that they, 

themselves, are not going to perpetrate identity theft 

when they start to look at all these credit reports with 

consumers. It would be a bad thing for the Federal 

Government study to foster identity fraud in the 

marketplace. And, also, I think, you need to make sure 

there's no bias with those experts themselves, that the 

people who are being trained themselves don't bring a 

bias to how they feel about credit reporting, the credit 

reporting experience, the credit granting experience and 

so on. 

So, there's some issues, I think, with 

controlling bias even on the part of the filter through 

which the experts themselves are going to work on that -- 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

26

work with that consumer. 

And then my final observation is, I guess, 

there's a lot of challenges to getting consumers to 

really truly commit themselves at a distance to really 

getting down through all the information in their file. 

It depends on how motivated those consumers will be. I'm 

assuming there will be some self-selection that results 

from the kind of consumer that has the propensity to want 

to get involved in looking through a credit report, I 

guess, at the level that you would like to get. So, 

those are just some first thoughts. 

MR. BUTTERS: (Inaudible) I think those are 

excellent comments. If I had had half an hour or an 

hour, I would have reached more of them. I was just 

talking to Karlene before about your materiality issue. 

That's something we want to be able to do. The credit 

score is only the first step. Which credit score is an 

issue? For which decisions and do we have numerical 

measures, how much -- how likely the effect is of a 

change in credit score on a behavior. So, all these 

things we would like to do. It's one of the things we 

have 11 years for, I guess. 

MR. CATE: Fred Cate. Let me say, and I think 

everyone here would reflect the same thing, you'll 

probably hear this a great deal today. The more I've 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

27

thought about this issue, getting ready for today, the 

sorrier I've begun to feel for you. 

MR. BUTTERS: Thank you. 

MR. CATE: I assume you didn't invite us here 

just for sympathy, but I think you have a nearly 

impossible task and I'm not -- I guess the challenge 

today is to see if we can get beyond merely emphasizing 

how impossible it is to actually find ways of making it 

more possible. 

I would, I guess, just raise three issues that 

overlap at a small point with what Stuart has said. One 

is I think the definitions are absolutely critical here 

and I think the definitions, as currently stated, for 

accuracy and completeness, while straightforward and 

appealing, I think their straightforwardness sort of 

obscures the fact that defining accuracy has been one of 

the key problems in all of the studies we've seen to 

date. 

You know, the GAO noted this in its 2003 

report. We just don't have an agreement on what accuracy 

means. In fact, I don't think we even have a stab at 

what accuracy means. I think the studies we're going to 

look at today will reflect that. Does accuracy mean, I 

didn't dispute it? Does accuracy mean the three bureaus 

agree on it? Does accuracy mean what? And I think 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

28

really plumbing the debts of that issue is something that 

desperately needs to be done, and the reason for that is 

because -- I keep coming back to remind myself this is 

for Congress. Now, I understand many things for Congress 

are subsequently ignored by Congress, but let's just 

assume for a moment Congress is going to use this for a 

purpose and that purpose is presumably to detect flaws in 

the system and recommendations for improving it. 

So, the question is, even once we have some 

concept of what is accuracy, it's why is there 

inaccuracy. Is it inaccurate because furnishers reported 

inaccurate information? Is it inaccurate because of the 

timing of when the data were reported or when they were 

recorded in the file? Is it a result of mismatching 

data? In other words, is it a bureau problem? We got 

the data, the data were accurate, but we somehow messed 

up getting it in the right place. Is it a calculation of 

the score problem? In other words, are credit granters 

using elements from the files that they should be not 

using? 

This, of course, obviously is going to be 

circular. It comes back to what is inaccuracy. Do we 

care about inaccuracies if they're not used by the score? 

 You know, inaccuracies, if they're not material or 

they're not relevant to calculating the score, why does 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

29

this matter? 

The third point, and I always go around saying 

things like this and I always get in trouble for it, but 

I think we have to be frank here and we're primarily 

among, you know, friends or professional colleagues. 

But, you know, we're not on CNN. Consumers are a lousy 

place to judge accuracy. I would argue they are, 

perhaps, the worst possible measure of judging accuracy. 

 I don't mean that, by the way, for malfeasance. I don't 

mean -- I mean, I think there are consumers who would 

deliberately lie or whatever we know -- you know, a 

majority of people say they'll cheat on their taxes, they 

lie on their resumes. 

So, I don't know what makes me think they're 

going to be incredibly full of integrity in this process. 

 But I'm more worried about, you know, most of us don't 

know. You know, the vast majority of people, 70 

something percent, don't balance a checkbook. They have 

no idea what credit account was opened 10 years ago. 

They have no idea what was closed, what was done jointly. 

You raised the point, which I think was a very 

good one, about the legal constraints. You know, that 

was before I was divorced, that was after a name change, 

that was -- and so, I think in terms of whatever our 

measure of accuracy ultimately is, whatever your measure 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

30

of accuracy ultimately is, I understand consumers will 

clearly have to play a role in the process and I think 

all of the studies on the table today have involved 

consumers at some point in triggering like I request a 

report or I dispute something and I don't know a way 

around that. 

But I think to think of consumers as being the 

measure of accuracy, like I say, well, you know, now 

that's accurate, I mean, I think is laughable. I don't 

think it's a practical thing to do for the simple fact 

that I think most of us, including probably some of us in 

this room, at least the lawyers in this room, would find 

it very difficult to know if our reports were accurate or 

not. 

UNIDENTIFIED MALE: Let me briefly respond to 

that point. All of your points are well taken and we are 

daunted by them. I think the element that we're trying 

to introduce here is, we're not just going to take the 

consumer's word for it. Just because the consumer says 

it, that doesn't make it so. That's why we are thinking 

in terms of coaches and a review process. So, the review 

process is either a formal dispute process or it's an 

informal process where, through the coach, a furnisher is 

contacted and they agree, okay, it's been straightened 

out. We hear that there are lots of circumstances in 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

31

which there are people who already play this kind of 

role, who work with the consumer in a very positive way 

to straighten things out, to straighten out 

misunderstandings. But the key is, we are not just 

taking the consumer's word for it, and that's why Gerry 

gave those different kinds of categorizations and we will 

ultimately note, you know, that there were certain errors 

where both consumer and furnisher agreed. Now, if they 

both agree, that's the most important kind of error and 

there are these other classifications where we are less 

sure. 

But I think that's the main way in which we're 

going to try to handle the observations that you're 

making. 

MR. VANDER NAT: Bob? 

MR. AVERY: Yes. My first reaction, actually, 

was -- 

MR. VANDER NAT: Bob, excuse me, could you 

state your name? 

MR. AVERY: Bob Avery of the Federal Reserve. 

My initial reaction was we were originally penciled in to 

do this study and how glad I was -- 

(Laughter.)

MR. AVERY: All of the reasons that you raised. 

 Let me just make a couple of comments about the specific 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

32

design here. My gut reaction is that this is just not 

feasible. Our survey of consumer finances cost about $6 

million. It's an in-person survey of about 4,000 people. 

 The issues are about as complex as this. You're talking 

about a recurring survey that you want to do on a 

basis -- you know, every X number of years, I think you 

have to -- realistically, you can't do this probably by 

mail if you're going to use coaches and email. It's just 

not going to work. And you're -- unless you're committed 

to lay out those kinds of resources, my suspicion is 

you're not going to get the kind of quality in this data 

that you seek. 

Let me suggest two other possibilities that are 

variants on this you might want to think about. One is a 

low level survey that you simply ask people their 

perceptions of the quality of their credit and you 

compare that with their score. And you're picking up 

their perceptions as to orders of magnitude, of how good 

their credit is, relative to what the Bureau would say, 

and what you're picking up there is whether there's a 

disconnect. Now, it may not become inaccuracies and 

errors, but you're also looking, at an operational level, 

whether there is a difference between the information 

recorded on the Bureaus and the information that's 

perceived by the consumers. That's a very cheap survey 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

33

and you can do it every year and see if there's a change 

over time. 

The second complimentary item, you're viewing 

the frame as a frame of people. There's an alternative 

frame, which is a frame of line items and Bureau data, 

and there are millions of them. So, you view every item 

in the Bureau data as an item. 

And think about the way that they do audits for 

the securities industry. You get a -- I don't get a 

letter for every -- asking me here's a list of all the 

stock you own, tell me about each one, whether it's true. 

 What I get is individual letters asking do you own this 

stock. One of the things you could do if you thought 

about that frame is you could go to the Bureaus, you 

could sample items that are much more likely to be in 

error, such as public records, collection account items. 

 Stay away from the B of A reports that are probably 100 

percent accurate, the large lenders. Look at files that 

have not been updated. 

So, you can key -- you can do very stratified 

sampling, keying in on items that are much more likely to 

be in error. Do them an item at a time. The consumer is 

sent a letter and you validate it one way or another. 

But you frame -- you've got lots of degrees of freedom. 

If you think about that as your frame, you've got 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

34

hundreds of millions of items there. All large numbers 

can work for you. What you're doing here is you're 

bundling. You're taking each consumer and sending them 

40 different questions to the same consumer and making 

them check off each one. My alternative is one where you 

don't have to -- you know, each consumer might get one. 

The loss of this is you won't learn about 

errors of omission, but it seems to me the gain is you 

learn -- you can have a much, much, much more focused 

sampling technique. You can probably do it by mail and 

it's going to probably be a lot cheaper, and it's 

something that over time, if you repeated this exercise, 

you could see, are there gains in error. 

MR. VANDER NAT: Thank you. We'll just take 

one more comment at this point and then we'll move on to 

the next presentation, and there is going to be further 

time for all of the discussion. So, we're going to get 

back to these points later in this morning's session and 

also throughout the day. 

I think it was Paul that had a comment. 

MR. WOHKITTEL: Yes. Paul Wohkittel. I 

couldn't help but -- as Fred was talking about accuracy, 

I couldn't help but draw a parallel to the definition of 

pornography. That is, you know it when you see it. Now, 

I realize this group is comprised of different 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

35

industries. I only represent the reselling industry of 

the mortgage. But at our level, we dig very, very 

deeply, along with the consumers, into the data. 

Now, my point in bringing this up is that this 

is a very feasible task. I agree with the premise that 

the consumer is the greatest expert about his or her own 

history. But I think along with that, you have to have 

an expert guiding them along and I think there's plenty 

of experts out there to do that in many different 

industries who are very knowledgeable and can delve 

independently into the problem. 

MR. VANDER NAT: Thank you. Just to keep 

things on track, I'm going to ask at this point -- 

(End of Tape 1, Side A)

MR. VANDER NAT: -- Interest Research Group to 

give us a presentation on their views and no doubt a 

report on their very latest study that we've all heard 

about the last couple of weeks. Please go ahead. 

MR. MIERZWINSKI: Thank you, Peter. I'm Ed 

Mierzwinski with U.S. PIRG and with me is Alison Cassady, 

a research director, and we've got extra copies if people 

didn't get it of our latest report, Mistakes Do Happen, 

which came out just two weeks ago. 

I want to thank the FTC for inviting us. We'll 

have, I think, a brief presentation with a lot of time 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

36

for questions and comments afterwards. I came to 

Washington in 1989 when Congress began its review of the 

credit reporting industry that actually took them 15 

years. If you look at the 1996 amendments as requiring, 

from the industry's perspective, additional 2003 

amendments, because otherwise the states, horror of 

horrors, would regain the right to try to make credit 

reports more accurate and prevent identity theft. So, 

Congress last year really -- I don't see the Congress 

doing a lot on the credit reporting industry again until 

perhaps some of the FTC's new studies come out. But it 

took them 15 years. 

I think they've come a long way, and other than 

the unacceptable limit on state authority, we've 

addressed a number of issues either in the new bill, 

specific requirements or in the studies that are being 

required to be done. I personally wish some of the 

studies were actually in the law as specific requirements 

on the industry, but I'm hoping that the result of the 

studies will be that we will get some more amendments to 

the law once the studies begin to roll out. 

When we first got involved, Consumers Union, 

Consumer Federation of America and PIRG were very 

concerned that the mistakes in credit reports or 

complaints about credit reports and the service levels at 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

37

the Bureaus were something that were the number one 

complaint to the Federal Trade Commission in the early 

'90s. So, we focused a lot on service at the beginning, 

as well as accuracy. The new FACTA Act provides, I 

think, some of the fundamental things that are going to 

help consumers who are really the data subjects, look at 

their reports. So, all consumers are going to be, in 

effect, contributing to a broader study than the study 

that the FTC is doing because all consumers are going to 

be looking at their credit reports for free for the first 

time. They're the data subjects. That's a very 

important public policy outcome. 

And after years of fits and starts, and 

California's ultimate leadership, although it's not free 

from the Bureaus, we will have access to credit scores. 

And as I've pointed out in other contexts, the credit 

score business model and the Fair Isaac will certainly 

speak to this, I think their business model has changed 

dramatically. With the passage of the California law, 

they recognized there was a direct consumer channel and 

that consumers would benefit from having this 

information. So, we have the credit score disclosure, 

the list of descriptions of the credit scores. 

The other fundamental changes that were made to 

the law that result in accuracy, we have a new 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

38

requirement allowing consumers to dispute directly to the 

creditors. That will also help to engage the creditors 

and force them to improve the way that they do business 

to the extent that they're responsible for mistakes and 

it's not the issues of the way that the data are 

collected. 

And, finally, the business model of the 

industry changed dramatically in the 1990s and risk-based 

pricing became the norm as computer power became so much 

cheaper. More consumers were given credit yes, but, 

rather than no, you're not getting credit. Yes, but 

you'll have to pay more. Those consumers weren't getting 

adverse action notices. Now, they will be getting 

adverse action notices. So, I think that that will also 

help to contribute to the bigger study. 

In terms of the other studies, even though 

Peter and others have pointed out that this study is a 

massive undertaking, I wish we could somehow incorporate 

some of the other studies into this study and come up 

with one big uber study. In particular, you've mentioned 

-- and I think this is one of the biggest problems you're 

going to have to wrestle with -- the issue of what you 

call same reports, what we call the subscribed report 

problem, somehow I think there's a separate study that's 

got to be done on the feasibility of giving consumers the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

39

subscriber report. 

Now, there's already language in the law that 

gives employment applicants the report provided by the 

employer. So, I don't think that's unfeasible at all. I 

think somehow your study has got to make sure that it 

blindly compares and incorporates data from subscriber 

reports that are provided to subscribers within the view 

of many at the table, a different set of verification 

algorithms than the reports that are provided to 

consumers which may therefore -- the report you look at 

after you're denied is probably much more accurate than 

the report the subscriber may have used to either deny 

you or charge you more. I wish -- I think it's very 

important that you figure out a way to have some part of 

your sample be studying both reports. 

Second, I think there's other issues in the 

law, not the least of which is the study of disparate 

impact of credit scoring and the study of how to expand 

missing information. We talk about missing information 

such as the Fed study found where a company on its own 

decides to mask a consumer's true credit profile to make 

them look worse so they won't be given offers by other 

companies. But there's also the missing information 

where the consumer doesn't have national accounts and is 

a victim of a thin file, as you pointed out. So, there's 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

40

a study of that as well. 

But all of these are some of the issues that 

we're encouraged that the Federal Trade Commission is 

going to look into. I'd like to turn it over to our 

research director, Alison Cassady, who will talk 

specifically about our methodologies of our several 

studies, and also about some of the views that we have as 

to how you can improve or respond to some of the 

questions you list. 

MS. CASSADY: Can everyone hear me? 

(No response.) 

MS. CASSADY: As you all know, in June, PIRG 

released an update to its 1998 study, Mistakes Do Happen. 

 We asked adults all over the country to order their 

credit reports and complete a survey on the report's 

accuracy. Just briefly, to go over the main findings of 

the report, we found that one in four of the credit 

reports surveyed contained serious errors that could 

result in the denial of credit, and by -- we defined 

serious errors as accounts that are incorrectly marked as 

delinquent, accounts inaccurately listed as being in 

collections, accounts listed that do not belong to the 

consumer whether or not in good standing and 

bankruptcies, tax liens and other judgments that do not 

belong to the consumer or are still listed as open even 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

41

though they've been resolved. 

Altogether, we found that eight out of ten of 

the credit reports we surveyed contained either these 

serious errors or other mistakes of some kind, such as 

missing accounts, inaccurate demographic information, 

closed accounts that are listed as open, inaccurate 

credit limits and loans and mortgages listed more than 

once. 

If you all have questions about the report=s 

findings, Ed and I would be happy to answer them, but I 

want to focus on the report=s methodology and the 

challenges we faced in implementing a survey that is 

based almost entirely on a consumer=s ability to recognize 

errors in their own reports. Hopefully, this will be 

somewhat instructive for the FTC pilot project. 

So, our report is based on the files held by 

the so-called big three credit bureaus, Equifax, Experian 

and Trans Union, and in the spring of 2004, we sent 

emails to thousands of PIRG citizen members across the 

country requesting their voluntary participation in the 

survey about accuracy of credit reports. We couldn=t 

offer anybody any monetary restitution for ordering the 

reports or anything like that, so it was all completely 

based on voluntary participation. 

We directed the consumers to our website where 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

42

they could indicate if they were willing to participate 

in the survey and from which credit bureau they planned 

to order their report. If someone volunteered to 

participate, but did not complete a survey within seven 

days, we sent a reminder email with a direct link to the 

survey form. In addition, we asked PIRG staff, coalition 

partners, friends and family from all around the country 

to complete the survey. In total, we collected 200 

surveys from 154 adults in 30 states, the vast majority 

of which came from PIRG citizen members. The 

participants ranged in age from 20 to 81 and the average 

age of a person responding was 40. 

The FTC asked us to focus on the challenges 

inherent in conducting the type of study we did, in which 

we rely on the consumers themselves to identify errors 

and accurately fill out the survey. Overall, our 

methodology is adequate to give us a representative 

sample of credit reporting problems. The results tracked 

the findings of other surveys conducted by consumer 

groups and also tracked the results of massive computerbased credit report surveys conducted by the Fed and CFA. 

The first challenge I want to focus on that we 

faced, and I=m sure the FTC will face as well, is just 

getting people to participate. Foremost, it is a 

struggle to get consumers to voluntarily spend $9 for a 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

43

single credit report. I can imagine that it=s even more 

difficult to get people to purchase three credit reports, 

plus their credit scores. That=s probably about $40 a 

person, and I=m not sure if the FTC is planning on 

reimbursing the survey participants or offering money, 

but this cost is prohibitive to many folks that we found. 

We were unable, like I said, to offer any reimbursement. 

On a less practical level, some people are 

literally afraid to look at their credit reports. They 

prefer to live in denial. They don=t like seeing their 

credit history in black and white. We had several people 

say that. And others are confused and think that by 

ordering their credit reports, they can somehow lower 

their credit score. Others are concerned about the 

privacy of the personal information when transmitting it 

over email or sharing it with a perfect stranger, such as 

myself. 

And overall, it is just a challenge to get 

consumers to participate in these types of surveys 

because it involves multiple steps. You=ve got to get 

them to commit to do it, then they have to order their 

credit reports. It doesn=t always work online. Sometimes 

you have to call and send stuff through the mail, and 

then once they get the stuff in the mail or online, they 

actually have to sit down, find the time and pour through 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

44

the credit report survey. Usually, the answers aren=t 

simple yes and no. So, that=s a task and we struggled, 

you know, over a period of several months to get enough 

people to complete our survey. 

The second main challenge, obviously, is 

relying on the consumer to understand their credit 

reports. Our report is based on the following 

fundamental premise, that consumers themselves know 

better than anyone whether or not their personal 

information is right and whether all of the account 

information on the report belongs to them and is 

accurate. I mean, if I were to look at a stranger=s 

credit report, I would not be able to identify missing 

accounts, inaccurate credit history or accounts that do 

not belong to the consumer or other errors. 

We=ve also found, basically, that the 

institutional memory, so to speak, of the consumer is 

most reliable in the areas that matter the most, whether 

he or she has filed for bankruptcy, for example, whether 

he or she has ever opened a significant line of credit 

with a major bank, ever faced collections, that sort of 

thing, and those are the things that -- errors in those 

components have the most direct effect on the credit 

score. 

The FTC pilot project you mentioned will look 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

45

at the efficacy of hiring a consultant to help consumers 

identify errors in their credit reports, and I=m 

unconvinced that it will be completely helpful, but, I 

mean, I=m open to -- it sounds interesting. I think it=s 

going to be particularly challenging and the first 

presenter, you know, did go over some potential snafues 

with finding consultants and finding the appropriately 

trained people and unbiased folks. 

But the pilot project also suggested, at least 

in the materials that we received beforehand, that you 

would do most of these phone -- these consultations over 

the phone. We relied almost entirely on email to conduct 

our surveys. So, I can=t speak specifically to how 

effective a telephone based survey would be. But I can 

imagine that a phone conversation discussing the accuracy 

of three credit reports would be quite lengthy, and I 

think that this just provides another obstacle to having 

consumers participate. Not many people want to spend 45 

minutes on the phone with a government employee talking 

about their personal credit history. I just think it 

might be one more thing that people have to get past in 

order to participate in the survey. 

So, in my opinion, I mean, crafting carefully 

worded questions on the survey instrument and providing 

detailed instructions and frequently asked questions 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

46

about how to understand the credit report is the best way 

to implement the project and it would be helpful, 

definitely, to have consultants available by telephone 

and email to answer specific questions, like, you know, 

the problem that was raised about, do I have a legal 

obligation to -- am I still responsible for the joint 

account that I held with my ex-husband. 

But those are definitely -- I just don=t 

necessarily think that a consultant helping each person 

is going to get past -- we still need to rely on the 

consumers as the people who know best about what they 

have done in the past. 

And then in terms of selecting the sample and 

the people in the sample, there are a few important 

factors to consider when selecting the population and 

these may seem kind of obvious, but these are the ones 

that jumped out at us most when we were looking at the 

credit surveys we received through our survey. First is 

age of the people. Obviously, 21-year-olds are less 

likely to have an extensive credit history than a 50-

year-old, and they are, therefore, less likely to have 

serious errors in their reports. Most of the people in 

the 20, 21-year-old bracket that we found had no errors 

in their reports whatsoever. 

The state of residence is also something to 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

47

consider. We did not break down our findings by state, 

but states that have offered consumers free credit 

reports for a while may have a different rate of errors 

than other states, since ostensibly it=s been easier for 

consumers to obtain their credit reports in the past and 

examine them for errors. This may be something you might 

want to look at when doing your analysis, breaking it 

down by free credit report states versus not and then 

comparing it over time as the states roll out their free 

credit reporting. 

A third thing that you touched upon is marital 

status. Again, we didn=t look at this specifically, but 

anecdotally, it appears that consumers who are married or 

divorced commonly noted that their spouse=s information, 

in some way, appeared on their credit reports. 

And the fourth thing is income. Obviously, 

income may be a bigger factor in the credit score aspect 

of your study, but in terms of the accuracy, it could 

affect it as well. Wealthy consumers, obviously, have a 

more extensive credit reporting history than more trade 

lines and such than poorer consumers. On the other hand, 

a poorer consumer may have more checkered credit history 

with bankruptcies, collections and such that offer 

additional opportunities for error. 

So, these are just -- obviously, if you=re doing 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

48

a truly random national sample, you may not be able to 

correct for these things, but there are indicators I 

wanted to flag, something that could affect the results. 

Ed? 

MR. MIERZWINSKI: Alison asked me to be the -- 

to sum up, but she really covered everything. I will 

point out that I totally agree that I think you -- and I 

look forward to Richard=s comments and Terry=s comments, 

in particular, on the issue of how do you address 

consumer=s understanding what a joint account is, which 

should be on your credit report and should count in your 

score, and a cosigned account, versus an authorized user 

account, which is frequently, in the view of many 

experts, from the consumer=s side of the business, a big 

concern. And the coding of those kinds of accounts can 

be a significant problem. 

The second point I just wanted to restate that 

Alison made, we totally feel that on the big picture 

issues -- I think Gerry made a valid point on the 

memories of the consumers and that will be an issue for 

whether you paid your credit card late or on time, that 

sort of thing. But you certainly remembered whether you 

cosigned a loan, whether you ever filed for bankruptcy, 

whether you=re married to some stranger. Those big 

picture mistakes on the credit reports, I think, are the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

49

ones that are most important for causing denial or 

putting you in the sub-prime when you=re not in the subprime. 

I don=t know if anybody saw the piece on 

Frontline this weekend on probably the nation=s leading 

bankruptcy expert, who=s done a number of major studies on 

bankruptcy, Liz Warren, of Harvard Law School. I was 

just rereading one of her papers and she says she=s 

participated in a number of big studies and they=re like 

helicopters. Helicopters are 20,000 loose bolts flying 

in formation and you have to make a lot of assumptions. 

Sometimes the assumptions and the decisions about what 

data are good data or bad data just have to be made. But 

I encourage you to go forward with this and I look 

forward to the rest of the discussion. 

MR. VANDER NAT: Thank you, Ed and Alison. 

Before I open it up for discussion, I just want to make a 

few comments on some of the points that were raised. The 

same report issue that Gerry referenced is an important 

issue and that falls under Section 318 of FACTA. I hope 

this mic is working fairly well. 

There are a number of different studies, which 

we called small studies, not because they=re really small 

because they=re due this December 1 and, therefore, they 

had to be small studies. But just because they=re due 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

50

this December 1, we can revisit a number of those issues 

so that it isn=t automatic that just because we gave a 

report up-front once on it that it=s over, because we will 

indeed discover that certain things are reoccurring 

issues and that they need a closer look. So, any of the 

318 studies can ultimately be reincorporated back into 

the broader 319 study because it=s an open-ended mandate. 

 You shall study -- an ongoing matter. So, it also talks 

about improving the accuracy and completeness of 

information. 

So, under that broad mandate, it is, in 

essence, up to the FTC to decide the variety of issues 

that it must revisit and, certainly, the 318 studies can 

be revisited. 

On the matter of the telephone contact, we 

certainly like the idea of email contact and the more 

consumers we can contact through email, the better. It=s 

just that we couldn=t just assume it. So, we=re hoping 

for a combination of telephone and email. 

On the matter of reimbursement, we do 

anticipate that we want to reimburse the consumer. Our 

funding office will not be very happy about this, but we 

think that is an impediment and we have to say up-front 

to the consumer, okay, we will pay for those costs and we 

might even pay a bit more in terms of some of the time 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

51

that you have to spend on this. So, that is a definite 

issue. We don=t know how that plays out, but we are 

definitely thinking along those lines. 

On the issue of the self-selection bias -- 

getting people to say, yes, I am willing to go along with 

you, okay, we imagine that if there is going to be a 

national survey, it will involve several thousands of 

consumers, meaning we invite them to participate. 

Whatever list we put out there of please participate, 

let=s say it=s a list of 5,000, you might get 2,000 saying 

yes and 3,000 saying no, we=re hoping and we=re planning, 

if this all works out, that we will get the complete 

redacted information on the 3,000 who said no, so that we 

can, in essence, assign the bias and study the nature of 

the bias. 

But we are very aware that there is a potential 

bias in saying, yes, I will versus no, I won=t. But by 

having the redacted information, let=s say, on the 3,000 

who said no, we can ask, well, is there average credit 

score any different from the ones who said yes, et 

cetera. We can look at the characteristics of the ones 

who said no versus yes and to see if we can assign this 

bias. So, we are very much aware of the issue of a selfselection bias. 

I just wanted to throw out those comments 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

52

immediately, and let=s have about another 10-minute 

discussion here before we move to the next presenter and 

the discussion is cumulative. So, now you can respond to 

what Ed and Alison have said or what Gerry has said or 

any of the comments that have been made. 

Alan? 

MR. WESTIN: Alan Westin. I was very cheered 

by the comment you just made because having done 50 or 60 

national privacy surveys, I>ve been very aware of the fact 

that we have a very skewed sample whenever we use 

telephone as the instrument because it=s the people who 

have caller ID, the people who will absolutely refuse to 

participate that often have the highest privacy concerns. 

 So, any time you do a telephone sample -- and people who 

do surveys can tell you the embarrassing figures about 

non-participation in standard telephone surveys -- that 

your suggestion or your commitment that you=re going to 

analyze the non-responders seems to me to be absolutely 

central to having any kind of trust in the sample of 

those who will participate. 

So, apart from all the other problems we=re 

discussing, it seems to me you=ve got a very good approach 

to that particular piece of problem. 

MR. VANDER NAT: Bob? 

MR. HUNT: Bob Hunt, Federal Reserve Bank of 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

53

Philadelphia. I just wanted to interject real quickly on 

the subscriber report idea. In Sweden, when you apply 

for credit, you have to get the same report the lender 

got. That=s the good news, so they don=t have this 

technological problem. The bad news is they don=t do 

prescreening, and so, the question is, how do you think 

about that in a system where prescreening is very 

important? 

MR. VANDER NAT: Greg? I=m sorry, Nick. I have 

to learn all your names yet. 

MR. SOULELES: Hi, Nick Souleles from the 

University of Pennsylvania. I just wanted to point out 

that there is a sub-field on survey methodology and 

within that field, the question isn=t whether people=s 

reports are accurate or not, the interesting questions 

are, how accurate and in which ways are they inaccurate 

and how do you minimize the inaccuracies. 

I=m sure to the people to whom you sent the 

pilot project proposals, they have survey methodologists 

on their staff, but to give you a sense of the sorts of, 

you know, issues that arise from other consumer surveys, 

like the FCF that many of us use, some of the lessons 

will show up in some of the designs that we=re going to 

talk about today, but just, you know, briefly, when you 

ask people a simple thing like their age in one year and 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

54

then you re-interview them a year later, sometimes the 

ages are going up by a couple years, sometimes they=re 

going down, even basic things like that. So, people have 

come up with techniques to try to deal with this sort of 

thing. 

A second way of comparing data, sometimes some 

studies have been able to compare self reports of things 

like income or amounts in a pension plan with the actual 

administrative data, say either from the IRS or the plan 

provider. And, again, of course there are inaccuracies 

and the question is how large and how material. 

Another sort of way of determining 

inaccuracies, and this is related to the prior point 

that, sure, you would expect that the larger the stakes, 

the more likely people would get it right, but it=s still 

the case that people get wrong even important things. 

One study that=s very close to the issues at hand, the 

Panel Study of Income Dynamics, is one of the best 

surveys that we have of households, and the great thing 

about this study is that it attracts the same households 

over time and a lot of tremendous work has gone into 

maintaining the sample and cleaning the data and so 

forth. Many of you know that a few years ago, they asked 

people -- they added a module about bankruptcy to the 

regular ongoing study, and these are retrospective 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

55

questions asking people, have you filed for bankruptcy 

and when and so forth. And there are sampling weights 

assigned to the study. It's possible to try to aggregate 

up the study and get something that is nationally 

representative. 

Now, when you go and look at these 

retrospective answers of did you file and aggregate up 

and then compare them to numbers -- the number of 

aggregate bankruptcy filings that we know are out there, 

you only get about half the aggregate, okay? So, 

somewhere there's an error. 

Now, it could be that there's a little bit of 

error in the aggregate, but it's got to be that the -- or 

maybe the sampling, the weights might be wrong. It might 

be that because of various biases in that study, the 

aggregation isn't working exactly right. But if it's not 

that, then the respondents -- and this is one of the best 

data sets -- many of us have made our research careers 

using this sort of data. I'm not -- it's a great data 

set, it's one of the best we have. But even with all the 

work that goes into making it as good as possible, the 

fact is that the numbers are only half of what they 

should be. 

Is it that people forget? Is it that people 

are embarrassed? We don't know. But that gives you a 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

56

sense of the sorts of errors that could be made even on 

important matters. 

How to deal with that? Well, again, that's 

a -- that's what we're going to talk about. I just want 

to point out that this is -- these issues are generic to 

survey design and studies of the consumer. I, myself, am 

more of a user of the data, not a methodologist. But 

there are people who work on the methodology of survey 

design, and I don't know if there are many here, but 

that's a group that you'll want to tap into. 

MR. VANDER NAT: Thank you very much. Evan? 

MR. HENDRICKS: Yes, thank you, Peter. 

Apologies for being late. On one, I think, relatively 

minor item where the consultant can help in the survey is 

the -- sometimes retailers use a bank to process their 

credit cards. So, it might be something like Lane Bryant 

or some retailer and they used either GE Monogram Bank or 

Household, and so, a consumer can look at their credit 

report and say, I don't have a household account, not 

realizing that that is actually the retail store card 

account. So, that's a minor issue. 

I think on the deeper issue, sometimes 

inaccuracies arise because of the difference between what 

happens when a consumer gets their own report and the 

strict algorithm that's used -- when you ask for your own 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

57

report, it requires a nine for nine matching of the 

Social Security number, all nine digits, and as well as 

all the rest of the data fields and you have to 

authenticate yourself and it's very rigorous and should 

be that way. 

But when subscribers, creditors order credit 

reports, sometimes it can be as much as a seven for nine 

matching of the Social Security number and that's why 

sometimes we have a difference between the subscriber 

report, what the subscriber sees about a consumer, and 

what the consumer sees about himself. 

So, one of the themes that I would like to 

develop throughout the day is that the credit reports are 

based on the information in the credit reporting agencies 

and the credit reporting agencies have monthly snapshots 

which are called -- sometimes called frozen data scans or 

name scans. And one methodology that we should consider 

is to compare what the consumer sees in their own report 

as opposed to what's shown in the back-up data and what 

potentially could be disclosed to creditors, knowing how 

these algorithms work. We discuss frozen data scans on 

page 132 in the book that you've ordered. 

MR. VANDER NAT: Mall? 

MR. DUNCAN: Mallory Duncan, National Retail 

Federation. First of all, I want to underscore a couple 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

58

of things that were said by Stuart and by Fred with 

respect to the importance of looking to materiality in 

this as a baseline. 

And secondly, as to the difficulty that Fred 

pointed out in accomplishing this, as Professor Westin 

pointed out, telephone introduced a bias. If you look at 

the issues that Alison was explaining in terms of her 

report, you've got people who are volunteers, that 

increases the bias, the fact that people who are familiar 

with the web is likely to introduce a bias, that it 

happened to be citizen members, all these elements. And 

there's no easy way to remove those. For that reason, I 

want to double underscore what Fred says, I don't -- I 

despair the Commission being able to pull this off with 

the degree of accuracy anywhere near the plus or minus 2 

percent that's necessary. 

I think the whole idea really should be sort of 

put to the side and we should instead look at, what are 

the possible areas that errors could occur in the report. 

 Well, there could be an error in the information that's 

actually furnished, sort of a filing error, a furnishing 

error. The second is that there could be an error in the 

way the information is compiled and presented back to the 

consumer. It's called a -- I call that a filing error. 

And the third possibility I would call, say, are errors 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

59

of malice. This is most commonly looked at as the 

identity theft error. And perhaps the Commission would 

be better served in its resources, rather than trying to 

get a very specific percentage for these errors, but to 

try to perform a qualitative study. I mean, 

fundamentally, you're going to go back to the consumer, 

you're going to have to ask them their opinion, you're 

going to have to judge that against what the -- a 

reinvestigation of the furnisher says. You're never 

going to have the level of accuracy that's going to 

maximize the sample. There's just a disconnect there. 

Perhaps you're better off to come back to 

Congress to say, look, we've looked at these three areas 

of potential errors and over the course of the 10 or 11 

years, here are suggestions we might make in terms of 

areas that are going up, that are going down. Keep in 

mind that while this process is going on, as Ed pointed 

out, consumers will be getting free credit reports. 

There will be changes in the system just by the fact that 

people will become more familiar over time. 

In the short run, it's impossible, absolutely 

impossible to provide the kind of expertise to consumers 

that even this room could probably not agree upon the 

particular points, to each consumer who has their report 

in front of them. So, you may as well accept that, take 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

60

a relatively small sample of consumers and try to pull 

qualitative information out of it rather than try to have 

a strict year-to-year we're up 5 percent, down 4 percent 

in terms of errors. 

MR. VANDER NAT: Thank you. Let me give a 

chance to this side. It's Bill Reeder, right? 

MR. REEDER: Bill Reeder at HUD. You've 

annunciated a lot of different goals, basically, and it 

strikes me that -- and you probably have different ways 

of getting at those. But it seems to me in terms of 

understanding the dynamic essentially of errors and error 

resolution and the materiality of errors, that -- 

following different approaches, rather than looking at a 

panel study and going forward or looking in those terms, 

perhaps backcasting, and I don't know whether one could 

enlist the credit reporting industry and the credit 

bureaus, a joint participation rather than kind of 

viewing it as we're going to go in and audit and try and 

see, to the extent to which there are errors. I think 

it's in everybody's interests to try and reduce errors 

and find out ways of mitigating those things and reducing 

them. 

So, if you could get the participation or 

enlistment of the big repositories, Equifax, Experian, 

Trans Union and other credit reporting agencies to come 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

61

together and participate, you have a lot of electronic 

data basically. Credit scores that summarize credit 

reports. You could, working with credit repositories, 

examine score change over time. This is the materiality 

problem basically. What is an error that makes a 

difference? Well, errors, that could be, I believe, from 

lots of things, registered in a score change. So, you 

have a dramatic change in score from some pattern and 

then something happens adversely. 

One could basically go back, identify those or 

stratified sampling on those types of things, look at the 

reason for those score changes. In the data that already 

exists, backcast within a year or two, find out how many 

were disputed essentially, look at this dispute 

resolution process essentially, look for undisputed items 

or score change and go back further. 

Ultimately, I don't know what the credit law 

permits, but getting permissible purpose or getting the 

agreement of consumers. If the credit industry was 

involved itself in terms of resolving issues, it may be 

that repositories have the ability or credit reporting 

agencies to actually go and solicit the participation or 

solicit information from the consumer to resolve the 

issue and explain or try to -- so, I guess what I'm 

saying is, essentially, look at the electronic data that 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

62

already exists, that pattern that already exists, the 

data -- credit data that already exists and look for 

patterns on the big ticket items or the items that make a 

material difference in score. Go back and analyze, get a 

picture or profile or what are the things that cause the 

score differences, how many are in error, how many are 

not in error. Ones that were disputed, you have some 

idea of what were the resolution of those, whether they 

were, in fact, not errors, but you can go all the way 

back to the consumer and essentially resolve the specific 

issue. 

Then you have a picture of, well, baseline, how 

well -- how well are the -- for things that matter, what 

are the causes of it and how much is in error, how much 

is not in error. And you can do that into the future, 

basically, the same study or follow it through time to 

see what happens. I think it would probably be a lot 

less costly if you can list the participation of the 

(inaudible) partners essentially of the credit reporting 

industry to essentially, in terms of cost, not have to 

buy all the scores. There would be some sort of cost 

sharing, perhaps, but it's in everybody's interest to try 

and find ways of mitigating that and improving the 

situation. 

And then second, if you were going forward 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

63

on -- okay, counting up the number of errors, you wanted 

to get a panel study, start at the beginning of a 

large -- reasonably large group of consumers, get their 

buy-in or something and simply follow the credit scores 

forward, examining changes and have permission to go back 

and resolve and identify, okay, what was the source of 

change that way, and it might be easier where you simply 

observe changes that you have to have an answer where you 

send out the mail -- kind of, did this happen and some 

things are a matter of record. 

You could follow things going forward as 

opposed to -- and not have to involve lots of (inaudible) 

credit counselors or at least a limited number where you 

have to have (inaudible) resolution. I don't know. But 

it seems to me breaking up your study into a couple of 

methodologies, a couple of approaches, and utilizing the 

data that already exists, that capacity. 

MR. VANDER NAT: Just one more (inaudible). 

Richard? 

MR. LE FEBVRE: Yeah, Richard Le Febvre. I 

want to first address what Fred and Stuart and Robert had 

talked about and that is that, first of all, I think we 

have to keep in mind, one, to a certain degree 

(inaudible) what Stuart was talking about with regard to 

consumers and credit repair, but first of all, I think we 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

64

have to focus on what the statute is, the consumer 

statute and that's what we're all here to talk about. 

And with regards to -- under the FCRA, I think we have to 

define reasonable, because right now, a reasonable 

investigation to the normal individual versus the 

Bureaus, there's a big issue there. 

Going back to what Paul said is, first, 

consumers do not have the knowledge to look at what 

affects their credit scores. They don't. While they 

have some input into it, but dealing with consumers for 

15 years in the credit reporting industry, I can tell 

you, they don't have a clue. 

While I agree that a certain percentage of the 

study should be with the consumer, I think it's real 

important that consumers, whatever you want to call them, 

experts, review those reports, get their idea by -- like 

to see a (inaudible) study and the NCRA study, it was a 

comparison. Take the comparison and then have the 

consumer look at them on top of it, because one of the 

things that that study lacked is what if all three 

repositories were in errors versus just doing a 

comparison? 

MR. VANDER NAT: There's one more presentation 

this morning, and then we'll have a bit more of a 

discussion, from Fair Isaac. We want to focus on those 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

65

errors that have a material change in the credit score, 

ultimately. And, hopefully, we have a process whereby 

the different parties agree that this has occurred and 

that's the most important outcome. We've asked Fair 

Isaac, again, just for the record, to remind us of what 

kind of information affects the credit score the most and 

whatever else you want to add to that. 

UNIDENTIFIED FEMALE: I was asked to provide 

just information on what information in the credit report 

is predictive in the FICO score. I am not including in 

this presentation our opinion on the study or other 

opinions that I've already gathered in the past hour, 

which are quite a few. So, I'm going to just spend 10 

minutes right now to talk about what information is 

predictive in the FICO score and interject some ideas as 

well. 

The first important point to me, and this has 

been mentioned by several folks in here, is that the FICO 

score is a summary of the information on the credit 

bureau report. It's certainly not a magic number that is 

all-perfect and all-consuming and could cover everything 

that is credit worthy on an individual. There's other 

information that typically goes into a credit decision. 

It is a single three-digit number, but the other 

important number here is that it rank orders consumers. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

66

The higher the score, the lower the risk, the better the 

credit quality over time. 

Now, the guts of the information that was being 

asked for here is, what information actually goes into 

the FICO score? When we develop a FICO score, we start 

out with over 300, 400 characteristics or variables that 

are potential characteristics into the scoring system. 

We pair this down to about 40 characteristics that end up 

in the final model, and out of these 40 characteristics, 

I divided it up here into five workable categories just 

so it's easy to talk about. 

These categories, and I'm going to speak to 

them in very generic or high level terminology today, but 

certainly if this study or this methodology is the one 

selected to perform this study, than more detail would 

probably be provided -- or would be provided. So, this 

is fairly high level. 

These five categories are listed here in order 

of significance of how they contribute to the FICO score. 

 The first category is payment history, in other words, 

how has a consumer paid their credit in the past is very 

predictive of how they're going to pay in the future. 

And I'm going to provide a little more detail on that in 

the next slide. That makes up about 35 percent of the 

weight in the scoring system. So, that's a pretty big 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

67

amount -- pretty big proportion of the weights. 

The second category, outstanding debt, or what 

I would like to think of how do consumers use their 

current credit. It makes up about 30 percent of the 

weights in the scoring system. So, what that means is 

between how they have paid their credit and how they use 

their available credit, this makes up at least 65 percent 

of the weight in the scoring system. So, this is pretty 

big. This is where the bulk of the data comes from. 

The remaining three categories are predictive, 

but not nearly to the same degree. The third category, 

credit history, how long someone's had credit, 

contributes about 15 percent of the weight in the scoring 

system. The fourth category, pursuit of new credit, 

contributes about 10 percent, and the credit mix 

contributes about 10 percent. 

I'm just going to provide a little more 

information on all five of those categories. The first 

one being payment history, so how consumers have paid 

their credits in the past is predictive of how they're 

going to pay in the future. We look at this in three 

different ways. We look at the recency of these 

delinquencies. So, did this delinquency occur last month 

or did it occur a year ago, and we found that to be very, 

very predictive. The severity, was it a 30-day 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

68

delinquency, was it a bankruptcy or a charge-off? That's 

also very predictive. And then the prevalence, if I have 

10 -- you have two consumers that both have 10 credit 

obligations and one has one delinquency and one has nine 

delinquencies, 90 percent, then that makes a huge 

difference as well. 

Some example characteristics that fall into 

this category include -- and these are example 

characteristics -- what's the highest level of 

delinquency in the past year, the number of months since 

the most recent collection and the number of times a 

consumer has been two months delinquent. 

An example of how predictive this is -- and now 

I'm just looking at one characteristic within this 

category -- I'm looking at the characteristic of months 

since the most recent major delinquency. On the 

horizontal access, this represents the number of months 

ago the delinquency occurred. So, for instance, this is 

the delinquency -- the major delinquency, the most recent 

one occurred in the last 11 months, or it occurred 12 to 

23 months ago and so on. And on the vertical axis is the 

risk that this represents. So, the higher the bar, the 

higher the risk, and you can see -- and this is fairly 

intuitive, but statistically we demonstrated this, that 

the delinquency occurred in the last 11 months. That 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

69

represents a pretty high risk for future defaults, okay? 

As time goes by, that risk lessens over time. 

In fact, initially, it lessens quite significantly. And 

then, as time goes by, if the delinquency, the major 

delinquency is four years old, it's not nearly as 

predictive as many people think it is. In fact, it's 

pretty darn close to it if they have no delinquency 

whatsoever. 

The outstanding debt, this information comes 

from the trade lines and we're answering questions such 

as the magnitude of balances, the utilization of 

available credit, and the type of credit, the amount on 

bank cards and new trades, and we look at this in a 

variety of different ways. Some example characteristics 

in this category include the average balance on revolving 

trades, the ratio of revolving debt to revolving limits, 

and this is a pretty darn predictive characteristic that 

encompasses a lot of good information, and then the 

percentage of outstanding on installment loans. So, 

we're trying to look at both revolving and installment. 

The remaining three categories, the three 

dimensions that are listed on that pie chart initially 

was the length of credit history, and an example that 

we're looking at there is how long have they had credit 

and we look at that in a variety of ways; the mix of 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

70

credit includes how many credit obligations and what type 

of credit obligations they've had; and then the last 

category is pursuit of new credit, and actually I think 

it was listed as number four on the pie chart, but it 

contributes the same weight as the mix of credit. 

(End of Tape 1, Side B)

UNIDENTIFIED FEMALE: -- consumer disclosure, 

and I know that consumers still aren't aware of that 

because I'm answering that question every day. Consumers 

ask me, wow, you know, if I can (inaudible) my score, 

that's going to lower my credit report. In fact, a cab 

driver asked me that yesterday. I said, no, no, you're 

okay. By today, he will be at myfico.com getting his 

score, trust me. 

So, the lender account reviews, when lenders 

are pulling scores for account review in a batch process, 

that doesn't count. Prescreen inquiries, those inquiries 

don't count. Employment and insurance inquiries also 

don't count in the FICO score. In addition to that, most 

of you are probably aware that we also eliminate or try 

to eliminate the effect of credit shopping. We do this 

by saying, well, we're pretty darn sure that a savvy 

shopper is going to check a variety of lenders when 

they're out shopping for a mortgage or an auto loan. So, 

therefore, we've put some routines in place that try to 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

71

identify this search for similar credit. Well, it looks 

like they have maybe 10, 15 different inquiries, but 

they're really only buying one automobile or one car. 

So, we do that with a buffer and de-dupe (phonetic) 

process and in general -- basically what that is is we do 

not count any auto or mortgage inquiries that have 

occurred in the last 30 days from the scoring date. In 

addition to that, we group together any auto or mortgage 

inquiries that occur over a certain period of time. If 

you're using our older versions of scores, that period of 

time is 14 days, and if you're using our most recent 

model versions, then that period of time is 45 days. 

Which brings up another point, we're 

continuously identifying issues, particularly consumer 

issues that evolve over time, and rate shopping is one of 

those issues -- is an example of one of those issues. 

Rate shopping was not prevalent in 1990 when we first 

started developing FICO scores. And as a result, that 

type of de-duping wasn't an issue, it wasn't necessary. 

Inquiries -- if you were searching for a car, you 

probably went to your primary banker for that credit. 

You probably went to one mortgage lender for your 

mortgage loan. But that's not the case today. 

So, we are continuously incorporating routines 

in the algorithms to compensate for the change in 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

72

consumer behavior. We're also -- we also include 

routines that compensate for potential errors to the best 

degree that we can. We don't try to reinterpret the 

data. That's a pretty important point. But what we do 

is we try to identify -- and an example is Peter asked me 

the other day, what if there are two mortgaged trades 

that represent the same mortgage line of credit on a 

consumer's report? And the answer is, well, you know, if 

there's two trades -- two duplicate trades, chances are 

one of them hasn't been reported over a certain period of 

time. If it hasn't been reported over a period of time, 

then we have a routine in place that looks at the last 

report date and says, you know, it's really old, so 

therefore, this is not a valid trade and we're not going 

to include in the score calculation. 

Those types of routines we're continuously 

researching and identifying the needs for new routines or 

updating existing routines to be sure -- to try to 

mitigate errors, duplications, omissions, et cetera -- 

not omissions, I guess, but (inaudible). 

So, with that said, the last slide that I have 

here is simply to demonstrate the predictiveness of the 

FICO score. This is on a mortgage portfolio, and all 

I've done here is I took an odds chart -- if you're 

familiar with our odds chart, and that is based on our 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

73

development sample of a million consumer files, and what 

we do is we say, out of these million files, what is the 

distribution of the population and then how do they 

perform by score, and that's what you see up here. This 

is the performance by score from low score to high score 

and then the delinquency is on the vertical axis. You 

can see that, of course, the higher the score, the lower 

the delinquency. 

What always strikes me when I look at this 

slide is the degree that it changes so quickly. It's, of 

course, very, very low up here. There's still some 

delinquency, but it is very low. And then right here, in 

the low 600s, all of a sudden, it starts to skyrocket up 

quite significantly. So, the point being that even 

though there are some issues with the credit report and 

there are probably some errors in the reports that we're 

actually using in our development, it's still extremely 

predictive as it is. That's it. 

MR. VANDER NAT: Thank you. We have a few 

minutes set for discussion. Let's enter into it now and 

it's cumulative. Brad, you had your hand raised. 

MR. SCRIBER: Thank you, Peter. My first 

comment I want to make is that I think that -- I'd like 

to thank the FTC for being so thoughtful in the 

construction of the design. I think a lot of the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

74

shortcomings, which everyone around this table would 

admit in their own studies or that they've seen in other 

studies, have been considered and you're really trying to 

address it and get at the basic question of are credit 

reports accurate, and I hope that's the goal of everyone 

around the table. I think it's a very good start and I 

just want to respond to a few of the comments that have 

been made on a couple different issues. 

On the question of materiality, the -- a couple 

comments have been made that small changes may not 

matter. But we've also noted that this is an 11-year 

study and the world is going to be different in 11 years 

and small changes that may not result in a denial today 

may result in a denial in 11 years. In order to 

understand if we're getting better or worse, we have to 

really look at even small changes now. 

Also, the items which have the greatest impact 

are going to change over time. I think as Fair Isaac 

would probably confirm, the algorithms change as we learn 

more about what data relate to what circumstance. So, 

the credit scores have to be considered and the scoring 

developers should be involved. It may be best if the FTC 

had a clearer vision of what the algorithms were to 

understand exactly how various elements impact a credit 

score and how those change over time, especially if the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

75

scores themselves are trying to correct for errors. They 

may introduce errors as well. So, I think it's worth 

considering. 

This would also -- of course, if the FTC had 

the algorithm, they wouldn't have to purchase the credit 

scores, it would lower the cost of the study. 

(Laughter.)

MR. SCRIBER: In terms of bias in the system, 

there have been some comments. A lot of focus on how 

consumers are going to introduce bias in the system. I 

hope we don't lose sight of the history of the system. 

We began with a black list system which is biased 

completely against the consumer. Thankfully, we've 

evolved somewhat to a more balanced system that includes 

positive and negative information. 

But the history of where this industry came 

from goes back to a black list and, also, if we look at 

the algorithms for including information on a subscriber 

report, which has been discussed, the bias seems to 

include as much information as possible which may relate 

to this consumer in case there's a negative piece of 

information that we didn't catch. So, that also skews 

more towards negative information. So, I agree that 

consumers are going to introduce some bias, but, you 

know, we need to be aware of the fact that the system 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

76

essentially is pretty well fortified to include negative 

information. So, not to overstate the influence of a 

consumer on that. 

And, of course, you've considered the 

mitigation with experts and verification and rescoring as 

possible ideas. 

I think it's very encouraging that you're 

looking at demographic information because credit reports 

don't include directly race into the credit score, 

sometimes it's difficult to assess whether errors or 

scores are fair across the board. I think that this 

study offers an excellent opportunity and actually has an 

obligation to take this on, to look at the impact of 

different types of errors on different demographic 

sectors. There's clear evidence, and even essentially a 

wrist slap from regulatory agencies, for missing 

information directed at lower income and minority 

consumers who use sub-prime loans. So, understanding how 

various errors affect different communities is very 

important, and I'm thankful that it's been already 

introduced and I would strongly support that. 

Just one more point on the question of cost, we 

heard last year from most of the people around this table 

that the accuracy and completeness of information in 

credit reports was vital to the economy of the United 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

77

States of America. I don't think it would be overstating 

to say that if we look at the testimony, it could be 

perceived to be one of the most important factors in the 

economy. So, if it costs money, it's worth it. 

MR. VANDER NAT: Thank you. Terry? 

MR. CLEMANS: Terry Clemans with the National 

Credit Reporting Association. I've heard a lot of 

comments about feasibility. So, there are several things 

I'd like to address, and that's one of them. 

It is not a small task. As somebody who has 

personally worked with consumers on their own credit 

reports for over a decade, I understand the challenges 

that come with it. I understand that there are consumers 

that are going to blatantly lie to you about what is in 

their histories. I've dealt with it, I've received 

fraudulent documents that are supposedly from creditors. 

 And it is a very difficult challenge to walk that fine 

line when you are given the position to try to interpret 

the truth. So, I think it's key that you have a column 

that says, yes, this is accurate, no, this is accurate, 

and yes, they just do not agree, the repositories and the 

consumer just do not agree on this particular issue. 

But I don't think it's unrealistic at all the 

way you have it outlined. One of the key elements to it 

is the expert. You have to have -- draw upon some 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

78

experts to help buffer what the consumer says and get to 

the bottom of what the real truth is because, ultimately, 

if we are going to judge accuracy, we've got to get down 

to the truth. Just getting repeated and accurate 

information, be it from, you know, well-intentioned 

organizations, either large companies, small companies or 

government entities, being county courthouses providing 

inaccurate records because somebody transposed something, 

we've got to get to the bottom of it. There is a large 

database out there. 

I say this because I represent a large group of 

them and this is something they do every day and they do 

it with a large responsibility in the consumer's mortgage 

and they have a responsibility to both the consumer and 

the mortgage company. If they make a mistake, they have 

liability either way. That's one of the reasons NCRA has 

such a large interest in accuracy. If our members 

overstate a consumer's credit worthiness, they have 

liability to the lender. If they understate it, they, 

obviously, via FCRA, have liability to the consumer. So, 

yes, there's going to be costs involved; yes, it is a 

very daunting task, but it is definitely a task that can 

be undertaken and you do have to start with the consumer. 

Even though they present some problems, they 

know, as it's been pointed out, they know if some of the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

79

basic things are correct, and you build from that along 

with reinvestigation. It's something that, you know, the 

score will help a lot upon and I don't want to go into 

part of our presentation for later, but it's going to 

have to be a layered multi-faceted study. I greatly, 

greatly appreciate the efforts you're putting forth to 

try to come to a sound methodology for this study going 

forward. 

MR. VANDER NAT: Thanks. Let's take a 10-

minute break here at this point. We will have lots of 

chances to discuss, and you will see that throughout the 

day, the discussion is cumulative. You can always come 

back to prior points as you wish to do so. So, at 11:00, 

we'll come back together. 

(A brief recess was taken.)

 SESSION II: METHODOLOGY FOCUSING ON RECORDS OF CREDIT 

 BUREAUS AND FURNISHERS OF INFORMATION

MR. VANDER NAT: Thank you all for getting us 

off to an excellent start. We're a little bit behind 

now, but maybe we can take five minutes off the lunch 

period. We have a rather substantial lunch period. It's 

an hour and 10 minutes, so we might shave off five 

minutes there or something to help us out. 

If you'd look at your agenda, please, we're 

moving to Session II and we're looking at a different 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

80

type of methodology now, one that focuses on the records 

of credit bureaus and furnishers of information, often 

known as an Arthur Andersen type of study. We do not 

mean to imply that all of these methodologies are 

exclusive to each other. In fact, there is a fair amount 

of overlap at certain points and we recognize that. But 

we also recognize each study tends to rely on one thing 

more than another. So, we're trying to localize the 

differences that way. 

We've asked Stuart Pratt from the Consumer Data 

Industry Association to start us off with a discussion of 

this methodology and whatever pertinent comments he wants 

to make about it. Stuart? 

MR. PRATT: Well, thank you very much again for 

having me here. Let me just touch on a couple of themes 

that I think we can draw out of the first session this 

morning. It was very helpful, certainly, for me to hear 

the broad views around the table and I want to thank FICO 

for a nice presentation on just what goes into the 

scoring side of this. I think it's important we have 

that on the table. 

First of all, I just want to go back to saying 

how important it is that we're defining accuracy as 

opposed to the extent this study is on completeness, also 

what we mean by completeness. I know that Gerry's 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

81

presentation attempted to start to put a finer point on 

that. Let me just say that incompleteness, you know, the 

error of omission, I guess as some are referring to it, 

concerns us quite a bit in terms of how this study goes 

forward simply because at its very core, the 30,000 data 

furnishers who provide data into the credit reporting 

system don't have to. That's the most important first 

point. None of them have to provide any data to anybody. 

 We're glad that they provide data, but they don't have 

to provide data if they don't so choose, and there's no 

penalty out there in the marketplace for them really, 

more or less, in terms of providing that data. 

So, that means there are going to be some data 

furnishers who probably don't supply data to any of the 

national credit reporting systems, and that may go to 

Ed's observation about a local lender in a local 

marketplace. It may also be true that that local lender 

simply through the years has done business with only one 

of what are now called the big three and so they tend to 

report to just that one and not the others. So, that may 

be a reason. 

There may be lenders, and we've certainly heard 

testimony about this, some lenders choose not to report 

on the credit limit. There you get into some -- by the 

way, I thought the paper the Fed did was a very helpful 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

82

paper just in terms of outlining the demographics of all 

the issues, if you will, that you see in the credit 

reporting system. In that sense, it was very helpful 

just to say, here's all these different kinds of 

variances that you can identify, and in this case, do you 

delete the data from the file completely because you 

don't have a credit limit, or do you build a credit score 

that will deal with the absence of a credit limit where 

that appears in the system. 

We think building a credit score where there's 

an absence of data and building it to be smart about -- 

that's the better approach because you're still using 

more full and complete predictive data in terms of making 

your final lending decision a good lending decision for 

the consumer and a good lending decision for lenders as 

well. 

I think that we've been going to this truth 

thing a little bit, you know, what is the truth about 

accuracy, and I want to at least add one other column 

here in terms of how we're going to debate this, and that 

is the effectiveness of the credit reporting system in 

the marketplace. A lot of people pshaw this and say, 

well, we shouldn't even focus on that at all, we're here 

to focus on just the individual consumer as well, and of 

course, lenders are ultimately focused on individual 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

83

consumers. They want to grant loans, they want to 

approve loans, they want to report accurate information. 

 We want to have accurate information, we want to make 

accurate information available, whether it's directly 

from the National Credit Reporting System or whether it's 

through one of the NCRA members in terms of delivering it 

to a local mortgage lender or mortgage broker, for 

example. 

There has been some very excellent work done on 

this and we have some of those folks around the table. I 

think the IPI's work that was funded through the U.S. 

Chamber of Commerce is exceptionally important context 

for our discussion of credit reporting. We have more 

credit available to more people than ever before. Ed, 

himself -- I know we have some differences in terms of 

interpreting the data. See, Ed, I'm trying to find 

agreement here. You know, it's a scary thing, and Ed's -

- you see Ed's eyebrows go up there. He was nervous. 

Where is Stuart going with this? 

But I think we both agree there are more yeses 

than nos. Now, Ed may say, I don't like all the yeses, I 

think some of the yeses could be better yeses, but we're 

getting to yes much, much more often in our country today 

than ever before historically. Home ownership is at 

higher rates than ever before historically. There are 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

84

more credit cards in the marketplace being managed 

effectively than ever before. Some 85, 86 percent of 

files contain only positive data based on what the Fed's 

review showed. So, there are incredible data out there 

showing how successful the overall system is. 

If we don't produce a report from the Federal 

Trade Commission that acknowledges that that is the 

context in which we're having this discussion, I believe 

the report fails miserably in terms of informing Congress 

first and foremost about that very fact, that we are 

looking at a system that is better than any other system 

in the country. 

I was invited to go to Seoul, Korea earlier 

this year because Seoul, Korea was literally failing to 

administer any kind of credit reporting marketplace and 

had 23 percent default rates on portfolios, 23 percent 

default rates. We showed up. Why? Because they're 

interested in everything from how do you get people to 

report data. I had a gentleman from Qatar stand up and 

say, how do you even get them to report data at all? Our 

banks just don't want to do it. So, I start with that 

because I think it's very important to start from that 

point rather than from the point that there's a big, big 

problem on the table that we're trying to wrestle with. 

We don't believe there's a big problem on the table that 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

85

we're trying to wrestle with. 

We do believe that these studies have to focus 

on not only the involvement of the consumer, if that is a 

route that you are planning to take, but then you must -- 

and I think your proposal encompasses this, but let me 

just emphasize it -- you must look at the data 

furnisher's experiences, how do they feel and believe in 

the data, you must look at the data user's experiences. 

Ed may believe that a single 30-day delinquency was 

consequential. I'm personalizing it to Ed. Alison, I'm 

leaving -- I'm going to focus on Ed, Alison. I'm just 

going to focus on Ed. 

But it may not be. As you saw on that chart, 

over time, that same scoring system may decide that that 

is progressively less and less significant to the overall 

lending decision. So, those kinds of moving dynamics do 

make it very difficult. You know, in some way, what 

Richard said is very true. Consumers have a hard time 

actually looking at a file and fully understanding how a 

score analyzes everything in that file. There are 

fantastic tools, by the way, to help with that, better 

than ever before. But those are all contexts in which, I 

think, we need to operate here. 

But I think errors of omission can be choices 

by voluntary data furnishers simply not to participate, 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

86

to participate with one, to participate with more than 

one. Credit bureaus have no power over a data furnisher 

to compel them to report anything to anybody. If the 

report does not acknowledge that, it fails miserably in 

its obligation to properly inform the Congress. 

So, for me, those are exceptionally important 

starting out points in terms of how we get to where we 

want to be. I'll reserve some comments on cross 

comparisons of files because I know we have another good 

discussion coming up where we'll have a chance to talk 

about that kind of approach. 

Finally, I just want to say recalling what we 

know -- consumers are the best test. I actually thought 

Alison did a very fair job of describing how -- it's kind 

of hard. Consumers will look, they can identify some 

things. By the way, consumers don't always know they've 

missed a 30-day late payment, they just don't know it. 

They don't remember ever not writing the check and they 

just don't know it and so they're pretty ticked off when 

they look at their file disclosure and see it on there. 

There is a staffer on the Hill who collects 

credit cards from every single baseball park in the 

country and turns them into magnets that he puts onto his 

refrigerator. I asked him if he closed all those 

accounts, he said no. I said, do you understand in the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

87

world of fraud today you probably should close all those 

accounts. They're wonderful refrigerator magnets, but 

you -- so, then he said, well, maybe I've closed some, 

maybe I've closed some, I don't remember. If he looked 

at his credit report -- the PIRG study acknowledged this, 

a reseller study that we worked with with our members 

acknowledged this, closed by the consumer versus closed 

by the lender are very significant differences in files, 

very significant in terms of scoring systems. 

Consumers don't always remember what they've 

closed or not closed. It doesn't mean consumers are not 

good potential starting points for looking at a file, but 

they are certainly not an endpoint by any means 

whatsoever. 

We did -- now I'm going to get to the study -- 

don't worry, Peter, you don't have to pull the hook on me 

here. We, too, took a look at the question of accuracy, 

if you will. Really, we looked more so at reliability, I 

guess is a better way to describe it, and in essence, 

we're looking at consequence, how consequential is a 

change to the system. This was a study conducted back in 

the very early '90s, '91 or so. Many lending decisions 

were more binary, a yes or a no approval, not approval. 

I think Ed said it right. We have a more stratified 

lending decision today where you -- so there are some 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

88

differences in terms of how we loan money today versus 

what we have in the study. 

What I liked about this study, though, I think 

what we liked about the study is we did employ an outside 

firm, in this case ironically, Arthur Andersen. Yes, we 

can all laugh about this now. You notice we now just 

talk about the CDIA study, we no longer call it the 

Andersen study. 

We went out and we talked -- we identified 

credit lenders in four different marketplaces around the 

country. So, we tried to distribute the involvement of 

lenders across the country. We asked them to give us a 

listing of adverse actions that had been taken against 

consumers, the identifying information. It was with that 

approach that we were then able to identify out of the 

100,000 plus adverse action notices, that we were able to 

extract a statistical sampling of 15,000 of those 

consumers. 

What we then started to do was to -- well, we 

asked ourselves some very straightforward questions. We 

just said, well, how many of those consumers, first of 

all, ever pulled their consumer report? How many of them 

ordered the free disclosure that Evan and Ed and others 

are so enthusiastic about? 

I was disappointed, by the way, I didn't get a 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

89

free book at the break. Evan, I'm still waiting for 

that. 

UNIDENTIFIED MALE: That's right. If the book 

is called Credit Scores and Credit Reports, it should be 

free. 

MR. PRATT: It should be free. I think that's 

right. I think that's absolutely right. I'm hoping to 

download it onto the net and make it widely available. 

UNIDENTIFIED MALE: Books are yearning to be 

free. 

MR. PRATT: Right. 

UNIDENTIFIED MALE: Perhaps it's in a library. 

MR. PRATT: You still haven't rung the bell, 

that's good, Peter. 

UNIDENTIFIED MALE: How about that report, 

Stuart? 

MR. PRATT: Right. 

(Laughter.)

MR. PRATT: 15,703 files were used for adverse 

lending decisions. Out of that, interestingly enough, 

only 7.7 percent of the consumers ever order their file 

disclosure. We don't know why, but we do know that. We 

just know that 7.7 percent or 1,223 consumers order their 

file disclosure. Now, of that, only 304 consumers 

disputed information in their file. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

90

You know, a lot of times we talk about, do 

consumers -- we need studies of consumers to review 

files. Of course, every year, millions of consumers are 

reviewing files as a result of this. This is a small 

sample, of course, ultimately, but this is drawn from a 

very large statistical sample. So, 304 consumers, which 

is about 2 percent of the study sample, disputed 

information. 

What we then did was go back and actually ask 

the lender, once file maintenance was conducted, the 

reinvestigation was completed and the results were then 

included in the file and the question was then, would the 

lending decision have changed? Again, in the yes or no 

binary world in which we live, it was really, did you 

approve what you previously had declined? And of that, 

36 cases resulted in a reversal of the decision. 

Now, what did that tell us? That told us that 

a very, very small percentage of the files in this study, 

at that time out in the marketplace, the real 

marketplace, drawing down from statistical sampling 

techniques, drawing down from samples drawn from across 

the United States, that very, very small percentages of 

files were ever implicated in a serious problem that 

resulted in a material effect. The material effect in 

this case was an approval where there was previously a 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

91

denial. 

Now, I have to say I do share the basic value 

around the table, and that is, would we like to have had 

all of those cases not occur? Absolutely. Would we like 

to have all information reported as completely and as 

routinely and as timely as possible? Absolutely. But 

when you're talking about now not just even two billion 

but closer to four billion data elements being updated 

every month in credit reporting systems, that's a pretty 

incredible proxy, if you will, when you look at that 37 

and you look at where we are now and we look at how 

accurately we're extracting data, bringing that data back 

to the lending community, bringing that data back to 

automated underwriting. 

So, for us, that study was compelling and the 

study was compelling because it did involve consumers 

reviewing their files, it did involve lenders, data 

furnishers than agreeing with or not -- with the 

consumer's reinvestigation request and it did ultimately 

involve lenders actually determining the consequence of 

the change. And if a study fails to have those factors 

in it that you really haven't gotten to the kernel of 

this, there will be some volatility in scores, I suspect, 

shifting back and forth as a result of any file 

maintenance. That would happen in a completely positive, 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

92

accurate credit reporting system, right? 

In the absence of any error, in the absence of 

any negative data, there would still be score shifting 

going on month by month by month. Surely, that is not 

what we're driving towards is a score which remains 

exactly the same on a monthly basis. Surely not. We're 

driving towards a system that works well for not just the 

individual consumer, as heretical as that sounds, but for 

the lenders and the safety and soundness of the system 

that we have in this country because this system is what 

has been the engine that has driven this economy. In 

that sense, I agree. It is critically important the 

system works well. 

I think perhaps where we move apart a little 

bit is, I'd say it is working incredibly well for the 

vast majority of Americans in this country, and that's 

what we know as a result of this data that we've now 

presented. 

MR. VANDER NAT: Thank you. We've asked 

several other people to comment further. There's Bob 

Hunt from Philadelphia -- 

(End of Tape 2, Side A)

MR. VANDER NAT: -- ask Bob to give us his 

comments at this time. 

MR. HUNT: I have to start with this strange 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

93

statement which is, these views are mine and not 

necessarily those of the Federal Reserve Bank of 

Philadelphia or the Federal Reserve System. I have to 

say this any time I do a talk. 

Let me start with just a point about Section 

319, which is what -- of the FACTA, which gives us the 

study requirement. It simply says that we're supposed to 

be looking at accuracy and completeness of credit bureau 

files. Now, I would argue that really what we want to be 

thinking about is how do you maximize the net social 

benefits of having a credit reporting system, and in this 

country, a voluntary arrangement that does this. So, 

probably the goal at the end of the day is not 

eliminating all inaccuracies. 

It's going to be about -- and I'm going to 

conjecture here because I have no idea. But suppose for 

the moment that the inaccuracies are really decisive in 

10 percent of the instances and that's very costly for 

people, but whatever you're going to recommend is going 

to affect policies by lenders and bureaus everywhere, 

you're going to need to have a handle on the costs and 

the benefits to be able to decide, you know, what are the 

best policy responses. 

Now, let's talk for a minute about -- the 

reason why I make this point is because then the study -- 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

94

you have to design the study with that in mind. What is 

the information that you're going to need to do this. 

The Arthur Andersen study -- I should say the CDIA study 

is -- if you look at the results of it, it's actually an 

interesting way to think about this problem. Stuart did 

a nice summary. You've got a bunch of people that were 

denied credit. A sample of those are pulled, about 

15,000. Identifying information on those people is sent 

to a bureau, okay? Then as consumer disputes come in, 

they try to match that identifying information with the 

identifying information that consumers provided, and 

that's how they generate an account which said roughly 

1,200 people asked for a copy of their report; of that, 

300 disputed something in their report. 

By going back to the lenders and asking them to 

rescore the new file, they found that 36 instances they 

would have changed their credit decision. As Stuart was 

saying, from the sample population, that means that 

roughly 7 to 8 percent of these people who were denied 

credit asked to look at their report. About 2 percent of 

that sample population disputed something and about twotenths of a percent of that sample population might have 

gotten a different credit decision with whatever 

inaccurate information was taken out of it. So, that's 

one way to look at the data. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

95

The other way is to think conditionally, that 

is, amongst the guys that request you to report, how many 

disputed? And the answer would be a quarter of them did, 

okay? So, when they saw the report, 25 percent were 

worried about something that was in it. 

Of the people that requested their reports, 3 

percent of those credit decisions would have been 

changed. So, all I've done is I've looked at the data 

differently and said, the guys that were concerned enough 

to request their reports, how did this behave? And we 

can cut it even more finely. For example, just take the 

guys that decided to dispute their reports, the 300 or so 

people. How often was the credit decision reversed 

amongst that set of people? And the answer is 13 

percent. So, when you're thinking about sort of the 

policy question at the end of the day, you have to ask 

yourself both about the concentrated impacts, which is 

what these sort of conditional probabilities tell you 

about. 

But you also have to go back and think about 

sort of these larger sample numbers. For example, I'm 

going to make a number up here but it's probably not that 

far off. Suppose there are about 400 million credit 

decisions a year made at the time that this study was 

done. Two-tenths of a percent reversal rate on a credit 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

96

decision is many thousands of credit decisions. Now, 

that's actually the wrong way to do the numbers because I 

guarantee you that in this sample, there were more 

problems in these files on average then on files in 

general. The problem is is we have no idea about the 

files in general. So, you can easily get into trouble 

generalizing if you don't gather the data that you need. 

Now, I actually like this study for a lot of 

reasons. First of all, they drew a random sample, which 

was a pretty novel thing at the time to do. Secondly, 

they have a very intuitive definition of what is a 

critical error in a report, and that is you didn't get 

credit when you should have. That's pretty neat. As 

Stuart was pointing out, that's harder to do today 

because what you're going to get is an acceptance with 

higher interest rates or other terms and that becomes 

problematic today. 

The other point I would make is it's a 

relatively efficient approach in the sense that by 

relying on both lenders and the Bureaus, what they could 

do is essentially use the data and the systems that both 

these guys use in exactly the way they use it, so the 

incremental cost of them to participate in the study was 

relatively low. And as we were talking about today, with 

the surveys, we're not sure that that's true. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

97

Now, there are disadvantages as well. There's 

not -- we don't know a whole lot about the nitty gritty 

of this study. So, it's difficult to be able to 

generalize. For example, we don't know anything about 

the information that was corrected. For example, was it 

utilization? Was it duplicated lines of data, et cetera? 

 We have no idea what it was. If we were to replicate a 

study like this today, we probably would like to know 

those characteristics of the item that turned out to be 

decisive. 

Also, it's important to know what forms of 

credit were applied for. I can see the implications 

being different if you're talking about a mortgage, a car 

loan, a credit card loan, you'd probably want to segment 

accordingly. The most simple way I can say this is that 

if I was going to do the study again, I would want 

control groups. That is, I would want a set of consumers 

from the same lenders whose applications by these five 

institutions were accepted and I would like to be able to 

follow their credit report and follow their score. That 

way, I can compare them to the guys that were rejected. 

Even more generally, what I might do is just 

randomly draw a sample of credit bureau files at the 

bureaus, okay? Now, why would I do this? The first 

control group says, these guys were actively looking for 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

98

credit. The second control group says, these guys are 

credit using Americans but they're not necessarily 

actively looking for credit today. And if you can 

compare these groups with the group the study did cover, 

which is the guys denied credit and requesting the 

reports, then you get a much better sense of the 

distribution of mistakes and the implications of mistakes 

across all of the bureau files. I think you need to know 

those kinds of things. 

Let me -- now, suppose the Federal Trade 

Commission was to go this route, what are some things 

that they're going to have to think about today? The 

first thing is obvious. We have different privacy laws 

and constraints and even if the laws are not binding, 

there's this issue about whether people are going to be 

willing to participate. For example, if I want to get 

that universe of credit files, do I have to get 

permission first? I'm not really sure how that works. 

Also, there may be some issues about trade 

secrets in the sense that if we really want to dig down 

and find out why it is that this kind of information 

generates big jumps in scores or exactly how we actually 

pull all of this stuff together into the consumers' 

files, you know, you guys can correct me, you know better 

than me. But do you worry about how much you're giving 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

99

away, about what is your business? I don't know. 

Let's see. One thing -- this is true both in 

the consumer survey idea and in this approach is you'd 

like -- you're going to need some demographics. So, even 

if you can't get people to participate, if you could get 

them to give you some demographics, that's going to help 

you in setting up a control group and evaluating what it 

is that you're learning. 

I'm going to -- I'll stop there, I think that's 

enough. Thank you. 

MR. VANDER NAT: Since the comments are fairly 

brief right now, let's just move on the next person, and 

then after Fred Cate has spoken, then we'll have a group 

discussion. 

MR. CATE: Thank you very much. I was given, I 

don't know, either enviable or hellish task depending 

upon how you want to look at it. I was asked to talk 

about this study and then Peter said, well, we don't 

really want you to talk about the substance of the study 

because Stuart's going to do that, and he said, of 

course, Robert Hunt is going to talk about the 

methodology of the study, you just talk about whatever's 

left. So, I'm going to feel free to do that. 

I would also say, for those of you who missed 

th earlier plug, you really should go out and buy Evan's 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

100

book. He'll sell it to you right here. He calls me a 

top gun in it, and as a law professor I don't often get 

described in such militaristic metaphors and I am 

delighted by it. So, I would encourage each of you to 

buy a copy and buy one for your friends as well. 

Let me just make a half a dozen observations 

about the study. Some of these are going to overlap what 

has already been said, and I would note, said by someone 

who frankly knows considerably more about doing studies 

like this than I do. First of all, of course, the study, 

I would assume, it would appear would be biased towards a 

higher error rate because we are only following up on 

people who had some reason to object. Therefore, we 

would assume that whatever the error rate is, whether we 

say it's the 2 percent or 12 percent or however we 

measure it, that the overall -- that when we try to 

extrapolate that to the entire population, that's skewed 

towards the negative. 

Remember, also, even of those who were denied, 

only a quarter even followed up by asking for a report. 

So, again, we have to assume the other three-fourths 

didn't think they saw anything in their report that was 

worth following up on, that there was -- so, again, 

another reason we might think the data itself would be 

skewed towards a somewhat negative outcome. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

101

A second observation is that, of course, this 

type of study always has the problem of we don't know why 

people ask for or don't ask for their file, and we can 

make certain assumptions about that, but we know, 

particularly those of us in the room who deal with 

privacy, there are lots of reasons people don't opt out 

or don't opt in or don't whatever, and it may or may not 

have something to do with discovering errors in the file. 

 I think it's intuitive common sense that if you 

saw a glaring error, a bankruptcy that you didn't 

declare, a significant delinquency that you didn't think 

was appropriate, you would be more likely to follow up, 

but we know there are many other good reasons, apathy 

being probably leading among them why we -- why people 

just don't follow up, don't ask for their report to start 

with. 

The third, as has already been noted, although 

this was not, I think, a problem at the time the study 

was done, today, you know, differential risk-based 

pricing makes this type of study a little more difficult 

to do because you would have to figure out what does it 

mean to be denied. It's obviously not going to be that 

binary question anymore. 

The sample in the study was large, it was 

comfortably large, but not necessarily representative. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

102

So, it was chosen through methods we really don't know 

because Arthur Andersen didn't make the methodology in 

this point public. You know, we have less than 10 

percent of all of the denials who were actually part of 

the study. Those were, again, information sought from 

only specific creditors and only in certain geographic 

markets. There's no reason, and in fact, Arthur Andersen 

went out of its way to suggest that there was no reason 

why this would skew the sample, but nevertheless, the 

difference between a random and a representative sample 

is one which the Commission, of course, is wellacquainted with and I think would have to be taken into 

account in future studies. 

I also, like Bob Hunt, find this fairly 

attractive because, of course, we have a factual, actual 

resolution. Did it change that whether credit would be 

granted or not? And, therefore, we have a measure of 

materiality, we have an outcome that is fact-based rather 

than opinion-based. I, nevertheless, would just 

highlight, and I made this point earlier, again, the 

whole triggering mechanism is consumer concern. Do I, 

the consumer, see something I'm not happy with and just 

two sort of observations specifically about that. One is 

it means we're not measuring people who got credit, but 

should not have, and I think -- Stuart made this point 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

103

earlier and I think it is a very important point. If 

we're talking about accuracy, we can't just mean harm to 

an individual consumer as part of accuracy. We must also 

be thinking about harm to all consumers, harm to the 

system, harm to the efficiency and economic reliability 

of the system. 

The other thing that this reliance on the 

trigger of consumer action tells us, and again, I'm not 

quite sure what to make, of the 267 people who actually 

got a report and disputed something in the report, so 

these are the consumers who say there's something wrong 

in my report, and we got to full investigation. So, in 

other words, they weren't pending as of the time the 

study's results were announced, only 36 resulted in any 

change in whether credit was granted or not. 

Now, that means if we're going to try to 

measure the effectiveness of the consumer I think there's 

something wrong as an indicator, only about one in eight 

times was the consumer I think there's something wrong 

relevant. The other seven out of eight times, there 

either was nothing wrong or there was nothing materially 

wrong sufficient to result in a change in outcome. So, 

that suggests what I would call the sort of fundamental 

inefficiency of relying on Consumers as a place to start. 

 If they're wrong seven out of eight times, that's 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

104

something we should be concerned about. There is 

certainly data from other studies that would support 

similar numbers. 

Robert's already made the point and I don't 

want to expand on it other than just to say, I also 

thought this was a particularly efficient study. It 

didn't require layers of consumer counseling and lots of 

contact and so forth. It took advantage of an existing 

process. Obviously, I have no earthly idea what it cost 

to do, but for what it generated, I would think it was 

economically efficient as well. 

That leaves just one last comment which, again, 

goes back to something I said earlier, so I will try not 

to belabor it too far, which is it doesn't distinguish 

among types and causes of errors, and I think for what 

the FTC has been asked or told by Congress to do, I think 

that's critical to make a real effort to build a study so 

that it has a likely chance of telling us what are the 

types and causes of errors. 

And, again, I think at a minimum, this is a 

very simplistic, sort of top gun law professor's view of 

this, but, you know, we have at least three -- we have 

furnishers, we have bureaus and we have the credit 

grantors or the users of the credit report, and it seems 

to me that a study that does not make at least a serious 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

105

good faith stab at separating out what types of errors so 

we might know where they occurred -- was it furnisher 

error, is it a situation where we need more furnisher 

information, more consistent furnisher information, 

mandatory furnisher information? Was it a bureau error, 

was it put in the wrong place, was it sat on a desk for 

weeks and weeks before somebody got around to putting it 

in a file? Was it a data processing error there or was 

it a grantor error? I think that's one thing we would 

like to know. I think there are ways you could begin to 

know that from this type of a study, by looking at not 

just how many credit granting decisions, or in this case, 

pricing decisions were reversed, but also looking at why, 

what was the nature of the error, trying to categorize 

those errors in a way that might provide more useful 

information for the Commission and for Congress. Thank 

you. 

MR. VANDER NAT: Thank you. We'll have a very 

good time here for a discussion period, so we can just 

start right in. As I said before, the discussion can 

build, you can refer back to prior things, to the 

immediate things. Let's just begin. Alan? 

MR. WESTIN: Alan Westin. Isn't it interesting 

that nobody has said yet that the system of consumer 

reporting is not going to be a target that stands still? 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

106

 In fact, it's going to be undergoing enormous change as 

a result of the FACTA Act and the Free Credit Report 

System. So, the dilemma is what's your trend line, what 

is your baseline against which you're then going to try 

to say what difference does it make that people can now 

get a free credit report from all three of these, what 

effect will that have on furnisher behavior, on CRA 

behavior, and recognize that even though we might say, 

well, consumers may not take advantage of this as much as 

some predictions, the fact of the matter is that we know 

that every time there's a column in Newsweek or every 

time there's a report on major television and cable 

stations, you get up-spikes in people who listen and say, 

gee, I guess I really ought to look at my report. 

So, to the extent that the media and consumer 

organizations and industry associations incent consumers 

to go and look at their report, there are going to be a 

whole new wave of pressures and impacts on the furnisher 

side and on the consumer reporting agency side. So, the 

dilemma is how do you measure what will happen to the 

accuracy and completeness levels in such a turbulent 

environment that is going to be undergoing reactive 

changes to the whole driving force of the FACTA Act and 

the free credit reports. 

MR. VANDER NAT: Um-hum. Brad? 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

107

MR. SCRIBER: I think that's a great point. 

Unfortunately, this study wasn't done from 1995 to 2004, 

so we know the impact of the 1996 laws. I would say get 

started now and, unfortunately, from our point of view, 

consumers aren't going to all get their credit report at 

the same time, but perhaps for this study there is a 

hidden silver lining to the roll-out of consumer reports. 

 So, getting detailed demographic information about what 

state people are in will allow you to look at the impact, 

at least in the first year, and presumably this is going 

to take a while to really settle in. 

As a consumer organization, I'd love to think 

that by next September, every American is going to look 

at their credit report, but I'm more of a realist than 

that. So, we're going to have an ability to look over 

time with each of these interim reports at what is 

changing, is it sinking in, are consumers looking at 

their reports and is it having an impact? 

MR. VANDER NAT: Paul? 

MR. WOHKITTEL: Paul Wohkittel. I want to 

agree with a lot of what Stuart said. In addition to 

owning a credit reporting agency in Baltimore, I serve as 

a consultant on three different international 

construction projects for credit bureaus. In Kazakhstan, 

we figured out a way around the participation and the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

108

errors of omission and that was just real simple, just 

pass a law and make it mandatory to report. 

(Laughter.)

MR. WOHKITTEL: I don't think that's going to 

work over here. I don't think it should work over here. 

 I think it's great for a former Soviet country, but 

that's what distinguishes us from a former Soviet 

country. 

But that being said, because someone doesn't 

report, that doesn't mean there are not errors of 

omission. This is where experts come in and also, I 

think, one major flaw in my thinking just because 

somebody does not dispute what they do not see in their 

credit report doesn't mean there are not errors. 

As an example, we -- at the reseller level, 

we're down in the trenches and we're working with 

consumers for the purpose of getting approved for their 

mortgage, and a very common occurrence is that when they 

see their denial because of too much derogatory 

information, and this happens quite frequently, a lot of 

times the reaction is, oh, gosh, I knew not paying my 

Sears bill on time three years ago was going to kill me. 

 That's it, it's dropped. They have no idea that there's 

a medical collection filed by Prince George's County 

Hospital four months ago for something they never even -- 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

109

they had. This is, again, where the experts come in. 

This is the guidance that they need, but not only to 

guide them from not -- from over-dwelling on what might 

not be an error, but from also picking up what might not 

be perceived. 

MR. VANDER NAT: Thank you. Evan? 

MR. HENDRICKS: I wanted to thank Fred. I 

thought that was an excellent presentation and I truly 

agree with your points that -- I think one of the 

important things I'd like to develop is to look at areas 

where we have at least anecdotal evidence of chronic 

inaccuracy and that that should be a subset. I think 

Mallory raised this before. Where are the problems and 

how can we understand them? 

I think that -- you know, we talk about bias. 

In the Arthur Andersen study, there was bias there 

because, one, the context this was going on is this is 

when the legislative debate was firing up and PIRG had 

released its studies raising serious questions about 

accuracy and this was the industry responding with their 

version of it. You know, at that time, there was a very 

low consciousness about the importance of credit reports. 

 They weren't as important then -- they were very 

important, but they weren't as important then as they are 

now. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

110

Two, credit reports were very hard to read and 

understand back then. I think -- I give the bureaus 

credit for actually making their reports much more user 

friendly. A lot of people still have trouble 

understanding them, but they're a lot better than they 

used to be. 

But the real problem, I think, with that study 

is that it defines -- the only problem it defines is if 

you're actually denied credit and then you change it and 

it reverses the credit decision. Now, in the Federal 

Reserve Board and among economists, these are the main 

things that we're looking at is what is the effect on the 

economy, on interest rates, on how much people spend, but 

the Federal Trade Commission has, within its 

jurisdiction, the issue of fair information practices in 

consumer protection, and I think if Paul talked further, 

you would hear that there are some situations where there 

are inaccuracies in credit reports which can harm the 

individual or be considered harmful if they discover 

them, which might not necessarily lead to credit denials 

because that person's in a situation where they're not 

applying for credit, but they might be the victim of 

identity theft. They might not like being characterized 

along as a fraudster or as a deadbeat, though they might 

not be out applying for credit. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

111

I think that we have to remember our -- you 

know, we have in the room Alan Westin, the creator of 

Fair Information Practices, and its legacy, and I think 

that we have to keep that in mind, that there is more at 

stake than just whether you're denied credit. There is 

people's good name and the accuracy of their information. 

Now, I think that the other thing is, in moving 

forward, I would like to see Bill Reeder's point of 

enlisting the CRAs to really cooperate in many of the 

things that can be done here in terms -- because they are 

sitting on the information that determines what's in 

people's credit reports. So, you know, the thing I 

think, Stuart, the one commitment that you can make here 

today is to say that, you know, that the CRAs will be 

willing to work with the Commission and help foster these 

studies by making -- possibly creating certain pilot 

audits that we can discuss and explore to find the most 

effective way. But I see that as a real shortcut to 

getting to the underlying data and getting some very 

useful information. Thank you. 

MR. VANDER NAT: I think Gerry wants to make a 

few comments. 

MR. BUTTERS: Just Peter, when he asked me to 

speak at the beginning said, don't be afraid to say 

things that are obvious. Please say the things that are 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

112

obvious. And I plead guilty already because one of the 

things as an economist that I think is obvious is that 

we're not just looking at how many errors there are and 

how many inaccuracies there are, but we want to know what 

is the consequence of these errors and we want to know 

the cost and benefits of making changes in the system. 

That's actually part of Congress' mandate because they're 

asking us -- they didn't say specifically costs and 

benefits, I believe, but they did say to look at possible 

changes in the system. 

So, I fully agree with the thrust of the 

comments from many people here that we should be doing 

that. It's not just enough to look at the errors, but 

also what's the cost of changing things. And one cost is 

that at least if you try to be very careful to get rid of 

all false negative information in a consumer's file, that 

you might drive a lot of information out of the files 

entirely which could lower the usefulness of credit 

reports in general in terms of their predictive accuracy 

and that could be harmful to consumers, at large, in 

terms of their ability to get credit. 

So, those points are well taken. But the 

reason I hadn't brought them in yet was because I hadn't 

-- maybe I'm too slow here, but I haven't figured out yet 

how to get that discussion integrated into the design of 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

113

the study. So, that's where you could help me. You 

could tell me, okay, how -- when we're studying accuracy, 

what is the connection between these costs and benefits 

that ultimately we want to work into the analysis and how 

we should design our measurement of accuracy and 

completeness. 

MR. VANDER NAT: And here I can remind you of 

the point, even though we cannot fully discuss that here 

or may not want to here, anyone who has suggestions on 

this and comments on this are invited to give them to us 

in writing and we will very carefully consider them. 

UNIDENTIFIED MALE: Peter, could I follow up to 

that quickly? Back to basics, you know, the Fair Credit 

Reporting Act tolerates inaccuracy. It's not a strict 

liability statute and it doesn't require 100 percent 

accuracy. It requires reasonable procedures to maximize 

accuracy. So, it tolerates a certain level inaccuracy. 

In fact, furnishers get a free bite of the apple. I 

mean, they're allowed to report inaccurate information. 

It's only after that they're notified that the 

information is inaccurate or that they know that it's 

inaccurate that the liability kicks in. 

So, I think policy -- you know, keep in mind 

the policy does tolerate a certain level of inaccuracy. 

You balance that against the purpose of the act and 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

114

that's the credit reporting agencies and, I think, the 

credit grantors have to live up to their grave 

responsibility to do everything to make sure the 

information is relevant, complete and accurate. So, I 

think those are the parameters to examining an accuracy 

study. 

UNIDENTIFIED MALE: I have one request as we go 

forward with this discussion and it goes back to this 

omission question. I know that we all choose our words 

because sometimes in the public fora, particularly in the 

media fora, it's convenient to use them because they're 

nice, they're inflammatory, they help with your message 

because it will get published. But there's really no 

such thing as an error of omission. It's not an error. 

If a lender doesn't want to report its data, that's not 

an error. If a lender only reports to one of the three 

systems, it's not an error. 

From what perspective? From the perspective of 

the FCRA which regulates individual consumer reporting 

agencies, individual data furnishers and individual data 

users on a company-by-company basis. The FCRA is not a 

law which regulates the industry holistically as one 

monolithic law. So, this whole -- would we like to solve 

the question of how to get more data into the file to 

ensure a more complete decision? Absolutely. Do we work 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

115

at that? Yes, we do. But I think that, again, the study 

does an incredible disservice and really promulgates 

disinformation if it embraces the term that there is any 

such thing as an error of omission, particularly in the 

cases where a data furnisher simply does not want to 

report to anybody or where a data furnisher chooses to 

report to one of all three. 

One of the interesting outgrowths of the free 

file disclosure question is to what extent will there be 

a dispute rate associated with free file disclosures and 

to what extent will some data furnishers decide, well, 

I've had enough and I'm going to contribute to the 

omission side of this balance sheet because I'm going to 

stop reporting because I no longer want to report because 

I can't afford to keep handling disputes about 

perceptions of inaccuracy, but where I don't think I have 

a real inaccuracy at the table. 

MR. VANDER NAT: I will briefly respond first 

and then I'll let others respond, also. You've raised 

the important point of definitions, how do you define an 

error of omission, and you've pointed out a way in which 

it's very difficult. 

UNIDENTIFIED MALE: It's not an error. 

MR. VANDER NAT: Right. But here's another way 

that -- in which it could be an error. Let's look at the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

116

information that is in the databases of the CRA, okay? 

It's in the databases, it has been reported by somebody 

and the question is, does it have bearing on credit 

worthiness, would it have an impact on the consumer's 

credit score, and we could say there's an error of 

omission if there's something that's in the databases and 

it is -- it has bearing on credit decisions, but somehow 

it didn't get into the report. Now, that would be a 

definition under which you would have an error of 

omission. 

UNIDENTIFIED MALE: Well, this goes back to the 

discussion, I suppose, of data matching, Peter, that 

we're having right now as well. 

MR. VANDER NAT: Yes. But I'm just trying to 

suggest that there are definitions. 

UNIDENTIFIED MALE: But there's no error. 

There's no -- if the file polls all of the -- if the file 

polls all of the data which can be polled based on the 

information submitted by the consumer on the application 

and that is delivered, there's no error. 

MR. VANDER NAT: So, in terms of -- 

UNIDENTIFIED MALE: (Inaudible) how the law 

operates. I think it's very important that the BOE try 

to align its thinking with what the law requires or 

doesn't require. Peggy and I were having a quick sidebar 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

117

just on, well, how -- for example, some of the 

suggestions you're making in terms of, well, we're going 

to score all the consumers who are in non-response 

groups. Well, I don't know how you get to that because 

if they haven't responded, then they have no permissible 

-- you have no permissible purpose by which you're going 

to be able to score non-respondents to a survey of this, 

at least based on the way the law operates currently. 

I'm just saying this -- I'm asking you to align 

your thinking in terms of how the BOE is going to 

approach the study and this is a relevant discussion to 

the whole comparison, file comparison question. You 

know, Richard and I had this discussion over at the CFA 

meetings before, that bureaus are not obligated to go 

checking each other's files to see what data the other 

file has which I might not have. 

MR. VANDER NAT: Okay, thank you. Ed? 

MR. MIERZWINSKI: You know, I just want to say 

from my perspective, I think that the notion of getting 

into a debate, you know, some semantic debate of whether 

the law does or does not require this kind of thing to be 

an error, I think, is something that would be a wrong way 

for the study to head. I think there's very clear 

Congressional intent and intent by the administrative 

enforcers of the Fair Credit Reporting Act that 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

118

information in credit files be as accurate and up-to-date 

as possible and that Congress change some of the wording 

of the Act this year with the integrity language on 

credit files to try to address this problem without going 

as far as we would have liked to address it further. But 

we have documented evidence from Senator Shelby's hearing 

that some credit bureaus -- excuse me, some creditors, 

some furnishers, are intentionally -- they know -- we 

talked a little bit earlier about trade secrets and we 

talked about, well, you know, we don't want consumers 

reverse engineering the credit scores so they can improve 

their ability to get credit. The creditors have already 

reverse engineered the credit score. They've figured it 

out. They've figured out that incomplete reporting 

results in their consumers' good names being cloaked and 

having lower credit report scores generated from the 

reports. 

Now, I agree that the credit scoring model 

should try to account for the fact that not everybody 

provides complete information, but I totally feel that an 

error of omission is an error and that there are many 

different kinds of errors of omission. Some of the 

errors of omission have a huge, huge effect and impact on 

consumers. Sallie Mae, extremely large secondary student 

loan securitizer, was not reporting information on young 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

119

consumers, young consumers with very thin files. 

So, this may not be -- there may be a semantic 

argument that can be made that you can't be sued for not 

having this information, but the point is Congress wants 

this looked at and I think it wants it looked at pretty 

seriously. 

UNIDENTIFIED MALE: And I think from our 

perspective, all we're saying is that if a Sallie Mae or 

any other data furnisher comes up to us and says that we 

can furnish information, we want to furnish information, 

and yes, we can live by the accuracy and integrity rules 

that the federal agencies are going to promulgate and so 

on, I think we're then in a position to say we're happy 

to accept that information and to make that part of that 

consumer's file and to do that accurately. We agree with 

that, Ed. I'm just saying that it's hard for a bureau to 

be held to the term "error" when a bureau doesn't have 

the power to march in to a particular data furnisher and 

say, well, darn it, you're reporting to the one down the 

street, but how about us. That's very important. 

I don't think that's semantics. I think that 

that has to do with how the report then informs the 

Congress of subordinate issues, if you will, that are 

derived from whatever the study is going to be. That's 

the reason for drilling down on that a little bit. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

120

MR. VANDER NAT: Mike? 

MR. TURNER: I just wanted to touch on two 

themes that I've been hearing mentioned quite a bit. The 

first is that consumers should be the ultimate arbiter of 

what is and isn't accurate and the second deals with 

what's being done to address or identify the inaccuracies 

or incompleteness and address it. I think that's -- the 

second point is very important. 

If I could piggyback on Fred Cate and his 

earlier remark about the somewhat quixotic nature of 

this, when you really try and get your arms around these 

studies and what Congress is asking, I think it's even 

more complicated and I really caution against this 

overemphasis and potentially an over-reliance on the 

consumer. You know, when I first thought about this 

particular intellectual puzzle and how to approach these 

studies, I was immediately reminded of Rashomon 

(phonetic) and that accuracy, even assuming we can agree 

on a definition, varies from perspective to perspective. 

 

And here, I think there are at least four -- 

and I could even be convinced five perspectives are 

important, accuracy from the perspective of the data 

furnisher, accuracy from the perspective of the bureaus, 

accuracy from the perspective of the two end users, 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

121

meaning institutional lenders and consumers, and I can 

imagine scenarios in which, for example, a data furnisher 

reports data to all three bureaus, but that data is 

inaccurate for keystroke or for whatever reason, and so, 

it's not an inaccuracy from the perspective of the 

bureaus. They transcribe that data quite accurately. 

It's in the reports accurately. 

So, you know, how do we catch that, what is the 

error? Consumers may or may not identify that and we've 

discussed the reasons why that can happen. But these are 

not necessarily going to be caught if we rely just on 

analysis of the consumers. And this gets to my second 

point. The assumption is that very little is being done 

to mitigate against the inaccuracies and incompleteness 

already. We've heard Fair Isaac say, in fact, in their 

model, they're already continually developing 

mitigations. I know for a fact that the bureaus, in 

fact, audit data that comes to them from data furnishers 

for data integrity and accuracy. I know for a fact that 

on the end user community, that financial institutions 

have various means, including zip tables and others, and 

they use other data to mitigate inaccuracies or legacy 

systems that have incomplete or errors of omission, 

assuming we can even agree that it's an error of 

omission. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

122

So, there are mitigations already in the system 

at every note, and in fact, rather than replicate the 

wheel, I think that much of the study needs to be 

informed by what the players in each of the (inaudible) 

are already doing to identify inaccurate or incomplete 

data and mitigate against it. Because, again, that's 

important to the extent that what we're really talking 

about here are the consequences from inaccurate or 

incomplete data rather than the existence. 

In fact, I think the best perspective on this 

was put forth by Howard Beales some months back and he 

said, I can eliminate deceptive and false advertising, 

all I have to do is rule out all advertising. Well, you 

know, that can't be the goal in this endeavor. We have 

to accept that there are going to be inaccuracies and 

incompleteness, and I think Bob Hunt had really a -- and 

I tend to agree with the economic perspective on this, we 

need to focus on a different goal, not the elimination of 

inaccuracies or incompleteness, but in terms of 

maximizing the social benefits from improving the 

consumer credit reporting system. 

MR. VANDER NAT: Other comments? Terry? 

MR. CLEMANS: I'd like to talk a little bit 

about the omission and whether we want to call it 

semantics, whether it's an error or not, I think we have 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

123

to focus on the fact that omission is very important and 

it should not be a system that requires every one of the 

three to have exactly the same data. Obviously, you 

know, that's not going to happen. There would be no 

reason for three with that. However, we can't ignore the 

fact that omission has a major impact on consumers. It 

has a major impact on the lenders just as well. Fannie 

and Freddie, I think, have set the standard for that by 

the creation of non-traditional credit reports. 

So, if there was no desire for the omitted 

data, they would have never come up with standards for 

ways to get data that is not traditionally in the 

repository files into consideration. So, you know, 

whether we say it's an error, it's not an error, I think 

something we all have to agree on is there's some way we 

need to address and that this study needs to address 

omitted data and take it into consideration. It is also 

something that maybe needs to be addressed how some data 

is omitted not because the lender doesn't want to report 

it but because there's policies and procedures in place 

that screen some lenders out. 

The whole thing needs to be reviewed and I 

do -- you know, I would like to point out that, Stuart, I 

agree with you entirely. The system does work very well 

for most consumers. However, what if you're one of the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

124

consumers that it doesn't work so well for? I believe 

that's why Congress has got us here today. So, while we 

might be focusing a lot on the negatives, we do have to 

pay homage to the fact that it does work very well for 

the vast majority of Americans and get to how many are 

being hurt and how to correct that and make the system 

better for our overall economy as well as their concerns. 

UNIDENTIFIED MALE: I think from our 

perspective, to the extent that we can get the 

information and put it in the file, we're happy to do 

that. You know that and I think I know our members know 

that. I think what Gerry said was very helpful, though, 

that we want to understand consequence. I think that's 

helpful to the study and I think the cost benefit of the 

extent of the changes is also very helpful. 

In representing resellers, we, too, have 

resellers who will work with files in the mortgage 

reporting context to update, for example, information in 

a file which the Fed study might identify as stale 

information or older information in order to show that 

something was closed or that a balance was older than a 

certain number of days. In fact, Fannie and Freddie both 

have guidelines which require things to be pushed 

forward, if you will. 

So, we see that in that case, you know, really 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

125

the very members that you represent, that we represent, 

are part of the market solution to the fact that you 

can't compel or change the entire data furnisher system 

to make it work in just a certain way all the time. I 

think Paul is right. I mean, we're probably not going to 

be able to replicate Kazakhstan here in the United 

States. We probably don't want to do that either. So, 

in that sense, we're there. 

MR. VANDER NAT: Jeffrey, I haven't heard yet 

from you. 

MR. FEINSTEIN: Apparently, it's easy to get 

picked if you dump ice right next to you. A little tip 

for people who are trying to get called on. 

(Laughter.)

MR. FEINSTEIN: We've been talking a lot about 

accuracy and measures of accuracy and there's three, I 

think, that were discussed so far in this forum, and I'm 

sure there are others that haven't been discussed. 

Consumer reporting, in other words, call up the consumer 

and say, hey, is this accurate; the dispute resolution 

process, just to see -- track the results of that; and 

just comparing credit bureau reports. 

I'd like to suggest a fourth one that I think 

gets at a lot of what we're talking about and that is 

Stuart's study involved the collaboration of lenders and 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

126

lenders do have at their disposal the data on which 

statements are developed and cut. Keep in mind, this 

isn't going to make the study easier or cheaper. But, 

presumably, those statements are reviewed by consumers on 

a monthly basis. It has an ongoing track record of a 

consumer saying, hey, I have a late fee, was I late; hey, 

I have a balance, is that really my balance. And that 

type of measure compared to a credit bureau report might 

help to assess the accuracy of the contents of the credit 

report. 

I'm not suggesting this is a replacement for 

the other measures because there are some things that 

this measure wouldn't get at, but it does offer a little 

bit of insight as to the accuracy of reporting and might 

preempt some of the issues that a consumer might have 

where they might go, oh, I don't know about this straight 

line. 

Personally, what I would think as somewhat of 

an expert in credit reporting, in fact, I could probably 

be one of those expert consultants, I've reviewed my 

credit report twice, found two errors that upon research 

I realized weren't errors on two separate occasions. I 

didn't go through the dispute process. I made some phone 

calls, but the fact is, I, as an expert, was confused by 

my credit report and thought I had errors that I 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

127

probably, if on the phone in an interview, I would go, 

yeah, that's an error, that's a mistake, because I didn't 

have at my disposal the fact that the sales finance 

(inaudible) I opened up five years ago was going to be 

reported by Security Pacific Finance. So, I started 

making phone calls. 

So, I'd like to suggest this alternative 

measure as another way to get at accuracy because it does 

have a consumer component of review in that consumers 

review their bills, or at least there's a presumption 

that consumers review their bills. 

I really like the methodology of Stuart's 

study. I would like to emphasize, though, going on to 

another point, Robert mentioned the need for control 

groups and I think that that's a really, really important 

component of it. What I'm getting at here is consumers 

that apply for credit tend to be riskier than those who 

don't apply for credit. So, that particular study was 

dealing with people who, independent of scoring solutions 

and accuracy, tended to be a riskier population. Given 

that, they were rejected -- chances are they were 

rejected for legitimate reasons, given that this was a 

self -- 

(End of Tape 2, Side B)

UNIDENTIFIED MALE: -- the actual information 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

128

reported would not help, on the contrary. 

MR. VANDER NAT: Thank you. We'll have one 

more comment before we break. Brad, go ahead. 

MR. SCRIBER: Just real quick. On this 

question of completeness, the mandate for this study, not 

a law mandating that everyone report or a new policy, but 

this study is to assess accuracy and completeness. So, I 

think it's a moot point of whether the study should look 

at it. It's a Congressional mandate. You have to look 

at completeness. 

It also gets to the point of what would be the 

impact? Well, we don=t really know how many people are 

voluntarily reporting. We=ve heard that it works, but is 

it 50 percent, is it 20 percent, is it 98 percent? 

Assessing that and establishing a baseline and even 

tracking it over time will help us assess whether certain 

provisions do have the everyone goes fleeing from the 

system effect that we=ve heard so often. So, I think it=s 

important to look at completeness. It=s also important 

because this is a major factor in credit reports. 

Back to my point about involving the credit 

score developers, one of the most frequently received 

explanations for a credit score that we found in our 

study was the ratio of available credit -- you know, 

utilization rate. This was an overwhelming top three 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

129

reason that we saw for credit scores being what they 

were. So, it matters. If it=s wrong, it has an impact 

and you have a mandate to look at it. 

MR. VANDER NAT: Thank you. Let=s break for 

lunch at this time. Marie, could you give us our lunch 

announcement, please? 

MARIE: Sure. For our speakers, lunches will 

be provided for you in the hall, and for those of you who 

do not have a lunch, we recommend two Irish pubs, the 

Dubliner and the Irish Times, which is right down F 

Street towards Union Station. It=s a less than threeminute walk. And we also have the SunSpot Deli here in 

our building around the corner. Enjoy your lunch and we 

will resume at 1:30. 

MR. VANDER NAT: Thank you. 

(Whereupon, a luncheon recess was taken.)
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

130

AFTERNOON SESSION

 SESSION III: METHODOLOGY FOCUSING ON CREDIT REPAIR 

 REPORTS PERTAINING TO THE SAME PERSON 

MR. VANDER NAT: Well, if you have your agendas 

in front of you, you know that we=re moving into Session 

III and we=re looking at a methodology that focuses on the 

credit reports pertaining to the same person and tries to 

infer from that various things about inaccuracy and 

incompleteness. We=re going to start off the afternoon 

with a presentation by Brad Scriber and Terry Clemans. 

MR. CLEMANS: Thank you, Peter. On behalf of 

Brad and myself, we really appreciate the opportunity to 

be here today and talk to you guys about the research we 

did in 2002. Some of this we=re going to go through 

fairly quickly because we=ve talked about a lot of it 

already and I=m sure most of you are very familiar with 

it. As far as the way it will work, I=ll go over some of 

the background on the research project and the study 

design itself and then Brad will come up and go over the 

findings and the recommendations. 

This study was a joint venture between the 

Consumer Federation of America and the National Credit 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

131

Reporting Association. I=m sure everyone is aware of the 

CFA. I=m not sure if some of you are not aware of NCRA, 

but we are a national trade association that=s been around 

since 1992. We represent primarily the resellers. 

When we started to look at the design for this 

study and discuss the possibility of doing one, we looked 

at the studies that were done previously that we=ve been 

discussing here today. We looked at the Consumer Union 

and the PIRG study. We looked at the Andersen/CDIA study 

and we also looked at some research that we did two years 

after we were founded back in 1994. 

That research was done where we looked at the 

differences between the three-bureau mortgage credit 

report that was starting to come into existence based on 

the need to fulfill the automated underwriting systems of 

the GSEs and the old two-bureau residential mortgage 

credit report. 

It should be interesting to note that those 

RMCRs that we all used to do did include a consumer 

interview any time there was disagreement in the data 

that was found between the two repositories and the data 

that was on the back of the 1003 that was taken from the 

consumer in the loan application process. 

In looking at those studies, we looked at areas 

that we felt could use some improvement. We looked at 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

132

some of the common criticisms that were given to the 

studies, you know, small samples by some of the consumer 

group studies, partiality by either the consumer group 

study or of the Andersen/CDIA study and some of the 

methodology that might be questioned in those studies. 

We also critiqued our own study as well and looked at the 

perceived partiality of our study also being industry, 

and again, the small sample. 

We tried to solve these problems. We addressed 

the issues of the previous studies by looking at 

partiality in creating this joint venture. We thought if 

the research was done jointly, you know, every file was 

reviewed, the manual files were reviewed, were done with 

a CFA researcher, Brad Scriber, and a credit reporting 

expert, someone from within the industry evaluating those 

files and comparing and contrasting that data. We 

obtained the data from all three repositories, something 

that had not been done previously except with the study 

that we did in >94, and to make a substantial size study, 

we used automation to compare the scores and to do a 

large data sample. 

That large data sample, of course, would be, 

you know, what we felt not worth much unless you had the 

ability to do some in-depth investigation to get at the 

root of some of those problems. We also -- when this 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

133

information was all gathered, we had it peer reviewed by 

relevant parties within the industry. We compared 

502,623 consumer credit files and the three bureau files 

from each one of those consumers based on the score 

alone. Now, this was blind data. We had to eliminate 

all references to the consumer with this data. The 

closest we got to the consumer was the zip code and we 

generated a generic code to keep the data separated. 

 We focused primarily on the threshold between 

the prime and the sub-prime and we talked about errors in 

this room and we talked about how important various 

errors are. One of the things that we felt with the way 

lending has evolved is look at those who are potentially 

harmed the most, those that are right at the threshold of 

the prime to sub-prime market where an error, not a 

substantial error that might push them from yes to no 

because that is not really relevant with today=s lending, 

but yes, but to a yes plus cost, and that might be a 

smaller error. When you=re looking at lending decisions 

that are sometimes based on as small as 20-point FICO 

increments to price the loan, the size of the error is 

much more important now than ever before. 

This study data was pulled randomly. It was 

from archived files that represented every state in the 

country and all U.S. territories. The credit reports and 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

134

the scores were from consumers that were actively seeking 

mortgages and loans at the time. The lenders had pooled 

almost all three files simultaneously and we did notice a 

variety of differences in the data and our study 

investigation processes were different for each phase of 

the study. 

We know there are many challenges to what we 

did, also. One of the biggest challenges we had to 

overcome in the very beginning to even do this study was 

how could we investigate this in a joint venture with a 

consumer group using data that was pulled for a 

permissible purpose and not violate the FCRA or the GLB. 

 Well, the answer to that was to go to the totally blind 

data. You know, you can=t have a consumer report if you 

remove the consumer from the legal advice that we got. 

So, no consumer identity, no consumer report, the FCRA, 

GLB contractual obligations, we felt, were met. 

But a blind data study like that created other 

problems. There was no way to bring the consumer or the 

creditor into the investigation. Any attempt to verify 

data with a consumer or any attempt to verify data with a 

creditor when you=ve lost account numbers, when you=ve 

lost the information that would document that particular 

file left some holes in our report that were noted by the 

GAO. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

135

In the study data, we broke it down into three 

segments. The first segment we talked about was the half 

a million consumers. That=s over a million and a half 

blind credit scores. And just identifying those scores 

by generic code and the zip code that was from the 

consumer=s residence, this was a total electronic 

calculation that produced some of the results that used a 

quantification or quality control factor for the smaller 

studies, Sample B and B2, that were 1,751 files where we 

dug into the data more in-depth and compared and 

contrasted the information from each of the three 

repositories. 

Now, again, we know there are some limitations 

here; however, we know if you=re looking at three accounts 

from the same creditor and you=re getting three different 

methods of reporting that, you know there=s obviously some 

problems somewhere. 

Interesting findings between all these subsets, 

and those Brad will go into in a few moments, but one of 

the things that we did keep consistent were that we 

collected the scores and the reason codes from each one 

of them. While we looked at the assessments of impact in 

only the smaller one and the errors of omission and coomission from the vary smallest file that is due to the 

problematic issue of how much time it takes to really dig 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

136

into these files. Ed, as you well know, going through 

these, it is not easy. I think actually our job was a 

little harder because we couldn=t contact the consumer or 

the creditor. I think this could actually be done 

quicker by bringing them into it to get some information 

from them, from the consumer, verify it with the 

creditor, instead of looking at the three files and 

trying to draw some conclusions. 

One of the most important things we feel that 

our study did was to focus on the common element from all 

three phases of the study and that was looking at the 

credit score as a common denominator so that 

investigation that=s done, that=s a layered approach, that 

focuses on direct investigation, that might focus on the 

Andersen type study where you take a subset that is 

triggered from a denial or is also triggered from an 

approval, so you have a control group and layer it with a 

more massive automated approach using the credit score as 

that common denominator can come up with some very 

interesting findings, and to get to the findings, I=m 

going to turn it over to Brad. 

MR. SCRIBER: I=m going to go quickly through 

this because I think a lot of you are familiar with the 

findings, and if you aren=t, I=d be happy to give you a 

copy of the study or talk with you afterwards. But I 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

137

want to make sure we have a chance to talk through 

reactions. 

This is an overview. The scores vary 

significantly for a given consumer among the various 

bureaus. The impact varied depending on the nature of 

the errors and where a consumer was on the credit 

spectrum. Some errors helped consumers, some errors 

harmed consumers. In a significant number of cases, it 

was difficult for us, again without consumer contact, to 

make an assessment of what the impact would be. But for 

those that we feel we could make the assessment on, the 

potential harm was significant. 

The reason codes we=re not going to go into very 

much, but they were rather vague and didn=t seem to offer 

a lot of help to consumers. 

Additional reports and mixed files were very 

common and this seemed to indicate a level of imprecision 

with matching data and a consumer at a given point. 

Something that I think our study offers that is very 

helpful is it looked at a subscriber report or -- I mean, 

on this same report issued, this is a report that a 

lender did look at in making a decision and it really 

gets at impact, you know, what is going to happen to 

consumers if they have these errors. So, I think that=s a 

benefit. And there omissions and commissions. You 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

138

decide if they were errors. 

Also, a number of consumers lacked information 

to assign a credit score, which again gets to the 

question of are people who may have reports that 

aren=t -- or may have accounts that aren=t reported being 

treated fairly in the system? 

So, on the largest sample -- and, again, the 

findings between our Sample A and Sample B were very 

similar. We used this to sort of do an internal check on 

if our two approaches were getting at the same 

information. Fifty-point variations on 29 percent of 

files, 100-point variations on 4 percent of files, 

average variation was 41, median 35. This gives you a 

sense of how big the variations were and how often 

variations of different sizes occurred. Most often 

centered around the average, the median, but a 

significant number of consumers had very high variations 

in credit score. 

And, again, we did a check on the different 

scoring models that are used to try and control for any 

impact that different models would have on the credit 

scores. 

On our next sample, we=re looking manually 

through the information. We found very similar results, 

50-point variation on 31 percent compared to 29 percent, 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

139

100-point variation on 5 percent compared to 4 percent, 

and the average in the medians were very similar as well. 

 There didn=t seem to be a dramatic difference on how 

severely you were affected based on where you were on the 

credit spectrum. We saw high and low variations for good 

and bad credit consumers. The skew was somewhat towards 

higher variation on the low end, but overall, we were a 

bit surprised at how uniform -- everyone is susceptible, 

essentially. 

And this gets also to a point I made earlier 

about being careful to look at the impact all along the 

spectrum because we know in 11 years a 750 or a 740 will 

have an impact that it may not have today, assuming that 

those numbers are still relevant, but the equivalent. 

Okay, Terry=s talked a little bit about this, on 

our second sample, how we defined at risk consumers. We 

used the 620 cut-off and a couple different ways of 

determining if you were close enough to be considered 

part of the bubble. In making the assessment, again, 

this is essentially a blind assessment, looking for 

logical inconsistencies that shows there definitely is an 

error. We tended to actually, we think, understate the 

impact because if there wasn=t contradictory information, 

we assumed it was correct. So, for example, if one 

bureau had an account that the other two didn=t, we 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

140

assumed that the account was correct on the one bureau 

that had it. So, if this is inaccurately included or the 

other two bureaus had received a dispute and removed the 

account, we would overstate the negative -- those 

consumers that are not being affected. 

Again, some of the limitations of not being 

able to consult with the creditors or the consumers, I 

would add, the CRAs or some of the other groups that have 

been mentioned here, and in most cases, we essentially 

withheld our assessment. This sort of graphically 

represents this. About 20 percent of the sample was 

considered to be at risk, on the bubble, and of those, 

about 20 percent were considered to benefit, 20 percent 

were considered to have some harm occur, and in most 

cases, we felt that in a blind study, we didn=t have the 

information we needed to have an assessment that would be 

reliable. 

I mentioned this earlier. Additional reports 

or missing credit scores were a problem, and this gets at 

a segment of the population that=s going to be 

increasingly important as credit scores increase in their 

currency. As more people use them and use them for more 

applications, those who aren=t captured or don=t have a 

trace in this -- not one database, but who aren=t recorded 

will be invisible and this is a population we need to not 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

141

forget. 

Then our very small sample, this is not meant 

to be statistically significant. We know it=s small, but 

it gives you a sense of what was underlying some of these 

errors. And, again, omissions and commissions occurred. 

And the nature of them were missing accounts 

that had no derogatory information, accounts with late 

payments, but not in collection and collections. You 

know, some positive, some negative information was 

omitted and the general tendency was for positive 

information to be less frequently reported than negative 

information. So, again, the skew towards reporting 

negative information that I reported earlier. 

Commission -- errors of commission, 

inconsistencies in how frequently consumers had been late 

and these are primarily in what I would call historical 

data, so not the most recent month. So, the majority of 

this, we didn=t quantify it, but the majority is not 

attributable to loading errors or timing errors. Most of 

these are historical two months or more in the past. 

Credit limit, again, not a statistically 

significant sample here, but if you hold our data next to 

the Federal Reserve data that shows 70 percent of -- I 

think it was accounts, 70 percent of accounts were -- or 

was it consumers? 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

142

MR. CLEMANS: Consumers. 

MR. SCRIBER: Seventy percent of consumers had 

a missing account -- a missing credit limit on an 

account. We found, also, a very high number. This 

matters. It was one of our most frequently found reasons 

for the credit score and if consumers are given that as a 

guide as to how to address errors, you know, again, we=re 

either leading them to a wall with no door or we need to 

address it. Balances also were a problem and dates, 

which are significant in terms of the aging of accounts. 

 

This approach has a number of strengths. As 

Terry mentioned earlier, we tried to build on what we saw 

as positive in previous studies. Some limitations, we 

couldn=t incorporate all of what we thought was helpful. 

But reviews to subscriber report, it combines the benefit 

of sort of the consumer interests and industry expertise. 

 The comparison of data and credit scores allows for 

quick quantification. The sort of nested studies give us 

a sense of the qualitative nature of those errors, but a 

large sample that can be summarized quickly was very 

helpful. And in some cases, we could assess whether the 

errors were helping or harming consumers. 

Our recommendation for what this study and the 

319 should look like, in our opinion, there are strengths 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

143

in all of the approaches and some of the limitations that 

we faced may not constrain the FTC in the same way, and 

some sort of approach that combines the strengths of each 

type of study will probably, in our opinion, yield the 

best results. 

Consumer interviews would provide information 

that we couldn=t get. Demographic information that I 

mentioned earlier we think is extremely significant; very 

important given the track record of sub-prime lenders and 

others in obscuring data, and just getting a sense of 

whether the system works fairly for everyone. I=ll 

mention here, also, potential implications for women. 

There=s been some discussion of divorces and, I think, 

maiden names may have an impact on women that don=t affect 

men and other institutional or family dynamics. Getting 

a sense of whether men or women have equivalent error 

rates is important. Missing accounts, if something was 

not reported to any bureau, we didn=t catch it. Also, if 

it was reported the same to every bureau, we didn=t catch 

it if it=s incorrect. 

Identity theft related accounts, if it was 

reported accurately, we didn=t catch it. Mixed files and 

credit limits were errors that we and others have found 

as significant and should be a focus, and any disputed 

information that=s been reinserted, again, if it was 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

144

consistent, we didn=t catch it. 

Rescoring, I think, is an approach that has 

been discussed, although maybe not directly today, that 

would quantify the impact of errors and also the benefit 

of helping the consumers who actually participate in the 

study. That may be an incentive. There was some concern 

about participation, but if consumers feel that their 

report will be improved, that may be an incentive to 

remove that barrier. This could also help locate the 

source of the error. This is important. 

A lot of people had mentioned this and I agree 

very much. There=s not one entry point for errors. CRAs, 

furnishers, consumers, the way they apply for credit, 

there=s lot of places where errors can be introduced, and 

the more we understand about where that happens, the 

better we=ll be able to address it and move towards higher 

accuracy. 

The dispute date, this has been mentioned and 

I=m glad Bob sort of addressed this point already. It=s 

relevant, I think, to talk about the Andersen study. It 

was done 11 years ago and that=s about the time -- I guess 

now it=s about 12 years ago that it came out. That=s the 

same time frame we=re looking for now, moving forward, and 

we need to be sure that we don=t look back at the study we 

do now and see mistakes that we should have addressed. I 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

145

think the biggest shortcoming here that needs to be 

addressed is really doing the correct analysis. Not 

making an assumption that 92 percent of reports that 

aren=t reviewed have no errors is a shortcoming. 

Also, looking only at errors large enough to 

reverse a credit denial, we know now that the world is 

different, and I think the parallel we can draw is that 

we need to look not exclusively at the bubble -- I mean, 

our report looked at the bubble, but we need to have an 

understanding of the extremes as well, because in 11 

years, those extremes may matter. 

This, you know, shows that an alternative 

interpretation of the Andersen data, which Bob has 

already mentioned, is consistent with the PIRG study, the 

latest PIRG study, that a quarter of reports consumers 

see as having errors. So, if you were looking for common 

ground, Ed, there it is. 

Dispute data can be very helpful. I think 

there=s a way to use this data that can give you a lot of 

insight, that=s important, that can help industry improve, 

that can help consumers. If we understand what types of 

errors motivate consumers to dispute, what are the -- 

when does it happen? Is it after people receive adverse 

action notices, after they=ve received risk-based pricing 

notices, when they get their free annual report? 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

146

Understanding the dynamics of the dispute could 

be very helpful in understanding the nature of accuracy 

in consumer reports. 

What=s the outcome? What happens to disputes? 

Essentially, that=s what Arthur Andersen looked at. We 

think that sort of an ongoing comprehensive review of 

dispute information would be very helpful to understand, 

again, if consumers are disputing information that they 

shouldn=t be or reaching -- I think the columns that the 

pilot study outlines are very helpful. A consumer may 

dispute and it gets fixed and everyone agrees that it was 

an error, a consumer may have something explained and 

then realize that it wasn=t an error or some of the other 

columns that were proposed. This probably would be a 

reasonable procedure to ensure maximum possible accuracy 

on the bureau part. 

This study really is essential to the success 

of FACTA. There was concern in the past year that we 

didn=t understand the accuracy issue and it made it 

difficult to make decisions. We do know that a key 

objective of FCRA is maximum possible accuracy, not 

pretty good, not right some of the time, but maximum 

possible accuracy. That is a fantastic standard and it's 

something that we can meet. 

Again, I mentioned that several people 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

147

highlighted how vital accurate information and complete 

information -- I mean, there was incredible emphasis on 

complete information, that is people pull out, the system 

will fall apart. Well, in the same way, if we can get 

more people in the system, the system will be stronger. 

So, this is important and the GAO study pointed out that 

it's very difficult without trend data to understand the 

impact of the '96 amendments and, in general, you know, 

what's improving, what's getting worse. 

So, the possible impact of this study is very 

large. Ongoing data every two years, it tells us what's 

happening, what's getting better, where we need to 

improve, it's really going to be the linchpin for FACTA, 

and it will be the standard that we're measured by, I 

think. 

The interim reports also allow for adaptation 

to market changes. Credit scoring is going to evolve. 

New applications, different people are going to be using 

this tool in different ways that we can't anticipate 

right now completely. We can do our best to start the 

study with a broad baseline that allows us to compare 

data that may not matter now, but we think will matter 

later so that we have trend data and not just playing 

catch-up. Again, the importance of various errors are 

going to change. Some of this I've already mentioned. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

148

You know, changes both in the credit scoring models and 

how people use them. 

That's the end of our comments. Thank you. 

MR. VANDER NAT: Thank you for your 

presentation. I think at this point we could move to 

about a 10 or 15-minute discussion of what has been 

presented because quite a lot was presented here, and 

then after that point, we will receive further comments 

from Michael Turner. 

So, who would like to begin the discussion with 

an observation, comment or question? Please. 

MR. HENDRICKS: You realize, in some ways, you 

don't need to reinvent the wheel. This was the best 

study of credit report accuracy ever done and they've 

laid out a methodology, which I think we can build on, 

and I think the infrastructure for conducting this study, 

having the resellers pulling reports like this, provides 

a very rich field of information to do a study. So, I 

think that we should build on this and then look at the 

weaknesses that Brad has identified and Terry and then go 

further by using the position the FTC is in to make it 

that much better. 

MR. VANDER NAT: Thank you. Stuart? 

MR. PRATT: In contrast to what Evan just said, 

I would say the study falls short of one of the FCRA 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

149

requirements because it attempts to insert that there's 

an error of omission concept into what the FCRA currently 

requires, or at least how Evan characterized what the CFA 

study attempted to do. So, to the extent that a study of 

cross comparisons is going to lead to a conclusion that 

there's an error rate associated with the fact that a 

lender chose not to report to all or chose not to report 

to any, we think that that part of the study in 

particular is troublesome. 

Just to point out, an error of omission was 

that it's reasonable procedures to ensure maximum 

possible accuracy, not simply the objective of maximum 

possible accuracy. That reasonableness standard was 

inserted for the very reason that you couldn't possibly 

build a system based on purely a maximum possible 

accuracy standard when you're managing four billion 

updates of information and 200 million credit reports 

every month of every year. That's very important. 

So, I think if we're going to stay on the same 

plane here, then we ought to make sure we're describing 

the operational law accurately in terms of how it 

operates relative to our members and Terry's members, by 

the way. I think that's very important, and I'm sorry to 

see that you omitted that from the presentation in the 

first place, Brad. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

150

MR. SCRIBER: It's in there. 

MR. PRATT: I'm sorry. 

MR. SCRIBER: I could go back to the slide and 

show you where I do have it in there. 

MR. PRATT: Well, I'm just saying you -- all 

right. At the end of the day, the FCRA is the 

measurement, the benchmark, I believe, and so I know the 

study is looking at accuracy and completeness. But the 

key here is, I guess, Peter, that it can't -- a study 

that says, ah-ha, there's somebody who's not reporting, 

well, you really just need to decide why somebody's not 

reporting in the first place. That's not an error, 

that's not a failure of the system, if you will. There's 

just maybe some -- and by the way, depending on how the 

accuracy and integrity rules are issued, there may be a 

community of lenders who are even larger who say I just 

can't continue to work under this regime or this set of 

requirements that the FCRA requires of me. I think I'm 

going to back out. That may happen. 

We don't know that that's going to happen, I'm 

not saying that that's going to happen. But I think 

that's relevant, if you will. So, to the extent that the 

study begins to create a finger-pointing kind of effect 

by saying, look, look, there's something missing and 

trying to find an accountability for that, I think the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

151

study falls short of what it really should do in terms of 

informing Congress at the end of the day. 

In terms of score variance, I think that this 

study speaks very directly to what the marketplace would 

acknowledge anyway. You've never heard the CDIA say, no, 

it's not true that there are score variances because 

there are score variances. There's no doubt about it. I 

mean, you could read the Fed Report and get a lot of 

reasons even from the Fed Report in terms of here's 

differences in terms of how files operate in the 

marketplace. So, I don't think we should run away from 

that issue. If we could have more complete information 

in every file, I think that our members would be in 

support of that as well. So, again, I don't see any real 

difference in goal here when it comes to that as well. 

I guess what I'm concerned about is making sure 

that we're asking the right question. When we see score 

variances, the real question is, what should a lender do 

about the score variances that they see. That's the real 

question on the table at that point in time. That has 

nothing to do with an accuracy study or a completeness 

study. That just really has to do with a question of 

what should a lender do when it sees three different 

scores and they vary by 250 points. 

MR. SCRIBER: Well, I mean, first of all, I 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

152

think -- I'm willing to agree with you, Stuart, that -- 

MR. PRATT: Can we make a note of this? 

MR. SCRIBER: Yes. 

(Laughter.)

UNIDENTIFIED MALE: That you're both at the 

same meeting is (inaudible). 

MR. SCRIBER: That it's not really -- some of 

the things we're talking about are not errors of omission 

and if we take the example of Capital One not reporting 

credit limits and only the balance, which tends to harm 

consumers if it makes them look like they're maxed out on 

their credit report -- 

MR. PRATT: And we would like to have that 

credit limit. 

MR. SCRIBER: Right. And we would like you to 

have it. And -- 

MR. PRATT: There's more agreement right there. 

 This is very exciting how this is happening. 

MR. SCRIBER: And then, also, I think in terms 

of finger-pointing, we should be -- it should be from the 

get-go. This is an issue that's not really the 

responsibility of the credit reporting agencies and no 

study should say that this problem is the fault of the 

credit reporting agencies, that some lenders don't report 

credit limits. I mean, that's a decision by the credit 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

153

lenders and you have no leverage to make them change. 

On the other hand, we are trying to get a study 

on the accuracy and completeness of information, and 

systemwide, we do have a problem if consumers' credit 

limits are not reported. This is just one example. And 

that ends up making them look less creditworthy than they 

are. So, I mean, maybe we shouldn't define that as an 

error of omission, maybe we should define that as an 

attempt to shaft consumers, you know. But the fact is 

that it does directly impact accuracy, completeness, 

relevancy and fairness and so, therefore, it should be a 

subject of the study. 

MR. PRATT: I'm not suggesting that. What I'm 

trying to make sure we're doing here, though, is properly 

characterizing one element that you may want to look at 

in this study which is whether some data is present or 

not present. But the extent that it's characterized as 

an error, you're automatically converting this into the 

pejorative and saying that it is always an attempt to 

shaft the consumer. Our challenge may be, and candidly 

for lender data furnishers out there, the challenge may 

well be that they say I have a choice in terms of how I 

think I'm doing in the marketplace. I'm either going to 

report in the way that I wish to report or not at all, 

one or the other, because it's important to me and it 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

154

costs me a lot of money to acquire each of those 

consumers that I have in my portfolio. 

So, from our perspective, and I think that's a 

sensitivity you have to have in all of this, that a data 

furnisher, too, may say I just need to have the 

flexibility to report what I believe I can report and 

it's not an attempt to shaft as much as it's an attempt 

to protect the very positive market that they have. I'm 

going to let data furnishers speak for themselves. 

But from our perspective, it's just simply a 

fact that at the end of the day, an omission of data is 

not an error and we're going to stay on this theme 

throughout this discussion and we'll just probably bore 

you to death with it at the end of the day, Gerry and 

Peter, but it's very important that this element of this 

discussion doesn't remain under the rubric of error. 

MR. VANDER NAT: We understand your point. 

Gerry? 

MR. BUTTERS: I was just going to say, we knew 

coming into this that incompleteness would be a source of 

a lot of strong feelings and disagreement and we are even 

more aware now and we will be very careful how we 

characterize things. 

UNIDENTIFIED MALE: I'll just add one comment 

here. I agree with Stuart's statement that when you see 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

155

this variance in score, what do you make of it, and the 

question is what does a creditor do with it, okay? But 

there's another element here which is a hypothesis which 

is open to a test and you could, I think with sufficient 

resources, ask the following question, does the variance 

in the score on the same report, among the three, have 

any correlation ultimately with the inaccuracy of the 

report? It is a fair hypothesis, although not proven, 

that the greater that variance is, the more likely it may 

be that there's a problem somewhere in that report. Now, 

that's a testable hypothesis if you had enough resources 

to devote to it. 

So, it may ultimately be that the variance in 

the score is related to the issue of accuracy or 

inaccuracy, but that we have not yet done the study to 

show what that connection is. 

MR. VANDER NAT: Anyway, I'll take other 

observations. Karlene? 

MS. BOWEN: I think it goes without saying that 

everyone here agrees that the more data and the more 

accurate data, the better the whole system is going to 

be, and that goes without saying and we all agree with 

that. But there's also a cost associated with that and I 

think that's part of the point that Stuart's getting to 

over and over and over again, that the cost involved in 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

156

having perfect data is going to come back to, guess who, 

the consumer. And so, trying to balance that out is the 

humongous task that you have when you perform this study, 

and I don't envy you trying to do this. 

There are -- when I think about how you choose 

to do the study, that's one important phase of this, and 

as you go through that process of selecting which 

methodology you're going to use, I'm sure that once you 

finally select it, you'll come up with another set of 

questions to ask many of the people in this room to fine 

tune that. So, I know this is just the first phase. 

So, I'm not going to comment on the particular 

benefits or cons of the various study options at this 

point, but I do want to step back a second and mention 

something that Paul mentioned earlier and a couple other 

people mentioned, that we have to also remember the value 

of the system we have in the United States. We do 

business internationally and we are the only country that 

has this type of system and it is such an incredible 

benefit to consumers. That doesn't mean it should stop 

here and that the improvement should stop here. I'm not 

saying that, but we do have to sit back and remember that 

once in a while. 

So, now to get to what I really wanted to say, 

as you're performing this study, I think there are three 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

157

phases of it, and the first phase is selecting and doing 

the study, and that's what we're focusing on today. It's 

easy to think that that is the most important phase, 

that's the most important part of doing the study. To 

me, that is an important part, but you're going to do a 

good job. I don't think anyone here would argue with 

you. You're going to find a way to do the study and 

there will be issues no matter what process you select. 

But the next two pieces are, to me, what's more 

important, and that is how you interpret the data. And 

Brad brought up a very good example of that. Gosh, 

there's two ways to have looked at that same piece of 

information and both of them tend to have a biasness to 

it. Both ways tend to have a bias to it. I like the 

first way personally, that only .2 percent have errors. 

But there's -- it's going to be so important how the data 

is interpreted and how that analysis is written up. 

There was a study done and you may have heard 

this example before, but there was a study done that 

said, wow, every time there is a house fire, the more 

fire engines that show up at the house fire, the higher 

the probability that someone's going to die in that house 

fire. Therefore, there must be -- it must be in our best 

interest not to send fire engines to house fires, okay? 

Well, that soon defeats the whole purpose and that's what 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

158

we're all trying to prevent from happening here, gosh, 

that we don't put so many things in place that all of a 

sudden we have nothing here to protect the consumers in 

the first place. That's going to depend a lot on how you 

analyze the data that you do eventually get out of the 

study. 

And then the third point is the summary of the 

analysis. That sounds a little redundant, but it's 

really not. The summary of the analysis is really where 

the sound bites are going to come from that go to 

Congress, that go to the media, and that's what we're 

going to hear repeated over and over and over again. 

When I think of past studies that have been done, what 

sticks in my mind, and believe me, I read these studies, 

but what really sticks in my mind is, oh, my gosh, 33 

percent of the files have errors in it or 66 percent of 

the files have errors or whatever the numbers are. 

That's what sticks with me. 

So, we have to be so careful in those final 

little sound bites to make sure that they are fair and 

accurate and are actionable sound bites. Because if we 

say, wow, 60 percent of the files have errors in them, 

consumers will react to that adversely and there will be 

constraints put in place that may result in some of the 

things that Stuart's saying that, wow, then all of a 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

159

sudden it's not going to be worth it for a credit grantor 

to report because the constraints will be so extreme. 

So, I would not want to be in your shoes today, 

but I want to keep in mind that there are those three 

points to keep in -- as you go through the analysis, that 

doing the study is only the first piece. How you analyze 

the data and then summarize that for Congress is hugely 

important as well. 

And I want to make one comment on this lack of 

data, lack of credit limit reporting. Sometimes someone 

will score lower and sometimes someone will score higher. 

 It will go both ways and I just want to keep that in 

mind. So, there's pros and cons. That's a minutia piece 

of information, but I just wanted to point that out. 

MR. VANDER NAT: At this point, let's turn to a 

presentation by Michael Turner. 

MR. TURNER: I just wanted to thank Peter and 

Gerry for inviting me and actually some of my colleagues 

from the Information Policy Institute. I'm going to try 

and be very efficient primarily because much of what I 

wanted to say has already been said and I'll avoid 

redundancy. Also, it's 3:15 in the morning for me right 

now. I just came back from India and arrived in this 

bizarro parallel universe where Stuart Pratt and Evan 

Hendricks are agreeing. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

160

(Laughter.)

MR. TURNER: So, I think that sets the context 

for the afternoon, in fact. 

I'm going to keep my discussion very narrowly 

focused. I was asked to speak to the NCRA/CFA study, 

both methodologically in terms of soundness and also some 

of the conclusions. So, inasmuch as I said much has 

already been addressed, I'm going to skip around 

somewhat. I apologize for the incoherence. I hope you 

understand and I promise to be twice as coherent next 

time I speak. 

A couple of remarks initially. I am on the 

record as having praised this study at the Info Flow 

Workshop that the FTC put together last year. I do think 

it's very quality work for a first generation study and I 

think some serious findings did fall out of this 

analysis. That's not to say that I agree with everything 

and that's not to say that there aren't weaknesses. But, 

you know, it's important and it took me a long time in 

graduate school to realize that I would never write the 

perfect article or the perfect paper, and there would 

always be flaws. Yes, there are flaws, but there's some 

valuable learning in this paper. So, it's maybe even a 

more bizarre universe because now I'm agreeing with Evan. 

(End of Tape 3, Side A)
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

161

MR. TURNER: -- but with that said, there are 

some -- I think your summary of analysis point is quite 

accurate. My problem isn't necessarily with the 

methodology, it's the way that the conclusions were 

presented. And what the press latched on to when the 

study was released were the conclusions that came from 

phase three of this analysis where you had a sample size 

of 51 files. That's a minuscule sample size and that's 

been a problem not just in this study but in many of 

these studies on the completeness and accuracy of 

consumer credit reports. No meaningful inference can be 

drawn from a sample size of 51 files. I'm sorry and I'm 

sorry those findings made it into the press because it 

does, in fact, impact consumer's perception of the 

accuracy of their credit reports and that's, in fact, 

what the press did focus on. 

But with that said, phase one and phase two of 

this analysis actually were quite helpful and quite 

useful. I do think that the decision by the NCRA and CFA 

to focus on the potential impact of score variation on a 

group of consumers -- and here they selected a bubble 

group in and around 620 which was the cut-off between 

prime and sub-prime, it's a helpful first cut. I think 

that moving forward a more nuanced analysis is necessary. 

 I think that breaking up the trenches into prime, near-
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

162

prime, non-prime and sub-prime, and in looking at the 

risk tiers within those trenches would be much more 

helpful in terms of assessing the impact of score 

variation, if any were identifiable on the differences of 

consumers, because one can imagine all sorts of scenarios 

within the three criteria that they established where a 

variation of the consumer score -- and 50 points may not 

even be one standard deviation by the way -- wouldn't 

have much of an impact. So, I think it does matter. 

So, you really need to look at are we talking 

about -- and, again, it's not just a binary approval or 

non-approval, how does it affect the impact of assess to 

credit, if at all, and how does it impact the terms of 

the credit, and that's something that can only really 

fall out of a more nuanced analysis. But, again, that 

there were potential impacts in those is suggestive 

that this is meritorious and it does warrant, 

methodologically, further analysis. So, I think that 

that's the most helpful contribution, at least of this 

study. 

In phase two -- and this is to Robert Hunt's 

point, it's quite important to discuss the universal or 

the broader findings as well as the concentrated 

findings. When they discuss the phase two findings they 

basically concluded that phase two findings with the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

163

broader sample or their baseline were very consistent 

with the smaller group of phase one. And, you know, you 

look at 5 percent in their phase one analysis had a 

variation of 100 points or greater, but in phase two in 

the larger sample size, it was 4 percent. Well, if you 

present it like that, you know, 1 percent difference, 

that's not very important. But then, actually, 1 percent 

and 5 percent is a 20 percent decline and that's pretty 

significant. 

So, you know, how you package and present the 

data, you know, industry typically tends to be accused of 

playing statistical games, this does matter, that's a 

significant drop. So, I think we need to be paying 

careful attention to the presentation of the data 

because, you know, there are very different 

interpretations that fall out. 

And here, again, to Stuart's point, many of the 

errors of omission and errors of commission, there were a 

whole variety of errors that were identified in this 

study and it's entirely unclear to me which of them 

matter, if at all -- and, you know, we covered this 

earlier. I think it's quite important that maybe some 

stratified analysis occur where we focus only on those 

errors that seem to have the most significant impact and 

that's where the learning from Fair Isaac and others who 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

164

are developing algorithms for risk modeling comes into 

play. But, you know, in many cases, these errors of 

omission are really not at all errors and they're not the 

fault of the credit bureaus certainly. They're not 

unintentional omissions. So, we need to be very careful 

about calling those errors and making too much about the 

frequency of these errors. 

Certainly, the government has been aware of 

this, some gaming of the system. There have been 

attempts made to remedy this both in terms of letters 

from government officials, but also the credit bureaus 

have taken actions, in some senses, to encourage data 

furnishers not to under-report. So, it's not a phenomena 

of which people are unaware and it's not one that hasn't 

been addressed at least in some capacities. But, you 

know, it may, in fact, be an area where additional 

analysis needs to occur. 

I also think that -- and I'll take this 

opportunity to plug the longitudinal component of this 

analysis, you know, I don't think it's a negative that 

this is an 11-year mandate. Most of the analysis that's 

occurred in terms of accuracy of consumer credit reports 

has really occurred with interest rates that have been 

steadily declining really since the late '90s. Well, we 

need to look at the impact of a complete business cycle 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

165

on the efficacy of the National Consumer Credit Reporting 

System and I'm not sure what the Federal Reserve will do 

today or in the future in terms of the federal funds 

rate, and obviously, the ever ballooning federal deficit 

certainly will have an impact on longer term interest 

rates and 30-year fixed mortgages. But, you know, we do 

need this type of analysis -- longitudinal analysis to 

assess the impact of the business cycle, if, again, this 

study can be done with all of the problems, various and 

sundry, that have been discussed. 

So, you know, I think -- the other real 

criticism I have of this is that the phase three 

analysis, the findings that fell off of a smaller group 

of phase one, well, there were significant differences, 

even though they were understated between phase one and 

phase two and, you know, the inaccuracies and 

incompleteness were more pronounced in phase one. So, to 

select a group from phase one, in fact, overstates the 

degree of inaccuracies and incompleteness. And, again, 

that's something just we need to be mindful. 

Methodologically, you know, the approach wasn't 

fundamentally flawed, but it's just the selection 

criteria involved here. So, I will say as a first 

generation study, these studies are very difficult and 

there is learning in this as there's learning in the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

166

Andersen study and I wouldn't throw the baby out with the 

bath water. I think that -- I don't know if I would say 

it's the best study ever done and this is the foundation, 

but, you know, certainly the authors of the study do 

deserve serious laud for their efforts. 

So, I think I finished in eight minutes. I'll 

turn the floor over. 

UNIDENTIFIED MALE: I'll just make one comment 

here about the issue of how the data is ultimately 

packaged, and at another effort at transparency, there's 

a lot of in-house discussion about those very same 

issues, so that whatever report is first written, you can 

be sure it goes through many drafts and it has lots of 

review by lots of different people and part of what 

they're thinking about is what is the sound bite effect 

of this. And so, be comforted that that receives very 

careful scrutiny before anything is ever put out as a 

report. 

MR. VANDER NAT: Let me open again to the floor 

discussion, comments, observations. Who would like to go 

first? Fred? 

MR. CATE: This is really, as much as anything, 

thinking out loud, but this seems the right group in 

which to do that. I think there's a ton of value and 

great interest in comparing reports across bureaus, and 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

167

it's something factual, it's something we have, it's one 

way of moving away from something that just relies on 

consumer interpretation. I guess I think going back to 

much of the comments made earlier, it does heighten the 

importance of figuring out what are we doing with that 

data; in other words, what's the value of that 

comparison? 

It's not quite clear to me what it is. It 

seems to me that to say -- I was very sorry to hear Evan 

say this was the finest accuracy study ever because it 

might make you also doubt my top gun attribution, but I 

mean, that's nonsense. It doesn't say a lot about 

accuracy at all. 

I think it does say something very important 

and I think we run the risk of undermining the study and 

of taking away the value when we say or when it's 

promoted as this tells us a lot about errors and 

inaccuracies and so forth. It really doesn't. You know, 

it tells us a lot about how the credit reporting system 

may operate. It tells us a lot maybe about the problems 

of furnishers and the fact that furnishers aren't 

obligated to furnish. It may tell us something about 

timing of the way data is recorded. And I guess this 

still keeps coming back to me about the importance of 

defining what it is you want out of this study, you know, 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

168

what's going to be -- and then what would be most 

valuable in terms of, not so much even thinking just the 

packaging, but what would -- Congress presented with 

these numbers, other than the sort of flashy conclusion, 

a third of all credit reports are inaccurate, which the 

study, to my mind, doesn't support, what might we be able 

to actually tell Congress out of a study that would be 

valuable. 

And this is where I think it might be useful, 

and maybe this is for the next panel, and I really do 

mean to pose this as a question for people like Bob Hunt 

or Robert Avery, people who have done a lot of this type 

of work in the past. What types of elements would make 

it useful to get what types of outcomes? And maybe by 

thinking of it both ways, what outcomes do you need, what 

categories of outcomes, would come back to say what types 

of methodologies would be most useful. 

MR. VANDER NAT: Thank you. Other 

observations? Yes, Nick? 

MR. SOULELES: I have a follow-up actually on 

that question. I don't know whether -- let me take a 

step back. I was going to ask earlier, do we know what 

fraction of lenders get multiple reports, you know, two 

or three, and then if there are differences, how do they 

respond. It's related to the issue of material 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

169

differences before. You could ask, well, in that 

situation, is a lender more likely to turn someone down? 

 Even that isn't sufficient because if one lender turns 

the person down, they might go to someone else. So, how 

would you get at the ultimate effect on credit supply? 

Well, there is one potential answer here. If 

you have panel data from these bureau reports, you could 

look at the total amount of credit held by these guys. 

Now, you're going to have to make some assumptions about 

credit demand, credit supply, you know, it might not be a 

random draw that these guys had differences in their 

scores. But one very interesting question is -- and I 

don't know if you have the data to answer it, but if you 

do, it would be tremendous -- when you see someone with a 

large divergence of scores, compared to some control 

group, and who knows what that would be, how much less 

credit do they get? If the answer is no -- none, then 

you draw different implications in terms of the effects. 

So, I guess one way to put the question is do 

you have a panel of the amount of credit held by these -- 

it's just a -- it's a cross-section. 

UNIDENTIFIED MALE: No, we don't have that. 

MR. SOULELES: Well, one answer is, gee, if you 

need to see -- even in a cross-section you could say, you 

know, do they have less credit -- it's not clear how the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

170

-- the causation is tricky there. But one answer to your 

question is, what are the implications of this on the 

amount of credit (inaudible). 

MR. VANDER NAT: Thank you. Yes? 

UNIDENTIFIED MALE: I was just going to say in 

terms of the question that you're trying to answer in 

Congress and whether it's an error of omission or how you 

want to characterize it, the proper characterization, if 

the question is to what extent do -- what's reported in 

the credit bureau files representative for everyone or to 

what degree do you have accurate representation of credit 

profile for each individual so that what is true in terms 

of their behavior and their credit, as opposed to what 

gets reported into that profile. And so, trying to 

assess a systematic bias or some sort of systematic 

problem in -- not a problem, but not an accurate 

representation for some lenders, and some individuals, 

they operate in a particular area of a market where 

people simply don't report because it's not costeffective or it's too costly to report on them. So, I 

guess what you're really trying to do is get an accurate 

representation of a credit profile for each individual. 

Now, to what extent do the profiles that appear 

in credit bureau files differ from a "true" reflection of 

the profile that should be reflected for an individual 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

171

for all the credit activities that they're involved in. 

So, if you get a description or somehow get a 

characterization of what that ought to be from a study -- 

so if you establish the truth and then you say, well, 

what is represented in the actual bureau files and how 

does it differ from the truth, then the question is, 

well, what are the methods for accommodating or dealing 

with that in terms of you make everybody -- the people 

who aren't reporting on a systematic basis report. Well, 

it would be too costly for a lot of those people, 

perhaps, and they wouldn't report, they'd drop out and 

the system goes down. 

Well, are there compensations that the system 

can make once you know what the truth would be or should 

be for people reporting? In other words, maybe in the 

sub-prime market are people who get shunted to a subprime market. Lots of people find themselves there 

because certain lenders don't find it in their interest 

or it's too costly to report for them. Well, you could 

make some sort of allowance for that profile of an 

individual or some -- how you deal with the scores that 

you do have. Fair Isaac probably already does that to 

some degree for score providers to build that and model 

that. 

So, I guess the study, in my mind, ought to be 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

172

establishing, well, what are the pieces of information 

that get systematically left out as opposed to better 

left out here and there but it's not systematic. So, 

characterize what the state of affairs is. 

Then the cost-benefit analysis, well, what do 

you do about it and that sort of thing? It might -- it 

may be nothing. The cheapest thing to do is just 

recognize that this is the way things are and figure out 

what compensations might be reasonable to take actions. 

So, at least that's the question, I guess, that I would 

be asking and trying to get a handle around and a 

description of. Rather than characterize it as errors of 

omission, just how do things deviate from a perfect 

system basically if you had a perfect system at work? 

The other thing in terms of an observation they 

couldn't -- you guys couldn't, in your study, go back to 

consumers and have them reconcile this because you had a 

blind study. I don't know if someone were to do this 

again or to go to bureau records and analyze a blind 

file, a large blind file just to establish universes or 

samples, basically, that characterize different subsets 

of consumers for which you wanted to get more 

information. 

If one got a blind file but with some kind of 

sequence number or something on it such that once you 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

173

established through analysis of a blind file, a lot of 

patterns and questions, hypotheses that you wanted to go 

explore, if there was a way where bureaus themselves 

could -- we've designed an instrument or something like 

that where you wouldn't be able to go and have -- 

administer that instrument and send it out, but if 

bureaus could send out an instrument with a sequence 

number tied to it, solicit consumer reaction to a sample 

that was designed after you got a -- after you developed 

a sample, send out a questionnaire, get answers or have a 

blind phone kind of question -- solicitation basically or 

actually have repositories solicit participation of 

individuals and they say yes or no, that they would agree 

to be contacted or whatever, such that you could link 

back through the sequence number reactions from consumers 

obtained various ways to augment what you observed in the 

blind file with the credit data alone and build that 

other information in there, that seems like a way of 

going beyond just what you could essentially observe in 

the file. 

MR. VANDER NAT: Terry, I noted that you wanted 

to make a comment on this. 

MR. CLEMANS: Yeah, I've got a couple comments. 

 Your first question, in the research we did in 1994, we 

wouldn't call this the perfect report because it's not 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

174

utilized by the GSEs primarily now. HUD still recognizes 

it, but the residential mortgage credit reports, 19 

percent of the RMCRs, the two-bureau files, had data on 

them that was disclosed by the consumer to the loan 

officer on the back of the application or was discovered 

with the consumer interview in the process of developing 

that report. So, I mean, that's a significant number in 

'94. I would guess that number is probably getting 

larger today as there's fewer affiliates in the network 

that are no longer gathering as much data locally from 

smaller creditors. There's a lot of small creditors that 

don't get into the system, not because they don't want to 

be in the system, because they're not allowed in the 

system. So, that's an issue. 

In regards to your second point about the 

blindness of it and a way to bring the consumer into the 

process, we would hope that with the powers of the FTC 

and a mandate from Congress, there would be a way to get 

the consumer actively involved where it wouldn't have to 

be a blind study, perhaps. I don't know, that's more of 

a question for you guys. 

MR. VANDER NAT: There are many in-house 

discussions on admissible purposes and what actually the 

authority is to engage in this study, what are the 

implied things that we can do and we have to leave it to 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

175

our legal staff to tell us that because it is a delicate 

problem and always the safe thing is to say within the 

permissible purposes of the FCRA and the obvious one is 

the consumer gave you permission to look at their report. 

 That's what you want. 

Now, if you didn't get this -- it is a very 

difficult legal issue and since I'm not a lawyer, I won't 

speak to it, I just recognize the issue. But ultimately 

our legal staff is going to advise us as to what we can 

and cannot do in this area. 

Yes, Nick? 

MR. SOULELES: I'm not a lawyer either, but 

just to push on that point, another alternative is you 

might decide in order to get the right sampling, if your 

interpretation of current law doesn't let you pull 

reports on people who don't want to participate or won't 

allow you to add the consumer reports to the analysis, 

one possibility, maybe it's naive, is you could go back 

and say to achieve your goals, Congress, we need an 

exception. 

MR. VANDER NAT: Again, that's one point of 

discussion. I mean, these are things that are out there 

for consideration and that's certainly been recognized. 

But we don't know the answer to it. I mean, it's -- 

MR. SOULELES: I'm not a lawyer, I was just 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

176

making the point that there is a distinction between your 

interpretation of current law and regulation and -- 

MR. VANDER NAT: Right, right, right. 

MR. SOULELES: -- possible future -- 

MR. VANDER NAT: And certainly you raise a 

point that can be considered, yes. Other observations, 

questions? Ed? 

MR. MIERZWINSKI: I wanted to get back just 

briefly to one of what I think is the most significant 

points of the CFA and CRA study, and I think it gets 

to -- a lot of people have been referencing the fact that 

we have the greatest credit system in the world and I 

think even Evan and I agree that we have a good credit 

system, but it could be a lot better. That's all we're 

here to do. 

Maybe you could use the language from the Act, 

from Section 607 that I think Stuart was quoting earlier. 

 You've got to use reasonable procedures to come up with 

the maximum possible accuracy of your study to try to 

figure it out. 

(Laughter.)

MR. MIERZWINSKI: But we respect that you're 

going to have to do that. You're going to have to make 

some assumptions and do that. But anyway, the most 

important part of the study to me is the idea that -- the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

177

finding, excuse me, the finding that up to eight million 

Americans could be wrongly characterized as sub-prime due 

to variances in their credit scores, and I think the 

study has to ultimately really take into account not just 

the cost to the industry, that's one part of maximizing 

net social benefit is to weigh the benefits against the 

costs of the system. But you've also got to look at what 

happens to the consumers who are hurt the worst and how 

do you quantify and value the cost that they are paying 

in the cost of increased credit, in the cost of denied 

credit, in the costs that they're paying going through 

life, paying too much for credit? 

Now, if everybody all paid equally too much 

because of variables in the system, maybe you would argue 

the system is working well with its variances and 

problems that it has. But if there are eight million 

Americans that are being treated badly, I just want to 

reiterate that you've got to make sure that you address 

that in your methodology and really try to drill down 

into that further. That's all I wanted to say. 

MR. VANDER NAT: Other comments, observations, 

advice? Yes, please, Mallory? 

MR. DUNCAN: A few thoughts. One -- and, 

again, I hate to be the one that keeps coming up with the 

frustrating comments on this panel, but it strikes me 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

178

that achieving a true measure of accuracy is going to be 

impossible. It's just -- it's a doomed effort. Take a 

consumer who is 30 days late. If you pull the consumer 

in, show them the report, they can say, I wasn't 30 days 

late and then you can begin the investigatory process. 

But my guess is that on looking at a report, there will 

be a consumer who, in fact, was 59 days late but the 

creditor reported and said, paid as agreed. Now, that is 

an inaccuracy, and yet no questioning of the consumer is 

going to uncover that, no reinvestigation is going to 

uncover that and we don't know if there are far more of 

those than there are the 30-day late. So, you've already 

got this large hole in your investigation. 

Secondly, in terms of variations in numbers, if 

I were to take the SAT twice, I might score 1300 the 

first time and 1375 the next time. Is one accurate, one 

inaccurate? It's the same me, same data going into a 

test on two different days. The real question is, what 

does the creditor do with that information? How do they 

compensate? Now, in this case, the creditor is the 

school. Some schools will take the higher score, some 

schools will average the scores. And so the real 

question is, what's -- that brings us back to 

materiality. So, I think we're going to keep circling 

back to that rather than looking for very small 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

179

variations. 

Lastly, in terms of accomplishing the study, if 

you want to get even a full accuracy, you're going to 

have to involve the consumer very directly. Now, think 

what that means. If a consumer pulls a report, potential 

data points in that report, say that they've got 20 

credit lines, 12 reports per year over the last minimum 

seven years, that's potentially 1,700 data points. Few 

consumers are going to be able to analyze that on their 

own and give you a very accurate figure. So, you're 

going to need an expert and a very few experts who can 

stay with those consumers and treat every one of them the 

same. 

So, the idea of having 1,000 reports out there 

with 50 different analysts isn't going to work because 

the inconsistency just among the analysts is -- you can't 

deliver the kind of training that would be necessary. 

You need a very few people talking with a core of 

consumers and try to pull out of that the kind of 

recommendations that you can use going forward with 

Congress. Anything to try to come up with a very precise 

number is not going to work there. 

MR. VANDER NAT: Thank you. 

UNIDENTIFIED MALE: Just one observation. In 

the presentation it talked about increasing number of 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

180

price points and that's probably true, there will be 

more. You know, today we have more stratification of 

offers than we probably have had in the past. I think 

this goes to Gerry's observation about cost benefit, 

though. At some point, you're going to have to debate 

with yourself the cost benefit of trying to perfect price 

point offers relative to certain amounts of information 

versus other effects that that might have on that very 

same system. We've talked about some of these in sort of 

the hypothetical, you know, to drive towards always 

getting it just exactly right for every consumer rather 

than 95 percent of consumers. 

It is probably -- and I think this is important 

for the study, at least for all of you who are developing 

the study to consider some of these issues as trade-offs 

rather than a linear path down which you can go and you 

will inevitably, at the end of the day, get to something 

better on the other side. You might be able to get to -- 

you know, one kind of result could be getting to better I 

have less data, but it is more timely. Another might be 

I have more data, but some of it is less timely. Get rid 

of all the old data, right, because it's mostly older 

data. 

How about not reporting mortgage information 

because derog mortgage information, I think in your 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

181

study, was one of the areas where you had the least 

updated amount of information, right? So, just get rid 

of that data. 

Now, hopefully, some of us are cringing when we 

start to think about these kinds of options out there, 

right? But I think it's very important to begin to 

realize this, we're moving beyond the pale of perfection 

and towards what I think is a trade-off kind of analysis. 

 Many of these analyses may enter benefits to the 

ultimate kind of discussion that we have to have. 

But I think it's very dangerous to think that 

we're heading towards some pure truth that exists out 

there that we have not yet found and that we're going to 

get there and we'll be able to get something that is just 

absolutely definitely better in all the different ways. 

The files will be more complete, the files will have more 

information, the files will be more timely, that all of 

this will exist. 

In the voluntary system that we have with the 

fact that amazingly enough -- and I guess I still have to 

keep saying it, we live in not just a good credit system, 

we live in an incredible credit system. I had an 

Austrian Data Protection Commissioner go, we like our 

system because there's not much information, everybody 

looks the same, so everybody will get a credit offer. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

182

UNIDENTIFIED MALE: It doesn't work. 

UNIDENTIFIED MALE: Right. At least economists 

are all laughing at that. 

(Laughter.)

UNIDENTIFIED MALE: And I said, well, thanks 

for sharing. What else can you say at that point? We 

were so just diametrically opposed in our view of where 

we needed to be. 

So, I believe you're heading towards a tradeoff debate, not just purely a linear truth here that 

you're going to get to at the end of the day. The tradeoff is going to be that -- and I think also some of this 

is really going to be predicated not so much on the 

purity of the data, but the smartness and the incredible 

smartness of the lending community to work around some of 

the variances in data that exist today, and there's an 

incredible intelligence out there in terms of how well 

our credit lending system operates, how smart the lending 

community is, how smart the score developer community is. 

 So, some of the issues that we have raised 

today are, yep, not perfect. I think I heard a score 

developer say, I could do better if I had all the credit 

limits out there. But would it be worse if you just had 

no data whatsoever, if you did not have any of that 

tradeline upon which to base part of your lending 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

183

decision? I suspect that is a worse outcome. 

So, I think that it's very important you begin 

to get your balance sheet out on this one because this is 

not going to be a clean discussion down toward the linear 

truth. I think that that is probably as important as 

anything, that there isn't a silver bullet. I think 

Congress is kind of looking for that. Isn't there a 

silver bullet? Isn't there something that's just 

definitely better? You know, one of the market responses 

to the fact that there are variances in files -- and, you 

know, we were in there with the RMCRs back when they were 

being used as well -- was that you had resellers. You 

know, mortgage (inaudible) companies even in our 

membership would update information in files. That is a 

market response to the fact that there's some unevenness 

in the kind of information that the Fed analyzed in its 

study. 

So, I think a report has to acknowledge that, 

that there are, first of all, trade-offs potentially to 

some of the kinds of ideas you might drive towards and 

also that there are marketplace work-arounds all the 

time, that lenders are always looking at ways to pick the 

very best file that's most predictive for them, which by 

the way is another good thing, right? I mean, it should 

be a good thing. Portfolios that perform well are good 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

184

portfolios overall. That's just framing thoughts, I 

think. 

MR. VANDER NAT: Gerry? 

MR. BUTTERS: Your comments prompt me to raise 

a fairly speculative question for everyone. If the 

industry generally has the incentives to get things right 

because through the combination of the lenders and the 

credit reporting agencies working to try to get good 

information to lenders to make good credit decisions, if 

this works out in a way so that the incentives are 

generally right, then it would be difficult for the 

government then to follow up and know in as much depth 

what the industry is doing and to try to improve on it. 

So, it seems to me that the argument that the 

industry is getting things wrong must be based on the 

premise that there's some systematic bias in the 

incentives of the lenders and/or the credit reporting 

agencies, and -- so, I'd like to know if people think 

they're -- especially I'd like to hear from the 

economists here. Is there such a -- do you think there 

is such a market failure, so to speak, that needs 

correction or is there not? 

UNIDENTIFIED MALE: Well, one of the things 

that we've talked about in the past was in terms of 

collecting as much information and keeping it as accurate 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

185

as possible is allowing the bureaus to get together and 

overcome some of the antitrust concerns, because as it is 

now, a creditor can play off one bureau against another 

if one bureau decided to try and enforce more 

constrictive reporting. 

So, I mean, there are market failures in that 

sense in the sense that one company can play off a bureau 

against another and it's hard for the bureaus 

individually to enforce as much accurate reporting as 

possible. 

One thing that I wanted to emphasize was this 

question of materiality. In my mind, at the end of the 

day, the worst thing that could happen is you guys come 

out with a report that says 75 percent of the reports 

have errors. If it turns out that most of them are 

immaterial -- as it's been pointed out, the marketplace 

does a lot of things to overcome data quality concerns. 

One of the things we don't -- we didn't refer 

to the issues that we discovered as errors because it's 

in the eye of the beholder and many of them are clearly 

not errors, but they certainly are data quality concerns. 

 Certainly knowing what the models consider in terms of 

how the attributes are put together is critical to decide 

what information you should bother having your consumers 

look at and challenge, if you will, or question, because 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

186

it doesn't do anybody any good to have consumers get free 

reports, identify lots of irrelevant things that are 

wrong with their reports or incomplete with their reports 

and have everybody spending money tracking them down. 

The second thing is the market also responds or 

has responded in how they use the data. So, for example, 

data -- in the area of prescreening, typically one report 

is pulled and it's used. In the area of mortgage lending 

where three reports may be pulled and there's 

reconciliation of the differences, the market may have 

already responded in the area of mortgage credit and 

there are data quality concerns, but maybe they're all 

resolved or substantially resolved through the market 

mechanism we have. 

So, depending on the kind of credit or the type 

of use we're talking about, this materiality makes a 

difference. Some issues are important for some parts of 

the market and they're not very important for others. 

The harm to a consumer is different. If I don't receive 

a solicitation because they happen to pull the Trans 

Union file and it was incomplete, that's a lot less 

important to me than whether I get my mortgage or not. 

So, I think you have to think, also, about how it's used 

in the market, the kinds of products involved. 

MR. VANDER NAT: Let's take about a 10-minute 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

187

break here. Let's come back at about three minutes or 

four minutes after 3:00. 

(End of Tape 3, Side B)

(A brief recess was taken.)

SESSION IV: WHAT HAVE WE LEARNED - IS THERE A BEST 

 COMBINATION OF THE REVIEWED METHODOLOGIES

MR. VANDER NAT: -- and we're going to continue 

with that here in the final session and we've asked 

various people -- and you can see their names there on 

the agenda -- to just take the lead and offer their input 

and whether in favor of our earlier consumer survey idea 

or not, we're equally interested in hearing critique and 

hearing what may be a problem there, and we've heard 

that, too. But we certainly want to hear from everyone 

their true and frank opinion. 

So, we'll start with Robert Avery from the 

Federal Reserve. 

MR. AVERY: Thanks, Peter. What I'd like to do 

-- my sense is that, A, I'm really pleased that you're 

doing it and we're not doing it. Let me keep reiterating 

that. This is very hard and my guess is that you're not 

going to find any one study that's going to answer all 

your questions. You're going to get pieces of the puzzle 

from various different sources. 

Let me comment on what is an alternative that 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

188

hasn't been mentioned but one that we've been pursuing. 

We have now just received our third national 

representative sample of -- it's really a dump of all the 

electronic files that typically would be used by a scorer 

from one of the three bureaus. We got one in 1997, 1999 

and 2003. It's about 250,000 people. It's identical to 

the data that FICO would get when they do their -- 

develop their performance model. 

We have written one article, we're updating it, 

but where we've tried to use this as a source of 

information on addressing the issue of error, and let me 

suggest or at least make some suggestions about how this 

might be potentially a complimentary tool to other 

sources in getting at this issue. 

First of all, let me say what you can't do with 

this. You clearly can't get at the issue of frag files 

or misapplication of files to consumers because you 

basically see what the bureau's assigned. So, you really 

can't address that at all. 

Secondly, you can't address or at least you 

can't directly address errors of omission because, 

obviously, they're not in the file. But what you can do 

is we do think that there are a number of things that you 

can use these files particularly if you do it on a 

recurring basis and sort of track things over time, that 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

189

you can learn from these that are relevant to the 

question of whether you use errors -- data quality issues 

that Glenn used. 

It seems -- so, let me just recite -- go 

through a number of the things that we think you could 

use it for based somewhat on our own experience. You can 

certainly identify some sources of missing information. 

Credit limits is an obvious example of that. 

But there are other items that you can track. 

One is you can identify lenders that don't report any 

minor delinquencies. One use of the file we found was to 

sort things by lenders. You can identify how many 

lenders only report derogatories despite -- look at how 

many that don't have any accounts in good standing. You 

can report how many don't report minor delinquencies. 

You never see a 30 or 60-day. You can clearly also 

identify lenders that no longer report anyone and these 

are clearly signals of lenders for whom the files can't 

be updated. You know, we can identify someone who's not 

reported any borrower in the last year. If you have a 

very large national representative sample, presumably you 

should be able to identify those lenders or at least 

approximate it. 

You can also use the data to -- one of the 

areas that we focused a lot on is stale information. And 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

190

just to -- again, along with the sort of legalistic 

discussion of errors we've had, let's say stale 

information we define as information that was accurate

 when it was reported but no longer necessarily 

reflects the consumer's present circumstances. How's 

that as a -- we had a lot of difficulty figuring out what 

to call it. 

But in some cases, it's clearly inaccurate. 

Let's say a consumer that reported -- was reported as 

being 60 days delinquent a year ago and it's carried that 

way. It may still go on the FICO score. If it's five 

months ago, it would probably still go in. Well, it 

can't be 60 days delinquent. It's either been paid off, 

it's gone to 90 or it's gone to -- but it's probably not 

60. That's probably the best bet. 

And you can look and see how prevalent these 

kinds of -- whatever the -- data quality issues there 

are, track them over time and -- another example -- 

further -- we argue you could go a little further with 

that. You could try to look at what the implications 

would be if you had updated. You can simulate various 

scenarios about what might be the case. For example, if 

those 60 days were all paid off, let's suppose they were 

-- instead of -- what they really were were a mortgage, 

change servicing, it was refinanced, they didn't get the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

191

notice out to the department handling the servicing of 

the mortgage, they just didn't get a payment, they 

recorded it 60 days past due, but, in fact, the mortgage 

was already re-fied. So, it's inaccurate and they just 

don't report it anymore. That -- you could say well, 

what -- that should have been reported correctly. What 

would be the impact on somebody's score? So, that's the 

kind of simulation exercise you can go through. 

One of the things that we're doing now -- you 

can also look at duplications. One of the things we've 

tried to do is to look at, for example, potential 

overlaps. If an account is sent to a collection company, 

it should be taken off the tradelines and then just 

reported in the collection accounts. We tried to look at 

-- guess using various items how often it appeared that 

they were duplicated. We also looked at collection items 

themselves which are just loaded with a lot less 

information in the collection items and it looks like 

they are messier. I'll just use that term, public 

records. How often they appear to be potentially 

duplicated. You see exactly the same item, but two 

slightly different dates, a different docket number. 

Is it really a person went bankrupt twice in 

the three-month period or have they just refiled but 

under slightly different terms? Those are the kinds of 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

192

things you can track getting this nationally 

representative sample. 

Now, where we think it is most useful is this 

issue of materiality. The problem you're going to have, 

unless you can piece together errors and relate them back 

to an individual and an individual score and then track 

how the score would change were the error not to have 

occurred, you really can't get out the issue of 

materiality. So, one of the things that we are using our 

data for is as a template for simulating materiality. 

Let me, in two seconds, just describe kind of the 

procedure. 

We had Paul Calem (phonetic), one of our 

colleagues, estimate -- fit a function to the bureau 

score, which is on our records, to approximate it using 

some sort of linear weighting of the attributes that are 

believed to go into this. So, we have a linear 

approximation to the score. And then we reverse engineer 

all the attributes, so that we actually figure out how 

they calculate what is the months since most recent 

delinquency for a retail trade. In the latter, we've 

done very accurately. We have 99.8 percent, I think, 

accuracy in the reverse engineering. So, we're 

reasonably confident in that. 

And now, we take an example, like let's assume 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

193

that the mortgage that was reported as 60 days, let's 

assume all of them were actually paid off accurately, and 

you just go back in the data, you reset them all to paid 

as agreed and rescore and look at the change in the score 

that would be implied by running it through your model. 

An example of where we did this is the student 

loan reports. We simulated, well, what would be the 

effect is Sallie Mae didn't report student loans and we 

simply took half of the -- they had about half the 

student loans. I think we took a set of lender numbers 

that added up to half the student loans in our file and 

we simply removed that and rescored all the people. 

Interestingly, just to show you that this is much more 

complicated than you might think, that exercise showed to 

us, on average, this helped students, it didn't hurt 

students. The reason being they are thin files. So, if 

you have a derogatory, you're going to get in the 

marginal rate. You're going to get 620, 580. You don't 

have any good reports. The students that had good 

reports still are thin even though they had one -- so, 

now they have one good loan rather than -- or two good 

loans rather than one. 

The help to the group that were in the -- the 

marginal students were much more likely to have a 

derogatory on their student loans. So, in total, by 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

194

removing the set of people -- you know, like 25 to 30 

that had student loans -- were collectively helped by 

Sallie Mae not reporting it. So, that's an example of 

where you really would get a very different inference 

than you would -- everybody would think, well, my gosh, 

this is just hurting all those small thin file students 

that don't have much of a record and you're taking away 

that one. Sure, you are hurting those students. But 

interestingly, collectively as a group, as a whole, 

you're actually benefitting them by not reporting it. 

That's an example. 

Some of them -- another thing we did to show 

you how you might even look at the issue of omissions is 

take the set of lenders we identified as sub-prime, by 

having half of their loans be made to people with scores 

below a certain level, that did report satisfactory 

accounts -- so that's the conditions -- and now you say, 

well, what would happen if they simply had not reported 

the satisfactory accounts for those people? And you take 

them out and rescore it. That is an exercise that 

simulates what would the impact be if a sub-prime lender 

chose not to report their loans. 

Now, it doesn't tell you how often that 

happens. We don't know. That's an area of omission. 

But I'm citing that as an example of how you could use 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

195

the file to look at issues of that type. Even where you 

can't measure the incidence, you could use it as a 

template. My guess is having this kind of a template 

would allow you to take bits of information from other 

sources and use it as a way of simulating any potential 

impacts. 

Now, just in closing, there's a cost to this, 

and it's actually a pretty significant cost. We have 

received just unbelievable support and assistance from 

the bureau we've been working with. In a way, I often 

have trouble understanding why they're doing it. But 

these have -- there is zero documentation. We receive a 

tape and it just -- we receive a tape and we have to 

figure out what's on there, what all the codes are. 

There's a little assistance from them. It's not always 

right when we get it. They are not used to dealing with 

people that get a dump of their files and want to go look 

at it the way we're looking at it. 

So, we have had to -- and we have a mainframe 

and this thing is a big file. It uses a lot of -- 

thankfully, it's free, but it's not an easy file to use. 

 So, there's a very significant investment of time and 

effort in -- this is not something you can casually use. 

 Reverse engineering it took a lot of effort. But I 

think the reward is, if you do make that investment, that 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

196

you can get a much better understanding of the nuts and 

bolts of how this works, and that may be a critical piece 

of glue as you put together your study to think about. 

So, that's -- I'd just encourage you to go down 

that route as using that as a potential complimentary -- 

not that there aren't other things. I'll let others 

comment on the value of the other things. 

MR. VANDER NAT: Thank you much. Now, we'll 

hear from Greg Elliehausen from the Credit Research 

Center. 

MR. ELLIEHAUSEN: I was asked to discuss the 

ideas of using consumer survey methods to review and 

identify possible errors in credit reports as a basis for 

assessing the accuracy of data. The type of survey I 

will discuss is one similar to the one outlined in the 

RFQ for the FTC pilot study. 

I've had a quite a bit of experience with 

consumer surveys over my career and I'll discuss this 

survey on the basis of my experience. 

First, in my judgment, a survey of a 

representative sample of consumers would be a necessary 

part of the analysis of accuracy of credit reports. Only 

a consumer can identify some errors or omissions. 

Discrepancies in credit reports do not necessarily 

indicate errors and the absence of discrepancies does not 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

197

mean that there's no error. Focusing on certain events 

such as credit denials on mortgage applications provides 

an unrepresentative sample of consumers and credit 

histories. Thus, if one wants data on the overall 

prevalence of errors or omissions in credit reports, a 

consumer survey appears to be needed. Since consumer's 

credit opportunities, for example solicitations, often -- 

or the type of solicitation that they get, often depend 

on information on credit reports, concern with the 

accuracy of credit reports is warranted. 

Consumer surveys are not free of errors, of 

course. Consumers may not be completely representative 

because the sample frame does not have complete coverage 

of the target population and because sample consumers 

refuse to participate and respondents may make errors in 

responding to questions. The remainder of my remarks 

will address briefly each of these areas. 

Inadequate sample frame for representative 

sample of consumers, I believe, is not a big issue. 

Credit use is not a rare event by far. Most consumers 

use credit. Random digit dialing, which samples 

telephone numbers, is widely used to obtain 

representative samples and would work for a survey of 

credit reports -- about credit reports. This method 

obviously does not cover households without telephones. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

198

About 6 percent of households, which are mainly low 

income households, do not have a telephone. Because of 

the great cost advantage of telephone surveys over faceto-face interviews, telephone surveys are commonly used, 

even to study disproportionately low income populations. 

It is often -- the under-coverage is often 

compensated for construction of weights. This may be an 

imperfect compensation. Households who do not have 

telephones may disproportionately use low income market 

retailers, rent-to-own stores or pawn brokers who do not 

report and may be untypical of other low income 

consumers. But I think there's no low cost way of 

avoiding this bias and these customers are perhaps 

another study. 

A mail survey would not be an attractive 

alternative to a telephone study. Any benefits from 

representativeness in the sample frame would be more an 

offset by the low response rate typical in mail surveys. 

 

It is not clear that any special sampling is 

needed to analyze particular groups. If one is concerned 

with errors in accounts with negative information, the 

random sample of a few thousand respondents would be 

sufficient for analysis of this group. As measured by 

FICO scores, about a fifth of consumers have information 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

199

in credit reports that causes them to be considered high 

risk of serious delinquencies or other serious adverse 

behavior, and about half of consumers have delinquencies 

of 30 or more days over a four-year period. 

Turning next to the problem of refusal to 

participate, the process of interviewing consumers about 

errors in credit reporting is more complicated than the 

typical consumer survey. There are multiple steps in the 

process with breaks in the process and the need to 

recontact respondents. 

Let me go through the steps. You have to 

contact a consumer, agree -- obtain agreement to 

participate, request credit reports, then there would be 

a break. The consumer receives a credit report. You 

have to recontact the consumer, interview, identify 

possible errors and omissions and decide to request an 

investigation, another break in the process. Send a 

consumer a form requesting for investigation, the 

consumer submits the form requesting an investigation, 

another break in the process. A consumer receives report 

of the investigation. You recontact a consumer in an 

interview. This process is cumbersome and raises 

concerns about participation and attrition. 

The incentive to participate in the survey may 

be greater than the typical survey. There are both 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

200

public and private incentives to participate. The public 

benefit is to provide information that may guide public 

policy. This benefit should not be underestimated. This 

may be one reason why government-sponsored surveys often 

have higher response rates than private surveys. The 

private benefits are the receipt of credit reports at no 

cost, assistance in understanding the information in 

credit reports and assistance in investigating possible 

errors. There may also be -- at the end of this, it may 

be possible to offer a monetary incentive for completion 

of the program, that may also reduce attrition. 

Provided that there's an assurance of 

legitimacy and confidentiality, resistance to initial 

participation is probably not a big problem. However, 

the breaks in the process provide the consumer with 

opportunities to refuse to cooperate further. This seems 

the greater concern. 

A good questionnaire design and interviewer 

training can help mitigate attrition. The interview 

length must not be too long. You have to limit the 

amount of information that you collect. That means you 

can't collect information about all possible errors. One 

has to focus on certain items that are deemed most 

important. 

Making the process of investigating errors as 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

201

easy as possible for the respondent will encourage the 

respondent to follow through to the end. And as I 

mentioned, it's possible a monetary incentive for 

completing the process would be effective. 

Finally, I'd like to discuss response errors. 

I will use a cognitive framework of the response process 

to do this. This process consists basically of five 

stages, end coding, comprehension, information retrieval, 

judgment and communication. In formulating an answer, 

the respondent must first have the knowledge required to 

provide a valid response. Most respondents likely have 

knowledge of their own credit history; however, 

consumer's conception and that of a creditor or 

researcher may differ. An example that was discussed 

many times already today, a consumer may view a long 

inactive credit card account as closed, but creditor may 

not. Many consumers may not be willing to adjust their 

views. The researcher needs to recognize such situations 

in designing a questionnaire and collect information that 

allows them to identify these situations. 

The next stage is comprehension. There must be 

a shared meeting among the researcher, the interviewer 

and the respondent with respect to questions. Credit 

reports have terminology that's unfamiliar to many 

respondents. The questionnaire will have to be designed 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

202

to help respondents understand the information in the 

credit report. Training interviewers to answer 

respondent's questions about credit reports and 

appropriate responses can facilitate respondent's 

comprehension and thereby increase the likelihood that 

responses will be accurate. 

The next stage in the response process is the 

retrieval of information. Ideally, the respondent would 

access records, but the consumer may not retain records 

for some accounts and records for other accounts may not 

correspond because differences in timing. If no records 

exist or if retrieving information from records is too 

difficult, the respondent may attempt to recall 

information or estimate the correct response. This is 

likely to be typical. Some respondents may (inaudible), 

that is, they'll exert minimal effort in responding, 

providing answers that they view as good enough rather 

than precise results. 

An interview who probes completely and 

persistently until an acceptable response is obtained can 

counteract some of the effects of satisfisance 

(phonetic). 

The last two stages of the decision process 

involve the respondent's judgment of the appropriate 

response and communication of that response to the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

203

interviewer. For closed-in questions, the respondent 

decides which response category is retrieved -- fits the 

retrieved information. For an open-end question, the 

respondent formulates his own response. If a respondent 

perceives legal, economic or social threats, he may alter 

his response accordingly. Deliberate mis-reporting can 

often be mitigated by emphasizing the neutrality of the 

interviewer and the survey sponsor. 

If adequate assurances about sponsorship and 

confidentiality, the subject matter would not seem to be 

so sensitive, but deliberate mis-reporting is especially 

a great concern. The Federal Reserve Board's survey of 

consumer finances have, for many years, contained 

extensive questions about credit use and payment 

behavior, including questions about delinquency and 

bankruptcy. The data are subject to error, but they have 

been useful in predicting consumer credit behavior in 

many studies. 

Reporting errors are much more likely to occur 

because of consumer's inability to recall some 

information precisely or because a satisfising (phonetic) 

behavior. There may be some situations in which 

consumers perceive that an error is in their favor and be 

reluctant to identify an error regardless of whether it 

is in their favor or not. In these situations, it is 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

204

unlikely that consumers would be willing to initiate an 

investigation or even admit it. One cannot force 

consumers to seek an investigation against their will. 

This possibility is unavoidable, but it's not, in itself, 

a reason to reject the use of survey methods. 

In summary, I believe that a consumer survey is 

a necessary part of a comprehensive analysis of consumer 

credit reports. The consumer survey is the only way to 

obtain data for a representative sample of consumers. 

Such a survey will be difficult and very expensive. I 

think Bob mentioned that the Survey of Consumer Finances 

cost $6 million. This has many more steps than the 

Survey of Consumer Finances. So, it will be very 

expensive. 

The data will also not be without error. 

However, there is sufficient reason to believe that 

consumers will be willing to participate in such a survey 

and they will be able and willing to report much of the 

requested information. It's important to use 

professional interviewers for these surveys. The 

training of interviewers in persuading respondents to 

cooperate and provide valid answers is very important. 

They will require additional training on credit 

reports -- on interpreting credit reports. 

Ultimately, a pilot study like the one that the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

205

Federal Trade Commission proposes is the only way to 

obtain more information about how severe the problems are 

and a more precise estimate of the costs of such a 

project on a national scale. It's an effort that's, I 

think, well worth doing. 

MR. VANDER NAT: Thank you, Greg. I think that 

during this session that we'll just have various people 

state their views and then at the end of the various 

statements then we'll conclude with some more group 

discussion. So, right now, we'll move to Alan Westin 

from Privacy in American Business. 

MR. WESTIN: In his famous oath, as you may 

recall, Hippocrates told physicians, first do no more. 

After reading all the reports that have been published in 

surveys, listening to the discussion, I'm seriously 

concerned that there's going to be nothing but harm from 

this exercise, and I want to take a few minutes and 

explain why. 

First, a little historical perspective. I 

think I'm the only person in this room that testified in 

Congress for the Fair Credit Reporting Act when it was 

originally introduced after having studied the way that 

the credit bureau system had developed in the 1960s. And 

my first perception is that the Fair Credit Reporting Act 

with all the amendments down to last year is actually the 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

206

most successful statute in the world on data subject 

access correction of errors, moving an industry into the 

proper relationship in a fair information practices sense 

to the American consumer, and it has made the American 

credit reporting and consumer system astonishingly 

effective. 

Sometimes there's been a discussion here as 

though credit is in short supply and we've got to protect 

people and they need to get to it. Hey, anybody gets 

credit. The industry pushes credit out constantly. And 

even though there can be some wrong situations, the 

general one is that with the Fair Credit Reporting Act as 

the framework and the incentives of the American 

marketing system driving the provision of the offering of 

credit, ours is an extraordinarily lubricated credit 

reporting system. 

When I travel and meet with data protection 

commissioners around the world, they're the ones that 

say, look, you passed the first major statute in the 

world on fair information practices. It works well and 

more people look at their record and get their record 

corrected than in any other system in employment, 

insurance, health, anywhere under any legislative 

framework. 

I'd be the first one to say that it's been an 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

207

interesting interaction between the industry and 

consumers on making that happen, as there have been 

changes in technology and applications, the industry has, 

in a real sense, been dragged kicking and screaming 

before Congress so that each time in 1996 and then in 

2003, very important expansions and enhancements and 

improvements of FCRA have been legislated, and then 

generally after that, the industry said, gee, that's 

great. I mean, we can live with this and it's going to 

work and it will make our system better. That's the 

dynamic of any industry in relationship to consumer 

protection or legislation. 

But I think what's important is to see that 

this industry now as a result of the 2003 Act and other 

things is in a tremendous state of potential change. The 

FACTA Act, as I mentioned earlier, will have each 

individual able to look at their credit report and that 

will create new interactions and great pressures, 

expenses on furnishers and on the industry, and there 

will be a major adjustment to that which we really don't 

know the parameters of that yet. 

And then I.D. theft is already corrupting many 

of the record systems because of the activities of very 

creative and dangerous fraudsters which means that 

instead of having a proper data system without criminal 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

208

intrusions, you've got data systems that are being 

corrupted by the activities, not just of the identity 

theft people, but also of the credit repair clinics and 

other kinds of harmful inputs to the data systems that 

are going to be studied here. 

Then you have the merger of banks, insurance 

and investment companies which makes some of them say, 

gee, we don't need credit reporting at all, we can do 

this ourselves with our own data about our consumers, and 

so you have a change in the way in which credit reports 

may be used in the future inside some very, very big and 

important players in the financial services industry. 

Then, again, you've got some major conflicts unfolding 

between privacy protection in the society and accuracy of 

credit reporting. As we limit the use of the Social 

Security number and other forms of identification for 

good reason and to try to limit identity theft and other 

kinds of forays on consumers, we're making it harder to 

accurately connect pieces of data and to achieve nonfragmented records about consumers. 

In the same way that HIPAA has resulted in some 

significant problems in hospitals and law enforcement and 

other kinds of things, you sometimes pay a price for 

privacy protection. We know, many of us, that it's worth 

it, but it has an effect on these systems that we're 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

209

looking at. So, how to factor in changing privacy limits 

on higher accuracy in credit reporting systems is really 

very important to keep into account. 

So, what do I conclude from this? I'm very 

worried that as I listen to the conversation, there seems 

to be a consensus that it will take multiple surveys with 

different samples and different criteria in order to try 

to assemble the mosaic that could give us some good 

conclusions. I don't think you're going to have the 

money to do that and I'm afraid that what's going to 

happen is the FTC is going to say, gee, we have limited 

funds, we're going to have to compromise, we won't do all 

the studies that we need to do or we'll do them but we'll 

cut back on the samples because we can't afford it. 

I very much agree with all the people who had 

said, are you really capturing the dollars that are 

involved here? You need experts, you need a variety of 

samples. You may have to pay for the credit reports. 

You may have to give monetary incentives to the people 

who are going to be giving you the information. You may 

have to go back and do record checks with the furnishers. 

 If you add up those kinds of costs, I think you're going 

to be in a staggering large, relative to the federal 

budget, kind of cost for these surveys. And it's normal 

behavior to say, hey, we've got to work with the best 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

210

we've got, Congress gave us a cattle prod and so we've 

got to do what we're told, and there will be fatal flaws 

in this exercise. 

Now, we all agree that you can't pursue perfect 

so you settle on what could be done. But if what can be 

done has really fatal errors and wounds in it, then you 

have to ask yourself what can you do. Obviously, go to 

Congress and ask for more money. That may not be the 

most realistic Washington solution, but it may be, in 

fact, the reality one. 

Is there anything else you can do? Well, at 

the least, I think, this first couple of studies ought to 

be called experimental. There ought to be a way of 

saying we draw no statistical conclusions from our first 

set of studies. We are trying to get our arms around a 

problem that everybody agrees ought to lead to improved 

credit reporting and improved consumer decision making, 

et cetera. 

So, that means that you have to communicate in 

a way that all the spinmeisters will not be able to 

change, that this is not yet a dependable study. This is 

experimental, it's provisional, it really has to be 

communicated as something that is trying to get it right. 

 You will not get it right, I'm sure, the first time with 

the amount of money that you will have. So, the trade-
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

211

off is present this as something that is a small work in 

progress and build toward in the second iteration or the 

third iteration what could be reliable studies from which 

statistical judgments could be made. 

MR. VANDER NAT: Thank you, Alan. In response 

to that, we did follow that advice in our very first 

pilot study where we say expressly that no statistical 

conclusions will be drawn from the pilot study. 

MR. WESTIN: Carry on. 

(Laughter.)

MR. VANDER NAT: All right. Let's continue 

with Richard Le Febvre from the American Credit Bureau 

and AAA Credit Expert Counseling Services. 

MR. LE FEBVRE: Well, everybody who knows me, 

everybody knows I'm pretty blunt, and so I want to have a 

couple of disclaimers. One, with regards to 

repositories, this is not an attack on the repositories, 

nor is it an attack on Fair Isaac, but I want to show you 

what reality truly is. I'm going to skip some of these 

because I've been given a time frame, but, again, 

basically pretty much what we've talked about here as far 

as what an inaccuracy is. 

Now, my interpretation as a past CRA, a credit 

inaccuracy is any misleading, incomplete and/or outdated 

data which fails to convey the full and true picture of 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

212

the consumer's credit risk, credit capacity, credit 

standing, credit reputation and credit worthiness. 

Who's to blame? Is it basically the 

repositories? Absolutely not. And as you see me go 

through a lot of this. There are issues here with regard 

to incentives with regards to credit furnishers. 

Accounts and bankruptcy, between the three 

repositories you'll have two -- basically two different 

types of ways in which they handle bankruptcy tradelines. 

 Trans Union and Experian makes the account number match 

to the list of creditors. Equifax has a different 

approach. Re-aging, again, another way of 

regaming the system with regards to changing the date of 

occurrence or date of first delinquency. The seven-year 

statute of limitations starts at the date of occurrence. 

 So, if it's being re-aged, it's a way to keep the 

statute running. 

Duplication of accounts, consumers got a double 

and triple ding for the same bad debt creating a false 

representation. This shows a false picture of a 

consumer's credit reputation and credit (inaudible). 

We've talked about omission. Don't touch that 

one. 

(Laughter.) 

MR. LE FEBVRE: Now, we did -- I want to tell 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

213

everybody, one, you're dealing with some -- I've been in 

the credit industry for about 15 years. There's probably 

more education in this room that it's absolutely scary. 

Everybody's got an MBA. 

What we did for 1999 through 2000, we logged in 

all of the credit reports and we broke them down to these 

basically six categories. We classified them as minor 

errors and what we classified as major errors. Now, this 

is a scoring system not to compete with Fair Isaac 

because it's not a predictor. It's an accuracy checker 

is what we called it. I created this back in 1998. 

Because we were one of the first resellers that had the 

ability to rescore, we needed some way of determining 

what the error rate was in a consumer's credit file. We 

went back to elementary school and we produced this to 

the mortgage brokers so they could turn around and give 

it to their consumer, especially when you're sitting at a 

672 and you need a 680 to get 2 percent down versus 10 

percent down. 

Here's our study. Basically, what we did is we 

tracked approximately 10,000 credit reports, untouched 

and we looked at them based -- if you can see here what 

we classify as missed reporting and gross errors, and 

again, on the other slide, we broke it down to exactly 

what the differences are. Again, after rescoring -- does 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

214

everybody understand what rescoring is, by altering the 

data that was inaccurate at the bureaus and update, you 

re-pull the file and it generates a new score. 

Could consumers need rescoring? Again, this 

tracks the consumers who are sitting at 750, they should 

have had a 770 hypothetically. So, it's not breaking it 

down according to scale. 

Now, here's what happens in reality. This is 

what goes on each and every day in the mortgage arena. 

It's a cartoon made to make a point. Now, here's our 

first example, and you can see what happened. You have a 

difference in score of 142 points. First item, right 

here, Discover Card was reported to Experian, one times 

30, date of delinquency 5/96, reporting to Trans Union as 

agreed, reporting to Equifax, again with that same one 

times 30, late 1996. Verizon, date last reported, 

September of 2000, August of 2000, August of 2000 again 

with the other bureau. It is feasibly impossible to be 

60 days past due with one month date of last reporting. 

And this is basically the rest of the credit report. 

So, you're looking at basically two derogatory 

items in a consumer's credit file, look at the 

discrepancies in the scores, 583, 725, 680. 

Retired female, 106-point differential based on 

a one times 30, one times 90, one times 120 and one times 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

215

150. Scores, right here, right here and right here. 

Only derogatory item on her credit file is Macy's. New 

scores, 743, 733, 744. 

This is probably the most alarming one that we 

did over our period of time. Minority consumer, there's 

a big issue on the Hill these days, is credit scoring 

discriminatory? Consumer, three-year unpaid credit file, 

mortgage paid on time. The only derogatory item on the 

consumer's credit file is this past due 30 days with 

Discover, which was an error. So, again, a thin file, 

one $10 mispayment. This is -- I think I addressed this 

before. This is what we called an unmerged file. Here's 

Discover reporting to all three bureaus. It's reporting 

past due 30 to Experian as agreed, with Trans Union as 

agreed, with Equifax. Scores, here's the differences 

right here. Scores went up with Experian because they 

updated the file. Scores took a negative nosedive, 587, 

551, 587 and 588 for one $10 mispayment, which, again, 

was an error. 

Example four, CVC pulled a report on behalf of 

a lender, Beacon score 520, 506 Empirica and Experian 

541. The problem with this file is when it was pulled, 

there are a total of 20 charged-off accounts belonging to 

his son. Everything on this gentleman's credit file 

belongs to his son. When we pulled it, because they 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

216

wanted rescoring, we pulled it, the only file that we got 

was Equifax with -- and, again, this was still inaccurate 

because it still belonged to his son. But the point is, 

you have different credit reporting agencies using 

different software to access credit files. You have one 

that reported 20 charged-off or collection accounts and 

you have another one, when we pull it, which was one. 

Merged blending problem, again, quality control 

by the lender, denied the consumer, based on this 

tradeline right here. General Motors was reporting one 

times -- three times 30. In the mortgage arena, any 

delinquencies in the last 12 months is a major no-no. 

That's CISCO, CBC. Again, General Motors reporting three 

times 30. Now, here's the problem. All three 

repositories -- again, this is not a repository issue, 

this is a furnisher problem. What happens, this is 

called mortgage blending. If you add Trans Union, 

Equifax and Experian together, you get three times 30. 

It's a dispute that can never be corrected if the 

consumer disputes the repositories at three times 30. 

This is -- example number six was the 

Washington Post article where we took a lawyer out of 

Orange County and rescored her and moved her 200 points. 

 Basically, the problems in her file were repossession, 

numerous account balance errors, late payments and paid-
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

217

in-full mortgages. Mercedes Benz was reporting a car 

that she didn't own as repossessed. I'm not showing you 

the whole file, but I make my point. Her scores, 597, 

569, 580. After rescoring, 780, 738. And there's the 

problem, Mercedes Benz was reporting the account -- it 

says, paid-in-full, but the account was not even hers. 

Example seven, differences -- I gave you an 

example of Sears before, we see this every single day. 

Reposit -- in this case, Unified, REF -- I don't know 

exactly who that is -- reports seven times 30 to Equifax, 

eight times 30 to Experian, two times 30 to Trans Union. 

 Look at the discrepancies in the date of last 

delinquency, '97, 2000. Big issue with a consumer's 

credit score. Again, here, three times 30 with Equifax 

and Experian, never delinquent with Trans Union. 

Consumer reporting -- Greenpoint reported her 

six times 30 days delinquent, all within the last 12 

months. All three repositories in this particular issue, 

meaning Greenpoint reported it the same to all three. 

You have -- this is what we'll call a universal data 

form. This is the form in which the credit grantor sends 

to the bureau on a manual or an automatic basis. You see 

right here, May 25th, 2001, first UDF to correct the 

mistake. Still not correct. Again, April 2001, very 

clearly, it's to remove the six delinquencies that were 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

218

in error. Again, May 14th, 2001, still reporting six 

times 30. It didn't get corrected until we actually 

rescored the consumer file. 

Last example, public record, one of the things 

that Hogan and most of the other credit -- public record 

data furnishers, they don't -- they have not figured out 

a way to report that that's a $9 judgment in favor of the 

plaintiff. It comes up as a $9 judgment against the 

plaintiff. 

And here's my issues with regards to -- 

(End of Tape 4, Side A)

MR. LE FEBVRE: -- create a standard of 

practice to define clearly what an inaccuracy is in a 

consumer report. Use a combination, in my opinion, of 

the CFA, NCR study in conjunction with certain percentage 

of consumer overview. A panel of independent experts or 

at the FTC then must review all credit files in order to 

determine a true inaccuracy. And that's basically my 

presentation on what I saw for 15 years. This is not 

abnormal, this is what we saw every day. There's truly 

nothing the repositories can do to stop it because as 

resellers, we saw their variations in scores, we saw 

them. And when you get the same furnisher reporting 

totally different sets of data, you question the 

integrity of our credit system. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

219

MR. VANDER NAT: Thank you, Richard. We will 

have one more presenter and then we'll enter into a group 

discussion. We have Joseph Duncan from the Information 

Policy Institute. 

MR. DUNCAN: I was waiting for Evan to do his. 

 That's all right. Did you skip Evan? 

MR. VANDER NAT: I'm sorry, I was reading too 

quickly. I did not mean to miss you Evan. I truly did 

not. 

MR. HENDRICKS: That's an omission, we don't 

know if it's an error. 

(Laughter.)

MR. VANDER NAT: No, that was an oversight. 

UNIDENTIFIED MALE: We tend not to think that 

was an error, actually. 

MR. HENDRICKS: Quiet. 

(Laughter.) 

MR. HENDRICKS: Anyway, Richard had a great 

segue because as someone who watches this system very 

closely through cases by working with attorneys in this 

area and by doing the research that I've done, I think 

that we do have strong evidence of inaccuracy at least in 

certain circles. We just don't know how pervasive it is. 

 That's why this study is so valuable. Therefore, I'd 

like to sort of identify some of the areas of accuracy 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

220

and inaccuracy and come up with ideas as to how we might 

want to approach this. 

But I, too, favor an experimental approach, but 

also a pluralistic approach. I think that we've talked 

about consumer surveys, we've talked about looking at the 

universe trying to get a global feel. But I think also 

part of this survey ultimately should be drilling down 

and looking at what -- looking at the areas of chronic 

inaccuracy and finding out what is causing those chronic 

inaccuracies. I think that will be very helpful. 

Professor Westin already mentioned identity 

theft. Identity theft is a direct assault on the 

accuracy and integrity of credit reports and the main 

damage from identity theft is cleaning up the pollution 

of the consumer's credit report. I think also, notably, 

identity theft started out as a person by person crime; 

now it's sort of gone wholesale. Now, gangs are doing 

it. They're getting into databases, they're hiring 

insiders, they're hitting many mailboxes at a time. If 

they can't convert the mail into an identity theft 

instrument, then they sell it to a fence which can. 

There's many different ways that identity theft 

is happening and from identity theft, we have this issue 

that's come out recently in the press, the issue of 

subfiles that when an identity thief applies in their own 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

221

name, but uses the Social Security number of the victim, 

then that creates a subfile and when the victim tries to 

find out what's in their polluted credit report, they 

can't get the subfile because it's in the name of someone 

else. So, this algorithm allows the disclosure of the 

victim's file but doesn't allow the victim to come back 

and get his or her file so they can start cleaning it up. 

 This is part of an ongoing investigation in Utah and I 

think there's other litigation pending over this. It's a 

very significant problem. Again, we don't know how 

extensive it is. 

Similarly, mixed files was identified as a 

problem in the early 1990s through the PIRG studies and 

through enforcement actions by the Federal Trade 

Commission and State Attorney Generals. All the evidence 

I see, including the CFA/NCRA study indicates that mixed 

files continues to be a problem here 13 years later, and 

I know that at least one CRA and possibly all of them 

have not significantly altered their algorithms for 

matching and merging data. 

Therefore, despite the fact that we've had 

these consent decrees -- the consent decrees form the 

basis for the 1996 amendments and the need for even a 

higher and more specific standard of care has led to 

FACTA, but there still has not been, in my opinion, 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

222

significant adjustment upwards of the rigor in which we 

try and make sure that people are who they say they are 

when we merge their data into the (inaudible) credit 

report and disclose it to a credit granter. And, of 

course, identity thieves have benefitted from this 

situation. It all goes to, again, a direct assault on 

accuracy. 

Then we have -- another thing that's 

interesting, and it's a little farther out there, but I 

think it's something that we have to keep in mind just to 

see if there's any correlation is that bankruptcy 

statistics spiked. They basically held steady around 

840,000 a year, but then in 1995-1996, they spiked to 

about 1.1 million bankruptcies. Elizabeth Warren has 

done a lot of terrific research on bankruptcy, but I 

thought it was interesting that this happened to coincide 

with when preapproved credit card offers became much more 

prevalent and those, of course, are based on credit 

reporting. 

Now, I'd be the first one to say that there's 

no way that I think that the rise in bankruptcy is caused 

by credit reports or the credit reporting system, yet I 

think we have to be careful to make sure we look to see 

if there's a link there and I don't think anyone's done 

that yet. I think one way you could go from bankruptcy 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

223

cases and maybe back up is another -- maybe that's a 

study for another day, but it's something that we want to 

keep in mind. 

And then the final issue is that as we talked 

about free reports, the credit reporting agencies were 

known to have said that they weren't as concerned about 

giving out the free reports as they were about handling 

all the disputes that would come in after people started 

reading them. 

Now, you still could say that that's because 

people are going to misunderstand their credit report and 

things that are not truly inaccuracies are going to cause 

those disputes, but I still think that's an indication 

that there's a significant level of inaccuracy out there, 

and as Professor Westin pointed out, we're going to learn 

a lot more about this in the coming 18 months as more and 

more people, we expect, will be getting their credit 

reports. 

So, in terms of looking at this universe of 

chronic inaccuracy, I think one place that we'd consider 

starting is complaints to the Federal Trade Commission. 

It's my understanding that the FTC has gotten some 15,000 

complaints per year or so about problems with credit 

reports, and that's a universe of people that clearly 

have some kind of problem that have figured out that you 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

224

can complain to the FTC and have written to you, and I 

think that's a cry for help. I think that's a universe 

of people that would be easy to get in contact with and 

that would be very revealing in terms of are there 

consistent patterns running through that universe of 

people. I think they would be very willing to cooperate. 

Similarly, we have victims of identity theft 

and victims of mixed files, I think, would be harder to 

reach, but certainly both the FTC survey and the identity 

theft resource center have already shown that there's a 

direct correlation between being a victim of identity 

theft, problems with your credit report, the damages 

they're trying to clean up and the chronic inaccuracy 

that occurs. In the FTC study, I remember there were 

percentages of how many people were satisfied with their 

relations with the credit bureaus is one of the 

statistics buried deep in there. So, I think there's 

potential there to understand what are the patterns. 

I think another very fascinating possibility, 

and this is something you'd really have to find the right 

scope to make it manageable, like maybe just do it in a 

zip code or -- and this is where we really need ideas, 

but to do a Social Security number audit. To take a 

defined universe, because there should only be one person 

associated with one Social Security number, yet there's 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

225

anecdotal evidence to indicate that there's a lot of 

different names and addresses -- because of mixed files, 

because of identity theft and because of, you know, 

perhaps the matching algorithms, there are a lot of 

different names and addresses associated with one Social 

Security number. That would be something you could do in 

an automated way to get some really interesting leads and 

then decide, is this manageable to dig down further and 

get more -- is there a chronic inaccuracy here? 

Finally, in the General Accounting Office 

report, I think that the one that came out this last 

summer, Stuart's group provided some statistics where 

they said -- this goes to the issue of investigations, 

reinvestigations, disputes. According -- the CDIA said 

that 46 percent of the disputes were verified as 

reported, and I'll stop right there because we've seen 

several cases because, when people dispute a lot of 

times, I mean, they're supposed to investigate but the 

credit bureaus and the credit grantors don't really 

investigate. They just do an automated comparison of 

what they reported before to what's being disputed. So, 

rather than really investigate is what they reported 

before accurate, they just say, we confirm, we verify 

that's what we reported before, and they side-step the 

issue. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

226

So, we've seen a lot of cases where when they 

say this is verified as reported, they were not going to 

the underlying truth, and they were perpetuating 

inaccuracy and they were perpetuating problems for that 

consumer trying to get it cleaned up. So, the whole -- 

we know that most people request their credit reports 

after they get an adverse action notice. So, the whole 

issue of drilling down into this universe's statistics 

could be very revealing. Besides those that were 

verified as reported, 27 percent were modified or updated 

per the furnisher's instructions, 10.5 percent had data 

deleted per furnisher's instructions, 16 percent had data 

deleted due to statutory time limit. So, all those -- 

some very interesting possibilities, that's data that's 

already captured. 

In closing, I'd like to say, you know, 

we talked about the materiality of, you know, if 

someone's -- the real issue is if someone's denied 

credit, and we know in risk-based pricing that the 

harm -- the credit and the pecuniary harms are going to 

be much more subtle, but they'll be real nonetheless. 

But as economists, I think you also want to identify the 

other costs that are involved here. There are harms in 

costs that are recognized by the law as actual, and 

that's the harm and the cost of trying to clean up a 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

227

credit report that's inaccurate, the time it takes. So, 

it's time, energy and there's sometimes lost opportunity 

because you have to spend your time doing that rather 

than something else. I think there's also a certain 

level of frustration and emotional distress, which is 

also recognized as a harm and a cost. 

So, I think that we want to -- as we look at 

the costs, realize that there's more at stake here for 

consumers than just the pecuniary costs. Thank you. 

MR. VANDER NAT: Thank you. I guess now we 

move to Joseph Duncan from Information Policy Institute. 

MR. DUNCAN: Thank you very much. It's tough 

to follow all these comments that have been made today. 

What I want to do is offer some summaries about what I 

think are four areas that need some attention. But, 

first, I'd like to congratulate FTC for having this 

forum. I think an awful lot of interesting comments have 

been made that will be nuggets for you to consider, that 

will be useful. I think an awful lot of good information 

has been revealed. Clearly, there's a Congressional 

mandate to work to improve the credit reporting function, 

which I happen to believe, along with some people around 

this room, is one of the key drivers of the U.S. economy. 

Alan Westin did a nice job of pointing out some 

of the changes that will flow because of the law itself 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

228

in terms of people looking at their credit reports. I 

would just like to add to that that we also have 

increased use of searching for, shopping for credit and 

automated underwriting. Those are our fundamental 

changes in the way that credit is allocated. Those are 

accelerating and changing the environment. 

I'd also like to comment on the fact that we 

are leaving, in an economic sense, an unprecedented 

period of low interest rates. During the refinancing of 

housing, a huge percentage of refinancing has been in 

variable rates. Some are variable in three years from 

now, some are variable one year from now. As interest 

rates rise, households are going to find a squeeze on 

their finances because of that adjustable rate taking 

money out of their pocket. 

So, while we begin to look at the impact of 

credit reporting, we're going to be doing it in an 

environment of growing credit risk. So, that adds 

another dimension, which I think we need to track. This 

really brings me to my major point, which is that the FTC 

has a unique opportunity because this is the start of a 

longitudinal program which will hopefully extend for 11 

years and is almost destined to be 11 years of dramatic 

change from our present condition. 

So, I think the early design needs to try to 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

229

capture information at this turning point and see what 

the effects are. I think that will be something we 

haven't talked about today but may, in the long term, be 

very valuable. 

The other three things I want to comment on 

besides the longitudinal analysis are the issues relating 

to the consumer and the need for robust sample design, 

which was covered a little bit earlier, and the 

definitions of accuracy. I, obviously, can't deal with 

those in detail, but I have a couple thoughts on those 

four subjects. 

When you think of longitudinal research, there 

are two forms of it. One is cross-sectional. You're 

doing a similar survey in different years or maybe every 

two years, maybe every year, and you get a crosssectional picture of what's going on at that time. But 

another form of longitudinal survey is to have a panel or 

a cohort that you follow over time, and I would suggest 

that when we're looking at credit, we actually need to do 

both of those. We need a profile of the credit reporting 

function to look at some of the issues that have been 

posed around this table on a cross-sectional basis year 

after year. But we need to select a sample of people to 

follow over time, and I think if we capture that we'll -- 

the two approaches will provide an enhanced perspective 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

230

on a longitudinal basis. 

Turning to the consumer, we really focused on 

the consumer today in many ways because the consumer is 

the ultimate person in terms of knowing about their own 

credit report, although I would suggest, as many other 

people have, they don't understand what credit reporting 

is, they don't understand how to read the files and they 

would be absolutely bewildered by the previous 

presentation as to what that means for them. I don't 

think any of the consumers, including some of the experts 

around this room, would be able to pick out all those 

points. 

So, there is a problem of educating the 

consumer that has to be part of a consumer survey. Now, 

we've talked about using experts. I would like to add to 

the experts two other ideas. I think it's possible to 

develop a set of written materials that explain to 

consumers some of the elements we're investigating. In 

fact, that could be part of a mailing that they receive 

as they participate in the survey. So, the consumer can 

read some materials and learn about what they're looking 

at on their own. 

Additionally, I think we ought to seriously 

consider using computer-aided instruction to train the 

experts. Now, I don't mean to teach the experts, but to 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

231

give the experts a format for approaching and dealing 

with the consumer so you have consistency across the 

realm. Otherwise, you have, as has already been 

mentioned, I'm just underscoring it, different experts 

treating different people and getting different results. 

 It's the experts' differences that you're analyzing, not 

the behavior of the consumer. So, I think we do have to 

help the consumer along and the consumer is a focus, but 

I think those two things would help. 

The third topic is really the demographics that 

we've talked about. When you sit -- I'm a statistician 

fundamentally and when you sit down and you draw up a set 

of survey designs that say I want to fill in the 

following cells, you suddenly discover that we're talking 

about an awful lot of cells because we want to look at 

geography, we want to look at ratios, we want to look at 

income and other basic demographics. We want to look at 

various markets, like credit cards versus mortgages 

versus insurance. Now, you can't do that, I don't 

believe, in one survey. That's already been commented on 

by a couple people, but I think you really have to follow 

some of the suggestions made today to develop a set of 

modules to answer some of these key questions. 

You first have to determine out of all of the 

dialogue today, which are the key questions you want to 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

232

focus on. You can't do them all in the beginning, 

regardless of what the budget is. You have to constrain 

it to some extent. 

But I think there needs to be, at the very 

beginning, a clear commitment to a large sample that will 

let you get that fine grained or you're not going to get 

anywhere in the end. You won't answer many of the 

questions that have been posed. I think a modular 

approach is very, very possible to be economical. 

And then finally, and I leave it until last 

because I think it's the most difficult, we really had an 

extraordinary discussion about what is accuracy, and we 

haven't really decided that today and this kind of a 

group is not going to decide that. 

But I think the FTC has to set down very 

carefully and define two things, what kinds of accuracy 

are you looking at and what metric are you going to use 

to measure them? That has to be done before the whole 

thing begins or we'll be in trouble down the way. But if 

you combine a good longitudinal survey design, a good 

relationship with the consumer, sufficient sampling and 

clear definitions of accuracy, I think this is 

potentially a very powerful tool for keeping the U.S. 

economy healthy. 

MR. VANDER NAT: Thank you very much. We have 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

233

a few minutes left of group discussion. I suppose you're 

all rather tired and you've all heard an awful lot. I'm 

not sure how many people will still volunteer to say 

anything, but we're very happy yet to hear from some 

speakers. Tom? 

MR. DANCHIK: Just a couple things. I know 

everything I'm going to say you've already heard, but 

material impact, the key there that I see is it's going 

to differ. That material impact is going to differ 

across each data furnisher or each lender who's trying to 

make this lending decision based upon this information 

that's presented in this report. It makes the FTC's job 

much more difficult to determine what's material. There 

might be different pricing points across different 

lenders. If you brought five or ten lenders in here, 

showed them all the same file and said what price are you 

going to give them, I think that's going to differ. Some 

are going to even say no to that consumer. It depends 

which area of the market they're focused on. 

So, you're going to have to segment beyond just 

yes, no and what price point, you're going to have to 

look at, well, what segment of the population does the 

lender typically work in. I would agree that the system 

works better than any other system that's out there. We 

have also had companies come from China to say, how 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

234

should we create our credit reporting agency so that we 

can do the kinds of things that you can do in the United 

States. So, I agree with the folks, and I think everyone 

has said, yes, it works very well. There's that question 

about how much better can we make it. 

Any inaccuracy that's out there that is 

typically focused on one of the CRAs is going to be dealt 

with by the marketplace. If there's one CRA that has a 

problem, the lenders have no problem going to that 

particular CRA and letting them know of our displeasure 

with something we're seeing on the file. Now, I don't 

recall having to do that recently, but that is something 

that we certainly would not be hesitant to do at all. 

So, there's a set of checks and balances that 

are out in the market today. You heard how the mortgage 

industry pulls all three files. Well, why do they do 

that? They do that in order to make sure they can give 

the person applying for that mortgage the best decision 

they can. So, depending on what kind of issuer you are, 

sometimes you might pull one, you might pull two, you 

might pull all three bureaus. 

Just completeness, is more data better? Yes, 

typically. But there -- you can get credit if you want 

credit. I would have to say you can find someone out 

there that will grant you credit. So, if I say no, 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

235

you'll go find someone else. 

And then the last thing I want to say is, I've 

just got to take a slight issue to something that Evan 

said that said the CRAs and the data furnishers don't do 

an investigation in regard to disputes. I 100 percent 

disagree with that. We do do an investigation. We look 

into our records to see what's there, we look at 

statements if we need to look at statements, to make sure 

that that dispute is responded to correctly. 

The other issue that's a little misleading is 

the fact that, you know, 46 percent of them were 

verified. Well, out of the ones that weren't verified, 

if somebody's disputing a balance, that can change very, 

very quickly, or if someone's disputing a date on a 

credit report, that can also change very, very quickly. 

And as a furnisher of data, I have to say, well, modify 

the data, even if it -- the date just changed because the 

account cycled. So, you have to be very careful when you 

look at those things in regards to timing. Thanks. 

MR. VANDER NAT: Thank you. Yes, David? 

DAVID: Hi, David (inaudible) from Credit 

Expert. I just wanted to touch base on two different 

points of feasibility that people have addressed briefly 

I think today, but not in any depth. 

Two points of feasibility. First, feasibility 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

236

in terms of solutions. That's something that we haven't 

really discussed very much today. There's a lot of 

debate about the approach and trying to measure 

inaccuracy, regardless of how you define it. And that's 

definitely a very important first step, but we do 

recommend that concurrent with that, there is an effort 

to determine possible solutions and the feasibility of 

those solutions. I think that would tie into what a few 

other people have voiced today where then, therefore, we 

would be able to -- as results come out from findings 

regarding amounts of accuracies or inaccuracies, you'd be 

able to combine that with -- or temper it with potential 

solutions and the feasibility of those solutions and 

provide some cost-benefit guidance so that it is more of 

a fair assessment of the current situation in the 

industry. 

Feasibility of the solutions should be a 

relatively -- I look at things very simply. I would 

think that it would be a very simple easy thing to 

determine. The sources of errors that are occurring 

today, regardless of how you define them, we know them, 

we know what they are. There's no mystery. So, people 

could feasibly start looking into this now. We know 

what's causing the problems, how do we address them, 

what's the cost of doing that, the cost to the industry, 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

237

how does that roll down to the consumer? 

And then additionally, the other type of 

feasibility that I wanted to address is the feasibility 

of performing the studies. I've heard a number of people 

say today, you know, talk about limitations on resources, 

access, funding. I would ask the Commission to really 

look outside the box and consider that. Don't apply the 

same constraints to this study going forward that 

traditionally or historically existed in previous 

studies. There are things available, mechanisms and 

technology that is available today that can automate 

processes, making it more efficient, making it less 

costly, allow for larger sample sizes, as Mr. Duncan 

pointed out. Also, to take some of the subjectivity out. 

 Technology that can benefit the experts or coaches that 

are involved in the process by providing a more objective 

basis from which they can work. 

MR. VANDER NAT: Thank you. Anybody else want 

to offer any concluding comments or observations? Yes? 

UNIDENTIFIED MALE: Actually, I guess -- I 

would simply summarize by saying, it looks like to me 

that there are two tasks that ought to be taken in 

echoing Mr. Duncan's comments in terms of the first 

looking at the type and incident of inaccuracy on items 

that are important for various different credit decisions 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

238

and differing populations, and that seems to me was 

offered in a number of people going through data that 

already exists, repository data and designing a study 

that is -- or the types of populations that you believe 

have perhaps differing behaviors or differing kinds of 

outcomes or things that are of interest, stratifying 

sample -- getting a stratifying sample of those 

individuals, looking at credit bureau reports, a crosssection to identify within files the type of information, 

assess difference in scores, identify score pattern 

differences, score changes. 

Those score changes basically can help you then 

identify -- and perhaps variance in terms of things that 

may not be embedded fully in a score, but lenders 

underwrite on in terms of their override sorts of things 

that beyond what's in a FICO score, people -- lenders may 

look at, bankruptcy at a certain time or different points 

in time. So, additional credit factors that are in a 

file that you know that various lender operations 

underwrite on, look for those types of changes or 

impacts. If it was there before, now it -- I mean, it 

wasn't there before, now it's there essentially. That 

would give you a population to go -- further do kind of a 

study on to see was it disputed. There would be dispute 

things in the files and non-dispute things in the files, 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

239

something like the study that was done before. 

But in designing that population initially from 

your basic sample design, keep a sequence number or 

something in the file such that you could ultimately, 

after you narrow down on a population or a subpopulation, sub-strata, if you could obviously go in with 

names beforehand and draw a sample or -- from this which 

becomes a blind sample which you can then analyze and 

find out where are these patterns to be investigated, one 

could later go back and associate names, not necessarily 

directly, and see if repositories or someone can solicit 

additional information -- participation. People have to 

participate but they could be approached, basically, to 

see if there's resolutions. 

One is identify what's important. You have 

pricing points that are different for different lenders, 

different types of things. To the extent that it's riskbased pricing on pricing points, it's easier to access 

harm necessarily like this simulated sort of a thing 

because when we have score differences, that translates 

into, okay, what would be the impact of that if somebody 

were underwriting this. So, you can more or less look at 

the magnitude of is it important or not, under what 

circumstances is it important. 

The second thing, once you learn from that kind 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

240

of analysis of files, you can basically have a panel -- 

set up a panel -- a stratified random survey panel which 

you follow forward with people buying in at the beginning 

and giving the authorization to examine their credit 

reports over time, how they change, and you can 

periodically go and see -- approach the people and have 

some ability to follow financial -- get financial 

information directly from the consumer, like what kinds 

of credit have you opened up, see if it gets reflected in 

the file, a way of basically independently sampling a 

population through either survey or a contact or written 

records that they hold or something like that that you 

could match to see does that get reflected in credit 

files, on credit scores, if you're trying to get an idea. 

Not on everything necessarily, but on the 

things that you've determined are important and have for 

credit decisions and essentially make a difference. 

While you won't necessarily get accuracy on the whole 

state of affairs, you can develop over time some -- in 

fact, does the system work well for the things that are 

important essentially and where does it not work as well 

and are there things that could be used to improve it or 

is this as good as you can get. Do a cost-benefit 

assessment where you find out, well, things aren't 

perfect, but this may be as perfect as you can get it for 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

241

the costs that have to be incurred to try and get it more 

perfect. 

MR. VANDER NAT: Thank you. Donato, did I see 

you have your hand up? 

MR. VACCARO: I was just going to make the 

suggestion like the gentleman before about thinking 

outside the box. One thing that I think would cost a lot 

of money that -- Alan Westin earlier was saying one of 

the biggest -- one of the big concerns is about the cost 

that would be incurred by carrying out such a grand 

project, and one of the ways to reduce the cost is to 

maybe focus on a group of people that may not be 

representative of the national population but may, in 

fact, help us to do what we want to do, which is to, 

first of all, explore what some of the inaccuracies and 

problems are in credit reporting. 

So, by somehow -- I can't come up with 

something creative immediately, but somehow coming up 

with a creative sample, getting a group of people that 

would, first of all, be interested in doing this kind of 

thing and participate -- that's another fundamental 

obstacle is just getting people to be so involved and 

spend so much time to give us the information about their 

personal lives. But by obtaining a really eager sample, 

by engaging them and getting information we need from 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

242

them, though it may not be nationally representative, it 

may, in fact, provide us the information that we need. 

All I'm doing is suggesting that we step 

outside the box and consider that we don't necessarily 

need a nationally representative sample at first and, in 

fact, that might be exactly what helps us to move along 

further with the process of getting to that goal 

eventually. 

UNIDENTIFIED MALE: I mean, that would fit into 

Alan's suggestion that the design is experimental. From 

that subset you just mentioned, you would not be drawing 

statistical conclusions, I imagine. 

MR. VACCARO: Well, you could -- you wouldn't 

be able to draw statistical conclusions that would be 

generalizable to the national population, but as long as 

you specified caveats and explained what, in fact, your 

conclusions related to, do we know what a certain 

subgroup of the population said or people like this, then 

anyone using that information could make their own 

conclusion about what they think that means. But the 

point would be just very clear about what your population 

is and then, you know, use that information as needed. 

UNIDENTIFIED MALE: You could identify where 

the system isn't working as well and follow that 

population through to find out if there are ways to -- 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

243

the extent to which it's not working and if there are 

ways to improve it basically. 

MR. VANDER NAT: Please. 

UNIDENTIFIED MALE: Along a similar line to 

your point, if you look at the social sciences 

literature, most of the research done there is on college 

students, and yet, they're making these extrapolations to 

the population at large, what will somebody do in a 

certain situation. 

But what that really does is identifies -- 

whatever the social science is, whether it's psychology 

or sociology, it identifies an underlying process that 

might occur given different operationalizations of that 

process. And I think your suggestion really is -- it's 

not about looking at specific consumers and types of 

consumers, it's really just looking at credit reporting 

in general and by just kind of looking at an eager 

population, you're really getting at some of the 

fundamental processes underlying reporting because 

there's really nothing intuitively that says to me that 

an eager population is less likely to have some sort of 

reporting error than a non-eager population. As I say 

that, I can think of exceptions at that point. An eager 

population is probably more likely to pay their bills on 

time, i.e., not have collections and so forth. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

244

But along that same theme of eager population, 

I hear a lot of people talking about longitudinal studies 

and I think that's a great idea, it would be very 

expensive. But just something for you to keep in mind 

when scoping that out is given how expensive the study 

could potentially be if you did a longitudinal study and 

how painful this could be for a consumer to track them 

over 11 years and every year or two have this interview, 

there's going to be a big attrition issue, and that 

attrition is probably not going to be random. In other 

words, it's going to be in the people that are most 

likely to have those painful phone calls or those painful 

interviews. 

So, in planning out such a study, you really 

need to keep attrition in mind as you pursue these goals, 

and that might be a reason to not pursue a longitudinal 

study. You really have to look at what could you 

potentially gain from a longitudinal study as opposed to 

just doing those kind of fixed snapshots over time of 

different operations. So, just a thought for you to keep 

in mind. 

MR. VANDER NAT: Thank you. 

UNIDENTIFIED MALE: (Inaudible). 

MR. VANDER NAT: Yes. 

UNIDENTIFIED MALE: That is it may be that if 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

245

you define the areas you want to study rather than trying 

to do -- and this is a variation of what people have said 

-- one large study, you do several different studies 

focused on that particular issue. So, that, for example, 

if the question is, are furnishers furnishing information 

accurately, look for a population, perhaps students, who 

are brand new to the credit market, and there's likely to 

be very few other effects showing up in their credit 

report other than whether or not they're being furnished 

accurately, and use that. Then put that issue aside and 

say you found X percent error. You can then apply that 

error to a more robust study of another population, are 

their files -- an older population, are their files being 

merged properly, and you don't have to reexamine the 

underlying furnishing issue when you go to the merged 

issue and so forth, for each portion of the study. 

UNIDENTIFIED MALE: I have a slightly different 

take on the last few minutes of discussion. We already 

know an awful lot about where there are differential 

weaknesses in the current system, and I'm thinking of the 

(inaudible) study in particular. So, I think it's 

awfully important to get to a representative sample. I 

don't -- you know, if we want to know something about 

accuracy of the system, I think it's awfully important to 

get to a representative sample as quickly as possible. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

246

So, from my perspective as an econometrician, I 

can -- the only reason I would see to use an 

unrepresentative sample is for early feasibility studies 

as you initially discuss. I don't see the value of 

studying college kids at all. I just don't know how 

easily that sort of example generalizes. We already know 

a lot about where to proceed. 

As far as I can tell, the two main sampling 

issues are -- that have been thrown out today are, do you 

start with the grand design that Greg had mentioned where 

you start with a sample of consumers and get everything 

on them and check -- and start from there with the 

checks, or do you do something along the lines of what 

Bob and Terry and Brad have done which is, the expert 

starts examining the data for inconsistencies and it only 

goes to the consumers at that point? The latter, as 

people have pointed out, will not find all possible 

errors, but it is a lot easier and cheaper. I think -- 

as far as I can tell, Bob hit the nail on the head, the 

main trade-off is this sampling issue. Do you go for 

everything at the start? Is it worth the cost? 

But I strongly feel it's got to be 

representative awfully quickly. 

MR. VANDER NAT: I saw a hand here somewhere 

else. Yes, Bob? 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

247

MR. AVERY: Well, I mean, Nick just said it, I 

would urge you all -- of all of the comments I've heard 

today, the one thing I think would be most dangerous is 

not to do a representative sample. This whole issue has 

suffered from convenience samples, from self-serving 

studies, and I think if you go down that track, you're 

just -- you can write the spin right now as to what's 

wrong with it. I think you'd be much better off with a 

somewhat imperfect design, but one which is 

representative. You can always attempt to prune a 

representative sample. You can take a sample of people 

and look at those that just got credit within your 

representative sample of consumers and maybe focus in on 

them. But you know the framework within which you're 

dealing. 

The one area, though, that I might urge you to 

put a lot of energy into thinking about -- it's very 

clear, for example, if you did a CFA study and you start 

with Social -- people who just got a mortgage, they 

almost surely started with Social Security number and 

they just got a mortgage or they applied for one so they 

probably have clean credit. You could think about doing 

a different design, which is starting with a mailing 

list, starting with a mailing list that's typically used 

for preapprovals or from Social Security numbers or from 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

248

random digit dialings. Each of those frames is actually 

different, even if you did a representative sample of the 

frame. And it may be one of the things you can do is to 

see how different a CFA study would be if you started 

with a mailing list, a list of names and no Social 

Security. 

How much error is there across the bureaus in 

that compared with Social Security numbers, or take the 

same set of people (inaudible) get a Social Security 

number for some people and see how much more accurate the 

scoring -- the variances -- whether they're different if 

you sent in the Social Security numbers versus matching 

by name. But not -- in any of those instances, I'd urge 

you not to deviate from representativeness of whatever 

the target population you're going after. I just think 

that's a big mistake. 

MR. VANDER NAT: Yes, Stuart? 

MR. PRATT: I couldn't resist one more comment 

here at the end of the day. I'm going to leap over all 

the methodological discussions that we've had. The FTC 

brings with it a different weight in terms of what this 

study will mean to many different audiences. The speed 

with which you rush to conclusions before you've 

understood the system really will dictate, I suppose, in 

some ways the success or the failure of the study. I 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

249

think that taking your time in terms of experimenting 

with the best methodologies is important and you 

shouldn't feel the need to meet a December 1st deadline 

of determining the first and best and only methodology or 

methodologies. If it takes you two years to experiment, 

to move towards the one that you think is best, I think 

you should do that. Because whatever you publish, if it 

is wrong, it could have very serious effects on a system 

which is working very well. 

We shouldn't be driven by the emotions of any 

member of Congress individually or corporately in terms 

of what you do. What you do will be different than 

anything we have done by extraordinarily large margins. 

So, the extent to which you produce data which is somehow 

(inaudible) benefit to the debate, that's fine. But the 

extent to which you rush towards managing the politics of 

the Hill, I think the more you drive yourself in the 

wrong direction. 

So, I encourage you all. I'm sure I'm 

preaching to a choir in some ways, but I encourage you 

all to approach this with extraordinary care, to 

experiment carefully with the methodologies that you 

believe are best, to apply the cost-benefit analysis in 

terms of ideas and to encourage a penetrating dialogue 

of, I suppose, the ideas that you come out -- that you 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

250

draw from and the conclusions that you draw from the raw 

data sets that you ultimately develop. If you develop 

your conclusions and your analyses without additional 

input along the way from, candidly, many of us around 

this table who live and breathe this industry day in and 

day out, then the study, again, could fall short of the 

methodology and the raw data could be fine, but the 

conclusions drawn from it could fall far short. 

For me, at the end of the day, it really goes 

back to this -- Karlene's observation. What you put in 

the bullet points is going to be the final word, and I 

don't care what you do to try to control that. So, at 

the end of the day, my concern is that the bullet points 

are right the first time through. 

MR. VANDER NAT: Thank you very much. I think 

we have come to the end of the day. In some ways, 

Stuart, you have anticipated my own concluding comments, 

so I won't repeat too many (inaudible). I've gotten to 

know Stuart very well and he's sent me a number of emails 

over time on different topics, and quite often, he's 

already anticipated what it is that I'm going to say. 

So, he has some perception here. 

One of the great benefits of this roundtable to 

us is that we've gotten to know many of you and we do 

intend to keep in contact with many of you because we do 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

251

need this kind of expertise. So, you will receive back 

from us emails and we appreciate your comments. Let me 

say, again, for some of you, if you feel that you want to 

submit something in writing to us, do so and we will be 

glad to consider it very carefully. I want you to know 

that in-house, the FTC has a lot of discussion about 

these points and there are similar in-house discussions 

and debates. And at this point, nothing is a settled 

matter. But we certainly appreciate it, and I think 

we've learned an awful lot from this day. I'm glad that 

this session is being transcribed because we're going to 

reread this transcription very carefully because there 

were wonderful nuggets all along the way which we 

couldn't fully appreciate at the moment, but upon 

rereading them, we definitely will benefit from them. 

So, I think we've come to the end of this 

session and I want to thank you all very much for having 

come to join us today. Thank you. 

UNIDENTIFIED MALE: Thank the FTC. 

(Applause.)

(The roundtable discussion was concluded.)
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

252

C E R T I F I C A T I O N O F T Y P I S T

MATTER NUMBER: P044804 

CASE TITLE: FACTA IMPLEMENTATION 

 TAPING DATE: JUNE 30, 2004 

 TRANSCRIPTION DATE: AUGUST 4, 2004 

 

I HEREBY CERTIFY that the transcript contained 

herein is a full and accurate transcript of the tapes 

transcribed by me on the above cause before the FEDERAL 

TRADE COMMISSION to the best of my knowledge and belief. 

DATED: AUGUST 4, 2004 

 

 ELIZABETH M. FARRELL 

C E R T I F I C A T I O N O F P R O O F R E A D E R

I HEREBY CERTIFY that I proofread the transcript for 

accuracy in spelling, hyphenation, punctuation and 

format. 
 For The Record, Inc. 

 Waldorf, Maryland 

 (301)870-8025 

253

 

 KATHY J. DE MENT