p.1 

OAR Box 1112 

Prepped by Ryan Dugan 

Document Number: 

13) ll-B-2k 

Docket Number: 

A-83-22 

Printed 4/16/2008 9:12:06 AM Sheet 13 of 97 
P.2 

The L.L. Thurstone Psychometric Laboratory 

THE UNIVERSITY OF NORTH CAROLINA 

AT 

CHAPEL HILL 

Januar y 8, 1985 

The University of North Carolina at Chapel Hill 

Davie Hall 013 A 

Chapel Hill. N.C. 27514 

MEMORANDUM 

TO: John Haines, Chief 

Ambient Standards Branch, Section B 

FROM: Thomas S. Wallstenl^Q 

RE: Development of IQ Protocol and selection of consultants 

The purpose of this memorandum is to describe the process by 

which we have developed the protocol for encoding probabilistic 

judgments about lead-induced IQ decrements, and to indicate which 

experts we have consulted. 

Our original intention was to develop a protocol analogous 

to the one for hemoglobin decrement, according to which we would 

encode experts' probabilistic judgments about dose-response functions for lead-induced IQ decrements. However, as will be described in this memorandum, the nature of the uncertainties is 

vastly different in the two cases, and an entirely different 

protocol is required. Instead of asking about dose-response 

functions, we will pose ideal hypothetical experiments and ask 

about their outcomes. The probabilistic responses to these questions will then be used to estimate probabilities about doseresponse functions and about mean dose-effect functions. 

In preparing to develop the protocol we initially read the 

draft Criteria Document (CD) and consulted with suitable experts, 

some of whom had aided in writing the relevent portions of the CD 

and all of whom had published research on neural, behavioral, or 

cognitive effects of lead. These experts were Drs. Vernon 

Benignus, Robert Bornshein, Kim Dietrich, David Otto, and Stephen 

Schroeder. 

We then developed a first draft of an IQ protocol. It was 

designed to elicit probabilistic judgments about dose-response 

curves for adverse effects defined in terms of IQ decrements. 

For example, an IQ decrement of 2 points or of 7 points might be 

specified, and the dose-response curve would indicate the percentage of the population who suffers at least that decrement at 

each lead dose level. The protocol was designed to elicit probabilistic judgments about this curve. 
P.3 

T 

Mr. John Haines January 8, 1985 Page 2 

This draft was discussed with most of the experts mentioned 

above as well as with a number of others. The consultants utilized at this point were Drs. Vernon Benignus, Claire Ernhart, 

Lester Grant, Lloyd Humphries, Lyle Jones, Herbert Needleman, 

David Otto, Stephen Schroeder, and Bernard Weiss. Their responses were generally favorable and very constructive. However, 

in discussing with these people how they would think about and 

respond to the questions in the protocol, two factors became 

apparent. 

First, although the concept of a dose-response curve here is 

well defined, it is not commonly used in this context. This is 

because IQ decrements cannot be observed directly, but can only 

be inferred statistically. Furthermore, the matter is so complex 

that the -statistics are not at all simple. The second factor was 

that in attempting to respond to the questions, the experts were 

envisioning hypothetical experiments, considering what the outcomes might be, and then extrapolating from that to the doseresponse functions of interest. 

It seemed to me that it would be far more consistent with 

the experts usual way of thinking and be easier for them to 

respond if we encoded directly their probabilistic judgments 

about outcomes of suitable hypothetical experiments. In addition, by properly designing these hypothetical experiments, we 

could eliminate the need to worry about some of the complexities 

that arise in actual research. We could then do the extrapolations mathematically that they were trying to do mentally. 

Accordingly, a new version of the protocol was developed to 

elicit probabilistic judgments about the outcomes of ideal hypothetical experiments. This protocol also indicated how the judgments would be used and the assumptions that would be incorporated in working with them. 

The new draft was discussed with Drs. Vernon Benignus, Anita 

Curan, Claire Ernhart, Lloyd Humphries, Lyle Jones, Marcy 

Lansman, Herbert Needleman, David Otto, and Stephen Schroeder. 

They found it much easier and more natural to respond to this 

version than to the previous one. However, various comments lead 

to a third draft of the protocol incorporating an improved ideal 

hypothetical experimental design. This draft was discussed with 

most of the people who had seen the previous one, and some very 

minor changes were made. 

The protocol was then used with two pilot subjects, one an 

EPA environmental health scientist and the other a psychology 

graduate student doing a related dissertation. The judgmental 

probability encodings went very well. However, in analyzing the 

judgments and discussing these analyses with Dr. Lyle Jones, we 

recently noticed that certain additional judgments were needed. 

The protocol is now structured in a way that our consultants 

consider reasonable and in which they would feel comfortable 

providing the desired probabilistic judgments. The additional 
P.4 

Mr. John Haines January 8, 1985 Page 3 

< <*o ••#> 

judgments will be obtained with a very minor change in the protocol. Nevertheless, the final version of the protocol will be 

discussed with a subset of the people who saw the previous one to 

assure that the change is acceptable. 

cc: J. Cohen 

T. Feagans 

R. Whitfield