p.1 

OAR Box 1112 

Prepped by RyanDugan 

Document Number: 

34) ll-C-3b 

Docket Number: 

A-83-22 

Printed 4/16/2008 9:12:07AM Sheet 34 of 97 
P.2 

THE UNIVERSITY OF NORTH CAROLINA 

AT 

CHAPEL HILL 

u> 

^3- ^ 

. C-34 -

The L.L. Thurstone Psychometric Laboratory The University of North Carolina at Chapel Hill 

Davie Hall 013 A 

Chapel Hill, N.C. 27514 

September 25, 1984 

MEMORANDUM 

TO: John Haines, Chief 

Ambient Standards Branch, Section B 

FROM: Thomas S. Wallsten T&i/ 

RE: Development of hemoglobin protocol and selection of 

consultants 

The purpose of this memorandum is to describe the process by 

which we have developed the protocol for encoding probabilistic 

judgments about dose—response functions for hemoglobin decrement, 

and to indicate which experts we have consulted. 

OAQPS specified the adverse health effects for which 

probabilistic judgments about dose-response functions were 

required, including hemoglobin decrement. After becoming 

generally familiar with the Draft Criteria Document (CD) with Bob 

Kallam's and Jeff Cohen's help, we then read more carefully the 

relevant portions of the CD and discussed them at length with 

suitable experts, who in fact for the most part were the authors 

of those portions. We then prepared a first draft of the 

protocol sections 2-6 , which structured the problem, and went 

back with it to these experts as well as to others. The 

additional experts were drawn from the population of people whose 

judgments might be encoded, and whenever possible included 

members of the Risk Assessment Subcommittee. We worked with 

these experts, structuring the problem in the manner we would if 

we were going to encode their judgments, and revising the 

protocol accordingly. In this fashion, the protocol went through 

a number of drafts. Wherever it was clear that all the 

respondents viewed the problem in the same way, this way was 

specified in the protocol. Wherever they differed, or indicated 

that others might differ from them, we provided freedom to 

individualize the scenario for the specific expert, subject only 

to the constraints that the obtained judgments could map into tr-.-. 

rest of the model, and be comparable across experts. 

We worked primarilybwith five experts in developing th 

protocol: Drs. Anita Curran, Julian Chisolm, Lester Grant, b 
P.3 

Memorandum 

Mr. John Haines 

September 25, 1984 

page 2 

Hammond, and Paul Mushak. The protocol is structured in a way 

they consider reasonable and in which they would feel comfortable 

providing the desired probabilistic judgments.. 

Section 1 of the protocol is an introduction, indicating 

what we are attempting to do and why we are doing it. Sections 2 

through 5 precisely specify all the parameters of the problem, so 

that the dose-response functions are well defined. 

Section 6, entitled "Factors to Consider," is particularly 

important. The point of this section is to elicit discussion 

from the expert on relevant theories and data. This will help us 

to insure that we have the problem structured properly for him or 

her, cause the expert to consciously review his or her knowledge 

prior to providing probability judgments, and provide a basis for 

understanding discrepant judgments, should they occur. 

Section 8 is designed to elicit the expert's opinion about 

possible adverse health effects associated with elevated 

erythrocyte protoporphyrin levels. Material in this section will 

be discussed following the formal probability encoding. 

Sections 7 and 9 relate to the probability encoding, and 

derive from our prior work on encoding probability judgments 

about dose-response functions (Wallsten, Forsyth, and Budescu, 

1983). Section 7 provides suggestions on how to minimize various 

possible biases in probability encoding, and Section 9 is a 

rather detailed discussion of the actual probability encoding 

procedure. 

It is not our intention that the protocol simply be read by 

the expert, following which probability judgments are collected. 

Rather, the protocol is to be used by the analyst to guide 

discussion and interaction with the expert. Sections 1 through 8 

of the protocol will have been sent to the expert ahead of time 

for review. Then, both parties will have the protocol in front of 

them, and will actively work together, using it as a focus. 

cc: J. Cohen 

T. Feagans 

R. Whitfield