# OAR Box 1112 Prepped by RyanDugan



### Document

Number: 

# 36 ) ll-C-3 d Docket Number: 

# A-83-2 2

Printed 4/16/20089:12:07

AM Sheet 36 of 97 p. 1P. 2

ARGONN

E NATIONA

LLABORATOR

Y J £- 6 " 3 ^ 9700 Sout hCas s Avenu e Argonne , Illinoi s 6043 9

ANL/EES-TM-XXX

ESTIMATIN

GTHE RISK

S O FLEAD-INDUCED HEMOGLOBI N DECREMENT S UNDE RCONDITION S O FUNCERTAINTY : A METHODOLOGY , PILO T JUDGMENTS , AN D EXAMPL

E CALCULATIONS * b y Ronal dG . Whitfiel d an d Thoma s S . Wallsten* * Energ y an dEnvironmenta l System s Divisio n Integrate dAssessment s an dPolic y Evaluatio n Grou p

Septembe r

24 , 198 4

wor k sponsore db y U.S . ENVIRONMENTA

LPROTECTIO N AGENC Y Offic e o f Ai r Qualit y Plannin g an dStandard s "Authorshi p i s share dequally . *LL . Thurston e Psychometri c Laboratory , Universit y o f Nort hCarolin aa t

Chape l Hill .P. 3

CONTENT S ABSTRAC T 1 1 INTRODUCTIO N 2 1. 1 Backgroun d2 1. 2 Motivatio n 3 1. 3 Outlin e 3 2 PROBABILIT Y ENCODIN G5 2. 1 Goal s o f

th e Encodin g Session s 5 2. 2 Metho d6 2.2. 1 Preparatio n 6 2.2. 2 Probabilit y Encodin g 7 2. 3 Subject s 1 0 2. 4 Result s 1 0 2.4. 1 Pilo t

Subjec t 1 1 0 2.4. 2 Pilo t

Subjec t 2 1 3 2. 5 Discussio n 1 6 3 MATHEMATICA LFORMULATIO N 1 7 3. 1 Criteri a1 7 3. 2 Th e Best-Fittin g an dEqual-Varianc e Normal-On-Log-Odd s Distribution s 1 7 3. 3 Combinin g Blood-Lea dDistribution s wit hProbabilit y Judgment s abou t Dose-Respons e Function s 2 1 4 ILLUSTRATIV E CALCULATION S 2 5 4. 1 Fittin g Normal-On-Log-Odd s Distribution s t o Judgment s 2 5 4.1. 1 Pilo t

Subjec t 1 2 5 4.1. 2 Pilo t

Subjec t 2 2 7 4.1. 3 Judgment s o f

Othe r Expert s 3 0 4. 2 Illustrativ e Ris k

Calculation s 3 3 5 COMMUNICATIN GTH E RIS K ASSESSMEN T RESULT S T O A

DECISIO N MAKE R3 6 6 OBSERVATION S AN D RECOMMENDATION S 3 8 7 REFERENCE S 4 0 APPENDI X A : Draft

Protoco l fo r

Encodin g Probabilit y Judgment s abou t Dose-Respons e Curve s fo r Lead-Induce d Hemoglobi n Decrement s 4 1 APPENDI X B : Detaile dMathematica l Formulatio n 5 6 APPENDI X C : Detaile dIllustrativ e Calculation s 6 6P. 4

FIGURE S 2. 1 Subjec t

l' s Probabilit y Judgment s abou t Fractio n o

f Populatio n of

U.S . Childre n Age d0- 3 Showin g Hemoglobi n Level s Les s Than o r Equal t o 1 1 g/d l a t

Eac hIndicate dBlood-Lea dLevel , L1 2 2. 2 Subjec t

2' s Probabilit y Judgment s abou t Fractio n of

Populatio n of

U.S . Childre n Age d0- 3 Showin g Hemoglobi n Level s Les s Than o r Equal t o 1 1 g/d l a t

Eac hIndicate dBlood-Lea dLevel , L1 5 3. 1 A Hypothetica l Famil y of Probabilisti c Judgment s abou t th e Fractio n of

a Well-Define dPopulatio n Tha t Wil l Suffer aWell-Define dAdvers e Healt h Effec t a t Eac hIndicate dBlood-Lea dLevel , L1 8 3. 2 A Hypothetica l Derive dProbabilit y Distributio n abou t th e Fractio n of

a Well-Define dPopulatio n Tha t Wil l Suffer aWell-Define dAdvers e Healt h Effect , Obtaine dby Combinin g Probabilit y Judgment s suc ha s in Fig . 3. 1 withaBlood-Lea dDistributio n suc ha s th e NHANES-I I 2 3 4. 1 Equal-Varianc e Normal-On-Log-Odd s CDF s fo r th e Probabilit y Judgment s o f Subjec t 1 Tha t Wer e Previousl y Show

n i n Fig . 2. 1 2 8 4. 2 Th e 0.05- , 0.50- , an d0.95-Cumulativ e Probabilit y Estimate s of Respons e Rat e Ra t

Variou s Blood-Lea dLevels , Base don th e Equal-Varianc e Normal-On-Log-Odd s Distribution s Fitte dt o th e Judgment s o f Subjec t 1 2 9 4. 3 Regresse dEqual-Varianc e Normal-On-Log-Odd s CDF s fo r th e Probabilit y Judgment s o

f Subjec t 2 Tha t Wer e Previousl y Shown i

n Fig . 2.2 , a s Wel l a s fo r Intermediat e Lea dLevel s 3 1 4. 4 Th e 0.05- , 0.50- , an d0.95-Cumulativ e Probabilit y Estimate s of Respons e Rat e Ra t Variou s Blood-Lea dLevel s LBase don th e Regresse dEqual

Varianc e Nor

rnal-On-Log-Odds Distribution s Fitte dt o th e Judgment s of Subjec t 2 3 2 4. 5 Derive dRis k

Distributio n Base don Equal-Varianc e Normal-On-Log-Odd s Distribution s Fi t

t o Subjec t

l' s an dSubjec t

2' s Judgments , Showin g in Eac hCas e aCumulativ e Probabilit y Ftha t Fractio n Rof

th e Populatio n of

U.S . Childre n Age d0- 3 Hav e Hemoglobi n Level s Les s Than o r

Equa l t o 1 1 g/d l 3 5 C. l Best-Fittin g Nor

mal-On-Log-Odd s CDF s an dth e Probabilit y Judgment s of Subjec t 1 Tha t Wer e Previousl y Shown i

n Fig . 2. 1 6 9 C. 2 Best-Fittin g Normal-On-Log-Odd s CDF s fo r th e Probabilit y Judgment s of Subjec t 2 Tha t Were Previousl y Shown in Fig . 2. 2 7 2 C. 3 A Complet e Se t

o f Six Modele dNormal-On-Log-Odd s CDF s fo r th e Respons e Rat e Variabl e Base don th e Paramete r Estimate s i

n Tabl e C. 5 7 5P. 5

FIGURES (Cont'd) C. 4 Derive dRis k

Distributio n Base don Best-Fittin g Normal-On-Log-Odd s Distribution s fo r

Subjec t l' s Judgments , Showin g th e Cumulativ e Probabilit y FTha t

Fractio n Ro

f th e Populatio n of

U.S . Childre n Age d0- 3 Hav e Hemoglobi n Level s Les s Than o r Equa l t o 1 1 g/d l 7 9 C. 5 Derive dRis k

Distributio n Base do n th e Best-Fittin g an dEqual-Varianc e Normal-On-Log-Odd s Distribution s fo r Subjec t l' s Judgments , Showin g th e Cumulativ e Probabilit y FTha t Fractio n Ro

f th e U.S . Childre n Age d0- 3 Hav e Hemoglobi n Level s Les s Than o r Equa l t o 1 1 g/d l 8 2 C. 6 Derive dRis k

Distributio n Base do n th e Equal-Varianc e an dRegresse d Best-Fittin g Normal-On-Log-Odd s Distribution s fo r Subjec t 2' s Judgments , Showin g th e Cumulativ e Probabilit y FTha t Fractio n Ro

f th e U.S . Childre n Age d0- 3 Hav e Hemoglobi n Level s Les s Than o r Equal t o 1 1 g/d l 8 3 C. 7 Probabilit y Densit y Function s fo r Derive dRis k

Distribution s Base don th e Equal-Varianc e Normal-On-Log-Odd s Distributio n Applie dt o th e Judgment s o

f Subject s 1 an d2 8 4

TABLE S 2. 1 Subjec t

l' s Probabilit y Judgments , F , abou t Fraction , R , of

Populatio n o f U.S . Childre n Age d0- 3 Showin g Hemoglobi n Level s Les s Than o r Equal t o 1 1 g/d l a t

Eac hIndicate dLea dLevel , L1 1 2. 2 Subjec t

2' s Probabilit y Judgments , F , abou t Fractio n R , of

Populatio n o f U.S . Childre n Age d0- 3 Showin g Hemoglobi n Level s Les s Than o r Equal t o 1 1 g/d l a t

Eac hIndicate dLea dLevel , L1 4 4. 1 R-Squar e Value s fo r th e Best-Fittin g an dEqual-Varianc e Normal-On-Log-Odd s Distribution s Fitte dt o th e Probabilit y Judgment s of Subject s 1 an d2 Conditiona l o n th e Indicate dBlood-LeadLevel s 2 6 4. 2 Regressio n r 6 Value s fo r th e Fi t of Best-Fittin g an dEqual-Varianc e Normal-On-Log-Odd s Distribution s t o Judgment s of Expert s Encode dby Wallsten , Forsyth , an dBudesc u (1983 ) 3 3 B. l Summary o

f Variable s Pertainin g t o th e Normal-On-Log-Odd s Functio n 60 C. l Normal-On-Log-Odd s Transformatio n of Probabilit y Judgment s i n Tabl e 2. 1 fo r Subjec t 1 67 C. 2 Paramete r

Estimates , Goodnes s o

f Fi t Statistics , an dMea

n Respons e Rate s fo r Best-Fittin g Normal-On-Log-Odd s Distribution s Fitte dt o th e Judgment s o

f Subjec t 1 6 8P. 6

TABLE S (Cont'd) C. 3 Paramete r

Estimates , Goodnes s o

f Fi t Statistics , an dMea

n Respons e Rate s fo r th e Equal-Varianc e Normal-On-Log-Odd s Distribution s Fitte dt o th e Judgment s o

f Subjec t 1 7 0 C. 4 Paramete r

Estimates , Goodnes s o

f Fi t Statistics , an dMea

n Respons e Rate s fo r th e Normal-On-Log-Odd s Distribution s Fitte dt o th e Judgment s of Subjec t 2 7 3 C. 5 Paramete r

Estimates , Goodnes s o

f Fi t Statistics , an dMea

n Respons e Rate s fo r th e Linearl y Regresse dNormal-On-Log-Odd s Distributions , Applie d t o th e Judgment s of

Subjec t 2 7 4 C. 6 Paramete r

Estimates , Goodnes s o

f Fi t Statistics , an dMea

n Respons e Rate s fo r Normal-On-Log-Odd s Function s wit hEqual Varianc e Fitte dt o th e Encode dJudgment s of

Subjec t 2 Throug hth e Us e of

Linea r Regressio n Technique s a s Describe di n th e Tex t

7 6 C. 7 Calculation s fo r

aMedia n Estimat e of

th e Fractio n o

f th e Populatio n Affected , Base don Best-Fittin g Normal-On-Log-Odd s Distribution s Fitte dt o th e Judgment s o f Subjec t 1 7 8

V IP. 7

ESTIMATIN

GTH E RISK S O FLEAD-INDUCED HEMOGLOBI N DECREMENT S UNDE RCONDITION S O FUNCERTAINTY : A METHODOLOGY , PILO T JUDGMENTS , AN D EXAMPL

E CALCULATION S b y Ronal dG . Whitfiel dan dThoma s S . Wallste n

ABSTRACT Thi s repor t

focuse s o n probabilisti c judgment s abou t dose

respons e relationship s o f lea dexposur e an dsubsequen t hemoglobi n decrements . I t

include s method s fo r

obtainin g th e judgments , repre sentin g the min suitabl e mathematica l form , an dcombinin g the mwit h ameasur e of

lea dexposur e t o produc e healt hris k estimates . Th e methodolog y i s easil y extende dt o othe r healt heffects , a s well . Judgmenta l dose-respons e function s fo r lead-induce dhemoglobi n decrement s wer e obtaine dfo r tw o pilo t

subject s a s par t

o f an exercis e t o tes t

th e viabilit y o

f aprotoco l t o obtai n suc hjudgment s fromhealt hexperts . Th e judgment s wer e represente din mathemati

ca l form(th e normal-on-log-odd s function ) t o facilitat e th e proces s o f combinin g the mwit hNHANES- H blood-lea ddat a( ameasur e of exposur e t o lead ) t o calculat e illustrativ e healt hris k estimate s fo r U.S . childre n age d0- 3 years . Attentio n i s als o give n i n th e repor t

t o way s of summarizin g an dpresentin g larg e amount s o f comple x ris k assessmen t

dat aan dcalculation s in aformunderstandabl e by an d usefu l t o decisio n makers . Th e pilo t

encoding s an dth e ensuin g calculation s hav e provide dvaluabl e experienc e an dpreparatio n fo r planne dencoding s o

f th e judgment s o

f expert s o n th e healt heffect s o f lead , an dth e developmen t

o f ris k estimate s partiall y base don thos e judgments .P. 8

1 INTRODUCTIO N 

### 1. 1 BACKGROUN D Th e Environmenta l Protectio n Agenc y (EPA ) i s charge dby th e Clea n Ai r Ac t wit hsettin g an drevisin g Nationa l Ambien t Ai r Qualit y Standard s (NAAQS ) fo r selecte d pollutants , a t

level s sufficien t

t o protec t

th e publi c healt hwitha n adequat e margi n of safety . Th e scientifi c base s fo r NAAQ S ar e presente dan dreviewe di n criteri adocu

ments . I n suppor t o f th e ongoin g revie w of

th e lea dNAAQS , EP

A i s preparin g ane w Air Quality Criteria for

Lead. I t

present s scientifi c evidenc e fro mwhic hth e mos t susceptibl e population s ca n b e determine dan dfro mwhic hvariou s advers e healt heffect s ca n b e identified . Th e criteri adocumen t summarize s an devaluate s th e availabl e clinical , epidemiological , an danima l o r toxicologica l laborator y evidenc e wit hregar dt o th e physiologica l an dadvers e healt heffect s of lead , an dtherefor e represent s th e mos t up t o dat e knowledg e on lea deffects . A s on e aspec t

of th e revie w process , EP A assesse s healt hrisk s by identifyin g th e mos t sensitiv e population s fo r eac hpollutant , an destimatin g probabilisticall y th e number s o

f peopl e i

n th e population s wh

o may suffe r

eac hof

variou s well-define d advers e healt heffect s attributabl e t o th e pollutant . I t i s believe dtha t

informatio n abou t th e healt hrisk s associate dwit hvariou s potentia l standard s wil l ai dth e EPA administrato r

i n selectin g tha t

standar dwhich , in hi s judgment , protect s th e publi c healt hwitha

n adequat e margi n of

safety . Becaus e th e ris k

estimate s tha t

EP A seek s ar e ofte n base din par t on dose

respons e relationship s an dlea dexposur e estimate s whic har e uncertain , i t i s necessar y t o mak e probabilit y judgment s abou t th e relevan t

dose-respons e function s base do n th e availabl e evidenc e an dt o probabilisticall y estimat e lea dexposur e unde r alternativ e lea d NAAQS . Obtainin g th e healt hris k estimate s the n involve s combinin g th e dose-respons e an dexposur e probabilit y estimates .P. 9

1. 2 MOTIVATION 5 Thi s repor t

i s primaril y concerne dwithtw o of th e thre e component s of healt h ris k estimates : representin g probabilisti c judgment s abou t dose-respons e relationships , an dcombinin g thes e wit hameasur e of exposur e t o lea dt o produc e ris k estimates . A t thi s time , w e d

o no t trea t th e proble mof

probabilisticall y estimatin g exposur e t o lea d unde r alternativ e lea dNAAQS , bu t simpl y tak e th e NHANES-I I dat at o represen t

curren t blood-lea ddistributions . Probabilit y judgment s abou t dose-respons e function s fo r lead-induce dhemoglobi n decrement s hav e bee n obtaine dfro mtw o pilo t subject s a s par t

o f an exercis e t o tes t

th e viabilit y of

aprotoco l t o obtai n suc hjudgment s fro mrecognize dhealt hexpert s som e tim e i n th e nea r

future . Thi s pape r

present s th e metho dfo r obtainin g th e judgment s a s wel l a s th e pilo t

judgment s themselves , an dillustrate s how (1 ) suc hjudgment s ca n b e represente di

n mathematica l form ; (2 ) the y ca n b e combine dwithexposur e informatio n t o generat e ris k

estimates ; an d(3 ) th e ris k informatio n ca n b e summarize din aform understandabl e b

y adecisio n maker . 1. 3 OUTLIN E Sectio n 2 present s th e pilo t probabilit y encodings . I t include s adescriptio n of th e metho db y whic hth e judgment s wer e obtained , th e result s fo r bot hsubjects , an da discussio n o

f thos e result s indicatin g wh y i t i s desirabl e t o represen t the m mathematically . Sectio n 3 summarize s th e mathematic s employe dt o represen t th e judgment s an d t o calculat e ris k

estimates . A ful l developmen t of th e mathematic s i s provide din an appendi x t o thi s report . Sectio n 4 applie s th e tool s presente di n th e previou s sectio n t o th e judgment s of th e pilo t subject s an dof

expert s who ha dparticipate din th e stud y b y Wallsten , Forsyth , an dBudesc u (1983) . I t present s th e fit s o f equal-varianc e normal-on-log-odd sP.1 0

distribution s t o th e judgments , an dillustrativ e ris k calculation s base do n combinin g thes e judgmenta l distribution s wit hNHANES-II blood-lea ddistributions . Detaile ddiscussio n of alternativ e representation s o f th e judgment s i s containe din a n appendix . Sectio n 5 illustrate s how suc hris k estimate s ma

y b e communicate dt o adecisio n maker , an dSectio n 6 contain s observation s abou t th e pilo t

encodin g sessions , th e mathematica l representations , th e subsequen t ris k estimates , an dlesson s learne dwhic h ar e relevan t

fo r futur e encodin g efforts .P.1 1

2 PROBABILIT Y ENCODIN G 2. 1 GOAL S O F THE ENCODIN G SESSION S Th e primar y goa l of

th e encodin g session s i s t o obtai n experts ' probabilisti c judgment s abou t

dose-respons e function s fo r lead-induce dhemoglobi n decrements . Specifically , th e expert s wil l b e aide din encodin g fo r eac hof variou s blood-lea dlevel s thei r subjectiv e probabilitie s abou t th e fractio n of

apopulatio n withhemoglobi n level s les s tha n o r

equa l t o aspecifie dvalue . Becaus e th e shap e an dlocatio n of an y dose-respons e functio n depend s o n numerou s populatio n an dexposur e conditio n parameter s a s wel l a s o n th e exac t

definitio n of th e advers e healt heffect , i t i s necessar y firs t

t o specif

y thes e factors . Suc h specificatio n precisel y define s th e dose-respons e function , an dassure s tha t an y difference s betwee n expert s in probabilit y judgment s i s du e t o tru e difference s of opinio n an dnot merel y t o consideratio n of differen t conditions . A secondar y goa l o

f th e encodin g session s i s t o discus s th e literatur e o n lead

induce dhemoglobi n decrements . Thi s i s t o b e don e prio r t o th e actua l probabilit y encoding , an dserve s thre e purposes . First , i t aid s th e exper t

in reviewin g th e literatur e i n hi s o r he r own mindbefor e beginnin g th e forma l judgmen t process . Second , i t

provide s abasi s fo r

possibl y understandin g conflictin g judgment s amon g experts , shoul dthe y arise . Third , i t

provide s an opportunit y fo r th e exper t

t o mak e an y point s h e o r sh e feel s ma y b e usefu l t o th e user s of th e subsequen t

ris k assessments . Th e actua l probabilit y encodin g wit hrecognize dhealt hexpert s i s planne dt o tak e tw o session s pe r

expert , wit htw o t o fou r week s intervening . Th e firs t sessio n wil l las t approximatel y eigh t

hour s an dth e secon dabou t fou r hours . I t

i s anticipate dtha t

roughl y one-thir dt o one-hal f

o f th e firs t sessio n wil l b e devote dt o preparatio n an ddiscussion , a s indicate dabove , withth e remainde r spen t on th e actua l probabilit y encoding . Th e preparator y materia l wil l b e reviewe di n th e secon dsession , bu t mos t o f th e tim e wil l b eP.1 2

give n t o probabilit y encoding . Base don th e earlie r wor k b y Wallsten , Forsyth , an d Budesc u (1983) , th e probabilit y judgment s obtaine din th e secon dsessio n wil l b e th e bette r one s t o us e in calculatin g ris k estimates . However , fo r th e presen t

pilo t stud y i t wa s deeme dsufficien t

t o wor k wit heac hexper t

i n asingl e sessio n o

f four hours . 2. 2 METHO

D 2.2. 1 Preparatio n A complet e protoco l fo r encodin g th e probabilit y judgment s i s attache da s Appendi x A t o thi s report . Thi s protoco l wa s sen t t o th e pilo t

subject s abou t

on e wee k prio r t o th e schedule dsession s s o tha t the y coul dfamiliariz e themselve s wit hit . Section s 1-

7 o f th e protoco l discus s th e hemoglobi n leve l t o b e considere dadvers e fo r regulator y purposes , th e populatio n a t risk , th e populatio n exposur e conditions , th e populatio n physiologica l conditions , feature s of th e relevan t

literature , an dfactor s t o b e awar e o f when makin g probabilit y judgments . (Sectio n 8 of th e protoco l specifie s how th e probabilit y judgment s ar e t o b e encoded ; th e subject s wer e aske dt o ignor e tha t sectio n prio r

t o th e session . Thi s sectio n wil l no t b e sen t ou t t o th e healt hexperts. ) Approximatel y one-and-a-hal f t o tw o hour s wa s spen t wit heac hsubjec t reviewin g th e materia l in Section s 1 throug h7 of th e protocol , reviewin g th e literature , an ddiscussin g procedure s fo r encodin g judgmenta l probabilities . Th e stipulatio n o f factor s in thos e section s whic hdefin e th e dose-respons e function s abou t whic h probabilit y judgment s wer e t o b e elicite dar e a s follows . Tw o hemoglobi n level s wer e specifie da s adverse , 9. 5 an d1 1 g/dl , bu t i n thi s pilo t stud y judgment s wer e obtaine donl y wit hrespec t t o th e functio n define dfo r hemoglobin s les s tha n o r

equa l t o 1 1 g/dl . Th e populatio n a t ris k i s al l U.S . childre n fro mconceptio n throug hthei r

sevent hbirthdays . Th e subject s wer e give n th e optio n o f judgin g separat e dose-respons e curve s fo r childre n age d0- 3 an d4-7 , i f the y though t

tha t appropriate .P.1 3

The y wer e furthe r

t o assum e tha t

th e blood-lea dleve l unde r consideratio n fo r

agive n judgmen t wa s in equilibriu ma s aresul t of asufficientl y long-ter mconstan t lea d exposur e withou t

gros s excursion s above o r belo w th e state dlevel . The y wer e als o t o assum e tha t

iron , zinc , copper , vitamin s A , C , D , an dE , calcium , phosphorus , an d magnesiu mwer e distribute di n th e populatio n a s the y believ e thes e mineral s an dvitamin s in fac t are . 2.2. 2 Probabilit y Encodin g Probabilit y encodin g commence dafte r th e relevan t

literatur e an dth e factor s definin g th e dose-respons e functio n ha dbee n thoroughl y discusse dan dth e exper t ha d indicate dwhethe r

h e though t

childre n age d0- 3 an d4- 7 wer e describe dby th e sam e o r differen t functions . I

n thi s pilo t

stud y onl y asingl e functio n wa s encode dpe r expert , bu t in th e actua l session s judgment s wil l b e obtaine dwit hrespec t t o tw o o r

fou r function s (on e eac hfo r

9. 5 an d1 1 g/d l eithe r fo r al l childre n t o thei r

sevent hbirthday s o r separatel y fo r

th e tw o ag e groups) . Probabilit y judgment s wer e encode dregardin g possibl e populatio n respons e rate s a t blood-lea dlevel s o

f 5 , 15 , 25 , 35 , 45 , an d5 5 ug/dl . Onl y pilo t subjec t 1 als o provide d judgment s regardin g possibl e blood-lea dlevel s supportin g respons e rate s of 0.2 0 an d0.65 . T o determin e th e usefu l point s in th e dose-respons e spac e fo r encodin g probabilities , eac hsubjec t wa s firs t

aske dt o suppl y broa d"99 % confidenc e interval " range s fo r

possibl e respons e rate s a t eac hof th e si x blood-lea dlevels . Th e analys t aide d th e exper t

i n determinin g thes e range s by posin g suitabl e question s (e.g. , followin g initia l specificatio n o

f arang e th e analys t

woul das k

whethe r th e exper t

coul dconstruc t

a plausibl e explanatio n i

f h e wer e t o lear n on e day tha t th e tru e respons e rat e wa s outsid e tha t range) . Th e range s fo r eac hblood-lea dleve l wer e the n divide dint o fiv e equall y space dinterval s fo r

detaile dencodin g a t

si x points . Fo r

example , i

f th e respons e rat e rang e for

aparticula r

blood-lea dleve l wa s 0.2 0 t o 0.70 , subsequen t encoding s a t

tha t lea dleve l woul doccu r

fo r respons e rate s o

f 0.20 , 0.30 , 0.40 , 0.50 , 0.60 , an d0.70 .P. M 8 Fo r pilo t subjec t 1 , probabilitie s wer e encode donl y a t th e point s predetermine d i n th e manne r

jus t described . As wil l becom e apparen t whe n th e result s ar e presented , a more dynami c approac hwa s required . Thus , fo r pilo t subjec t 2 additiona l point s outsid e th e specifie drange s wer e encode dwhe n tha t becam e necessary . Pilo t subjec t 1 wa s encode dby asingl e analys t (T.S.W.) , who wa s sufficientl y bus y durin g th e sessio n tha t h e di dno t notic e th e nee dfo r

dynami c adjustment . Pilo t subjec t 2 wa s encode db y tw o analyst s (T.S.W . an dR.G.W.) , an ddynami c adjustmen t wa s easil y accomplished . Withi n ablood-lea dlevel , respons e rate s wer e presente dfo r encodin g in a n unsystemati c orde r

(bu t not selecte dwit hth e ai do f arandomizatio n device) . Encodin g wa s accomplishe dwit hth e us e o f aprobabilit y wheel . Thi s i s awhee l radiall y divide d int o tw o sector s o

f adjustabl e relativ e areas , on e orang e an don e blue . Th e whee l ca n b e spun s o tha t

i t randoml y stop s witheithe r secto r unde r apointer . Fo r

aparticula r respons e rat e Ra t

agive n blood-lea dleve l L , th e whee l wa s se t a t som e relativ e are ao

f blue , an dth e exper t

wa s aske dt o conside r

th e followin g question : "D o you conside r

i t mor e probabl e tha t

th e tru e respons e rat e a t bloo d lea dLi s les s tha n Ro r tha t on arando mspi n th e whee l woul dsto p wit hth e pointe r

on blue? " Thus , th e exper t

wa s askedt o weig htw o probabilitie s an ddecid e whic hi n hi s min dwa s greater . Dependin g on hi s answer , th e relativ e are ao f blu e wa s increase do r

decreased , an dth e questio n wa s pose dagain . Thi s procedur e continue dunti l awhee l settin g was reache dsuc htha t

th e exper t

considere di t

equall y likel y tha t

th e whee l woul drandoml y lan do n blu e a s tha t

th e tru e respons e rat e i s les s tha n Rfo r blood-lea dL . Th e relativ e are aof blu e wa s the n take n provisionall y a s th e judge dcumulativ e probabilit y Fn/, . Th e procedur e wa s repeate dfo r anothe r respons e rat e unti l a n entir e cumulativ e probabilit y distributio n ove r

Rfo r 0. 0 < R< 1.0 ha dbee n encode dfo r blood-lea dleve l L . At tha t point , th e distributio n wa s graphed , show n t o th e expert , an ddiscusse d withhim . Inconsistencies , i f any , wer e pointe dout

an dresolved , generall y wit hth e ai d o f th e wheel . A

n inconsistenc y occur s when , fo r an y pai r o f respons e rate s R

j an dR: ,P.1 5

R j > R- , bu t

FR./ T < FR./L * ^ n inconsistenc y quit e simpl y a s adi p i n th e grap hwhic h shoul dotherwis e ris e monotonicall y a s respons e rat e goe s fro m0. 0 t o 1.0 (Se e Wallsten , Forsyth , an dBudescu , 1983 , fo r details. ) Implication s of

th e encode ddistributio n wer e als o discussed . A s jus t on e example , i

f fo r R j > Rj , F R,r

- 0.2 5 an dF^/ L= 0.75 , thi s implie s tha t th e exper t consider s i t

equall y likel y tha t

th e tru e respons e rat e fall s inside th e (Rj , Rj ) interva l a s outsid e it . I

f th e exper t

disagree dwiththi s o r an y o f variou s othe r

implications , adjustment s wer e made , agai n generall y withth e ai do f th e wheel . Whe

n th e exper t

wa s satisfied , an dconsidere dth e encode ddistributio n t o represen t hi s probabilisti c judgmen t anothe r blood-lea dleve l wa s selecte dan dth e proces s wa s repeated . Whe n distribution s wer e encode dfo r al l selecte dblood-lea dlevels , thes e wer e plotte do n asingl e grap han ddiscusse dwit hth e expert . Inconsistencie s betwee n distributions , i

f any , wer e resolved , agai n withth e wheel . A n inconsistenc y betwee n distribution s occur s wheneve r tw o distribution s cross , i.e. , fo r bloo dlead s L

j an dL

- suc h tha t L j > Lj , an dfo r

respons e rate s R

j an dRj , suc htha t

R j > R- , F R/L < > F ^ « . bu t F R-/L

- * ^R/L

- ^ se e Wailsten » Forsyth , an dBudescu , 1983) . Whe n th e exper t

wa s satisfie dtha t

th e entir e se t

o f distribution s represente dhi s probabilisti c judgments , th e encodin g wa s completed . Additiona l consistenc y check s wer e carrie dou t wit hpilo t

subjec t 1 . H e als o encode dsubjectiv e probabilit y distribution s ove r blood-lea dlevel s a t

tw o respons e rates , i n amanne r

analogou s t o tha t

described , resultin g i

n cumulativ e distribution s Fj/^ . A s Feagan s an dBille r

(1981 ) hav e proved , consistenc y require s tha t FJWT + F , ,-o - 1 . Inconsistencie s wit hthi s requiremen t wer e pointe dou t

t o th e subject , an dh e wa s aide d i n resolvin g the mt o hi s satisfaction . Fo r pilo t subjec t 1 , response-rat e range s wer e obtained , an dprobabilit y judgment s ove r

respons e rate s wer e subsequentl y encoded , fo r blood-lea dlevel s i n th e orde r 15 , 55 , 35 , 5 , 45 , an d2 5 ug/dl . I n addition , followin g th e 5 5 ug/d l condition ,P.1 6 1 0 probabilit y judgment s ove r blood-lea dlevel s wer e encode dfor.respons e rate s 0.6 5 an d 0.20 , in tha t

order . Fo r pilo t subjec t 2 , respons e rat e range s wer e obtaine dfo r blood-lea dlevel s in th e orde r 55 , 15 , 25 , 45 , 5 , an d3 5 ug/dl . Du e t o tim e limitations , probabilitie s wer e encode dove r

respons e rate s onl y fo r

blood-lea dlevel s 55 , 15 , an d3 5 ug/dl , in tha t order . 2. 3 SUBJECTS Th e tw o pilo t

subject s ar e EP

A environmenta l healt hscientist s involve dwit h analysi s an dinterpretatio n o f scientifi c informatio n o n lead , who ar e familia r wit hth e literatur e on lead' s hematopoieti c effects . 2. 4 RESULT

S 2.4. 1 Pilo tSubject1 I n th e opinio n o

f subjec t 1 , childre n age d0- 3 an d4- 7 hav e differen t dose

respons e curves , primaril y du e t o difference s in iro n status . Judgment s wer e encode d onl y fo r th e functio n pertainin g t o th e younge r ag e group . Th e probabilit y judgment s ar e presente din Tabl e 2. 1 an ddepicte din Fig . 2.1 . The y may b e considere dcumulativ e judgmenta l probabilit y distributio n function s (CDFs) . Tha t

is , fo r eac hblood-lea dleve l L , th e ordinat e i n Fig . 2. 1 an dth e colum n labele dFin Tabl e 2. 1 represen t th e judge dprobabilit y tha t th e tru e respons e rat e Rrr . i s les s tha n th e rat e Rshow n a s th e abciss ao f th e figur e an di

n th e labele dcolum n of th e table . Not e in Fig . 2. 1 th e differenc e i n CDF s fo r

L= 5 an d1 5 ug/d l versu s th e CDF s fo r th e remainin g lea dlevels . Thi s subjec t i s rathe r

certai n tha t

populatio n respons e rate s ar e lo w fo r

th e tw o lowes t

lea dlevel s employed . He consider s th e respons e rate s t o b e highe r

a t th e highe r

lea dlevels , but

i s mor e uncertai n a s t o thei r actua l values , a sTABL E 2. 1 Subjec tl' s Probabilit y Judgments , F , abou tFraction , R , o f Populatio n o f

U.S . Childre n Age d 0- 3 Showin g Hemoglobi n Level s Les s Tha n o rEqua l t o 1 1 g/d l a tEac h Indicate d Lea d Level , L Blood-Lea dLevel s L A= 5 u/d l L 1 = 1 5 u/d l L 6 = 2 5 p/d l L 3 = 3 5 g/d l L 5 = 4 5 p/d l L 2 =5 5 u/d l R a F b RFRFRFR FR F 0.0 0 0.0 0 0.0 0 0.0 0 0. 2 0.2 7 0.1 0 0.0 7 0.3 5 0.2 0 0.3 0 0.0 3 0.0 2 0.1 5 0.0 2 0.0 5 0. 3 0.5 2 0.2 6 0.2 8 0.4 8 0.4 8 0.4 4 0.2 4 0.0 4 0.6 5 0.0 4 0.3 8 0. 4 0.8 7 0.4 2 0.4 6 0.6 1 0.7 1 0.5 8 0.4 5 0.0 6 0.7 9 0.0 6 0.5 5 0. 5 0.9 7 0.5 8 0.8 0 0.7 4 0.9 8 0.7 2 0.7 2 0.0 8 0.8 9 0.0 8 0.7 0 0. 6 1.0 0 0.7 4 1.0 0 0.8 7 1.0 0 0.8 6 0.9 9 0.1 0 0.9 8 0.1 0 0.8 3 0. 7 1.0 0 1.0 0 1.0 0 1.0 0 1.0 0 1.0 0 1.0 0

a R i s populatio nrespons e rat e a s a decima l fraction ; Rwa s presente d a s a percentag e to th

e subjec t t o avoi d confusionwit hprobability . ^ P i s th

e judgmenta l probabilit y tha t th

e respons e rat e i s les s tha n o r equa l t o R .P.1 8 1 2

X i t o X. o u C u C U > e m.

Populatio n Respons e Rate , Expresse dAs Fractio n o f

Populatio n Affected , R

FIGUR E 2. 1 Subjec t

l' s Probabilit y Judgment s abou t Fractio n of

Populatio n of

U.S . Childre n Age d0- 3 Showin g Hemoglobi n Level s Les s Tha n o r Equa l t o 1 1 g/d l a t Eac hIndicate dBlood-Lea dLevel , LP.1 9 1 3 indicate dby th e mor e graduall y risin g CDFs . Thi s patter n suggest s tha t th e subjec t believe s ther e t o b e athreshol dbetwee n 1 5 an d2 5 ug/dl , belo w whic hlead-induce d hemoglobi n reductio n i s minimal . Upon seein g thes e distributions , h e independentl y remarke dtha t

the y wer e consisten t

wit hhi s belie f

in suc hathreshold . Not e als o in Fig . 2. 1 tha t th e encode dCDF s fo r

L= 2 5 an d4 5 ug/d l d o no t begi n sufficientl y clos e t o 0.0 . Encodin g too k plac e fo r

th e respons e rat e range s establishe d initiall y fo r

thos e tw o lea dlevels , but despit e al l attempt s th e range s wer e no t extende d lo w enough . Thi s i s th e proble mallude dt o earlier , in whic hdynami c adjustmen t of th e point s t o b e encode dwa s required , but overlooked . Th e entir e se t o f CDF s i s internall y consisten t

i n tha t non e o f the mdi p an dther e ar e n o crossovers . However , i f th e CDF s fo r L= 2 5 an d4 5 ug/d l ha dbee n encode da t thei r lowe r ranges , crossover s may hav e occurre dan dth e subjec t woul dhav e bee n aide di n eliminatin g them . Th e encode djudgment s fo r blood-lea dlevel s conditiona l on eac ho f respons e rate s 0.2 0 an d0.6 5 wil l not b e reproduce dhere . The y wer e intende donl y a s furthe r consistenc y check s o

n th e judgment s jus t

presented . I t

turne dou t

tha t wheneve r

thes e judgment s conflicte dwithth e others , i t wa s thes e tha t

th e subjec t

changed . 2.4. 2 Pilo tSubjec t2 Thi s subjec t

als o considere dther e t o b e differen t dose-respons e curve s fo r th e tw o ag e groups . Thus , a s wit hth e othe r subject , judgment s wer e encode donl y fo r a dose-respons e functio n fo r childre n age d0-3 . Thes e judgment s ar e show n in Tabl e 2. 2 an dFig . 2.2 . Figur e 2. 2 give s no suggestio n tha t thi s subjec t

believe s ther e t o b e athreshold , no r di dth e existenc e o

f on e com e up in discussion . Th e thre e CDF s wer e encode dove r thei r ful l range s an dar e internall y consistent .P.2 0 14 TABL E 2. 2 Subjec t2' s Probability Judgments , F , aboutFraction , R

, o f Populatio n o f

U.S . Children Age d 0- 3 Showing Hemoglobi n Level s Les s Tha n o rEqua

l t o 1 1 g/d l a tEac

h Indicate d Lead Level, L

Blood-Lea

d Level s L 2 = 1 5 u/d l L 3 = 3 5 u/d l L 1 = 5 5 u/d l

R F R F 0.1 0 0.0 0 0.1 7 0.0 0 0.2 1 0.1 8 0.2 0 0.0 3 0.3 2 0.4 5 0.3 3 0.3 2 0.4 3 0.6 8 0.4 6 0.5 0 0.5 4 0.9 0 0.5 9 0.7 0 0.6 5 0.9 7 0.7 7 0.9 2

0.8 5 0.9 5 Ri s populatio n respons e rat e a s adecima l fraction ; Rwa s presente da s apercentag e t o th e subjec t

t o avoi dconfusio n wit h probability . Fi s th e judgmenta l probabilit y tha t th e respons e rat e i s les s tha n o r equa l t o R . 2. 5 DISCUSSIO N First , i t i s of interes t t o compar e judgment s fro mth e tw o subjects . A s alread y noted , subjec t

1 believe s ther e t o b e athreshol dfo r lead-induce dhemoglobi n reduction , wherea s subjec t

2 gav e no evidenc e of suc habelief . Consisten t wit hthi s difference , subjec t 1 believe s populatio n respons e rate s t o b e lowe r a t 1 5 ug/d l tha n doe s subjec t 2 . Th e probabilit y judgment s o f th e tw o subject s cove r overlappin g respons e rat e range s fo r L= 3 5 an d5 5 ug/dl , however .

R a 0.0 5 0.1 4 0.2 3 0.3 0 0.4 0 0.5 0 F b

0.0 0 0.1 1 0.5 5 0.9 0 0.9 5 0.9 9P.2 1

1 5



### 5 0, 6 

### - Xi m X i o cu > 3 E 3 U 

### S 0. 4 

### -

Populatio n Respons e Rate , Expresse dA s Fractio n o f

Populatio n Affected , R

FIGUR E 2. 2 Subjec t2' s Probability Judgments abou tFractio n o f Populatio n o f

U.S . Children Age d 0- 3 Showing Hemoglobi n Level s Les s Tha n o rEqua

l t o 1 1 g/d l a tEach Indicate d Blood-Lea d Level , LP.2 2 1 6 But not e als o tha t

subjec t 2' s CDF s ar e shallowe r tha n subjec t

l's , i.e. , the y spa n abroade r range . Thus , th e judgment s o f subjec t 2 represen t

mor e uncertaint y abou t th e dose-respons e functio n tha n d o thos e o f subjec t 1 . Beyondcomparison s o

f thi s sort , however , th e judgment s o f th e tw o subject s ar e t o b e use dfo r

th e purpos e o

f calculatin g ris k

estimates . Thi s wil l b e don e b y usin g th e probabilisti c judgment s t o generat e ase t o f possibl e dose-respons e functions . Thes e wil l b e isoprobabilit y dose-respons e function s i n th e sens e tha t

th e probabilit y i s constan t tha t agive n leve l o

f Lwil l generat e arespons e rat e les s tha n o r equa l t o R . Thi s se t

of function s ca n the n b e combine dwit hsuitabl e estimate s of blood-lea ddistribution s t o yiel dase t

o f probabilisti c estimate s fo r numbe r of peopl e affected . Sinc e numerica l judgment s ar e b e obtaine donl y a t discret e point s fo r

agive n lea dlevel , respons e rate s correspondin g t o particula r probabilitie s wil l b e obtaine di n genera l by interpolatin g betwee n encode dpoints . Thi s i s t o b e don e by fittin g asuitabl e probabilit y distributio n t o th e encode dvalues . Th e criteri afo r

selectin g aparticula r distributio n ar e discusse din th e nex t section , a s ar e th e technique s fo r fittin g adistributio n t o a n individual' s probabilit y judgments . Th e technique s use dt o interpolat e betwee n distribution s a t particula r

blood

level s t o obtai n isoprobabilit y dose-respons e relationship s ar e als o discussed . Sectio n 4 the n evaluate s thes e technique s b y applyin g the mt o th e judgment s of eac hsubjec t

an d derivin g ris k

estimates . Sectio n 5 indicate s ho w suc hestimate s ca n b e communicate d understandabl y t o adecisio n maker , an dSectio n 6 summarize s wha t ha s bee n learne d fro mth e pilo t

encodin g session s an dth e subsequen t analysis .P.2 3 

### 1 7 3 MATHEMATICA

L FORMULATIO

N 3. 1 CRITERI

A Th e firs t requiremen t

i s t o specif

y afamil y of

distribution s whos e shape , o r rang e of shapes , matc hthos e o f th e judgments . I

n addition , th e distribution s mus t b e one s fo r whic hparameter s ca n b e estimate dsuc htha t th e judgment s ar e fitte da s closel y a s possible , in awell-define dsense , an dsuc htha t th e fi t ca n b e assessed . Finally , th e fitte ddistribution s fo r a n exper t mus t no t demonstrat e interna l inconsistencies . Specifically , thi s mean s tha t th e fitte ddistribution s mus t no t cros s a t an y point . Th e famil y of

equal-varianc e normal-on-log-odd s distribution s meet s al l thes e requirements , an di s presente dnext . Followin g that , ametho di s presente dfo r derivin g ris k estimate s b y combinin g aspecifie dblood-lea ddistributio n wit ha n expert' s probabilit y judgment s abou t dose-respons e relationships , wher e th e latte r i s represente d by normal-on-log-odd s distributions . Th e ful l mathematica l formulatio n i s give n in Appendi x B , an donl y asummar y i s presente di n thi s section . 3. 2 TH E BEST-FITTIN

G AN D EQUAL-VARIANC

E NORMAL-ON-LOG-ODD S DISTRIBUTIO

N FUNCTIONS Th e uncertai n quantitie s o f interes t in thi s pape r

ar e populatio n respons e rates , denote dR , a t

eac hof severa l blood-lea dlevels , denote dL . Throug hencodin g session s wit hrecognize dexperts , judgment s wil l b e elicite drepresentin g acumulativ e densit y functio n (CDF ) fo r

apopulatio n response . rat e Rgive n ablood-lea dleve l L . Take n together , thes e judgment s wil l provid e afamil y of probabilisti c relationship s a s idealize d in Fig . 3.1 , an dalread y presente dmor e realisticall y i n Figs . 2. 1 an d2.2 . Eac hcurv e in Fig . 3. 1 represent s th e judgmenta l probabilit y tha t th e populatio n respons e rat e ( a fractio n betwee n 0 an d1 ) wil l b e les s tha n o r equa l t o aparticula r

valu e R , give n a blood-lea dleve l of

L , fo r awell-define dpopulatio n of interest , unde r

well-define d exposur e conditions .P.2 4 1 8

X. ra X ) o u o - > 3 E 3 C J l . U



### 0 . 8

0. 6

0 . 4



### 0 . 2

0 . 0 

### 1 I

* J / 

# 1

« o / * J / 

# 1 /

" / 

# ' 1/

" / 

# ' l /

« o / » /



# 1 • 1 /

« o 7



### I 

# ' 1 /

1 ' !/ '



### I 1

. —

—

—111



### 0. 0 

### O. S 0. 4 

### 0 . 6 0. 8 

### 1. 0

Populatio n Respons e Rate , Expresse d As Fractio n o f

Populatio n Affected , R

FIGUR E 3. 1 A Hypothetica l Famil y o f Probabilisti c Judgment s abou tth e Fractio n o f aWell-Define d Populatio n ThatWil l Suffe raWell-Define d Advers e Healt h Effec ta t Eac h Indicate d Blood-Lea d Level , LP.2 5 1 9 A s alread y discussed , i t wil l generall y b e necessar y t o fi t adistributio n t o th e assesse dpoint s i n orde r t o perfor mris k

calculations . Becaus e th e probabilisti c judgment s abou t

th e dose-respons e relationship s appea r generall y t o b e "S-shaped " ove r th e close d[0,1 ] interval , a s demonstrate di n Figs . 2. 1 an d2.2 , a s wel l a s b y th e judgment s of al l th e healt hexpert s encode db y Wallsten , Forsyth , an dBudesc u (1983) , i t ca n b e difficul t t o represen t

the mwithsimpl e mathematica l functions . On e particularl y usefu l functio n tha t

i s relativel y eas y t o wor k wit hi s th e normal-on-log-odd s distributio n function . Th e normal-on-log-odd s distributio n i s obtaine db y fittin g anorma l distributio n t o th e natura l lo g of

th e odd s implie db y th e populatio n respons e rate s R . Thus , th e followin g relationship s ar e defined : X = JL, 0 < X < » Y = ln(X) , - « < Y < - wher e X i s th e odd s variabl e an dY i s th e log-odd s variable . I t i s assume dtha t

Y i s normall y distribute dwithmean u an dvarianc e o . Th e degre e t o whic hth e assumptio n i s appropriat e ca n b e teste dwit heac hse t of judgments . I

f Y i s normall y distributed , the n X i s lognormall y distribute dan dth e distributio n of interes t induce don Ri s calle da normal-on-log-odd s distribution . Althoug hth e normal-on-log-odd s distributio n i s no t expressabl e in aclose dform , al l probabilitie s of interes t ar e easil y derive dfro mth e norma l distributio n ove r Y , a s discusse din Appendi x B . A separat e normal-on-log-odd s distributio n functio n ca n b e fi t t o th e elicite d CDF s (probabilit y judgment s ove r respons e rat e R

) a t eac hblood-lea dleve l LJ , j=l , ... , m . I n othe r words , afunctio n i s fi t t o eac hse t

of point s depicte din Fig . 3.1 . A uniqu e normal-on-log-odd s distributio n i s determine da t eac hlea dleve l b y derivin g least-square s estimate s o f

th e parameter s U

J an daj , denote du

j an da; , fo r

th e norma l distributio n applie dt o th e log-odd s transforme dvariabl e Y , in th e manne r develope di

n Appendi x B . Th e se t of distributions , eac hhavin g separat e least-square s estimate s fo r u : an daj , i sP.2 6

2 0 referre dt o a s th e best-fittin g normal-on-log-odd s distributions . Furthermore , ameasur e i s availabl e of

ho w wel l th e probabilit y judgment s a t agive n lea dleve l ar e represente d by th e derive ddistribution . Thi s measure , discusse di n Appendi x B , i s th e standar d regressio n r 2 statisti c fo r th e goodnes s of

fi t of th e norma l distributio n wit hparameter s U J an da - t o th e CD Fove r

th e transforme dvariabl e Y . A s presente dfull y in Appendi x C , an dsummarize di n Sec . 4 , th e normal-on-log

odd s distributio n describe s th e encode djudgment s ver y well , wit hr value s generall y exceedin g 0.95 . (Th e maximu mpossibl e r 2 valu e i s 1.0 , whic hwoul dindicat e aperfec t fit. ) Nevertheless , i t

may b e necessar y t o sacrific e goodnes s of fi t t o asmal l degre e b y fittin g a n expert' s probabilit y judgment s withafamil y of equal-varianc e normal-on-log

odd s distributions . Doin g s o woul dassur e tha t th e derive ddistribution s a t tw o differen t lea dlevel s neve r

cros s on e another . (Crossin g woul dimpl y tha t exceedin g aspecifie d respons e rat e i s judge dmor e probabl e a t blood-lea dleve l Lj tha n a t L2 , wher e L 2 i s greate r tha n Lj. ) Becaus e of

th e manne r

i n whic hjudgment s wer e obtaine dfro mth e pilo t subjects , crossing s di dnot occur . Furthermore , judgment s wil l b e obtaine dfro mth e healt hexpert s ove r

awide r probabilit y rang e s o tha t crossing s woul donl y b e possibl e a t cumulativ e probabilitie s ver y nea r t o 0 an d1 . However , eve n crossing s suc ha s thes e woul drepresen t

an inconsistenc y an dtherefor e shoul dno t b e allowed . Equal-varianc e normal-on-log-odd s distribution s ar e derive db y firs t obtainin g a least-square s estimat e of th e standar ddeviatio n poole dove r lea dlevels , a' , the n settin g al l th e 0 ' = a ' an dfindin g ne w leas t square s estimate s of

th e means , u' , a s detaile di n Appendi x B . Th e resul t i s afamil y of

normal-on-log-odd s distributions , on e membe r fo r eac hblood-lea dleve l L- , tha t hav e acommo n varianc e an ddiffe r onl y i n th e mean . Thei r goodnes s of

fi t ca n als o b e assesse db y mean s o f r 2 calculate dfo r

th e fi t of th e variabl e Y . I f afunctiona l relationshi p ca n b e establishe dbetwee n blood-lea dlevel s an dth e u' , ajudgmenta l probabilit y densit y functio n ove r Rca n b e calculate dfo r an y blood-lea dP.2 7 2 1 level , shoul dtha t

b e necessar y fo r

interpolatin g betwee n blood-lea dlevels . Thi s continuu mof

densit y function s in essenc e i s aprobabilit y surfac e ove r th e dose-respons e plane . I f fo r eac hblood-lea dleve l L- , probabilit y judgment s ar e elicite dove r virtuall y th e entir e [0,1 ] interval , an di f thes e judgment s exhibi t n o crossings , the n i t

woul db e expecte dtha t

th e goodnes s o

f fi t o f th e equal-varianc e normal-on-log-odd s distribution s woul db e clos e t o tha t

of th e best-fittin g normal-on-log-odd s distributions . Sinc e th e equal-varianc e distribution s coul dno t giv e ris e t o inconsistencies , the y woul db e take n a s th e preferre drepresentatio n o f hi s o r he r judgment s fo r

purpose s o

f developin g ris k estimates . 3. 3 COMBININ G BLOOD-LEA D DISTRIBUTION S WIT H PROBABILIT Y JUDGMENT S ABOU T DOSE-RESPONS E FUNCTION S Usin g eithe r

equal-varianc e o r

best-fittin g normal-on-log-odd s function s fi t t o th e judgment s i

n th e manne r

describe dabove , i t

i s possibl e t o estimat e dose-respons e function s tha t

correspon dt o an y give n judgmenta l cumulativ e probability . Fo r example , amedian dose-respons e curv e basedon a n expert' s probabilit y judgment s i s obtaine db y determinin g th e respons e rat e tha t correspond s t o acumulativ e probabilit y o f 0.5 0 a t eac hblood-lea dL . Th e dose-respons e curve , suc htha t ther e i s a5 % chanc e of th e tru e respons e rat e lyin g abov e th e curv e a t an y give n bloo dlead , i s obtaine db y determinin g th e respons e rat e tha t

correspond s t o acumulativ e probabilit y o f 0.9 5 a t eac hL , an ds o forth . A s show n i n Appendi x B , an y blood-lea ddistributio n ca n the n b e combine dwit h th e dose-respons e curv e correspondin g t o judgmenta l cumulativ e probabilit y Ft o yiel da probabilit y estimat e fo r th e overal l fractio n o f th e populatio n Rsufferin g th e advers e healt heffect . I

f ther e i s als o uncertaint y abou t th e blood-lea ddistribution , tha t mus t b e take n int o accoun t

a s well , but

tha t proble mwil l no t

b e treate din thi s report .P.2 8

2 2 W e wil l conside r

her e onl y th e empirica l blood-lea ddistributio n fro mth e NHANES-I

I survey . Th e distributio n i s presente din discret e form , wit hinterval s of 0- 9 ug/dl , 10-1 9 ug/dl , ... , 60-6 9 ug/dl . Appendi x B shows , in particular , how t o combin e th e NHANES-I

I blood-lea ddistributio n wit hadose-respons e curv e correspondin g t o an y judgmenta l probabilit y F , fo r al l value s of

Fbetwee n 0. 0 an d1.0 . Fo r example , i

f th e NHANES-I

I distributio n i s combine dwit hth e dose-respons e curv e correspondin g t o ajudge dcumulativ e probabilit y of 0.50 , th e resul t

i s a n estimat e o f th e fractio n o

f th e populatio n sufferin g th e advers e healt heffect , suc htha t th e tru e fractio n i s equall y likel y t o b e abov e o r belo w it . I

n general , combinin g th e blood-lea d distributio n wit hth e dose-respons e curv e correspondin g t o judgmenta l cumulativ e probabilit y Fyield s a n estimat e of th e fractio n of

th e populatio n Rsufferin g th e healt h effect , suc htha t

th e tru e valu e lie s belo w Rwit hprobabilit y F . B y carryin g ou t thi s calculatio n fo r

al l value s of

F , 0 < F< 1 , an dplottin g eac hFa s afunctio n of th e derive d R , ajudgmenta l CD Fi s obtaine dove r affecte dfractio n of

th e population , a s illustrate d i n Fig . 3.2 . W

e cal l thi s CD Faderive dris k distribution . Figur e 3. 2 represent s ahypothetica l derive dris k distributio n fo r

aparticula r populatio n an daparticula r advers e healt heffect , derive dfro ma n expert' s probabilit y judgment s abou t

dose-respons e function s combine dwit hablood-lea ddistribution . On e ca n rea dfro mit , fo r

example , tha t

wit hprobabilit y 0.90 , les s tha n abou t 62 % of th e populatio n suffe r

th e advers e healt heffec t (an dwit hprobabilit y 0.10 , mor e tha n 62 % do) , wit hprobabilit y 0.75 , les s tha n abou t 47 % suffe r

i t (an dwit hprobabilit y 0.25 , mor e tha n 47 % do) , an ds o forth . A separat e ris k

distributio n i s derive db y combinin g eac hexpert' s probabilit y judgment s abou t

dose-respons e curve s wit hablood-lea ddistribution . Thi s give s ris e t o a famil y of ris k distribution s tha t

represent s th e rang e of uncertaint y abou t

th e prevalenc e o f th e advers e healt heffect . Thi s famil y ca n b e show n an dexplaine dt o th e decisio n maker .P.2 9 2 3



### 1. 0

a. m-A

x . ra X i o u P . > 3 E 3 O



### 0* 4 0. 6 

### 1. 0 Populatio n Respons e Rate , Expresse d A s Fractio n o f

Ponulatio n Affected. . R

FIGURE 3. 2 A Hypothetica l Derive d Probabilit y Distributio n abou tth e Fractio n o f aWeU-Define d Populatio n Tha tWil l Suffe raWell-Define d Advers e Healt h Effect , Obtaine d b y Combinin g Probabilit y Judgment s Suc h a s i n Fig . 3. 1 wit h a Blood-Lea d Distributio n Suc h a s th e NHANES-I IP.3 0 2 4 I f alternativ e blood-lea ddistribution s ar e hypothesize d(say , unde r alternativ e NAAQSs) , an dther e i s uncertaint y abou t them , i t

i s asmal l matte r

t o generaliz e th e presen t procedure s t o incorporat e thi s riche r informationa l structure .P.3 1 2 5 4 ILLUSTRATIV

E CALCULATION

S Th e analytica l method s derive din Appendi x B an dsummarize din th e las t sectio n shoul db e applie dt o th e judgment s of healt hexpert s onl y afte r th e method s hav e bee n teste dan dshow n t o describ e th e judgment s wit hsufficien t accuracy . I

n Appendi x C best-fittin g normal-on-log-odd s distribution s an dequal-varianc e normal-on-log-odd s distribution s ar e derive dfo r th e tw o pilo t

subjects , an dthei r

fit s ar e assessed . I

n addition , ris k

estimate s ar e derive dbase don th e probabilit y judgment s fro meac hpilo t subject . Thi s sectio n summarize s th e materia l in Appendi x C . Also , t o establis hth e generalit y of

th e approach , distribution s hav e bee n fi t t o th e probabilit y judgment s of th e four lea dexpert s encode db y Wallsten , Forsyth , an dBudesc u (1983) , an dth e result s ar e presente din thi s section . 4. 1 FITTIN

G NORMAL-ON-LOG-ODD S DISTRIBUTION S T O JUDGMENTS 4.1. 1 Pilo tSubject1 Recal l tha t

th e probabilit y judgment s fro mthi s subjec t ar e show n in Tabl e 2. 1 an dFig . 2.1 . Th e best-fittin g an dequal-varianc e normal-on-log-odd s distribution s bot h fi t thi s subject' s probabilit y judgment s ver y well , a s indicate db y th e r-squar e statistic s summarize din Tabl e 4. 1 fro mTable s C. 2 an dC. 3 of Appendi x C . Furthermore , th e r-squar e value s fo r th e equal-varianc e normal-on-log-odd s distribution s ar e almos t a s grea t

a s thos e fo r

th e best-fittin g ones , excep t a t L= 3 5 ug/dl . Thi s suggest s tha t wit hth e possibl e exceptio n of th e fi t a t L= 3 5 ug/dl , littl e descriptiv e powe r

i s los t b y relyin g o n th e equal-varianc e functions . Th e reaso n fo r

thi s patter n of

result s i s tha t

th e variance s of

th e best-fittin g distribution s wer e in fac t nearl y equal , excep t

fo r tha t o f th e functio n fi t

a t th e blood

lea dleve l L= 3 5 ug/dl , whic hgreatl y exceede dth e othe r variances . Thi s outcom e i sP.3 2 2 6 TABL E 4. 1 R-Squar

e Value s fo rth e Best Fittin g an

d Equal-Variance Normal-On Log-Odds Distribution s Fitte d t o th e Probability Judgments o f Subject s 1 an d 2 Conditional on th e Indicate d Blood-Lea d Level s

Blood Lead 5 ug/d l 1 5 ug/d l 2 5 ug/d l 3 5 ug/d l 4 5 ug/d l 5 5 ug/d l

a Bes t Fitt i Subjec t 1

B.F. a E.V. b 0.9 7 0.9 5 0.9 9 0.9 9 0.9 8 0.9 3 0.9 7 0.6 2 0.9 7 0.9 4 0.9 8 0.9 8 ng Subj < B.F . NA C 0.9 7 N A 0.9 9 N A 0.9 5 >c

t 2 E.V . N A 0.8 7 N A 0.9 8 N A 0.9 1

Equa l Variance . c No t Appropriate . Judgment s wer e no t obtaine d a

t thi s blood-lea d level . explore din detai l i n Appendi x C an dca n b e trace dt o th e fac t tha t th e CDF s encode dfo r th e neighborin g blood-lea dlevel s of L= 2 5 an d4 5 ug/d l di dno t begi n sufficientl y clos e t o 0.0 , aproble mpreviousl y discusse di n Sec . 2.4.1 . Th e larg e varianc e a t L= 3 5 ug/d l cause dtha t

normal-on-log-odd s distributio n t o cros s thos e a t L= 2 5 an d4 5 ug/d l an dt o hav e amean les s tha n th e mean a t L= 2 5 ug/dl . (Se e Tabl e C. 2 an dFig . C. l fo r details. ) Thes e ar e clearl y inconsistencie s tha t subjec t 1 sough t

t o avoi dan dtha t

d o no t occu r wit hth e equal-varianc e normal-on-log-odd s distributions . Thus , al l thing s considered , th e equal-varianc e distribution s a s agrou p appea r t o provid e th e bette r description s o

f subjec t l' s probabilit y judgments .P.3 3 2 7 Figur e 4. 1 show s th e CDF s fro mth e equal-varianc e normal-on-log-odd s distribution s alon g withth e encbde dprobabilit y judgment s of subjec t 1 . I t ca n b e see n tha t th e fi t i s quit e satisfactory . Anothe r interestin g representatio n of th e subject' s judgments , derive dthroug h th e equal-varianc e normal-on-log-odd s function , i s th e famil y of dose-respons e curve s shown i n Fig . 4.2 . Th e 0.5 0 cumulativ e probabilit y functio n show s th e media n judge d respons e rate , R , a t

eac hlea dlevel , L . Th e 0.9 5 an d0.0 5 function s sho w th e respons e rates , R , a t

eac hlea dlevel , L , suc htha t ther e i s a5 % chanc e of th e tru e respons e rat e being , respectively , abov e o r belo w th e indicate dvalue . Thus , i n asense , th e tw o extrem e dose-respons e function s ma p ou t 90 % credibl e interval s fo r respons e rat e a t eac hblood-lea dlevel , basedon subjec t l' s probabilit y judgments . 4.1. 2 Pilo tSubject2 Th e probabilit y judgment s fo r thi s subject , liste din Tabl e 2. 2 an ddisplaye di n Fig . 2.2 , wer e als o fi t wit hbest-fittin g an dequal-varianc e normal-on-log-odd s distributions . Th e r-squar e statistic s fro mTable s C. 4 an dC. 6 ar e reproduce di n Tabl e 4.1 , wher e i t

ca n b e see n tha t

th e best-fittin g function s d o an excellen t job , an dth e equal-varianc e one s ar e almos t a s good . Th e greates t

discrepanc y occur s a t

L= 1 5 ug/dl , but eve n her e th e lesse r

r-squar e valu e i s quit e high . Unlike th e result s wit hsubjec t 1 , th e variance s o

f th e best-fittin g distribution s fo r subjec t 2 increase dwit hlea dleve l (cf. Tabl e C.4) . I f ther e ha dbee n tim e i n th e encodin g sessio n t o elici t judgment s fo r

mor e tha n th e thre e lea dlevel s o f 15 , 35 , an d5 5 ug/dl , s o tha t

adjacen t level s woul dhav e bee n mor e closel y spaced , the n th e consistenc y requiremen t

tha t CDF s no t

cros s woul dhav e cause dth e variance s t o b e mor e nearl y equal . A

n ancillar y poin t

pursue di

n Appendi x C concern s th e us e o f linea r regressio n technique s t o estimat e normal-on-log-odd s function s a t intermediat e lea dlevel s wher e2 8 P.3 4



### 1. 0

ta. >A X i ra x o OJ > 3 e 3 u

Populatio n Respons e Rate , Expresse dA s Fractio n o f

Populatio n Affected , R

FIGUR E 4. 1 Equal-Varianc e Normal-On-Log-Odd s CDF s fo r th e Probabilit y Judgment s of

Subjec t 1 Tha t Were Previousl y Show n i n Fig . 2. 1P.3 5

29

t3 0 1 U 0 1 c o 3 a o

c o u ra u l. U



### 0. 8

0. 6

0. 4

0. 2



## n. o 

## 1

— 

# 1 : 1 1 1 1

0.9 5 Cumulativ e ^ ^ Probabilit y*>^

/ 0.5 0 Cumulativ e ^s' / Probabilit y^ ^

/ ^ 0^m. " 0.0 5 Cumulativ e / ^**Probabilit y 

## 1 1 I 1

• ^—

^—



# 1 1 5 2 5 3 5 4 5 5 5 Blood-Lea d Level , L (ug/dl )

FIGUR E 4. 2 The 0.05- , 0.50- , an d 0.95-Cumulativ e Probability Estimate s o f Respons e Rate Ra tVarious Blood-Lea d Level s L , Base d on th e Equal-Varianc e Normal-On-Log Odd s Distribution s Fitte d t o th e Judgments o f Subjec t1P.3 6 3 0 encodin g di dnot

tak e place . Suc hestimatio n become s necessar y t o carr y throug hth e subsequen t

ris k calculations . Figur e 4. 3 show s th e regresse dequal-varianc e normal-on

log-odd s distributio n CDF s fo r al l si x lea dlevel s alon g wit hth e encode dprobabilit y judgment s fo r

th e level s a t

whic hthe y wer e obtained . Figur e 4. 4 show s th e 0.05- , 0.50- , an d0.95-cumulativ e probabilit y dose-respons e function s base do n th e CDF s i n th e precedin g figure . Thes e diffe r fro mthos e fo r

subjec t 1 (Fig . 4.2 ) primaril y a t th e lowe r lea dlevels . 4.1. 3 Judgments o f

Othe rExperts A s afurthe r

tes t of th e generalit y o

f th e technique s develope dhere , best-fittin g an dequal-varianc e normal-on-log-odd s distribution s wer e fi t t o th e probabilit y judgment s o f th e lea dexpert s previousl y encode dby Wallsten , Forsyth , an dBudesc u (1983) . A s wit h th e pilo t subjects , thes e expert s provide dprobabilit y judgment s ove r respons e rate s a t fiv e blood-lea dlevel s regardin g lead-induce dhemoglobi n reduction . Uppe r an dlower , rathe r tha n sharp , probabilitie s wer e encode da t eac hdose-respons e point . Distribution s wer e fi t t o th e mean s of

th e uppe r

an dlowe r

probabilities , whic hca n b e take n a s approximation s of

th e value s tha t

woul dhav e bee n obtaine dha dshar p probabilitie s bee n sought . Tabl e 4. 2 show s th e mean s o f th e fiv e r-square s fo r

th e best-fittin g normal-on

log-odd s distribution s fo r eac hexpert , a s wel l a s th e poole dr-squar e fo r th e fi t of th e equal-varianc e distributions . I t i s apparen t

fro mTabl e 4. 2 tha t

th e individua l normal-on

log-odd s distribution s provid e excellen t fit s t o th e judgments . Th e equal-varianc e distributio n provid e respectabl e fits , bu t no t a s goo da s thos e see n previously . However , th e range s o

f probabilitie s tha t

wer e encode dfo r

eac hdistributio n in thi s stud y wer e quit e restricted , averagin g onl y 0.5 8 ove r th e fou r experts . Hadmor e complet e range s bee n encoded , maintainin g no distributio n crossovers , th e equal-varianc e function s woul d hav e fi t muc hbetter .3 1 P.3 7



### 1. 0



### 5 0.6 - X ) ra X I o u a. o i > 3 E 3 C J

Populatio n Respons e Rate , Expresse d A s Fractio n o f

Populatio n Affected , R

FIGUR E 4. 3 Regresse d Equal-Varianc e Normal-On-Log-Odd s CDF s fo rth e Probability Judgments o f

Subjec t2 ThatWer

e Previousl y Show

n i n Fig . 2.2 , a s Wel

l a s fo r Intermediat e Lead Level sP.3 8 3 2



### 1. 0

0. 8

0 1 u 0 1 c o 3 c o p 

### 0. 6 -

: 0.4 - c o u ra 

### 0. 2 

### -

0. 0 0.9 5 Cumulativ e

Probabilit y

0.5 0 Cumulativ e Probabilit y

Blood-Lea d Level , L (ug/dl )

FIGURE 4. 4 Th e 0.05- , 0.50- , an d 0.95-Cumulativ e Probabilit y Estimate s o f Respons e Rate s Ra tVariou s Blood-Lea d Level s L Base d o n th e Regresse d Equal-Varianc e Normal-On-Log-Odd s Distribution s Fitte d t o th e Judgment s o f Subjec t2P.3 9 3 3 TABL E 4. 2 Regressio n r 2 Value s fo rth e Fi to f Best-Fittin g an

d Equal-Varianc e Normal-On-Log Odd s Distribution s t o Judgments o f Experts Encode d by Wallsten , Forsyth , an d Budescu (1983 ) Exper t r 2 E F G H Best-Fittin g 0.9 6 0.9 6 0.9 5 0.9 5 Equal-Varianc e 0.9 2 0.8 5 0.8 5 0.8 3 T o summariz e al l th e results , th e normal-on-log-odd s functio n applie dt o individua l subjectiv e probabilit y distribution s generall y provide s excellen t fits , wit hr square s o n th e orde r

of 0.9 5 an dhigher . Constrainin g th e normal-on-log-odd s distribution s t o hav e equa l varianc e fo r a n entir e se t

of distribution s stil l yield s ver y goo dfittin g functions , bu t with , of

course , somewha t

lowe r r-squares , whil e simultaneousl y avoidin g th e possibilit y of inconsistencies . Since , a s discusse dpreviously , probabilit y judgment s wil l b e obtaine dfro mth e healt hexpert s ove r virtuall y th e ful l [0,1 ] interva l fo r

si x lea dlevels , i t

ca n b e expecte dtha t

equal-varianc e function s wil l b e quit e accurate . 4. 2 ILLUSTRATIV

E RIS KCALCULATIONS Havin g establishe dtha t th e normal-on-log-odd s distributio n describe s th e probabilit y judgment s ver y accurately , th e nex t ste p i s t o combin e th e judgmen t distribution s wit hth e NHANES-II blood-lea ddistributio n t o yiel dderive dris k distribution s i n th e manne r describe di n Appendi x B an dsummarize din Sec . 3.3 . Thi s i s don e i n Appendi x C wit hth e judgment s of bot hpilo t

subject s usin g th e NHANES-II distributio n fo r

childre n age d6 month s t o 5 years , t o illustrat e th e method . Th e result s ar e summarize dhere .P.4 0 3 4 Derive dris k

distribution s of

th e sor t depicte din Fig . 3. 2 wer e calculate dusin g bothth e best-fittin g an dth e equal-varianc e normal-on-log-odd s distributio n representation s of

eac hsubject' s judgments . Th e result s fo r subjec t 1 wer e virtuall y identica l fo r

th e tw o representative s (Fig . C. 5 in th e appendix) , bu t som e difference s occurre dfo r

subjec t 2 (Fig . C.6) . I

n th e latte r

case , th e ris k

distributio n wa s slightl y more shallo w (i.e. , span s alarge r range ) base don th e equal-varianc e tha n o n th e best

fittin g representation . However , th e tw o curve s wer e ver y clos e in th e uppe r thir dof th e cumulativ e distribution , whic hwoul db e th e point s of mos t interes t t o adecisio n maker . Figur e 4. 5 show s th e derive dris k distribution s base don th e equal-varianc e normal-on-log-odd s distributio n representatio n o f eac hsubject' s probabilit y judgments . Utilizin g subjec t

l' s judgments , th e media n estimat e i s tha t 10.5 % of

U.S . childre n age d 0- 3 hav e hemoglobi n level s les s tha n o r equa l t o 1 1 g/dl . Th e media n estimat e utilizin g subjec t 2' s judgment s i s 20.1% . Base don subjec t l' s judgments , ther e i s a0.1 0 probabilit y tha t

mor e tha n 20.5 % o

f th e populatio n suffe r

th e advers e healt heffect , whil e subjec t

2' s judgment s lea dt o th e statemen t tha t ther e i s a0.1 0 probabilit y tha t mor e tha n 39.1 % o

f th e populatio n do so . Othe r value s ca n b e rea dsimilarly . Fro m anothe r perspective , on e coul dfocu s o n particula r populatio n respons e rate s (e.g. , 10% ) an drea dth e probabilitie s tha t th e tru e respons e rate s ar e les s tha n tha t valu e base do n eac hexpert' s judgment . Th e differenc e betwee n th e derive dris k distribution s base don th e tw o subject' s judgment s i s easil y trace dbac k t o particula r

feature s o

f thei r probabilit y judgments . Subjec t 1 considere drespons e rat e t o b e muc hlowe r a t 1 5 ug/d l bloo dlea dtha n di d subjec t 2 (cf. Figs . 2. 1 an d2. 2 o r 4. 2 an d4.4) , an daccordin g t o th e NHANES-II blood

lea ddistribution , 63 % of th e populatio n hav e bloo dlead s i n th e correspondin g interva l of 10-1 9 ug/dl . Thus , quit e naturally , subjec t l' s judgment s lea dt o lowe r ris k estimate s tha n do subjec t

2's .P.4 1 3 5



### 1. 0

0. 8

u.

x ra x o u a . 0) > 3 6 3 U 

### 0. 6 

### -

0 . 4



### 0. 2

0. 0 Subjec t

1



### 0 . 0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7

Populatio n Respons e Rate , Expresse dA s Fractio n o f

Populatio n Affected , R

FIGUR E 4. 5 Derive d Ris kDistribution s Base d o n Equal-Varianc e Normal-On-Log-Odd s Distribution s Fi tt o Subjec tl' s an d Subjec t2' s Judgments , Showin g i n Eac h Cas e aCumulativ e ProbabUit y F tha tFractio n Ro f th e Populatio n o f

U.S . Childre n Age d 0- 3 Hav e Hemoglobi n Level s Les s Tha n o rEqua l t o 1 1 g/d lP.4 2 3 6 5 COMMUNICATINGTH E RIS K ASSESSMEN

T RESULT S T O A DECISION MAKE R Althoug hth e mathematic s t o thi s poin t ar e of moderat e complexity , the y resul t i n probabilisti c statement s abou t dose-respons e curves , derive dris k distribution s an dris k estimate s base do n th e judgment s of expert s (two pilo t

subject s wer e use dhere ) tha t ca n b e clearl y communicate dt o adecisio n maker . A grap hsuc ha s Fig . 4. 5 t o represen t derive dris k

distribution s base do n eac hexpert' s judgment s should , wit hsom e explanation , b e understandabl e b y adecisio n make r an dhi s o r

he r staff . Fo r example , fro mFig . 4. 5 i t i s relativel y eas y t o se e that , wit hprobabilit y of 0.05 , a t leas t 24 % of th e populatio n hav e hemoglobin s les s tha n o r equa l t o 1 1 g/d l accordin g t o subjec t

l' s judgments , an da t

leas t 46 % d o s o accordin g t o subjec t 2's . Fro m anothe r perspective , i

f th e decisio n make r

wishe s t o focu s o n aparticula r populatio n respons e rate , becaus e i n hi s o r he r opinio n tha t

i s th e maximu mallowabl e rat e consisten t

wit hprotectin g th e publi c health , the n h e o r sh e ca n rea dth e probabilit y tha t i t i s exceede dgive n eac hexpert' s judgments . Fo r example , i

f th e decisio n make r

chos e t o focu s on apopulatio n respons e rat e of 10% , the n h e o r

sh e woul dse e tha t

thi s rat e i s currentl y exceede dwit hprobabilit y 0.5 5 accordin g t o subjec t l' s judgment s an dwit hprobabilit y 0.8 5 accordin g t o subjec t 2's . Derive dris k

distribution s suc ha s thos e in Fig . 4. 5 fo r eac hdefinitio n of

th e advers e healt heffec t

(9. 5 an d1 1 g/d l o

f hemoglobin ) fo r

eac ho f th e tw o ag e group s (0- 3 an d4- 7 year s old) , o r

simpl y combine dove r

bot hag e groups , wil l giv e th e decisio n make r agoo dpictur e of

th e curren t

ris k estimates . Whil e illustrativ e value s rea dfro mth e graph s shoul db e include din accompanyin g text , th e decisio n make r determine s wha t definitio n of

th e advers e healt heffect , wha t populatio n respons e rates , an dwha t probabilitie s constitut e "protectin g th e publi c healt hwit ha n adequat e margi n of safety. " Havin g mad e thes e decisions , h e o r sh e ca n the n consul t

th e derive dris k distribution s t o determin e whethe r th e rang e of

exper t opinio n suggest s tha t

th e criteri aP.4 3 3 7

ar e currentl y bein g met . I f estimate dalternativ e blood-lea ddistribution s ar e availabl e give n alternativ e NAAQ S (a n issu e no t

addresse din thi s report) , the n simila r ris k distribution s ca n b e derive dfo r th e alternativ e NAAQS , an dth e decisio n make r ca n consul t thos e a s well . I t i s likel y tha t

th e judgment s o

f th e actua l expert s wil l b e mor e simila r t o eac h othe r tha n wer e thos e o

f th e pilo t

subjects , leadin g t o mor e simila r derive dris k distributions . Bu t

shoul dtha t

not b e th e case , the n th e decisio n make r wil l b e wel l serve dby seein g th e rang e o f ris k estimates , rathe r

tha n asimpl e "average " estimate . A secon dkin dof

outpu t tha t ma y b e understandabl e by adecisio n make r an dhi s o r he r staf f i s th e famil y of

equal-probabilit y dose-respons e functions , suc ha s thos e show n in Figs . 4. 2 an d4.4 . I f suc hindividual s ar e familia r withth e usua l empirica l dose

respons e curves , includin g associate dstandar derro r bars , the n thes e figure s ar e asmal l conceptua l leap . The y woul dserv e t o illustrat e th e rang e o f exper t judgmen t

regardin g th e dose-respons e relationshi p betwee n lea dan dth e advers e healt heffec t fo r th e populatio n of

interest , withou t

requirin g th e estimatio n o f blood-lea ddistributions .P.4 4 3 8 6 OBSERVATION S AN

D RECOMMENDATION S W e hav e foun dth e pilo t

encodin g session s an danalysi s t o b e ver y valuable . The y hav e enable du s t o wor k ou t man y smal l difficultie s tha t coul donl y b e discovere db y experience , an dmak e u s wel l prepare dt o begi n wor k wit hactua l experts . Th e foUowing section s contai n observation s an drecommendation s base don thes e tw o experience s describe din Sees . 2 an d4 . 1 . Th e initia l broa drange s on respons e rat e give n b y subjec t 1 prove d t o b e inappropriat e i

n man y cases . Becaus e of limite dtim e an d th e fac t tha t onl y on e analys t

wa s present , thi s wen t unnotice d durin g th e session . Thi s proble mshoul db e overcom e b y usin g tw o analysts , provide dtha t the y ar e carefu l t o obtai n probabilit y judgment s fo r

eac hlea dleve l tha t

adequatel y spa n th e 0. 0 t o 1.0 probabilit y interval . 2 . I f th e normal-on-log-odd s distributio n i s t o b e used , probabilitie s andrespons e rate s o

f 0 an d1 ar e no t

helpful . Instead , value s aroun dth e extreme s of

0.0 1 an d0.9 9 shoul db e sought . 3 . Whe n inconsistencie s aros e durin g th e encoding s conditiona l on respons e rates , subjec t 1 tende dt o chang e hi s judgment s conditiona l on respons e rat e (R) rathe r tha n thos e conditiona l o n • blood-lea dleve l (L) . Analyst s shoul db e awar e o f thi s possibl e reactio n and , i

f i t occurs , shoul dconside r spendin g mor e tim e on obtainin g judgment s conditione don L , o r possibl y no t

obtai n judgment s conditiona l on Ra t all. * 4 . Subjec t

1 remarke dtha t

startin g a t

1 5 ug/d l wa s no t

ver y usefu l t o himbecaus e ther e i s essentiall y nothin g happenin g a t thi s level . Thi s ma y hav e le dt o th e larg e sprea dbetwee n th e L= 1 5 ug/d l an d L= 2 5 ug/d l judgments . I f subject s see mt o hav e troubl e gettin g starte da t

lo w lea dlevels , th e analys t shoul dshif

t t o ahighe r

leve l an dthe n retur n late r

t o th e lowe r

levels . 5 . Subjec t

1 judge dtha t

separat e function s wer e appropriat e fo r childre n age d0- 3 year s an d4- 7 years . Thi s woul dhav e require d encodin g fou r

set s of probabilitie s (sinc e th e pla n i s t o encod e a t tw o level s o

f hemoglobin) . Th e judgment s wer e encode di n abou t four hours , includin g discussio n o f th e literature , a n explanatio n of •Becaus e encoding s conditiona l on Rar e potentiall y ver y valuable , the y provid e aric h topi c fo r extensiv e researc hsom e tim e i n th e future .P.4 5 3 9 th e probabilit y wheel , etc' . I t woul dhav e take n a t

leas t tw o mor e hour s t o complet e th e othe r thre e set s o

f probabilit y judgments , indicatin g tha t

afu U da y shoul db e reserve dfo r th e firs t roun dof encodings t o enhanc e th e chance s of obtainin g al l of

th e neede d judgments . 6 . Only thre e set s of judgment s coul db e obtaine di n th e tim e availabl e wit hsubjec t

2 . However , th e abilit y t o develo p a complet e se t

of probabilisti c judgment s (neede dt o complet e calculation s of

ris k distributions ) i s crucia l an dshoul db e guaranteed . 7 . Becaus e mor e tha n 95 % o f th e childre n in th e NHANES-I I surve y ha dblood-lea dlevel s belo w 3 0 ug/dl , i t migh t hav e bee n bette r

t o hav e encode do n L= 2 5 ug/d l instea dof L= 3 5 ug/d l (whic hwa s th e thir dcurv e encoded ) fo r subjec t 2 . I f tim e appear s t o b e a constrain t

in th e future , analyst s shoul dtr y t o maximiz e th e chance s of

gettin g th e mos t

critica l judgment s (i

f the y ar e know n a t th e tim e of

th e encodings) , eve n i f i t require s changin g th e planne d(possibl y randomized ) order . 8 . Equal-varianc e normal-on-log-odd s function s fi t ver y closel y th e encode djudgment s o

f subject s 1 an d2 . 9 . Th e normal-on-log-odd s distributio n i s appealin g becaus e i t permit s an analytica l metho dfo r accuratel y representin g encode djudg

ments . However , i

f an y suc hmathematica l distributio n i s used , i t an dit s implication s mus t b e thoroughl y explaine dt o th e exper t whos e judgment s ar e used . Ultimately , th e result s mus t accuratel y portra y an dsummariz e eac hexpert' s judgments . Therefore , ideall y th e distribution s wi U b e fi t whil e wit hth e exper t

t o assur e tha t h e o r sh e agree s tha t

the y accuratel y represen t hi s o r he r judgments .P.4 6 4 0 7 REFERENCE S 1 . Feagans , T.B. , an dW.F . Biller , A General Method for Assessing Health Risks Associated

with Primary National Ambient Air Quality Standards, U.S . EP A Draf t Repor t (Ma y 1981) . 2 . Wallsten , T.S. , B.H . Forsyth , an dD.V . Budescu , Stabilit y and Coherence of

Health Experts! Upper

and Lower Subjective Probabilities about Dose-Response Functions, Organizationa l Behavio r an dHuma

n Performance , 31:277-30 2 (1983) .P.4 7 4 1 APPENDI X A DRAF T PROTOCO L FO RENCODIN G PROBABILIT Y JUDGMENT S

ABOU T DOSE-RESPONS E CURVE S FO R LEAD-INDUCE D HEMOGLOBI N DECREMENT S OUTLIN E A.l . INTRODUCTIO N A.2 . LEAD-INDUCE D HEMOGLOBI N DECREMENT S A.2. 1 Definitio n o f

Hemoglobi n Decremen t a s a n Advers e Healt hEffec t A.3 . POPULATIO N A T RIS K A.4 . EXPOSUR E CONDITION S A.5 . PHYSIOLOGICA LCONDITION S A.6 . FACTOR S T O CONSIDE R A.7 . FACTOR S T O KEE P I N MIND WHEN MAKIN GPROBABILIT Y JUDGMENT S A.7. 1 Sequentia l Consideratio n o f Informatio n A.7. 2 Effect s o f

Memor y o n Judgmen t A.7. 3 Estimatin g Reliabilit y o f Informatio n A.7. 4 Relatio n Betwee n Even t Importanc e an dProbabilit y A.7. 5 Estimatio n o f

Probabilitie s A.7. 6 Recommendation s A.8 . FINA LPREPARATIO N FO RELICITATIO N O FPROBABILIT Y JUDGMENT S

Versio n 84.6. 5P.4 8 4 2 APPENDI X A DRAF T PROTOCO L FO RENCODIN G PROBABILIT Y JUDGMENT S

ABOU T DOSE-RESPONS E CURVE S FO R LEAD-INDUCE D HEMOGLOBI N DECREMENT S A.1 . INTRODUCTIO N Th e Environmenta l Protectio n Agenc y (EPA ) i s charge db y th e Clea n Ai r Ac t withsettin g an drevisin g Nationa l Ambien t Air Qualit y Standard s (NAAQS ) fo r selecte d pollutants , a t

level s sufficien t

t o protec t

th e publi c healt hwit ha n adequat e margi n of safety . A s you know , th e scientifi c base s fo r NAAQ S ar e presente dan dreviewe din criteri adocuments . I

n suppor t of th e forthcomin g revisio n of th e lea dNAAQS , EPA ha s jus t prepare danew Air

Quality Criteria for

Lead. I t

present s scientifi c evidenc e fro m whic hth e mos t

susceptibl e population s ca n b e determine dan dfro mwhic hvariou s advers e healt heffect s ca n b e identified . Th e criteri adocumen t summarize s an d evaluate s th e availabl e clinical , epidemiological , an danima l o r toxicologica l laborator y evidenc e withregar dt o th e physiologica l an dadvers e healt heffect s o f lead , an d therefor e represent s ou r mos t up t o dat e knowledg e o n lea deffects . As on e aspec t

o f th e revie w process , EP A assesse s healt hrisk s b y identifyin g th e mos t sensitiv e population s fo r eac hpoUutant , an destimatin g probabilisticall y th e number s o

f peopl e i n th e population s who may suffe r eac ho f variou s well-define d advers e healt heffect s attributabl e t o th e poUutant . I t i s believe dtha t

informatio n abou t th e healt hrisk s associate dwit hvariou s potentia l standard s wil l ai dth e EPA administrato r

i n selectin g tha t

standar dwhich , in hi s judgment , protect s th e publi c healt hwit ha

n adequat e margi n of

safety . Becaus e th e ris k

estimate s tha t

EP A seek s ar e ofte n base di n par t o n dose

respons e relationship s an dlea dexposur e estimate s whic har e uncertain , i t i s necessar y t o mak e probabilit y judgment s abou t relevan t

dose-respons e function s base don th e availabl e evidence , an dt o probabilisticall y estimat e lea dexposur e unde r alternativ eP.4 9 4 3 NAAQS . Obtainin g th e healt hris k estimate s the n involve s combinin g dose-respons e an d exposur e probabilit y estimates . Th e proble mof

estimatin g dose-respons e relationship s i s simila r t o tha t whic h exist s in clinica l medicin e when ther e d o no t exis t dat atha t

bea r precisel y o n th e patient' s problem . I

n tha t cas e i t i s necessar y t o us e scientifi c judgmen t t o extrapolat e fro mth e dat at o mak e abes t decisio n fo r

th e patient . Here , too , i t i s necessar y t o us e scientifi c judgmen t

t o extrapolat e fro mth e availabl e data . Th e extrapolatio n i s no t certai n and , therefore , w e wil l ai dyou t o represen t you r opinio n probabilisticaUy . Furthermore , sinc e th e extrapolatio n depend s o n one' s interpretatio n of th e literature , differen t peopl e wil l hav e differen t judgments . Fo r

eac hhealt heffec t

w e inten dt o obtai n th e probabilisti c judgment s of abou t fiv e expert s t o sampl e th e rang e o f respecte d opinions . Th e mode l fo r estimatin g risk s wi

U no t merg e thes e judgment s int o asingl e averag e judgment , bu t

rathe r wi U estimat e th e rang e of risk s base do n th e rang e of judgments . I

f w e a s ris k

analyst s d o ou r

jo b properly , the n no t

onl y wil l w e b e abl e t o sho w th e EP

A administrato r

th e rang e of

estimate s base don th e rang e of judgments , bu t w e wi U als o b e abl e t o sho w som e of th e source s o

f th e disagreements . Indeed , asid e benefi t of thi s exercis e i n whic hw e prob e you r knowledg e in astructure dmanne r ma y b e t o hel p identif

y source s of

greates t disagreement . Base do n th e evidenc e i n th e lea dcriteri adocument , tw o population s hav e bee n identifie da s bein g mos t susceptibl e t o th e effect s of lea dintoxication . On e i s childre n fro mbirt hthroug hth e sevent hbirthday , an dth e othe r i s pregnan t

women , o r

mor e precisely , th e fetuse s bein g carrie db y th e pregnan t women . A numbe r

of advers e healt h effect s hav e bee n identifie dfo r whic hw e woul dlik e t o estimat e dose-respons e function s includin g th e on e discusse dbelow . A.2 . LEAD-INDUCE D HEMOGLOBI N DECREMENT S W e woul dlik e t o represen t in probabilisti c formyou r opinio n abou t

th e locatio n an dshap e of

dose-respons e function s fo r certai n we

U define dlevel s of

lead-induce dP.5 0 44 hemoglobi n reductio n in th e populatio n of U.S . childre n fro mconceptio n throug hth e sevent hbirthday . W

e realiz e tha t

availabl e dat ad o no t

full y defin e thes e functions ; i f the y did , the n i t

woul db e unnecessar y t o obtai n you r an dothe r

experts ' opinion s abou t them . Nevertheless , i

f th e population , th e exposur e conditions , an dth e healt heffect s ar e a U precisel y defined , the n suc hfunction s in fac t exis t an dw e woul dlik e you r bes t judgmen t abou t wha t the y woul dloo k

lik e i f th e dat acoul db e collected . A.2. 1 Definitio n o f

Hemoglobi n Decremen ta s a n Advers e Health Effec t Ther e ar e difference s of opinion , of

course , a s t o wha t

degre e of

reductio n in hemoglobi n leve l constitute s ahealt hrisk . Th e Clea n Ai r Ac t make s i t

clea r tha t EPA shoul dse t

it s standard s t o protec t agains t advers e healt heffects . However , th e reductio n in hemoglobi n leve l considere dt o b e advers e may b e differen t fo r regulator y purpose s tha n fo r

clinica l action . Sinc e i t i s generall y agree dfo r childre n tha t

a hemoglobi n leve l of

abou t 1 2 gram s pe r

decilite r

i s norma l an dabou t

9 gram s pe r decilite r i s anemic , w e wi

U specif y tw o hemoglobi n level s i n tha t interval , namel y Hg an d H j (curren t

candidate s ar e 9. 5 an d1 1 gm/dl) , an dtrea t eac ha s th e physiologica l effec t fo r purpose s o

f specifyin g adose-respons e function . W e wil l elici t

fro myo u probabilit y judgment s abou t

th e shap e an dlocatio n of th e dose-respons e curv e fo r eac hof th e tw o level s o f effect s in th e sensitiv e population . A.3 . POPULATIO

N A T RIS K W e ar e definin g th e mos t susceptibl e grou p a s al l Unite dState s childre n fro m conceptio n throug hthei r sevent hbirthday . However , yo u may believ e o n variou s ground s tha t younge r an dolde r

childre n i

n thi s grou p hav e differen t dose-respons e function s fo r lead-induce dhemoglobi n reduction . I f so , the n w e wil l elici t

you r judgment s separatel y fo r th e ag e group s 0- 3 an d4-6 .P.5 1 4 5 A.4 . EXPOSURE CONDITION S W e wi U b e askin g you r

judgmen t

abou t populatio n respons e rate s a t variou s blood-leadlevels . Assum e tha t th e blood-lea dleve l unde r consideratio n fo r

agive n judgmen t i s in equilibriu ma s aresul t of asufficientl y long-ter mconstan t lea dexposur e withou t gros s excursion s above o r belo w th e state dlevel . A.5 . PHYSIOLOGICA L CONDITION S Becaus e th e effec t

of lea don hemoglobi n leve l depend s o n man y parameter s of a person's system , i t

i s necessar y t o specif y assumption s abou t

thos e parameter s i n th e population . Th e effec t

of lea din th e syste mdepend s o n th e person' s nutritiona l an d metaboli c status . Th e incidenc e o f iro n deficienc y i s greate r in childre n tha n in adults , an dgreate r

ye t i n childre n ag e 0- 3 tha n ag e 4-6 , wit hawid e rang e o f iro n level s i n childre n up t o thei r

sevent hbirthday . Assum e tha t th e distributio n of

iro n level s i s tha t whic hyo u believ e i t

actuaU y t o b e in th e ag e group s 0- 3 an d4-6 . Th e effec t of lea dals o depend s o n th e level s of zinc , copper , vitamin s A , C , D , an dE , calcium , phosphorou s an d magnesium . Assum e in al l case s tha t thes e nutrient s ar e distribute di n th e tw o ag e group s a s you believ e the y in fac t are , takin g int o accoun t th e wid e rang e of

diet s an d nutritiona l level s o

f U.S . children . A.6 . FACTOR S T O CONSIDE R I n orde r t o hel p you brin g t o min dth e relevan t evidenc e s o tha t

yo u may conside r i t systematicaUy , an dals o in orde r t o hel p u s t o interpre t

you r judgments , w e woul dlik e t o as k yo u t o discus s briefl y you r interpretation s of

variou s aspect s of

th e literature . Firs t o f all , coul dyou tel l u s somethin g abou t how , in you r

judgment , lea dan diro n interac t t o reduc e hemoglobi n level s an dsomethin g abou t th e relatio n betwee n iro n an d lea dlevel s in th e body ? Similarly , w e woul db e intereste din you r opinion s abou t

th e sam e question s wit hregar dt o zin c an dlea dan dt o calciu man dlead . I t appear s tha t

lea dP.5 2 4 6 affect s hemoglobi n level s throug hthre e somewha t separat e routes : interferenc e wit h hem e synthesis , interferenc e withglobi n synthesis , an ddecrease dlif e expectanc y of erythrocytes . Briefly , wha t i s you r interpretatio n of

th e literatur e o n thes e variou s effect s o f lead ? Finally , w e know tha t som e group s of

childre n ar e a t

greate r ris k of exposur e t o lea ddu e t o thei r livin g in deterioratin g pre-195 0 housin g o r i n urba n area s withlarg e amount s of

vehicula r

traffic . Thes e childre n woul dten dt o hav e hig hlea d levels , but

d o yo u believ e tha t

ther e ar e als o independen t reason s t o thin k

tha t thei r dose-effec t

curve s wi

U b e differen t

fro mchildre n wh

o liv e in othe r

circumstances ? Fo r example , thes e may b e th e sam e childre n who ten dt o hav e poore r diet s an dles s acces s t o medica l care . Wha

t i s you r opinio n regardin g th e possibilit y of acorrelatio n betwee n increase dexposur e an dincrease dsusceptibility ? Ar e ther e othe r factor s t o conside r

in thinkin g abou t

th e dose-respons e function s fo r lead-induce dhemoglobi n decrement s tha t w e shoul ddiscus s now ?

A.7 . FACTOR S T O KEEPI N MIN D WHE N MAKIN G PROBABILIT Y JUDGMENT S Ther e i s usuall y uncertaint y associate dwit hconclusion s tha t w e dra w fro m research , an dmor e generall y in ou r everyda y thinking . However , no t everyon e i s awar e o f a U th e source s tha t

shoul dcontribut e t o thei r uncertainty , no r

ar e mos t

peopl e familia r wit hth e proces s of

actuaU y expressin g thei r uncertaint y i n probabilisti c terms . Whe n an exper t

i s aske dt o mak e probabilit y judgment s o n sociall y importan t matters , i t i s particularl y importan t tha t h e o r sh e conside r

th e relevan t

evidenc e i n asystemati c an deffectiv e manne r

an dprovid e judgment s tha t represen t

hi s o r he r opinion s well . Experimenta l psychologist s an ddecisio n analyst s hav e amasse daconsiderabl e amoun t o f dat aconcernin g th e way peopl e forman dexpres s probabilisti c judgments . Th e evidenc e suggest s tha t

whe n considerin g larg e amount s of comple x information , mos t peopl e emplo y simplifyin g heuristic s an ddemonstrat e certai n systemati c distortion s o f thought , i.e. , cognitiv e biases , whic hadversel y affec t thei r judgments . Th e purpos e o fP.5 3 47 thi s sectio n i s t o mak e you awar e o f thes e biase s an dheuristic s s o that , a s muc ha s possible , you ca n avoi dthe mi n makin g probabilit y judgments . W e wil l firs t

revie w th e mos t widesprea dbiase s an dheuristics , an dthe n offe r som e suggestion s t o hel p you mitigat e thei r

effects . A.7. 1 Sequentia l Consideratio n o f Informatio n GeneraUy , th e orde r

in whic hevidenc e i s considere dinfluence s th e fina l judgment , althoug hlogicaU y tha t shoul dno t

b e th e case . O

f necessity , piece s of informatio n ar e considere don e by on e in asequentia l fashion . However , thos e considere dfirs t

an dlas t ten dt o dominat e judgment . I n part , initia l informatio n ha s it s undu e influenc e becaus e i t provide s th e framework whic hsubsequen t

informatio n i s the n tailore dt o fit . Fo r

example , peopl e usuall y searc hfo r evidenc e t o confirm thei r initia l hypotheses ; the y rarel y loo k fo r evidenc e tha t

weigh s against

them . Th e late r

evidenc e ha s it s undu e effec t

simpl y becaus e i t

i s freshe r

in memory . Relate dt o thes e sequentia l effect s i s th e phenomeno n o f anchorin g and adjustment. Base do

n earl y partia l information , on e form s an initia l probabilit y estimat e regardin g th e even t

i n question . Thi s ancho r judgmen t

i s the n adjuste da s subsequen t informatio n i s considered . Unfortunately , suc hadjustment s ten dt o b e to o conservative . I

n othe r words , to o littl e weigh t i s attache dt o informatio n considere d subsequen t

t o th e formatio n of

th e initia l judgment . A.7. 2 Effect s o f

Memor y on Judgmen t I t i s difficul t

fo r mos t peopl e t o conceptualiz e an dmak e judgment s abou t large , abstrac t universe s o r

populations . A natura l tendenc y i s t o recal l specifi c member s an d the n t o conside r

the mrepresentativ e of th e populatio n a s awhole . However , th e specifi c instance s ofte n ar e recalle dprecisel y becaus e the y stan dou t in som e way , suc ha s bein g familiar , unusual , especiall y concrete , o r of persona l significance . Unfortunately , th eP.5 4 48 specifi c characteristic s o f thes e singular

example s ar e the n attributed , ofte n incorrectly , t o all th e member s of

th e populatio n of

interest . Moreover , thes e memor y effect s ar e ofte n combine dwit hth e sequentia l phenomen adiscusse dearlier . Fo r example , in considerin g th e evidenc e regardin g th e dose-respons e curv e of aparticula r

poUutant , yo u might naturall y firs t

thin k of astud y you o r

apersona l frien drecentl y completed . O r yo u migh t thin k of astud y you recentl y read , o r on e tha t

was unusua l an dtherefor e stand s out . Th e tendenc y migh t the n b e t o trea t

th e recaUe dstudie s a s typica l of th e populatio n o

f relevan t

research , ignorin g importan t difference s amon g studies . Subsequen t

attempt s t o recal l informatio n coul dresul t i n thinkin g primaril y of evidenc e consisten t

wit hth e initia l item s yo u though t of . A.7. 3 Estimatin g ReliabUity o f Informatio n Peopl e ten dt o overestimat e th e reliabilit y of information , ignorin g factor s suc h a s samplin g erro r

an dimprecisio n of

measurement . Rathe r

the y summariz e evidenc e in term s o f simpl e an ddefinit e conclusions , causin g the mt o b e overconfiden t in thei r judgments . Thi s tendenc y i s stronge r whe n on e ha s aconsiderabl e amoun t of intellectua l and/o r persona l involvemen t

in aparticula r

field . I

n suc hcases , informatio n i s ofte n interprete di

n away whic hi s consisten t withone' s belief s an dexpectations , result s ar e overgeneralized , an dcontradictor y evidenc e i s ignore do r underestimated . A.7. 4 Relatio n Betwee n Even tImportanc e an d ProbabUit

y Sometime s th e importanc e of events , o r

thei r possibl e cost s o r

benefits , influenc e judgment s abou t th e certaint y of

th e event s when , rationally , importanc e shoul dnot

affec t probability . I

n othe r words , one' s attitude s toward s ris k ten dt o affec t one' s abilit y t o mak e accurat e probabilit y judgments . Fo r example , man y physician s ten d t o overestimat e th e probabilit y o f ver y sever e diseases , becaus e the y fee l i t i s importan t t o detec t an dtrea t

them ; an dsimilarly , many smoker s underestimat e th e probabilit y o fP.5 5 4 9 advers e consequence s of smoking , becaus e the y fee l tha t th e odd s d o no t

appl y t o themselve s personally . A.7. 5 Estimatio n o f

Probabilitie s Anothe r limitatio n i s relate dt o one' s abilit y t o discriminat e betwee n level s of uncertainty , an dt o us e appropriat e criteri aof discriminatio n fo r

differen t

range s of probability . On e resul t o f thi s fac t

i s tha t peopl e ten dt o estimat e bot hextrem e an d mid-rang e probabilitie s in th e sam e fashion , usuall y doin g apoo r jo b in th e extremes . I t help s her e t o thin k

i n term s o

f odd s a s wel l a s probabilities . Thus , fo r example , changin g aprobabilit y estimat e fro m0.51 0 t o 0.50 1 i s equivalen t t o achang e i

n odd s fro m1.041: 1 t o 1.004:1 , bu t

achang e fro ma

n estimat e o

f 0.99 9 t o 0.99 0 change s th e odd s by afacto r of abou t 1 0 fro m999: 1 t o 99: 1 Th e close r t o th e extreme s (eithe r 0 o r 1 ) tha t on e i s estimatin g probabilities , th e greate r th e impac t

of smal l changes . A.7. 6 Recommendation s Althoug hextensiv e an dcarefu l trainin g woul db e necessar y t o eliminat e al l th e problem s mentione dabove , som e relativel y simpl e suggestion s ca n hel p minimiz e them . Mos t importan t

i s t o b e awar e of

one' s natura l cognitiv e biases , an dt o tr y consciousl y t o avoi dthem . To avoi dsequential

effects kee p i n min dtha t th e orde r

in whic hyou thin k

of informatio n shoul dno t

influenc e you r

fina l judgment . I t

may b e helpfu l t o actuall y not e on pape r th e importan t

fact s you ar e considerin g an dthe n t o reconside r the min tw o o r mor e sequences , checkin g th e consistenc y of you r judgments . Try t o kee p an ope n min d unti l you hav e gon e throug ha U th e evidence , an ddon' t le t th e earl y informatio n yo u conside r swa y you mor e tha n i s appropriate . To avoi dadvers e memory effects, defin e variou s classe s of informatio n tha t

you dee mrelevant , an dthe n searc hyou r memor y fo r

example s of

each . Don' t

restric t you rP.5 6 50 thinkin g onl y t o item s tha t stan dou t

fo r specifi c reasons . Mak e aspecia l attemp t

t o conside r conflictin g evidence , an dt o thin k o f dat atha t

may b e inconsisten t

wit ha particula r

theory . Also , b e carefu l t o concentrat e o n th e give n probabilit y judgmen t an d do not le t you r own value s (ho

w you woul dmak e th e decisio n yourself ) affec t thos e judgments . T o accuratel y estimate the reliability of information, pa y attentio n t o suc h matter s a s sampl e siz e an dpowe r of th e statistica l tests . Kee p in min dtha t dat aar e probabilisti c i

n nature , subjec t

t o element s o

f rando merror , imprecis e measurements , an dsubjectiv e evaluatio n an dinterpretation . I n addition , th e furthe r on e mus t extrapolate , o r

generalize , fro maparticula r stud y t o asituatio n of interest , th e les s reliabl e i s th e conclusio n an dth e les s certaint y shoul db e attribute dt o it . Rel y mor e heavil y on informatio n whic hyo u conside r

mor e reliable , bu t

d o no t trea t i t a s "absolut e truth. " Kee p i

n min dtha t

the importance of

an event or an outcome should

not influence its judged

probability, i t

i s rationa l t o le t

th e costlines s o r

severit y of

a n outcom e influenc e th e poin t

a t whic hactio n i s take n withrespec t t o it , bu t

no t th e judgmen t

tha t i s mad e abou t

th e outcome' s likelihood . FinaUy , in makin g probabilit y judgments , thin k primaril y i n term s of

th e measur e (probabilit y o r

odds ) wit hwhic hyou fee l mor e comfortable , bu t sometime s translat e t o th e alternativ e scale , o r eve n t o measure s o

f othe r event s (e.g, , th e probabilit y of th e even t not happening) . Whe

n estimatin g ver y smal l o r ver y larg e likelihoods , i t i s usuall y bes t t o thin k i n term s of

odds , whic har e unbounded , instea do f probabilities , whic har e bounded . Fo r

example , on e ca n mor e easil y conceptualiz e odd s of 1:20 0 tha n a probabilit y of

.005 .P.5 7 5 1 A.8 . FINA L PREPARATIO N FO RELICITATIO N O F PROBABILIT Y JUDGMENT S Th e shap e an dlocatio n o f th e dose-respons e curv e a s w e define di t abov e i s uncertain , becaus e th e existin g dat ad o no t determin e th e dose-respons e relationshi p exactly . Yet , i

f w e hav e define dth e dose-respons e curv e precisely , i n amathematica l sens e suc harelationshi p doe s exist . Ou r goa l i s t o hav e yo u represen t probabilisticall y your ow n uncertaint y abou t

th e locatio n an dshap e of thi s mathematicall y existen t function , base don you r expertis e an dth e availabl e knowledge . I n respondin g t o th e question s w e wi

U as k you , pleas e thin k

carefuU y abou t

th e relevan t

informatio n reviewe d i n th e criteri adocument , an dconsul t it , th e literature , o r you r file s a s yo u dee m appropriate . Th e previou s sectio n suggest s way s t o thin k abou t th e relevan t

data . Th e purpos e o f tha t sectio n i s t o hel p minimiz e th e biasin g effect s tha t frequentl y accompan y th e informatio n overloa dnaturall y resultin g fro mrapi dconsideratio n o f larg e amount s of comple x evidence . Yo

u ma y fin di t helpfu l t o revie w th e sectio n o r rais e question s abou t th e point s mad e i

n i t befor e w e begin . Uncertaint y abou t

th e locatio n of

ados e respons e relationshi p ca n b e represente d probabilisticaU y in tw o differen t ways . 1 . Uncertaint y abou t

th e percentages , o r

fractiles , o

f th e define d populatio n tha t

woul db e affecte db y give n blood-lea d concentrations ca n b e represente dprobabilistically . 2 . Uncertaint y abou t

th e blood-lea dconcentration s tha t woul db e require dt o affec t

given percentages, or fractiles of

the defined population ca n b e represente dprobabilistically . W e wi U concentrat e on on e way a t atime , focusin g primaril y o n th e firs t one . Fo r thes e purpose s i t

i s helpfu l t o imagin e tha t everyon e i n th e populatio n ha s a specifie dblood-lea dleve l tha t ha s becom e stabilize din th e manne r describe dabov e unde r Exposure Conditions. Then , a s aresult , som e percentag e of th e populatio n wil lP.5 8 52 suffe r th e hemoglobi n response . Sinc e th e member s o f th e populatio n ca n b e ran k ordere daccordin g t o thei r susceptibility , whe n w e spea k of aparticula r

percentag e of th e populatio n bein g affected , sa y 10% , w e mean precisel y tha t everyon e i n th e populatio n up t o an dincludin g th e perso n i n tha t fractile , i n thi s cas e th e .1 0 fractile , ha s bee n affected . Now , i n orde r

fo r u s t o determin e you r

uncertaint y abou t

th e fractile s of

th e populatio n tha t

woul db e affecte dby give n bloo dlead levels w e mus t

introduc e a definition . Le t

C b e th e give n blood-lea dconcentratio n i n question . Definition : Th e R(C)-benchmark i s th e fractil e of

th e populatio n fo r

whic ha blood-lea dleve l of

C woul dcaus e th e define dhemoglobi n healt heffect . Thus , R(C ) i s precisel y th e fractio n of th e populatio n tha t

woul dsho w arespons e i f th e entir e populatio n hadblood-lea dlevel s of C unde r th e condition s define dabove . Th e valu e o

f R(C ) fo r

agive n C i s uncertain , an dw e woul dlik e t o obtai n probabilit y judgment s fro myou abou t it s possibl e values . W e wil l elici t

you r judgment s abou t th e possibl e value s of

R(C ) b y specifyin g aparticula r percentag e o r

fractil e r , an d havin g you conside r

ho w likel y i t

i s tha t th e R(C)-benchmar k

i s les s tha n tha t

value . T o hel p you mak e you r

probabilit y judgments , w e wi U mak e us e of

adevic e calle da probabilit y wheel , whic hha s adjustabl e sector s o f blu e an dorange . W

e ca n rea don th e bac k of th e whee l th e percentag e of th e whee l tha t

i s eac hcolor . I

n makin g you r judgment s you ar e t o imagin e tha t th e whee l i s aperfectl y fai r rando mdevice , an dtha t therefor e th e probabilit y of it s stoppin g wit hth e pointe r o n blu e i s exactl y represente d b y th e relativ e are atha t

i s blue . W e wi U the n procee da s follows . Fo r eac hbloo dlea dconcentratio n C , w e wil l specif y aparticula r

fractil e r , an dals o se t th e probabilit y whee l t o hav e aspecifi c relativ e are ao

f blue . The n you ar e t o conside r carefuU y th e question :P.5 9 5 3 Do you conside r

i t mor e probabl e tha t

th e R(C)-benchmark

i s les s tha n r

or tha t th e whee l woul dsto p withth e pointe r o n blu e (on arando mspin) ? Yo u ca n giv e on e of

thre e responses : (1 ) Yo u judg e i t

t o b e mor e probabl e tha t R(C ) i s les s tha n r ; (2) Yo u judg e i t

t o b e mor e probabl e tha t th e whee l woul dsto p wit h th e pointe r

in blue ; (3 ) Yo u canno t judg e eithe r

even t a s mor e probabl e tha n th e other . Fo r aparticula r

concentratio n C an dfractil e r , som e whee l setting s wil l hav e a smal l enoug hrelativ e are ao f blu e tha t

you wil l fee l confiden t makin g respons e (1) . Othe r whee l setting s wil l hav e alarg e enoug hrelativ e are aof blu e tha t

you wi U fee l confiden t makin g respons e (2) . Th e intermediat e setting s wi U b e mor e difficul t

t o judge . However , fo r

afixe dbloo dlea dC , an dpopulatio n respons e rate , r , w e wil l manipulat e th e whee l setting s t o fin dth e on e fo r whic hyo u fee l mos t

comfortabl e makin g respons e (3) . Onc e w e hav e determine dth e poin t a t whic hyou ar e mos t

comfortabl e wit h respons e (3) , an dstil l focusin g on th e give n bloo dlea dC , w e wi U specif y ane w fractil e r' , rese t th e whee l t o ane w relativ e are aof blue , an dprocee dt o obtai n you r judgment s regardin g th e probabilisti c relatio n betwee n r ' an dth e R(C)-benchmark . Thi s wil l continu e fo r

th e give n C unti l w e hav e obtaine dcomparison s of variou s fractile s t o th e benchmark

R(C) . The n w e wil l hav e elicite don e of th e probabilisti c representation s of your uncertaint y abou t

th e dose-respons e relationshi p w e need . W e wil l obtai n th e nex t probabilisti c representatio n b y specifyin g ane w C an dcontinuin g a s before . W e wil l d o thi s fo r anumbe r

o f value s of

C . I t frequentl y happen s tha t a n expert' s judgment s alte r somewha t

ove r th e cours e of asessio n suc ha s this , a s h e o r sh e consider s th e evidenc e fro mvariou s perspective s an dthink s abou t

th e variou s response s calle dfor . Hence , w e wil l grap hyou r response s and , a t appropriat e times , sho w the mt o you fo r you r consideratio n an dcomparison . A t thes e time s you may wis ht o chang e som e of th e judgment s yo u gav e earlier .P.6 0 54 Also , althoug hth e probabilit y judgment s ar e entirel y you r own , w e mus t introduc e on e logica l constraint : namely , tha t you r final judgment s ar e coheren t i n a sens e tha t

w e ca n explai n a s w e g o along . I t i s quit e commo n fo r

initia l judgment s t o exhibi t som e incoherencies . Tha t i s anothe r

reaso n w e wi

U conside r

togethe r th e graph s of your judgments . Our

objectiv e i s t o obtai n acoheren t se t of judgment s tha t

represen t you r opinion s we

U b y th e en dof

thi s elicitatio n process . Yo u ar e no t expecte dt o giv e u s suc hase t immediately . Al l of

thi s wi U becom e cleare r

a s w e g o along . W e shoul demphasiz e tha t th e judgment s w e ar e askin g you t o mak e ar e no t simpl e ones , no r

of cours e ar e ther e know n correc t answers . Rather , w e wan t you r bes t an dmos t

considere djudgmen t

i n ligh t of th e availabl e relevan t scientifi c data , Therefore , pleas e reflec t on th e availabl e dat acarefuUy , feelin g fre e t o consul t th e lea d criteri adocumen t

o r othe r source s a s you wis ha s you formulat e you r judgments . (Encod e judgment s fo r

tw o concentration s of on e dose-respons e function , the n continu e wit hinstructions. ) Recal l tha t

uncertaint y abou t

th e dose-respons e functio n ca n als o b e expresse din term s o f th e blood-lea dconcentratio n necessar y t o produc e th e effec t in agive n percentag e o r

fractil e of

th e populatio n r , i

f th e entir e populatio n ha dth e sam e blood

lea dconcentration , stabilize da s th e resul t of exposur e in th e manne r discusse dearlier . Mor e specifically , proceedin g a s before , le t u s introduc e adefinition : Definition : Th e C(R)-benchmark i s th e blood-lea dleve l tha t woul dcaus e th e define dhemoglobi n healt heffec t in th e perso n a t

fractil e Rof

th e population . Th e valu e of

C(R) i s uncertai n fo r agive n R , an dw e wan t t o obtai n you r judgment s abou t

it s possibl e values . Analogousl y t o wha t w e hav e alread y don e in obtainin g th e R(C)-benchmark , fo r agive n Rw e wi

U specif y concentration s c an das k youP.6 1 55 t o conside r

how likel y i t

i s tha t th e C(R)-benchmar k

i s les s tha n th e specifie dc . Mor e specificaUy , utilizin g th e whee l a s before , w e wi U as k you th e question: Do you conside r

i t mor e likel y tha t

th e C(R)-benchmark

i s les s tha n c , or

tha t th e whee l woul dsto p wit hth e pointe r on blu e (on arando mspin) ? Yo u ca n giv e on e of

thre e responses : (1 ) Yo u judg e i t mor e probabl e tha t

C(R) i s les s tha n c ; (2) Yo u judg e i t

mor e probabl e tha t

th e whee l woul dsto p wit hth e pointe r in blue ; (3 ) Yo u canno t judg e eithe r

even t a s mor e probabl e tha n th e other . Afte r determinin g th e whee l settin g a t whic hyo u fee l mos t

comfortabl e wit h respons e (3) , w e wil l specif y anew c an drepea t

th e process . A s before , fo r eac hRw e wi U elici t you r comparativ e judgment s fo r variou s value s o

f c .5 6 APPENDI X B DETAILE D MATHEMATICA L FORMULATIO N B. 1 OUTLINE Thi s appendi x present s detaile ddiscussion s of th e mathematica l technique s tha t ar e summarize din Sec . 3 o f th e report . Th e technique s ar e use dt o represen t th e expert' s probabilit y judgment s abou t dose-respons e relationships , an dt o combin e thes e judgment s withablood-lea ddistributio n t o obtai n aris k estimate . First , basi c definition s an dnotatio n ar e introduced . The n th e normal-on-log-odd s functio n i s discusse da s adistributio n t o b e fitte dt o th e expert' s probabilit y judgment s tha t meet s certai n o

f th e criteri aspecifie di n Sec . 3.1 . Method s ar e presente dfo r obtainin g least-square s estimate s of th e parameter s of

anormal-on-log-odd s distributio n an dfo r assessin g th e goodnes s o f fit . A famil y o

f equal-varianc e normal-on-log-odd s distribution s ma

y sacrific e somethin g i n term s o

f fi t t o th e judgments , bu t

i t meet s al l th e criteri aspecifie di

n Sec . 3.1 . Thi s famil y i s presente dnext , alon g wit hmethod s fo r obtainin g least-square s estimate s of it s parameter s an dfo r assessin g it s goodnes s of fit . Finally , technique s ar e presente dfo r derivin g ris k

estimate s b y combinin g ablood-lea d distributio n withan expert' s probabilit y judgment s abou t dose-respons e relationships , wher e th e latte r

i s represente deithe r

b y independen t

normal-on-log-odd s distribution s o r by afamil y of

equal-varianc e normal-on-log-odd s distributions . B. 2 BASI C DEFINITION S AN D NOTATIO N Thi s subsectio n present s definition s an dnotation . I t i s assume dtha t

th e reade r

i s familia r wit hth e basi c concept s o f probabilit y theor y (i.e. , definition s of rando m variables , probabilit y densit y functions , expectatio n an dhighe r moments , etc.) . Fo r uncertai n quantit y X , le t f x 0 0 = judgmenta l probabilit y densit y functio n (PDF ) fo r X F x (x 0 ) = cumulativ e judgmenta l densit y functio n (CDF ) fo r XP.6 3 5 7 x 

### = J °f

x 

### (x)dx . Thu s Fv(x Q ) i s th e judgmenta l probability * tha t X i s les s tha n o r

equa l t o x Q . Furthermore , le t

u „ = E(X ) = expecte dvalu e o

f X f x f v ( x )d x J o X o o

o £= V(X ) = varianc e o

f X

= J ( x - E(X)) 2 f v ( x )d x . J o X o o Th e uncertai n quantitie s of interes t i n of thi s pape r

ar e populatio n respons e rates , denote dR , a t

eac ho f severa l blood-lea dlevels , denote dL . Throug hencodin g session s withrecognize dexperts , judgment s representin g acumulativ e densit y functio n (CDF ) fo r

arespons e rat e Rgive n apopulatio n blood-lea dleve l Lwi U b e elicite d(i.e. , judgment s representin g FR/L(R) ) f° r anumbe r

of blood-lea dlevels . Take n together , thes e wi U provid e afamil y o

f probabilisti c relationship s a s idealize di n Fig . 3.1 , an d alread y presente dmor e realisticaU y in Figs . 2. 1 an d2.2 . Eac hcurv e i n Fig . 3. 1 represent s th e judgmenta l probabilit y tha t th e populatio n respons e rat e ( afractio n betwee n 0 an d1 ) wil l b e les s tha n o r equa l t o aparticula r

valu e R , give n ablood-lea d leve l of L , fo r aweU-define dadvers e healt h affect , awell-define dpopulatio n of interest , an dunde r weU-define dexposur e conditions . As alread y discussed , i t wi U generall y b e necessar y t o interpolat e betwee n specificall y assesse dpoint s i n performin g ris k calculations . Suc hinterpolatio n i s bes t

•Hereafter , a

U probabilitie s referre dt o shal l mean judgmenta l probabilities , eve n i f no t explicitl y stated .P.6 4 5 8 accomplishe db y fittin g afunctio n t o th e judge dprobabilities . Th e nex t subsectio n describe s ametho dfo r

doin g this . B. 3 TH E NORMAL-ON-LOG-ODD S FUNCTIO N Becaus e th e probabilisti c judgment s abou t th e dose-respons e relationship s appea r generall y t o b e "S-shaped " ove r th e close d[0,1 ] interval , a s demonstrate din Figs . 2. 1 an d 2.2 , a s we

U a s b y th e judgment s o f al l th e healt hexpert s encode db y Wallsten , Forsyth , an dBudesc u (1983) , i t

ca n b e difficul t

t o represen t

the mwit hclosed-for mmathematica l functions . On e particularl y usefu l functio n (whic hi s relativel y eas y t o wor k with ) i s th e normal-on-log-odd s distribution . Th e normal-on-log-odd s distributio n i s obtaine db y 4 fittin g anorma l distributio n t o th e natura l lo g o f th e odd s implie db y th e populatio n respons e rate s R . Thus , th e followin g relationship s ar e defined : x= x^ , o < x < Y = ln(X) , - » < Y < » wher e X i s th e odd s variabl e an dY i s th e log-odd s variable . I t i s assume dtha t

Y i s normall y distribute dwit hmea n u an dvarianc e o . Th e degre e t o whic hth e assumptio n i s appropriat e ca n b e teste dwit heac hse t of judgments . I

f Y i s normall y distributed , the n X i s lognormall y distributed ; an dwhil e ther e i s n o closed-for mexpressio n readil y availabl e fo r

th e distributio n on R , a U probabilisti c an dstatistica l result s o f interes t o n Rca n b e obtaine dthroug hth e distributio n o n Y sinc e pr( R< r ) = pr( X < ^ ) = pr( Y < In^) ) an d 

### pr( Y < 

### y ) = 

### pr( Z < ^ 

### !l_ ) = 

### *{I° !ll )

—o v—a, , ^ OA,P.6 5 5 9 wher e pr(- ) denote s probabilit y an dZ i s th e uni t norma l rando mdeviat e fo r whic h*(Z) i s extensivel y tabulate dan davailabl e in compute r libraries . Thu s

y Tabl e B. l summarize s quantitie s of interes t fo r thes e variables . Th e parameter s u an daca n b e estimate din aleast-square s sens e fro man assesse ddistribution . Le t th e n assesse dpoint s fo r aCD Fb e denote d(Rj , Fj ) fo r i=l , ... , n wher e R

j i s th e populatio n respons e rat e an dF j i s th e associate dcumulativ e judgmenta l probability . Leas t square s estimate s u an do fo r u an daca n b e obtaine db y definin g

. 2 r f

y i " *\ 2 E 

### ' = 

### l[

z 

### , - _ i y takin g derivative s wit hrespec t t o u an do an dequatin g th e result s t o zero ,

3 U

3E i 3 a . = 0 u = u

. = 0 o= oP.6 6 6 0

TABL E B. 1 Summar y o f Variable s Pertainin g t o th e Normal-On-Log-Odd s Functio n

Defining Equation s Distribution 3

Mean b

Varianc e

Media n

Mod e Y

Y= ln(X ) N(u,o 2 )



### u

o 2

u



### u 2

a X Variabl e X X= e Y X= R/(l-R ) A(u, o )

_u+a 2 / 2 u = e X 2 2 2u + o r ° = e l e -

e » 2 e u- a •1 ) R



### R »-5 _ R1+ X 7 2 uo X X X(1+ u /

X 2 f 1 ^ 4 ° x^1+ u ]

X u e l*e u 2 u- o e 2 l*e M ~ °

a N(u,a 2 ) denote s th e norma l PDF ; A(u',o 2 ) denote s th e lognorma l PDF ; a closed-for mexpressio n fo r th e PD F fo r R i s no t obtainable . Mea n an d varianc e expression s fo r R ar e approximate ; the ywer e derive d vi a th e metho d o f moments ; othe r estimate s ca n b e obtaine d vi a numerica l method s o n F Y = N(u,o 2 ) .P.6 7 6 1

an dsolvin g fo r

u an da , wher e

z . = •" 1 (F. ) 1 1 i s th e valu e o

f th e uni t norma l variat e tha t

correspond s t o acumulativ e probabilit y of Fj , an dy } = ln(Rj/(l-Rj)) . Th e result s ar e 

## ( I 

## v\) 

## il 

## « L 

## ) - (I 

## /-)( ! JO u =

a= 

### (I y j - "

M 

### il'i) (B.l )

(B.2 ) wher e a U o f th e summation s ar e fo r i=l , ... , n . Thes e estimate s ca n als o b e obtaine db y linearl y regressin g th e Z j o n th e yj ; o i s th e reciproca l of

th e slop e o

f th e regressio n equatio n an dG i s th e y-intercept , i.e. , th e valu e of y correspondin g t o z = 0 i n th e regressio n equation . Sinc e mjudgmenta l distribution s ar e assessed , on e fo r eac hblood-lea dleve l j fo r j = 1 , ... , m , ther e ar e m mean s (ui , ... , u m ) an dstandar ddeviation s (a^ , ... , a m ) , on e pai r fo r eac hof th e judge ddistributions . Th e goodnes s of

fi t of eac htransformed , assesse ddistributio n i s give n b y th e standar dregressio n r 2 statisti c

2 r LK

l l ' 1 - — ^ 3^

- (B.3 ) 

### K« i 

## - z ) wher e z denote s th e estimate dZ

- value s {i.e. , z ^ = (y ^ - u)/o } an dZ i s th e mea n o f Z . I f r " i s sufficientl y high , the n i t ca n b e claime dtha t

th e normal-on-log-odd s distributio n withparameter s u ; an do ; describe s th e judgment s wel l fo r lea dleve l j . Thi s distributio n ca n the n b e use dfo r interpolation .P.6 8 6 2 To summarize , formula s wer e presente dt o calculat e th e parameter s of anormal

on-log-odd s distributio n whic hca n b e use dt o represen t ajudge dprobabilit y distributio n ove r populatio n respons e rat e fo r agive n blood-lea dlevel . Goodnes s of fi t i s give n b y th e regressio n r 2 statistic . Th e normal-on-log-odd s distributio n variabl e an dit s PD Fca n b e use dt o estimat e judgmenta l probabilitie s fo r th e respons e rat e variabl e Rin orde r t o calculat e ris k

estimates . B. 4 REPRESENTIN G A PROBABILISTI C DOSE-RESPONS E SURFAC E WIT H A FAMIL Y O F EQUA L VARIANC E NORMAL-ON-LOG-ODD S DISTRIBUTION S I f th e o ; ar e approximatel y equal , i t i s desirabl e t o se t

the mequa l t o asingl e poole dvalu e befor e proceedin g furthe r (i.e. , se t

o : = o ' = a = o fo r

al l j) , becaus e thi s assure s tha t

th e derive ddistribution s a t tw o differen t

lea dlevel s neve r

cros s on e another . (Crossin g woul dimpl y tha t exceedin g aspecifie drespons e rat e i s judge dmor e probabl e a t

blood-lea dleve l L , tha n a t L« , wher e L ? i s greate r tha n L,. ) Becaus e of

th e manne r i n whic hjudgment s wer e obtaine dfro mth e pilo t subjects , crossing s di dno t occur . Furthermore , judgment s wil l b e obtaine dfro mth e healt hexpert s ove r awide r probabilit y rang e s o tha t crossing s woul donl y b e possibl e a t cumulativ e probabilitie s ver y nea r t o 0 an d1 . However , eve n crossing s suc ha s thes e woul drepresen t an inconsistenc y th e expert s ough t

t o avoid , an dtherefor e shoul dnot b e allowed . A poole destimat e of

o , denote da ' ca n b e obtaine db y firs t subtractin g of

f th e mea n u. - fo r eac hse t o f assesse dpoints ,

y . . = y . . - u. , i = l » ... , n. , j-1 , ... , m wher e n - i s th e numbe r

o f assesse dpoint s fo r

Fn/ , an dy- j i s an assesse dpoin t o n ^R/L- " ^" h e y ' • ar e the n use dt 0 ca l c u l a t e amean (whic hshoul db e ver y clos e t o zero ) an davarianc e (whic hwi U b e th e pooled , leas t

square s estimat e a* ) usin g Eqs . B. l an dP.6 9 63 B.2 , respectively . FinaUy , th e individual , least-square s estimate s of th e mean s ar e recalculate db y notin g tha t Eq . B. 2 ca n b e repeatedl y solve dfo r eac hu i i n term s of

o' . Denot e th e result s of

thes e las t

step s a s uT , ••• , u' . Goodnes s of fi t agai n ca n b e 1 m calculate dwit hEq . B.3 . I f afunctiona l relationshi p ca n b e establishe dbetwee n blood-lea dlevel s an dth e u ' , ajudgmenta l probabilit y densit y functio n ove r Rca n b e calculate dfo r an y blood-lea d level , shoul dtha t

b e necessar y fo r

interpolatin g betwee n blood-lea dlevels . Thi s continuu mof

densit y function s i

n essenc e i s aprobabilit y surfac e ove r th e dose-respons e plane . B. 5 COMBININ G BLOOD-LEA D DISTRIBUTION S WIT H PROBABILIT Y JUDGMENT S ABOU T DOSE-RESPONS E FUNCTION S Usin g eithe r

th e equal-varianc e o r

th e best-fittin g normal-on-log-odd s function s fi t t o th e judgment s i

n th e manne r

describe dabove , i t

i s possibl e t o estimat e dose

respons e function s tha t

correspon dt o an y give n judgmenta l cumulativ e probability . Fo r example , amedia n dose-respons e curv e base don a n expert' s probabilit y judgment s i s obtaine db y determinin g th e respons e rat e tha t correspond s t o F^ ^ = 0.5 0 a t eac hbloo d lea dL . Th e dose-respons e curve , suc htha t ther e i s a5 % chanc e of th e tru e respons e rat e lyin g abov e th e curv e a t an y give n bloo dlead , i s obtaine db y determinin g th e respons e rat e tha t

correspond s t o F^ ^ = 0.9 5 a t eac hL , an ds o forth . Le t Dp(L ) represen t th e dose-respons e curv e tha t correspond s t o cumulativ e probabilit y F . Any blood-lea ddistributio n ca n the n b e combine dwit hth e dose-respons e curv e correspondin g t o judgmenta l cumulativ e probabilit y F , Dp(L) , t o yiel dan estimat e of th e overal l fractio n o

f th e populatio n Rsufferin g th e advers e healt heffec t wit hcumulativ e probabilit y F . I

f ther e i s als o uncertaint y abou t th e blood-lea ddistribution , tha t mus t b e take n int o accoun t

a s well , bu t

tha t proble mwi

U no t b e treate din thi s report .P.7 0 6 4 W e wi U conside r

her e onl y th e empirica l blood-lea ddistributio n fro mth e NHANES-I

I survey . Th e distributio n i s presente di n discret e for

mwithinterval s of

0-1 0 ug/dl , 10-1 9 ug/dl , ... , 60-6 9 ug/dl . Le t P(L: ) = proportio n o

f th e populatio n in th e interva l centere da t L

fo r j=l , ... , 6 . I

f judgmenta l probabilitie s fo r dose-respons e relationship s ar e encode d conditiona l on blood-lea dlevel s L: , the n th e NHANES-I I distributio n ca n b e combine d wit han y D F (L ) t o obtai n th e overal l estimat e of Rwit hcumulativ e probability . F , accordin g t o

i 6

R= F X (R ) = I

P(L.)- D (L. ) (B.4 ) j = 1 J J wher e F^ 1 (R) i s th e invers e o f F^(R) , th e derive dcumulativ e probabilit y distributio n fo r Rlik e tha t

indicate di

n Fig . 3.2 . I

f fo r som e reaso n judgmenta l probabilitie s ar e no t encode dconditiona l o

n aparticula r

Ls , Dp(L: ) ca n b e obtaine dby interpolatio n o n th e probabilit y surfac e derive dove r th e dose-respons e plane. * Fo r example , i

f th e P(L: ) ar e combine dwit hD g 5Q(L: ) in Eq . B.4 , th e resul t i s a media n estimat e of

th e fractio n o

f th e populatio n sufferin g th e advers e healt heffect . I n general , combinin g th e blood-lea ddistributio n withadose-respons e curv e correspondin g t o judgmenta l cumulativ e probabilit y Fyield s an F-fractil e estimat e of th e fractio n of th e populatio n sufferin g th e healt heffect . By carryin g ou t thi s calculatio n fo r

al l value s of F , 0 < F< 1 , an dplottin g eac hFa s afunctio n of th e derive dR , ajudgmenta l CD Fi s obtaine dove r

affecte dfractio n of

th e population , a s illustrate din Fig . 3.2 . W e cal l thi s CD Faderive dris k

distribution .

•I t shoul dals o b e note dtha t Eq . B. 4 i s easil y generalize dshoul dablood-lea ddistributio n b e presente di

n acontinuou s o r

adifferen t

discret e form .P.7 1 6 5 Figur e 3. 2 represent s ahypothetica l derive dris k distributio n fo r

aparticula r populatio n an daparticula r advers e healt heffect , derive dfro man expert' s probabilit y judgment s abou t

dose-respons e function s combine dwithablood-lea ddistributio n accordin g t o Eq . B. 4 fo r al l value s of

F , 0 < F< 1 . On e ca n rea dfro mi t amedia n estimat e of

th e affecte dfractio n o f th e population , a95 % credibl e interva l fo r tha t fraction , o r

whateve r

i s desired .P.7 2 6 6 APPENDI X C DETAILE D ILLUSTRATIVE CALCULATION S Th e analytica l method s derive di n Appendi x B shoul db e applie dt o th e judgment s o f healt hexpert s onl y afte r th e method s hav e bee n teste dan dshow n t o describ e th e judgment s withsufficien t accuracy . I

n thi s appendi x best-fittin g normal-on-log-odd s distribution s an dequal-varianc e normal-on-log-odd s distribution s ar e derive dfo r th e tw o pilo t subjects , an dthei r

fit s ar e assessed . I

n addition , ris k

estimate s ar e derive dbase d o n th e probabilit y judgment s fro meac hpilo t subject . C. l FITTIN G NORMAL-ON-LOG-ODD S DISTRIBUTION S T O JUDGMENT S C.l. l PUo tSubjec t1 Recal l tha t

th e probabilit y judgment s fro mthi s subjec t ar e show n i

n Tabl e 2. 1 an dFig . 2.1 . Transformin g th e judgment s vi ath e normal-on-log-odd s functio n yield s th e result s in Tabl e C.l . Applyin g Eqs . B. l an dB. 2 t o thes e dat ayield s th e estimate s U J an dOJ ( j = 1 , ... , 6) liste di

n Tabl e C.2 , a s wel l a s th e regressio n r value s show n in th e table . Figur e C. l show s th e CDF s fro mth e best-fittin g normal-on-log-odd s distribution s an dth e assesse dvalues . Th e hig hregressio n r-square s in Tabl e C. 2 (th e smalles t i s 0.97 ) indicat e ver y goo dfit s of

th e separat e normal-on-log-odd s functions . I f th e transformation s departe d appreciabl y fro mnormaU y distribute drando mvariables , the n th e regressio n r-square s woul db e muc hlower . Thes e goo dfit s ca n b e see n visuall y in Fig . C.l , wher e th e normal

on-log-odd s function s closel y matc hth e assesse dprobabilities . However , not e i

n Fig . C. l tha t

th e CD Ffo r

L= 3 5 crosse s thos e fo r L= 25 , 45 , an d5 5 ug/dl . Thi s resul t occurred , because , a s alread y discusse din Sec . 2.4.1 , probabilitie s wer e no t

encode dfo r

asufficientl y ful l rang e fo r L= 2 5 an d4 5 ug/dl . I

f th e lowe r probabilitie s ha dbee n encode dfo r thes e tw o lea dlevels , the n th e subjec t woul dP.7 3

6 7 TABL E C. l Normal-On-Log-Odd s Distributio n Transformatio n o f ProbabUit

y Judgments i

n Tabl e 2. 1 fo rSuojec t1

Blood-Lea d Level s R aY b Z cRY Z 5 ug/d l 1 5 ug/d l 2 5 ug/d l 0.0 2 -3.8 9 -1.0 4 0.0 2 -3.8 9 -1.6 5 0. 2 -1.3 9 -0.6 1 0.0 4 -3.1 8 0.3 9 0.0 4 -3.1 8 -0.3 1 0. 3 -0.8 5 0.0 5 0.0 6 -2.7 5 0.8 1 0.0 6 -2.7 5 0.1 3 0. 4 -0.4 1 1.1 3 0.0 8 -2.4 4 1.2 3 0.0 8 -2.4 4 0.5 2 0. 5 0.0 0 1.8 8 0.1 0 -2.2 0 2.0 5 0.1 0 -2.2 0 0.9 5 3 5 ug/d l 4 5 ug/d l 5 5 ug/d l 0.1 0 -2.2 0 -1.4 7 0.3 5 -0.6 2 -0.8 4 0.3 0 -0.8 5 -1.8 8 0.2 6 -1.0 5 -0.5 8 0.4 8 -0.0 8 -0.0 5 0.4 4 -0.2 4 -0.7 1 0.4 2 -0.3 2 -0.1 0 0.6 1 0.4 5 0.5 5 0.5 8 0.3 2 -0.1 3 0.5 8 0.3 2 0.8 4 0.7 4 1.0 5 2.0 5 0.7 2 0.9 4 0.5 8 0.8 6 1.8 2 2.3 3

a R i s respons e rate .

C Z = 4 (F) , wher e F correspond s t o R fo r each blood-lea d leve l a s listed in Tabl e 2.1 .P.7 4 68 TABL E C. 2 Paramete rEstimates , Goodnes s o f Fi t Statistics , an

d Mea n Respons e Rate s fo rBest Fittin g Normal-On-Log-Odd s Distribution s Fitte d t o th e Judgments o f Subjec t1

Bloo d Regressio n

" 2 ~ a Lea d u j O j r R 1 5 ug/d l -3.3 0 0.5 9 0.9 7 0.0 4 2 1 5 ug/d l -2.8 4 0.6 7 0.9 9 0.0 7 3 2 5 ug/d l -0.9 9 0.5 4 0.9 8 0.2 8 4 3 5 ug/d l -0.4 4 1.1 3 0.9 7 0.1 9 5 4 5 ug/d l -0.0 6 0.5 9 0.9 7 0.4 7 6 5 5 ug/d l 0.3 7 0.6 7 0.9 8 0.5 6

a R : represent s the mea no f the respons e rat e distributio na t eac ho f th e lea d levels ; th e formul a fo r calculatin g R - i s liste d i n Tabl e B.l . J hav e modifie dhi s judgment s t o avoi dcrossovers , possibl y lowerin g th e judgment s a t L= 3 5 ug/dl , an dth e modele ddistribution s woul dals o no t hav e crosse d(except , a s alread y indicated , possibl y a t

value s ver y clos e t o 0 an d1) . Equal-varianc e (i.e. , th e pooled-varianc e estimate ) normal-on-log-odd s distribution s wil l no t

demonstrat e th e crossovers , an dindee dappea r rathe r consisten t withth e probabilit y judgment s othe r tha n thos e a t

L= 3 5 ug/dl . I t

ca n b e see n i n Tabl e C. 2 that , excep t

a t L= 3 5 ug/dl , th e O J rang e fro m0.5 4 t o 0.67 . Thus , ther e i s goo d reaso n t o determin e apoole destimat e fo r o 2 an dt o fi t

th e equal-varianc e normal-on

log-odd s distributions . Th e poole destimat e o' , calculate da s describe din Sec . B.4 , i s 0.71 , wit ha P poole dregressio n r * valu e o f 0.94 . Th e revise dleast-square s estimate s of th e means , u ' , base don apoole dvarianc e ar e liste di n Tabl e C.3 . A s expected , th e tw o estimate s6 9 P.7 5

J 2 ra ja o u a. Si > 3 E 3 U



### 0. 4 0. 8 

## 1. 0 Populatio n Respons e Rate , Expresse dA s Fractio n o f

Populatio n Affected , R

FIGURE C. l Best-Fittin g Normal-On-Log-Odd s CDF s an d th e ProbabUit y Judgment s o f Subjec t1 Tha tWer e Previousl y Show n i n Fig . 2. 1P.7 6 7 0 TABL E C. 3 Paramete rEstimates , Goodnes s o f Fi t Statistics , an

d Mea n Respons e Rate s fo rth e Equal Variance Normal-On-Log-Odd s Distribution s Fitte d t o th e Judgments o f

Subjec t1

j 1 2 3 ' 4 5 6 Bloo

d Lead 5 ug/d l 1 5 ug/d l 2 5 ug/d l 3 5 ug/d l 4 5 ug/d l 5 5 ug/dl . u '

J -3.3 8 -2.8 4 -1.0 9 -0.5 8 -0.1 1 0.3 7 0 '

P 0.7 1 0.7 1 0.7 1 0.7 1 0.7 1 0.7 1 Revised 3

2 r Value s 0.9 5 0.9 9 0.9 3 0.6 2 0.9 4 0.9 8 

## I .

J 0.0 4 0.0 7 0.2 6 0.3 5 0.4 5 0.5 5

a Tha t is , fo r th e equal-varianc e normal-on-log

odd s functions . R ' represent s th e mea n of th e respons e rat e distributio n a t

eac ho f th e lea dlevel s fo r th e equal-varianc e normal-on-log-odd s _ functions ; th e formul afo r calculatin g R ' i s liste di n Tabl e B.l . J

o f u (comparin g Table s C. 2 an dC.3 ) do no t diffe r substantiall y fro meac hothe r excep t fo r th e L= 3 5 ug/d l case . Similarly , regressio n r-square s fo r th e equal-varianc e normal

on-log-odd s function s ar e abou t th e sam e a s thos e fo r

th e best-fittin g normal-on-log-odd s function s excep t

fo r L= 3 5 ug/dl . Th e CDF s fro mth e fitte dequal-varianc e distribution s hav e bee n show n i n th e bod y o f th e repor t

a s Fig . 4.1 . Als o liste din Table s C. 2 an dC.3

- ar e th e mean value s of th e respons e rat e variable s (

Ran dR' , respectively ) fo r th e best-fittin g an dequal-varianc e normal-on

log-odd s functions . Not e tha t i n Tabl e C.2 , th e best-fittin g function s resul t in amea n a t L= 3 5 ug/d l tha t

i s lowe r tha n a t L= 2 5 ug/dl . However , fo r th e equal-varianc e fits , th e mean values , th e R ' i n Tabl e C.3 , increas e monotonicall y withincreasin g blood-lea d levels , a s the y shoul don rationa l grounds . Thes e consideration s suppor t th e appropriatenes s of

th e equal-varianc e mode l fo r describin g thi s subject' s judgments .7 1 A fina l interestin g representatio n o f thi s subject' s judgment s wa s show n a s Fig . 4.2 , base don th e equal-varianc e normal-on-log-odd s distribution s i n Tabl e C.3 . Th e curv e labele d0.50-cumulativ e probabilit y i s th e dose-respons e curv e D g -Q(L) , plottin g media n respons e rat e r ' a s afunctio n of bloo dlea dleve l L ; , wher e r ' = s— , an d

- »] J u 1 + e J th e u ' ar e thos e liste di n Tabl e C.3 . Thi s i s adose-respons e curv e representin g th e media n respons e rat e a t

eac h lea dlevel , basedo n th e judgment s of

subjec t 1 . The " dose-respons e curve s fo r 0.05

- an d0.95-cumulativ e probabilities , DQ.OS^ ) an dD g gc(L) , respectively , ar e calculate di n an analogous fashio n t o tha t fo r On.50 ^ an dtogethe r

indicat e 90 % credibl e interval s on respons e rates . Tha t is , th e tw o curve s indicat e th e rang e o

f respons e rate s a t

an y give n lea dleve l tha t ha s a0.9 0 probabilit y o f containin g th e tru e respons e rat e base don subjec t l' s judgments . Not e th e S shap e of al l thre e dose-respons e function s (especiall y th e to p two) , mirrorin g th e subject' s belief , discusse dearlier , o

f athreshold . To summarize , bot hth e best-fittin g an dth e equal-varianc e normal-on-log-odd s distribution s fi t

th e subject' s judgment s ver y well . Th e latte r ha s no inconsisten t implication s an dfit s th e judgment s almos t a s wel l a s doe s th e former , suggestin g tha t i t i s suitabl e fo r

us e in obtainin g ris k

estimates . Calculatio n o f ris k estimate s base do n pilo t subjec t l' s judgment s wi

U b e iUustrate dfoUowing furthe r evaluatio n of

th e normal

on-log-odd s distributio n on th e othe r subject' s judgments . C.1. 2 PUotSubjec t2 A s fo r subjec t 1 , Eqs . B. l an dB. 2 wer e use dt o fi t normal-on-log-odd s distribution s t o th e encode dprobabilit y judgment s fo r subjec t 2 tha t ar e liste din Tabl e 2. 2 an dshow n in Fig . 2.2 . Th e resultin g estimate s u ; an da; , a s wel l a s th e regressio n r value s ar e show n i

n Tabl e C.4 . Figur e C. 2 show s th e CDF s fro mth e fitte dnormal-on

log-odd s distribution s an dth e assesse dvalues . Recal l tha t judgment s wer e obtaine donl yP.7 8



## 1. 0 7 2



### 0. 8 0. 4 

## 0. 6 0. 8 

## 1. 0 Populatio n Respons e Rate , Expresse d A s Fractio n o f

Populatio n Affected . R

FIGURE C. 2 Best-Fittin g Normal-On-Log-Odd s CDF s fo rth e ProbabUit y Judgment s o f Subjec t2 Tha tWer e Previousl y Show n i n Fig . 2. 27 3 TABL E C. 4 Paramete rEstimates , Goodnes s o f Fi t Statistics , and Mea

n Respons e Rate s fo rth e Normal-On-Log-Odd s Distribution s Fitte d t o th e Judgments o f

Subjec t2l-l. 1 2 3 Blood Lead 1 5 ug/d l 3 5 ug/d l 5 5 ug/d l 

# h

-1.2 8 -0.6 7 -0.0 5 

# h

0.5 1 0.6 9 0.8 9 Regressio n

r 0.9 7 0.9 9 0.9 5 

# V

0.2 3 0.3 4 0.4 1

a R - represent s th

e mea no f th

e respons e rat e distributio na

t eac ho f the lea

d levels ; th e formul a for calculatin g R - i s liste d i n Tabl e B.l . fo r blood-lea dlevel s of

15 , 35 , an d5 5 ug/dl . Again , th e hig hr-squar e value s indicat e goo dfit s o

f normal-on-log-odd s function s t o th e judgments . Not e th e stron g linea r relationshi p betwee n blood-lea dleve l an dbot hu an da, whic hwa s no t th e cas e fo r subjec t 1 . Becaus e judgment s wer e not obtaine da t

thre e o f th e desire dblood-lea dlevel s (du e t o alac k

of tim e wit hsubjec t

2) , i t was necessar y t o estimat e the min orde r t o calculat e th e desire dris k estimates . Thi s was don e b y linearl y regressin g th e u j an do ; i n Tabl e C. 4 ont o th e bloo dlea dlevel s L : t o obtai n th e revise destimate s v'. an da', show n a t

eac hof th e si x blood-lea dlevel s liste di n Tabl e C.5 . Figur e C. 3 show s th e ful l se t

of normal-on-log-odd s distribution s correspondin g t o th e paramete r estimate s in Tabl e C.5 . Als o liste din Tabl e C. 5 ar e th e regressio n r-square s fo r th e fitte dfunction s a t

15 , 35 , an d5 5 ug/dl ; the y ar e essentiall y th e sam e a s th e r-squar e value s liste din Tabl e C. 4 becaus e of th e hig hdegre e o

f linearit y betwee n blood-lea d leve l an dth e u an da values . Th e mea n value s of th e populatio n respons e rate s a t eac h blood-lea dlevel , base don th e fitte dnormal-on-log-odd s functions , ar e als o liste di n Tabl e C.5 . The y rang e fro m0.1 8 a t L= 5 ug/d l t o 0.4 1 a t

L= 5 5 ug/dl .P.8 0

7 4 TABL E C. 5 Paramete rEstimates , Goodnes s o f Fi t Statistic s and Mea

n Respons e Rate s fo rth e Linearly Regresse d Normal-On-Log-Odd s Distributions , Applie d t o th e Judgments o f Subjec t2

j 1 2 3 4 5 6 Blood Lea d (ug/dl ) 5 15 2 5 3 5 4 5 5 5 A . a

U . J -1.5 9 -1.2 8 -0.9 7 -0.6 7 -0.3 6 -0.0 5 3

0.4 1 0.5 0 0.6 0 0.6 9 0.7 9 0.8 9 2

r Value s NA C 0.9 7 NA C 0.9 9 NA C 0.9 5 R- b

0.1 8 0.2 3 0.2 8 0.3 4 0.3 8 0.4 1

a Linea r regressio n model s of th e result s i n Tabl e C. 4 wer e use dt o estimat e th e u ' an da", values . J J Estimate dmea

n value s o f

th e modele d distribution s fo r

populatio n respons e rat e conditiona l on blood-lea dlevel . c No t appropriate ; judgment s wer e no t obtaine da t

thi s blood-lea dlevel . However , a s alread y discussed , equal-varianc e normal-on-log-odd s distributio n representation s of

th e probabilit y judgment s ar e preferabl e o n rationa l grounds . Thes e wer e derive dby combinin g al l of th e encode djudgment s t o yiel dapoole destimat e of a , denote da' , i n th e manne r describe di

n Appendi x B . Th e estimate dpoole dvarianc e i s o ' = 0.74 , withapoole dregressio n r-squar e valu e of 0.92 . Th e correspondin g least

square s estimate s fo r

th e u ' , calculate da s describe di n Appendi x B , ar e -1.28 , -0.67 , an d-0.0 5 fo r

L= 15 , 35 , an d5 5 ug/dl , respectively . Regressin g thes e estimate dvalue s o n blood-lea dleve l produce s th e ~ . , value s fo r equal-varianc e normal-on-log-odd s function sP.8 1 7 5



### 1. 0

ra J O O > 3 E 3 U

Populatio n Respons e Rate , Expresse d As Fractio n o f

Populatio n Affected , P .

FIGUR E C. 3 A Complet e Se to f Si x Modele d Normal-On-Log-Odd s CDF s fo rth e Respons e Rat e Variabl e Base d on th e Paramete rEstimate s in Tabl e C. 5P.8 2

7 6 tha t ar e liste din Tabl e C.6 . Als o liste din th e tabl e ar e th e r-squar e value s fo r th e associate dnormal-on-log-odd s function s an dth e mean populatio n respons e rat e a t eac h o f th e si x blood-lea dlevel s o f interest . Thes e ar e slightl y lowe r tha n th e r-squar e value s i n Tabl e C.4 , bu t

sti U quit e respectable . Th e normal-on-log-odd s distribution s correspondin g t o th e parameter s i n Tabl e C. 6 wer e show n i n th e repor t a s Fig . 4.3 , alon g wit hjudgment s elicite da t th e thre e blood-lea dlevels . Figur e 4. 4 showe dth e correspondin g 0.05- , 0.50-(o r median ) an d TABL E C. 6 Paramete rEstimates , Goodnes s of Fi t Statistics , an

d Mea n Respons e Rate s fo rNormal-On Log-Odds Function s with Equa l Varianc e Fitte d t o th e Encoded Judgments o f Subject2 Throug

h th e Us e of

LinearRegressio n Technique s a s Describe d in th e Tex t

j 1 2 3 4 5 6 Bloo

d Lea d (ug/dl ) 5 15 2 5 3 5 4 5 5 5 u .

J -1.8 1 -1.4 6 -1.0 9 -0.7 3 -0.3 7 -0.0 0 a'

P 0.7 4 0.7 4 0.7 4 0.7 4 0.7 4 0.7 4 2 r Value s NA C 0.8 7 NA C 0.9 9 NA C 0.9 1 R " b

0.1 6 0.2 0 0.2 6 0.3 2 0.3 9 0.4 7 Resul t fromalinea r

regressio n of

th e u ' value s (liste di n th e text , froma n equal

varianc e representation ) o n blood-lea dlevel , Estimate dmea

n value s o f

th e modele d distribution s fo r

populatio n respons e rat e conditiona l o

n blood-lea dlevel . c No t appropriate ; judgment s wer e no t obtaine da t

thi s blood-lea dlevel .P.8 3 77 0.95-cumulativ e probabilit y estimate s o f respons e rat e a t

eac hof th e si x blood-lea d levels . Unlike th e curve s fo r subjec t 1 , th e curve s o

f equa l probabilit y fo r

subjec t 2 ar e ver y nearl y straigh t

lines . T o summariz e thes e an dth e previou s results , th e normal-on-log-odd s functio n applie dt o individua l subjectiv e probabilit y distribution s generall y provide s excellen t fits , wit hr-square s on th e orde r o f 0.9 5 an dhigher . Constrainin g th e normal-on-log-odd s distribution s t o hav e equa l varianc e fo r a n entir e se t

o f distribution s stil l yield s ver y goo dfittin g functions , but with , of

course , somewha t

lowe r r-squares , whil e simultaneousl y avoidin g th e possibilit y of inconsistencies . Since , a s discusse dpreviously , probabilit y judgment s wil l b e obtaine dfro mth e healt hexpert s ove r virtuall y th e ful l [0,1 ] interva l fo r

si x lea dlevels , i t

ca n b e expecte dtha t

equal-varianc e function s wil l b e quit e accurate . C. 2 ILLUSTRATIV

E RIS KCALCULATION

S Havin g establishe dtha t th e normal-on-log-odd s distributio n describe s th e probabilit y judgment s ver y accurately , th e nex t ste p i s t o combin e th e judgmen t distribution s wit hth e NHANES-I I bloo dlea ddistributio n t o yiel dderive dris k distribution s i

n th e manne r

describe din Sec . B.5 . Thi s i s don e her e withth e judgment s of bot hpilo t

subject s usin g th e NHANES-II distributio n fo r childre n age d6 month s t o 5 years , t o illustrat e th e method . C.2. 1 PUotSubjec t1 Tabl e C. 7 iUustrate s th e calculation s require dt o obtai n amedia n estimat e of th e percen t of childre n age d0- 3 wit hhemoglobin s les s tha n o r equa l t o 1 1 g/dl . Thes e calculation s ar e base don th e best-fitting , no t th e equal-variance , normal-on-log-odd s distributions . Fo r

eac hlea dlevel , L : = 5 , 15 , ... , 5 5 ug/dl , th e tabl e show s th e mea n of th e normal-on-log-odd s distribution , uj , th e associate drespons e rate , r^ , th e NHANES-II7 8 TABL E C. 7 Calculation s fo raMedia n Estimat e o f

th e Fractio n o f

th e Populatio n Affected , Base d on Best Fittin g Normal-On-Log-Odd s Distribution s Fitte d t o th e Judgments of

Subject1

j 1 2 3 4 5 6 Lj(ug/dl )

5 1 5 2 5 3 5 4 5 5 5 * j

-3.3 0 -2.8 4 -0.9 9 -0.4 4 -0.0 6 0.3 7 ° j 0.5 9 0.6 7 0.5 4 1.1 3 0.5 9 0.6 7 ' / 0.0 4 0.0 5 0.2 7 0.3 9 0.4 9 0.5 9 P(L j ) b 0.12 2 0.63 3 0.20 5 0.03 5 0.00 3 0.00 1 1.000 c P(L j )» J

0.00 5 0.03 2 0.05 5 0.01 4 0.00 1 0.00 1 0.110 c

a r . = e" j /( l • e Uj . b P(L: ) denote s th

e fraction o f NHANES-I I childre n wit n blood lead s in th e rang e o f L - ± 5 ug/dl . CoLumn s do no t alway s su m du e t o rounding . fractio n of

childre n wit htha t

bloo dlead , P(L:) , an dth e product , P(L.-)r : fo r eac hblood

lea dlevel . Summin g ove r th e produc t

an dmultiplyin g by 100 % yield s th e media n estimat e tha t

11.0 % of

th e childre n age d0- 3 hav e hemoglobin s les s tha n o r equa l t o 1 1 g/dl . A s discusse di n Sec . B.5 , combinin g th e dose-respons e curve s correspondin g t o al l cumulativ e probabilitie s F , 0 < F< 1 , wit hablood-lea ddistributio n yield s ajudgmenta l CD Fove r

affecte dfractio n o

f th e population , whic hi s aderive dris k distributio n base d o n th e expert' s judgments . Th e derive dris k distributio n calculate dfro mthi s subject' s probabilit y judgment s i s give n in Fig . C.4 . I t ca n b e see n fro mthi s ris k distributio n that : th e media n estimat e o f th e percen t

of th e populatio n tha t

ar e affecte di s 11.0 % (a s alread y discussed) ; wit hprobabilit y 0.0 5 mor e tha n 23.4 % of th e populatio n ar e affected ;P.8 5

7 9

ra Xi o u a. 0 ) > E 3 U



## 0.0 0 0.0 5 0*1 0 0.1 5 0.B 0 0.2 5 0.3 0 0.3 5 Populatio n Respons e Rate , Expresse d As Fractio n o f

Populatio n Affected , R

FIGUR E C. 4 Derive d Ris kDistributio n Base d on Best-Fittin g Normal-On-Log-Odd s Distribution s fo rSubjec tl' s Judgments , Showing th e Cumulativ e Probability F ThatFractio n Ro f

th e Populatio n of

U.S . ChUdre

n Age d 0- 3 Hav e Hemoglobi n Level s Les s Tha

n o rEqua

l t o 1 1 g/d lP.8 6

8 0

an dwithprobabilit y 0.0 1 mor e tha n 30.5 % ar e affected . Th e estimat e of th e mea n o

f th e distributio n i s arespons e rat e of 11.3% . Th e precedin g ris k

calculation s use dth e best-fittin g normal-on-log-odd s distributions . Repeatin g th e calculation s utilizin g th e equal-varianc e normal-on-log-odd s distribution s yield s almos t identica l results . Th e media n ris k estimat e i s tha t

10.5 % of th e populatio n suffe r

th e healt heffec t

(compare dt o th e previou s estimat e of 11.0%) . Wit hprobabilit y 0.05 , mor e tha n 24.3 % ar e affecte d(compare dt o 23.4%) , an dwit h probabilit y 0.01 , mor e tha n 32.4 % ar e affecte d(compare dt o 30.5%) . Th e mea n estimat e i s 11.5% . Figur e C. 5 compare s th e tw o ris k estimates , showin g thei r essentia l identity . Sinc e th e equal-varianc e distribution s ar e bette r o n theoretica l grounds , thei r us e seem s preferable . C.2. 2 PUo

tSubject2 Combinin g F-fractil e dose-respons e curve s base do n th e judgment s of subjec t 2 withNHANES-I

I dat ao n blood-lea ddistributio n fo r childre n age d6 month s - 5 year s produce s adifferen t

se t of ris k estimates . Th e soli dcurv e in Fig . C. 6 iUustrate s th e derive dris k

distributio n usin g th e regresse dbu t best-fittin g normal-on-log-odd s distribution s in Tabl e C.5 . Accordin g t o thi s ris k distribution , th e media n percen t of th e populatio n affecte di s 22.8% , whil e wit hprobabilit y 0.05 , mor e tha n 40.9 % of th e populatio n suffe r

th e healt heffect , an dwit hprobabilit y 0.01 , mor e tha n 49.5 % d o so . Th e estimate dmean o

f th e distributio n i s 23.6% . Th e broke n curv e i n Fig . C. 6 show s th e derive dris k distributio n obtaine db y combinin g th e NHANES-II blood-lea ddistributio n wit hth e F-fractil e dose-respons e curve s base do

n th e equal-varianc e normal-on-log-odd s distribution s i n Tabl e C.6 . Thi s ris k distributio n yield s aslightl y lowe r media n estimat e (20.1% ) an dslightl y highe r 0.95

- an d0.99-fractil e estimate s (45.6 % an d58.0% , respectively ) tha n doe s th e othe r one . Th e estimate dmean i s arespons e rat e o f 21.4% .8 1

C. 3 COMPARISO

N O F RESULTS FO

RSUBJECT S 1 AND 2 Only th e ris k

distribution s basedo n th e equal-varianc e normal-on-log-odd s distribution s wil l b e compare dhere . Figur e 4. 5 showe dth e derive dris k distribution s base do n th e equal-varianc e normal-on-log-odd s function s fo r subjec t 1 (th e parameter s o f whic har e summarize din Tabl e C.3 ) an dsubjec t 2 (th e parameter s of whic har e summarize din Tabl e C.6) . Thi s figur e wa s obtaine dby usin g th e appropriat e curve s fro m Figs . C. 5 an dC.6 . Subjec t l' s probabilit y judgment s combine dwit hth e NHANES-II blood-lea ddistributio n lea dt o lowe r an dtighte r

ris k estimate s tha n d o subjec t 2's . Th e estimate s ar e sai dt o b e lowe r becaus e th e distributio n base don subjec t 1 i s entirel y t o th e lef t of tha t base don subjec t

2 , resultin g i n lowe r

estimate s of

fractio n o f

th e populatio n affecte dfo r

an y give n probabilit y level . Fo r example , th e media n fractio n affected , correspondin g t o acumulativ e probabilit y o f 0.05 , i s 10.5 % accordin g t o subjec t l' s judgment s an d20.1 % accordin g t o subjec t 2's . Th e ris k

estimate s base do n subjec t 1 judgment s ar e tighte r

becaus e th e distributio n i s steeper . Thus , fo r example , base do n subjec t l' s estimates , wit hprobabilit y 0.9 0 th e fractio n affecte di s betwee n 4 % an d24% ; base do n subjec t 2' s estimates , i t

i s betwee n 7 % an d46% , wit hprobabilit y 0.90 . Ther e i s considerabl e agreemen t in th e tw o set s of

ris k estimates , however , a s i s apparen t in th e plo t

of th e probabilit y densit y function s (PDFs) , show n i n Fig . C.7 , whic h wer e derive dfro mth e CDF s of Fig . 4. 5 b y differentiatin g wit hrespec t t o respons e rate . A measur e of

th e degre e of

agreemen t

betwee n th e tw o ris k

estimate s i s obtaine d a s foUows . Fo r

an y pai r of PDFs , suc ha s in Fig . C.7 , cal l th e left-mos t PD F1 an dth e right-mos t

2 . The n ther e i s exactl y on e fractio n of th e population , R , suc htha t are a unde r 1 t o th e righ t

o f Requal s th e are aunde r 2 t o th e lef

t of R . Th e su mof

thes e tw o area s i s ameasur e o

f agreement . I

f the y agre e completely , th e sumi s 1.0 , an di f the y d o no t agre e a t all , th e sumi s 0.0 . I

n Fig . C.7 , thi s fractio n Ri s 0.145 , an dth e su mi s 0.60 . Thi s measur e of

agreemen t

ma y b e usefu l i

f i t become s necessar y t o quantif y th e degre e o f concensu s amon g experts . On e measur e of agreemen t

woul dthu s b e calculate d fo r ever y possibl e pai r

of experts .8 2



### 1. 0

0. 8 

### - 

## 5 

# 0. 6 J 3 ra x > o 0 1 

### i 0. 4 3 E 3 U 

### 0. 2

0. 0 Bes tFittin g Equal Varianc e 

# 1 0.0 0 0.0 5 0.1 0 0.1 5 0.2 0 0.2 5 0.3 0 0.3 5

Populatio n Respons e Rate , Expresse d A s Fractio n o f

Populatio n Affected , R

FIGUR E C. 5 Derive d RiskDistribution s Base d o n th e Best-Fittin g an d Equal-Varianc e Normal-On-Log-Odd s Distribution s fo rSubjec tl' s Judgments , Showin g th e Cumulativ e ProbabUit

y F Tha tFractio n Ro f

th e U.S . ChUdre

n Age d 0- 3 Hav e Hemoglobi n Level s Les s Tha n o rEqua

l t o 1 1 g/d lP.8 9

83



## 1. 0

0. 8

X. ra x. o u C L 0 1 > E 3 u 

### 0. 6

0. 4

0. 2

0. 0 

### 0 

## 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7

Populatio n Respons e Rate , Expresse d As Fractio n o f

Populatio n Affected . R

FIGUR E C. 6 Derive d RiskDistribution s Base d on th e Equal-Varianc e an d Regresse d Best-Fittin g Normal-On-Log-Odd s Distribution s fo rSubjec t2' s Judgments , Showin g th e Cumulativ e ProbabUit y F ThatFractio n Ro f

th e U.S . ChUdre

n Age d 0- 3 Have Hemoglobi n Level s Les s Tha n o rEqua

l t o 1 1 g/d l8 4



### 8

C A c 0 1 Q

J 2 ra o U PL . Subjec t 1



## 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 

## 0. 7 Populatio n Respons e Rate , Expresse dA s Fractio n o f

Populatio n Affected , R

FIGUR E C. 7 ProbabUit y Densit y Function s fo r Derive dRis k

Distribution s Base do n th e Equal-Varianc e Normal-On-Log-Odd s Distributio n Applie dt o th e Judgment s of Subject s 1 an d2