# OAR Box 1112
Prepped by Ryan Dugan



## Document

Number: 

# 27 ) ll-C-1 e
Docket Number: 

# A-83-2 2

Printed 4/16/20089:12:07AM Sheet 27 of 97 P. 1

# ' ARGONN E NATIONA L LABORATOR Yk-c-t

970 0 SouT h CAS S AVENUE , ARGPWE , Ittoi s 6045 9 "felEphoN E 512/972

- 8 4 3 0 P. 2

September 16 , 198 2

Mr . Thoma s R . McCurd y U.S . Environmenta l Protectio nAgenc y Offic e o f Ai r Quality Plannin

g an d Standard s ResearchTriangl e Park , NC 2779 1
Dear Tom : Argonne Nationa l Laborator y i s please

d t o hav

e th e opportunity

t o revie w th e tw o ris k assessmen t methodologie s develope d b y SR I Internationa l (SRI ) an

d Woodard-Clyde Consultant s (WCC) . Th e fundin

g o f thes e tw

o development s b

y th e Environmenta l Protectio nAgency

ha s provide

d a unique an

d noteworth

y set o f studie s i nth e ris k analysi s field . Sinc e th e guideline s wer e sufficientl y

non-restrictive , eachwa s abl e t o develo p a solutio

nthat wa s essentiall y

Independen t o f th e other . I t ha s bee ninterestin

g t o stud

y an d compar e thes e reports . Thi s lette r i s structure d aroun

d th e evaulatio

ncriteri a tha t EPA specifie

d inth

e scop e o f wor k wit hANL . However , a s you wil l see , w e hav e adde d anothe r notio

nwhic hwe fee l i s essentia l injudgin g th e suitability

of thes e ris k assessmen t approache s t o th e regulator

y environmen t inwhic hEP A mus t operate . Inconsiderin g th e strength s an

d weaknesse s o f thes e methodologies , we hav e trie d t o pu

t ourselve s i nth e positio

no f th

e EPA . Thus , we use

d a needs/value structur e for EPA whic hi s base

d o n ou r understandin

g o f EPA' s rol e i nth e standard s settin

g process . I n a fe w words , ou

r opinio

ni s that th

e methodolog

y develope

d b y WC C i s bette r suite

d thanth

e SR I methodolog

y for furthe r developmen t an d applicatio

n to th e analysi s o f th

e risk s o f alternativ e nationa l ambient ai r qualit y standard s fo

r criteri a pollutant s b y th e Environmenta l Protectio nAgency . Th

e rationa l fo

r thi s conclusio

ni s explaine

d inth e remainde r o f thi s letter . For man y reason s i t wa s difficul t t o mak e a definitiv e choic e between th e tw o contractors . Bot hare highl y qualified , broadl y experienced , an

d d o goo d work . We hav

e suppresse

d wha t we kno

w generall y

about bot hgroup s i n doin g thi s evaluatio

nan d hav e trie d t o focu s onth e content s o f th e documents . Inlookin g at performanc e base d o nthei r fina l reports , i t appear s t o u s tha t th

e WC C methodolog

y bes t meet s th

e requirement s o f a formalize d assessmen t o f th e healt hrisk s due t o variou s criteri a pollutants . Furthermore , i t appear s tha t th e WC C grou

p i s mor e awar e o f an d mor e responsiv

e t o th e standard s settin

g environmen t i nwhic hEPA mus t operate. * Discussio

no f eacho f th e si xcategorie s o f evaluatio ncriteri a follows .

US

. DEppM-rnv-EN

T o f ENERq

yT k

LKiKERsiT

yo f OHJCAQ O ARCAN E UNIVERSITIE S ASSOCIATIO NP. 3

•T . R . McCurd

y Pag e 2 o f 7 Sept . 16 , 198 2

Opennes s an

d Understandability Th e firs t categor

y o f th

e EPA-supplied criteri a i s opennes s an d understandability . I t i s important tha t th e methodolog

y be ope

nt o revie

w b y intereste

d parties , that methodolog y assumption s b

e explicit , an

d that output s b e understandabl e b

y decisionmaker s an

d th e public . Thes e ar e ver y important considerations , perhap s th e mos t important o f th e si xcategorie s supplie d b y EPA . I f result s ar e t o be accepte d b y th e scientific , environmenta l health , industry , an

d publi c communities , the y mus t b

e reproducible . Whil e neithe r SR I no r WC C provide s enoug hinformatio

ni nthei r fina l report s t o allo w a thir d part y t o reproduc e th

e results , th

e WC C approachleave s fewer unanswere d questions . Th

e principa l reaso

nfo r thi s i s th

e fac t tha t SR I ha s introduce d anothe r layer o f complexit y t o th e proble

m b y developin

g sub-model s inorder t o determine th

e eventua l dose-respons e relationship s tha t are needed .
AN L agree s that thi s i s th e righ t solutio

ni na nidea l world . However , i t require s mor e informatio nan d experimentatio

ntha ntha t required fo r directl y

quantifyin

g dose-respons e curves . Thi s als o complicates , we think , th e standard s settin g proces s for EPA . Whil e th e modelin

g approac hdoe s theoreticall y

mak e th

e entir e proces s mor e explici t ( a plu s fo r SRI) , th

e pric e i s probabl y

prohibitiv

e fo r a number o f reasons . I t i s no t necessaril y tru e that additiona l modelin g wil l mak e th

e result s "better. " Whenmor e variable s ar e introduce d t o a model , th e possiblity

o f cross-correlation s increases . Sinc e ther e i s littl e dat a o nwhic hto bas e direc t measuremen t o f th e dose-repsons e curves , thes e dat a mos t certainl y woul d no t b

e ver y usefu l fo r investigatin

g cross-correlations . A s a result , independenc e between variable s woul d mos t likel y hav e to b e assume

d an d thi s usuall y

lead s to a large r sprea

d i nth

e calculate

d results . Sucha sprea d coul d b

e overl y

an d unncessaril y

cautious . SR I develope

d a numbe r o f "sub-models" . For example , th e coronar

y obstructiv

e pulmonar

y diseas e (COPD ) mode l containe d a simulatio

nmode l that wa s onl y

briefl y

allude d to inth

e report . Becaus e there i s n o descriptio

nof thi s model , i t i s impossibl e t o reproduc e th e result s o f thi s section . I n addition , th

e output o f thi s mode l wa s expresse d i nterm

s o f "lif e shortening" . Thi s i s a ne w output measur e fo r th e standard s settin g process . Tha t ma

y pos e problems . Mor e seriou s i t seems , i s th e fac t that th

e output measure s EP

A need s (bench-mar

k an d hea d count risks ) d

o no t fal l ou

t of sucha model . The WC C approach , whic hinvolve d direc t assessmen t o f th e dose-respons e curves , i s muc heasie r t o follow . Ther e i s a ver y seriou s questio

na s to whethe r or no t i t i s possibl e to accuratel y measur e thes e dose-respons e curve s directly . SR I reporte d that the

y turne d t o a modelin

g approachbecaus e th e healt hexpert s the y Interviewe

d wer e no t abl e t o answe r question s aime d a t directl y

assessin

g dose-respons e curves . Sinc e WC C wa s successfu l a t directl y measurin

g dose-respons e curves , th e important questio ni s "Ho

w goo d wer e thes e assessments? "P. 4

.T . R . McCurd

y Pag e 3 o f 7 Sept . 16 , 198 2

Fortunatel y

ther e wa s overla

p i nth

e output measure s used b y SR I an d WCC , s o we ca

nge t som

e indicatio

no f th

e acceptibility

o f th

e direc t assessments . For example , bot hproduce d nationa l estimate s o f excessiv e death s du

e to ASHD an

d excessiv

e angin

a attacks . Afte r translatin g th e result s t o th

e sam e basi s (that is , relativ e t o th

e strictes t standard) , an d recognizin

g tha t ther e ar e difference s inother assumption s leadin g u p t o thes e results , i t appear s that :

• Th e expecte

d number s o f exces s death s calculate d b y SR I are generall y

a littl e les s (b

y a factor o f about 0.25 ) tha nth e lowes t estimate s produce

d b y WCC .

• WCC' s medianestimate s are about three time s highe r tha nSRI' s mea

n estimate s fo

r nationa l ASHD deaths .

• SR I estimate s o f th e number s o f exces s angin a attack s are about th e sam e a s th e highes t estimate s made b y WCC .

• WCC' s medianestimate s are about 40 % lower tha nSRI' s meanestimate s for exces s angina attacks . Thes e observation s ar e base d o ndat a inSRI' s Tabl e 9. 2 an d WCC' s Tabl e 7.17 . Th e Lo s Angele s result s (SRI' s Table s 5. 4 an d 5.5 , WCC' s Tabl e 7.1 ) hav e fewer source s fo r differences . Th e conclusion s are essentiall y th e sam e a s fo r th e nationa l data :

• SR I estimate s o f death s ar e generall y a littl e lower (abou t 10 % lower ) tha

nWCC' s lowes t estimates .

• WC C medianestimate s o f death s are about tw o time s highe r tha

nSRI' s mean estimates .

• SR I estimate s o f exces s angin a attack s ar e generall y about th

e sam e a s th e highes t estimate s mad e b y WCC .

• WC C medianestimate s o f exces s angin a attack s ar e abou t 45 % lower tha n SR I estimates . Althoug hi t woul d b

e difficul t t o establis ha statistica l tes t o f th e hypothsi s that thes e estimate s ar e "th e same" , i t appear s tha t the y ar e o f th

e sam e "order o f magnitude" . Acceptin g this , on

e ma y reasonabl y

conclude that th e WC C estimate s o f th e dose-respons e curve s ar e "acceptable" . Efficiency

- Th e secon d categor

y o f evaluatio

ncriteri a i s efficiency . Th e concern s her e are tha t th

e method :P. 5

T . R . McCurd

y Pag e 4 o f 7 Sept . 16 , 198 2

• mak e appropriat e us e o f logic , mathematics , an d mathematica l models ;

• make maximu

m feasibl e us e o f relevant informatio nan d expertise ; an

d

• comprehensivel y

dea l wit hal l o f th e uncertaintie s Th e WC C metho

d bette r addresse s thes e issues , althoug hther e i s a notabl e tradeof f betweenth e firs t an

d thir

d points . Th

e ke y word here is"appropriate " becaus e thi s i s a mai npoin t whic hseparate s th e SR I approac

h fro m th e WC C approach . As discusse d earlier , th

e SR I modelin

g approachi s mor e rigorou s but probabl y no t appropriat e fo r us e i nth

e current standard s settin g process . Thi s approachintroduce s consideration s whic har e no t addresse

d i nth

e C O criteri a document . On e exampl e i s th

e us e o f th

e doubl e produc t instea

d o f "reduce

d tim e t o onse t o f pain " t o predic t angina attacks . O nth e othe r hand , thi s mor e rigorou s modelin g approachdoe s make explici t mor e o f th e underlying , uncertai nphysiologica l factor s tha t relat e t o healtheffects . I f th e dat a existe

d to suppor t it , an d th e standard s settin g proces s wer e equippe

d to handl e it , the na mor e extensiv

e modelin

g approachwoul d b e appropriate . A s thing s stand , i t seem s likel y

that th

e SR I metho

d woul d plac e unrealisticall y

heav y demand s o

navailabl e informatio nan d expertise . Arbitrarines s The inten

t o f thi s evaluatio ncategor

y i s that th

e methodologie s minimiz e arbitrarines s an d als o conside r divers e scientifi c opinions . Whil e ther e i s nothin

g inherent ineithe r methodolog y whic hviolate s thes e desirabl e features , th

e SR I repor t ha s man y mor e instance s wher e assumption s wer e made i norde r t o mak e a calculation , assumption s tha t wer e no t explaine d o r defended . Fo r example , th e probability

tre e i nFig . 7. 5 used for th e COP D mode l contain s probabilitie s tha t ar e no t explained ; th e sam e i s tru

e in Fig . 5. 5 whic hi s a probability tre e for probablisti c analysi s o f th e angin a model ; als o inth e probablisti c phas e analysi s fo r angin

a i s th

e assumptio

n tha t MO C i s lognormall y distributed . I f ther e wa s a rationa l for thes e choices , i t shoul d hav e bee nmentioned . WC C di d mak e similar assumptions . Th e lognorma l distributio ni s at th

e cor e o f thei r dose-respons e curv e relatin

g th e percentag e o f additiona l angin a attack s t o angin

a patient s vs . COH bleve l (Fig . 6.1) . However , WC C gav e som e argument s (statistica l goodnes s o f fit ) i nth e tex t fo r th e choice . The sam

e figur e i s als o illustrativ e o f WCC' s mor e obviou s us e o f divers e exper t opinion . The ra w dat a fro

m Tabl e 6. 1 wer e used t o derive , th e dose-respons e curves , whic hgraphicall y sho w tha t uncertainty

exist s i n th

e respons e eve nwhe nth e dos e i s known . Th e explici t us e o f divers e exper t opinio ni s carrie

d alon g throug

hth e analysi s al l th

e wa y t o th e outpu t table s (7. 2 - 7.20) , wher e low , median , an d hig hris k estimate s ar e listed .P. 6

T . R . McCurd

y Page 5 o f 7 Sept . 16 , 198 2

Generality The generality

categor

y require s tha t th

e method :

• handl e different kind s o f healt heffects ;

• wor k fo

r al l NAAQ S pollutants ;

• wor k fo

r standard s wit hvariou s averagin g times ; an

d

• handl e different type s o f informatio n(e.g. , epidemiological , clinical , toxicologica l evidence) . AN L knows o f n

o reaso

nwh y eithe r metho

d shoul d b

e deficien t o

r necessaril y

bette r tha

nth e othe r o

nan y o f thes e points . Bot hmethod s are reall y approache s or "school s o f thought " o nho w t o mode l an

d asses s uncertainty . A s such , eac hi s sufficientl y

adaptabl e to accomodat e a wide variety o f problems , eve

ni nth e highl y

focuse d aren a o f settin

g standard s fo

r NAAQ S pollutants . Th e mos t seriou s weaknes s seem s t o be SRI' s detaile d modelin g approach , whic hwoul d mos t likel y introduc e consideration s whic hwil l no t b e addresse

d i nth

e othe r criteri a documents . Bu t again , thi s i s a poin t raise d earlie r an

d shoul d no t b

e belabored . Flexibility The fifthevaluatio ncategor

y i s flexibility . Issue s relevant t o thi s categor y includ

e th e ability

o f a methodolog y t o incorporat e ne w methodologica l improvement s an d ne w information . Comment s o nthi s categor

y ar e th e sam e a s thos e fro

m th e previou s categor

y concernin

g generality . Sinc e bot hmethod s are base d o nBayesia

nstatistics , bot hwoul d incorporat e ne w informatio

n b y updatin

g prior estimate s conditiona l o nth e ne w Information . Bayes ' rul e i s use d to perfor

m th e mathematic s insuc ha situation . Ne w consideration s woul d be accomodated (i ntheory ) b

y eac hmethodolog

y b y simply addin g a ne w variabl e an

d adjustin

g th e remainde r o f th

e mode l a s necessary . Neutrality Th e las t categor

y i s neutrality . Th e concern s her e ar e vita l considerin

g th e regulator

y environmen t i nwhic hEP A mus t operate . I t i s important that th

e ris k estimatio

nprocedur e minimiz e th e incorporatio

n o f socia l value s an

d minimiz e bias' . Bot hmethodologie s incorporat e assessmen t protocol s whic hd

o enchanc e th

e likelihoo

d o f obtainin

g estimate s whic hare value fre e an

d unbiased . I t i s almos t unavoidabl e tha t value judgment s ar e a majo r factor in computin

g a composit e ris k estimate . Thi s occurre d i nChapte r 1 0 o f SRI' s repor t whic hdevelope d an"equivalent days o f restricte d activities " (EDRA ) metri c fo

r rankin

g decisio

nalternatives . Thi s chapte r shoul d b e viewe d only a s a nexampl e o f wha t coul d b e don e t o aggregrat e th

e impact s o f man

yP. 7

T . R . McCurd

y Page 6 o f 7 Sept . 16 , 198 2

different healt heffects ; bu t i t i s als o important to not e tha t th e assignment o f suc hmeasure s i s a value judgment . For example , automobil e death s ar e "mor e important " tha nfeta l death s becaus e th e EDRAs ar e 3 0 year s an d 7. 5 year s respectively . A ste p inth

e directio

no f a composit e ris k estimat e approache s th

e notio no f "ris k evaluation" . I f suc ha measur e an

d assignment s wer e actuall y

used , the

y woul d hav

e t o b e chose

nwit hgrea t an

d carefu l thought . A suggestio

nfo r futur e wor k I s tha t assessmen t protocol s b e varie d whe nassessin

g anexper t fo

r a numbe r o f probability densities . Researchha s show nthat th

e phrasin

g an d orde r o f question s ca ninfluenc e responses . Whil e consistenc

y check s shoul d solv e an y suchproblems , i f the y arise , i t i s possibl e to desig

nassessment protocol s whic hexplicitl y account fo

r thes e subtleties . I f th e asssessmen t procedur e alway s begin s b y assessin

g th e 50 % fractil e onth

e cumulativ

e probabilty

curve , a systemati c bia s calle d "anchoring " ma

y b e introduced . Th e en d resul t i s usuall y tha t th

e assesse

d distributio

ni s to

o narrow , peake

d aroun

d th e mode . I t ma

y b e helpfu l to begi nsom

e assessment s a t th e 95 % fractile , som e at th e 5 % fractile , som e a t th e 50 % fractile , etc . Randomizatio no f th

e startin

g poin t woul d enhanc e objectivity

o f th

e procedure . The ni f ther e ar e enoug hassessments , th

e presenc e o f undesirabl e biase s ma y b e detectable . Statement o f Work

I n additio

n to thes e si xcategorie s inevaluatio ncriteria , it i s als o appropriat e t o conside r one important issu e fro m th e statement o f wor k which EP A specifie

d to SR I an

d WCC . Th e contractor s wer e give na large degre e o f freedo m i ndecidin

g wha t output measure s o f huma nhealt hrisk s wer e appropriate , wit hsom e guidanc e a s t o th e minimu

m need s o f th

e EPA . Thes e minimu m need s are critica l sinc e th e measure s ar e specificall y use d i nth

e standard s settin g process . Thes e measure s are :

• Th e probability

that th

e numbe r o f person s experiencin g an d th e expected numbe r o f incidence s of , eac hadvers e healt heffec t equal s o r exceed s a give

nvalue fo

r a specifie

d tim e period .

• Th e expecte

d numbe r o f person s experiencing , an d th e expecte

d numbe r o f incidence s of , eachadvers e healt heffec t ina give ntim e period .

• Th e probability

tha t n Z o f th

e mos t sensitiv

e group , i f expose d t o ai r quality

jus t attainin

g a give

nstandard , wil l experienc e a n advers e healt heffec t i na specifie d perio d o f time . . Onl y WC C provide

d estimate s fo

r eac ho f thes e outpu t measures . SR I provide d som e o f thes e plu s som e ne w one s o f thei r ow

ncreation . Eventhoug hth e ne w one s ar e important measure s o f th e risk s to huma

nhealth , th

e standard s settin g proces s i s no t set u p t o dea l wit hthem . Mos t important i s th e fac t that SR I did no t mee t th e minimu

m need s o f th

e EPA . I t als o seeme d that SRI ha d som e difficulty

wit hth

e notio no f who comprise s a sensitiv e group .P. 8

T . R . McCurd

y Page 7 o f 7 Sept . 16 , 198 2

Summar y
Man y word s hav

e bee nuse d In th

e precedin

g page s t o evaluat e th e SR I an d WC C risk s assessmen t methodologies . Th e mai npoint s ar e th e following :
T • WC C i s judge d to b e bette r tha

nSR I i nthre e o f th e si xevaluatio

n categorie s considered , an d the y ar e equa l i nth

e othe r three .

• Th e mos t important difference , whic hripple s throug hth e entir e evaluation , i s philosophical ; SRI' s approachi s base d o ndetaile

d modelin g whic hultimatel y complicate s th

e standar

d settin

g process , whil e WC C used direc t assessment s o f critica l dose-respons e relationships .

• WC C bette r addresse d th e task s specifie

d inth

e statement o f work , specificall y

providin

g th e benchmark an

d hea d count ris k estimate s neede d b y EPA . Th e fina l poin

t i s that th

e EPA shoul d b

e complimente

d fo r sponsorin

g thi s typ

e o f research . I t i s ver y important tha t th e bes t informatio

n availabl e b

e use d inth

e standard s settin

g process * Analysi s o f th e risk s to huma nhealt hi s fraught wit huncertainty an d incomplet e data . A subjectiv

e vie w o f probability

i s th e bes t wa

y to ti e wha t i s know nan d wha t i s unknow

n togethe r s o tha t somethin g rationa l ca

nbe sai d i nsuppor t o f critica l decision s whic hmus t b e made regardin

g th e standard s fo

r criteri a pollutants .
Respectfully , 

# /LM;$>(Mf^

Ronal d G . Whitfield , Ph. D Decisio nan d Syste m Science s Sectio

n Energ y an d Environmenta l System s Divisio

n

cc : T . Surle s T . Wolsk o

'"RG W 

### :s m