## Hypothesis **- Based**  **We igh**  **t of E**  **vidence: An approach t**  **o structured ev aluation of comple**  **x evidence**  **bases f or r**  **egulatory judgmen ts about risk and causality**  

## Lor **enz Rhomber g**  , PhD , F ellow ATS 

## Gr adien t
**EP A/**  **FWS Meeting**  6 Oct ober 2014**2** Copyright Gradient 2013 

###  

## The r egulatory pr ocess cannot sus ta in pur e science’ s suspension of judgmen t until ultimate r esolution. A basis fo r actions is needed.

 Skepticism and consider at ion of alternativ es as part of the scientific method  div ersity in interpr etations among scientis ts 

###  

## The r egulatory pr ocess needs “**findings** ,” and judgmen t is **deleg ate**  **d** to particular people ta sked with r epr esen ting the larger body of informed scien tific opinion.

 So , whose judgments and how they are jus tified become key 

## The general **Wo E**  **Ques tion is as big as “Scienc** **e, ”**  **but the application to** **Chemic al R** **egulation en ta** **ils some special f** 

**eatures: 3**  Copyright Gradient 2013 

# “W **EIGHT OF EVIDENCE”** 

 

### As a metaphor  

### As a method  

### Use all the data  

### Syste ma tic ev aluation  

### Aim at objectiv e pr ocedur es that la y out the pr ocess of scien tific pr ofessional judgmen t  

### Question: In view of incomplete and co ntradictory evidence, how compelling is the case fo r the e xis te nce and poten tial magnitude of risk ?**4** Copyright Gradient 2013 

# Desider**ata :** 
 Syste ma tic Appr oach to Liter atur e Inclusion  inclusion/exclusion criteria  not just positive or “featur ed” out comes of studies  

### Syste ma tic and Consis te ntReview of Studies  es ta blished pr ocedur e, ta bulation  

### Eva luations of Study Str engths and W eaknesses  BUT – what to do with “lesser ” studies?  omit? down - weight? int erpr et?  study design, power

, confounders, pot ential pr oblems 

## An Es **ta blished Process for E** **valuation and Combination**

**into Inf erences** **5** Copyright Gradient 2013 

# In **fe re**  **nces in**

**HHRA vs**  **EC O**  **vs ESA** **6** Copyright Gradient 2013 

## Rules based vs. guided **judgmen t:**  **Sailing be**  **tw een Scylla and**  **Charybdis** A **“Known Human**  **Carcinogen”** is one for which the evidence is sufficient to conclude that it is a human carcinogen. A **“Known Human**  **Carcinogen”** is one for which, fo llowing the fr amework, one ends up in the “K nown Human Car cinogen” bo x. 

### “J **UDGMENT ”** 

 guided evaluations with r ecorded re sults  Judgments ar e pr oposed explanations of the arr ayof re sults  Judgments ar e jus tified by citing basis and showing superiority over alt ernatives 

### “RULES”

**“STRUCTURED JUDGMENT**  **” 7**  Copyright Gradient 2013 

# “W **EIGHT OF EVIDENCE”**  **HO W MUCH DOES EVIDENCE WEIGH?**  i.e ., what process f

or the evaluation of evidence provides a means f or judging how compelling it is and how to trade off among apparent contradictions ?
 

### “ Ev idence” has no meaning e xc ept in re lation to a specific hy pothesis  

### The hy

pothesis is ev aluated w. r. t . the evidence (not the other wa y ar ound)  

### So **Wo E should be organized around e**  **valuating specific h** **ypotheses against dat a** **8** Copyright Gradient 2013 

# Articula **te an Hypothesis**  What is the proposed basis fo r inferring that a particular phenomenon seen in s tudies of a chemical's effects will also happen in ta rget populations?
 

### Ma y be h ypothesiz ed MoA -- or g ener al (“ animals pr edict humans”)  

### A generalization , not jus t an e xtr apolation (should apply to all cases within its r ealm )  

### What manifes tations of the h ypothesis ar e e xpect ed and not ex pect ed? Check ag ains t all actual obser ve d re sults.**9** Copyright Gradient 2013 

# Key**W oE Ques** **tions**

 Based on obser ve d positives, what h ypothesiz ed c ausal pr ocesses ar e necessary? Suf ficien t?  How do they gener aliz e? What other manifes tat ions should they ha ve ?  If h ypothesis wer e wr ong , how else would one e xplain the arr ayof out co mes?**10** Copyright Gradient 2013 

# Fo **r Obser ve**  **d Out comes tha**  **t ar**  **e Candida te**  **s**

**fo r “E** **vidence”**  Wh y w e think they happened wher e they did.  Wh y w e think they didn’t happen wher e they didn’t .  Wh y w e think the “did - happen” fact or s w ould also apply t o the ta rge t popula tion.  Might apply? Pr obably apply? K nown to apply? 

###  Ar e ther e discr epan t observ ations, and if so , how do w e acc ount fo r them?  Ar e our “wh ys”  Observ able underlying causes?  Re asonable guesses based on wider knowledge, other cases?  Ad hoc assumptions without evidence, needed to e xplain otherwise puzzling phenomena?**11** Copyright Gradient 2013 

# Some Pitf**alls:**
 

### Overr eliance on Con ve nt ions, Heuris tics  

### Narr ative, citing “consistent ” information  

### Fa ilur e to addr ess null or contr adict ory findings on same endpoin ts (especially when combining endpoints)  

### Multiple Comparisons (choosing fr om among man y par allel alt ernative analyses of same dat a)  

### Mutually contr adict ory ex planations of individual s tudy r esults  

### Ad Hoc Explanations (distinguish discovery fr om support)  

### Accommodation**12** Copyright Gradient 2013 

# Ev**alua te** **How Compelling the Case is f or the**  **Pr oposed Basis**  **of Human Risk in View of:** “ **Predictions** " of h ypotheses that ar e confirmed in the obser vat ions  Mor e weight to "risky" and specific pr edictions  Less weight when subsidiary assumptions or e xplanations ar e needed  Both Po sitive and Negative Pr edictions! 

### Apparent R **efutations** (counterex amples)  Fa ilur e to r epeat r esult acr oss studies  Non - re sponding se xe s or species  Unpr edicted but clearly r elevant phenomena **A h ypothesis**  **can often be reconciled with apparent**  **refutations by either modifying**  **it or adding subsidiary**  **assumptions –**  **but this entails**  **a weight "penalty" 13**  Copyright Gradient 2013 

# The P **erils of Implicit Accommoda**  **tion**

 

### X caused incr eased activity in r ats  

### In another lab, X c aused sligh tly decreased activity  but activity measur ed during da y (ins tead of night) 

###  

### Is this supportive evidence?**14** Copyright Gradient 2013 

# Re **la tiv** **e Cr**  **edence in Compe**  **ting “A**  **ccoun ts”** 
• “A ccoun t” = an articulated set of pr oposed ex planations fo r the set of observ ations • Re levant Causation – but also chance, err or , confounding fa ct or s, gener al - knowledge possibilities, plausible assumptions, assertions of irrelevance, and “unknown re asons” 

### Cert **ain Findings Indicat**  **e Ta**  **rget -**  **Po pulation Risk**   re asoning wh y  how contr adictions r esolv ed  wh y assumptions r easonable 

### Those Findings Do **Not Indicate**  **Ta rget**  **- Po**  **pulation Risk**   re asoning wh y not  how findings ar e otherwise ex plained  w hy assumptions r easonable 

### Can we measure the weigh **ts? 15**  Copyright Gradient 2013 

## Adv**an ta**  **ge s**  **of Hypothesis -**  **Based Wo**  **E**

 

### Shows which endpoin ts ar e the mos t compelling  

### Makes re asoning e xplicit – shows both str engths and we aknesses  

### Allows debate about particular data and interpr etations  

### Fr ames Wo E classifications as scientific state men ts (falsifiable, poin ts to tests)  

### Does not blend all endpoin ts t ogether in an unin te rpr etable , gener aliz ed “risk” state men t  

### Informs the QRA pr ocess**16** Copyright Gradient 2013 

## Sir Aus**tin Br** **adf or**  **d Hill**  **on the Hill Crit eria**  “. . . **the fundamental question –**  **is there an y other way**  **of e xplaining the set of facts before us, is there an**  **y other answer equally** 

**, or more, lik**

**ely than cause and ef** 

**fect? ”**  A. Br adford Hill (1965) Proc Roy Soc Medicine **58** :295. 

### “set of facts” =  

### all the epi (+ and - )  

### mode of action  

### animal studies  

### other pot en tial e xplanations**17** Copyright Gradient 2013 

## Applications **of Hypothesis -**  **Based We**  **igh t**  **of E vidence (**  **HB Wo**  **E )** 
 Chloryprifos neur odev elopmental tox icity

 Prueitt, RL ; Goodman, JE; Bailey , LA; Rhomberg, LR . 2011. Crit. Rev . To xicol. 42(10):822 - 903.  Goodman, JE; Prueitt RL ; Rhomberg, LR . 2012. Dose Response 11(2):207 - 219.  Methanol carcinogenicity

 Bailey , LA; Prueitt, RL ; Rhomberg, LR . 2012. Re gul. To xicol. Pharmacol. 62:278 - 291.  Dio xins th yr oid hormone perturbation

 Goodman, JE; K erper

, LE; P etito Boyce, C; Prueitt, RL ; Rhomberg, LR . 2010. Re gul. To xicol. Pharmacol. 58(1):79 - 99.  Formaldeh yde as a leukemogen

 Rhomberg, LR ; Bailey

, LA; Goodman, JE; Hamade, AK; Mayfield, DB. 2011. Crit. Rev . To xicol. 41(7):555 - 621 .  Naphthalene car cinogenicity

 Rhomberg, LR ; Bailey

, LA; Goodman, JE. 2010. Crit. Rev . To xicol . 40(8):671 - 696.  Meth ylmethacrylate nasal t ox icity

 Pe mberton, M; Bailey , EA; Rhomberg, LR . 2013. Regul . To xicol . Pharmacol . 66(2): 217 - 233.  To luene Diisocyanate carcinogenicity

 Goodman, JE; Prueitt, RL ; Rhomberg, LR . 2013. Crit. Rev . To xicol . 43(5):391 - 435.**18** Copyright Gradient 2013 

# Public **at ions on the**  **HB Wo**  **E Me**  **thod**

 Rhomberg, LR . 2014. "Hypothesis - Based W eight of E vidence: An Appr oach to Assessing Causation and its Application to R egula tory T ox icology." Risk Analysis . doi : 10.1111/ risa.12206 .  Rhomberg, LR ; Goodman, JE; Bailey , LA; Prueitt, RL ; Beck, NB; Bev an, C; Honeycutt, M; K aminski, NE; Pa oli, G; Po ttenger , LH ; Scher er

, RW ; Wise, KC; Becker , RA. 2013. "A Surv ey of Fr ameworks fo r Bes t Pr actices in W eight - ofEv idence Analyses." Crit. R ev . To xicol. 43(9):753 - 784.
 Lutter , R; Abbott, L; Becker

, R; Borgert , C; Br adley

, A; Charnley

, G; Dudley

, S; Felsot , A; Golden, N; Gr ay , G; Juberg , D; Mitchell, M; Rachman , N; Rhomberg, L ; Solomon, K; Sundlof , S; Willett, K. 2014. "Impr oving W eight of E vidence Appr oaches to Chemical E va luations." In Press in Risk Analysis .