APPENDIX D 

EXTERNAL PANEL REVIEW: 

Method of Analysis to Perform a Tissue-Based Cumulative Risk Assessment for 

Mixtures of Chemicals (NCEA-C-1602)

D-1
The following panelists were identified and retained by Versar, Inc. 

Richard J. Bull, Ph.D. 

MoBull Consulting 

Richland, WA 

Harvey Clewell (Chair) 

CIIT Centers for Health Research 

Research Triangle Park, NC 

Gary L. Ginsberg, Ph.D. 

Connecticut Department of Public Health 

Hartford, CT 

Margaret MacDonell, Ph.D. 

Argonne National Laboratory 

Argonne, IL 

Moiz M. Mumtaz, Ph.D. 

Agency for Toxic Substances and Disease Registry (ATSDR) 

Chamblee, GA 

Clifford P. Weisel, Ph.D. 

Environmental & Occupational Health Sciences Institute (EOHSI)/UMDNJ 

Piscataway, NJ

I. PRE-MEETING COMMENTS 

Following dissemination of the draft document and before the convening of the panel 

review, comments were solicited from panelists. Two sets of comments were received 

(Clewell and Ginsberg); those pre-meeting comments were discussed by the review 

panel. Where concerns remained, those concerns were treated in the final review 

comments. As such, there are no EPA responses to pre-meeting comments. 

1.A. Harvey Clewell 

(1) Method of Analysis to Perform a Tissue-Based Cumulative Risk Assessment 

for Mixtures of Chemicals (NCEA-C-1602) 

1. Does the document clearly distinguish interactions at the level of cumulative risk 

assessment and pharmacokinetics? Does the document present the benefits of 

employing tissue dose, rather than environmental concentration as the basis for 

cumulative risk assessment? 

D-2
Reviewer Response: The document does provide a reasonably clear distinction 

between interactions as defined in default cumulative risk assessment and 

interactions in pharmacokinetics, although there are some awkward sentences, 

e.g.: 

“Form [stet] an “interactions” perspective, it is possible that pharmacokinetic 

interactions may explain chemical mixture (cumulative risk) interactions, that may 

not be “interactions” as defined by cumulative risk; such departures from 

additivity may only be departures when considered at the level of the external or 

applied dose, and may be strictly additive when “dose” is expressed as “dose 

metric” – the concentration of the toxicologically active chemical species in the 

target tissue. “ 

I think this sentence should be expanded into a paragraph that takes the 

necessary time to lay out the idea it is being attempted to convey. 

The document does a good job of describing the benefits of using tissue dose, 

but it would be strengthened if examples were presented where considering 

tissue dose made a difference. 

2. The decision point for continuing to develop a comprehensive cumulative risk 

assessment (a PBPK-based approach and assessment) is presented in Step 4 of the 

analysis plan. It is based on the assumption of chemical additivity, and includes a 

Hazard Index type analysis coupled with an uncertainty factor intended to address 

interactions (in the broad definitional sense). This point has drawn appreciable 

comment previously. The intent of the document is to communicate considerations 

undertaken in the series of choices relating to resource expenditure for CRA. Please 

comment on the clarity and strength of the rationale presented and propose other 

considerations you feel may be useful to incorporate. Please comment specifically on 

the proposed 1000-fold uncertainty factor and on conditions when it may be warranted. 

Below is an example comment and response relating to this point: 

Proposed “Rules of Thumb” for Estimating Threshold of Toxicological 

Interactions 

INTERNAL REVIEWER COMMENT: “Thus, for practical purpose of conducting CRA” 

Interesting rule, but the stated rationale (stated at the end on page 19) is pretty weak. 

The screening calculation might make more sense if stated in terms of an average 

hazard index (i.e., sum of each chemical’s exposure potential /RfD, and that sum then 

divided by N) or relative potency or some other normalizing method, then applying an 

additional screening factor (like an uncertainty factor or margin of safety factor) to 

account for potential synergistic interactions. One could also evaluate the potential for 

antagonism and a lower level of hazard for a mixture. A fixed screening factor for 

synergistic interactions (100 or 1000) could be consistently applied for all CAGs, but I 

think a CAG specific factor would be better. The case by case factor could include 

some weighting for the number in the group, although I think better based upon what we 

know about the chemicals’ metabolism or other binding, what we know about 

D-3
competitive inhibition/binding at different dose levels for the CAG chemicals, or about 

chemical reactivity for the chemicals in the CAG. A similar method could apply for a 

cancer index using potential exposure over the 10^6 risk level (from slope factors or 

linear extrapolations from the NOAELs as the POD 10%). 

RESPONSE: This is a very good comment; likewise it was echoed in those from the 

other reviewer. This issue (how best to develop a method to cull out potential scenarios 

for which a Comprehensive Cumulative Risk Assessment should be performed) will be 

raised for discussion at the level of external review. It may be that such a point (when 

to not do such an assessment) should not be specifically “codified.” but be left to be 

addressed on a case-by-case basis. This issue will be forwarded to the external review 

panel for comment. 

Reviewer Response: I agree with the reviewer above. The rule of thumb should 

be characterized in terms of the hazard index for the mixture, and the numerical 

value of the “screening factor” for a given mixture should be left to scientific 

judgement. 

3. Are you aware of any additional information which can be used to improve the 

present draft report? 

Reviewer Response: No. 

4. Are there assumptions or uncertainties being made in this exercise that are not 

articulated? 

Reviewer Response: No. 

5. Are the figures and tables informative? Would revision of the tables or figures 

improve the clarity of the report or its conclusions? If so, what are they? 

Reviewer Response: The figures and tables are adequate.

6. Are there important publications missing from the reference section? 

Reviewer Response: I think the report would be improved by doing a literature 

search and adding examples of interaction modeling that demonstrate its value. 

7. What additional information would you like to see presented? 

Reviewer Response: Examples of modeling of interactions and how the 

modeling provided a quantitative understanding of the impact of the interaction 

on cumulative risk. 

8. Please comment on the overall quality of the report and analysis. What is your 

overall evaluation of the scientific content, readability and utility of the draft report? Do 

D-4
you have any suggestions relative to structure or content that would improve the quality 

of this draft report? 

Reviewer Response: For the most part, the report is well written and 

scientifically sound. It should be useful as a an initial source of discussion on the 

merits of tissue dosimetry in cumulative risk assessment. It sometimes feels 

rather shallow, however, because there are no concrete examples of the 

successful use of tissue dosimetry to improve a cumulative risk assessment. I 

believe such examples exist for chemicals other than those discussed in the 

report, although they admittedly are for high exposure interactions (e.g., 

TCE/VDC, BTEX). 

There is one scientific inaccuracy I noticed: the reaction of OPs with AChE is not, 

in general, irreversible, as suggested on p. 43. In fact, characterization of the 

relative rates of regeneration (reversible binding) and aging (irreversible binding) 

is an important aspect of the identification of the more dangerous OPs (e.g., 

soman). 

1.B. Gary L. Ginsberg, Ph.D. 

General Comments

This document presents some interesting arguments on how to screen multiple 

chemical exposures and determine whether an extensive PBPK modeling assessment 

would be needed to evaluate cumulative risk. In this regard it is useful. However, it is 

written more like a white paper than a framework or general guidance document and so 

would likely to be difficult to follow for the average risk assessor. The ultimate user of 

this document is unclear (researchers/modelers? headquarters risk assessors involved 

in policy? Risk assessors in the field?). It would be helpful at the outset to describe the 

intended audience and how they might consider using the approach. Regardless of the 

audience, a summary of the proposed approach which highlights the key analytical 

phases and steps is needed. The document is wordy and somewhat redundant with the 

purpose of individual sections relative to the overall flow not always clear. Starting with 

a flow diagram or outline of the proposed approach (e.g., the 10 steps divided into 2 

phases with brief outline of purpose of each step) would help keep the various sections 

in perspective. Then at the end, a section is needed which summarizes the overall 

approach to tie it together for the intended user. For example, it might state that the 

screening level approach can be conducted by most risk assessors to quickly evaluate 

whether interactions are possible and whether a more quantitative (modeling) approach 

is needed. This then would require recruitment of analysts which can gather the PK 

information, build interaction models, etc. etc. Right now, the document makes a 

number of good suggestions but does not do a good job of presenting them as an 

integrated and practical approach. 

D-5
Charge Questions and Reviewer Responses

1. Does the document clearly distinguish interactions at the level of cumulative risk 

assessment and pharmacokinetics? Does the document present the benefits of 

employing tissue dose, rather than environmental concentration as the basis for 

cumulative risk assessment? 

Reviewer Response: The document spends considerable effort distinguishing 

between interactions than can be judged based upon additivity of effect vs. those 

that occur on a biochemical or toxicokinetic basis. While this generally comes 

across, it can be more clearly described in Section 1. For example, the 2nd

paragraph of page 7 thru page 8 is the place where this distinction is made for 

the first time. However, the paragraph starts out with other concepts 

(interactions that are passive/partitioning vs. active – metabolic) that are an aside 

to the key point that the interaction can be on a PD basis (additivity of effect such 

as with the OPs) or on a PK basis (interference with metabolic activation, 

detoxification, clearance, binding, etc.). From such an introduction, one could 

then go on to state how these types of interactions are different and need to be 

analyzed differently (e.g., one with RfD approach, one with PBPK analysis) but 

that both types of interactions need to be considered jointly as part of the overall 

CRA. Setting up the differences while also explaining the need for a common 

analytical framework is critical to the success of the rest of the document. Right 

now the reader has to do too much work to figure out the implications of the 

distinction that is being made on pages 7-8. I don’t think this is clarified to a 

satisfactory extent at later points where the difference is brought up again. 

However, I must add that the 2 types of mixtures examples as discussed on 

pages 13-14, the OPs vs. the chlorinated solvents, does help clarify the 

distinction in that they exemplify the two types of interactions quite well. Some 

framing verbiage would help solidify that understanding (e.g., “The two case 

study mixtures were selected to exemplify ......, OPs on the one hand represent 

......, while the chlorinated water contaminants represent the other type of 

interaction at the PK level. This document uses these different examples to 

show that different types of interactions can be analyzed jointly within a unified 

framework”). 

The second part of this charge question, whether the case is adequately made 

for the benefits of analyzing at the tissue dose rather than environmental 

concentration level. The case for this is argued fairly strongly with at times 

overzealous language (e.g., - page 4 - “wonderful”, “the best”). However, it is still 

somewhat unclear whether the tissue (PBPK) level of analysis is needed for 

CRAs involving interaction at the PD level. If this is the case (and it may well be), 

it is not argued well in this document. In contrast, the need for internal dose 

resolution for PK interactions is clear. 

2. The decision point for continuing to develop a comprehensive cumulative risk 

assessment (a PBPK-based approach and assessment) is presented in Step 4 of the 

D-6
analysis plan. It is based on the assumption of chemical additivity, and includes a 

Hazard Index type analysis coupled with an uncertainty factor intended to address 

interactions (in the broad definitional sense). This point has drawn appreciable 

comment previously. The intent of the document is to communicate considerations 

undertaken in the series of choices relating to resource expenditure for CRA. Please 

comment on the clarity and strength of the rationale presented and propose other 

considerations you feel may be useful to incorporate. Please comment specifically on 

the proposed 1000-fold uncertainty factor and on conditions when it may be warranted. 

Below is an example comment and response relating to this point: 

Proposed “Rules of Thumb” for Estimating Threshold of Toxicological 

Interactions 

INTERNAL REVIEWER COMMENT “Thus, for practical purpose of conducting CRA” 

Interesting rule, but the stated rationale (stated at the end on page 19) is pretty weak. 

The screening calculation might make more sense if stated in terms of an average 

hazard index (i.e., sum of each chemical’s exposure potential /RfD, and that sum then 

divided by N) or relative potency or some other normalizing method, then applying an 

additional screening factor (like an uncertainty factor or margin of safety factor) to 

account for potential synergistic interactions. One could also evaluate the potential for 

antagonism and a lower level of hazard for a mixture. A fixed screening factor for 

synergistic interactions (100 or 1000) could be consistently applied for all CAGs, but I 

think a CAG specific factor would be better. The case by case factor could include 

some weighting for the number in the group, although I think better based upon what we 

know about the chemicals’ metabolism or other binding, what we know about 

competitive inhibition/binding at different dose levels for the CAG chemicals, or about 

chemical reactivity for the chemicals in the CAG. A similar method could apply for a 

cancer index using potential exposure over the 10^6 risk level (from slope factors or 

linear extrapolations from the NOAELs as the POD 10%). 

RESPONSE: This is a very good comment; likewise it was echoed in those from the 

other reviewer . This issue (how best to develop a method to cull out potential 

scenarios for which a Comprehensive Cumulative Risk Assessment should be 

performed) will be raised for discussion at the level of external review. It may be that 

such a point (when to not do such an assessment) should not be specifically “codified”, 

but be left to be addressed on a case-by-case basis. This issue will be forwarded to the 

external review panel for comment. 

Reviewer Response: This screening approach is both intriguing and troubling. 

The attraction to it is that it offers a unified way to screen different types of 

interaction mechanisms for potential likelihood of occurrence. The problem is 

that one approach may not fit different interaction mechanisms. The RfD/N 

additivity approach comes out of the common PD MOA – additivity mechanism in 

which each ingredient in the mixture has a common PD target/endpoint (e.g., 

brain AChE inhibition). In this case, a simplistic hazard equation (exposure 

dose/modified RfD) makes some sense in that the RfD for single compounds is 

set to be below an effects level for even sensitive individuals and lowering the 

RfD to take into account interacting chemicals that have the same MOA has 

D-7
some rationale. It may be a conservative default if the various chemicals have 

similar potency. One gets nervous about this approach for similarly targeted 

chemicals when one considers the malathion/dicrotophos comparison presented 

on pages 28-29. Here are similarly acting OPs which have vastly different dose 

response for the same endpoint. If we are evaluating an interaction between 

these 2 OPs, does it make sense to only lower the malathion RfD (the less potent 

ingredient) by a factor of 2? One might guess that the high potency of 

dicrotophos might cause it to be more influential in an interaction scenario and so 

should be more heavily weighted. Perhaps whats needed is an approach which 

weights each ingredient’s contribution to the interaction based upon its potency 

and its dose level in the system. If dicrotophos is present at levels far below its 

RfD, then it likely doesn’t matter that its so potent. Thus, rather than adding 

malathion and dicrotophos together as two equally important constituents 

(1+1=2) and then dividing each RfD by 2, perhaps one should consider the 

dicrotophos contribution relative to the malathion contribution as follows: 

(Malathion RfD/Dicrotophos RfD) * (Dicrotophos dose level/ Dicrotophos RfD) 

If Malathion RfD is 1000x greater than the Dicrotophos RfD and the dicrotophos 

dose is 1000x lower than its RfD, then the above equation yields 1 and the 

dichrotophos contribution to the cumulative interaction would equal that 

contributed by malathion and you could divide the malathion RfD by 2 (N=2) as in 

the proposed screening method. However, if the dicrotophos exposure level is 

0.1 of the RfD, then the malathion RfD would be divided by 1 (for malathion) + 

100 (from above equation for dicrotophos contribution) (N = 101). Obviously this 

can become more complicated and unwieldy for ternary or higher order 

interactions and so one needs a simplifying assumption that the potency 

adjustment is only needed for the more potent ingredients and only if they 

exceed a given ingredient’s potency (lower RfD) by 3x or more. Thus, if you 

have 5 OPs in the mixture with 3 having similar potency and 2 being considerably 

more potent, you would only need to run the above equation twice to calculate 

how much RfD lowering is contributed by these 2 more potent ingredients. 

The above discusses issues with the screening approach for chemicals acting via 

similar PD MOAs. However, this approach may not be so relevant for chemicals 

which interact strictly via PK mechanisms (e.g., the chlorinated solvents 

interacting at the level of CYP2E1). In this case, there may be no relationship 

between a chemical’s RfD and its ability to compete for throughput via 2E1. For 

example, a chemical may have a high RfD but also a high affinity for 2E1 which 

may make it a stronger metabolic interactor than a chemical with a low RfD but 

weak affinity for 2E1. Thus, for PK-interacting chemicals, screening on the basis 

of RfD/N may have little relevance to what is going on biochemically, especially 

since these chemicals may have different PD targets and MOAs (e.g., one 

solvent affecting CNS while other affects liver). The example of fenitrothion and 

parathion interaction is a good example of the fact that a chemical does not have 

to exert a measureable toxic effect (only a biochemical effect) to synergize 

D-8
another chemical. Fenitrothion ties up peripheral carboxylesterases so that at 

doses well below those required for fenitrothion AChE inhibition, a synergistic 

interaction with parathion is set up. 

It may well be that in most cases, the RfD is set low enough so that not only are 

risks for toxicity averted but also there is no biochemical perturbation relevant to 

interaction with other chemicals at the RfD/N dose. However, the case for this 

needs to be explored and made stronger. For example, dioxin is a potent 

inducer of the CYP1A family which may cause interactions with other CYP1A 

family substrates. Do we know that RfD/N for dioxin (or PCBs) will make this 

type of biochemical/interaction mechanism moot? 

I believe that evidence supporting this screening approach may be obtainable via 

interactive case studies which involve PBPK analyses exploring what goes on at 

high dose vs. what occurs at RfD-type doses. My guess would be that for simple 

competition interactions and protein binding interactions, the RfD/N approach 

would be a reasonably conservative screen. However, this may be less clear for 

enzyme induction interactive mechanisms, or perhaps for non-competitive 

inhibition mechanisms. The fenitrothion/parathion interaction mentioned above 

would be interesting to test in this regard. 

This charge question also raises the issue of whether a default approach 

(NOAEL / 1000 fold UF) is a useful default in the case where an RfD is 

unavailable for a particular constituent. The idea is that a 1000x cumulative UF 

will create an RfD that is low enough to protect against the possibility of 

synergistic interactions. This approach does not appear to be well thought out. 

The RfD should be set based upon the relevance and strength of the underlying 

database. The greater the uncertainty, the larger the cumulative UF. RfDs are 

set based upon the dose-response assessment for individual chemicals and are 

not based upon whether they have an interaction with other chemicals. It would 

be preferable to stick with the standard risk assessment approach to RfD setting 

when starting from a NOAEL and not use a default factor that incorporates 

concerns over synergistic interaction. The analysis of interaction, whether 

screening or detailed, should be a separate step from dose-response 

assessment. Since IRIS RfDs do not incorporate a group interaction UF, to use 

one in the case of a missing RfD would create an apples and oranges situation in 

which the different types of RfDs would not be comparable. 

It is important to note that the focus should not be on how much potentiation is 

possible in a synergistic interaction but whether the concentrations of the 

reactants are high enough for a meaningful interaction to take place. If this is so, 

then one moves beyond the screening level to Phase II in the 10 step process. 

Hopefully during this phase one can determine whether synergy is possible and 

to what extent. 

D-9
If there is a particular concern over the possibility of synergy during the screening 

phase of the analysis (e.g., OPs that can interact the way that fenitrothion 

interacts with malathion) one could incorporate it into the RfD/N approach by 

increasing the value of N for synergistic chemicals in a manner similar to the 

dichrotophos example above. Such a method could be tested with known 

synergistic interactions on a pharmacokinetic level of analysis to document that 

the screening level approach is protective against synergists. Such analyses 

may show what additional factor (if any) is needed to modify N in the RfD/N 

paradigm to ensure protection against synergistic interactions. 

In summary, I would recommend against adjustment of the way in which one 

goes from a NOAEL to an RfD but would rather adjust for synergistic interactions 

on a case specific basis by adjusting the value of N in RfD/N. 

3. Are you aware of any additional information which can be used to improve the 

present draft report? 

Reviewer Response: As stated above, the fenitrothion/parathion synergistic 

interaction is important to mention. The document should also describe other 

types of interactions (non-competitive inhibition, enzyme induction, etc) and how 

they can be brought within the CRA analytical framework. 

4. Are there assumptions or uncertainties being made in this exercise that are not 

articulated? 

Reviewer Response: No, nothing not already talked about. 

5. Are the figures and tables informative? Would revision of the tables or figures 

improve the clarity of the report or its conclusions? If so, what are they? 

Reviewer Response: As described above, a framework overview figure would 

be very helpful at the outset and a summary version at the end showing key 

decision points would also be helpful. 

Figure 1 does not capture the interaction between OPs at the level of peripheral 

carboxylestases (CE). A prime example of this is the fenitrothion synergism of 

parathion. This type of interaction should be included in the chart. 

Figure 2 might be improved by indicating the direction of the interaction at each 

step: e.g., if 1 inhibits 3 at level of 2E1 then there might be more of 3 available for 

GSH conjugation and renal toxicity. Without this context the figure can seem 

overwhelming and may not relate as clear a message regarding the importance 

of the interactions.. vis-à-vis the text. 

D-10
6. What additional information would you like to see presented? 

Reviewer Response: The paper lacks citations of PBPK modeling studies in 

which interactions were evaluated. I believe several studies have been 

published along these lines. Additional background information on the various 

types of PD and PK interactions possible would be helpful (e.g., interaction at 

level of competitive or non-competitive enzyme inhibition; at level of GSH 

depletion predisposing to second toxicant that needs GSH; at level of enzyme 

induction, etc. 

7. Please comment on the overall quality of the report and analysis. What is your 

overall evaluation of the scientific content, readability, and utility of the draft report? Do 

you have any suggestions relative to structure or content that would improve the quality 

of this draft report? 

Reviewer Response: For overall quality, readability and utility, see general 

comments above. To reiterate, the intended end user(s) should be stated up 

front, how and when they might use the document should also be stated, and 

then this should be kept in mind throughout the writing. This may enable it to 

become less of a white paper and more of an analytical framework document. 

Some specific comments are as follows: 

Section 1.1 –the lengthy quotes from the Hansen and Gilman memos do not 

appear to be necessary. The main points can be highlighted much more briefly, 

using selective quotes. 

Page 16, large para – This paragraph addresses the problem of aggregating 

exposures across chemicals, each of which has its own data distribution. The 

discussion here seems too complex and misses what I think is the main option – 

a screening level approach where an upper bound value is used for each 

contaminant. If there are not signficiant interactions with these across the board 

upper bound concentrations, then there is little need to worry about the full data 

distributions for each analyte. There are a number of options for more refined 

analyses. One in particular that may be attractive is to run the interaction 

scenario 4 times (if there are 4 chemicals interacting in the mixture). Each run 

uses the 95th percentile for one chemical and the average concentration for the 

other 3 chemicals when the first chemical is at its 95th percentile. The point 

sampling approach on pages 16 to 17 is worth mentioning but appears to be 

impractical. 

Page 19, bottom para – The issue of age-related variability is briefly discussed. 

Some contextual discussion is needed of the types of immaturities that may 

predispose to PK or PD interactions in early life as opposed to adults. One can 

cite the early life vs. adult OP studies showing greater sensitivity in young rats 

with mechanistic work demonstrating both PK and PD bases for the sensitivity 

D-11
differences. This has obvious bearing on the potential for OP to OP interactions 

in young vs. old rats. 

Page 27 – middle of page – “some number of components would be at a 

significant fraction of the NOAEL. These components would survive the initial 

analysis.“ 

What is meant by a significant fraction? Several lines later it states that 

others may be present at levels below or near RfD/N and so could be excluded. 

This language is too vague. The screening level RfD/N approach needs to be 

clarified with regards to what exposure level (relative to RfD/N) would constitute a 

significant interaction potential. 

Pages 36-38 and beyond – discussion of PBPK modeling of chemical 

interactions – this section mentions some key parameters that will be needed but 

does not describe how these parameters can come together to predict the 

outcome of the interaction – e.g., that the ratio of tissue concentration to Km is a 

key factor for competitive interactions involving M-M kinetics. It would be useful 

to have a table of the key interactive equations (and parameter definitions) for the 

major types of interactions: M-M enzyme competive, non-competitive; 

competition for protein binding sites (e.g., albumin, CEs), enzyme induction, etc. 

Page 38, bottom: should describe some of the issues with recombinant CYP 

systems (e.g., expression systems may not also express electron transport 

enzymes or they may be in wrong proportions). 

II. FINAL REVIEW COMMENTS 

II.A. Document Overview 

Dr. Lipscomb provided an overview of the document and reviewed the charge 

questions. He explained that pharmacokinetics is important in evaluating human 

exposure and refining the doses used in the dose-response relationship. It is important 

in cumulative risk and mixtures assessment because the internal doses of some 

components of the mixture can alter the tissue dosimetry of other components. Dr. 

Lipscomb provided as background information the fact that Congress mandated that 

EPA conduct a cumulative risk assessment for organophosphorous (OP) pesticides that 

share a common mode of action. Because of that mandate, the EPA’s guidance 

determines that when chemicals have the same mode of action, one can use the doseaddition or response-addition approach to evaluate cumulative risk. One of EPA’s 

challenges is to develop methods to estimate risks from environmental mixtures 

containing a variety of chemicals, often acting through different modes of action. PBPK 

modeling is a valuable tool for developing the approaches to conduct cumulative risk 

assessments, with internal tissue doses being a critical step in the process. Dr. 

Lipscomb suggested that the group should keep this important issue in mind when 

reviewing the first document. 

D-12
Method of Analysis to Perform a Tissue-Based Cumulative Risk Assessment for 

Mixtures of Chemicals is a document that attempts to take a 10-step process, 

developed by the Office of Pesticide Programs under a Congressional mandate, and 

inform a cumulative risk assessor of the benefits and considerations that need to be 

recognized when the assessor applies the 10-step process. The document presents 

two examples of chemical mixtures, OP pesticides and volatile organic chemicals. Dr. 

Lipscomb stated that the key step is the decision point which is after Step 5 in the 10-

step process. The document discusses comprehensive cumulative risk assessment 

which means a physiologically- or tissue-based approach. Dr. Lipscomb believes that it 

will be difficult to get to that stage of risk assessment due to the lack of reliable data. 

He stated that the analysis still needs to be done but there might be uncertainty in the 

outcome. 

Dr. Lipscomb indicated that confusion may arise about the term “interaction,” which in 

cumulative risk assessment means that the observed toxic response cannot be 

predicted by the addition of the doses of the individual components or responses from 

the addition of individual components. Pharmacokinetics can assist in considering 

metabolic interactions and estimating chemical concentrations in the target tissue. 

Dr. Lipscomb explained that the document is intended for NCEA internal use. He 

reviewed the charge questions and highlighted that he would like comments regarding 

the decision point step of the process. 

II.B. General Comments 

Mr. Clewell requested general comments from the reviewers on the document, Method 

of Analysis to Perform a Tissue-Based Cumulative Risk Assessment for Mixtures of 

Chemicals. Dr. Richard Bull stated that the reaction network modeling references 

interrupt the flow of the document and could be made into footnotes. Dr. Gary Ginsberg 

recommended toning down the “overzealous” language (terms like “wonderful”). Dr. 

Moiz Mumtaz felt that the document did not reflect the EPA 2000 “Supplementary 

Guidance for Mixtures” document. Also, he felt that the approach for assessing 3-4 

chemicals is not adequate for more realistic environmental mixtures containing 

hundreds of chemicals. Dr. Clifford Weisel felt that the section on pharmacodynamics 

was rushed and Dr. Bull agreed; they decided to address this issue in the responses to 

charge questions. Mr. Clewell agreed with the previous comments and added that there 

should be discussion of all the different levels of interactions that can occur, not just 

metabolic enzyme interactions. He also added that these metabolic interactions 

typically do not occur at low doses, which is needed for the assessment to be 

interesting in terms of evaluating interactive effects. Dr. Ginsberg agreed that it was 

important to point out that other pharmacokinetic interactions exist, but to state that the 

document focuses on the 10-step process. Dr. Mumtaz agreed that other types of 

interactions should be mentioned and acknowledged, but that it should be pointed out 

that they wouldn’t be completely explained or solved in the document. Dr. Ginsberg did 

D-13
not feel that the document was user-friendly and would like a flowchart of the 10-step 

process included up front in the document to help the reader understand where they are 

in the process as they move through the steps. Mr. Clewell agreed and would also like 

a diagram of the levels of interaction included. 

II.C. Response to Charge Questions 

1. Does the document clearly distinguish interactions at the level of cumulative risk 

assessment and pharmacokinetics? 

Reviewer Response: Yes, it does, for the most part, but more discussion in the 

early part of the document would be helpful (section 1.6). Examples of published 

interactions and their impact would also help. E.g., mention of interaction 

threshold dose studies. Case studies could be used to demonstrate the different 

levels of interactions. 

There needs to be some discussion of other levels of interactions (e.g., 

pharmacodynamic) between the two extremes currently identified in the 

document. 

EPA Response: Section 1.6 has been substantially expanded. Examples of 

some published interactions have been added. The second paragraph contains 

a description of a work by Dobrev and colleagues that demonstrates interactions 

thresholds. This constitutes one case study where an interaction threshold has 

been determined. 

Some discussion of other levels (types) of interactions has also been added to 

section 1.6. The fifth paragraph contains the example of piperonyl butoxide 

(PBO) and a Detoxicated insecticide. Here, PBO exerts an effect which is 

unlikely associated with the toxicity of PBO, but which markedly decreases the 

ability of the organism to metabolize the active insecticide. Also, the example of 

thioacetamide and carbon tetrachloride has been included in paragraph 6. This 

example demonstrates how pretreatment with one toxic compound can be 

protective against subsequent and otherwise lethal doses of another compound. 

Finally, paragraph 8 contains an example indicated in the reviewers’ comments 

to charge question 3. The OP inhibition data from the Chambers et al. and 

Cohen et al. publications have been included in this paragraph. 

Does the document present the benefits of employing tissue dose, rather than 

environmental concentration as the basis for cumulative risk assessment? 

Reviewer Response: A better case is made for the value of tissue dose for 

pharmacokinetic interactions than for its value for cumulative risk assessment. 

Need to include a hypothetical case study of receptor interaction in section 

2.3.1.4. 

D-14
EPA Response: The modifications made in response to Charge question 1 

largely address this comment. The additions specifically include measures of 

internal dose (especially the example by Dobrev and colleagues) for use in 

interactions as well as in cumulative risk assessment. The report has been 

challenged to include characterizations of receptor interactions in multiple 

locations have been suggested by one particular reviewer. We feel that the 

document adequately conveys its general points without the inclusion of a 

receptor interaction example. 

2. The decision point for continuing to develop a comprehensive cumulative risk 

assessment (a PBPK-based approach and assessment) is presented in Step 4 of the 

analysis plan. It is based on the assumption of chemical additivity, and includes a 

Hazard Index type analysis coupled with an uncertainty factor intended to address 

interactions (in the broad definitional sense). This point has drawn appreciable 

comment previously. The intent of the document is to communicate considerations 

undertaken in the series of choices relating to resource expenditure for CRA. Please 

comment on the clarity and strength of the rationale presented and propose other 

considerations you feel may be useful to incorporate. Please comment specifically on 

the proposed 1000-fold uncertainty factor and on conditions when it may be warranted. 

Below is an example comment and response from the Internal Review relating to this 

point: 

Internal Review Comment, Proposed “Rules of Thumb” for Estimating Threshold 

of Toxicological Interactions 

INTERNAL REVIEWER COMMENT “Thus, for practical purpose of conducting CRA” 

Interesting rule, but the stated rationale (stated at the end on page 19) is pretty weak. 

The screening calculation might make more sense if stated in terms of an average 

hazard index (i.e., sum of each chemical’s exposure potential /RfD, and that sum then 

divided by N) or relative potency or some other normalizing method, then applying an 

additional screening factor (like an uncertainty factor or margin of safety factor) to 

account for potential synergistic interactions. One could also evaluate the potential for 

antagonism and a lower level of hazard for a mixture. A fixed screening factor for 

synergistic interactions (100 or 1000) could be consistently applied for all CAGs, but I 

think a CAG specific factor would be better. The case by case factor could include 

some weighting for the number in the group, although I think better based upon what we 

know about the chemicals’ metabolism or other binding, what we know about 

competitive inhibition/binding at different dose levels for the CAG chemicals, or about 

chemical reactivity for the chemicals in the CAG. A similar method could apply for a 

cancer index using potential exposure over the 10^6 risk level (from slope factors or 

linear extrapolations from the NOAELs as the POD 10%). 

RESPONSE: This is a very good comment; likewise it was echoed in those from the 

other reviewer . This issue (how best to develop a method to cull out potential 

scenarios for which a Comprehensive Cumulative Risk Assessment should be 

performed) will be raised for discussion at the level of external review. It may be that 

such a point (when to not do such an assessment) should not be specifically “codified”, 

D-15
but be left to be addressed on a case-by-case basis. This issue will be forwarded to the 

external review panel for comment. 

Reviewer Response:

A. Conceptually, such a screening rule of thumb is a good idea. Practically, the 

difficulty is knowing the relationship of the RfD and the interaction threshold. 

Should use target organ toxicity data, not most sensitive NOAEL, since the 

latter may not be for same site. 

B. A default factor of 1000 should not be applied; instead an RfD equivalent 

should be derived for the appropriate effect, as explained in the TTD 

document. Secondary endpoints (e.g., developmental) should also be 

considered. 

C. The document should make the point that the interaction threshold may be 

determined by pharmacokinetic factors (for example, saturation of enzyme 

systems and enzyme inhibition or induction), and not a toxicity threshold. The 

interaction threshold may determined by data on the dose/response for the 

induction of an enzyme, for example, but this should not be confused with a 

threshold for a toxic effect. 

D. Case by case informed scientific judgment based on knowledge of the 

mechanism of action is necessary. The proposed rule of thumb may be 

valuable for the organophosphate common mechanism group, but it is 

questionable to apply it for metabolic interactions, without data on the 

relationship of the dose-responses for metabolic interactions and toxicity. 

EPA Response: 

A. The proposed rule of thumb has been removed and the issue has been 

characterized as one that should be undertaken on a case-by-case basis. 

B. The default factor of 1,000 has been removed. The entire Rule of Thumb text 

has been removed. Section 2.2.4 has been substantially expanded and now 

includes a worked example of a Target Organ Toxicity Dose example as 

suggested by the reviewers. 

C. That toxicokinetics (e.g., metabolic interactions) can be the determinant of 

mixtures interactions has been clearly included, see response to charge 

question 1 and section 1.6 of the document. 

D. The approach has been restructured and communicated in a manner to 

clearly indicate that the decision to proceed with developing a physiologically 

based pharmacokinetic analysis to mixtures risk should be undertaken on a 

case bycase basis. 

3. Are you aware of any additional information which can be used to improve the 

present draft report? 

Reviewer Response: 

A. Examples from the literature, demonstrating the effect of interactions on 

toxicity. Presentation of the literature on receptor interactions. Mention of other 

D-16
types of interactions (enzyme induction, non-competitive inhibition, etc.). 

Equations describing interactions. 

B. A good example of a mechanism of interaction that would have very little to 

do with the proposed RfD-based cumulative analysis is the potentiation of OP 

toxicity by tieing up serum carboxylesterases. Pretreatment with low toxicity 

pesticides like fenitrothion and other low toxicity chemicals such as bis-pnitrophenyl phosphate (BNPP) can potentiate malathion, soman, etc. The 

LOAEL, NOAEL and RfD for the potentiating agents would be based upon other 

endpoints that would not reflect their ability to interact with OPs. 

References: Chambers, et al., Effects of 3 reputed carboxylesterase inhibitors 

upon rat serum esterase activity. Neurosci Biobehav Rev 15: 85-88, 1991. 

Cohen, Mechanisms of toxicological interactions involving organophosphate 

insecticides. FAT 4: 315-324, 1984. 

EPA Response: 

A Other types of interactions have been included, specifically in section 1.6. An 

example of enzyme induction has not, however, been included, neither has one 

on receptor binding been included. The report is constructed to guide the risk 

analyst through the process of justifying and undertaking physiologically based 

analysis of mixtures and or cumulative risk. As such, more than a few examples 

are not warranted. 

B The information contained in the references suggested by the reviewers has 

been included in passages inserted into section 1.6. This passage describes OP 

and AChE interactions in more detail. 

4. Are there assumptions or uncertainties being made in this exercise that are not 

articulated? 

Reviewer Response: Use of Hazard Index approach for estimating threshold for 

metabolic interactions. (although effect threshold is ultimate issue.) 

EPA Response: The application of the Hazard Index approach does not and is 

not intended to address metabolic interactions. Perhaps the reviewer 

misunderstood something. No change made. 

5. Are the figures and tables informative? Would revision of the tables or figures 

improve the clarity of the report or its conclusions? If so, what are they? 

Reviewer Response: Need flow chart of 10 steps and diagram of multiple levels 

of interactions (PK, PD). 

Need table of definitions. 

D-17
EPA Response: A flow chart of operations has been developed and inserted as 

Figure 1. It divides the activity into the Initial Assessment, comprising steps 1-5, 

and the Dosimetry-Based Cumulative Risk Assessment, comprising steps 6-10. 

6. Are there important publications missing from the reference section? 

Reviewer Response: Mehendale et al. (effects on repair of injury). Swartout et 

al. (combining HQs). Target organ toxicity doses (EPA/ATSDR publication). 

Krishnan et al. (BTEX). Andersen et al. (TCE/DCE). Bull et al. (TCA/DCA). 

Chambers et al., Pope et al. (OP mixture studies). ATSDR interaction profiles. 

Rider et al. (human OP studies). Paul Price (P3M). Herzberg (MIXTOX 

database). 

EPA Response: 

• The reviewers point out several areas of work which could enhance the depth 

of the report and/or broaden its treatment of uncertainty. The report has been 

revised to include an example from Mehendale (Tissue repair: an important 

determinant of final outcome of toxicant-induced injury. Toxicol Pathol. 33:41-

51, 2005) on pre-exposure and its effect on stimulating tissue repair near the 

end of section 1.6. 

• Swartout’s reference on probabilistic approaches to developing reference 

doses (A probabilistic framework for the reference dose (probabilistic RfD) 

Risk Anal. 18:271-82, 1998) was considered but not included due to its 

indirect relationship to pharmacokinetics and the assessment of exposure. 

• A complete and substantial treatment of Target Organ Toxicity Doses has 

been incorporated in section 2.2.4. References from the US EPa and from 

Mumtaz have been cited. 

• The reviewer suggests inclusion of a publication by Krishnan and colleagues 

on BTEX. Instead, the report has been revised to include a reference from 

Dobrev and colleagues that addresses the same issue (thresholds for 

metabolic interactions) as does the Krishnan paper. This example can be 

found a couple of pages into section 1.6. The Bull publication on TCA/DCA 

could be one of several that, en masse, conclude that dichloroacetic acid is a 

metabolite of trichloroacetic acid and that the liver tumors produced by DCA 

and TCA are phenotypically different, when cell membrane proteins are 

evaluated. Bull and colleagues concluded that, when administered 

separately, TCA and DCA promoted the outgrowth of different subpopulations 

of spontaneously-arising tumors in rodents. A complicating factor for 

estimating internal doses of DCA is that administration of DCA at high doses 

tends to inhibit its own metabolism, prolonging biological residence time. It is 

not immediately clear how the inclusion of these points would improve the 

manner in which the document informs the risk assessor in choosing whether 

and how to implement PBPK modeling to refine mixtures or cumulative risk 

assessment. 

• A PubMed search revealed no returns for organophosphate mixture studies 

published by Jan Chambers or by Carey Pope. 

D-18
• The reviewer suggests that the reference list should include some 

Interactions Profiles developed by the ATSDR. A reference to that database 

has been added to section 2.2.1. 

• Extensive searching for Rider and human organophosphate returns was 

fruitless. However, a chapter in the PhD dissertation (2005) for Cynthia V 

Rider demonstrates the development and application of a mathematical 

model for inhibition of a detoxicating enzyme in a mixtures toxicity context: 

An Integrated Addition and Interaction (IAI) model of mixture toxicity was 

constructed and validated using a ternary mixture of organophosphates 

(malathion and parathion) and the P450 inhibitor piperonyl butoxide. Individual 

chemical concentration-response parameters and binary interaction data were 

used in the model. Modeled data was compared to experimentally derived data 

from Daphnia magna acute toxicity assays. The IAI model provided a good fit to 

the data. Results indicated that toxicokinetic interactions could be quantified and 

incorporated into mixture toxicity models. 

http://www.lib.ncsu.edu/theses/available/etd-01022006-

223335/unrestricted/etd.pdf 

However, this is not a human data set, and the example is the one for 

piperonyl butoxide, already incorporated into Section 1.6. No change was 

made. 

• The reviewer has suggested that the document could be improved by 

including reference to the work done by Paul Price and collaborators, 

especially that from the P3M database. This database contains measured 

variability in physiological parameters in humans. Because variation in these 

parameters may influence internal dosimetry, they can influence risk. 

Application of the data in the database would improve the estimates of 

variability in internal dosimetry for a given chemical, but because variability in 

a given direction for one parameter can increase dosimetry for one compound 

while decreasing dosimetry for another compound, it is not clear that inclusion 

of this work would increase the document’s treatment of internal dosimetry as 

a modulator of risk, except when a PBPK model has already been developed 

for the chemical(s) of interest. As such, this work (abstract from the 2003 

meeting of the Society for Risk Analysis below) has not been included in the 

revisions to the report. 

Modeling Inter-individual Variation in Physiological Factors Used in PBPK 

Models of Humans, by PS Price et al. SRA, 2003, 

Modeling interindividual variation in internal dose in humans using PBPK models 

requires data on the variation in the physiological parameters across the 

population of interest. These data should also capture the correlations between 

the values in each person. In this project, we developed a tool to provide such 

data and its correlations. The tool provides a source of data for human 

physiological parameters where 1) the parameter values for an individual are 

correlated with one another, and 2) values of parameters vary according to 

interindividual variation in the general population, by gender, race, and age. The 

parameters investigated in this project include: 1) volumes of selected organs 

and tissues; 2) blood flows for the organs and tissues; and 3) the total cardiac 

D-19
output under resting conditions and average daily inhalation rates. These 

parameters are expressed as records of correlated values for the approximately 

30,000 individuals evaluated in the NHANES III survey. Software was developed 

that allows records to be retrieved randomly from the database with specification 

of constraints on, age, sex, and ethnicity. The [P3M] database and 

accompanying software together provide a convenient tool for parameterization 

of human PBPK models for the study of interindividual variation. In addition, the 

data provides a useful information on the variation in physiological parameters in 

adults and children. This work was funded by the American Chemistry Council. 

• The reviewer also suggested the MixTox database developed by Hertzberg 

and collaborators. While this database was useful, especially in assessing 

the likelihood of interactions in binary mixtures, it has not been maintained 

and is no longer publicly available. 

7. What additional information would you like to see presented? 

Reviewer Response: 

A. Receptor theory (e.g., from textbook by Pratt and Taylor). Mechanism vs. 

mode of action and implications for interactions (AChE reversible inhibition vs. 

aging vs. ion channel disruption. The document over-simplifies the nature of OP 

inhibition of AChE, which can be either reversible or irreversible (aging). 

Complex mixture issues (DBPs, gasoline). 

B. Some discussion of time element (variation in exposure) and linkage to 

exposure modeling. 

C. How to identify potential exposures is not covered. Need indication of 

completed exposure pathway for mixture to define need for CAG. 

D. Need clearer criteria for step 5 cutpoint. (availability of data?, evidence of 

interaction?). Just common metabolism enzymes is not the only criteria. 

EPA Response: 

A. 

• The document will not be revised to include receptor theory. The reviewer 

seems to indicate that the document would benefit from inclusion of the 

distinction between mechanism and mode of action. The basis for deciding 

the grouping of chemicals for CRA is the Common mechanism Group (CMG) 

per US EPA guidance. Note that this distinction is made at the level of 

“Mechanism”, not Mode. A key point underlying this comment is perhaps 

sentiment that critical toxicologic interactions can be based on effects that are 

not directly involved in the mechanism of action, or are based on events 

included in the mode, but not the mechanism. Such an interaction based on 

the former is exemplified by piperonyl butoxide, as can now be found in 

Section 1.6. The opening passages in section 1.6 have also been revised to 

include a distinction between Mechanism and Mode and a caveat about how 

D-20
focusing only on mechanism can result in an overlooking of important bases 

for chemical interactions. 

• The document has been revised to include a more technical treatment of OP 

interactions, specifically to include binding half-life and aging of enzyme 

complexes. This can be found in a separate paragraph near the end of 

section 2.2.4. 

• The reviewer indicates that complex mixtures such as gasoline and drinking 

water disinfection byproducts should be addressed. It is felt that the 

considerations presented in this document apply equally to simple and to 

complex mixtures. No revision made. 

B. The reviewer indicates that complex mixtures such as gasoline and drinking 

water disinfection byproducts should be addressed. It is felt that the 

considerations presented in this document apply equally to simple and to 

complex mixtures. No revision made. 

C. Per the reviewer’s suggestion, a passage has been added to the end of 

section 2.2.2 which communicates the 5 steps identified by ATSDR and 

references their 2005 guidance. 

D. There is not going to be a more clearly made cut-point. The document 

instead espouses a case by case approach to the decision. 

8. Please comment on the overall quality of the report and analysis. What is your 

overall evaluation of the scientific content, readability and utility of the draft report? Do 

you have any suggestions relative to structure or content that would improve the quality 

of this draft report? 

Reviewer Response: Excellent explication of mixtures issues within the context 

of a standard paradigm. The document makes its main point (the importance of 

considering metabolic interactions) well. The main requirement to improve it is 

some discussion of other levels of interaction (PD) and examples from the 

literature of the impact of PK and PD interactions on cumulative risk assessment. 

Put reaction network discussion in footnotes or delete. 

Tone down language (“wonderful”). 

“Comprehensive” implies more than just target tissue based. Refer to as 

“dosimetry-based” instead. 

EPA Response: 

A. Several additional interaction types and results have been included in the 

report including metabolic induction, metabolic inhibition, and alterations of 

biological response (dynamics, induction of tissue repair). 

D-21
D-22

B. Reaction network modeling was mentioned in two places in the draft report. It 

was retained in the first instance, but was there relegated to a footnote. It was 

deleted in the second instance. 

C. There were several subjective and biased terms included in the external 

review draft. These sporadic instances of overzealousness have been replaces 

with more objective terminology. 

D. CCRA has been globally changed to Dosimetry-Based Cumulative Risk 

Assessment (DBCRA).