# Peer Review of Assessing the Effect of Five Gasoline Properties on Exhaust Emissions from Light

- Duty Vehicles certified to Tier

- 2 Standards

(EPAct/V2/E

- 89: Phase 3)

Part II: Data Analysis and Model Development

Prepared for

Assessment and Standards Div ision

Office of Transportation and Air Quality

U.S. Environmental Protection Agency

Prepared by

Systems Research and Application Corporation

652 Peter Jefferson Parkway, Suite 300

Charlottesville, VA 22911

EPA Contract Number: EP

- C

- 11

- 007

February 2012Page 2



# Peer Review of Assessing the Effect of Five Gasoline Properties on Exhaust Emissions from Light

- Duty Vehicles certified to Tier

- 2 Standards

(EPAct/V2/E

- 89: Phase 3)

Part II: Data Analysis and Model Development

Table of Contents

Peer Review of the EP

Act Study Analysis , Conducted by SRA International p. 3

a. Background p. 3
b. Description of Review Process p. 4

c. Compilation of Review Comments p. 4
d. References p. 2 2 Appendices

A. Resumes of Peer Reviewers p. 2 3

B. Conflict of Inter est Statements p. 49
C. Peer Review Charge p. 5 7

D. Reviews p. 6 0Page 3

TO: Kent Helmer, Connie Hart , U.S. Environmental Protection Agency, Office of Transportation

and Air Quality (OTAQ)

FROM: Brian Menard, SRA International

DATE: February 10

, 201 2

SUBJECT: Peer Review of Assessing the Effect of Five Gasoline Properties on Exhaust Emissions from

Light

- Duty Vehicles certified to Tier

- 2 Standards (EPAct/V2/E

- 89: Phase 3)

– Part II: Data Analysis and Model Development



### 1. Background

Past fuel effe cts models, such as the EPA Predictive Model and EPA’s Complex Model, were based on data collected using 1990s

- technology cars and trucks meeting Tier 0 and Tier 1 vehicle emission standards at pollutant levels an order of magnitude higher than compliance levels for current Tier2 vehicles. With the

current on

- highway fleet turning out much lower

- emitting vehicles than in past years, the U.S. Congress, the EPA and a variety of stakeholders are interested in generating an updated data set for fuel effects
mod els to guide vehicle emissions policy. Further, Section 1506 of the Energy Policy Act (EPAct) of 2005 directed EPA to generate a more current fuel effects model representing future gasoline vehicle fleet

emissions. The anti

- backsliding studies requested by Section 211(q) and 211(v) of the Clean Air Act would
benefit from the output of updated fuel effects models.

This report

, Assessing the Effect of Five Gasoline Properties on Exhaust Emissions from Light

- Duty Vehicles certified to Tier

- 2 Standards (EPAct/ V2/E

- 89: Phase 3)

– Part II: Data Analysis and Model Development (EPAct Study Analysis)

, describes the analysis and modeling of data collected in Phase 3 of the EPAct/V2/E

89 light duty gasoline vehicle fuel effects study (referred to here as “the EPAct p rogram”, or “Phase 3 of the EPAct program”). This study examined the exhaust emission impacts on vehicles of changing levels of five fuel properties (ethanol, T50, T90, aromatics, and RVP (specified as DVPE)) in a matrix of 27 gasoline fuels, developed to implement an optimal design allowing estimation of 11 effects (5 linear effects, 2 quadratic effects and four 2

- way interactions) . The range in the levels of fuel properties tested spans the ranges expected in current market fuels and in potential mid

- leve l ethanol blended fuels. The test vehicle sample consisted of 15 new 2008 light

- duty cars and trucks, selected from among high sales makes and models, to provide a representative sample of a fleet of vehicles meeting the U.S. Federal Tier 2 vehicle

emissio n standards. Given the relatively low level of emissions from these vehicles, a number of design and procedural steps were undertaken to minimize the impacts of measurement variability and other artifacts on data quality.

Vehicle emission test data coll ected in the program included concentrations and rates for typical regulated pollutants on an aggregate ( bag ) and continuous ( second

- by

- second ) basis , plus data for speciated
emissions for subset s of vehicle s and fuels . Complete data was generated for 926 tests, with 30 additional tests containing valid measurements for regulated emissions. The analysis of this data focuses on producing reduced mixed

- effect models by emission bag for the pollutants of interest (nitrogen oxides, total hydrocarbon, non

- methan e organic gases and carbon monoxide) after screening for outliers and other data quality issues. Fuel effects models were developed start ing with the set of five linear effects and the target set of six 2 nd

- order terms for which the fuel matrix was optimiz ed (total of 11 terms) , plus a set
of additional 2 nd

order terms comprising possible remaining interactive terms (for a total of 16 terms) and evaluating fit parameters for both sets of models . A conservative approach was taken on outl ying
observations

, such that statistical parameters outside specified thresholds were generally not sufficient toPage 4

remove data without identification of an underlying measurement or procedural problem. Relatively few
outliers were removed on the basis that their values were u nrealistically high ( limited to four measurements, all from the particulate mass emission dataset), but subsets of data representing particular
vehicles were removed after data quality review suggested some very low

- level measurements from these
vehicles s ubstantially affected by measurement error. Another issue encountered during analysis was
treatment of non

- detects (zero values), which were addressed via use of Tobit regressions ( employing the PROC LIFEREG procedure in SAS).

The primary objective of thi s program is to generate data to produce models with good predictive abilities for vehicles and fuels beyond those specifically tested in this program. Thus, minimizing retention of terms that are based largely on measurement or design artifact or are othe rwise overfit to these vehicles and
fuels is a concern.

EPA sought a n expert peer review of the EPAct Study Analysis , including reviewers’ opinion on the appropriateness of the statistical techniques described in the report and their appropriateness in th e context of data accuracy/quality issues . This report documents the peer review. Section 2 of this memorandum describes the process for selecting reviewers, administering the review process, and closing the peer review. Section 3 summarizes reviewer co mments according to the series of specific questions set forth in the peer review charge. The appendices to the memorandum contain the peer reviewers’

resumes, completed conflict of interest and bias questionnaires for each reviewer, and the peer review

c harge letter.



### 2. Description of Review Process

In October 2011, OTAQ contacted SRA International to facilitate the peer review of the EPAct Study Analysis

. EPA provided SRA with a list of subject matter experts from academia , consulting, and industry t o serve as a starting point for identification of peer reviewer candidates. SRA selected three independent (as
defined in Sections 1.2.6 and 1.2.7 of EPA’s Peer Review Handbook, Third Edition) subject matter experts to conduct the requested reviews. SRA selected subject matter experts familiar with statistical analysis and
vehicle emissions . To ensure the independence and impartiality of the peer review, SRA was solely responsible for selecting the peer review panel. Appendix A of this report contains t he resumes of the three peer reviewers. A crucial element in selecting peer reviewers was to determine whether reviewers had any actual or perceived conflicts of interest or bias that might prevent them from conducting a fair and impartial review of the E PAct Study Analysis . SRA required each reviewer to complete and sign a conflict of interest and bias questionnaire. Appendix B of this report contains an explanation of the process and standards for judging conflict and bias along with copies of each rev iewer’s signed questionnaire.

SRA provided the reviewers a copy of the most recent version of the EPAct Study Analysis as well as the peer review charge containing specific questions EPA asked the reviewers to address. Appendix C of this report contains the memo to reviewers from SRA with the peer review charge.

SRA delivered the final review comments to EPA by the requested date. These reviews , contained in Appendix D of this report, include the reviewers’ response to the specific charge questions an d any
additional comments the y might have had

.



### 3. Compilation of Review Comments

The EPAct Study Analysis was reviewed by Dr. Xuming He (University of Michigan), Dr. Christian Lindhjem (ENVIRON), and Mr. Brian West (Oak Ridge National Laboratory) . Appendix A contains detailed resumesPage 5

for each of the reviewers. This section is a compilation of their comments. The comments have been categorized as specific, general, and editorial. All textual edits and corrections provided by the three
reviewers are indica ted in bold type. The reviewers’ complete comments may be found in Appendix D.



### 3.1 Specific Technical Comments

The reviewers provided a significant number of specific technical comments and were generally favorable
in their reviews of the technical an d statistical aspects of the EPAct Study Analysis . This section contains
their comments and is divided into those that specifically address either questions or requests contained in the peer review charge and additional technical comments that reviewers c hose to provide

.

3.1.1 Charge Questions & Requests for Comment

In addition to encouraging reviewers to best apply their particular area(s) of expertise to review the overall study, EPA drafted six questions or requests for comment to serve as a focus for the reviewers. These were included in the peer review charge provided to the reviewers. In varying degree, the three reviewers provided direct responses to the questions or requests for comments.

1. Was the process of imputation of NMOG/NMHC results for tests/bags with missing speciation data reasonable and statistically sound? (Section 2.2)

He: Due to the speciation schedule described in Section 2.2 of the report, most tests in the dataset do not have alcohol and carbonyl measurements for bags 2 and 3. As NMOG and NMHC are calculated emission results that use speciation data, they could not be computed for the portions of the dataset without speciation. The study used imputation based on an alternate measure of hydrocarbon emissions to fill in the mis sing values. Linear location

- scale

- type models were used to imputation with special substitution of value zero for small NMOG. The report made a convincing case that the models used for imputation fit the data well, and should result in small errors and

va riability due to imputation.

It seems that the imputed values were deterministic in the study given the variables x NMHC in Equations 8

- 11. If so, statistical variability could be under

- reported in the subsequent studies. One approach to recommend here is to use multiple imputations to account for the variability. Based on the descriptions in the report, I do not think that the additional variability due to imputation would be a significant factor, but I would prefer to see a more explicit discussion and e xamination of this issue in the study.

Lindhjem: For examp l e, this statement o n Page 36 (“ The alterna t e measure “N M HC as measured by FID” (NMH C F I D ), was collected for the entire dataset, and it very tightly corre la ted with bo t h

NMOG and “tr u e” NMHC. It is thus possible to estimate NMOG and NMHC resu l ts for tests without speciation b y using correlations genera t e d from those with speciation. This technique essentially estimates the offset bet w een th e response of th e FID and the fully characteri z e d

emission s t ream, due to the incomplete measurement of ox y gena t es by the FID. For NMOG, this
estimated v alue is typically between 2‐20% higher than th e NMHC FID m e asuremen t

, depending

on emiss i on bag and fuel ethanol l e v e l. ” ) re f e rs t o an apparently unique measureme nt NMHC FID and data handling approach, and it would be useful to understand this measurement in order to
understand the validity o f this statement. The correlation (descri b ed i n section 3.2 Imputation

o f Speciated Hydrocarb o ns (NMOG, NMHC)) o f NMHC FID to N MOG and NMHC appears to

indicate that fuel ethanol level has no effect o n this correlat i on (same slope for all levels o fPage 6

ethanol, Tables 10, 1 1 & 13, 14) for Bag 2 and 3, but an ethanol slope term for Bag 1. Yet, the

cor r elations o f ac etaldehyde, formaldeh yde, and et h anol in Tab l e 82 for Bag 2 demonstrate
ethanol still increa s e s these oxyge n ated species and so sh o u ld increa s e NMOG (consisting usually o f NMHC + oxyge n ated carbo n ). Perhaps there is something unique about this NMHC FID

measure m ent that allows o xygenated species t o be measured at some level. However, t h e approach t o correlati n g NMOG an d NMHC with NMHC FID appears inconsistent for Bags 2 and 3 compared t o Bag 1.

West: Estimating NMOG from available NMHC appears reasonable and is consistent with what was done in the DOE V4 program (ORNL/TM

- 2011/234 and ORNL/TM

- 2011/461). For a given ethanol level,
NMOG emissions have been shown to be a linear function of NMHC emissions.

2. Was the decision to remove very low emitting and influential vehicles from th e NOx and NMOG analyses reasonable? (Sections 5.5 and 6.1)

He: Modifying or removing outliers and influential observations could raise questions about the validity of a study, especially when ad hoc decisions are made after the data are collected and

e xamined. The report paid serious attention to data quality, and described how outliers and
influential observations were identified.

At the end of Section 5.2, it was mentioned that Run 6281 in Bag 3 was removed even though it was not flagged as influent ial. The specific reason for this decision was lacking in the report.

West: These discussions were largely reasonable and convincing; however one case (run 6281) was not explained.

3. Please comment on the use of the “design set” of 11 terms as the basis of the final models, versus allowing model to fit all 17 terms, including the adequacy of the justification of this
decision. (Section 7)

He: Table 36 report (s) correlation coefficients between the linear

- effects and the additional terms (interactions ). I would suggest including the canonical correlation between the set of linear

- effects and the set of interactions. This would assess linearity beyond pairwise correlations. The considerations and justifications on finding final models (Section 7.2) were reasonable and well thought

- out. Because some subjectivity was involved in the final model selections, it is hard for me to tell whether each detailed review and scrutiny reported in Section 7.3 is “optimal”. On the other hand, there is no single model th at is likely to be the best . In reality, it is generally the case that several models are (almost) equally good given the limited amount of data, and the final selection can be made with some subjectivity. The analyses given in Section 7.3 appear reasonabl e.

Lindhjem: The approach to model i ng is well considered given the low emissions rates o f these vehicles and
the relatively small fu e l effects, often below normal detection limit s

. It was apparent that the

iterati v e process using not only novel statis t ic al techniques (compa r e d with other fuel effects evaluations), but an understanding o f th e testing limitations (low emission rates coupled with detection limits) was a necessary method to determine relevant f u el para m e ters to include in the
evaluation.

I a gree with the approach o f limiting the number o f statistically fit terms (especially t h e second
order term s ) to only those that assist in explaining the fuel effects. Perhaps the discussion of thePage 7

magnitude o f the residuals could further highlight the lack o f impact o f terms that have been dropped from the correlations such as in Figure 70, wher e a trend may exist, but is relative l y small in ma g nitude.

The overall statistical a p proach is sound using s e veral meth o ds to discover and systematically
elimina t e terms. The approach to identify influential data used appropriate physical (investigate
detection limits and o t her laboratory variables) and statistical (significance level, BIC, and residual evaluation) me t hods to eliminate o r keep data.

West: This app roach seems reasonable based on the discussion.

4. Comment on the use of the Tobit regression (SAS PROC LIFEREG) for modeling datasets with large numbers of censored values. (Section 5.3)

He: I think that the use of left

- censored models and the Tobit reg ression is appropriate. My only question here is why the same approach is not taken for cases with light censoring.

I believe that the decision to use left

- censored models in the study is appropriate. This allows
linear models to remain valid for the dat a with left end points. Such practices have been used in the statistics and econometrics literature. There are, however, several minor issues to deal with.

(a) In Section 5.2, censored measurements were replaced by the minimum positive value
measured for the emission and bag. This substitution has the potential to lower the variance estimate of the statistical models, unless censoring is taken into account in the variance estimates. If the error variance estimates are deflated, we would see higher “studentize d residuals”, resulting in false positives in outlier detection. The issue needs to be carefully
examined. (b) In the analysis of Section 5.3.1, a distinction was made between light censoring and severe censoring. In the case of severe censoring, the Tobit re gression was used in the data analysis. Otherwise, the censored values were substituted. I do not see good reasons for handling the two scenarios differently. Why not use the Tobit regression in all cases? The current practice would raise a question about the stability and sensitivity of the results if one more or one fewer data point is censored. (c) In Section 5.3.2, both BIC (model selection criterion) and likelihood ratio tests were described and used. Although both approaches are valid and useful, they ha ve different goals in mind. Model selection based on BIC is to choose models, treating all competing models equally. The
chi

- square tests have a null hypothesis in mind, where the null hypothesis refers to the smaller models in the present analysis. Test d ecisions are designed to protect the null hypothesis, so the competing models are not treated equally. When both approaches are used, one needs to be clear how they work together, and what are to be achieved. I do not imply that anything has gone wrong her e, but this part of the analysis needs to be made clearer as to why one cannot simply use BIC.

West: This approach seems reasonable, although I would like to see “large” and “small” censoring levels explained further. Censoring of less than 5 values is considered small. Why? With over 900 datapoints, could the limit be set at a higher number? Please explain.

5. Comment on the decision to independently model the linear terms and interactions between fuel blends as presented in the final results?Page 8

He: Table 36 report (s) correlation coefficients between the linear

- effects and the additional terms (interactions). I would suggest including the canonical correlation between the set of linear

- effects and the set of interactions. This would assess linearity beyond pairwise correlations. The considerations and justifications on finding final models (Section 7.2) were reasonable and well thought

- out. Because some subjectivity was involved in the final model selections, it is hard for me
to tell whether each detailed review and scrutiny reported in Section 7.3 is “optimal”. On the other hand, there is no single model that is likely to be the best . In reality, it is generally the case that several models are (almost) equally good given the limited amount of data, and th e final selection can be made with some subjectivity. The analyses given in Section 7.3 appear reasonable.

Lindhjem:

Statement ( p . 9) “ The analysis involved o n going and i t erati v e interaction bet w een stat i sti c al modeling and additional physical and chem ical review of the data

. ” Page 23 “The final design is the result of an itera t ive process involving in t e ractions between r e s e a r c h goals, th e feasib i lity of fuel blending, and e xp er imental design.”

Comment The term “in t erac t ion” should refer only to statis tically fitted second order terms where fuel parame t e rs are mixed, s u c h as ZZ e a , to represent the second order etha no l x

aromatics term. The statement above appears to use “interaction” in a different con t ext and so is confusing. EPA should also search al l other uses o f “interaction” to ensure that there is no
confusion.

Statement ( p . 13) “ Note that this generalization does not account for the ef f e ct of interactions
between R V

P and other pro p ert i es, which are in some cases larger than t he underlying linea r effects

. ”

Comment I see only two RVP interacting terms affecting only running THC (not NMOG or NMHC)
and start CO, and this statement only appears to be true for the CO start emissions. I would

sugg e st either stating this more plainly o r striking the co mment. O r does this line refer to fuel

proper t ies that are affected when RVP is modified? For examp l e, reduced T5 0 o r diluted aromatics
occur when lighter compounds are added to increase RVP; t h en this statement would be true for
the other fuel properties than just for RVP.

West: This approach seems reasonable based on the discussion.

6. Please comment on the methods used to select reduced models. (Section 5)

He: Table 36 report (s) correlation coefficients between the linear

- effects and the addition al terms (interactions). I would suggest including the canonical correlation between the set of linear

- effects and the set of interactions. This would assess linearity beyond pairwise correlations. The considerations and justifications on finding final mod els (Section 7.2) were reasonable and well thought

- out. Because some subjectivity was involved in the final model selections, it is hard for me to tell whether each detailed review and scrutiny reported in Section 7.3 is “optimal”. On the other hand, there is no single model that is likely to be the best . In reality, it is generally the case that several models are (almost) equally good given the limited amount of data, and the final selection can be made with some subjectivity. The analyses given in Sectio n 7.3 appear reasonable.Page 9

West: This discussion was convincing, although I am not an expert in this area. I understand that reduced models have lower likelihood of the models describing the random error rather than the underlying fuel effects.

3.1.2 Ot her Specific Technical Comments

All of the reviewers provided specific technical comments in addition to their responses to the specific questions in the peer review charge.

He: I have some additional suggestions, some of which might be useful for futu re studies. First, if a similar study is planned in the future, it would be better to construct specific criteria for removing outliers prior to data collection. This would eliminate questions about biased interference in the data processing stage. Second, when imputation is used, more careful procedures should be in place to account for variability due to imputation. Multiple imputation is a common approach to take. Third, some sensitivity analysis using robust statistical methods can be performed to
under stand the effects of outlying/influential points and their handling on the final analysis. Most statistical techniques, including linear mixed models and the Tobit regression used in this study, are based on the assumption of Gaussian errors. Robust statis tical methods can help us understand the impact of non

- Gaussian errors.

Lindhjem:

[1] The fuel properties in testing matrix used in the evaluation are in two respects quite
different from previous studies. The higher levels o f ethanol, up to 20%, are b eyond what an y previous study has considered. Also, the lack o f olefins evaluation data elimina t ed one fuel parameter that was found to have a significant effect o n emissions in previous st u dies.

[2] One issue to be determined is how E PA o r others will ch oose to extend the fuel property relationships developed in this report to the general fleet, such as in MOVES. Because vehicles will naturally age and may respond differently t o fuel properties, these relationships may not continue
to hold true. These hig her emitting vehicles could potentially contribute to the emissions
inventories out o f pro p ortion to their numbers. Ca r e should be taken when extending the fuel
effects to other vehicles, w h ether a g ed late m odel light‐duty o r heav y ‐duty gasoline powered
vehicles.

[3] Statement (p. 181) “ 14 The typic a l hydrocarbon analyzer us e d for emi s sion testing uses a flame

ionization d e t e ctor (FID), which is calibrated to accurately count carbon atoms that are bonded to hydrogen. Carbons bonded t o oxygen, which occur in carbonyl and a l cohol emiss i ons from burning ethanol fuels, a r e not accurately counted by the FID, and thus emissions fro m ethanol fuels requ i re additional c haracteriza t ion methods to properly quantify as NMOG or VOC.”

Comment This statement is generally, but not s t rictly, acc ur ate in that most FID un i ts are calibra t e d

o n propane, and the hydrocarbon measure m ent assumes that all carbons in the sample res p ond
the same as the ca r bon atoms i n propane. Hydroca rbons do n o t n e cessarily respond identically as propane carbon atoms do, but carbons bound to oxygen respond at a rate order(s) o f magnitude
lower. The error for most hydrocarbons has been considered insignificant. The respo n se to oxygenate carbons has bee n historically co n sidered to be insignificant compared with other hydrocarbons in the sample, so carbonyl and alcohol compounds determined through alternative

me t hods are added to the NMHC weight measure as the weight o f single carbon aldehydePage 10

(form a ldehyd e) a n d alcohol (methanol). The different composition o f hydrocarbons and an y FID response to oxygen‐bound carbon influence the NMHC mea s urement, bu t those influences are

c o nsidered m i nor. With lower emissi o n rates of these newer vehicles, t h ese assumptions may not be as appropriate as for older vehicle designs with higher emission rates. In addit i on, does the ‘typical hydrocarbon analyzer’ differ from the NMHC FID measurement th at needs to be correla t e d

with NMHC before th e statistically modeling proceeds?

[4] “8.2.2.4 Vapor Pressure” (p. 182)

Comment I f o und this section description (and others that rationalize the effect mo d e led) more o f

an unsat i sfying hand waving exe r cise to justify what the data was telling. If indeed the older
studies and this most re cent evalua t ion are to b e belie v e d, then the modeled ef f e ct may be
temperature dependent when at high temperature the fee d back o f vapors to the e ngine may affect the emissions opposite to that modeled here. The suggestion is t o limit the speculation to
the results o f the analys i s presented and note o t her re f ere nc e s if the effect found needs to be justified.

West:

[ 1 ] In Section 7, mean residuals are plotted, presumably to demonstrate the quality of the models. It would be interesting to see range bars o n the data points to show the range of individual
residuals.

[ 2 ] (p. 9) This approach was followed for several reasons: (1) the candidate fuel effected identified for study were selected because we anticipated that they could be important for one or more

emissions. Not clear. Candidate fuels? Or candidate fuel effects?

[ 3 ] (p. 13) Ethanol : taken in isolation, the models indicate that increasing ethanol is associated with increases in all emissions, both for cold

- start and hot

- running emissions. The s ole exception to the pattern is CO, for which the response to fuel properties appears to change between start and running. The effects are strongest for PM, NOx and NMOG, although presumably, the underlying physical processes could vary. Interestingly not consistent with V1 (ORNL/TM

2008/117) or V4 (ORNL/TM

- 2008/234). Increasing ethanol (in splash blends with certification gasoline) decreased NMHC and THC (FID_HC), while NMOG was relatively flat. Ethanol and acetaldehyde increased, of course. V1 used LA 92 but V4 used FTp.

[ 4 ] (p. 36) For these reasons, the decision was made not to replace zeros in the dilute bag dataset with integrated continuous measurements. Agree with this decision.

[ 5 ] (p. 50) At the outset, it is helpful to get an overview of the raw results, sorted by vehicle and fuel, which gives an initial impression of variability among vehicles and fuels, as well as within
vehicles. Agreed. Also would be helpful to show variability of individual vehicles on individual fuels.

[ 6 ] (p. 51) In the plot for etOH×T50 (Figure 18), the view seems to indicate an upward trend from E0 through E20, but with some downward curvature above E10. Show all data? How much scatter is there in NOx for these cases?Page 11

[ 7 ] (p. 52) At first glance, the trend appea rs to “zig

- zag,” from low to high. Test to test variation of Bag 1 NOx in V1 study (LA92 cycle) ranged from 10% to over 100% (same veh, same fuel). In V4 program (FTP, not LA92), range of weighted composite NOx approached 100% in some cases. How do you e nsure that coincidental test

- to

- test variation is not erroneously attributed to a fuel effect?

[ 8 ] (p. 76) The Linear Effects plot for ethanol shows some mixed results (Figure 33), but with an apparent increase from 0% to 10% ethanol, followed by a leveli ng or decline at higher ethanol
levels. Decline would be expectation. Increase from E0 to E10 is odd.

[ 9 ] (p. 93) In this initial step, censored measurements were replaced with the minimum positive value measured for the emission and bag. Minimum acro ss all vehicles and fuels?

[1 0 ] (p. 96) For minimal levels of censoring, defined as five or fewer censored measurements ( n censored ≤ 5), we elected to substitute the minimum positive measured value for the missing
measurements. After substitution we fit mixed models as described above. N<5 seems reasonable if Total number of samples is large. Suggest noting Ntotal here. Why 5? That is, why not 4 or 6 or 10? Is 5 arbitrary or is there a citable reference to why 5?

[ 11 ] (p. 100) In this case, the BIC declines steadily as terms are removed, indicating an improvement in fit for each successive reduced model. Declining steadily by 0.1 to 0.2% does not seem significant or important. Is the improvement cited below truly due to the simpler model? Perhaps , but is the BIC really an accurate indicator?

[ 1 2 ] (p. 101) Table 20. Model Fitting History for PM, Bag 1 (FM9 selected as best

- fit model). Is the sensitivity of BIC such that there is a significant difference between 2862 and 2867? Six significant fi

gures and 0.1% change in BIC does not seem important. If this is important or significant, it should be explained. Perhaps the important point is that BIC is NOT INCREASING as the model is simplified, thus justifying the simpler model?

[1 3 ] (p. 120) In t his program, dilution air was HEPA

- filtered and presumed to be free of PM, so there was no background filter sample collected for later subtraction as is typical with other

emissions. Why no tunnel blanks to prove this assumption?

[1 4 ] (p. 120) Discussio n with EPA staff experienced with PM measurement suggests that for the data as collected in this program a variability of ±1 μg should be applied to all filter weights. Considering that the net PM result is calculated by subtracting two filter weights (av erage dirty minus average clean), it should be understood to have a variability range of ±2 μg, as the measurement error applies to both weights. Therefore, a net weight gain of 10 μg would have a relative error of 20% associated with it, a figure of the same order of magnitude as the fuel effects this program attempts to capture. Need to cite reference(s) or present data to establish this level of error. For example 2005

- 01

- 0193.

[1 5 ] (p. 149) Interaction plots for selected terms are shown in Figure

. Y axis title is “mean measurement.” Isn’t this a modeled result, not a measured result?

[1 6 ] (p. 149) Another way of viewing the interactions is to average and plot the residuals of the
linear effects model. Compare modeled results to measured results. Mean is good, but wouldPage 12

also be nice to see the range of residuals. Mean can be close to zero, but how much variation is there? (average of + 20 and

- 20 is zero).

[1 7 ] (p. 161) Figure 69. NOx (Bag 1): Mean R esiduals for the Linear Effects Model, vs. Target Fuel Properties for four pairs of terms: (a) Ethanol × Aromatics, (b) Aromatics ethanol, (c) Aromatics × T90, (d) T90×Aromatics, (e) RVP × T90, (f) T90 × RVP, (g) ethanol × T50, (h) T50 × etOH. Same

commen t as above. Mean residuals look good. What is range of individual residuals? Perhaps add error bars to show min and max or perhaps interquartile range of residuals?

[1 8 ] (p. 178) This question does not arise in the context of model application, but rat her with respect to model validation, in that the datasets available for validation include data that represent emissions from pre Tier

- 2 vehicles. Such as the DOE V1 and/or V4 datasets? Have the models been compared/validated against V1 or V4 data? Not e that V1 used splash blends and ran the LA92 cycle. V4 tests also used splash blends but ran the FTP.

[ 1 9 ] (pp. 180

- 81) Thus, despite much lower overall emission levels that have been achieved in Tier 2 vehicles through improved fuel control and catalys t efficiency, the effect of ethanol on
combustion and aftertreatment(?) appears to persist in certain modes of operation such as cold

starts and transients during warmed

- up operation. Effect of ethanol on NOx emissions may be
related to exhaust stoichiome try and catalyst efficiency more so than (or in addition to) changes in engine

- out NOx.

[ 20 ] (p. 181) (Should we address effect on THC/NMHC as well?) Yes. Note that NMOG in V4 was not affected by ethanol, but FID_HC and NMHC decreased (with splash ble nds).

[ 21 ] (p. 181) In the present study NMOG decreased with decreasing aromatics content, in agreement with earlier studies. Or increased with increasing aromatics...

[2 2 ] (p. 199) With respect to censoring, the following rule was applied. If the number of censored measurements was ≤ 5, we substituted the smallest measured positive value for the missing values, and proceeded with model fitting, using a mixed

- model approach. Why 5? Why not 4 or 6 or 10? Explain.

[2 3 ] (p. 199) However, if the number of censored measurements was > 5, we fit a model using Tobit regression (i.e., “censored normal regression”), an established technique for analysis of left

censored datasets. Ditto. What is significance of 5?

[2 4 ] (p. 203) Based on these results, the redu ced model FM7 was selected as the best fit. As noted in previous BIC discussion, do small variations in BIC truly indicate a difference in fit? FM5, FM6,
and FM7 all have similar BIC.



### 3.2 General Comments

The r eviewers provided general comments on the EPAct Study Analysis . Among these general comments

were evaluations of the report’s strengths, suggestions for improving and strengthening certain of its elements, and queries for further information.

He: Overall, I found the study well designed and car efully analyzed with generally accepted modern statistical tools. My comments will focus on the appropriateness of the data processing andPage 13

statistical techniques described in the report, with the purpose of improving the EPAct Study Analysis.

Lindhjem:

[1] The report

. . . presents a well‐documented appr o ach to estimating the effect that g asoline fuel properties h a v e o n late model vehicle emissions. In general, the approach to evaluating the data is equal to o r m o r e robust t h a n previous efforts, s u ch as the complex a n d predictive models

o r other fuel ef f e cts studies.

[ 2 ] I was unable to find the earl i e r r e ports “EPAct/V2/ E ‐89” referenced in the document, presumably “Assessing the Effect o f Five Gasoline Properties o n Exhaust Emissions from Light‐ Duty V ehicles certif i e d t o Tier‐2 St a ndards: Part I ‐ Study D e sign and Execution (EPAct/V2/E

‐ 89Test Progr a m Final Re po rt)”. This report likely d e s c ri b e s in better detail why the
fuel blending program could not produce an orthogonal fuel matrix. Likewise, questio ns about

t he measurement, vehicle conditioning, and other issues are not addressed in Part II. Th i s makes
it difficult to assess the relevance o f the modeling.

West:

[1] The report is long and contains tremendous detail regarding the statistics and mod eling approaches used in analysis of the EPAct/V2 data from the SwRI program. Considering its length and the technical detail, the report is well written and well organized.

[ 2 ] This reviewer has over 20 years experience in engine, emissions, and vehicle testing, but limited experience with many of the statistical methods and modeling approaches described. To the extent possible in the short time available, methods and terms were researched and explored while reviewing the EPA document. Explanations and a pproaches appear largely reasonable. Some specific questions are noted in comments in the report. For example, the Bayesian Information Criterion (BIC) is used to determine goodness of fit. In one example, the BIC for the full model
starts out at >2900 and decreases by 0.1

- 0.2% for subsequent reduced models. The discussion refers to “a steady decrease” in BIC. The reviewer suggests the significance of such a small change in BIC be discussed further. While the reduction in BIC appears insignificant , the fact that BIC is not increasing is perhaps the justification to use the reduced model (sec. 5.3.2).

[ 3 ] The authors appear to have taken great care to ensure that models are as representative as possible, and to avoid overly complex models or overfitting. The handling of “nondetects” appears reasonable. There is rigorous treatment and discussion surrounding background measurements, analyzer drift, limits of quantitation, censoring of data, etc. Nonetheless, the complexity of the problem makes it difficult to s ee whether all of the objectives were achieved or rather that some of the apparent results are any more than artifacts of an intricate math problem. As an example, all emissions data are shown by vehicle and include all tests on all fuels, to show the rang e of measurements for a given pollutant, by vehicle. These are very informative charts. However, no
data are shown to demonstrate the test

- to

- test repeatability for a given vehicle with a given fuel. Several figures (such as Figures 13

- 21) show averages, b ut would be more informative if range bars were shown to indicate max and min or perhaps interquartile range, or perhaps scatterplots with all data points.Page 14

[ 4 ] Authors discuss taking care to not overfit or model random scatter (which is good), in fact

sta ting that the Bag 3 models may be more vulnerable to measurement error due to the extremely low emissions, and opting not to report model coefficients for Bag 3 results. The reviewer agrees with and commends this decision. Handling of extremely low measure ments is difficult, and the authors discuss this issue extensively and convincingly. However, when it comes to the PM emissions measurement, there could be additional discussion and justification. For instance, one
issue that deserves additional discussion is the lack of tunnel blanks for background PM. Data from a few example tunnel blanks could provide convincing evidence of the zero background assumption. Furthermore, control data should be shown, or at least cited, to support the assumed
level of error (± 1 μg) in the PM filter weight measurements. The authors mention discussions with EPA staff as the basis for the assumed level of error. As the authors know, PM measurement is very sensitive to measurement precision and accuracy, with temperature, humidi ty, buoyancy effects, static charge and other factors greatly influencing results. In SAE 2005

- 01

- 0193, the authors detail exhaustive measures to attain an accuracy of better than ± 1 μg in their filter weights. Please provide evidence of the stated PM mea surement error.

[ 5 ] Models should be validated against independent datasets, as discussed in section 8.

[ 6 ] ( p.8) The program was conducted in three phases. Phases 1 and 2 were pilot efforts involving measurements on 19 light

- duty cars and trucks on thre e fuels, at two temperatures. This work was completed at Southwest Research Institute between September 2007 and January 2009. Have these results been published?

[ 7 ] (p. 16) This report describes the analysis of the dataset collected in Phase 3 of the E PAct/V2/E

89 program, conducted at Southwest Research Institute in San Antonio, Texas. A separate report describing the program design and data collection activities is available, but an overview is provided below. This report does not appear to be publi cly available at this time (January 2012)

[ 8 ] (p. 18) An initial sample of 19 test vehicles was chosen with the intent of representing of the
latest

- technology light duty vehicles being sold at the time the program was being launched (model year 2008). I n terms of regulatory standards, the test sample was to conform on average to Tier 2 Bin 5 exhaust levels and employ a variety of emission control technologies, to be achieved by including a range of vehicle sizes and manufacturers. I recall EPA staff ind icating (c. 2007) that the list of vehicles was a projection of future technology and/or engine families

[ 9 ] (p. 20) After some consideration, study participants agreed to rely on the aggregate data, while applying appropriate techniques do to address the resulting “censoring” of the data at low end of the range of values. Who are the “study participants?”

[ 10 ] (p. 31) As one

- stage standardization did not neutralize correlations among model terms, we applied a second stage of standardization to the 2 nd o rder terms. This report does not appear to be publicly available at this time (Jan 2012)

[ 11 ] (p. 32) Table 8 shows that the combination of one

- and two

- stage standardization neutralizes the remaining correlations, with the exception that between the etO H and T50 linear effects, as previously described. Unclear “that between the etOH and T50 linear effects (what)”Page 15

[ 12 ] (pp. 36

- 37) Based on strong correlations between these species, we developed statistical models to impute NMOG and NMHC from correspondi ng NMHC FID measurements. See ORNL/TM

2011

- 461

[1 3 ] (p. 51) Trends for individual vehicles show a general increase in NOx with increasing ethanol,

wit h some exceptions. Consistent with prior studies ORNL/TM

- 2011/234; SAE 2009

- 01

- 2723

[1 4 ] (p. 91) We ass ume that a very small but positive measurement existed but was not captured and quantified. Assigning a value of zero to these observations is an example of a common approach to censoring of observations, known as “substitution.” In this approach, a smal l but fixed quantity is substituted for the censored observations. Values used for substitution include zero, as mentioned, or small but positive quantities such as the smallest observation, a multiple of the smallest observation, the limit of quantitatio n (LOQ) or half the limit of quantitation (LOQ/2). The degree of censoring varied widely by emission and bag, as shown in Table 2 . At different stages of the analysis, we addressed censoring in different ways. These various approaches all make sense. Suggest adding couple of sentences about the various bags of the LA92. Bag 1 is cold with majority of emissions, hence less censored values. Bag 2 is hot, but includes some open

loop operation due to hard acceler ation. Bag 3 is hot start bag with very low emissions, hence majority of censored values.

[1 5 ] (p. 92) Table 1 5. Numbers of Censored Measurements, by Emission and Bag. Suggest showing total number of tests (in title or footnote)

.

[1 6 ] (p. 94) Table 17. Counts of Influential Measurements, by Emission and Bag (with “influential” defined as having a studentized

- deleted residual ≥ 3.5 or ≤

- 3.5). Suggest including total number of measurements or observations

[1 7 ] (p. 95) An additional me asurement in Bag 3 (run 6281) was removed, even though it was not flagged as influential. Explain further?

[1 8 ] (p. 95) The full sets of terms in the optimized design include terms anticipated to be meaningful for any of the emissions to be measured. Ho wever, it was not anticipated that all the terms included would necessarily be meaningful for all emissions in all bags. A closely related goal is to develop models that would be, to the extent possible, explicable in terms of knowledge of the relevant phy sical and chemical processes. Parsimonious models are preferred over full models for this purpose, as their simpler structure makes their behavior easier to assess and explain. Finally, with respect to explicability, it is much preferred to minimize the potential for overfitting, which could reduce the generality of models selected for prediction. To guide the process, we adopted several assumptions, described below. Model describes the random error rather than the desired underlying relationship. Good discussion.

[1 9 ] (p. 107) The approach to analysis of censored measurements, as described in 5.3.2, was also adopted based on guidance from the author of the DOE research. No public documents can be found on the EPAct/V2 study.

[ 20 ] (p. 116) If this pr ogram were simply trying to quan t ify the magnitude of NOx emissions from such vehicles, this level of error may be acceptable. However, since we are looking for meaningful differences in emissions between fuels, this large relative error is particularly p roblematic. Key point. Meaningful differences.Page 16

[ 2 1] (p. 117) This suggests it likely has higher measurement noise than data from the other vehicles, and thus many measurements may not be reliably distinguishable from background
levels. Good

[2 2 ] (p. 14 9) It is interesting to note that while (d) and (f) appear somewhat similar visually, the model considers the aromxT90 interaction highly significant but the RVP×T90 interaction insignificant. Explain.

[2 3 ] (p. 150) < insert physical interpretation of th ese interactions here?> Yes

[2 4 ] (p. 165) One is that the magnitudes of corresponding coefficients are generally larger for Bag 1 than for Bag 2 emissions, suggesting that the effects of fuel properties are more pronounced for
“cold start” than for “hot running” emissions. As expected!

[2 5 ] (p. 169) The reasons for these differences are not apparent, but it is clear that the relations among NOx, etOH and T50 are complex Or nonexistent?

[2 6 ] (p. 169) It may be appropriate to consider whether the Bag 3 results may be more vulnerable to measurement error attributable to low sample measurements relative to background, given the issues with measurement discussed in 6.1.1 (page 116). Yes

[2 7 ] (p. 183) Consider showing quantitative parameter changes and pe rcent change results for the models?) Good idea

[2 8 ] (p. 211) Detailed results, including models fit, fitting histories, coefficients and tests of effect are presented in Appendices Q.3

- W.3 for Bag 1 models, and Q.4

- W.4 for Bag 2 models. Not
available for review



### 3.3 Editorial Comments

Two of the reviewers undertook a thorough editing of the report

, provid ing s ignificant editorial comments. These reviewers noted typographical and formatting errors, incorrect word choice , and omissions, including miss ing references.

Lindhjem: There are num e r o us editorial corrections that nee d to be made, but most have already been
noted in the document itself o r are obvio u s from WORD program review. What fol l ows are specific suggestions either for improving the flow o r for feedback that would not otherwise be considered during normal editing. The original line is provided in “quo t ed italics” a n d the suggestions, questions, o r comments appear in suggested e d its (striketh r ough o r red) o r flat text.

Statement (p. 9) “ T

he models rep o rted in this section a r e as parsi m onious concise as the data and subject‐matter kno w ledge allow

. ”

Statement (p.13) “ Ethano l : taken in iso l at i on, the models indica t e that increas i ng ethanol is
associated with increases in all em i ssions, b o t h for cold‐start and hot‐running emiss i ons. The sole exception to the pa ttern is CO, for which the response to added e t han ol is lower CO
emissions du r

ing start but inconclus i ve for ru n n i ng conditions. fuel properties appears to change between start and runn i ng. ”Page 17

p. 22 5
“ This parameter was measured a s DVPE, but f or simplici t

y and consistency, we will refer to it as “RVP. ” ”

West: Several recommended edits and comments are provided in the marked up report, to correct typos or improve clarity. As an example, many figures could use larger fonts before final publication.
Several c ited references (especially Part I of the same report and the appendices) appear unavailable at present, and there are a few missing figures and references. But overall it is a very good draft.

( p. 8 ) An initial sample of 19 test vehicles was chosen with the intent of representing of the latest

technology light duty vehicles being sold at the time the program was being launched (model year 2008).

( p.9 ) Speciation also allowed independent analyses of selected toxics including acetaldehyde, formaldehyde, ac rolein, benzene and 1,3

- butadiene.

( p. 10 ) In generation of emissions, the effects of different fuel properties are not separable, in that it is difficult to modify one property without affecting one or more of the others.

( p. 10 ) However, the coeffi cients for different fuel properties can be directly compared, allowing assessment of the relative importance of the effects of the fuel properties on the emissions constituent being modeled.

( p. 13 ) Cold

- start (Bag 1 ) CO and hot

- running THC both have sma ll decreases in emissions with increasing aromatics. In terms of magnitude, the pattern is similar to ethanol, with PM, NOx and NMOG showing the strongest effects.

( p. 16 ) Since data on Tier 2 vehicles are critical to understanding the impact of fuel pro perty changes on the onroad vehicle fleet as increasing volumes of biofuels are introduced , EPA entered a partnership with DOE and CRC to undertake the largest fuels research program conducted since the Auto/Oil program in the early 1990s. This program is aimed specifically at understanding the effects of fuel property changes on regulated and selected unregulated exhaust emissions from later technology Tier 2 vehicles.

( p. 17 ) These five parameters were selected based on previous studies on older vehicle s as having potential to affect exhaust emissions [cite]. Missing reference.

( p. 17 ) The parameter ranges to be covered for T50, T90, aromatic content, and RVP were selected to represent the range of in

- use fuels based on a review of the Alliance of Auto mobile Manufacturers’ 2006 North American Fuel Survey [cite?]. Missing reference.

( p. 17 ) Test fuel parameter ranges were originally drafted to span roughly the 5 th to 95 th percentiles of survey results for in U.S. gasoline, though some test fuel paramet ers were adjusted after the actual blending process began.

( p. 18 ) An initial sample of 19 test vehicles was chosen with the intent of representing of the
latest

- technology light duty vehicles being sold at the time the program was being launched (model y ear 2008).Page 18

( p. 20 ) After some consideration, study participants agreed to rely on the aggregate data, while applying appropriate techniques do to address the resulting “censoring” of the data at low end of the range of values.

( p. 21 ) The methods used were very similar to those used for the modeling of the other emissions, with modifications to address issues of study design and measurement specific to these compounds.

( p. 22 ) The design and implementation of the study, including the aspects of fuel bl ending, measurement methods and logistics are described in a separate report4. Error! Bookmark not
defined. Missing reference.

( p. 22 ) It is well known for fuel properties to be moderately to strongly correlate d.

( p. 22 ) 1 This parameter was measured as DVPE, but f or simplicity and consistency, we will refer to
it as “RVP.”

( p. 25 ) Measurement methods are discussed in detail in the testing report4. Missing reference.

( p. 27 ) In addition to correlations among the linear effects and interactions, and correlations among interactions, we can see one fairly strong correlation s among the linear effects, specifically, between etOH and T50 ( R =

- 0.57).

( p.33 ) In this constant volume sampling system, the vehicle exha ust is mixed with a large amount of filtered dilution air, and a small portion of this stream is continuously withdrawn to fill a sealed bag over the course of a test cycle.

( p. 33 ) Its primary disadvantage is that the overall dilution ratio of background air to exhaust must be fixed for an entire test, and is set relatively high to avoid condensation of water vapor within the system during periods of high exhaust flow.

( p. 35 ) 1 This assessment of the situation ignores the possibility that the vehicle ac tually consumes or destroys a given pollutant species during parts of the test cycle, resulting in periods of “negative emissions”, such that the average emission level over a test is truly zero. While situation s may occur over a limited period for some e missions in a highly polluted environment, e.g., PM or NMHC in congested traffic, it is highly unlikely in an emission test cell.

( p. 36 ) Rather, these measurements to were treated as “censored.”

( p. 36 ) The alternate measure “NMHC as measured by FID” (N MHC FID ), was collected for the entire dataset, and it very tight e ly correlated with both NMOG and “true” NMHC.

( p. 37 ) On this basis, we fit linear models for NMOG and NMHC in terms of NMHC FID .

( p. 37 ) Scatterplots of NMOG vs. NMHC FID for Bag 2 are prese nted in Formatting.

( p. 38 ) For the bag 2 and 3 models, the counterpart to Equation 8 is Equation 11, which simplifies
for blends other th an E0 similarly to Equation 8, except that the slope term is always as in Equation

9.Page 19

( p. 51 ) A linear

- effects plot for ethanol is shown in Figure 13, which suggests that an ethanol effect
is visible when the data is are averaged across the other four fuel properties.

( p. 52 ) It is necessary to go a step further, and look at “interaction” or “conditional effects” plots

, starting with the interaction of ethanol and T50, which deserves special attention because

....

( p. 64 ) As in Bag 1, variability for most individual vehicles spans one third to half an order of magnitude. Missing reference?

( p. 64 ) The effects for T50 and T90 are similar in that the trends across the five T50 levels and three T90 levels are similar, with no overall trend apparent, for the same reasons noted above for Bag 1 (Figure 26, Figure 27). Missing reference?

( p. 76 ) The variability within vehicles is about 1

- 1.25 orders of magnitude. Missing reference?

( p. 77 ) The Linear Effects plot for T90 is clearly suggestive of an overall positive effect, when considering all vehicles (Figure 37).

( p. 77 ) The left

- hand point represents the same fuel as the left

- hand point in the green trend in the previous plot.

( p. 77 ) In contrast, the plots for ethanol and aromatics are suggestive of a positive or “reinforcement” interaction. In the etOH × arom view (Figure 40), the trend for the higher
aromatics level (green) appears steeper than for the lower aromatics level (black). Similarly, in the arom × etOH view ( Font.

( p. 77 ) In Figure 41.5, the etOH × T90 plot does not appear to suggest interaction, if we discount the green trend (T90=325°) as representing on ly two fuels. Missing figures

( p. 99 ) At each step, we tested the goodness

- of

- fit of each reduced model against that of the full
model using a likelihood

- ratio test.

( p. 102 ) All models based on the 16

- parameter full model, as shown in Equation 14 are

.... .

( p. 107 ) Thus, in running these models, censored values were replaced in with the minimum positive measured value in each bag for each emission.

( p. 116 ) This observation led to a closer examination of measurement error in the dataset.

( p. 117 ) This c ondition would be expected to give a zero result as discussed in Section 3.1.

( p.118 ) The data for the Sienna, though similar in their range of sample and background
measurements, was were not found to be exceptionally influential to model

- fitting and the refore

was were not removed from the dataset.

( p. 120 ) After the test is conducted, the filters are removed, placed back into the clean containers

, and returned to the clean room.Page 20

( p. 124 ) Figure 5 6 4 and Figure 5 7 5 show net measurements as a percentage o f sample for bags 2 and 3, as an attempt to understand the magnitude of the nets relative to the measurement error
where nets are small.

( p. 124 ) Two vehicles (Odyssey, Sienna) have the majority of points below those of other vehicles.

( p. 124 ) Plots of ambient and sample for bags 1

- 3 2 are shown in Figure 58, Figure 59, and Figure 60, followed by examination of nets as percentage of sample for bags 2 and 3 in Figure 61 and Figure 62.

( p. 138 ) At this point it is important to note that the results reporte d below differ from those reported above in 5.3, as well as from those reported for the DOE analysis8. Missing reference

(p. 151) Figure 65. ln(CO) (Bag 1): Two

- way Conditional Effects Plots for Three Interactions and one Quadratic term, viewed with res pect to Both fuel Parameters : (a) Ethanol×Aromatics, (b) Aromatics×Ethanol, (c) Aromatics×T90, (d) T90×Aromatics, (e) RVP×T90, (f) T90×RVP, (g)

etOH×etOH. Typo in fig c and e. T90 blue should be 300

(p. 153) Figure 66. CO (Bag 1): Mean Residuals for th e Linear Effects Model, vs. Target Fuel Properties for three Interactions: (a) Ethanol × Aromatics, (b) Aromatics × T90, and (c) RVP × T90, and the quadratic term etOH×etOH. Blue T90=300

( p. 158 ) Another possibility is that the effect of T50 in this pres entation is masked by variation in
RVP and T90 levels across these fuels.

( p. 164 ) The relations among the ethanol, aromatics and RVP coefficients are similar to Bag 1 and to each other, except that the aromatics coefficient is slightly higher than the et hanol coefficient for NMOG, while ethanol is more important to than aromatics for NMHC.

( p. 164 ) As in Bag 1, the etOH x arom and etOH x T50 interactions are reinforcements, whereas the

etOH x RVP interaction is an interference (etOH positive, RVP negative, int eraction negative).

( p. 169 ) When starting with the 16

- term model, the reduced model retains includes RVP and T90 linear terms (both insignificant), plus two interactions not included in the design model: arom×T90
and RVP×T90.

( p. 176 ) Ethanol : taken in isolation, the models indicate that increasing ethanol is associated with

increase in all emissions, both in bags 1 and 2.

( p. 180 ) The results of the present study are consistent, showing an increase in NOx emissions with an increase in ethanol level, regardless of whether it is from a statistical analysis of an orthogonal change in ethanol alone or when changes in other fuel properties typical of splash or match blending are included.

( pp. 180

- 81 ) Thus, despite much lower overall emission levels that have been achieved in Tier 2 vehicles through improved fuel control and catalyst efficiency, the effect of ethanol on combustion and aftertreatment(?) appears to persist in certain modes of operation such as cold

- starts and transients during warmed

- up oper ation.Page 21

( p. 181 ) In the present study NMOG decreased with decreasing aromatics content, in agreement
with earlier studies.

( p. 182 ) However, during cold start, the new data suggests the direction of the effect is dependent
on ethanol content.

(p. 182 ) The present study shows little or no effect of T90 on NOx, which is reasonable considering that NOx production and control are largely about heat release, and T90 represents a smaller
amount of combustible material at cold

- start temperatures, and thus le ss energy, than T50. Not clear, please review/reword.

( p. 184 ) More variation is seen in T50 and RVP, which span somewhat wider ranges of 160

- 195 °F
and 7.3

- 13.0 lb psi , respectively.

( p. 184 ) Use of the non

- standardized models is much more straightforw ard, in that they allow input of fuel properties in their original units, e.g., % for ethanol and aromatics, lb psi for RVP and
°F for T50 and T90.

(p. 185) The processes and methods for hydrocarbon speciation are described in greater detail in
the testin g report4. Missing reference

( p. 187 ) The “ G

- efficiency” for the full design was estimated at 51.6% for the eleven design parameters, as previously described in 2.1 2.1 above (page 22).

( p. 187 ) As shown in the table, the design efficiency drops sharply with inclusion of 2 nd order terms, to less than 5% for designs four and five.

( p. 190 ) In addition to estimating the sample measurements, as shown above, we estimated two variances, the first being the variance of the variance of the 5

- day moving average of the media
blanks ( ), and the second being a variance of random errors ( ).

( p. 193 ) The first two plots (Figure 8 1 2 ) show acetaldehyde vs. ethanol, by T50 level, in linear and logarithmic space.

( p. 202 ) If the p

- value for the test

- against

- previous is greater than the critical value, the null hypothesis of no significant differen c e t in fit between the reference and nested models is retained,
and the nested model is retained as the current best fit.

( p. 208 ) Finally, to illustrate the results of the jackknife replication procedure, Figure 83 shows cumulative distributions of coefficients for each jackknife replicate for the five linear linear

e E ffects.

( p. 208 ) The distributions for aromatics ( Z a ),T50 ( Z

5 ), a nd T90 ( Z

9 ) are similar in that they show noticeable lengthening in the lower tail, suggesting that 2

- 3 vehicle s may be influential in decreasing the values of the coefficients.

( p. 214 ) 10 References: Refs 4 and 8 cannot be foundPage 22



### 4. References

U.S. EPA (2011)

. Assessing the Effect of Five Gasoline Properties on Exhaust Emissions from Light

- Duty
Vehicles certified to Tier

- 2 Standards (EPAct/V2/E

- 89: Phase 3)

– Part II: Data Analysis and Model Development

.Page 23 

### Appendix A: Resumes of Peer Reviewers

CURRICU L U M V I T A E

N A M E: H e , X u m i ng

HO M E A DDR E S S : 657 7 H e r o n C t

. , A n n A r b o r, M I 48103

P hon e : ( 2 17 )

- 4 17

- 2 8 80

E

- m a il : h e

_ u i u c @y a hoo .c om

E DU C A T I O N B A C K G R O UN D :
P h

. D

. i n S t a ti s t i c s , 198 9

, U n i v e r s it y o f I lli no i s a t U r b a n a

- C h a m p a i g n , U S A M

. S

. i n M a t h e m a t i c s , 19 8 8

, U n i v e r s it y o f I l l i no i s a t U r b a n a

C h a m p a i gn , U S A B. S

. i n M a t h e m a t i c s , 1 9 84

, F ud a n U n i v e r s it y , S h a ngh a i

, C h i n a

.

P R E S E N T A N D F O R M ER P O S I T I O N S H EL D :

• P r o f e ss o r ( 201 1

- p r e s e n t )

, H

. C

. C a r v e r P r o f e s s o r o f S t a ti s t i c s , D e p a r t m e n t o f S t a ti s t i c s , U n i v e r s it y o f M i c h i g a n

• P r o f e ss o r ( 20 0 2

- p r e s e n t )

, P r o f e ss o r a n d J e rr y a n d A n n N e r a d P r o f e s s o r i a l S c ho l a r a t t h e C o l l e g e of L A S ( 20 0 5

- 2 0 0 8 )

, A ff i l i a t e d P r o f e ss o r a t D e p a r t m e n t o f C o m pu t e r S c i e n c e s ( 2 0 0 8

- )

, A ss o c i a t e P r o f e ss o r ( 1 9 9 6

- 2 0 0 2 )

, A ss i s t a n t P r o f e ss o r ( 19 9 3

- 1 99 6 )

, D e p a r t m e n t o f

S t a ti s t i c s , U n i v e r s it y o f I lli no i s a t U r b a n a

- C h a m p a i gn

• P r og r a m D i r e c t o r o f S t a t i s ti c s , N a ti on a l S c i e n c e F oun d a t i o n ( 20 0 3

- 2 0 0 5 )

• D i r e c t o r , I lli no i s S t a t i s ti c s O ff i c e ( s t a ti s t i c a l c on s u lti n g o ff i ce )

, U n i v e r s it y o f I lli no i s a t
U r b a n a

- C h a m p a i g n ( 20 0 0

- 2 0 0 3 )

• D i r e c t o r o f t h e S c i e n ti f i c C o mm itt e e , C e n t e r f o r S t a t i s ti c a l S c i e n c e s , C h i n e s e A ca d e m y o f

S c i e n c e , B e i j i n g ( 2 0 0 6

- )

• H ono r a r y P r o f e s s o r o f S t a ti s t i c s a t t h e S c hoo l o f M a n a g e m e n t

, F u d a n U n i v e r s it y ( 2 0 0 8

- )

• V i s iti n g C h a ngj i a n g P r o f e ss o r a t N o r t h ea s t N o r m a l U n i v e r s it y un d e r C hun g K on g S c ho l a r s h i p ( 2 00 8

- 20 10 )

• S t a t i s t i c a l C on s u lt a nt

, A r gonn e N a ti on a l L a bo r a t o r y ( 1 9 9 5

- 2 00 0 )

• H ono r a r y P r o f e s s o r , D e p a r t m e n t o f S t a t i s ti c s a n d A c t u a r i a l S c i e n c e , T h e Un i v e r s i t y o f H on g K on g ( 2 01 1

- )

• L e c t u r e r , D e p a r t m e n t o f M a t h e m a t i c s , N a t i on a l U n i v e r s it y o f S i ng a po r e ( 1 98 9

- 199 3 )

R E S E A RC H IN TE R E S T S :
T h e o r y a n d m e t h o d o l og y o f s t a ti s t i c a l i n f e r e n c e , w it h a pp l i c a ti on s t o d a t a a n a l y s i s fr o m a w i de r a ng e o f a r e as

T E A C H IN G E X P E RI E NC E:

H a v e t a ugh t s t a t i s t i c s c ou r s e s a t t h e und e r g r a d u a t e a n d g r a d u a t e l e v e l s s i n c e 1 989

P h

. D S T U D E N T S:
H a v e d i r e c t e d d o c t o r a l d i ss e r t a t i on s i n s t a ti s t i c s a n d b i o s t a t i s ti c s o f t w e n t y s t u d e n t sPage 24

E DI T O R I A L S E R V IC E S :

• C o

- E d it o r

- E l e c t ( 2 0 1 1 ) o f J o u r n a l o f t h e A m e r i c a n St a t i s t i c a l As s o c i a t i on , T h e o ry & M e th od s

.

• E d it o r o f t h e I M S B u l l e t i n ( 2 00 7

- 20 10 ) ; A ss o c i a t e E d i t o r o f J o u r na l o f t h e A m e r i c a n S t a ti s t i c a l Ass o c i a ti o n ( 200 4

- 20 10 )

, J o u r n a l o f t h e R o y a l S t a ti s t i c a l S o c i e t y : S e r i e s B ( 2 01 0

- p r e s e n t )

, S t a t i s t i

c a Si n i c a ( 19 96

- 2 0 0 2

, 200 5

- p r e s e n t )

, a n d T h e A n na l s o

f S t a t i s t i c s ( 20 04

- p r e s e n t ) ; C o

- E d it o r o f S an k h y a ( 2 00 4

- 200 7 ) ; C o

- E d it o r o f S t a t i s ti c s an d i t s I n t e r f a c e ( 2 00 7

- 20 10 ) ; A ss o c i a t e E d it o r o f S ta ti s t i c s a n d P r ob a b i l i t y L e t t e rs ( 199 4

- 20 10 ) ; E d it o r o f

J o u r n a l o f M u l t i v a r i a te A n a l

y s i s ( 1 9 9 8

- 2 00 4 ) ; G u e s t E d i t o r f o r a s p ec i a l i ss u e o f S c i e n c e i n C h in a , S e r i e s A ( 2 00 9 ) a n d C o m p u t a ti o na l S ta ti s t i c s & D at a A n a l y s i s ( 20 10 )

.

• Re f e r e e f o r m a j o r s t a t i s ti c s j ou r n a l s i n c l u d i n g T h e A m e r i c a n S ta ti s t i c ia n , T h e A n n a l s o

f

S t a t i s t i c s , J o u r n a l o f t h e A m e r i c a n S t a t i s ti c a l As s o c i a t i o n , B i o m e t r i k a , B i o m e t r i c s,
B i o i n f o r m a t i c s , J o u r nal o

f M ul ti v a r

i a t e S t a t i s t i c s, J ou r n a l o f St a t i s t i c a l P la n ni n g a n d

I n f e r e n ce s , J o u r n a l of N onp a r a m e t r

i c S ta ti s t i c s, J o u r

n a l o f t h e R o y a l S ta ti s t i c a l S o c i e t y , J o u r n a l o f C o m p u t a ti o na l and G r a ph i c a l S t a t i s t i c s, A n na l s o

f I n s t i t u t e o f S ta ti s t i c a l

M a t h e m a t i c s

, N e u r a l N e t w o r k s , E c o n o m e t r i c T

h e o r y , a n d s e v e r a l I EEE j ou r n a l s

.

O T H ER P R O F E S S I O N A L S E R V IC E S A N D R E C O G NI T I O N S :

• E l e c t e d F e ll o w o f t h e A m e r i ca n A ss o c i a ti o n f o r t h e A d v a n ce m e n t o f S c i e n c e ( A A A S) ;

• E l e c t e d F e ll o w , A m e r i ca n S t a t i s ti c a l A ss o c i a t i o n ( A S A ) ;

• E l e c t e d F e ll o w , I n s tit ut e o f M a t h e m a t i c a l S t a ti s t i c s ( I M S) ;

• M e d a l l i o n L e c t u r e r , 2 0 0 7 J o i n t S t a ti s t i c a l M e e t i ng s ;

• S p ec i a l I nv it e d a n d K e yno t e S p e a k e r a t t h e K o r ea n S t a ti s t i c a l S o c i e t y F a l l M e e ti n g , 200 8 ;

• D i s ti ngu i s h e d L e c t u r e r a t t h e 1 s t I M S A s i a P a c i f i c

- R i m M e e ti n g i n S e oul

, K o r e a , 2 0 09;

• P l e n a r y s p ea k e r

, t h e 2 01 0 I n t e r n a ti on a l C on f e r e n c e o n R obu s t S t a t i s ti c s , J un e 28

- J u l y 2

, 2 0 1 0

, P r a g u e , Cz e c h Re pub l i c ;

• C h a i r

- d e s i gn a t e o f t h e A S A C o mm itt e e o n M e e t i ng s ( 2 0 11 ) ; S e r v e d i n t h e A S A C o mm itt e e

o n S c i e n ti f i c Fr ee d o m a n d H u m a n R i gh t s ( 199 8

- 20 00 )

, A S A C o mm it t e e o n N o e t h e r A w a r d s ( 20 0 5

- 2 01 1 )

, S e a r c h C o mm itt e e f o r t h e A S A E x ec u ti v e D i r e c t o r ( 200 7 )

, A S A C o mm itt e e o n
F e d e r a ll y F un d e d R e s ea r c h ( 2 00 6

- 201 0 )

, a n d A S A C o mm itt e e o n M e e t i ng s ( 2 0 08

- 2 0 1 0 ) ;

• P r e s i d e n t o f t h e I n t e r n a t i on a l C h i n e s e S t a ti s t i c a l A ss o c i a ti o n ( I C S A ) ( 2 0 1 0 ) ; E l e c t e d bo a r d

m e m b e r o f I C S A ( 2 00 3

- 20 04 ) ; C h a i r o f t h e I C S A N o m i n a ti o n C o mm itt e e ( 200 5

, 20 06 ) ; M e m b e r o f t h e I C S A a d ho c C o mm itt e e o n a n e w A pp li e d S t a ti s t i c s J ou r n a l ( 20 06

- 2 0 0 7 ) ;

• I M S C oun c i l M e m b e r ( 20 0 4

- 2 0 0 7 b y e l e c ti on , 20 07

- 2 0 1 0 e x o ff i c i o ) ; E d it o r o f I M S B u l l e t in ( 2 00 7

- 20 10 ) ; C h a i r o f t h e I M S N o m i n a ti o n C o mm itt e e ( 2 00 5

- 200 6 ) ; C h a i r o f t h e I M S

T r a v el A w a r d C o mm i tt e e ( 200 4

- 20 06 ) ; C h a i r o f I M S a d ho c c o mm itt e e o n I M S

- C h i n a ( 20 07

- 2 0 0 8 ) ;

• S c i e n ti f i c C o mm itt e e m e m b e r o f t h e I n t e r n a t i on a l C on f e r e n c e o n R obu s t S t a t i s ti c s ( 2 00 4

- 20 11 ) ;

• C h a i r o f t h e P r og r a m C o mm itt e e o f t h e 5 8 t h W o r l d S t a t i s ti c s C ong r e s s o f t h e I n t e r n a ti o n a l

S t a t i s t i c a l I n s tit u t e ( I

S I ) t o b e h e l d i n H on g K on g 2 01 3 ;

• P r og r a m C h a i r f o r 2 0 1 0 J o i n t S t a ti s t i c a l M e e t i ng s i n V a n c ouv e r ; O r g a n i ze r o f I n v it e d

S e ss i on s at t h e 1 99 7

, 200 0

, 2 0 07

, a n d 2 0 1 1 J o i n t S t a ti s t i c a l M ee t i ng s ;

• C h a i r o f t h e I n t e r n a ti on a l P r og r a m C o m m itt e e , C on f e r e n c e o n Fr on ti e r s o f S t a ti s t i c s : B i o s t a ti s t i c s a n d B i o i n f o r m a ti c s , J u l y 0 6

, C h i n a ;

• C o

- C h a i r o f t h e S c i e nt i f i c C o mm itt e e o f t h e I M S

- C h i n a I n t e r n a t i on a l C on f e r e n c e , 2 00 8 ;Page 25

• Re vi e w C o mm itt e e f o r t h e I n s tit u t e o f M a t h e m a t i c s a n d I t s A pp li c a t i on s ( I M A )

, 20 0 9;

• M e m b e r o f S c i e n ti f i c C o mm it t ee , I n t e r n a t i on a l C on f e r e n c e o n S t a t i s ti c s a n d S o c i e t y , J u l y 1 0

- 1 2, 2 01 0

, B e i ji ng;

• C o

- C h a i r f o r t h e t r ac k Q ua n t i l e r e g r e ss io n a n d s e m ip a r

a m e t r i c m e tho d s a t t h e 3 r d

I n t e r n a t i on a l C on f e r e n c e o f t h e E RC I M W G o n C O M P U T I N G & S T A T I S T I C S

, D ece m b e r 1 0

- 1 2

, 2 01 0

, L on d on.

R E S E A RC H G R A N T S
, A W ARD S A N D S P O N S O RS

• P I o n t h e N a t i on a l S c i e n c e F oun d a ti o n A w a r d D M S

- 1 00 7 39 6 o n E f f i c i e n t M o d e l i n g i

n Q ua n t i le R e g r e s s i o n ( 0 8 /1 0

- 0 8 /1 3 )

• P I o n N a ti on a l I n s tit ut e o f H ea t h G r a n t 1 R 2 1 C A 12 9 67 1 0 1 A 1 o n N on pa r a m e t r i c

a na l y s i s of r ev e rs e

-

p h a s e p r o t e i n l y s a t e a r r a y d a t a ( w it h D r

. J

. H u a t M D A n d e r s o n Ca n c e r Ce nt e r ) ( 08 /

09

- 0 7 /1 1 )

• P I o n N a ti on a l I n s tit u t e o f H e a t h G r a n t 1 R 0 1 G M 0 80 5 0 3 0 1 A 1 o n L o w

- r an k

A p p r o x i m a ti o n to P r ob e

- l e v e l D a t a w i t h A p p l i c a ti o n t o E

x o n T i l in g Ar r a y s ( w it h D r

. J.

H u a t M D A n d e r s on Ca n c e r C e nt e r ) ( 0 7 /0 8

- 0 5 /1 2 )

• P I o n t h e N a t i on a l S c i e n c e F ound a t i o n A w a r d D M S

- 060 4 22 9 o n I n f e r e n t i a l M e th o d

s f o r Q u a n t i l e R e g r e s s i o n ( 0 8 /0 6

- 0 8 /1 0 )

• C o

- P I o n t h e N a ti on a l S c i e n c e F oun d a ti o n A w a r d D M S

- 0 7 247 5 2 o n S t a t i s t i c a l e v al u a t io n o f

m o de l

- b a s e d u n c e r t a in t i e s l e a d i n g t o i m p r o v e d c l i m a t e c h a ng e p r o j ec ti o n s ( 0 9 /0 7

- 0 8 /1 1 )

• P I o n t h e N a ti on a l N a t u r a l S c i e n c e F oun d a ti o n o f C h i n a G r a n t 1 0 828 1 0 2 o n T h e o ry a nd

a pp l i c a t io n s o

f no n p a r a m e t r i c m e t h o d s ( w it h D r

. N i n g

- Z hon g S h i a t N o r th ea s t N o r m a l

U n i v e r s it y , C h i n a ) ( 01 /0 9

- 1 2 /1 0 )

• S t a t i s t i c i a n o n t h e N a ti on a l I n s tit u t e o f H e a l t h G r a n t R 2 1 H L 09 0 45 5

- 0 1 o n M u s c l e q ua l i t y ,
e xe r c i se a n d w e i g h t l o ss i n o l d e r w

o m e n : A q u a n ti t a t i v e M RI s tu d y ( 2 0 0 7

- 2 00 9 ) ( P I : J oh n G e o r g i a d i s )

• S e n i o r p e r s onn e l o n t h e N a ti on a l S c i e n c e F ound a t i o n A w a r d D M S

- 0 8 00 6 3 1 o n St a t i s t i c a l

A p p r o a c h e s t o I n t e g r

a t i o n o f M a ss S p e c t r a l an d G e n o m i c D a t a o

f Y e a st H i s to n e
M od i f i c a t i ons ( 06 /

0 8

- 06 /1 2 ) ( P I : P i n g M a )

• P I o n t h e N a ti on a l S c i e n c e F ound a t i o n A w a r d D M S

- 0 63 0 95 0 o n A V i r tu a l C e n t e r t o
P r o m o t e C o l lab o r a t io n b e t w ee n t h e U

S

- an d C h in a

- b a s e d R e s e a r c h e rs i n S ta ti s t i c a l

S c i e n c e ( 0 8 /0 6

- 07 /

09)

• C o

- P I o f t h e N a ti on a l I n s tit ut e o f H ea l t h G r a n t R 0 1 D C 00 5 60 3 I n t e r n e t E v al u a t io n o

f

O r o ph a r y n g e a l S w a l l o w i n g F u n c ti o n ( 0 3 /0 4

- 02 /

09 ) ( P I : A d r i e nn e P e r l m a n )

• P I o f t h e N a ti on a l S c i e n c e F oun d a ti o n A w a r d D M S

- 010 2 41 1 o n Q ua n t i l e R e g r e s s i o n ( 0 7 /0 1

- 07 / 0 6 )

• P I o f t h e N a t i on a l S ec u r i t y A g e n c y G r a n t o n C e n s o r e d R e g r e s s i o n Q ua n t i l e ( 02 / 0 4

- 02 / 07)

• S t a t i s t i c i a n o n t h e N a t i on a l I n s tit u t e o f H e a lt h G r a n t R O 1

- D C 36 8 4 T h e C o o r d in a ti o n o f

R e s p i r a ti o n a n d D e gl u t i t i o n ( 0 1 /0 1

- 1 2 /0 4 )

• C o

- I n v e s ti g a t o r , R o bu s t n e ss i n G e n e r a li s e d S e m ip a r

a m e t r i c M i

x e d M o d e l s , f und e d b y t h e
R e s ea r ch G r a n t s C oun c i l o f H on g K on g ( 08 /0 3

- 0 7 /0 6 )

• P I o f t h e N a t i on a l S ec u r i t y A g e n c y G r a n t o n R e g r e s s i o n A na l y s i s ( 0 2 /0 2

- 0 2 /0 4 )

• P I o f t h e N a t i on a l S c i e n c e F oun d a t i o n A w a r d S B R

- 96 1 727 8 o n C o n s t r a in t s a n d F

l e x i b i l i t y i n M o d e l in g ( 0 9 /9 7

- 0 8 /0 1 )Page 26

• P I o f t h e N a t i on a l S ec u r i t y A g e n c y G r a n t M DA 90 4

- 9 6

- 1

- 00 1 1 o n R o b u s t m e t h o d s ( 02 / 96

- 0 2 / 98)

• S t a t i s t i c i a n , S t ud

y o n S e i s m i c R e h a b i l

i ta t io n C o s t s , f un d e d b y F e d e r a l E m e r g e n c y M a n a g e m e nt A g e n c y t h r oug h C E R L ( 2 00 2

- 20 03 )

• S t a t i s t i c i a n , C

h i l d W e l f a re S tu d i e s f un d e d b y t h e S t a t e o f I lli no i s t h r oug h t h e C h i l d r e n a n d

F a m i l y Re s ea r c h Ce n t e r , U I U C ( 2 0 0 0

- 2 00 1 )

• S t a t i s t i c i a n , D e s ig n a n d A n a l y s i s o f T r a c k W e a r T e s t s f und e d b y t h e A ss o c i a t io n o f

A m e r i ca n R a il r o a d s ( 2 0 0 1

- 2 00 2 )

• U n i v e r s it y o f I lli no i s C a m pu s Re s ea r c h B o a r d A w a r d s o n r obu s t s t a ti s t i c s ( 19 9 4

, 19 9 6

, 1 9 9 7 )

, c o m pu t a ti on a l s t a ti s t i c s ( 1 9 9 8 )

, i t e m r e s po n s e t h e o r y m o d e l s ( 2 0 0 0 )

, a n d o n b i o i n f o r m a ti c s ( 2 00 2 )

.

S ELE C TED C O N S U LT I N G E X P E RI E NC E:

• C on s u lt a n t ( 2 0 0 7

- 2 00 8 ) t o M i l l e nn i u m P h a r m ace ut i c a l s , I n c

. o n d e s i g n a n d a n a l y s i s o f

c l i ni c a l t r i a l s

• C on s u lt a n t ( 2 00 2

- 20 03 ) t o A r c h e r

- D a n i e l s

- M i d l a n d C o m p a n y o n m i c r o a rr a y d a t a a n a l y s i s

• C on s u lt a n t ( 200 1

- 20 03 ) t o t h e F e d e r a l C o mm o d it y F u t u r e s T r a d i n g C o mm i ss i o n ( C F T C ) on

e v a l u a ti n g e vi d e n c e o f i l l e g a l t r a d i ng

• C on s u lt a n t ( 19 95

- 2 0 0 2 ) a t t h e A r gonn e N a t i on a l L a bo r a t o r y o n r e s i d e n t i a l e n e r g y

c o n s u m p ti on ( s p o n s o r e d b y t h e U

. S

. D e p a r t m e n t o f E n e r gy)

• C on s u lt a n t ( 1 9 9 6 ) o n t h e r oo f i n g d a t a m a n a g e m e n t f o r U S A r m y C on s t r u c t i o n E ng i n e e r i ng Re s ea r c h L a bo r a t o r i es

• C on s u lt a n t ( 1 9 9 5 ) f o r t h e C u s t o m e r P ot e n t i a l M a n a g e m e n t C o r po r a ti o n a t E a s t P e o r i a , I lli no i s o n m o d e li n g m a r k e t i n g d a t a

S ELE C TED I N V I T ED P R E S E N T A T I O N S S I NC E 2001 :

• C on f e r e n c e o n C ont e m po r a r y M e t ho d s o f D a t a A n a l y s i s : T h e o r y a n d P r a c t i ce , A r g e n ti n a , M a r c h 2 001

• D e p a r t m e n t o f B i o s t a ti s t i c s , U n i v e r s it y o f N o r t h C a r o li n a a t C h a p e l H i l l

, A p r i l 2 0 0 1

• I n t e r n a ti on a l C on f e r e n c e o n R obu s t S t a ti s t i c s , A u s t r i a , J u l y 2 0 01

• S e ss i o n O r g a n i ze r f o r t h e 5 t h I C S A I n t e r n a t i on a l C on f e r e n c e , H on g K ong , A ugu s t 2 001

• D e p a r t m e n t o f S t a ti s t i c s , C o l u m b i a U n i v e r s it y , M a r c h 2002

• I n v it e d s p ea k e r a n d S c i e nt i f i c C o mm itt e e m e m b e r , t h e 2 n d I n t e r n a t i on a l C on f e r e n c e o n
R obu s t S t a t i s t i c s , V a n c ouv e r , J un e 20 0 2

• D e p a r t m e n t o f S t a ti s t i c s , U n i v e r s i t y o f V i r g i n i a , D e c e m b e r , 2 002

• I n v it e d s p ea k e r a n d S c i e nt i f i c C o mm itt e e m e m b e r , t h e 3 n d I n t e r n a t i on a l C on f e r e n c e o n
R obu s t S t a t i s t i c s , A n t w e r p , B e l g i u m , J u l y 2 0 0 3

• B I R S W o r k s ho p o n r e gu l a r i z a ti o n i n s t a t i s ti c s , B a n f f I n t e r n a t i on a l Re s e a r c h S t a t i o n f or
M a t h e m a t i c a l I nnov a ti o n a n d D i s c ov e r y , S e p t e m b e r , 2 003

• D e p a r t m e n t o f S t a ti s t i c s , N o r t h C a r o li n a S t a t e U n i v e r s it y , M a r c h

, 2004

• O r g a n i z i n g C o mm itt e e m e m b e r , t h e 6 t h I C S A C on f e r e n c e , S i ng a po r e , J u l y 20 0 4

• J o i n t S t a t i s t i ca l M e e t i ng s , T o r on t o , A ug u s t 2 0 04

• C o ll oqu i u m s p ea k e r “ A n E x c i ti n g T i m e f o r S t a ti s t i c i a n s ” a t t h e U n i v e r s it y o f V i r g i n i a a n d

Y a l e U n i v e r s it y , N ov e m b e r , 20 0 4Page 27

• I n v it e d s p ea k e r a t t h e 2 0 0 4 D y s ph a g i a R e s ea r c h S o c i e t y o n “ E f f ec t s o f A g e , G e nd e r , B o l us V o l u m e a n d V i s c o s it y o n A c ou s ti c S i gn a l s o f N o r m a l S w a ll o w i ng ”

, M on t r e a l

, O c tob e r 2 0 04

• T u t o r i a l l e c t u r e r o n “ Q u a n t i l e R e g r e ss i on ” a t t h e 2 00 5 E N A R M e e ti ng , M a r c h 2 005

• P ub l i c S c hoo l L e c t u r e , ” Fr o m D a t a t o D i s c ov e r i e s ” a t NU S H i g h S c hoo l

, S i ng a po r e , M a r c h 2

, 2 0 05

• D e p a r t m e n t o f M a t h e m a t i c s , U n i v e r s it y o f M a r y l a n d

, M a r c h

, 2 0 0 5

• D e p a r t m e n t o f B i o s t a ti s t i c s , C o l u m b i a U n i v e r s it y , A p r i l

, 2 0 0 5

• I n v it e d s e ss i o n o r g a n i ze r o n r ob u s t s t a ti s ti c s , t h e 2 5 t h E u r op ea n M e e t i n g o f S t a t i s t i c i a n s , O s l o , J u l y , 2 005

• D e p a r t m e n t o f B i o s t a t i s t i c s a n d A ppl i e d M a t h e m a t i c s , M D A n d e r s o n M e d i c a l Ce nt e r , O c t ob e r , 2 005

• S c hoo l o f I n d u s t r i a l a n d S y s t e m s E ng i n ee r i ng , G e o r g i a T ec h

, N ov e m b e r , 2 0 05

• T h e 5 t h A s i a n C on f e r e n c e o n S t a t i s ti c a l C o m pu ti ng , H on g K ong , D ece m b e r , 2 0 05

• T h e T M S & A M S J o i n t I n t e r n a t i on a l C on f e r e n c e , T a i w a n , D e c e m b e r , 2005

• D e p a r t m e n t o f S t a ti s t i c s & P r ob a b i l it y , M i c h i g a n S t a t e U n i v e r s it y , F e b r u a r y , 20 0 6

• D e p a r t m e n t o f I n f o r m a ti o n & O p e r a t i on s M a n a g e m e nt

, U n i v e r s it y o f S outh e r n C a li f o r n i a , F e b r u a r y , 20 0 6

• I n v it e d p a n e li s t a t t h e D y s ph a g i a Re s ea r c h S o c i e t y C on f e r e n c e , A r i z on a , M a r c h

, 2 0 0 6

• I n v it e d s p ea k e r a t t h e W o r k s ho p o n qu a n til e r e g r e ss i on , L M S m e t ho d a n d r obu s t s t a ti s t i c s i n t h e 21 s t ce n t u r y , E d i nbu r gh

, UK

, J un e , 2006

• I n v it e d s p ea k e r a t t h e 2 0 0 6 I n t e r n a t i on a l C on f e r e n c e o n R obu s t S t a t i s ti c s , L i s b o n , P o r t g a l

, J u l y, 2 006

• D e p a r t m e n t o f S t a ti s t i c s , U n i v e r s it y o f W i s c on s i n a t M a d i s on , O c t ob e r 4

, 2006

• D e p a r t m e n t o f M a t h e m a t i c s , W a s h i ng t o n U n i v e r s it y i n S t

. L ou i s , O c t ob e r 1 9

, 2 0 06

• D e p a r t m e n t o f M a t h e m a t i c s , Z h e j i a n g U n i v e r s i t y , C h i n a , S e p t e m b e r 1 3

, 2 0 06

• I n v it e d l e c t u r e a t t h e M i l l e nn i u m P h a r m ace ut i c a l s , I n c. , B o s t on , S e p t e m b e r 1 8

, 2 0 0 6

• I n v it e d s p e a k e r a t t h e B i o s t a t i s ti c s W o r k s hop

, I n s tit u t e f o r t h e M a t h e m a t i c a l S c i e n ce , S i ng a po r e , O c t ob e r 2 3

, 20 0 6.

• D e p a r t m e n t o f S t a ti s t i c s , W h a r t o n S c hoo l

, U n i v e r s it y o f P e nn s y l v a n i a , N ov e m b e r 1 5

, 2 0 06

• D e p a r t m e n t o f S t a ti s t i c s , T e x a s A & M U n i v e r s it y , D e ce m b e r 1 5

, 2 006

• D e p a r t m e n t o f S t a ti s t i c s , Y a l e U n i v e r s it y , F e b r u a r y 5

, 20 0 7

• D e p a r t m e n t o f S t a ti s t i c s , O r e go n S t a t e U n i v e r s it y , F e b r u a r y 2 6

, 20 0 7

• I n v it e d s p ea k e r a t t h e E NA R 20 0 7 m ee t i ng , A t l a n t a , M a r c h 1 3

, 2007

• D e p a r t m e n t o f S t a ti s t i c s a n d A c t u a r i a l S c i e n ce , U n i v e r s it y o f I o w a , A p r i l 1 2

, 2 00 7

• D e p a r t m e n t o f S t a ti s t i c s , T e x a s A & M U n i v e r s it y , Ma y 1 7

, 20 0 7

• I n v it e d P a r

t i c i p a n t

, W o r k s ho p o n S t a t i s ti c a l A n a l y s i s o f G e n e t i c D a t a , B a n ff

, Ca n a d a , J un e 2 4

- 2 9

, 2 007

• D e p a r t m e n t o f B i o s t a ti s t i c s , C o l u m b i a U n i v e r s it y , J u l y 1 2

, 2007

• M e n t o r f o r t h e A S A Wr i t e r s W o r k s hop

, S a l t L a k e C it y , J u l y 2 9

, 20 0 7

• K e yno t e s p ea k e r a t t h e I n t e r n a t i on a l C on f e r e n c e o n Fr on t i e r s o f S t a t i s ti c s

- H i g h

D i m e n s i o n a l D a t a , K un m i ng , C h i n a , A ugu s t 1 3

- 1 5

, 2007

• S c hoo l o f P ubl i c H e a l t h

, R o c h e s t e r U n i v e r s it y , S e p t e m b e r 6

, 2 0 07

• D e p a r t m e n t o f S t a ti s t i c s , U n i v e r s it y o f C a l i f o r n i a a t L o s A ng e l e s , O c t ob e r 2

, 2 0 07

• D e p a r t m e n t o f B i o s t a ti s t i c s , U n i v e r s it y o f N o r t h C a r o li n a a t C h a p e l H i l l

, N ov e m b e r 7

, 2007

• D e p a r t m e n t o f M a t h e m a t i c s a n d S t a t i s ti c s , M i a m i U n i v e r s it y , N ov e m b e r 1 3

, 20 0 7Page 28

• D e p a r t m e n t o f O p e r a t i on s Re s ea r c h a n d F i n a n c i a l E ng i n ee r i ng , P r i n c e t o n U n i v e r s it y , M a r c h 2 5

, 2 008

• I n v it e d s p ea k e r a t t h e f i r s t I n t e r n a t i on a l B i oph a r m ace ut i c a l S t a t i s ti c s C on f e r e n ce , S h a ngh a i

, J u l y 1, 2 008

• I n v it e d s p ea k e r a t t h e I n t e r n a ti on a l W o r k s ho p o n A p p li e d P r ob a b i l it y ( I W A P 20 08 )

, C o m p i e gn e , Fr a n ce , J u l y , 20 0 8

• I n v it e d s p ea k e r a t t h e S y m po s i u m o n A d v a n ce s i n S t a t i s ti c s , i n hono r o f t h e 6 5 t h
b i r t hd a y of P r o f e ss o r Z h i d on g Ba i

, S i ng a po r e , J u l y 2 0

, 20 0 8

• I n v it e d s p e a k e r a n d m e m b e r o f t h e S c i e nt i f i c C o mm itt e e o f t h e I n t e r n a t i on a l C on f e r e n c e o n
R obu s t S t a t i s t i c s , A n t a l y a , T u r k e y , S e p t e m b e r 8

- 1 2

, 20 0 8

• K e yno t e s p ea k e r a t t h e K o r e a n S t a t i s ti c a l S o c i e t y F a l l M e e t i ng , S e ou l

, O c t ob e r 3 1

, 2 008

• D e p a r t m e n t o f S t a ti s t i c s , U n i v e r s it y o f M i c h i g a n , N ov e m b e r 7

, 2 0 08

• I n v it e d s p ea k e r a t t h e E NA R M e e ti ng , S a n A n t o n i o , M a r c h 1 7

, 2 0 09

• D i s ti ngu i s h e d l e c t u r e r a t t h e 1 s t I M S A s i a P a c i f i c R i m M e e ti ng , S e ou l

, J u l y 1

, 200 9

• C o ll oqu i u m s p ea k e r a t t h e A S A O r e go n C h a p t e r M e e t i ng , a n d D e p a r t m e n t o f M a t h e m a t i c s , P o r t l a n d S t a t e U n i v e r s it y , O c t ob e r 2 3

, 2 009

• D e p a r t m e n t o f S t a ti s t i c s , U n i v e r s it y o f Mi nn e s o t a , N ov e m b e r 1 9

, 2 0 09

• I n v it e d s p e a k e r a n d d i s c u ss a n t a t W o r k s ho p o n S t a ti s t i c a l Fr on ti e r s , 20 0 9 I SS

, T a i w a n , D ece m b e r , 2 009

• Y a l e S c hoo l o f P ubl i c H e a lt h

, Y a l e U n i v e r s it y , J a nu a r y 1 2

, 2010

• C o ll oqu i u m s p ea k e r , D e p a r t m e n t o f B i o s t a t i s ti c s , H a r v a r d U n i v e r s it y , F e b r u a r y 1 8

, 20 1 0

• P l e n a r y s p ea k e r a t t h e 20 1 0 I n t e r n a ti on a l C on f e r e n c e o n R obu s t S t a ti s t i c s , J u l y 1

, 2 0 10.

P U BL I C A T I O NS
1

. H e

. X

, D

. G

. S i m p s o n a n d S

. L

. P o r t no y ( 1 9 9 0 ) : B r e a k d o w n R obu s t n e s s o f T e s t s , J o u r na l o f
the A m e r i c a n St a t i s t i c a l As s o c i a t i o n , vo l

. 8 5

, no

. 4 0

, 4 46

- 4 5 2.

2

. H e , X

. ( 19 90 ) : E s ti m a ti n g t h e r oo t o f a nonp a r a m e t r i c r e g r e ss i o n f un c ti o n i n a r obu s t f a s h i on, T h e A u s t r a l ia n J o u r n a l o f S t a t i s t i c s , vol

. 3 2

, no

. 2

, 21 7

- 22 6

. 3

. H e , X

. , J

. J u r e c k ov a , R

. K o e n k e r a n d S

. P o r t no y ( 199 0 ) : T a i l B e h a v i o r o f Re g r e ss i on
E s ti m a t o r s a n d T h e i r B r e a k d o w n P o i n t s , E c on o m e t r i c a , vol

. 5 8

, no

. 5

, 1 19 5

- 12 1 4.

4

. H e , X

. ( 1 9 9 0 ) : L i k e li hoo d r a t i o t e s t f o r d i s c o r d a n c y w it h s li pp a g e a l t e r n a ti v e s , C o m m u n i c a t io n s i

n S t a t i s ti c s, T h e o r y a n d M e th od s , vo l

. 1 9

, no

. 1 0

, 35 8 5

- 3 5 9 4.

5

. H e , X

. ( 19 90 ) : C o mm o n

- s e n s e m ono t on i c it y o f r obu s t e s ti m a t o r s , M e t r o n , vo l

. 4 8

, 1 4 1

- 1 47.
6

. H e , X

. ( 1 9 9 1 ) : A l o c a l b r e a k d o w n p r op e r t y o f r ob u s t t e s t s i n l i n ea r r e g r e ss i o n , J o u r n a l of M u l t i v a r

i a t e A n a l y s i s , vo l

. 3 8

, no

. 2

, 2 9 4

- 3 0 5.

7

. H e , X

. a n d D

. G

. S i m p s o n ( 19 9 2 ) : R obu s t D i r e c t i o n E s ti m a t i on , T h e A nn a l s o f S t a t i s t i c s , vol

. 2 0

, no

. 1

, 35 1

- 36 9

. 8

. H e , X

. ( 1 9 9 2 ) : R obu s t S t a ti s t i c s o f D i r e c t i on a l D a t a : A S u r v e y

. N o n p a r a m e t r i c S t a ti s t i c s and R e la t e d T

o p i c s , e d i t e d b y A

. K

. M d

. E. S a l e h

, N o r t h H o l l a nd

, N e w Y o r k

, 8 7

- 9 6.
9

. H e , X

. a n d S

. L. P o r t no y ( 1 9 9 2 ) : Re w e i ght e d LS e s ti m a t o r s c onv e r g e a t t h e s a m e r a t e a s
t he i n i t i a l e s ti m a t o r

. T

h e A n na l s o

f St a t i s t i c s , vo l

. 2 0

, no

. 4

, 216 1

- 21 6 7.

1 0

. H e , X

. a n d D

. G

. S i m p s o n ( 1 99 3 ) : L o w e r B oun d s f o r C on t a m i n a t i o n B i a s : G l ob a ll y M i n i m a x V e r s u s L o ca ll y L i n ea r E s ti m a ti on

. T h e A nn a l s o

f S ta ti s t i c s , vo l

. 2 1

, 31 4

- 33 7

. 1 1

. C h a n , Y

. M

. a n d H e , X

. ( 19 94 ) : O n M e d i a n

- t yp e E s ti m a t o r s o f D i r e c t i o n f o r t h e vo n Mi s e s

- F i s h e r D i s t r i bu ti on s

. B i o m e t r i k a , vo l

. 8 0

, 8 6 9

- 8 7 5.Page 29

1 2

. C h a n , Y

. M

. a n d H e , X

. ( 1 99 4 ) : A s i m p l e a n d c o m p e t i ti v e e s t i m a t o r o f l o c a t i on

. S t a ti s t i c s

a n d P r o b a b i l

i t y L e t t e r s , vo l

. 19

, 1 3 7

- 1 4 2.

1 3

. H e , X

. a n d P

. S h i ( 1 99 4 ) : C onv e r g e n c e r a t e o f B

- s p li n e e s ti m a t o r s o f n onp a r a m e t r i c

c on d i ti on a l qu a nt i l e f un c t i on s

. J

. N onp a r a m e t r i c St a t i s t i c s , vo l

. 3

, 2 99

- 3 0 8.

1 4

. M a r k a t ou , M

. a n d H e , X

. ( 1 9 9 4 ) : B ound e d i n f l u e n c e , h i g h b r e a k d o w n po i n t t e s ti n g
p r o ce d u r e s i n l i n ea r m od e l s

. J

. A m e r. St a t i s t

. As s o c

. , vo l

. 8 9

, 54 3

- 54 9

. 1 5

. H e , X

. ( 19 94 ) : B r e a k d o w n v e r s u s e ff i c i e n c y , you r p e r s p ec ti v e m a t t e r s , a s G u e s t E d it o r f or S t a t i s t i c s a n d P r o ba b i l i t y L e t t e rs , vo l

. 1 9

, no

. 5

, 3 5 7

- 3 6 0.

1 6

. H e , X

. a n d G

. W a n g ( 19 95 ) : L a w o f t h e I t e r a t e d L og a r i t h m a n d I nv a r i a n c e P r i n c i pl e f o r
M

- e s ti m a t o r s

. P r o c e e di

ng s o f t h e A m e r

i c a n M at h e m a t i c a l So c i e t y , vol

. 1 2 3

, no

. 2

, 563

- 5 7 3.

1 7

. C u i

, H

. a n d H e , X

. ( 1 99 5 ) C on s i s t e n c y o f s e m i p a r a m e t r i c e s ti m a t i o n o f e l l i pt i c d e n s iti e s , A c t a M at h e m a t i c a S in i c a , n e w S e r i e s , vo l

. 1 1

, 4 4

- 5 8.

1 8

. H e , X

. a n d G

. W a n g ( 1 9 9 6 ) : C r o ss

- c h ec k i n g u s i n g t h e m i n i m u m vo l u m e e lli p s o i d

e s ti m a t o r

. S t a t i s t i c a Si

n i c a , vo l

. 6

, no

. 2

, 36 7

- 37 4

. 1 9

. H e , X

. a n d P

. S h i ( 1 9 9 6 ) : A s y m p t o ti c s f o r M

- t yp e r e g r e ss i o n s p li n e s w it h a u x il i a r y s c a l e

e s ti m a ti on , Sa n k

h y a

- A

, 4 5 2

- 4 6 1.

2 0

. H e , X

. a n d Q

. M

. S h a o ( 19 96 ) : B a h a d u r e ff i c i e n c y a n d r obu s t n e s s o f S t u d e n ti ze d s c o r e
t e s t s , A nn a l s o

f I n s t i t

. S t a t i s t

. M a t h

. , vo l

. 4 8

, no

. 2

, 29 5

- 31 4

. 2 1

. H e , X

. a n d P

. S h i ( 19 96 ) : B i v a r i a t e t e n s o r

- p r o d u c t B

- s p li n e s i n a p a r tl y li n e a r m o d e l

, J o u r n a l o f M u l t i v a r i a t e A na l y s i s , vol

. 5 8

, no

. 2

, 1 6 2

- 1 8 1.

2 2

. H e , X

. a n d Q

. M S h a o ( 1 99 6 ) : A g e n e r a l B a h a d u r r e p r e s e nt a t i o n o f M

- e s ti m a t o r s a n d i t s

a pp li c a t i o n t o li n e a r r e g r e ss i o n w it h no n s t o c h a s ti c d e s i gn s , A nn a l s o f S t a ti s t i c s , vo l

. 2 4

,
no

. 6, 2 60 8

- 263 0

. 2 3

. H e , X

. a n d G

. W a n g ( 19 97 ) : A qu a l i t a ti v e r obu s t n e ss o f S * e s ti m a t o r s o f m u lti v a r i a t e l o c a t i on a n d d i s p e r s i on , S t a ti s t i c a N e e r la n d i c a , vo l

. 5 1

, no

. 3

, 2 5 7

- 2 6 8.

2 4

. H e , X

. a n d L. S h e n ( 1 9 9 7 ) : L i n ea r r e g r e ss i o n a f t e r s p li n e t r a n s f o r m a ti on , B i o m e t r i k a , vol

. 84, no

. 2

, 47 4

- 48 1

. 2 5

. H e , X

. ( 1 9 9 7 ) : Q u a nt i l e c u r v e s w it hou t c r o ss i ng , T h e A m e r i c a n S t a ti s t i c ia n , vo l

. 5 1

, no

. 2
18 6

- 1 92.

2 6

. H e , X

. a n d G

. W a n g ( 1 99 7 ) : C onv e r g e n ce o f d e p t h c on t ou r s f o r m u lti v a r i a t e d a t a s e t s , A n n a l s o f St a t i s t i c s , vo l

. 2 5

, no

. 2

, 49 5

- 50 4

. 2 7

. H e , X

. , P

. N g a n d S

. P o r t no y ( 1 9 9 8 ) : B i v a r i a t e qu a nt i l e sm oo t h i n g s p li n e s , J ou r n a l o f th e
R o y al S t a t i s t i c a l S o c i e t y

, S e r i e s B

, vol

. 6 0

, P a r t 3

, 5 3 7

- 5 5 0.

2 8

. H e , X

. a n d S

. P o r t no y ( 1 99 8 ) : A s y m p t o ti c s o f t h e d e e p e s t l i n e , c h a pt e r 5 o f A p p li e d

S ta ti s t i c a l S c i e n c e III

, pp

. 7 1

- 81

, N ov a S c i e n c e P ubl i s h e r s , I n c

. 2 9

. C h e n , L. a n d H e , X

. ( 1 99 8 ) : T e s t o f no r m a lit y f o r ce n s o r e d d a t a , c h a p t e r 8 o f A p p l i e d

S t a t i s t i c a l S c i e n c e III

, pp

. 11 3

- 12 5

, N ov a S c i e n c e P ubl i s h e r s , I n c

. 3 0

. H e , X

. a n d P

. S h i ( 1 99 8 ) : M ono t on e B

- s p li n e sm oo t h i ng , J o u r n a l o f t h e A m e r

i c a n S t a t i s ti c a l Ass o c i a ti o n , V ol

. 9 3

, N o

. 4 4 2

, 64 3

- 65 0

. 3 1

. S i m p s on , D

. G

. , H e , X

. a n d Y

. T

. L i u ( 1 9 9 8 ) : C o mm e n t s o n “ M

- t yp e sm oo t h e r s w it h e d g e
p r e s e r v i n g p r op e r ti e s ”

, J o u r n a l o f t h e A m e r

i c a n St a t i s t i c a l As s o c i a t io n , V o l

. 9 3

, N o

. 4 4 2

, 5 4 4

- 5 47.
3 2

. H e , X

. a n d P

. N g ( 1 99 9 ) : Q u a n t i l e S pl i n e s w it h s e v e r a l c ov a r i a t e s , J o u r n a l o f S ta ti s t i c a l

P l a nn i n g an d I n

f e r e n c e , vol

. 75 ( 2 )

, 34 3

- 35 2

.Page 30

3 3

. H e , X

. a n d P

. N g : ( 1 99 9 ) : C O B S : C on s t r a i n e d S m oo t h i n g v i a L i n e a r Pr og r a m m i ng, C o m p u t a ti o na l S ta ti s t i c s , vol

. 1 4

, 3 15

- 3 3 7.
3 4

. H e , X

. ( 199 9 ) : C o mm e n t s o n “ r e g r e ss i o n d e p t h ”

, J ou r n a l o f t h e A m e r i c a n St a t

i s t i c al

Ass o c i a ti o n , vo l

. 9 4

, 4 0 3

- 4 04.
3 5

. H e , X

. a n d W

. K

. F un g ( 1 99 9 ) : M e t ho d o f M e d i a n s f o r li f e t i m e d a t a w it h W e ibu l l

m o d e l s , S t a t i s t i c s i n M e d i c i n e , vo l

. 1 8

, 19 93

- 2 0 09.
3 6

. B a i

, Z

. D

. a n d H e , X

. ( 1 9 9 9 ) : A s y m p t o ti c d i s t r i bu ti on s o f t h e m a x i m a l d e p t h e s ti m a t o r s f or r e g r e s s i o n a n d m u l t i v a r i a t e l o c a t i on

. A nn a l s o f S ta ti s t i c s , vo l

. 2 7

, 16 16

- 1 6 3 7

. 3 7

. G a l f a l vy , H

. C.

, H e , X

. , a n d D

. G

. S i m p s o n ( 1 9 99 ) : C o n d i ti o n a l qu a n t i l e c u r v e s w it h a n

a pp li c a t i o n t o i n fr a s t r u c t u r e s t u d i e s

. A m e r i c a n S ta ti s t i c a l As s o c i a t io n 1 99 9 P r o c e e d i n g s

o f t he S e c t i o n o n S ta ti s t i c a l C o n s u l ti n g , 37

- 4 1

. 3 8

. H e , X

. a n d H

. L i a n g ( 2 0 0 0 ) : Q u a nt i l e r e g r e ss i o n e s ti m a t e s f o r a c l a s s o f li n e a r a n d

p a r t i a ll y li n e a r e rr o r s

- i n

- v a r i a b l e s m o d e l s , S ta ti s t i c a Si n i c a , vo l

. 10

, 1 2 9

- 1 4 0.
3 9

. H e , X

. a n d W

. K

. F un g ( 2 00 0 ) : H i g h b r ea k d o w n e s ti m a ti o n f o r m u lti p l e popu l a t i on s w it h

a pp li c a t i on s t o d i s c r i m i n a n t a n a l y s i s , J o u r n a l o f M u l t i v a r

i a t e A n a l y s i s , vo l

. 72

, 1 51

- 1 6 2.

4 0

. H e , X

. a n d Q

. M

. S h a o ( 200 0 ) : O n p a r a m e t e r s o f i n c r ea s i n g d i m e n s i on , J o u r na l o f

M u l t i v a r i a t e A n a l y s i s , vol

. 7 3

, 1 20

- 1 3 5.

4 1

. H e , X

. ( 20 00 ) : C o mm e n t o n “ M ul ti v a r i a t e L

- e s ti m a ti on ”

, T e s t

, vo l

. 8

, 29 7

- 30 1

. 4 2

. H e , X

. , D

. G

. S i m p s on , a n d G

. Y

. W a n g ( 20 00 ) : B r e a k d o w n po i n t s o f t

- t yp e r e g r e ss i on

e s ti m a t o r s

. B i o m e t r i k a , V ol

. 8 7

, 6 75

- 6 8 7.

4 3

. H e , X

. a n d S

. P o r t no y ( 2 00 0 ) : S o m e a s y m p t o ti c r e s u lt s o n b i v a r i a t e qu a nt i l e s p li n e s , J ou r n a l o f S t a t i s t i c a l P l a nn i n g an d I n f e r e n c e , V o l

. 9 1

, N o

. 2

, 34 1

- 34 9

. 4 4

. H e , X

. a n d W

. K

. F un g ( 2 00 0 ) : D i s c u ss i o n o f ” D i m e n s i o n Re d u c ti o n a n d V i s u a l i z a t i o n i n
D i s c r i m i n a n t A n a l y s i s ”

, A u s t r a l i a n & Ne w Z

e a la n d J ou r

n a l o f S ta ti s t i c s , V ol

. 43

, no

. 2

, 1 9 0

- 1 93.

4 5

. P o r t noy , S

. a n d H e , X

. ( 2 00 0 ) : A r obu s t j ou r n e y i n t h e n e w m i l l e nn i u m , J o u r n a l o f t h e A m e r i c a n St a t i s t i c a l As s o c i a t i o n , V o l

. 9 5

, N o

. 45 2

, 1 3 3 1

- 1 33 5

. 4 6

. H e , X

. , Z

. W

. S h e n a n d L

. X

. S h e n ( 200 1 ) : A d a t a

- a d a pt i v e k no t s e l e c t i o n s c h e m e f o r f itti ng s p li n e s

. I EEE S ig n a l P r o c e ss in g L e t t e rs , V o l

. 8

, N o

. 5

, 1 3 7

- 1 39.

4 7

. M on r a d

, D

. , W

. F

. S t ou t

, B

. A

. B a i l e y , J

. R

. Fr yx e l l

, R. L

. G ou l d

, H e , X

. , V

. R

. P l e s s n e r , a n d L.

R ou s s o s ( 2 00 1 ) : S t a t i s ti c s , t h e C r a f t o f D a t a C ol l e c t i on , D e s c r i p ti on , a n d I n f e r e n c e ( S ec ond E d i t i on )

. T e x

t boo k fr o m M ob i u s C o m m un i ca t i on s

. 4 8

. H e , X

. a n d M

. K i m ( 2 0 0 2 ) : O n M a r g i n a l E s ti m a ti o n i n a S e m i p a r a m e t r i c M od e l f or L ong it u d i n a l D a t a w it h T i m e

- i n d e p e n d e n t C ov a r i a t e s

. M e t r i k a ( s p ec i a l i ss u e )

, V o l

. 5 5

, 6 7

- 7 4.

4 9

. H u a ng , B.

, B a c h m a nn , K

. , H e , X

. , C h e n , R

. , M c a l li s t e r , J

. a n d W a ng , T

. ( 20 02 ) : I n a pp r op r ia t e p r e s c r i p ti on s f o r t h e a g i n g popu l a ti o n o f t h e U n i t e d S t a t e s : a n a n a l y s i s o f

t h e N a t i on a l A m bu l a t o r y M e d i c a l C a r e S u r v e y , 19 9 7

, P h a r m a c o e p i d e m i o lo g y a n d D r u g S a f e t y , V ol

. 1 1, 1 2 7

- 1 34.

5 0

. H e , X

. , Z hu , Z

. Y

. a n d F ung , W

. K

. ( 2 0 0 2 ) : E s ti m a t i o n i n a S e m i p a r a m e t r i c M o d e l f o r L ong it u d i n a l D a t a w it h U n s p ec i f i e d D e p e nd e n c e S t r u c t u r e , B i o m e t r i k a , V ol

. 8 9

, N o

. 3

, 5 7 9

- 5 90.

5 1

. F ung , W

. K

. , H e , X

. , L i u , L. a n d S hi

, P

. D

. ( 2 0 0 2 ) : D i m e n s i o n r e d u c t i o n b a s e d o n

ca non i cal c o rr e l a ti on , S t a t i s t i c a S in i c a , V o l

. 1 2

, 10 9 3

- 1 1 1 4

.Page 31

5 2

. C u i

, H

. , H e , X

. a n d Z hu

, L

. ( 2 0 0 2 ) : A s y m p t o ti c s o f D e

- no i s e d L ea s t S qu a r e s E s ti m a t o r s , S t a t i s t i c a Si

n i c a , V ol

. 1 2

, 1 1 9 1

- 1 20 6

. 5 3

. H e , X

. a n d H u , F

. ( 2 00 2 ) : M a r k o v c h a i n m a r g i n a l boo t s t r a p , J . A m e r i c a n St a t i s t i c a l

Ass o c i a ti o n , V ol

. 9 7

, no

. 45 9

, 7 8 3

- 7 9 5.

5 4

. F ung , W

. K

. , Z hu

, Z

. Y

. , W e i

, B

. C

. a n d H e , X

. ( 2 0 0 2 ) : I n f l u e n c e D i a gno s ti c s a n d O u tl i e r T e s t s f o r S e m i p a r a m e t r i c Mi x e d M o d e l s , J

. R o y a l St a t i s t i c a l S o c i e t y , S e r i e s B

, V ol

. 64 ( 3 )

, 56 5

- 57 9

. 5 5

. H e , X

. , N g , W

. K

. a n d S h i

, J

. ( 2 0 0 3 ) : M a r g i n a l v e r s u s J o i n t B ox

- C o x T r a n s f o r m a ti o n w it h

A pp li c a ti on s t o P e r ce nt i l e C u r v e C on s t r u c ti o n f o r I g G S ub c l a s s e s a n d B l o o d P r e ss u r e s , S t a t i s t i c s i n M e d

i c i n e , V o l

. 2 2

, 39 7

- 40 8

. 5 6

. H e , X

. ( 2 00 3 ) : R obu s t T e s t s i n S t a ti s t i c s

– a Re v i e w o f t h e P a s t D e c a d e

. F o r a s p e c i a l i ss u e

o f Es t a t i s t i c a , V o l

. 5 4

, 29

- 4 6

. 5 7

. C h e n , M

. H

. , H e , X

. , S h a o , Q

. M

. , a n d X u , H

. ( 20 03 ) : A M ont e Ca r l o G a p T e s t i n C o m pu ti ng H P D Re g i on s , i n D e v e lo p m e n t o f M o d e r n S ta ti s t i c s a n d R e la t e d T op i c s

- I n

ce l e b r a t i o n of P r o

f e s s o r Y i a o

- T i n g Z han g ’ s 7 0t h B i r

t hd a y , S e r i e s i n B i o s t a ti s t i c s , V o l

. 1

, W o r l d S c i e n ti f i c , S i ng a po r e

. 5 8

. Z hu

, Z

. Y

. , H e , X

. a n d F ung , W

. K

. ( 200 3 ) : L o c a l i n f l u e n c e a n a l y s i s f o r p e n a l i z e d

G a u ss i a n l i k e li hoo d e s ti m a t o r s i n p a r t i a l l y li n e a r m o d e l s

. S c a n di n a v ia n J o u r n a l o f

S t a t i s ti c s , V o l

. 3 0

, 7 6 7

- 7 80.

5 9

. B a i

, Z

. D

. a n d H e , X

. ( 2 0 0 3 ) : A C h i

- s qu a r e t e s t f o r d i m e n s i on a l i t y w it h non

- G a u s s i a n
D a t a. J o u r

n a l o f M u l t i v a r

i a t e A n a l y s i s , V o l

. 8 8

, 1 0 9

- 1 1 7.
6 0

. H e , X

. , F u , B

. a n d F ung , K

. W

. ( 2 00 3 ) : M e d i a n R e g r e ss i o n o f L ong it u d i n a l D a t a

. S t a t i s ti c s in M e d i c i n e , V ol

. 2 2

, 3 6 5 5

- 3 66 9

. 6 1

. C u i

, H

. , H e , X

. a n d N g , K

. W

. ( 2 00 3 ) : A s y m p t o ti c D i s t r i but i on s o f P r i n c i p a l C o m pon e n t s Ba s e d o n R obu s t D i s p e r s i o n s

. B i o m e t r i k a , V o l

. 9 0

, 9 5 4

- 9 6 6.

6 2

. H e , X

. a n d Z hu , L. ( 20 03 ) : A L a c k

- o f

- f i t T e s t f o r Q u a n t i l e Re g r e ss i on , J o u r n a l o f t he A m e r i c a n St a t i s t i c a l As s o c i a t i o n , V o l

. 9 8

, 1 01 3

- 102 2

. 6 3

. H e , X

. , C u i

, H

. a n d S i m p s on , D

. G

. ( 2 0 0 4 ) : L on g it u d i n a l D a t a A n a l y s i s U s i n g t

- t yp e
R e g r e ss i on, J o u r

n a l o f S t a t i s ti c a l P l a nn i n g an d I n f e r e n c e , V o l

. 1 2 2

, 2 5 3

- 2 6 9.
6 4

. Z uo , Y

. , H

. C u i a n d H e , X

. ( 2 0 0 4 ) : O n t h e S t a h el

- D onoh o E s ti m a t o r a n d D e p t h

w e i gh t e d M e a n s o f M u l t i v a r i a t e D a t a , A nn a l s o f S t a ti s t i c s , V o l

. 3 2

, N o

. 1

, 16 7

- 18 8

. 6 5

. C u i

, H

. , H e , X

. a n d N g , K

. W

. ( 200 4 ) : M

- e s ti m a t i o n f o r li n ea r m o d e l s w it h s p a ti a l l y

c o rr e l a t e d e rr o r s , S t a t i

s t i c s an d P r ob a b i l

i t y L e t t e r s , V ol

. 6 6

, N o

. 4

, 3 8 3

- 3 93.
6 6

. L i n , N

. , B a i l e y , B

. , H e , X

. a n d B ut t l a r , W

. ( 20 04 ) : A d j u s t m e n t o f M e a s u r i n g D e v i ce s

w it h L i n ea r M o d e l s , T

e c hn o m e t r i c s , V o l

. 4 6

, N o

. 2

, 12 7

- 13 4

. 6 7

. H e , X

. a n d P o r t noy , S

. ( 2 0 0 4 ) : C o mm e n t o n L o ca ti on

- S c a l e D e p t h

, J o u r n a l o f t h e A m e r i c an S t a t i s t i c a l A ss o c ia t io n , V ol

. 9 9

, N o

. 4 6 8

, 97 3

- 97 6

. 6 8

. H e , X

. a n d N g , K

. W

. ( 2 00 4 ) : P e r c e n t i l e c h a r t s f o r c o rr e l a t e d m ea s u r e s

. I n E n cy c l o p e d i a s of B i o ph a r m a c e u t i c a l S ta ti s t i c s , e d i t e d b y S h e i n

- C hun g C ho w , D e k k e r

. c o m

. 6 9

. K o c h e r g i n s k y , M

. , H e , X

. a n d M u , Y

. ( 2 0 0 5 ) : P r a c t i c a l C on f i d e n c e I n t e r v al s f o r Re g r e ss i on Q u a n ti l e s , J o u r na l o f C o m p u t a t i on a l an d G r

a p h i c a l S t a t i s ti c

s , V ol

. 1 4

, no

. 1

, 4 1

- 5 5

. 7 0

. P e r l m a n , A

. L

. , H e , X

. , B a r k m e i e r , J

. , a n d V a n L ee r , E. ( 2 00 5 ) : B o l u s L o ca t i o n A ss o c i a t e d

w it h V i d e o f l uo r o s c op i c a n d Re s p i r od e g l u t o m e t r i c E v e n t s

. J o u r n a l o f S p e e c h

, L a n gua g e

,
a nd H e a r i n g R e s e a r c h

, V o l

. 4 8

, no

. 1

, 2 1

- 3 3

.Page 32

7 1

. H u a ng , B

. , M a r t i n , S

. J

. , B ac h m a nn , K

. A

. , H e , X

. , R ee s e , J

. H

. , W e i

, Y

. , I w u a g w u C.

, M a o , Z

.,
a n d X i n , H

. ( 2 00 5 )

. A n a t i on w i d e s u r v e y o f phy s i c i a n o ff i c e v i s it s f oun d t h a t
in a pp r op r i a t e a n ti b i o t i c p r e s c r i pt i on s w e r e i ss u e d f o r b a c t e r i a l r e s p i r a t o r y t r a c t i n f e c t i on s i n a m bu l a t o r y p a t i e n t s

. J o u r na l o f C l in i c a l E p i d e m i ol o g y , V o l

. 5 8

, 41 4

- 42 0

. 72

. H e , X

. ( 2005 ) : D i s c u ss io n o n “ B r ea kdo w n a n d G r o u p s ”

, A nnal s o f Stati s ti c s , V o l

. 33

, 9 98

- 1000.

7 3

. H e , X

. , F ung , W

. K

. , a n d Z hu

, Z

. Y

. ( 2 00 5 ) : R obu s t E s ti m a t i o n i n G e n e r a l i ze d P a r t i a l L i n ea r
M od e l s f o r C l u s t e r e d D a t a

. J o u r na l o f t h e A m e r

i c a n S ta ti s t i c a l As s o c i a t i o n , V ol

. 1 00

, 1 1 7 6

- 1 18 4

. 7 4

. W a ng , H

. X

. , H e , X

. , Ba n d

, M

. , W il s on , C

. a n d L i u , L. ( 2 0 0 5 ) : A s t u d y o f i nt e r

- l a b a n d i nt e r

p l a t f o r m a g r ee m e n t o f DN A m i c r o a rr a y d a t a

. B M C G e no m i c s , V ol

. 6

, A r t

. N o

. 71.

7 5

. B o e n t e , G

. , H e , X

. a n d Z hou

, J. ( 200 6 ) : R obu s t E s ti m a t e s i n G e n e r a l i z e d P a r ti a ll y L i n e ar
M od e l s

. A nn a l s o

f S ta ti s t i c s , V o l

. 3 4

, 285 6

- 28 7 8.

7 6

. W e i

, Y

. , P e r e , A

. , K o e n k e r , R

. a n d H e , X

. ( 20 06 ) : Q u a nt i l e Re g r e ss i o n M e t h o d s f o r Re f e r e n ce G r o w

t h C h a r t s

. S t a t i s t i c s i n M e d i c i n e , V o l

. 2 5

, no

. 8

, 13 6 9

- 1 3 8 2

. 7 7

. W e i

, Y

. a n d H e , X

. ( 200 6 ) : C ond i t i on a l G r o w t h C h a r t s ( w it h d i s c u s s i o n s )

. A n n a l s o f

St a t i s t i c s , V o l

. 2 5

, no

. 8

, 136 9

- 13 8 2.

7 8

. L i n , N

. a n d H e , X

. ( 200 6 ) : R obu s t a n d e ff i c i e n t e s ti m a t i o n un d e r d a t a g r oup i ng

. B i o m e t r i k a , V o l

. 9 3

, no

. 1

, 9 9

- 1 12.

7 9

. Z uo , Y

. a n d H e , X

. ( 20 06 ) : O n t h e li m i ti n g d i s t r i bu ti on s o f m u lti v a r i a t e d e p t h

- b a s e d r a n k

s um s t a ti s t i c s a n d r e l a t e d t e s t s

. A nn a l s o f S ta ti s t i c s , V o l

. 3 4

, 287 9

- 28 9 6.
8 0

. L i n , G

. , H e , X

. , H a nl ee , J

. , S hi , L

. , D a v i s , R

. W

. , a n d Z hong

, S

. ( 20 06 ) : R e p r o d u c i b i l it y P r ob a b i l it y S c o r e : I n c o r p o r a ti n g M e a s u r e m e n t V a r i a b i l it y a c r o ss L a bo r a t o r i e s f o r G e ne S e l e c t i on

. N at u re B io t e c hn o lo g y , V o l

. 2 4 ( 12 )

, 1 47 6

- 147 7 ( w it h on li n e s upp l e m e n t a r y

m a t e r i a l )

8 1

. H u , J a n d H e , X

. ( 2 0 0 7 ) : E nh a n c e d qu a n t i l e no r m a l i z a ti o n o f m i c r o a rr a y d a t a t o r e d u c e l o ss of i n f o r m a ti o n i n t h e g e n e e xp r e ss i o n p r o f il e

. B i o m e t r i c s , V o l

. 63

, N o

. 1

, 5 0

- 5 9.
8 2

. M u , Y

. a n d H e , X

. ( 20 07 ) : P o w e r T r a n s f o r m a ti o n t o w a r d s a L i n ea r R e g r e ss i o n Q u a n til e

. J o u r n a l o f A m e r i c a n S t a t i

s t i c i a n As s o c i a t io n , V o l

. 10 2

, 26 9

- 27 9

. 8 3

. W a ng , H

. X

. a n d H e , X

. ( 2 0 0 7 ) : D e t e c ti n g D i ff e r e n t i a l E xp r e ss i o n s i n G e n eC h i p
M i c r o a rr ay S t u d i e s : A Q u a n t i l e A pp r o ac h

. J o u r n a l o f t h e A m e r i c a n S t a ti s t i c a l A ss o c ia t i o n , V o l

. 10 2

, 1 04

- 1 12.
8 4

. K o c h e r g i n s k y , M

. a n d H e , X

. ( 200 7 )

. E x t e n s i on s o f t h e M a r k o v C h a i n M a r g i n a l

B oo t s t r a p. S t a t i s t i c s a n d P r o ba b i l i t y L e t t e rs , V o l

. 7 7

, N o

. 1 2

, 1 2 5 8

- 1 26 8

. 8 5

. C u i

, H

. , H e , X

. a n d L i u

. L. ( 20 07 )

. T e s ti n g f o r A d d it i v it y w it h B

- S pl i n e s

. S c i e n c e i n C hi n a S e r i e s A: M a t h e m a t i c s , V ol

. 5 0

, N o

. 6

, 8 4 1

- 8 58.
8 6

. H e , X

. a n d H u , F

. ( 2 00 7 )

. C o mm e n t s o n ‘ I m p l e m e n t a ti o n o f E s ti m a ti n g

- F un c ti o n B a s e d

I n f e r e n c e P r o ce d u r e s w it h M C M C S a m p l e r s ’ b y T i a n , L i u a n d W e i

. J o u r na l o f t h e A m e r i c an S t a t i s t i c a l A ss o c ia t io n , V ol

. 1 0 2

, no

. 4 7 9

, 8 8 9

- 8 9 0.
8 7

. H u , J

. , H e , X

. , B a gg e r l y , K

. A

. , C oo m b e s , K

. R.

, H e nn e ss y , B.

, a n d M i ll s , G

. B

. ( 2 00 7 )

. N onp a r a m e t r i c Q u a n ti f i c a t i o n o f P r ot e i n L y s a t e A rr a y s

. B i oi n f o r m a t i c s , V ol

. 2 3

, N o

. 1 5

, 19 86

- 1 99 4

. 8 8

. Z hou , T. a n d H e , X

. ( 200 8 )

. T h r e e

- s t e p E s ti m a ti o n i n L i n ea r Mi x e d M o d e l s w it h S k e w

- t
D i s t r i bu ti on s

. J o u r

n a l o f S ta ti s t i c a l P la n ni

n g an d I n f e r e n c e , V o l

. 13 8 ( 6 )

, 1 5 4 2

- 1 5 5 5.
8 9

. H e n s on , R.

, R ou ss o s , L

. , D oug l a s , J

. a n d H e , X

. ( 200 8 ) : C ogn i t i v e d i a gno s ti c a t t r i bu t e l e v e l d i s c r i m i n a ti o n i n d i ce s

. A p p l i e d P s y c ho l og i c a l M e a s u r

e m e n t

, V ol

. 3 2

, N o

. 4

, 2 7 5

- 2 8 8.Page 33

9 0

. W a ng , H

. a n d H e , X

. ( 2 0 0 8 )

. A n E nh a n c e d Q u a nt i l e A pp r o ac h f o r A ss e s s i n g D i ff e r e n ti a l

G e ne E xp r e s s i o n s

. B i o m e t r i c s , V o l

. 6 4 ( 2 )

, 4 4 9

- 4 5 7.
9 1

. C h e n , C

. , H e , X

. , a n d W e i

, Y

. ( 20 08 )

. L o w e r R a n k A pp r ox i m a ti o n o f M a t r i c e s B a s e d o n
F a s t a n d R obu s t A l t e r n a t i n g Re g r e ss i on

. J o u r n a l o f C o m p ut a t i on a l a n d G r

a p h i c a l

St a t i s t i c s , V o l

. 1 7

, no

. 1

, 1 8 6

- 2 0 0.
9 2

. Z hou

, J. a n d H e , X

. ( 2 00 8 )

. D i m e n s i o n R e d u c ti o n B a s e d o n C on s t r a i n e d Ca non i ca l

C o rr e l a t i on a n d V a r i a b l e F i l t e r i ng

. A n n a l s o f S ta ti s t i c s , V o l

. 3 6

, no

. 4

, 1 6 4 9

- 1 66 8

. 9 3

. N o e , D

. a n d H e , X

. ( 2 0 0 8 )

. P a r t i a ll y Ba y e s i a n V a r i a bl e S e l e c t i o n i n C l a ss i f i ca ti o n T r ee s

. S t a t i s t i c s a n d i t s I

n t e r f a c e , V o l

. 1

, 1 5 5

- 1 6 7.
9 4

. T ho r p e M P

, J ac ob s e n E H

, L a y m a n DK

, H e X

, K r i s

- E t h e r t o n P M

, E v a n s E M ( 2 0 0 8 )

. A d i e t h i g h i n p r o t e i n , d a i r y a n d ca l c i u m a t t e nu a t e s bon e l o s s ov e r 4 m on t h s w e i gh t l o ss a n d

8 m o w e i gh t m a i n t e n a n c e r e l a ti v e t o a c onv e nt i on a l h i g h ca r bohy d r a t e d i e t i n a d ul t s

. J o u r na l o f N u t r i t i o n , V o l

. 13 8

, 10 9 6

- 1 1 0 0

. 9 5

. Z hu , Z

. Y

, F ung , W

. K

. , a n d H e , X

. ( 200 8 ) . A no t e o n t h e l o c a l a s y m p t o ti c s o f m a r g i n a l

r e g r e s s i o n s p li n e s w it h l ong it u d i n a l d a t a

. B i o m e t r i k a , V o l

. 94

, no

. 4

, 90 7

- 91 7

. 9 6

. H u , F

. , Z h a ng , L

. X

. , a n d H e , X

. ( 200 9 )

. E f f i c i e n t R a n d o m i z e d A d a p ti v e D e s i gn s

. A n n a l s of

S t a t i s t i c s , V o l

. 3 7

, N o

. 5 A

, 2 5 4 3

- 2 56 0

. 9 7

. B r og li o , S

. P

. , S o s no ff

, J

. , S h i n , S

. , H e , X

. , A l ca r a z , C.

, a n d Z i mm e r m a n , J. ( 2 0 09 )

. H e l m e t
i m p ac t s d u r i n g h i g h s c hoo l f oo t b a ll : A b i o m ec h a ni c a l a ss e ss m e n t

. J ou r n a l o f A th l e ti c
T r ai n in g , V o l

. 44 ( 4 )

, 34 2

- 34 9

. 9 8

. H u , Y

. , H e , X

. , T i a o , J

. , a n d N

. Z. S h i ( 200 9 )

. M o d e l i n g a n d p r e d i c ti o n o f c h i ld r e n ’ s
g r o w t h d a t a v i a f un c ti on a l p r i n c i p a l c o m pon e n t a n a l y s i s

. S c i e n c e i n C hi n a , S e r i e s A

, V o l

. 5 2

, N o

. 6, 1 34 2

- 135 0

. 9 9

. Y a ng , S

. K

. , L ong , C

. , M i l l e r , A

. , H e , X

. , Y a ng , Y

. , W u e bb l e s , D

. , a n d T i a o , G

. ( 2 00 9 )

. T h e

m o d u l a ti o n o f n a t u r a l v a r i a b i l it y o n t h e t r e n d a n a l y s i s o f t h e up d a t e d c oh e s i v e S B UV ( 2 )

t o t a l o z on e

. I n t e r

n a ti o na l J o u r

n a l o f R e m o t e S e n s in g , V ol

. 3 0

, 39 7 5

- 3 9 8 6

. 1 00

. F e ng , X

. a n d H e , X

. ( 2 0 0 9 )

. I n f e r e n c e o n l o w

- r a n k d a t a m a t r i ce s w it h a ppl i ca t i on s t o

m i c r o a rr a y d a t a

. A nn a l s o f A pp l i e d St a t i s t i c s , V ol

. 3

, N o

. 4

, 1 6 3 4

- 1 65 4

. 1 01

. S h i

, N

. Z

. , H e , X

. , a n d T a o , J

. ( 20 09 )

. U n d e r s t a n d i n g S t a ti s t i c s a n d S t a t i s ti c s E du ca t i on : A C h i n e s e P e r s p ec ti v e

. J o u r na l o f S ta ti s t i c s E

d u c a t i o n , V o l u m e 1 7

, N u m b e r 3

,
w w w

. a m s t a t

. o r g / pub li c a ti on s / j s e / v 17 n 3 / s h i

. h t m l

. 1 02

. H u , J, H e , X

. , C o t e , G

. J

. , a n d K r a f e , R

. ( 20 09 )

. S i ngu l a r V a l u e D ec o m p o s iti on

- b a s e d

A lt e r n a ti v e S pl i c i n g D e t e c ti on

. J o u r n a l o f th e A m e r i c a n S ta ti s t i c a l As s o c ia t io n , V o l

. 1 0 4

, N o. 4 87

, 9 4 4

- 9 5 3.
103

. H ong , F

. , F a ng , F

, H e , X

. , Ca o , X

. , C h i pp e rf i e ld

, H

. , X i e , D

. , W ong , W

. H

. , N g , H

. H

. , a n d

Z hong

, S

. ( 2 0 0 9 )

. D i ss ec ti n g e a r l y d i ff e r e nt i a ll y e xp r e ss e d g e n e s i n a m i x t u r e o f
di ff e r e n t i a t i ng e m b r

yo n i c s t e m ce l l s

. P

L o S C o m putationa l B io l

og y , V o l

. 5 ( 12 )

, d o i:10

. 1371 /

j o u r n a l

. p c b i

. 1000607.

1 04

. H ong , H

. G

. , a n d H e , X

. ( 2 0 1 0 )

. P r e d i c ti o n o f F un c t i on a l S t a t u s f o r t h e E l d e r l y Ba s e d o n

a N e w O r d i n a l R e g r e ss i o n M o d e l

. J o u r n a l o f t h e A m e r i c a n S ta ti s t i c a l As s o c i a t i o n , V o l

. 10 5

, N o. 4 91

, 9 3 0

- 9 4 1

. 1 05

. H e , X

. , X u e , H

. , a n d S hi

, N

. Z

. ( 201 0 )

. S i e v e M a x i m u m L i k e li h oo d E s t i m a ti o n f o r D oub l y S e m i p a r a m e t r i c Z e r o

- I n f l a t e d P o i ss o n M o d e l s

. J o u r n a l o f M u l t i v a r ia t e A na l y s i s , V o l u m e 1 01, I s s u e 9

, 2 02 6

- 203 8

.Page 34

1 06

. H e , X

. a n d Z hou

, J. ( 20 10 )

. D i s c u s s i o n o f “ E nv e l op e m o d e l s f o r p a r s i m on i o u s a n d

e ff i c i e n t m u lti v a r i a t e l i n ea r r e g r e ss i on ”

, S ta ti s t i c a S i n i c a , V ol

. 2 0

, N o

. 3

, 9 7 1

- 9 7 8.

1 07

. B r og li o , S

. P

. , S c hn e b e l

, B

. , S o s no ff

, J

. J

. , S h i n , S

. , F e ng , X

. , H e , X

. , Z i mm e r m a n , J. ( 201 0 )

. T h e b i o m ec h a n i c a l p r op e r ti e s o f c on c u ss i o n s i n h i g h s c hoo l f oo t b a l l

. M e d i c i n e &

S c i e n c e i n S p o r

t s & E xe r c i s e , V o l

. 4 2

, 20 64

- 1 0 7 1

. 1 08

. H e , X

. , H s u , Y

. H

. , a n d H u , M

. ( 2 01 0 )

. D e t e c ti o n o f T r e a t m e n t E f fe c t s b y C ov a r i a t e

a d j u s t e d E xp ec t e d S ho r t f a l l

. A n na l s o f A p p l i e d S t a t i s t i c s , V o l

. 4

, N o

. 4

, 21 1 4

- 2 1 25

. 1 09

. L i

, C

. , W e i

, Y

. , C h a pp e l l

, R

. J. a n d X

. H e ( 2 01 1 )

. Be n t L i n e Q u a n t i l e R e g r e s s i o n w it h

a pp li c a t i o n t o a n a ll o m e t r i c s t u d y o f l a n d m a mm a l s ’ s p ee d a n d m a s s

. B i o m e t r i c s , V o l

. 6 7

, 24 2

- 2 49.

110. Wang, X., Gui, J., and He, X. (2011). Finding the minimal set for collapsible graphical
models. Proceedings of the American Mathematical Society, Vol.139, 361

- 373.

1 11

. G uo , J

. , X u , P

. F

. , a n d H e , X

. ( 20 1 1 )

. A n I m p r o v e d I t e r a ti v e P r opo r ti on a l S c a l i n g
P r o c e d u r e f o r G a u s s i a n G r a ph i c a l M od e l s

. J o u r n a l o f C o m p ut a t i on a l an d G r

a p h i c a l

S t a t i s t i c s , V ol

. 20

, 417

- 431

.

S O F T W AR E:

• H e , X

. a n d N g

, P

. ( 2 0 0 1 ) : C on s t r a i n e d B

- s p li n e S m oo t h i n g ( C O B S)

, F o r t r a n f un c t i on s w it h S

P l u s i n t e rf a c e , a n d i n C R A N f o r R u s e r s

.

• K o c h e r g i n s k y , M

. a n d H e , X

. ( 2 0 0 3 ) : M C M B C on f i d e n c e I n t e r v a l s f o r Re g r e s s i o n Q u a n til e s ,
c u rr e n tl y a v a i l a bl e a s a c on t r i but e d p a c k a g e r q m c m b 2 i n C R A N f o r R u s e r s

. T h e s a m e

m e t ho d i s i n c l u d e d i n S A S P R O C QUAN T R EG und e r t h e m c m b opt i on.Page 35

Christian E . Lindhjem

EDUCATION
1987 PhD, Chemical Engineering, Rensselaer Polytechnic Institute

1981 MS, Chemical Engineering, University of Michigan

1980 BS, Chemical Engineering, Rose

- Hulman Institute of Technology

1980 BS, Chemistry, Rose

- Hulman Institute of Te chnology

EXPERIENCE Dr. Christian E. Lindhjem, a Senior Consultant at ENVIRON, is an expert on emissions from highway and nonroad vehicles and engines and fuels used in those engines. With ENVIRON for 12 years and with EPA’s Office of Transportation and A ir Quality (previously named Office of Mobile Sources) for 8 years, Dr. Lindhjem has worked on on

- road and off

- road mobile source regulation development, emission measurements and analysis, emission control strategies, and emission inventory modeling inclu ding regulated pollutants and chemical compositional analysis to estimate toxic emissions and other components of concern. Dr. Lindhjem has evaluated and continues to consult on a broad range local and national mobile source emission issues including both emission estimates and potential emissions reductions from a variety of planned or demonstrated control strategies for on

- road vehicles and off

- road equipment (such as construction equipment, locomotive, marine vessels, and other harbor sources) and heavy

- duty diesel vehicles. He works with local, regional and national officials and private clients to improve on

- road and off

- road (including agricultural, commercial marine, locomotive, construction and mining, recreational marine, and lawn and garden equipme nt types) emission inventories for regional evaluation as well as individual facilities including ports and rail yards. His other national experience included assistance for EPA in the development of the NONROAD model and a national commercial marine

emiss ion inventory and spatial allocation for deep draft vessels. Dr. Lindhjem holds a Ph.D. degree in Chemical Engineering from the Rensselaer Polytechnic Institute and a M.S. degree in Chemical Engineering from the University of Michigan.

 Commercial Marine, Rail, and Intermodal Emissions Evaluations : Conducted in depth analysis and method formulation to revise locomotive and commercial marine emissions for ports and railroads, states, and national agencies including those for the States of Arkansas, Illinois, Indiana, Michigan, Ohio, Wisconsin, and Wyoming and the metropolitan areas of Kansas City and Boise. Prepared a grid scale national emissions inventory from Category 3 (ocean

- going vessels) commercial marine engines for EPA combining near port and open

- oc ean emissions. Other projects include emissions and control strategy evaluations for the Ports of Long Beach, San Francisco, Oakland, San Diego and others, including the emission reduction potential from shore power and other control strategies. Evaluated averaging strategies to comply with the Canadian and U.S. Emission Control Area (ECA) designation for ocean

- going vessels using highly resolved activity and emissions calculations. Conducted detailed rail yard activity and emissions evaluations
in Californ ia and Kansas including locomotive and other off

- road and on

- road source categories within the yards.

 On

- Road Emission Inventories : Managed projects to estimate metropolitan area and project level emissions estimates using the EPA MOVES2010 and California ARB EMFAC models. Managing projects to incorporate CONCEPT link level modeling for Detroit, Atlanta, and Philadelphia including MOVES2010 model adjustments and vehicle volume and mix of heavy and light

- duty vehicles adjusted by time of data and road type. Performed evaluations of the on

- road vehicle mix and vehicle weight data from Automatic Traffic Recorders to better estimate on

- road emissions for EPA, Illinois, Ohio, Wisconsin, and Minnesota. Led programs to develop new strategies and uses forPage 36

alternativ e data to evaluate on

- road vehicle emissions for EPA and the Coordinating Research Council in preparation of the MOVES model.

 On and Off

- Road Mobile Source Control Strategies : Provides detailed technical analyses and planning and day to day support of on

- r oad and off

- road mobile source control strategies for local and state government agencies including the Houston

- Galveston, Dallas

- Ft. Worth, and East Texas nonattainment areas’ SIPs, for the Houston

- Galveston Area Council (HGAC), the East Texas Council
of Governments (ETCOG), North Central Council of Governments (NCTCOG), and the Texas Commission on Environmental Quality (TCEQ). Also assisting in revisions/updates to the mobile source emission inventory in Texas and Las Vegas. Providing the plan and techn ical guidance for the HGAC Voluntary Mobile Source Emission Reduction Program (VMEP) including marketing and online database and interactive programs, and reviews and evaluation of the Texas Emission Reduction PLAN (TERP) for the Houston Advanced Research Center (HARC).

 Off

- Road Emission Inventories : Provided service to state agencies by improving air emissions
inventories from all nonroad mobile sources. Conducted in

- depth locomotive and other emissions evaluations for BNSF railyards in California in prep aration for dispersion modeling impact analysis on the local community. For the Lake Michigan Air Directors Consortium (LADCO), a plan was developed and implemented for revising and developing activity estimates for all off

- road emission sources, incorpora te such estimates in the emissions inventory, and spatially allocate those emissions. Innovative methods were used for commercial marine to quantify the Lake and river commercial marine emissions, locomotive emissions from large and small railroads, and a unique method for recreational marine spatial allocation specifically. Prepared revised emissions inventories some categories or all nonroad (including aircraft, commercial marine, and locomotive sources) for the States of Arizona, Texas, Arkansas, Oklaho ma, Wyoming, Wisconsin, Illinois, Indiana, Michigan, Ohio, New Jersey, and the Western Regional Air Partnership (WRAP). Assisted EPA in improvements to the NONROAD model for off

- road emissions estimation including for instance detailed technical comparison s of EPA and CARB models for estimating emissions from nonroad mobile sources and providing revisions to modeling structure, activity data inputs and assumptions, and emission factors for nonroad spark

- ignition and compression ignition engines.
Another exa mple of activity includes providing technical guidance to Northeast States for Coordinated Air Use Management (NESCAUM) for the development of surveys and analysis of survey data to derive improved estimates of construction equipment activity in the Northe ast U.S.

 Scientific Emissions Research and Analysis : Performed scientific evaluation of emission modeling methods such as critically reviewing the project plan and available data for EPA’s new MOVES model in an extensive report (E

- 68) for the Coordinating Research Council (CRC). This report outlined the general method, specifics of data handling, and many other areas of interest in implementing this new modeling approach for on

- road mobile sources. The CRC project followed the previous work for EPA, which d eveloped and executed a method using on

- board emissions monitor data to estimate emissions from onroad and offroad vehicles and equipment for the Office of Transportation and Air Quality (OTAQ) of EPA. Driving behavior and emissions were related to the phy sical and operating parameters experienced of the vehicle or engine. Scientific evaluation of the effects of humidity and temperature on mobile source emissions was conducted and incorporated into Texas emissions for HARC and TCEQ. A scientific assessment of the emissions response from hybrid

- electric urban buses for the New York State Energy Research and Development Agency (NYSERDA).

 Chemical Constituents of Mobile Source Emissions : Performed a literature review and evaluation
of the chemical constituents including potential toxic components of mobile source emissions for the EPA. Estimated the emission reduction potential and impacts on air quality and toxicity from replacing diesel with biodiesel fuel in heavy

- duty diesel vehicles for the Department of En ergy’s National Renewable Energy Laboratory.Page 37

 Mobile Source Emissions Air Quality Impact Evaluation : Providing limited and innovative plans to evaluate and mitigate mobile source emissions at a project level. For instance, the City of Hawthorne, California , provided an evaluation of the impact and provided an innovated contracting method to mitigate air quality effects for garbage collection vehicles. Provided technical support for estimating on

- road emissions and air quality impacts for the expansion of an amphitheater and casino including preparing a report for the Environmental Impact Statement and an assessment of General Conformity.

Prior to joining ENVIRON, Dr. Lindhjem held the following positions:

Engineer, U.S. Environmental Protection Agency (EPA ), National Vehicle and Fuel Emissions Laboratory, Office of Mobile Sources, Ann Arbor, MI (1990

- 1998), 8 years

Nonroad Mobile Emissions

 Evaluation of rail and port intermodal yard emissions including emission activity from ships, locomotive, cargo hand ling equipment, and on

- site truck movements.

 Responsible for the theoretical framework behind EPA’s NONROAD model, the next inventory modeling tool for nonroad engines, analogous to EPA’s MOBILE model for highway vehicles, including nonroad industrial, con struction, commercial, residential, and marine engines and
vehicles.

 Developed the input estimates for the EPA’s NONROAD model for population, activity, load factor, average life, and compression and spark

- ignition emission factors for nonroad engines as w ell as estimates for the hydrocarbon speciation.

 Project engineer on a variety of test programs measuring emissions from nonroad engines, specifically construction and agricultural diesel engines, outboard and inboard recreational marine motors, and lawn & garden engines.

 Evaluated of the effect of test cycle on various diesel and gasoline nonroad engines through regulatory and nonregulatory steady

- state and transient comparisons. Developed a methodology to estimate emissions from commercial marine engines.

Highway Mobile Emissions Model

 Incorporated MOVES into emission inventory development tools such as SMOKE, CONCEPT, and project level emissions development.

 Have provided emission estimates for light

- duty and heavy

- duty vehicles exhaust and evaporative

e missions, and fuel effects for MOBILE4.1, MOBILE5, and MOBILE6.

 Specific areas of emissions modeling were estimating the effects of reformulated gasoline, evaporative emissions, and diesel engines emissions for heavy

- duty highway trucks and buses.

 Other mo deling efforts include analyzing chemical characterizations of exhaust and evaporative emissions to provide estimates currently used in EPA’s Speciate Database.

 Managed up to $1.4 million in contract funding per year.

 Research included lean NOx

- reduction catalyst evaluations, test cycle comparisons for highway and nonroad engines, and novel emission modeling techniques for diesel engines.

Engine Testing Project Management

 Managed the heavy

- duty, marine, evaporative, and light

- duty chassis testing programs with five technicians, two engineers, and a budget of about $500k per year.

 Multidisciplinary approach combining mechanical, electrical, computer, and chemical analysis.

 Supplied test results for a variety of EPA projects using a variety of engines and ve hicles; most notably, nonroad and highway diesel, inboard and outboard marine, and lawn and garden engines.Page 38

Clean Fuels Development

 Developed the Reformulated Gasoline Simple Model; evaluated the effect of fuel oxygen and volatility control on primary and toxic pollutants from gasoline vehicles.

 Team member for the Reformulated Gasoline Complex Model and Final Rulemaking providing estimates for effects of fuel parameter changes on primary and toxic pollutants.

 Conducted testing programs investigating the ef fects of reformulated fuels on vehicles and nonroad engines.

 Facilitated fuel additive emission testing and registration.

Research Engineer, Westvaco Corporation, (1988

- 1990), Laurel Research Center, Laurel, MD

 Designed coating formulations and products for the fine papers division.

 Successfully quantified subjective criteria of product quality and determined the mechanisms of in

use deterioration of our products during printing.

 Determined rheological failures in the production of coated papers and de veloped novel coating formulations for improved production.

 Managed 2 technicians and interactions with research and production.

EPA AWARDS

 EPA Bronze Medal for Commendable Service for Nonroad Engine Emission Controls Development,

1995

 EPA Bronze Medal fo r Commendable Service for Highway Heavy

- Duty Engine Emission Reduction,

1995

 EPA Science Achievement Award in Air Quality, 1992

 EPA Silver Medal for Superior Service for Clean Fuels Development, 1991

TEACHING EXPERIENCE

University of Michigan, (1991), “P ollution Control for Chemical Engineers,” Department of Chemical Engineering Rensselaer Polytechnic Institute, (1985

- 1987), “Air Pollution Control,” Department of Chemical Engineering

PROFESSIONAL MEMBERS HIPS

Air and Waste Management Association

Nation al Cooperative Highway Research Program Panelist EPA’s Mobile Source Technical Review Subcommittee for Nonroad Engine Emissions

PUBLICATIONS AND PRE SENTATIONS

Lindhjem C.E. 2010 "Use of MOVES2010 in Link Level On

- Road Vehicle Emissions Modeling Using

CON CEPT

- MV," C. E. Lindhjem, A. DenBleyker, M. Jimenez, J. Haasbeek, A. K. Pollack, ENVIRON International Corporation, CA; Z. Li, Clark County, Las Vegas, NV. 19th International Emission Inventory Conference, San Antonio, Texas, September 27

- 30, 2010.
Lin dhjem C.E. 2010 "Development of Drivers and Post

- Processing Scripts to Incorporate MOVES2010 Emission Factors with the Smoke Emissions Model,"C. E. Lindhjem, A. DenBleyker, M. Jimenez and A. K. Pollack, ENVIRON International Corporation. 19th Internationa l Emission Inventory Conference, San Antonio, Texas, September 27

- 30, 2010. Lindhjem, C. 2009. “Mobile Source Particulate And Semi

- Volatile Organic Carbon Ambient Modeling.” Presented at the 18th International Emissions Inventory Conference, Baltimore, MD, April 15, 2009.Page 39

Lindhjem, C. 2009 “Mobile Source Emissions: Adjustments to MOBILE6,” 19th CRC On

- Road Vehicle Emissions Workshop, March 23

- 25, 2009. Lindhjem, C. 2008. “Intermodal Yard Activity and Emissions Evaluations.” Presented at the 17th

Interna tional Emissions Inventory Conference, Portland, OR. June. Lindhjem, C.E. and Russell, J., 2006. “Development Of Gridded Ocean

- Going Vessel Emission Inventories,” Presented at the Air and Waste Management Association Emission Inventory Conference, New
Orl eans, LA, May 17. Lindhjem, C.E. and Shepard S. 2005. “Estimation and Effects of Vehicle Mix on On

- Road Emission Estimates,” Air and Waste Management Association Emission Inventory Conference, Las Vegas, Nevada, April 14. Lindhjem, C.E. and Chan L

- M. 2004. “Emission Control Technologies and Programs for Heavy

- Duty Diesel Vehicle Fleets in North America,” Paper No. 371. Air and Waste Management Association Annual Meeting, Indianapolis IN. Lindhjem, C.E., Chan, L

- M., Pollack, A.K., and Kite C. 2004. “Applying Humidity and Temperature
Corrections to On and Off

- Road Mobile Source Emissions,” Air & Waste Management Association Emission Inventory Conference, St. Petersburg Florida. Lindhjem, C.E., T. Stockenius. 2002. “On

- Board Emissions Data Analysis,” Present ed at the12th CRC On

Road Vehicle Emissions Workshop, San Diego, California, April. Lindhjem, C.E., A.K. Pollack, R. Chi. 2001. “Comparison Of Highway Mobile Source Emissions Inventory From MOBILE1 through MOBILE6,” 11th CRC On

- Road Vehicle Emissions Work shop, San Diego, California, March. Lindhjem, C.E., and D.A. Guerrieri. 1998. “Evaluation of Lean NOx Reduction Catalysts for Controlling Emissions from Diesel Engines”, Environmental Progress , Spring, 1998, page 48. Lindhjem, C.E., D.M. Swain, C.J. Jack son, and G.J. Hoffman. 1998. “A Method for Comparing Transient NOx Emissions with Weighted Steady State Test Results,” Society of Automotive Engineers, SAE

- 980408. Lindhjem, C.E., D.J. Korotney, V. Rao, and M.S. Sklar. 1995. “Reformulated Gasoline Eff ects on Exhaust Emissions: Phase III: Investigation on the Effects of the Oxygenate ETBE, Sulfur, Olefins, Volatility, and Aromatics and the Interactions Between Olefins and Volatility or Sulfur,” Society of Automotive Engineers, SAE

- 950782. Lindhjem, C.E. 1995. “The Effect of Gasoline Reformulation and Sulfur Reduction on Exhaust Emissions
from Post

- 1983 and Pre

- 1990 Vehicles,” Society of Automotive Engineers, SAE

- 950778. Lindhjem, C.E., S.C. Mayotte, V. Rao, and M.S. Sklar. 1994. “Reformulated Gasolin e Effects on Exhaust Emissions: Phase II: Continued Investigation of the Effects of Fuel Oxygenate Content, Volatility, Sulfur, Olefins, and Distillation Parameters,” Society of Automotive Engineers, SAE

- 941974. Lindhjem, C.E., S.C. Mayotte, V. Rao, and M. S. Sklar. 1994. “Reformulated Gasoline Effects on Exhaust Emissions: Phase I: Initial Investigation of Oxygenate, Volatility, Distillation and Sulfur Effects,” society of Automotive Engineers, SAE

- 941973. Lindhjem, C.E., D.J. Korotney. 1993. “Running L oss Emissions from Gasoline

- Fueled Motor Vehicles,” Society of Automotive Engineers, SAE

- 931991. Lindhjem, C.E. 1991. “The Particle Packing and Shape Effects on the Rheological Characteristics of Paper Coating Pigments,” Proceedings of the 1991 TAPPI Coa ting Conference , Montreal, Quebec, Canada, TAPPI Press, Atlanta, Georgia. Lindhjem, C.E., E.R. Altwicker. 1988. “Absorption of Gases by Drops,” AICHE Journal , v. 34. pp. 329

- 332.

SELECTED REPORTS Lindhjem, C.E. and Sturtz, T.M., “Development of Emission Estimates for Locomotives in the Kansas City Metropolitan Statistical Area,” Prepared for Kansas Department of Health and Environment (KDHE), May 13, 2010.Page 40

Parker, L., Lindhjem, C.E. and others, “Implement Port of Houston’s Current Inventory and Harmoniz e the Remaining 8

- county Shipping Inventory for TCEQ Modeling” Prepared for: Texas Commission on Environmental Quality, August 18, 2010. Lindhjem, C.E. and Sturtz, T.M., “Development of Link

- Level Spatial Allocation Methodology for Line

- Haul

Locomotive Emi ssions in Texas,” Prepared for Texas Commission on Environmental Quality, August 15,

2009. Lindhjem, C.E. 2008. “Evaluation Of Mobile Source Control Strategies For The Houston

- Galveston

- Brazoria State Implementation Plan,” Draft Report, December 2008.
Lind hjem, C.E. 2008. “Emission Profiles for EPA SPECIATE Database, Part 2: EPAct FUELS.” Prepared for EPA; Office of Transportation and Air Quality, Ann Arbor, MI. September.

Bar

- Ilan, A., C. Lindhjem. 2008. “Port of Oakland Berthing Load Study.” Prepared for the Port of Oakland, Oakland, CA. April. Lindhjem, C.E. 2008. “Emission Profiles for EPA SPECIATE Database.” Prepared for US Environmental Protection Agency. January.

Bar

- Ilan, A., C. Chandler

- Nogales, C. Lindhjem. 2007. “Massachusetts Diesel PM Em issions Reduction: Retrofit Technical Feasibility Analysis.” Prepared for NESCAUM, Boston, MA. October. Lindhjem, C.E. 2007. “Port of Oakland 2005 Seaport Air Emissions Inventory,” Prepared for the Port of Oakland, Available online at http://www.portofoakland.com/environm/airEmissions.asp , August. Lindhjem, C.E. 2007. “LADCO Nonroad Emission Inventory Project for Locomotive and Commercial Marine Emission Sources,” Prepared for Lake Mi chigan Air Director Consortium by ENVIRON, February. Lindhjem, C.E. 2006. “Los Angeles

- Hobart Railyard Toxic Air Containment Emissions Inventory,” “Commerce Eastern Railyard Toxic Air Containment Emissions,” “Commerce

– Mechanical Railyard
Toxic Air Con tainment Emissions,” “Wilmington Watson Railyard Toxic Air Containment Emissions,” Stockton Railyard Toxic Air Containment Emissions,” “Richmond Railyard Toxic Air Containment Emissions,” ”Barstow Railyard Toxic Air Containment Emissions,” ” San Diego Rai lyard Toxic Air Containment Emissions,” Prepared for BNSF Railway, Available Online at http://www.arb.ca.gov/railyard/hra/hra.htm Lindhjem, C.E. 2006. “Evaluation Of Mobile Source Control Strateg ies For The Houston

- Galveston

- Brazoria State Implementation Plan,” Prepared for the Houston

- Galveston Area Council, May. Lindhjem, C.E. and Yarwood, G. 2004. “Humidity and Temperature Effects on On

- road and Off

- road Emissions and Ozone Formation (HARC Proj ect H8B), Prepared for Houston Advanced Research Center (HARC), November. Lindhjem, C.E., Shepard, S. 2004. “Development Work For Improved Heavy

- Duty Vehicle Modeling Capability Data Mining

– FHWA Datasets Phase II: Final Report,” EPA Contract No. 68

- C

- 02

- 022 Work Assignment No. 2

- 6 Prepared for Evelyn Sue Kimbrough, Atmospheric Protection Branch, Office of Research and Development, U.S. Environmental Protection Agency, September. Lindhjem, C.E., Shepard, S., Pollack, A.K. 2004. “LADCO/MPCA Total Volume And Vehicle Classification Temporal Profiles,” Prepared for Lake Michigan Air Directors Consortium, September. Lindhjem, C.E. and Pollack, A.K. 2004. “Analysis of EPA’s Draft Plan for Emissions Modeling in MOVES and MOVES GHG, Project E

- 68, Prepared for Coord inating Research Council, Inc., May. ENVIRON. 2004. “Cold Ironing Cost Effectiveness,” Prepared for the Port of Long Beach by ENVIRON International Corporation, March 30, 2004. A. Pollack, R. Chi, C. Lindhjem, C. Tran, P. Chandraker. 2004. “Development o f WRAP MOBILE Source Emission Inventories”. Prepared for Western Governors’ Association, Denver, Colorado. February. A. Pollack, C. Lindhjem, T.E. Stoerckenius , C. Tran,G. Mansell, M. Jimenez, G. Wilson, and S. Coulter

- Burke 2003. “CRC Project E

- 64 Eva luation of the U.S EPA MOBILE6 Highway Vehicle Emission Factor Model,” Prepared for Coordinating Research Council, Inc., December.Page 41

Lindhjem, C.E., A. Pollack, R. Friesen, B. Sylte, D.Calkins, T. McGuire, D. Baldwin. 2003. “Potential Control
Measures for Reducing Visibility

- Related Pollutants in Maricopa County.” Prepared for Arizona Department of Environmental Quality. December. Lindhjem, C.E., A.K. Pollack, T.E. Stoerckenius, C. Tran, G. Mansell, M. Jimenez, G. Wilson, S.Coulter

- Burke.
2003. “Evaluat ion of the US EPA MOBILE6 Highway Vehicle Emission Factory Model. December. Lindhjem, C.E., A. Pollack, C. Tran, T. Stoeckenius, R. J. Downing, R. Schindler, E. Raisanen, D. Konopka, R. Sedlacek. 2003. “Maricopa County 2002 Comprehensive Emission Invent ory; For the Cap and Trade Oversight Committee.” Prepared for Arizona Department of Environmental Quality. October. Lindhjem, C.E., Shepard, S. 2003. “Development Work For Improved Heavy

- Duty Vehicle Modeling Capability, Phase I Final Report,” EPA Contra ct No. 68

- C

- 02

- 022 Work Assignment No. 1

- 7, Prepared for Evelyn Sue Kimbrough, Atmospheric Protection Branch, Office of Research and Development, U.S. Environmental Protection Agency, September. Lindhjem, C.E., P. Chandraker. 2003. “Updates to Off

- Road M obile Emissions for East Texas.” Prepared for the East Texas Council of Governments. September. Lindhjem, C.E., M. Keinath, C. Tran. 2003. “Literature Survey of Mobile Source Air Toxic Emission Studies.” Prepared for Office of Transportation and Air Qu ality; U.S. Environmental Protection Agency, September. Lindhjem, C.E., S. Shepard. 2003. “Development Work for Improved Heavy

- Duty Vehicle Modeling Capability.” Prepared for Atmospheric Protection Branch; U.S. Environmental Protection Agency, Septembe r. Lindhjem, C.E. 2002. “Improving Local Air Emissions Inventories From Nonroad Sources,” Prepared for Lake Michigan Air Director Consortium, November. Lindhjem, C.E., R.E. Morris, A.K. Pollack, G.E. Mansell, Ph.D., Y. Jia, G. Wilson. 2002. “ Impact o f Biodiesel Fuels on Air Quality and Human Health.” Summary Report. Prepared for National Renewable Energy Laboratory. Golden, CO. November. Lindhjem, C.E. 2002. “Review of EPA Draft Report: “Updating Fuel Economy Estimates in MOBILE6.3,” Prepared for U.S. Environmental Protection Agency, Office of Transportation and Air Quality, June. Lindhjem, C.E. 2002. “Commercial Marine Emission Inventory Development,” Prepared for U.S. Environmental Protection Agency, Office of Transportation and Air Quality, Apri l. Lindhjem, C.E. and Stoeckenius, T., 2002. “On

- Board Data Analysis and Collection for the New Generation Model, PR

- CI

- 01

- 12239,” Prepared for U.S. Environmental Protection Agency, Office of Transportation and Air Quality, Ann Arbor, MI. February. Lindhje m, C.E., A.K. Pollack and R. Chi. 2001. “MOBILE6 Beta Testing.” Prepared for U.S. Environmental Protection Agency, Office of Transportation and Air Quality, Ann Arbor, MI. January. Lindhjem, C.E., C. Tran and A.K. Pollack. 2000. “Draft NONROAD Model Gasoline Consumption Estimates

– Comparison with Other Data Sources.” U.S. Environmental Protection Agency, October. Lindhjem, C.E., 2000. “NONROAD and OFFROAD Models: Small (<25 hp) Spark

- Ignition Engine Exhaust
Emissions.” U.S. Environmental Protectio n Agency, October. Lindhjem, C.E., A.K. Pollack. 2000. “NONROAD and OFFROAD Models: Recreational Equipment Comparison.” U.S. Environmental Protection Agency, October. Lindhjem, C.E., (with others). 2000. “Study of Air Quality Programs in Clark Count y, Nevada,” Pursuant to Senate Bill 432, Chapter 529, Statutes of Nevada 1999. Prepared for Legislative Counsel Bureau. September. Lindhjem, C.E. 2000. “NONROAD and OFFROAD Models: Comparison of EPA and ARB Input Data for Emissions from Large Spark

- Ign ition Engines.” U.S. Environmental Protection Agency, September. Lindhjem, C.E. 2000. “Estimates of the Effect of the School Year on Vehicle Activity.” Prepared for the Texas Natural Resource Conservation Commission. September. Lindhjem, C.E., (with ot hers). 2000. “On

- Road Motor Vehicle NOx Emissions Reduction Measures for the Sacramento Metropolitan Area.” Prepared for the Sacramento Council of Governments. August.Page 42

Lindhjem, C.E., (with others). 2000. “Houston

- Galveston Area Marine Vessel Emissio n Inventory.” Prepared for the Port of Houston Authority and Texas Natural Resource Conservation Commission.

June. Lindhjem, C.E. 2000. “NONROAD and OFFROAD Models: Recreational Marine Comparison.” U.S. Environmental Protection Agency, February.
Lind hjem, C.E. 2000. “NONROAD and OFFROAD Models: Comparison of Modeling Input Data for Compression

- Ignition Engines.” U.S. Environmental Protection Agency, January. Lindhjem, C.E., (with others). 1999. “Control Measures for the Dallas

- Ft. Worth.” Prepa red for the North Central Texas Council of Governments. September. Lindhjem, C.E., Coralee Cooper. 1999. “Heavy

- Duty Nonroad Engine Activity and Emissions in the Northeast.” Northeast States for Coordinated Air Use Management, August. Lindhjem, C.E., A.K. Pollack. 1999. “TNRCC Construction Equipment Emissions Project.” Prepared for the Texas Natural Resources Conservation Commission. February. Lindhjem, C.E. 1998. “ARB OFFROAD and EPA NONROAD Models: Comparison of Model Structures.” U.S.

Environme ntal Protection Agency, December. Lindhjem, C.E., A.K. Pollack. 1998. “Comparison of EPA NONROAD and ARB OFFROAD models: Population Estimates and Allocation to Counties for Small (<25 hp) Spark Ignition (SI) Engines.” U.S. Environmental Protection Agenc y, December. Lindhjem, C.E., M. Beardsley, C. Harris. 1998. “Exhaust Emission Factors for Nonroad Engine Modeling

-- Spark Ignition.” Report No. NR

- 010, Documentation Supporting EPA’s Draft Release of the NONROAD Model. February. Lindhjem, C.E., M. Beard sley. 1998. “Exhaust Emission Factors for Nonroad Engine Modeling

-- Compression

- Ignition.” Report No. NR

- 009, Documentation Supporting EPA’s Draft Release of the NONROAD Model. February. Lindhjem, C.E., 1997. “Nonroad Engine Population Estimates.” Re port No. NR

- 006, Documentation Supporting EPA’s Draft Release of the NONROAD Model. December. Lindhjem, C.E., M. Beardsley. 1997. “Average Life, Annual Activity, and Load Factor Values for Nonroad Engine Emissions Modeling.” EPA/OMS Report NR

- 005, Docu mentation Supporting EPA’s Draft
Release of the NONROAD Model. December. Lindhjem, C.E., 1997. “Exhaust Emission Effects of Fuel Sulfur and Oxygen on Gasoline Nonroad Engines.” Report No. NR

- 003, Documentation Supporting EPA’s Draft Release of the NONR OAD Model.
November. Lindhjem, C.E., 1997. “Conversion Factors for Hydrocarbon Emission Components.” Report No. NR

- 002, Documentation Supporting EPA’s Draft Release of the NONROAD Model. November. Lindhjem, C.E., T. Jackson. 1997. “Update on Heavy

- Du ty Emission Levels (Model Years 1988

- 2004+) for Use in MOBILE6.” Draft Report M6HDE.001. October. Lindhjem, C.E., 1997. “Nonroad Mobile Emissions Modeling.” Air & Waste Management Conference on Emission Inventory: Planning for the Future, RTP, NC. Oct ober. Lindhjem, C.E., 1997. “Heavy

- Duty Diesel Emissions Modeling.” Air & Waste Management Conference on Emission Inventory: Planning for the Future, RTP, NC. October. Lindhjem, C.E. 1997. “Emission Rates from Highway and Nonroad Diesel Engines.” Worl d Car Conference, Riverside, California. January. Lindhjem, C.E., 1995. “Heavy

- duty Engine Testing Report; Emissions from Idling Truck Engines.” EPA Technical Report. May. Lindhjem, C.E., 1994. “The Effect of Fuel Reformulation on 4

- stroke Lawn and Gar den Engines.” EPA Technical Report. July. Lindhjem, C.E., 1993. “The Exhaust VOC Emission Inventory by Vehicle Emitter Class Following Implementation of an Enhanced Inspection and Maintenance (I/M) Program.” EPA Technical Report (for use in EPA's rulem aking on Reformulated Gasoline using the Complex Model). June.Page 43

Lindhjem, C.E., 1993. “Speciation of Evaporative Emissions.” EPA Technical Report. April. Lindhjem, C.E., 1992. “Exhaust, Evaporative, and Running Loss Speciation.” EPA Technical Report. Jul y. Lindhjem, C.E., 1992. “Vapor Pressure at 130 o F as the Parameter of choice for Running Loss and Hot Soak Emissions Correlation.” EPA Technical Report. April. Lindhjem, C.E., 1992. “Effect of Oxygenates on Emissions.” EPA Technical Report (for use in EPA's Rulemaking on Reformulated Gasoline using the Simple Model). January. Lindhjem, C.E., 1991. “Multiple Day Diurnal, Purge Failure, and Pressure Failure Evaporative Emissions.” EPA Technical Report (for use in EPA's MOBILE emission factors model fo r mobile source emission estimations). November.Page 44 Brian H. West, P.E. Deputy Director of Fuels, Engines, and Emissions Research Center

Oak Ridge National Laboratory NTRC Building, 2360 Cherahala Blvd Knoxville, TN 37932

865

- 946

- 1231

EDUCATION

M. S. Me chanical Engineering, Virginia Tech, 1988 B. S. Mechanical Engineering, Clemson University, 1986

PROFESSIONAL HISTORY

Oak Ridge National Laboratory, Oak Ridge, TN, 5/88

– present

 Development Engineer and Deputy Director of Fuels, Engines, and E missions Research

Center

 Design, planning, management, and execution of projects to evaluate advanced engines, catalysts and emission control systems, materials, and alternative fuels.

 Leader for ORNL vehicle research activities, including procedure/prot ocol development and assessment and characterization of alternative fuels, advanced diesel emissions control, full electric prototype USPS vehicles, unique European vehicles, etc.

 Leader for Department of Energy (DOE) / Coordinating Research Council (CRC) test program to investigate the effects of mid

- level ethanol blends on vehicles.

 Co

- chair of NOx adsorber technical workgroup for Advanced Petroleum

- Based Fuels

- Diesel Emission Control Program, an industry

- government cooperative program.

 Leader for D epartment of Transportation program to develop modal emissions and fuel consumption models for traffic simulations

MPR Associates, Incorporated, Alexandria, VA, 2/98

- 8/98

Senior Engineer at this Engineering Consulting Firm. Developed engine governo r troubleshooting manual for emergency diesel generators operated by nuclear power plants. Conducted electrical power plant assessments in Brazil, including coal

- fired, oil

- fired, and hydroelectric plants.

Hoechst Fibers Industries, Spartanburg, SC, 1/84

- 8/85

Co

- op Engineer at this synthetic fiber producer for 3 semesters during undergraduate study at Clemson University. Assisted senior engineering staff in maintenance and development engineering in the plant.Page 45

Motorola Portable Products, Fort Lauder dale, FL, 1/83

- 5/83

Co

- op Engineer at this pager development and manufacturing facility. Assisted Mechanical Laboratory manager and other engineers in testing components and maintaining equipment for shock, vibration, and strength testing.

Profession al Activities, Honors, Awards

 DOE R&D Award for Managing Mid

- Level Ethanol Blends Vehicle aging program (2010 DOE Merit Review)

 Member of Society of Automotive Engineers (23 years).

 Recipient of SAE Lloyd L. Withrow Distinguished Speaker Award (April 20 09).

 Two patents

 Invited Technical Panelist for DOE Merit Review

 Licensed Professional Engineer in Tennessee

Selected Publications

West, Brian H., C. Scott Sluder, Keith E. Knoll, John E. Orban, and Jingyu Feng, “Intermediate Ethanol Blends Catalyst Dura bility Program,” ORNL/TM

- 2011/234.

Sluder, C. Scott, and Brian West, “Limitations and Recommended Practice In the Use of Compression and Leak

- Down Tests to Monitor Gradual Engine Degradation,” SAE Int. J. Engines, Vol. 4, Issue 3, December 2011.

Sluder, C. Scott, and Brian H. West, “NMOG Emissions Characterizations and Estimation for Vehicles Using Ethanol

- Blended Fuels,” ORNL/TM

- 2011/461, October 2011.

Knoll, Keith, Brian West, Shean Huff, John Thomas, John Orban, Cynthia Cooper, “Effects of Mid

Level E thanol Blends on Conventional Vehicle Emissions,” Society of Automotive Engineers Paper

2009

- 01

- 2723, November 2009.

Knoll, Keith, Brian West, Wendy Clark, Ronald Graves, John Orban, Steve Przesmitzki, Timothy Theiss, “Effects of Intermediate Ethanol Blen ds on Legacy Vehicles and Small Non

- Road Engines, Report 1

– Updated,” ORNL/TM

- 2008/117, February 2009.

Norman, Kevin, Shean Huff, and Brian West, “Effect of Intake Air Filter Condition on Vehicle Fuel Economy,” ORNL/TM

- 2009/021, February 2009.

Parks, Ja mes E., Brian West, and Shean Huff, “Characterization of Lean NOx Trap Catalysts with In

Cylinder Regeneration Strategies,” SAE Paper Number 2008

- 01

- 0448, April 2008

West, Brian H., Alberto J. Lopez, Timothy J. Theiss, Ronald L. Graves, John Storey, and S am Lewis “Fuel Economy and Emissions of the Ethanol

- Optimized Saab 9

- 5 BioPower,” Society of Automotive Engineers Paper 2007

- 01

- 3994, October 2007.Page 46

Bechtold, R., J. F. Thomas, S. P. Huff, J. P. Szybist, T. J. Theiss, B. H. West, M. Goodman, T. A. Timbario, “Technical Issues Associated with the Use of Intermediate Ethanol Blends (>E10) in the U.S. Legacy Fleet: Assessment of Prior Studies,” ORNL/TM

- 2007/37, August 2007.

Swartz, Matthew M., Shean Huff, James Parks, and Brian West, “Intra

- Catalyst Reductant Chemistry and NOx Conversion of Diesel Lean NOx Traps at Various Stages of Sulfur Loading,” Society of Automotive Engineers Paper 2006

- 01

- 3423, October 2006.

West, Brian H., Shean Huff, James E. Parks, Matt Swartz, and Ronald L. Graves, “In

- Cylinder
Produ ction of Hydrogen During Net

- Lean Diesel Operation,” Society of Automotive Engineers Paper 2006

- 01

- 0212, April 2006

Huff, Shean, Brian West, James E. Parks, Matt Swartz, Johney Green, and Ronald Graves, “In

Cylinder Regeneration of Lean NOx Trap Catalysts Using Low Temperature Combustion,” Society of Automotive Engineers Paper 2006

- 01

- 1416, April 2006

Parks, James E., Shean Huff, Josh Pihl, Jae

- Soon Choi, and Brian H. West, “Nitrogen Selectivity in Lean NOx Trap Catalysis With Diesel Engine In

- Cylinder Re generation,” Society of Automotive Engineers Paper 2005

- 01

- 3876, October 2005.

Witze, Peter O., Shean P. Huff, John M. Storey, Brian H. West, “Time

- Resolved, Laser Induced Incandescence Measurements of Particulate Emissions During Enrichment for Diesel Le an NOx Trap Regeneration,” Society of Automotive Engineers Paper 2005

- 01

- 0186, April 2005.

West, Brian H., Shean P. Huff, James E. Parks, Jae

- Soon Choi, William P. Partridge, Sam A. Lewis, John M. Storey, “Assessing Reductant Chemistry During In

- Cylinder Regeneration of Diesel Lean NOx Traps,” Society of Automotive Engineers Paper 2004

- 01

- 3023, October 2004.

Sluder, C. Scott, and Brian H. West, “Effects of Regeneration Conditions on NOx Adsorber Performance,” Society of Automotive Engineers Paper 2002

- 01

- 2876, October 2002.

Sluder, C. Scott, and Brian H. West, “Performance of a NOx Adsorber and Catalyzed Particle Filter System on a Light

- Duty Diesel Vehicle,” Society of Automotive Engineers Paper 2001

- 01

- 1933, May 2001.

West, Brian H., and C. Scott Slude r, “NOx adsorber performance in a Light

- Duty Diesel Vehicle,” Society of Automotive Engineers Paper 2000

- 01

- 2912, October 2000.

Sluder, C. Scott, and Brian H. West, “Catalyzed Diesel Particulate Filter Performance in a Light

Duty Vehicle,” Society of Auto motive Engineers Paper 2000

- 01

- 2848, October 2000.

West, B. H., R. N. McGill, J. W. Hodgson, C. S. Sluder, D. E. Smith "Development of Data

- Based, Light

- Duty Modal Emissions and Fuel Consumption Models," Society of Automotive Engineers Paper 972910, Octob er 1997.Page 47

West, B. H., R. N. McGill, C. S. Sluder, and J. W. Hodgson, “Data

- Based Modal Emissions and Fuel Consumption Models for Transportation Assessments,” Air and Waste Management Association, Proceedings of Emission Inventory: Planning for the Future

, October 1997.

West, B. H., J. M. Storey, S. A. Lewis, G. L. Devault, J. B. Green, C. S. Sluder, J. W. Hodgson, and B. L. Moore, “Experimental and Numerical Assessment of On

- Road Diesel and Biodiesel Emissions,” Proceedings of 1997 Diesel Engine Emissions Reduction Workshop , July 1997.

West, B., J. B. Green, Effects of Piston Surface Treatments on Performance and Emissions of a
Methanol

- Fueled, Direct

- Injection, Stratified

- Charge Engine , National Renewable Energy Laboratory Report NREL/TP

- 425

- 6161, July 1 994. West, B. H., R. N. McGill, S. L. Hillis, and J. W. Hodgson, Federal Methanol Fleet Project Final Report , Oak Ridge National Laboratory Report ORNL/TM

- 12278, March 1993.

West, B. H., and R. N. McGill, "Oil Performance in a Methanol

- Fueled Vehicle Use d in Severe
Short

- Trip Service," Society of Automotive Engineers Paper 922298, October 1992.

McGill, R. N., R. L. Graves, B. H. West, and J. W. Hodgson, "The Federal Methanol Fleet

- Summary of Technical Data," Published in proceedings of IX International Symposium on Alcohol Fuels

, Firenze, Italy, November 12

- 15, 1991.

West, B. H., R. R. Judkins, R. L. Graves, C. S. Daw, R. P. Krishnan, A. L. Compere, and W. L. Griffith,
"Mild

- Gasification Product Characterization

-- 1991 Results," in Proceedings of the El eventh Annual Gasification and Gas Stream Cleanup Systems Contractors Review Meeting , Vol. 1, DOE/METC

91/6123, Morgantown, WV, August 1991.

McGill, R. N., and B. H. West, "The Federal Methanol Fleet

- A Demonstration of Methanol Vehicle Technology," Publi shed in Proceedings of Technical and Business Exposition and Symposium

, Huntsville, AL, May 14

- 15, 1991.

Brinkman, C. R., G. M. Begun, R. L. Graves, W. K. Kahl, K. C. Liu, and B. H. West, Influence of Diesel Engine Exposure on the Rupture Strength of Sili con Nitride and Partially Stabilized Zirconia , Oak Ridge National Laboratory Report ORNL/TM

- 6612, June 1990.

Daw, C. S., R. L. Graves, R. P. Krishnan, B. H. West, A. L. Compere, and W. L Griffith, "Mild

Gasification Product Characterization

-- 1990 Result s," in Proceedings of the Tenth Annual Gasification and Gas Stream Cleanup Systems Contractors Review Meeting , Vol. II, DOE/METC

90/6115, Morgantown, WV, August 1990.

West, B. H., A. L. Compere, W. L. Griffith, "High

- Alcohol Microemulsion Fuel Performance in a Diesel Engine," Society of Automotive Engineers Paper 902101, October 1990.

Graves, R. L., C. S. Daw, R. P. Krishnan, and B. H. West, "Mild

- Gasification Product Characterization," in Proceedings of the Ninth Annual Gasification and Gas Stream Cleanu p Systems Contractors Review Meeting , Vol. II, DOE/METC

- 89/6107, Morgantown, WV, June 1989.Page 48

Nelson, D. J., and B. H. West, "Fully Developed Natural and Mixed Convection in Inclined Rectangular Ducts," American Society of Mechanical Engineers paper publishe d in Proceedings of 1988 Winter Annual Meeting , December 1988.Page 49 

### Appendix B : Conflict of Interest Statements

Conflict of Interest and Bias for Peer Review

Background

Identification and management of potential conflict of interest (COI) and bias issues are vital to the successes and credibility of any peer review consisting of external experts. The questionnaire that follows is consistent with EPA guidance concerning peer reviews. 1

Definitions

Experts in a particular field will, in many cases, have e xisting opinions concerning the subject of the peer review. These opinions may be considered bias, but are not necessarily conflicts of interest.

Bias : For a peer review, means a predisposition towards the subject matter to be discussed that could influ ence the candidate's viewpoint.

Examples of bias would be situations in which a candidate:

1. Has previously expressed a position on the subject(s) under consideration by the panel; or

2. Is affiliated with an industry, governmental, public interest, o r other group which has expressed a position concerning the subject(s) under consideration by the panel.

Conflict of Interest : For a peer review, as defined by the National Academy of Sciences, 2 includes any of the following:

1. Affiliation with an orga nization with financial ties directly related to the outcome;

2. Direct personal/financial investments in the sponsoring organization or related to the subject; or

3. Direct involvement in the documents submitted to the peer review panel

... that could im pair the individual's objectivity or create an unfair competitive advantage for the individual or organization.

Policy and Process

● Candidates with COI, as defined above, will not be eligible for membership on those panels where their conflicts apply.

1 U.S. EPA (200 9 ). Sc ience Policy Council Peer Review Handbook . OMB (2004). Final Information Quality Bulletin for Peer

Review.
2 NAS (2003). "Policy and Procedures on Committee Composition and Balance and Conflict or Interest for Committees Us ed in the Development of Reports" (www.nationalacademies.org/coi).Page 50

● In general, candidates with bias, as defined above, on a particular issue will be eligible for all panel memberships; however, extre me biases, such as those likely to impair a candidate's ability to contribute to meaningful scientific discourse, will disqualify a candidate.

● Ideally, the composition of each panel will reflect a range of bias for a particular subject, striving for bal ance.

● Candidates who meet scientific qualifications and other eligibility criteria will be asked to provide written disclosure through a confidential questionnaire of all potential COI and bias issues during the candidate identification and selection pr ocess.

● Candidates should be prepared, as necessary, to discuss potential COI and bias issues.

● All bias issues related to selected panelists will be disclosed in writing in the final peer review record.Page 51 Conflict of Interest and Bias Questionnaire

E PAct Study Analysis Peer Review

Instructions to Candidate Reviewers

1. Please check YES/NO/DON'T KNOW in response to each question.

2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the circumstances.

3. Please make a reaso nable effort to answer accurately each question. For example, to the extent a question applies to individuals (or entities) other than you (e.g., spouse, dependents, or their employers), you should make a reasonable inquiry, such as emailing the questions to such individuals/entities in an effort to obtain information necessary to accurately answer the questions.

Questions

1. Are you (or your spouse/partner or dependents) or your current employer, an author, contributor, or an earlier reviewer of the doc ument(s) being reviewed by this panel?

YES___ NO_ X

_ DON'T KNOW_

_ _

2. Do you (or you spouse/partner or dependents) or your current employer have current plans to conduct or seek work related to the subject of this peer review following the completion of t his peer review panel?

YES_

__ NO

_ X

_ DON'T KNOW___

3. Do you (or your spouse/partner or dependents) or your current employer have any known financial stake in the outcome of the review (e.g., investment interest in a business related to the subject of pe er review)?

YES___ NO

_ X

_ DON'T KNOW___

4. Have you (or your spouse/partner or dependents) or your current employer commented, reviewed, testified, published, made public statements, or taken positions regarding the subject of this peer review?

YES_

__ N O

_ X

_ DON'T KNOW___

5. Do you hold personal values or beliefs that would preclude you from conducting an objective, scientific evaluation of the subject of the review?

YES___ NO

_ X

_ DON'T KNOW___Page 52

6. Do you know of any reason that you might be unable to pro vide impartial advice or comments on the subject review of the panel?

YES___ NO

_ X

_ DON'T KNOW___

7. Are you aware of any other factors that may create potential conflict of interest or bias issues for you as a member of the panel?

YES___ NO

_ X

_ DON'T KNO W___

Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I further declare that I have made r easonable effort and inquiry to obtain the information needed to answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change in circumstances that would require me to revise the answers that I have provided.

Xuming He
Name

____________________ ______ 11/15/11

________ Signature DatePage 53 Conflict of Interest and Bias Questionnaire

EPAct Study Analysis Peer Review

Instructions to Candidate Reviewers

1. Please check YES/NO/DON'T KNOW in response to each questi on.

2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the circumstances.

3. Please make a reasonable effort to answer accurately each question. For example, to the extent a question applies to individuals (or entities) other than you (e.g., spouse, dependents, or their employers), you should make a reasonable inquiry, such as emailing the questions to such individuals/entities in an effort to obtain information necessary to accurately answer the questions.

Questions

1. Are y ou (or your spouse/partner or dependents) or your current employer, an author, contributor, or an earlier reviewer of the document(s) being reviewed by this panel?

YES___ NO___ DON'T KNOW_ X

_
[Chris Lindhjem may have investigated this data on behalf of E PA to develop speciation (chemical composition) estimates for emissions from these fuels. The description of this work primarily focuses on regulated emissions, so speciation would not be an element of this review.]
2. Do you (or you spouse/partner or dep endents) or your current employer have current plans to conduct or seek work related to the subject of this peer review following the completion of this peer review panel?

YES_

__ NO___ DON'T KNOW

_ X

_
[ENVIRON has had a business relationship with an etha nol producer reviewing emissions data related to fuel certification.]
3. Do you (or your spouse/partner or dependents) or your current employer have any known financial stake in the outcome of the review (e.g., investment interest in a business related

to the subject of peer review)?

YES___ NO_ X

_ DON'T KNOW___Page 54

4. Have you (or your spouse/partner or dependents) or your current employer commented, reviewed, testified, published, made public statements, or taken positions regarding the subject of this peer review?

YES_

__ NO

_ X

_ DON'T KNOW___

5. Do you hold personal values or beliefs that would preclude you from conducting an objective, scientific evaluation of the subject of the review?

YES___ NO

_ X

_ DON'T KNOW___

6. Do you know of any reason that you mig ht be unable to provide impartial advice or comments on the subject review of the panel?

YES___ NO

_ X

_ DON'T KNOW___

7. Are you aware of any other factors that may create potential conflict of interest or bias issues for you as a member of the panel?

YES

___ NO_ X

_ DON'T KNOW___

Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I further declar e that I have made reasonable effort and inquiry to obtain the information needed to answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change in circumstances that would require me to revise the answers that I have pro vided.

Chris Lindhjem
Name

_____________________________ January 9, 2012 Signature DatePage 55 Conflict of Interest and Bias Questionnaire

EPAct Study Analysis Peer Review

Instructions to Candidate Reviewers

1. Please check YES/NO/DON'T K NOW in response to each question.

2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the circumstances.

3. Please make a reasonable effort to answer accurately each question. For example, to the extent a question applies to in dividuals (or entities) other than you (e.g., spouse, dependents, or their employers), you should make a reasonable inquiry, such as emailing the questions to such individuals/entities in an effort to obtain information necessary to accurately answer the q uestions.

Questions

1. Are you (or your spouse/partner or dependents) or your current employer, an author, contributor, or an earlier reviewer of the document(s) being reviewed by this panel?

YES___ NO

_ X

_ DON'T KNOW_

_ _

2. Do you (or you spouse/partner or dependents) or your current employer have current plans to conduct or seek work related to the subject of this peer review following the completion of this peer review panel?

YES_

__ NO___ DON'T KNOW

_ X

_
In 23 years at QRNL I have worked on a wide ran ge of fuels, engine, and emissions control technologies, conducting experiments in engine labs, vehicle dynamometer labs, or managing subcontracts at contractor facilities such as SwRI or TRC. I presume that I am a candidate reviewer because of this experi ence. Future work at ORNL could very well be related to the subject of the EPAct study, just as much of my past work experience has been; however, I do not currently have plans that are related to the EPAct study.

3. Do you (or your spouse/partner or depe ndents) or your current employer have any known financial stake in the outcome of the review (e.g., investment interest in a business related to the subject of peer review)?

YES___ NO

_ X

_ DON'T KNOW___Page 56

4. Have you (or your spouse/partner or dependents) or your current employer commented, reviewed, testified, published, made public statements, or taken positions regarding the subject of this peer review?

YES_

__ NO___ DON'T KNOW

_ X

_
I endorsed using DOE funds to augment the EPA funding for the subject tes t program. I managed a large DOE program (DOE V4 program) from 2008

- 2011 in which over 80 vehicles were aged and emissions tested with ethanol blends (at SwRI and TRC). Emissions measurements in the V4 program were largely limited to criteria pollutants. S everal national lab colleagues and I saw value in the detailed exhaust speciation work to be done at SwRI under the EPAct program, and endorsed having DOE leverage that effort. DOE funds (through NREL contract with SwRI) supported part of the EPAct (DOE V2 ) project to allow a wider range of fuels, including ethanol blends (of particular interest to DOE). I shared my position on the matter with DOE sponsors and NREL, ORNL, and EPA colleagues in meetings in 2008. I have not testified or published any statemen ts to this effect

5. Do you hold personal values or beliefs that would preclude you from conducting an objective, scientific evaluation of the subject of the review?

YES___ NO

_ X

_ DON'T KNOW___

6. Do you know of any reason that you might be unable to pro vide impartial advice or comments on the subject review of the panel?

YES___ NO

_ X

_ DON'T KNOW___

7. Are you aware of any other factors that may create potential conflict of interest or bias issues for you as a member of the panel?

YES___ NO

_ X

_ DON'T KNO W___

Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I further declare that I have made r easonable effort and inquiry to obtain the information needed to answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change in circumstances that would require me to revise the answers that I have provided.

Brian West

____________ ____ _ Name

_____________________________ ___________ Signature DatePage 57 

### Appendix C : Peer Review Charge

Charge to Peer Reviewers of Assessing the Effect of Five Gasoline Properties on Exhaust Emissions from Light

- Duty Vehicles certifie d to Tier

- 2 Standards (EPAct/V2/E

- 89: Phase 3)

Past fuel effects models, such as the EPA Predictive Model and EPA’s Complex Model, were based on data collected using 1990s

- technology cars and trucks meeting Tier 0 and Tier 1 vehicle
emission standards at pollutant levels an order of magnitude higher than compliance levels for
current Tier2 vehicles. With the current on

- highway fleet turning out much lower emitting vehicles than in past years, the U.S. Congress, the EPA and a variety of stakeholders are int erested in generating an updated data set for fuel effects models to guide vehicle emissions policy going forward. Further, Section 1506 of the Energy Policy Act (EPAct) of 2005 directed EPA to generate a more current fuel effects model representing future gasoline vehicle fleet emissions. Statutory requirements such as the one outlined in Section 209 of the Energy Independence and Security Act (EISA) of 2007, an anti

- backsliding assessment of the effects of increased renewable fuels use on air quality, are likewise dependent upon the output of updated fuel effects models.

This report describes the analysis and modeling of data collected in Phase 3 of the EPAct/V2/E

- 89 light duty gasoline vehicle fuel effects study (referred to here as “the EPAct program”, or “Phase 3 of the EPAct program”). This study examined the exhaust emission impacts on vehicles of changing levels of five fuel properties (ethanol, T50, T90, aromatics, and RVP (specified as DVPE)) in a matrix of 27 gasoline fuels, arranged according to a partial factorial design optimized for selected interactions of interest. The range in the levels of fuel properties tested is comparable to those seen in current market fuels and in potential mid

- level ethanol blended fuels. The test
vehicle fleet consi sted of 15 new 2008 light

- duty cars and trucks, selected from among high sales makes and models, to provide a representative sample of a fleet of properly

- operating vehicles meeting the U.S. Federal Tier 2 vehicle emission standards. Given the relatively low level of emissions from these vehicles, a number of design and procedural steps were undertaken to minimize the impacts of measurement variability and other artifacts on data quality.

Vehicle emission test data collected in the program included conc entrations and rates for typical regulated pollutants by bag and by second

- by

- second collection, plus data for speciated emissions for a subset of vehicle emission tests and bags. Complete data was generated for 926 tests, with
30 additional tests containi ng valid measurements for regulated emissions. The analysis of this data focuses on producing reduced mixed

- effect models by emission bag for the pollutants of interest (nitrogen oxides, total hydrocarbon, non

- methane organic gases and carbon monoxide)
aft er screening for outliers and other data quality issues. Fuel effects models were produced using the target set of six interactive terms, for which the fuel matrix was optimized, plus a set of terms generated by using all possible interactive terms (eleven ) and evaluating fit parameters for both. A conservative approach was taken on outliers, such that abnormal statistical parameters were generally not sufficient to remove data without an underlying measurement or procedural problem. Relatively few outliers were removed based on having an unrealistically high value (only four measurements, all from the particulate mass emission dataset), but portions of the nitrogen oxides data were removed after data quality review suggested some very low

- level

measurements from certain vehicles were suspected to contain a large amount of measurementPage 58

error. Another issue encountered during analysis was treatment of non

- detects (zero values), which were addressed via use of Tobit regressions (i.e., the PROC LIFEREG procedure in SAS).

The primary objective of this program is to generate data to produce models with good predictive abilities for vehicles and fuels beyond those specifically tested in this program. Thus, minimizing retention of terms that are based largely on meas urement or design artifact or are otherwise
overfit to these vehicles and fuels is a concern.

EPA is seeking your expert opinion on the appropriateness of the statistical techniques described
in the EPAct Study Analysis and their appropriateness in the co ntext of any data accuracy/quality issues. Given your knowledge and understanding of the accuracy of the test measurements, has EPA chosen the most useful means of statistical analysis? Model validation using external datasets and crossvalidation (testing predictive ability using portions of data omitted during model fitting) are only addressed briefly here and remain as possible subjects of ongoing report and comment and future study.

Some specific areas of focus include the following:

1. Was the proce ss of imputation of NMOG/NMHC results for tests/bags with missing speciation data reasonable and statistically sound? (Section X.X)

2. Was the decision to remove very low emitting and influential vehicles from the NOx and NMOG analyses reasonable? (Sect ions 5.5 and 6.1)

3. Please comment on the use of the “design set” of 11 terms as the basis of the final models, versus allowing model to fit all 17 terms, including the adequacy of the justification of this decision. (Section 7)

4. Comment on the use of the Tobit regression (SAS PROC LIFEREG) for modeling datasets with large numbers of censored values. (Section 5.3)

5. Comment on the decision to independently model the linear terms and interactions between fuel blends as presented in the final results?

6

. Please comment on the methods used to select reduced models. (Section 5)

In addition to addressing these issues, EPA encourages you to best apply your particular area(s) of expertise to review the overall study.

In your comments you s hould distingu ish between recommendations for clearly defined improvements that can be readily made based on data or literature reasonably available to EPA and improvements that are more exploratory or dependent on information not readily available

to EPA. Your comments should be sufficiently clear and detailed to allow readers to thoroughly
understand their relevance to the EPAct Study Analysis

. Please deliver your final written comments to SRA International by Friday, January 20 , 2012.

All materials provided to you a s well as your review comments should be treated as confidential, and should neither be released nor discussed with others outside of the review panel. Once EPA has made its report and supporting documentation public, the Agency will notify you that you

m ay release or discuss the peer review materials and your review comments with others.Page 59

Should you have questions about what is required in order to complete this review or need additional background material, please contact Brian Menard at SRA ( Brian_Menard@sra.com ) or (434

- 817

- 4133)

.Page 60 

### Appendix D: Reviews

Comments on the EPAct Study Analysis

Xuming He

The report under review describes the analysis and modeling of data collected in Phase 3 of the EPAct/V2/E

- 89 light duty gasoline vehicle fuel effects study. The study examined the exhaust emission impacts on vehicles of changing levels of five fuel properties (ethanol, T50, T90, aromatics, and RVP (specified as DVPE)) in a matrix of 27 gasoline fuels, arranged a ccording to a partial factorial design optimized for selected interactions of interest. A number of design and procedural steps were undertaken in this study to reduce the impacts of measurement bias and variability on data quality and statistical analysis . Overall, I found the study well designed and carefully analyzed with generally accepted modern statistical tools. My comments will focus on the appropriateness of the data processing and statistical techniques described in the report, with the purpose of improving the EPAct Study Analysis. The item numbers used below (1

– 6) correspond to the prompts given in the email of December 22 from Brian Menard. Some additional comments and suggestions are given at the end of
this review.

1. Imputation of NMOG/NM HC results for tests/bags with missing speciation data

Due to the speciation schedule described in Section 2.2 of the report, most tests in the dataset do not have alcohol and carbonyl measurements for bags 2 and 3. As NMOG and NMHC are calculated

emissi on results that use speciation data, they could not be computed for the portions of the dataset without speciation. The study used imputation based on an alternate measure of hydrocarbon emissions to fill in the missing values. Linear location

- scale

- type m odels were used to imputation with special substitution of value zero for small NMOG. The report made a convincing case that the models used for imputation fit the data well, and should result in small errors and variability due to

i mputation.

It seems t hat the imputed values were deterministic in the study given the variables x NMHC in Equations 8

- 11. If so, statistical variability could be under

- reported in the subsequent studies. One approach to recommend here is to use multiple imputations to account f or the variability. Based on the descriptions in the report, I do not think that the additional variability due to imputation would be a significant factor, but I would prefer to see a more explicit discussion and examination of this issue in the study.

2. Decision to remove very low emitting and influential vehicles from the NOx and NMOG analyses;

Modifying or removing outliers and influential observations could raise questions about the validity of a study, especially when ad hoc decisions are made af ter the data are collected and examined. The report paid serious attention to data quality, and described how outliers and influential observations
were identified.Page 61

I believe that the decision to use left

- censored models in the study is appropriate. This allows linear models to remain valid for the data with left end points. Such practices have been used in the statistics and econometrics literature. There are, however, several minor issues to deal with.

(a) In Section 5.2, censored measurements were re placed by the minimum positive value measured for the emission and bag. This substitution has the potential to lower the variance estimate of the statistical models, unless censoring is taken into account in the variance estimates. If the error

variance es timates are deflated, we would see higher “studentized residuals”, resulting in false positives in outlier detection. The issue needs to be carefully examined.

(b) At the end of Section 5.2, it was mentioned that Run 6281 in Bag 3 was removed even though it was not flagged as influential. The specific reason for this decision was lacking in the report.

(c) In the analysis of Section 5.3.1, a distinction was made between light censoring and severe censoring. In the case of severe censoring, the Tobit reg ression was used in the data analysis. Otherwise, the censored values were substituted. I do not see good reasons for handling the two scenarios differently. Why not use the Tobit regression in all cases? The current practice would raise a question about t he stability and sensitivity of the results if one more or one fewer data point is
censored.

(d) In Section 5.3.2, both BIC (model selection criterion) and likelihood ratio tests were described and used. Although both approaches are valid and useful, the y have different goals in mind. Model selection based on BIC is to choose models, treating all competing models equally. The chi

- square tests have a null hypothesis in mind, where the null hypothesis refers to the smaller models in the present analysis. Te st decisions are designed to protect the null hypothesis, so the competing models are not treated equally. When both approaches are used, one needs to be clear how they work together, and what are to be achieved. I do not imply that anything has gone wrong here, but this part of the analysis needs to be made clearer as to why one cannot simply use BIC.

3, 5 and 6. Use of the “design set” of 11 terms as the basis of the final models, versus allowing model to fit all 17 terms, including the adequacy of the justification of this decision; the decision to independently model the linear terms and interactions between fuel blends as presented in the final results; the
methods used to select reduced models.

Table 36 report correlation coefficients between the l inear

- effects and the additional terms (interactions). I would suggest including the canonical correlation between the set of linear

- effects and the set of interactions. This would assess linearity beyond pairwise correlations. The considerations and just ifications on finding final models (Section 7.2) were reasonable and well thought

- out. Because some subjectivity was involved in the final model selections, it is hard for me to tell whether each detailed review and scrutiny reported in Section 7.3 is “opt imal”. On the other hand, there is no single model that is likely to be the best . In reality, it is generally the case that several models are (almost) equally good given the limited amount of data, and the final selection can be made with some subjectivit y. The analyses given in Section 7.3 appear reasonable.

4. Use of the Tobit regression (SAS PROC LIFEREG) for modeling datasets with large numbers of censored
valuesPage 62

I think that the use of left

- censored models and the Tobit regression is appropriate. My only question here is why the same approach is not taken for cases with light censoring. See my comments under

Item #2.

Additional suggestions I have some additional suggestions, some of which might be useful for future studies. First, if a similar

s tudy is planned in the future, it would be better to construct specific criteria for removing outliers prior to data collection. This would eliminate questions about biased interference in the data processing stage. Second, when imputation is used, more ca reful procedures should be in place to account for variability due to imputation. Multiple imputation is a common approach to take. Third, some sensitivity analysis using robust statistical methods can be performed to understand the effects of outlying/inf luential points and their handling on the final analysis. Most statistical techniques, including linear mixed models and the Tobit regression used in this study, are based on the assumption of Gaussian errors. Robust statistical methods can help us underst and the impact of non

Gaussian errors.Page 63

January 20, 2012



## MEMORANDUM

To : Brian Menard

From: Chris Lindhjem, ENVIRON International Subject: Review o f “Assessing the Effect o f Five Gasoline Properties o n Exhaust Emissions

from Light‐ D u t y Vehicles certif i e d to Tier‐2 Standards (EPAct /V 2/E‐89: P h ase 3)

Part II: Data Analysis and Model Development” DRAFT RE P O RT December, 2011

INTRODUCTION The report, “Assessing the Effect o f Five Gasoline Properties o n Exhaust Emissions from Light‐ Duty Vehicles certif i e d t o Tier‐2 St a ndards (EPAct/V2/E‐89: Phase 3) Part II: Data Analysis and Model Development DRAFT REPORT” presents a well‐documented appr o ach to estimating the
effect that g asoline fuel properties h a v e o n late model vehicle emissions. In general, the approach t o evaluating the data is equal to o r m o r e robust t h a n previous efforts, s u ch as the complex a n d predictive models o r other fuel ef f e cts studies.

The fuel properties in testing matrix used in the evaluation are in two respects quite different
from previous studies. The higher levels o f ethanol, up to 20%, are beyond what an y previous study has considered. Also, the lack o f olefins evaluation data elimina t ed one fuel parameter
that was found to have a significant effect o n emissions in previous st u dies.

One issue to be determined is how E PA o r others will choose to extend the fuel property relationships developed in this report to the general fleet, such as in MOVES. Because vehicles will naturally age and may respond differently t o fuel properties, these re lationships may not continue to hold true. These higher emitting vehicles could potentially contribute to the

emissions inventories out o f pro p ortion to their numbers. Ca r e should be taken when extending
the fuel effects to other vehicles, w h ether a g ed lat e model light‐duty o r heav y ‐duty gasoline powered vehicles.

OVERALL REVI E W The approach to model i ng is well considered given the low emissions rates o f these vehicles and
the relatively small fu e l effects, often below normal detection limit s

. It was appar ent that the

iterati v e process using not only novel statis t ical techniques (compa r e d with other fuel effects evaluations), but an understanding o f th e testing limitations (low emission rates coupled with detection limits) was a necessary method to determin e relevant f u el para m e ters to include in the
evaluation.I agree with the approach o f limiting the number o f statistically fit terms (especially t h e

second order term s ) to only those that assist in explaining the fuel effects. Perhaps the
discussion of th e magnitude o f the residuals could further highlight the lack o f impact o f terms
that have been dropped from the correlations such as in Figure 70, wher e a trend may exist,
but is relative l y small in ma g nitude.

The overall statistical a p proach is sound us ing s e veral meth o ds to discover and systematically
elimina t e terms. The approach to identify influential data used appropriate physical (investigate detection limits and o t her laboratory variables) and statistical (significance level, BIC, and residual eva luation) me t hods to eliminate o r keep data.

SPECIFIC COMMEN T S

I was unable to find the earl i e r r e ports “EPAct/V2/ E ‐89” referenced in the document, presumably “Assessing the Effect o f Five Gasoline Properties o n Exhaust Emissions from Light‐ Duty Vehicles certif i e d t o Tier‐2 St a ndards: Part I ‐ Study D e sign and Execution (EPAct/V2/E

‐ 89 Test Progr a m Final Re po rt)”. This report likely d e s c ri b e s in better detail why
the fuel blending program could not produce an orthogonal fuel matrix. Likewise, questions
abou t t he measurement, vehicle conditioning, and other issues are not addressed in Part II.
Th i s makes it difficult to assess the relevance o f the modeling.

For examp l e, this statement o n Page 36 (“ The alterna t e measure “N M HC as measured by FID” (NMH C

F I D ), wa s collected for the entire dataset, and it very tightly corre la ted with bo t h NMOG
and “tr u e” NMHC. It is thus possible to estimate NMOG and NMHC resu l ts for tests without speciation b y using correlations genera t e d from those with speciation. This technique essentially estimates the offset bet w een th e response of th e FID and the fully characteri z e d

emission s t ream, due to the incomplete measurement of ox y gena t es by the FID. For NMOG, this
estimated v alue is typically between 2‐20% higher than th e NMHC FID m e a suremen t

, depending

on emiss i on bag and fuel ethanol l e v e l. ” ) re f e rs t o an apparently unique measurement NMHC FID and data handling approach, and it would be useful to understand this measurement in order

to understand the validity o f this statement. The co rrelation (descri b ed i n section 3.2 Imputation o f Speciated Hydrocarb o ns (NMOG, NMHC)) o f NMHC FID to NMOG and NMHC
appears to indicate that fuel ethanol level has no effect o n this correlat i on (same slope for all
levels o f ethanol, Tables 10, 1 1 & 13, 14) for Bag 2 and 3, but an ethanol slope term for Bag 1. Yet, the cor r elations o f ac etaldehyde, formaldehyde, and et h anol in Tab l e 82 for Bag 2
demonstrate ethanol still increa s e s these oxyge n ated species and so sh o u ld increa s e NMOG (consisting usually o f NMH C + oxyge n ated carbo n ). Perhaps there is something unique about
this NMHC FID measure m ent that allows oxygenated species t o be measured at some level. However, t h e approach t o correlati n g NMOG an d NMHC with NMHC FID appears inconsistent for Bags 2 and 3 comp ared t o Bag 1.

EDITORIAL SUGGESTIONS

There are num e r o us editorial corrections that nee d to be made, but most have already been
noted in the document itself o r are obvio u s from WORD program review. What fol l ows are specific suggestions either for improving the flow o r for feedback that would not otherwisePage 65

be considered during normal editing. The original line is provided in “quo t ed italics” a n d the suggestions, questions, o r comments appear in suggested e d its (striketh r ough o r red) o r flat

text.

Page 9

St atement “ The analysis involved o n going and i t erati v e interaction bet w een stat i sti c al modeling and additional physical and chemical review of the data

. ” Page 23 “The final design is
the result of an itera t ive process involving in t e ractions between r e s e a r c h goals, th e feasib i lity of fuel blending, and e xp er imental design.”

Comment The term “in t erac t ion” should refer only to statistically fitted second order terms where fuel parame t e rs are mixed, s u c h as ZZ e a , to represent the second order etha no l x

aromatic s term. The statement above appears to use “interaction” in a different con t ext and so is confusing. EPA should also search all other uses o f “interaction” to ensure that there is no
confusion.

Statement “ The models rep o rted in this section a r e as parsi m o nious concise as the data and subject‐matter kno w ledge allow

. ”

Page 13

Statement “ Ethano l : taken in iso l at i on, the models indica t e that increas i ng ethanol is
associated with increases in all em i ssions, b o t h for cold‐start and hot‐running emiss i ons. The sole exception to the pattern is CO, for which the response to added e t han ol is lower CO
emissions du r ing start but inconclus i ve for ru n n i ng conditions. fuel properties appears to change
between start and runn i ng. ”

Statement “ Note that this generalization does not account for the ef f e ct of interactions
between R V

P and other pro p ert i es, which are in some cases larger than t he underlying linear effects

. ”

Comment I see only two RVP interacting terms affecting only running THC (not NMOG or NMHC) and start CO, and this statement on ly appears to be true for the CO start emissions. I would sugg e st either stating this more plainly o r striking the comment. O r does this line refer to fuel proper t ies that are affected when RVP is modified? For examp l e, reduced T5 0 o r diluted

aromatics occ ur when lighter compounds are added to increase RVP; t h en this statement would be true for the other fuel properties than just for RVP.

Page 22

5 “ This parameter was measured a s DVPE, but f or simplici t

y and consistency, we will refer to it
as “RVP. ” ”Page 66

Pag e 181

Statement “ 14 The typic a l hydrocarbon analyzer us e d for emi s sion testing uses a flame

ionization d e t e ctor (FID), which is calibrated to accurately count carbon atoms that are bonded to hydrogen. Carbons bonded t o oxygen, which occur in carbonyl and a l cohol

emiss i ons from burning ethanol fuels, a r e not accurately counted by the FID, and thus

emissions fro m ethanol fuels requ i re additional c haracteriza t ion methods to properly quantify

as NMOG or VOC.”

Comment This statement is generally, but not s t ric tly, acc ur ate in that most FID un i ts are calibra t e d o n propane, and the hydrocarbon measure m ent assumes that all carbons in the sample res p ond the same as the ca r bon atoms i n propane. Hydrocarbons do n o t n e cessarily respond identically as propane carbon at oms do, but carbons bound to oxygen respond at a
rate order(s) o f magnitude lower. The error for most hydrocarbons has been considered insignificant. The respo n se to oxygenate carbons has been historically co n sidered to be insignificant compared with other hydrocarbons in the sample, so carbonyl and alcohol compounds determined through alternative me t hods are added to the NMHC weight measure as the weight o f single carbon aldehyde (form a ldehyde) a n d alcohol (methanol). The different composition o f hydrocarb ons and an y FID response to oxygen‐bound carbon
influence the NMHC mea s urement, bu t those influences are c o nsidered m i nor. With lower

emissi o n rates of
these newer vehicles, t h ese assumptions may not be as appropriate as for older vehicle designs with higher emission rates. In addit i on, does the ‘typical hydrocarbon analyzer’ differ
from the NMHC FID measurement th at needs to be correla t e d with NMHC before th e statistically modeling proceeds?

Page 182

“8.2.2.4 Vapor Pressure”

Comment I f o und this section description (and others that rationalize the effect mo d e led) more

o f an unsat i sfying hand waving exe r cise to justify what the data was telling. If indeed the older
studies and this most recent evalua t ion are to b e belie v e d, then the modeled ef f e ct may be
temperature dependent when at high temperature the fee d back o f vapors to the e ngine may affect the emissions opposite to that modeled here. The suggestion is t o limit the speculation
to the results o f the analys i s presented and note o t her re f ere nc e s if the effect found needs to
be jus tified.Page 67

Brian West, ORNL, westbh@ornl.gov 865

- 946

- 1231 January 2012

Review of “ Assessing the Effect of Five Gasoline Properties on Exhaust Emissions from Light

- Duty Vehicles certified to Tier

- 2 Standards (EPAct/V2/E

- 89: Phase 3)

- Part II: Data Analysis and Model Development

- DRAFT REPORT (December 2011)”

The report is long and contains tremendous detail regarding the statistics and modeling approaches used in analysis of the EPAct/V2 data from the SwRI program. Considering its length and the technical detail, the report is well written and well organized. Several recommended edits and comments are provided in the marked up report, to correct typos or improve clarity. As an example, many figures could use larger fonts before final publication. Several ci ted references (especially Part I of the same report and the appendices) appear unavailable at present, and there are a few missing figures and references. But overall it is a very good draft.

This reviewer has over 20 years experience in engine, emission s, and vehicle testing, but limited experience with many of the statistical methods and modeling approaches described. To the extent possible in the short time available, methods and terms were researched and explored while reviewing the EPA document. Expl anations and approaches appear largely reasonable. Some specific questions are noted in comments in the report. For example, the Bayesian Information Criterion (BIC) is used to determine goodness of fit. In one example, the BIC for the full model starts ou t at >2900 and decreases by 0.1

- 0.2% for subsequent reduced models. The discussion refers to “a steady decrease” in BIC. The reviewer suggests the significance of such a small change in BIC be discussed further. While the reduction in BIC appears insignifi cant , the fact that BIC is not increasing is perhaps the justification to use the reduced model (sec. 5.3.2).

The authors appear to have taken great care to ensure that models are as representative as possible, and to avoid overly complex models or overfi tting. The handling of “nondetects” appears reasonable. There is rigorous treatment and discussion surrounding background measurements, analyzer drift, limits of quantitation, censoring of data, etc. Nonetheless, the complexity of the problem makes it diff icult to see whether all of the objectives were achieved or rather that some of the apparent results are any more than artifacts of an intricate math problem. As an example, all emissions data are shown by vehicle and include all tests on all fuels, to sho w the range of measurements for a given pollutant, by vehicle. These are very informative charts. However, no data are shown to demonstrate the test

- to

- test repeatability for a given vehicle with a given fuel. Several figures (such as Figures 13

- 21) show a verages, but would be more informative if range bars were shown to indicate max and min or perhaps interquartile range, or perhaps scatterplots with all data points.

Authors discuss taking care to not overfit or model random scatter (which is good), in fa ct stating that the Bag 3 models may be more vulnerable to measurement error due to the extremely low emissions, and opting not to report model coefficients for Bag 3 results. The reviewer agrees with and commends this decision. Handling of extremelyPage 68

low m easurements is difficult, and the authors discuss this issue extensively and convincingly. However, when it comes to the PM emissions measurement, there could be additional discussion and justification. For instance, one issue that deserves additional disc ussion is the lack of tunnel blanks for background PM. Data from a few example tunnel blanks could provide convincing evidence of the zero background assumption. Furthermore, control data should be shown, or at least cited, to support the assumed level of error (± 1 μg) in the PM filter weight measurements. The authors mention discussions with EPA staff as the basis for the assumed level of error. As the authors know, PM measurement is very sensitive to measurement precision and accuracy, with temperature, humidity, buoyancy effects, static charge and other factors greatly influencing results. In SAE 2005

- 01

- 0193, the authors detail exhaustive measures to attain an accuracy of better than ± 1 μg in their filter weights. Please provide evidence of the stated PM measurement error.

In Section 7, mean residuals are plotted, presumably to demonstrate the quality of the models. It would be interesting to see range bars on the data points to show the range of individual residuals.

Models should be validated agains t independent datasets, as discussed in section 8.

Some specific questions were provided for consideration during the review:

1. Was the process of imputation of NMOG/NMHC results for tests/bags with missing speciation data reasonable and statistically s ound? (Section X.X) Estimating NMOG from available NMHC appears reasonable and is consistent with what was done in the DOE V4 program (ORNL/TM

- 2011/234 and ORNL/TM

- 2011/461). For a given ethanol level, NMOG emissions have been shown to be a linear function of NMHC emissions.

2. Was the decision to remove very low emitting and influential vehicles from the NOx and NMOG analyses reasonable? (Sections 5.5 and 6.1). These discussions were largely reasonable and convincing; however one case (run 6281) was not e xplained.

3. Please comment on the use of the “design set” of 11 terms as the basis of the final models, versus allowing model to fit all 17 terms, including the adequacy of the justification of this decision. (Section 7). This approach seems reasonable b ased on the discussion.

4. Comment on the use of the Tobit regression (SAS PROC LIFEREG) for modeling datasets with large numbers of censored values. (Section 5.3). This approach seems reasonable, although I would like to see “large” and “small” censoring levels explained further. Censoring of less than 5 values is considered small. Why? With over 900 datapoints, could the limit be set at a higher number? Please explain.

5. Comment on the decision to independently model the linear terms and interactions b etween fuel blends as presented in the final results? This approach seems reasonable based on the discussion.Page 69

6. Please comment on the methods used to select reduced models. (Section 5). This discussion was convincing, although I am not an expert in this a rea. I understand that reduced models have lower likelihood of the models describing the random error rather than the underlying fuel effects.
Brian West

Additional comments on the Draft Report

( p.8 ) The program was conducted in three phases. Phases 1 an d 2 were pilot efforts involving measurements on 19 light

- duty cars and trucks on three fuels, at two temperatures. This work was completed at Southwest Research Institute between September 2007 and January 2009. Have these results been published?

(p.8) An initial sample of 19 test vehicles was chosen with the intent of representing of the latest

technology light duty vehicles being sold at the time the program was being launched (model year 2008).

( p.9 ) Speciation also allowed independent analyses of s elected toxics including acetaldehyde, formaldehyde, acrolein, benzene and 1,3

- butadiene.

(p.9) This approach was followed for several reasons: (1) the candidate fuel effected identified for study were selected because we anticipated that they could be important for one or more emissions. Not clear. Candidate fuels? Or candidate fuel effects?

( p. 10 ) In generation of emissions, the effects of different fuel properties are not separable, in that it is difficult to modify one property without affecting one or more of the others.

( p. 10 ) However, the coefficients for different fuel properties can be directly compared, allowing assessment of the relative importance of the effects of the fuel properties on the emissions constituent being modeled.

( p. 1 3 ) Ethanol : taken in isolation, the models indicate that increasing ethanol is associated with increases in all emissions, both for cold

- start and hot

- running emissions. The sole exception to the pattern is CO, for which the response to fuel properties a ppears to change between start and running. The effects are strongest for PM, NOx and NMOG, although presumably, the underlying physical processes could vary. Interestingly not consistent with V1 (ORNL/TM

- 2008/117) or V4 (ORNL/TM

2008/234). Increasing et hanol (in splash blends with certification gasoline) decreased NMHC and THC (FID_HC), while NMOG was relatively flat. Ethanol and acetaldehyde increased, of course. V1 used LA92 but V4 used FTp.

( p. 13 ) Cold

- start (Bag 1 ) CO and hot

- running THC both hav e small decreases in emissions with increasing aromatics. In terms of magnitude, the pattern is similar to ethanol, with PM, NOx and NMOG showing the strongest effects.

( p. 16 ) Since data on Tier 2 vehicles are critical to understanding the impact of fue l property changes on the onroad vehicle fleet as increasing volumes of biofuels are introduced , EPA entered a partnership
with DOE and CRC to undertake the largest fuels research program conducted since the Auto/OilPage 70

program in the early 1990s. This progr am is aimed specifically at understanding the effects of fuel property changes on regulated and selected unregulated exhaust emissions from later technology Tier 2
vehicles.

( p. 16 ) This report describes the analysis of the dataset collected in Phase 3 of the EPAct/V2/E

- 89 program, conducted at Southwest Research Institute in San Antonio, Texas. A separate report describing the program design and data collection activities is available, but an overview is provided
below. This report does not appear to be publicly available at this time (January 2012)

( p. 17 ) These five parameters were selected based on previous studies on older vehicles as having potential to affect exhaust emissions [cite]. Missing reference.

( p. 17 ) The parameter ranges to be covered for T50, T90, aromatic content, and RVP were selected to represent the range of in

- use fuels based on a review of the Alliance of Automobile Manufacturers’ 2006 North American Fuel Survey [cite?]. Missing reference.

( p. 17 ) Test fuel parameter ranges we re originally drafted to span roughly the 5 th to 95 th percentiles of survey results for in U.S. gasoline, though some test fuel parameters were adjusted after the actual blending process began.

( p. 18 ) An initial sample of 19 test vehicles was chosen with the intent of representing of the latest

technology light duty vehicles being sold at the time the program was being launched (model year 2008). In terms of regulatory standards, the test sample was to conform on average to Tier 2 Bin 5
exhaust levels an d employ a variety of emission control technologies, to be achieved by including a range of vehicle sizes and manufacturers. I recall EPA staff indicating (c. 2007) that the list of vehicles was a projection of future technology and/or engine families

( p . 20 ) After some consideration, study participants agreed to rely on the aggregate data, while applying appropriate techniques do to address the resulting “censoring” of the data at low end of the range of
values. Who are the “study participants?”

( p. 21 ) The methods used were very similar to those used for the modeling of the other emissions, with modifications to address issues of study design and measurement specific to these compounds.

( p. 22 ) The design and implementation of the study, including the aspects of fuel blending, measurement methods and logistics are described in a separate report4. Error! Bookmark not defined. Missing reference.

( p. 22 ) It is well known for fuel properties to be moderately to strongly correlated.

( p. 22 ) 1 This parameter was measured as DVPE, but f or simplicity and consistency, we will refer to it as
“RVP.”

( p. 25 ) Measurement methods are discussed in detail in the testing report4. Missing reference.

( p. 27 ) In addition to correlations among the linear effects and interactions, and correlations among interactions, we can see one fairly strong correlation s among the linear effects, specifically, between etOH and T50 ( R =

- 0.57).Page 71

( p. 31 ) As one

- stage standardization did not n eutralize correlations among model terms, we applied a second stage of standardization to the 2 nd order terms. This report does not appear to be publicly available at this time (Jan 2012)

( p. 32 ) Table 8 shows that the combination of one

- and two

- stage s tandardization neutralizes the remaining correlations, with the exception that between the etOH and T50 linear effects, as previously
described. Unclear “that between the etOH and T50 linear effects [what]”

( p.33 ) In this constant volume sampling system, the vehicle exhaust is mixed with a large amount of filtered dilution air, and a small portion of this stream is continuously withdrawn to fill a sealed bag over the course of a test cycle.

( p. 33 ) Its primary disadvantage is that the overall dilution ra tio of background air to exhaust must be fixed for an entire test, and is set relatively high to avoid condensation of water vapor within the system during periods of high exhaust flow.

( p. 35 ) 1 This assessment of the situation ignores the possibility th at the vehicle actually consumes or destroys a given pollutant species during parts of the test cycle, resulting in periods of “negative emissions”, such that the average emission level over a test is truly zero. While situation s may occur over a limited period for some emissions in a highly polluted environment, e.g., PM or NMHC in congested traffic, it is highly unlikely in an emission test cell.

( p. 36 ) For these reasons, the decision was made not to replace zeros in the dilute bag dataset with integra ted continuous measurements. Agree with this decision.

( p. 36 ) Rather, these measurements to were treated as “censored.”

( p. 36 ) The alternate measure “NMHC as measured by FID” (NMHC FID ), was collected for the entire dataset, and it very tight e ly correl ated with both NMOG and “true” NMHC.

( pp. 36

- 37 ) Based on strong correlations between these species, we developed statistical models to impute NMOG and NMHC from corresponding NMHC FID measurements. See ORNL/TM

- 2011

- 461

( p. 37 ) On this basis, we fit line ar models for NMOG and NMHC in terms of NMHC FID .

( p. 37 ) Scatterplots of NMOG vs. NMHC FID for Bag 2 are presented in Formatting.

( p. 38 ) For the bag 2 and 3 models, the counterpart to Equation 8 is Equation 11, which simplifies for
blends other th an E0 s imilarly to Equation 8, except that the slope term is always as in Equation 9.

( p. 50 ) At the outset, it is helpful to get an overview of the raw results, sorted by vehicle and fuel, which gives an initial impression of variability among vehicles and fuel s, as well as within vehicles. Agreed. Also would be helpful to show variability of individual vehicles on individual fuels.

( p. 51 ) A linear

- effects plot for ethanol is shown in Figure 13, which suggests that an ethanol effect is
visible when the data is are averaged across the other four fuel properties.Page 72

( p. 51 ) Trends for individual vehicles show a general increase in NOx with increasing ethanol, with some exceptions. Consistent with prior studies ORNL/TM

- 2011/234; SAE 2009

- 01

- 2723

( p. 52 ) It is ne cessary to go a step further, and look at “interaction” or “conditional effects” plots, starting with the interaction of ethanol and T50, which deserves special attention because....

(p. 52) In the plot for etOH×T50 (Figure 18), the view seems to indicate a n upward trend from E0 through E20, but with some downward curvature above E10. Show all data? How much scatter is there in NOx for these cases?

( p. 52 ) At first glance, the trend appears to “zig

- zag,” from low to high. Test to test variation of Bag 1 NOx in V1 study (LA92 cycle) ranged from 10% to over 100% (same veh, same fuel). In V4 program (FTP, not LA92), range of weighted composite NOx approached 100% in some cases. How do you ensure that coincidental test

- to

- test variation is not erroneously at tributed to a fuel effect?

( p. 64 ) As in Bag 1, variability for most individual vehicles spans one third to half an order of magnitude. Missing reference?

( p. 64 ) The effects for T50 and T90 are similar in that the trends across the five T50 levels and t hree T90 levels are similar, with no overall trend apparent, for the same reasons noted above for Bag 1 (Figure 26, Figure 27). Missing reference?

( p. 76 ) The variability within vehicles is about 1

- 1.25 orders of magnitude. Missing reference?

( p. 76 ) T he Linear Effects plot for ethanol shows some mixed results (Figure 33), but with an apparent increase from 0% to 10% ethanol, followed by a leveling or decline at higher ethanol levels. Decline would be expectation. Increase from E0 to E10 is odd

( p. 7 7 ) The Linear Effects plot for T90 is clearly suggestive of an overall positive effect, when considering all vehicles (Figure 37).

( p. 77 ) The left

- hand point represents the same fuel as the left

- hand point in the green trend in the previous plot.

( p. 77 ) In contrast, the plots for ethanol and aromatics are suggestive of a positive or “reinforcement” interaction. In the etOH × arom view (Figure 40), the trend for the higher aromatics level (green) appears steeper than for the lower aromatics level (bla ck). Similarly, in the arom × etOH view ( Font.

( p. 77 ) In Figure 41.5, the etOH × T90 plot does not appear to suggest interaction, if we discount the green trend (T90=325°) as representing only two fuels. Missing figures

( p. 91 ) We assume that a very s mall but positive measurement existed but was not captured and quantified. Assigning a value of zero to these observations is an example of a common approach to censoring of observations, known as “substitution.” In this approach, a small but fixed quant ity is substituted for the censored observations. Values used for substitution include zero, as mentioned, or small but positive quantities such as the smallest observation, a multiple of the smallest observation, the limit of quantitation (LOQ) or half t he limit of quantitation (LOQ/2). The degree of censoring variedPage 73

widely by emission and bag, as shown in Table 2 . At different stages of the analysis, we addressed censoring in different ways. These various app roaches all make sense. Suggest adding couple of sentences about the various bags of the LA92. Bag 1 is cold with majority of emissions, hence less censored values. Bag 2 is hot, but includes some open

- loop operation due to hard acceleration. Bag 3
is hot start bag with very low emissions, hence majority of censored values.

( p. 92 ) Table 2 5. Numbers of Censored Measurements, by Emission and Bag. Suggest showing total number of tests (in title or footnote).

( p. 93 ) In this ini tial step, censored measurements were replaced with the minimum positive value measured for the emission and bag. Minimum across all vehicles and fuels?

( p. 94 ) As a measure of influence, we calculated the externally studentized or “studentized

- deleted” residual ( r

- i ). Divided by an estimate

( p. 94 ) Table 17. Counts of Influential Measurements, by Emission and Bag (with “influential” defined as having a studentized

- deleted residual ≥ 3.5 or ≤

- 3.5). Suggest including total number of measurements

or o bservations

( p. 95 ) An additional measurement in Bag 3 (run 6281) was removed, even though it was not flagged as
influential. Explain further?

( p. 95 ) The full sets of terms in the optimized design include terms anticipated to be meaningful for any
of t he emissions to be measured. However, it was not anticipated that all the terms included would necessarily be meaningful for all emissions in all bags. A closely related goal is to develop models that would be, to the extent possible, explicable in terms of knowledge of the relevant physical and chemical processes. Parsimonious models are preferred over full models for this purpose, as their simpler structure makes their behavior easier to assess and explain. Finally, with respect to explicability, it is much preferred to minimize the potential for overfitting, which could reduce the generality of models selected for prediction. To guide the process, we adopted several assumptions, described below.
Model describes the random error rather than the desire d underlying relationship. Good discussion.

( p. 96 ) For minimal levels of censoring, defined as five or fewer censored measurements ( n censored ≤ 5), we elected to substitute the minimum positive measured value for the missing measurements. After
substit ution we fit mixed models as described above. N<5 seems reasonable if Total number of samples is large. Suggest noting Ntotal here. Why 5? That is, why not 4 or 6 or 10? Is 5 arbitrary or is there a citable reference to why 5?

( p. 99 ) At each step, w e tested the goodness

- of

- fit of each reduced model against that of the full model using a likelihood

- ratio test.

( p. 100 ) In this case, the BIC declines steadily as terms are removed, indicating an improvement in fit for
each successive reduced model. D eclining steadily by 0.1 to 0.2% does not seem significant or important. Is the improvement cited below truly due to the simpler model? Perhaps, but is the BIC really an accurate indicator?

( p. 101 ) Table 20. Model Fitting History for PM, Bag 1 (FM9 se lected as best

- fit model). Is the sensitivity of BIC such that there is a significant difference between 2862 and 2867? Six significant figures andPage 74

0.1% change in BIC does not seem important. If this is important or significant, it should be explained. Perhaps the important point is that BIC is NOT INCREASING as the model is simplified, thus justifying the simpler model?

( p. 102 ) All models based on the 16

- parameter full model, as shown in Equation 14 are.....

( p. 107 ) The approach to analysis of censore d measurements, as described in 5.3.2, was also adopted based on guidance from the author of the DOE research. No public documents can be found on the EPAct/V2 study.

( p. 107 ) Thus, in running these models, censored values were replaced in with the minim um positive measured value in each bag for each emission.

( p. 116 ) This observation led to a closer examination of measurement error in the dataset.

( p. 116 ) If this program were simply trying to quan t ify the magnitude of NOx emissions from such
vehicles , this level of error may be acceptable.

( p. 116 ) However, since we are looking for meaningful differences in emissions between fuels, this large relative error is particularly problematic. Key point. Meaningful differences.

( p. 117 ) This condition w ould be expected to give a zero result as discussed in Section 3.1.

( p. 117 ) This suggests it likely has higher measurement noise than data from the other vehicles, and thus many measurements may not be reliably distinguishable from background levels. Go od

( p.118 ) The data for the Sienna, though similar in their range of sample and background measurements,

was were not found to be exceptionally influential to model

- fitting and therefore was were not removed
from the dataset.

( p. 120 ) After the test is c onducted, the filters are removed, placed back into the clean containers

, and
returned to the clean room.

( p. 120 ) In this program, dilution air was HEPA

- filtered and presumed to be free of PM, so there was no background filter sample collected for later subtraction as is typical with other emissions. Why no tunnel blanks to prove this assumption?

( p. 120 ) Discussion with EPA staff experienced with PM measurement suggests that for the data as collected in this program a variability of ±1 μg should be applied to all filter weights. Considering that the net PM result is calculated by subtracting two filter w eights (average dirty minus average clean), it should be understood to have a variability range of ±2 μg, as the measurement error applies to both weights. Therefore, a net weight gain of 10 μg would have a relative error of 20% associated with it, a figu re of the same order of magnitude as the fuel effects this program attempts to capture. Need to cite reference(s) or present data to establish this level of error. For example 2005

- 01

- 0193.

( p. 124 ) Figure 5 6 4 and Figure 5 7 5 show net measurements as a p ercentage of sample for bags 2 and 3, as an attempt to understand the magnitude of the nets relative to the measurement error where nets
are small.Page 75

( p. 124 ) Two vehicles (Odyssey, Sienna) have the majority of points below those of other vehicles.

( p. 124 ) Plots of ambient and sample for bags 1

- 3 2 are shown in Figure 58, Figure 59, and Figure 60, followed by examination of nets as percentage of sample for bags 2 and 3 in Figure 61 and Figure 62.

( p. 138 ) At this point it is important to note that the resul ts reported below differ from those reported above in 5.3, as well as from those reported for the DOE analysis8. Missing reference

( p. 149 ) Interaction plots for selected terms are shown in Figure

. Y axis titl e is “mean measurement.” Isn’t this a modeled result, not a measured result?

( p. 149 ) It is interesting to note that while (d) and (f) appear somewhat similar visually, the model considers the aromxT90 interaction highly significant but the RVP×T90 int eraction insignificant. Explain

( p. 149 ) Another way of viewing the interactions is to average and plot the residuals of the linear effects
model. Compare modeled results to measured results. Mean is good, but would also be nice to see the range of res iduals. Mean can be close to zero, but how much variation is there? (average of + 20
and

- 20 is zero).

( p. 150 ) < insert physical interpretation of these interactions here?> Yes

( p. 151 ) Figure 65. ln(CO) (Bag 1): Two

- way Conditional Effects Plots for Three Interactions and one Quadratic term, viewed with respect to Both fuel Parameters : (a) Ethanol×Aromatics, (b) Aromatics×Ethanol, (c) Aromatics×T90, (d) T90×Aromatics, (e) RVP×T90, (f) T90×RVP, (g) etOH×etOH. Typo in fig c and e. T90 blue should be 300

( p. 153 ) Figure 66. CO (Bag 1): Mean Residuals for the Linear Effects Model, vs. Target Fuel Properties for three Interactions: (a) Ethanol × Aromatics, (b) Aromatics × T90, and (c) RVP × T90, and the quadratic

term etOH×etOH. Blue T90=300

( p. 158 ) Another possibility is that the effect of T50 in this presentation is masked by variation in RVP
and T90 levels across these fuels.

( p. 161 ) Figure 69. NOx (Bag 1): Mean Residuals for the Linear Effects Model, vs. Target Fuel Properties for four pairs o f terms: (a) Ethanol × Aromatics, (b) Aromatics ethanol, (c) Aromatics × T90, (d) T90×Aromatics, (e) RVP × T90, (f) T90 × RVP, (g) ethanol × T50, (h) T50 × etOH. Same comment as above. Mean residuals look good. What is range of individual residuals? Pe rhaps add error bars to show min and max or perhaps interquartile range of residuals?

( p. 164 ) The relations among the ethanol, aromatics and RVP coefficients are similar to Bag 1 and to each other, except that the aromatics coefficient is slightly higher than the ethanol coefficient for NMOG, while ethanol is more important to than aromatics for NMHC.

( p. 164 ) As in Bag 1, the etOH x arom and etOH x T50 interactions are reinforcements, whereas the

etOH x RVP interaction is an interference (etOH positive, RVP n egative, interaction negative).Page 76

( p. 165 ) One is that the magnitudes of corresponding coefficients are generally larger for Bag 1 than for Bag 2 emissions, suggesting that the effects of fuel properties are more pronounced for “cold start” than
for “hot ru nning” emissions. As expected!

( p. 169 ) When starting with the 16

- term model, the reduced model retains includes RVP and T90 linear terms (both insignificant), plus two interactions not included in the design model: arom×T90 and

RVP×T90.

( p. 169 ) The re asons for these differences are not apparent, but it is clear that the relations among NOx, etOH and T50 are complex Or nonexistent?

( p. 169 ) It may be appropriate to consider whether the Bag 3 results may be more vulnerable to
measurement error attribu table to low sample measurements relative to background, given the issues with measurement discussed in 6.1.1 (page 116). Yes

( p. 176 ) Ethanol : taken in isolation, the models indicate that increasing ethanol is associated with

increase in all emissions , both in bags 1 and 2.

( p. 178 ) This question does not arise in the context of model application, but rather with respect to model validation, in that the datasets available for validation include data that represent emissions from pre Tier

- 2 vehicles. Such as the DOE V1 and/or V4 datasets? Have the models been compared/validated against V1 or V4 data? Note that V1 used splash blends and ran the LA92 cycle. V4 tests also used splash blends but ran the FTP.

( p. 180 ) The results of the present study ar e consistent, showing an increase in NOx emissions with an increase in ethanol level, regardless of whether it is from a statistical analysis of an orthogonal change in ethanol alone or when changes in other fuel properties typical of splash or match blend ing are included.

( pp. 180

- 81 ) Thus, despite much lower overall emission levels that have been achieved in Tier 2 vehicles through improved fuel control and catalyst efficiency, the effect of ethanol on combustion and aftertreatment(?) appears to persist in certain modes of operation such as cold

- starts and transients during warmed

- up operation. Effect of ethanol on NOx emissions may be related to exhaust stoichiometry and catalyst efficiency more so than (or in addition to) changes in engine

- out NOx.

( p. 181 ) [Should we address effect on THC/NMHC as well?] Yes. Note that NMOG in V4 was not affected by ethanol, but FID_HC and NMHC decreased (with splash blends).

( p. 181 ) In the present study NMOG decreased with decreasing aromatics content, in agreem ent with
earlier studies. Or increased with increasing aromatics...

( p. 182 ) The present study shows little or no effect of T90 on NOx, which is reasonable considering that NOx production and control are largely about heat release, and T90 represents a sma ller amount of
combustible material at cold

- start temperatures, and thus less energy, than T50. Not clear, please review/reword.

( p. 182 ) However, during cold start, the new data suggests the direction of the effect is dependent on
ethanol content.Page 77

( p. 183 ) Consider showing quantitative parameter changes and percent change results for the models? )
Good idea

( p. 184 ) More variation is seen in T50 and RVP, which span somewhat wider ranges of 160

- 195 °F and

7.3

- 13.0 lb psi , respectively.

( p. 184 ) Use of the non

- standardized models is much more straightforward, in that they allow input of fuel properties in their original units, e.g., % for ethanol and aromatics, lb psi for RVP and °F for T50 and

T90.

( p. 185 ) The processes and methods for hydrocarbon spe ciation are described in greater detail in the testing report4. Missing reference

( p. 187 ) The “ G

- efficiency” for the full design was estimated at 51.6% for the eleven design parameters, as previously described in 2.1 2.1 above (page 22).

( p. 187 ) As sh own in the table, the design efficiency drops sharply with inclusion of 2 nd order terms, to less than 5% for designs four and five.

( p. 190 ) In addition to estimating the sample measurements, as shown above, we estimated two variances, the first being the variance of the variance of the 5

- day moving average of the media blanks
( ), and the second being a variance of random errors ( ).

( p. 193 ) The first two plots (Figure 8 1 2 ) show acetaldehyde vs. ethanol, by T50 level, in linear and logarithmic space.

( p. 199 ) With respect to censoring, the following rule was applied. If the number of censored measurements was ≤ 5, we substituted the smallest measured positive value for the missing values, and proceeded with mod el fitting, using a mixed

- model approach. Why 5? Why not 4 or 6 or 10? Explain.

(p. 199) However, if the number of censored measurements was > 5, we fit a model using Tobit regression (i.e., “censored normal regression”), an established technique for a nalysis of left

- censored
datasets. Ditto. What is significance of 5?

( p. 199 ) For compounds affected by media contamination, we integrated the approach to censoring with an “Estimated Dependent Variable Model,” an approach to modeling datasets with meas urement uncertainty in the response variable. Ref 20

( p. 202 ) If the p

- value for the test

- against

- previous is greater than the critical value, the null hypothesis of no significant differen ce t in fit between the reference and nested models is retained, a nd the nested
model is retained as the current best fit.

( p. 203 ) Based on these results, the reduced model FM7 was selected as the best fit. As noted in previous BIC discussion, do small variations in BIC truly indicate a difference in fit? FM5, FM6, a nd
FM7 all have similar BIC.Page 78

( p. 208 ) Finally, to illustrate the results of the jackknife replication procedure, Figure 83 shows cumulative distributions of coefficients for each jackknife replicate for the five linear linear e E ffects.

( p. 208 ) The distr ibutions for aromatics ( Z a ),T50 ( Z

5 ), and T90 ( Z

9 ) are similar in that they show noticeable lengthening in the lower tail, suggesting that 2

- 3 vehicle s may be influential in decreasing the values of
the coefficients.

( p. 211 ) Detailed results, including m odels fit, fitting histories, coefficients and tests of effect are presented in Appendices Q.3

- W.3 for Bag 1 models, and Q.4

- W.4 for Bag 2 models. Not available for
review

( p. 214 ) 10 References: Refs 4 and 8 cannot be found