# Peer Review of t he Effects of Fuel Sulfur Level on Emissions from the In

- Use Tier 2 Vehicles

Prepared for

Assessment and Standards Division

Office of Transportation and Air Quality

U.S. Environmental Protection Agency

Prepared by

Systems Research a nd Application Corporation

652 Peter Jefferson Parkway, Suite 300

Charlottesville, VA 22911

EPA Contract Number: EP

- C

- 11

- 007

February 2012Page 2 

# Peer Review of t he Effects of Fuel Sulfur Level on Emissions from the In

- Use Tier 2 Vehicles

Table of Contents

Peer Review of the Fuel Sulfur Effects Report , Conducted by SRA International p. 3 1. Background p. 3 2. Description of Review Process p. 4 3. Compilation of Review Comments p. 4 4. References p. 2 0 Appendices
A. Resumes of Peer Reviewers p. 2 1 B. Conflict of Interest Statements p. 29 C. Peer Review Charge p. 37 D. Reviews p. 39Page 3

TO: Kent Helmer, Connie Hart , U.S. Environmental Protection Agency, Office of

Transportation and Air Quality (OTAQ)

FROM: Brian Menard, SRA International

DATE: February 23 , 201 2

SUBJECT: Peer Review of The Effects of Fuel Sulfur Level on Emissions from the In

- Use Tier 2

Vehicles (Fuel Sulfur Effects Report)



## 1. Background

Sulfur in gasoline has long been known to reduce the efficiency of auto mobile exhaust emission aftertreatment systems. Some emission studies have suggested an increase in catalyst sensitivity to sulfur ( i.e. , binding to active catalytic sites) with increasing stringency of vehicle exhaust emission standards. This may be a r esult of higher catalytic efficiencies required for compliance with recent emission standards. EPA, in promulgating its Tier 2 light

- duty vehicle emission standards, recognized the importance of fuel sulfur level. Reductions to new vehicle exhaust emissi on standards under the Tier 2 vehicle and fuels program were accompanied by corresponding reductions in fuel sulfur level to improve the cost and feasibility of the vehicle technology changes required for compliance.

In the 2005/2006 timeframe, EPA’s Mobi le Source Air Toxics/MSAT

- 2 study 1 examined the effects of fuel sulfur level on exhaust emissions from a test fleet of nine Tier 2

- compliant cars and trucks. The study examined four gasolines, without ethanol and blended in a step

- wise manner from a base fuel at 6 ppm S to the final fuel at 32 ppm S. FTP

- weighted emission reductions related to fuel sulfur changes in this program for exhaust emission s were all statistically significant (α = 0.9). D ue to specific catalyst prep procedures that compared a cleaned

- out catalyst with low sulfur fuel to a loaded catalyst with high sulfur fuel, these results were thought to represent a “worst case” only. Th e data, though, suggested reversible sulfur loading was occurring i n Tier 2 vehicle catalysts and significant emission reductions were likely possible through further fuel sulfur level control.

The test program described in this report used two fuels wit h properties similar to conventional federal certification gasoline, except that the “lower sulfur” fuel contained 5 ppm sulfur. The “higher sulfur” fuel had a level similar to the national average in

- use fuel at approximately 30 ppm sulfur. The types of data targeted by the program were the level of “reversible” loading in catalysts found in

- use Tier 2 vehicles and the relative emission differences due to sulfur re

- loading for two different sulfur level fuels in the same vehicle. In order to generate th ese emission data, privately

- owned in

- use vehicles were recruited for the study. It was not feasible to damage or destroy catalysts to directly measure any sulfur loading and, so, the behavior of emissions relative to a baseline was used as a proxy for ca talyst sulfur loading.

EPA sought a n expert peer review of the Fuel Sulfur Effects Report

. T his report documents the peer

r eview. Section 2 of this memorandum describes the process for selecting reviewers, administering the review process, and closing the peer review. Section 3 summarizes reviewer comments according to the

1 ““Proposed Rule: Control of Hazardous Air Pollutants from Mobile Sources” Preamble and Regulations (published March 29, 2006). See http://www.epa.gov/oms/toxics.htm

.Page 4

series of specific questions set forth in the peer review charge. The appendices to the memorandum contain the peer reviewers’ resumes, completed conflict of interest, and bias ques tionnaires for each reviewer, and the peer review charge letter.



## 2. Description of Review Process

In October 2011, OTAQ contacted SRA International to facilitate the peer review of the Fuel Sulfur Effects Report

. EPA provided SRA with a short list of su bject matter experts from academia , consulting, and industry to serve as a “starting point” from which to assemble a list of peer reviewer candidates. SRA selected three independent (as defined in Sections 1.2.6 and 1.2.7 of EPA’s Peer Review Handbook, Th ird Edition) subject matter experts to conduct the requested reviews. SRA selected subject matter experts familiar with statistical analysis and vehicle emissions . To ensure the independence and impartiality of the peer review, SRA was solely responsible for selecting the peer review panel. Appendix A of this report contains the resumes of the three peer reviewers. A crucial element in selecting peer reviewers was to determine whether reviewers had any actual or perceived conflicts of interest or bias t hat might prevent them from conducting a fair and impartial review of the Fuel Sulfur Effects Report . SRA required each reviewer to complete and sign a conflict of interest and bias questionnaire. Appendix B of this report contains an explanation of the process and standards for judging conflict and bias along with copies of each reviewer’s signed questionnaire.

SRA provided the reviewers a copy of the most recent version of the Fuel Sulfur Effects Report as well as the peer review charge containing spe cific questions EPA asked the reviewers to address. Appendix C of this report contains the memo to reviewers from SRA with the peer review charge.

SRA delivered the final review comments to EPA by the requested date. These reviews , contained in Appendi x D of this report, include the reviewers’ response to the specific charge questions and any additional comments the y might have had

.



## 3. Compilation of Review Comments

The Fuel Sulfur Effects Report was reviewed by Dr. Thomas Durbin (University of Californi a, Riverside);
Dr. Ron ald Heck (Independent Consultant); and Dr. S. Kent Hoekman (Desert Research Institute)

. Appendix A contains resumes for each of the reviewers. This section consolidates their comments. The complete comments of the three reviewers m ay be found in Appendix D.

The reviewers were asked on the basis of their work experience and expertise to provide expert opinion on all aspects of the report . I n particular, they were asked to assess the methodologies used to evaluate the effects of b oth “cleanout” and sulfur level on in

- use Tier 2 vehicle emissions and to indicate whether these techniques are likely to yield accurate results . In addition, they were asked to consider the appropriateness of the statistical techniques used to analyze t he data described in this report, their appropriateness in the context of data accuracy and quality issues and the overall conclusions drawn in this report pertaining to reducing fuel sulfur levels in the light

- duty fleet. Lastly, EPA provided a series of specific questions for reviewers to address in their comments.

The reviewers were in general agreement as to the importance of the fuel sulfur effects study and the significance of its results. In this section of the memorandum, t heir written comments are consolidated into three categories: (1) comments in direct response to questions set out in the peer review charge;Page 5

(2) other technical comments; and (3) editorial comments. All textual edits and corrections provided by the reviewers in Section 3.2 are indicated in strikethrough and bold type.



### 3.1 Technical Comments

The reviewers provided a significant number of specific technical comments and were generally favorable in their reviews of the technical and statistical aspects of the Fuel Sulf u r Effe cts Report . This section contains their comments and is divided into those that specifically address either questions or requests contained in the peer review charge and additional technical comments that reviewers chose to provide.



### 3.1.1 Charge Questions & Re quests for Comment

In addition to encouraging reviewers to best apply their particular area(s) of expertise to review the overall study, EPA drafted six questions or requests for comment to serve as a focus for the reviewers. These were included in the p eer review charge provided to the reviewers. In varying degree, the three reviewers provided direct responses to the questions or requests for comments.

1. Was the imputation method used for replacement of measurements with low concentration reasonable? W hat other alternatives may have been better in this case? (Section 7.1)

Durbin: The text talks about tailpipe emissions being greater than zero, while the actual question of relevance may be

– Are the tailpipe emissions greater than the ambient backgroun d concentrations? (presuming a relatively clean background) If the vehicle’s emissions is below background levels, than it would not be making an additive contribution to pollution levels. In this case, you could get a distribution of both positive and neg ative values fluctuating around

zero.

My concern with the imput value procedure is that it might introduce a slight positive bias. My preference would probably be to use zeros as the imput values, since this would represent a distribution of positive and negative values that would average to zero over a larger dataset. For the cases where the dilute exhaust concentrations are lower than the measured background. By eliminating, or essentially not allowing, negative or zero values, doesn’t this add a positi ve bias to the results? Or how do you ensure there is a not a positive bias.

p. 24

– first paragraph

– its states that a data point can either be deleted or replaced with an imput value. Couldn’t the value just simply be left as is? Also, if an imput valu e is deleted for a particular bag, presumably no composite emissions would be available. Would the corresponding test be eliminated in its entirety?

p. 25 last line

– it’s unclear how an outlier could be an imput value. Is this a case where the imput val ue is a very low value? A description of what these values might be should be added to the text. Again, if the imput outlier value is eliminated, how does that impact the result of the test?

Can Table 7

- 2 be modified to include not only the number of outl iers identified, but also the number that were eliminated? This should also be in the text in the paragraph for section 7.1.2.Page 6

Hoekman: [T]he method used to impute emissions values in cases where measured levels were lower than background is probably the best that can be done. The approach of developing an imputation value based upon each specific vehicle family seems better than using a single, fleet

wide imputation value. The fact that imputed values were applied to only about 20% of the vehicles provide s some assurance that this imputation method did not excessively distort the results. This was further confirmed by performing the statistical analyses with and without inclusion of the imputed values. One additional test that could be useful is to conduct the

s tatistical analyses using values of zero instead of the imputed emissions values.

2. Please comment on the use of mixed model in analyzing the data. Was the model fitting strategy in selecting the final model statistically sound? (Section 7.2)

Durbin : The use of a mixed model is a relatively standard method to treat data analysis of emissions, and appears to be appropriate for the study design given here. The fitting strategy of using a saturated model, then developing the most optimal covariance stru cture, and then fitting the final model appears to be statistically sound. The step

- wise backward elimination approach for the “Sulfur Effects data” also appears to be statistically appropriate.

One thought relating to the discussion of the univariate ANO VA and the multivariate MANOVA in section 7.2. Overall, it does not seem to add significant value to the report. Looking at the audience for this report, it would likely be composed of a mix of readers that may or may not have a statistical background. For those with a statistical background, the additional information on the univariate ANOVA and the multivariate MANOVA would not provide significant value. For those that are not as familiar with statistical methods, the added information would likely be

mor e confusing than clarifying. It is suggested that this information, although interesting, might be better placed in an Appendix.

Hoekman: Overall, the statistical modeling approach seems appropriate. As described in Section 7.2 (p p . 26

- 28) the structure and limitations of the emissions dataset make the linear mixed model a good choice for analysis, as this better accommodates missing data, irregularly spaced measurements, and within

- vehicle effects.

A significant question about the statistical modeling approach is the failure to include any information about the vehicles’ catalyst formulations, or other catalyst properties. It is not surprising to observe that vehicle type (car vs. truck) was not a significant model term, but it is surprising that there is no term related to the emissions control systems used across the range of vehicles. Surely not all catalysts were the same, and different catalysts would be expected to respond to changes in sulfur levels in different ways. This is an area that should be addressed. Inclusion of a catalyst term could result in quite different statistical models.

3. Were the model assumptions for the covariance structure reasonable given the data? (Section 7.3)

Durbin: For the “Clean

- out data”, an unstructured covariance structure was initially used (which did not converge), and then a compound symmetry structure was selected. The compound symmetry structure is appropriate since the measurements from the same vehicle should have a homogeneous variance, as included in the t ext. For the “Effect of Sulfur data, where multiple measurements are made at different mileage accumulations, the autoregression covariancePage 7

structure is appropriate since the since the correlations between measurements is expected to decline as the measure ments are further apart in terms of mileage accumulation, as mentioned in the text of the report.

4. Please comment on the methodology used in determining the in

- use sulfur effect for models with and without the sulfur and mileage interaction term. (7.3.3)

Durbin: Overall, the methodology for determining the in

- use sulfur effect for models with and without the sulfur and mileage interaction term appears to be sound. For cases where the sulfur level and mileage interaction term was not significant, the sulfur effect, sulfur loading and associated percentage differences between the high and low sulfur fuels should be constant as a function of increasing mileage. Thus, using the differences in the least squares means from the final model and reverse transforming them could be used to quantify the percentage differences between high and low sulfur for the case where the sulfur level and mileage interaction term was not significant.

For cases where the sulfur level and mileage interaction term is significant, the rate of sulfur loading between the low and high sulfur fuels would differ as a function of mileage, so a different approach would be needed. The methodology of using the in

- use emission level upon arrival (pre

- cleanout) from a larger clean

- out dataset and projecting it out to the mileage where the two lines intersect (in

- use equivalent loading) seems reasonable for estimating the in

- use sulfur level effect. Some clarification should be given on what is meant by the “larger clean

- out dataset” since it is no t immediately obvious what data these actually are. Additionally, how do the incoming sulfur levels for the “larger clean

- out dataset” compare with average in

- use values for the US fleet, or the fleet in different regions of the country.

5. Is the interpret ation of the mileage

- by

- sulfur interaction term presented correctly? (Section 7.3.3)

Durbin: As discussed under question #4, the interpretation of the mileage

- by

- sulfur interaction term does appear to be presented correctly. On p. 48, when the interpreta tion of the interaction term is first being discussed, I would also add in something about the percentage differences in emissions being constant as a function of mileage, just to make things clearer. For example, for the last line of the paragraph just af ter Table 7

- 9: In other words, the effect of high fuel sulfur on Bag 2 NO x exists immediately after clean

- out and remains constant on a percentage basis during subsequent driving of a vehicle.

6. Are the sensitivity analyses on effects of low

- level concentr ations, imputed values, and influential vehicles sufficient as performed? Do the results from the sensitivity analyses provide additional support for the robustness of the conclusion? (Section 7.3.4)

Durbin: Overall, the sensitivity analyses see to add a dditional support to the robustness of the conclusions. Here are a few thoughts on the sensitivity analyses.

If the sensitivity analyses were only done for the NO x bag 2 emissions, it would be worth mentioning that the NO x bag 2 emissions showed a higher percentage of measurements with zero values than most other pollutant/bag combinations, as illustrated in Table 7

- 1.Page 8

It would be interesting to get some feel for the vehicles/data eliminated in the sensitivity analysis for the low concentration measuremen ts. Since these would be low emitting vehicles, it seems like they might have more robust and sulfur tolerant catalyst systems. This is consistent with the results showing an increase in the emissions reductions as these data are pulled out.

Relating to the imput values, it would be interesting to see what the effect would be of making the imput values simply zero. Given the small differences between the model with and without the imput model, there would likely be only a minor impact, but this would add to the robustness of the results, although probably not change any conclusions.

For the influential vehicles, it is interesting to look at the vehicles selected. Vehicles 0007 and 0178 show a relatively large sulfur effect, whereas vehicle 0046 shows a sl ight reverse sulfur effect with tight variability. It is interesting that other vehicles with relatively strong sulfur effects, such as 0165, 0179, and 0011, did not have a strong influence. The result of this sensitivity analysis is reasonably intuitive, with the NOx differences between sulfur levels shrinking, which would be consistent with removing some vehicles with a stronger sulfur effect.

Hoekman: Were the sensitivity analyses described in Section 7.3.4 (p p . 56

- 59) only conducted for Bag 2 NOx? If so, this seems like a major limitation. Also, at the top of p. 59 it is mentioned that even when removing the influential vehicles from the analysis, sulfur effect “is still highly significant.” What are the significance levels before and after removing th ese influential vehicles?

In discussing the sensitivity analyses within the Summary and Conclusions Section (p. 60) it should be stated that these analyses were only performed for Bag 2 NOx emissions (if this is true).



### 3.1.2 Other Technical Comments

All of t he reviewers provided specific technical comments in addition to their responses to the specific questions in the peer review charge.

Durbin:

[1] Study Design

Was the fuel commercially available in the Ann Arbor area at the time of the study oxygenated or not? This is of relevance since the test fuel is non

- oxygenated. 30 ppm is given as the in

- use sulfur average. Is this based on actual data of in

- use sulfur samples, or is this based on regulatory average requirements. In discussing why the US06 x2 clea n

- out cycle was used, it might be worth discussing the clean

out procedures that were used in the MSAT and Umicore studies.

[2] Test Vehicle Selection, Recruitment, and Delivery

What is the average sulfur level for the as received vehicles?
Section 4.1 s hould specifically mention that 19 makes/models were utilized in the study.Page 9

Was any attempt made to characterize the vehicle categories by other factors such as level of long term fuel trim, or whether the vehicle used a closed coupled or double catalyst o r only an underbody, or catalyst metals? What incentive was provided to owners to participate in the program?

– add to section 4.2. More information should be given about the number of vehicles targeted for testing, and the number of vehicles in each vehic le make/model category. The first mention should be here, as opposed to in section 6.2. In that section, it talks about 5 vehicles for each make/model class. It would be useful to say the total number of vehicles tested at this point.

[3] Test Fuel Specs and Procurement

Some more details should be included about the fuel selection would be useful. How does this fuel compare to typical in

- use fuel, in terms of say aromatics and RVP. Were oxygenates in use in the recruitment area? Or compared to averages of in

- use used throughout the US?

[4] Test Procedures

It would be useful to add in fuel change points into the test procedure flow charts. There is some lack of clarity in reading through this section that can be shown by looking at the last paragraph. The last paragraph indicates that only 4 of 19 family used the modified short procedure, whereas the first sentence of the paragraph indicates that the change was made approximately halfway through the program. Then looking at Table 7.7, it indicates that bet ween 2

- 4 vehicles in each make/model class were given the modified short procedure, whereas it seems to imply in the paragraph that once the change was made, it was applied thereafter. Was the change just made for a subset of the vehicles tested in each ve hicle category? Also, in the last sentence it indicates that the change in the number of vehicles providing “sulfur level data” is seen in Table 7

- 7. Does this mean that the modified short vehicles were added to the “sulfur level data” as well as the Long or mo dified Long procedure vehicles? Related to the above comment, why not simply just say at the end of each procedure description how many vehicles were tested on each procedure? Figure 6

- 3 indicates that only the 28 ppm sulfur level data is used for the “clean out effect”, but the very first sentence under section 7 indicates that there is a set of “clean

- out” data at 5 ppm.

[5] Data Analysis and Results

p. 21

– 3 rd paragraph

– A statement should be added to answer the following question

– Why do the results of the Bag 2 NOx emissions have more substantial implications than those of other pollutants/bags? While it is useful to discuss the results at low concentrations, further information on the experimental methods might merit a section. For example, what analyzer is used? Can the detection limits of the analyzers be included in an Appendix? What methods/microbalance was used for the PM measurements, etc.? p. 24

– top of p. There should be a brief paragraph discussing Table 7

- 1 and the number of zero

v alue measurements before discussing the imput value methodologies. p. 24

– last sentence

– It says that the statistical analyses were run both with and without imputed values. In terms of the “without” input value case, does that mean that the values werePage 10

eliminated, were zeros used as the imput values, or were the original values used, whether they are positive or negative. p. 25

– second paragraph

– A new terms vehicle “families” is introduced. Presumably this term is the same as the vehicle make/models used earlier. Presumably these also have the same engine family and this should be included along with engine size, vehicle configuration, and weight. p. 30

– There is no discussion on the symbols in the plots presumably the plots show the average and medi an inside the bar. Then the bars represent the 95% confidence levels and the error bars the full data range. Then there are some other dots on the plot? p. 30

– It is not clear what is meant by “Some vehicle families show the presence of within

family var iability” Presumably this could also be due to differences in in

- use sulfur levels as well, since a handful of vehicles came in with in tank fuels with sulfur levels much higher than those for others. For the post

- cleanout variability, it seems like this s hould be independent of the in

use sulfur loading. Could some of the variability be related to the condition of the vehicle? Vehicle family M504 seems to show a very variability post

- cleanout. p. 32

– Related to BIC criterion. It indicates that the BIC per forms relatively better for small sample size settings. How would one define 5 here? Is the current experiment a small sample size? Figure 7

- 4 does not appear to add significant value. Much of the same information is available in the box plots in Figure 7

- 5, and that is a little easier to read. It talks about the vehicles with similar emissions profiles being grouped. However, it is not immediately obvious what grouping were used for the Figures 7

- 4 and 7

- 5.

[6] This report presents an interesting approac h to estimating the real effects of fuel sulfur in the Tier 2 in

- use fleet. The study was very extensive in terms of the number of vehicles and level of testing that was done, and the results of the study some potentially important impacts of differing in

- use sulfur levels. It would be interesting to evaluate these results further to determine the modeled impact of in

- use fuel sulfur levels on ambient quality. The analysis methodologies in the report appear to be statistically sound, and appropriate for rep eated measures types of testing. The presentation of the data is relatively extensive in terms of the descriptions of the statistical analysis procedures used. As such, the report would relatively straightforward to read for the some with a moderately stro ng background in statistics, but might a little too technical for some readers without a strong statistical background.

Heck: [1] It is very important that the authors discuss right up front that they have taken an approach where the vehicles are consider ed as “black boxes”. This is a totally different approach than that adopted by the Umicore paper (Ball D., Clark D., Moser D. (2011). Effects of Fuel Sulfur on FTP NOx Emissions from a PZEV 4 Cylinder Application. SAE 2011 World Congress Paper 2011

- 01

- 030 0. SAE International: Warrendale, PA.) In the “black box “ approach there is no consideration of the engine control strategies (rich or lean bias, cold start strategy, etc.), sensor responses (from aging), vehicle operation (e.g. fuel cut), catalyst tech nologies, catalyst configurations(close coupled, underfloor, cell density, volume, etc.) See James M. Lyons, David Lax, and Steve Welstand, Investigation of Sulfur Sensitivity and Reversibility in Late

- Model Vehicles, SAE 1999

- 01

- 3676 for work that takes a more phenomenological approach to sulfur effects. By the way I could not find this reference in the report. In the “black box” approach very little information is given besides the vehicle type, mileage, age, etc. Because of this lack of information i t will be difficult for anyone to repeat the authors’ experiments and challenge there results. In addition it will be difficult for the authors to explain data anomalies because therePage 11

are no specifics to tease out a possible explanation. Also since there are so many uncontrolled variables, the statistical analysis may be limited since it is only statistics! This is not meant to say the work has no merit only to provide caution in the final conclusions and merits of too much statistics!

[2] One concern I have is if some of the vehicles have a fuel cut strategies. If so, this could confound the results since in the US06 the catalyst with a fuel cut strategy will see lean excursions while a catalyst without will be stoichiometric during the cycle. Again s ince no information is given in the report on engine operation it is impossible to go back and uncouple this effect.

[3] Sulfur poisoning is a complex phenomenon for catalysts and the catalyst manufacturers have found recipes to compensate for high and lo w sulfur levels (see Harold N. Rabinowitz, Samuel J. Tauster, Ronald M. Heck, The effects of sulfur & ceria on the activity of automotive Pd/Rh catalysts, Applied Catalysis A: General 212 (2001) 215

– 222). If the catalyst companies can design for Tier 2 ve hicles at 30 ppm S, then this study may be in question or to put in another way, maybe some of the vehicles you studied already have these types of technologies and that is why you found some responses that didn’t show an effect of S level!

[4] The remova l of sulfur from the catalyst needs a consistent statement throughout the paper. You show Figure 2

- 1 from Ford but the text is not consistent in describing this removal which is due to temperature, air to fuel ratio, and sulfur level. Also the degree of removal is different for Pt, Pd and Rh and the oxygen storage materials (Ce, Zr, La, etc.). So this discussion needs a little more technology.

[5] [p. 3] “ Test fuels were two non

- ethanol gasolines with properties typical of certification fuel,
one at a s ulfur level of 5 ppm and the other at 28 ppm.”

– Is there any reason to expect different results with ethanol gasolines?

[6] [p. 4.] “ This indicates that the catalyst is not fully desulfurized, even after a clean out procedure, as long as there is sulfur in the fuel.”

– Not sure you can make this conclusion. It may be that the sulfur in fuel equilibrates instantaneously and it is a concentration effect not a desulfurization. To prove this you would have to vary the desulfurization time and see where it r eaches steady state.

[7] [p. 5] “ Other results, such as Bag 1 hydrocarbons, did show a significant miles

- by

- sulfur interaction.”

– Does this mean that the effect never equilibrates? Are you comfortable in saying this?

[8] [p. 7] “ The amount of sulfur re tained by the catalyst is primarily a function of its operating temperature, the active materials and coatings used within the catalyst and the concentration of sulfur oxides in the incoming exhaust gases.”

– Again air to fuel ratio is important as you

men tion in the next sentence for reducing conditions.

[9] [p. 8] “However, the temperatures necessary to release sulfur oxides can also lead to thermal degradation of the catalyst over time.”

– Nowhere do you mention that normal operation of the vehicle, the catalyst is constantly being exposed to rich/lean condition fromPage 12

the control system using an oxygen sensor. So the catalyst is being regenerated in

- situ form the perturbation around stoichiometric.

[10] [p. 11] “ The level of reversible in

- use sulfur sto rage and release (or loading) within an exhaust catalyst system can be assessed by measuring emissions from the vehicle as received, performing a high speed, high load clean

- out cycle, then measuring emissions again.”

– In describing the cleanout it is imp ortant to mention the temperature range and the air to fuel ratio. Also depending on the vehicle calibration the degree of richness and time at such could be different from vehicle to vehicle. How is this taken into account for the degree of cleanout?

[11] [p. 11] “ A vehicle with relatively high exhaust temperature at the catalyst location, and/or significant excess loading of certain platinum group metals (PGM) and other active materials in the catalyst may be relatively insensitive to sulfur loading r egardless of driver behavior.”

– Up to now you have not mentioned the effect of S on the oxygen storage components which will affect NOx and lightoff in cold start. Is there a reason this is not mentioned?

[12] [p. 12] “This loading continues over time w ith vehicle operation and can be observed as an increase in emissions (sometimes referred to in the auto industry as “NOx creep”).”

– Is this due to PGM and/or oxygen storage component deactivation?

[13] [p. 13] Table 3

- 1 Test Vehicles Recruited

– If you look at the Umicore study you will see details about the e3mission control system (close coupled, underfloor, etc.) You give no such detail in this study. Also, they also show a plot of engine lambda during the FTP. This is important background information to possibly explain outliers or unusual results. This study loses significant meaning without this information. You are treating the cars as black boxes and I think this is a mistake! Also, the oxygen s ensor response is very important as well as it age. This is never mentioned in this entire study.

[14] [p. 38] “ Furthermore, the reduction in emissions from cleanout shown in Error! Reference source not found. wo uld likely be larger if the low sulfur test fuel at 5 ppm had been used for the cleanout procedure and the tests immediately following the as

- received baseline emissions.”

– This looks like a conjecture. Present evidence.

[15] [p. 40] “ For example, vehic le IDs 0011, 0022, and 0178 clearly show large effect of fuel sulfur level on emissions while the effect is only marginal for vehicle IDs 0123 and 0264.”

– Differences can be A/F ratio, close coupled catalyst, oxygen storage components, temperature histor y. This is why more information on vehicle characteristics is important to explain unusual results.

[16] [pp. 59

- 60] “ Comparing emissions immediately following the clean

- out procedure on 5 vs. 28 ppm fuel, FTP composite NO x emissions were 18% lower, NMHC 9% lower, and CO 8% lower.”

– To make this conclusion you need to age with 28, clean out with 28 and test with 5 and compare age with 5, cleanout with 5 and test with 5. I went back in text and really didn’t see this discussed in analysis. If I missed i t OK otherwise need some discussion in analysis.Page 13

Hoekman:

[1] The experimental test design was suitable for the intended purposes, although clearly, other testing procedures could have been used as well. For example, the standard FTP test procedure

was u sed, at 75 F, to determine the emissions results from each vehicle test. Some comments should be offered regarding the suitability of the FTP driving cycle and a 75 F test temperature in providing representative/realistic operating conditions. For the in

- u se vehicle fleet, a set of standardized test conditions is clearly necessary to obtain repeatable and statistically significant results. However, the FTP test is now over 35 years old, and 75 F is just one of many temperatures that could be used. The overa ll quantitative effect of fuel sulfur on emissions as determined in this study would be more convincing if it were shown that similar results apply to other realistic operating conditions.

[2] The use of two, back

- to

- back US06 dynamometer cycles was an ap propriate choice for catalyst regeneration. Although a somewhat more aggressive cycle might provide more complete desulfurization, potential damage to the vehicle was a significant concern, as explained in this report.

Also, the approach of catalyst clean

- out, followed by repeated testing with a single fuel is a valid way to determine a new “equilibrium emissions level” at a specific fuel sulfur level. However, in real

- world driving, such an equilibrium level is never attained, because operating conditions are constantly changing, and fuel sulfur varies from tank

- to

- tank.

[3] It is not clear why this report places so much emphasis on Bag 2 results

– especially Bag 2 NOx. Typically, Bag 2 emissions concentrations are extremely low from properly functioning Tier 2 vehicles. In fact, as shown in this study, it is often difficult to distinguish Bag 2 levels from background levels. Large percentage differences between two very small numbers may not be very meaningful. To help put these results in perspective, i t would be useful to show the relevant certification emissions levels for these Tier 2 vehicles. Also, something should be said about the impact of fuel sulfur reduction on fleet

- wide emissions, and how large a reduction this represents in comparison to th e entire mobile

- source emissions inventory. In reality, this test program only captured a small portion of the entire in

- use fleet (only Tier 2 vehicles of model years 2007

- 2009).

[4] Some historical perspective about the “FTP bag method” would be helpf ul here. This sampling and analysis method was developed about 40 years ago, to measure emissions from uncontrolled (or slightly controlled) vehicles. Prior to introduction of emissions control systems, Bag 2 emission levels were routinely 2 orders of magn itude higher than today. Under such conditions, the FTP bag method was perfectly suitable for the intended purpose. Now, with Bag 2 emission levels being nearly indistinguishable from background, a different method should be employed for quantifying vehicl e emissions.

[5] In retrospect, it seems unfortunate that only non

- ethanol gasolines were used, as such fuels are increasingly uncommon. It is stated that similar results would be expected from ethanol

containing gasolines, but no data are provided to su pport this. If any subsequent testing has been done with ethanol

- containing gasolines, this should be mentioned.Page 14

[6] On page 2 (1 st paragraph) the quantity of sulfur present on the vehicle’s catalyst is said to be a function of temperature and fuel sulfur level. Isn’t the catalyst formulation/metallurgy another important factor? (Catalyst formulation is mentioned on page 7.) What is known about the catalyst formulations used in each of the test vehicles of this study?

[7] It is interesting to note that wh en reducing fuel sulfur from 28 ppm to 5 ppm, significant emissions reductions were observed for all pollutants except PM, (See Tables ES

- 2 and ES

- 3.) Is there a clear mechanistic reason for this lack of effect for PM? Are the catalyst systems used in Tier 2 vehicles ineffective in reducing PM (and PM precursors)? This lack of a PM effect should be mentioned along with the major findings that are shown in bullet form on page 6.

[8] On page 7 (final paragraph) it is stated that the impact of fuel sulfur on emissions was considered negligible under the Tier 0, Tier 1, and NLEV programs. I don’t think this is true. The emissions impacts of fuel sulfur have been well known for a long time, but more severe sulfur reductions were not thought to be a cost

- effecti ve emissions control approach until recently. To provide greater context, it would be useful to include a brief summary of gasoline regulations (including sulfur levels), and how they have evolved over the past 30

- years. Along with this, a history of LD ve hicle emissions standards should also be presented.

[9] On page 8, various detrimental effects of hot/rich catalyst operation are described

– including catalyst degradation and increased emissions of PM, NMOG, and CO. However, nothing is said about fuel e conomy effects

– either here or elsewhere in the report. In general, fuel economy is a concern with any type of fuel or vehicle modification. This topic should at least be mentioned. In addition, the dataset generated in this study provides an excellent op portunity to evaluate the impacts of sulfur reduction on fuel economy, by analyzing CO 2 as another pollutant, along with CO, THC, NOx, and PM.

[10] The 2005 MSAT study, which used nine Tier 2 vehicles, is described on pages 8

- 9. The emissions reductions upon lowering fuel sulfur from 32 to 6 ppm are said to be 33% for NOx, 11% for THC, 17% for CO, and 32% for CH 4 . Are these values simple averages of the nine vehicles, or were they computed by a more sophisticated method? Were similar statistical methodolo gies used to compute the fuel sulfur effects in both the MSAT study and the present study? Were PM emissions also measured in the MSAT study?

[11] When first describing the two test fuels on page 10, it would be helpful to refer the reader to Table 5

- 1 (p age 15), which provides a more complete listing of fuel properties. Also, did these fuels contain anything unusual in the way of additives; e.g. antioxidants, detergents, dyes, etc.?

[12] In selecting the vehicles, was the ratio of LD trucks/LD passenger cars representative of the in

- use fleet? Table 4

- 1 (page 13) which lists the 19 makes/models that were recruited, shows that the No. 1 U.S. sales rank vehicle is missing. Was this an oversight, or are the rankings shown in this table incorrect? Notice also that the Toyota Camry is shown as No. 23 in U.S. sales rank. This seems unbelievably low for what has traditionally been a “top seller.” Also, it would be useful to add a column to Table 4

- 1 to indicate the number of vehicles in each category that
were te sted in this study, comprising the total test fleet of 81. It might also be useful to indicate here the number of vehicles in each class that underwent the various test procedures: i.e. short procedure, long procedure, modified short procedure, and modifie d long procedure.Page 15

[13] The modified long testing procedure shown in Figure 6

- 2 (page 19) is confusing. The blue box indicating the short procedure shows only two post cleanout FTPs, although the wording still indicates “triplicate FTPs at 28 ppmS.” Also, in describing the modified short procedure on page 20, it is said that “... the change in the number of vehicles providing sulfur level data can be seen in Table 7

- 7 starting with Family ID N513.” It is not clear what this means. Does this table list the veh icle families in chronological testing order, so that the first 13 families listed were tested using the standard short procedure, and the last 4 families listed were tested using the extended short test? It is also confusing that Tables 7

- 5 and 7

- 7 provid e identical information.

[14] How is NMOG different from NMHC? What analytical procedures were used to measure these two pollutants?

[15] Displaying the background and sample measurements by FTP Bag (as done in Figure 7

- 1 and Appendix B) is very instruct ive. However, only NOx and THC results are shown in this way; it would be useful to also include CO and PM. It would also be helpful to draw ovals in Figure 7

- 1 to capture all background measurements for each vehicle family shown. Although it is not possib le to tell which sample points correspond to which background points, this graphical approach clearly illustrates the problem of very low emissions measurements relative to background in Bag 2. It also begs the question of why background levels of NOx are so variable. Figure 7

- 1 shows that these background concentrations varied by over an order of magnitude for many of the vehicle families tested.

[16] In the determination of outlier data points (described on page 25) what is the rationale for choosing an outlier screening criterion value of ± 3.5 for the studentized residuals? Also, although Table 7

- 2 identifies the number of outlier points for each pollutant/bag, it doesn’t indicate how many of these outlier points were actually excluded from the statisti cal analyses.

[17] As explained on page 26, log

- transformation of emissions measurements has commonly been used to analyze vehicle emissions data. However, many previous emissions modeling studies have utilized test fleets that included a variety of techn ology types having a wide range of emission levels. In the present study, only Tier 2 vehicles were used, and the emission levels did not vary drastically across the test fleet. Given this situation, is log

- transformation still necessary (and helpful)?

[1 8] On page 29 it is stated that the average starting odometer reading of the vehicles used to assess the clean

- out effect at 28 ppmS was 31,470 miles. Is this value the average of the 17 vehicle families shown in Table 7

- 3, or the average of all 81 vehicle s? Similarly, is the ± value of 1,578 miles the standard deviation of the 17 vehicle families or the 81 individual vehicles? (The same questions pertain to the vehicles used to assess the clean

- out effect at 5 ppmS, described on page 35; and the vehicles u sed to assess the sulfur effects, described on page 39.)

[19] The box plots shown in Figure 7

- 2 (page 31) are quite informative in displaying the relative variances between the pre

- and post

- cleanout vehicle tests. It would be useful to show similar plots for other pollutants and Bags. Also, a legend should be included to explain the different symbols used in these plots.

[20] When describing the dependent variable ( Y i ) and effects ( X

i and Z

i ) on pages 31

- 32, the reader should be reminded of the mixed mod el being used, shown as Equation 7

- 1 on page 28.Page 16

Also, it is stated on page 32 that “The significance of between

- family variation was observed graphically in Figure 7

- 2 ...” But this figure only show Bag 2 NOx results. Was similar between

family variation ob served with other Bags and pollutants?

[21] Figure 7

- 3 (pages 37

- 38) is not easy for the reader to process. I realize that the data are lumped by emission level, but displaying the data in three separate charts, over two pages, is confusing and difficult to understand. It would be preferable to show these data on a single page (similar to Figure 7

- 2, page 31) or even in a single chart (similar to Figure 7

- 1, using a logarithmic scale). It would also be better to order the vehicles along the x

- axis in the s ame way they are listed in Table 7

- 5. Finally, it would be helpful to see more Bags/pollutants presented in this way

– not just Bag 2 NOx.

[22] On page 38, the enhanced emissions reduction benefits of the 5 ppmS clean

- out are described as compared to the 28 ppmS clean

- out (comparing Tables 7

- 4 and 7

- 6). It is interesting to note that this does not apply to PM emissions. Some explanation should be offered as to why PM emissions show such a different behavior. Also, the last sentence on page 38 mentions conf irmation of results that have not even been presented yet. The placement of this sentence seems odd.

[23] The same comments given above regarding data display in Figure 7

- 3 apply to Figure 7

- 4 (pages 41

- 42). The current grouping of vehicles shown in Figur e 7

- 4 makes it very difficult to see the important point being made on page 40 that some vehicles within the same family had markedly different emission profiles. This would be much easier to see if a single chart were used to display all vehicles

– simila r in structure to Figure 7

- 1. Also, the box plots in Figure 7

- 5 (pages 43

- 44) should all appear on a single page, and should have a legend to explain the symbols being used. Further, it would be helpful to identify each vehicle ID number shown in Figure 7

- 5 by make/model; e.g. Vehicle ID 0003 is a Toyota Corolla. Again, additional figures should be shown for other Bags and pollutants.

[24] Figure 7

- 8 (page 50) is very helpful in convincingly demonstrating the reasonableness of the derived statistical model in predicting Bag 2 NOx emissions with both high and low sulfur levels. It would be useful to include similar plots for other Bags/pollutants.

[25] At the bottom of page 50, where Figure 7

- 9 is being discussed, it is stated that “the rate of sulfur loadi ng is the same for both high and low sulfur levels.” What is the basis for this statement? Sulfur loading is not measured directly. Is NOx emission rate taken as a surrogate for sulfur loading? (This statement about sulfur loading is also given in the Summ ary and Conclusions Section on page 59.) Also, Figure 7

- 9 shows quite clearly that for some vehicles, not only are the emission rates higher with the high sulfur fuel, but also the increase in emissions with mileage accumulation is higher. For example (see vehicles 0075, 0123, 0264, 0178, and 0179). This seems inconsistent with the statement that rate of sulfur loading is the same for both high and low sulfur levels.

[26] In Table 7

- 10 (page 53) the fixed effects in the NOx statistical model include only s ulfur level and miles for all three FTP Bags. Yet the FTP Composite result includes a sulfur level by miles interaction term. This seems strange, since the FTP Composite emissions are simply calculated as a combination of the individual Bag emissions. Some explanation should be offered.Page 17

[27] Pages 53

- 54 discuss the problem of comparing emissions from high and low sulfur levels when there is a sulfur level by mileage interaction. The approach taken to address this (as illustrated in Figure 7

- 10) seems rathe r arbitrary, although it may be as good as any other approach. However, to explore the impact of this selection, it might be good to conduct some sensitivity analyses using other methods.

[28] On page 55, it is stated that “For all models except CO Bag 1, CO Bag 1

– Bag 3, and CH 4 Bag 1

– Bag 3, the reduction estimates are statistically significant ...” However, this is not true for the pollutants NOx + NMOG or PM.



### 3.2 E ditorial Comments

All of the reviewers provided significant editorial comments. They p rovided suggestions for strengthening the report through clarification and elaboration. In addition, the reviewers noted typographical and formatting errors and textual omissions.

Durbin:

It might be useful to discuss the Umicore study cited in the Exe cutive summary, since usually the executive summary is more condensed than the Background. Executive summary

– the number of spaces between sentences differs from to in some cases 3 or even 4. This should be made consistent. p. 2 The end of the first sent ence and start of the second redundantly talk about catalyst efficiency. p. 2 The final sentence of the first paragraph should be split into two sentences. p. 2 Second to last sentence

-- rewrite... for a PZEV operated on a 3 ppm fuel compared to a 33 ppm fu el. p. 3 third sentence

– In response

.... To address this question, the EPA (spell out EPA first use) p. 3 last sentence

– split it

– performance . F or Example, p. 4 2 nd line

– fuel s (add s); Also split the second to last sentence. p. 5 need space for 5 ppm and 28 ppm, and period after “overall”. This sentence should probably also be split. p. 5 3 rd sentence

– rewrite

– a significant miles

- by

- sulfur interaction was not found from the model fitting.... p. 5 last sentence

– rewrite

– In this case, the relatively differences with sulfur level varied as a function of mileage, so determination..... p. 7

– 1 st sentence

– “has long been shown” is awkward . Also in executive summary p. 8, 9, and 10

– comma after e.g. , p. 8 comma after idles) , comma after in 2005 , p. 9

– 1 st line

– spell out MSAT acronym; line 7

– word “specimens” is a bit awkward ; 6 th line form bottom “benefits of further sulfur control” p. 11

– about line 10 comma after high temperatures), line 15 “conditions that are favorable” line 16 lines further do wn “A vehicle with a relatively..., reference 10 on the E

- 60 report is not referenced properly and should contain more author names. p. 12

– 1 st line operation , (add comma); 1 st line of section 4.1 “ with the intent of being to be representative of the latest ...”;Page 18

p. 12

– last sentence table 4

- 1 should not be underlined. This same changed should be incorporated throughout the report, as it appears there is something in the formatting underlining table and figure references in the text. p. 13

- First line

– “Veh icles recruited for testing were targeted to have a mileage between 12,000 and 40,000 miles and an age of less than three years old.” p. 14

- line 5 up front bottom

– exhaust leaks , (add comma) p. 16

- First full paragraph

– Following the fuel change , p. 16

- last sentence

– rewrite to “The Long and Short procedures are shown in Figure 6

- 1 and are discussed in greater detail below. The Long and Short procedures are identical in structure for the first six emissions tests. p. 17

- third line

– start a ne w paragraph about the Long procedure p. 20

– last sentence

– Split into two sentences. p. 21

– 3 rd paragraph

– detail

. but t The p. 27

– 2 nd paragraph line 4 “vehicles which was were “...., later in same paragraph Besides this , (add comma), 3 rd paragraph line 2

– effects , (add comma) p. 28

– line right after equation

- respectively , (add comma), 3 rd paragraph

– line 2

– model , (add comma)

– line 4 structure , (add comma) p. 29 1 st paragraph line 3 exchange “were” for was

- line 4

– levels , (add comma)

– p. 32

– last paragraph line 3

– covariance , (add comma) p. 33 line 6

– selected , (add comma) p. 35 last paragraph 1 st line

– “dataset include d 17” ...... p. 38

– 2 nd line from bottom

– Section 7.3.3 , (add comma) p. 40

– 3rd line from bottom

– i.e. , (add comma) p. 40

– 5th line from bottom

– rewrite

- considering each vehicle as a random effect s might be useful p. 42

– The sentence that includes Figure 7

- 6 should be broken up into two sentences. Example: This is shown in figure 7

- 6, which shows the log

- transformed em issions from individual vehicles by sulfur level. 5 th sentence from bottom

– add comma ... some vehicles , and suggests.... p. 44

– 2 nd line from bottom

– add comma ... vehicle to vehicle , and simple descriptive statistics... p. 45

– 1 st line

– rewrite .. a simila r top

- down model fitting statistical approach was applied to the clean

- out data, as described in Section 7.3.1. p. 45

– 2 nd sentence

– rewrite .. Furthermore, additional analysis was done that used a subset of the sulfur level dataset that isolates the emi ssion measurements immediately following the clean

- out to address the effectiveness of the clean

- out cycle in reducing emissions. p. 46

– 2 nd paragraph 4 th sentence

– add comma ... irregularly , where p. 46

– 3 rd paragraph 2 nd sentence

– ....exponentially with time , i.e., the variability.... p. 50

– 2 nd to last line

– operation , (add comma) causing an increase in emissions p. 58

– 1 st paragraph

– line 3 through the presence, line 4 (RLD), (add comma), line 5 after an iterative... p. 58

– 1 st paragraph

– line 3 th rough the presence, line 4 (RLD), (add comma), line 5 after an iterative... p. 59

– 2 nd paragraph under “summary and conclusions”

- ...US06 cycles . This data was used to examine the existence... “Summary and Conclusions”

– Under the first bullet point. Performi ng a clean

- out cycle with a 28 ppm fuelPage 19

The second bullet point could be clearer. For example: for a subset up vehicles tested on both 5 ppm and 28 ppm fuels, it was found that additional FTP composite reductions of 18% for NO x , 9% for NMHC, and 8% for CO were for the 5 ppm fuel in comparison with the 28 ppm fuel.

Heck:

In the report, there are a number of places where symbols are not explained in tables and in text. Also, I did not see an explanation about how one does a hot/cold start FTP. Please make sure this is explained in the text of the report. At end of papers, please a list of references. I had to go through the entire paper many times to see if a reference was cited. This was painful! p. 2. Fuel sulfur content has long been understood to aff ect the performance of emission after treatment catalysts in light duty vehicles, where the sulfur and/or its oxides adsorb to the active precious metal sites and oxygen storage materials , reducing the catalyst’s efficiency in destroying harmful pollutants

. p. 2. The quantity of sulfur present on the catalyst at any given time is a function of its temperature, air to fuel ratio

, and the fuel sulfur level . . . p. 2. . . . with elevated catalyst temperature , rich of stoichiometric operation, and lower fuel sulfur concentration both reducing sulfur loading. p. 4 [Table ES

- 1]. Need to explain what p

- value is in the text? Also what is in (?????)? p. 5. [Table ES

- 2]. Again what is p value and what is in (....)? p. 5. This analysis found highly significant reduct ions for several pollutants, as shown in Error! Reference source not found. ; reductions for Bag 2 NOx were particularly high, estimated at 59 percent between 28ppm and 5ppm overall

. p. 17 [Figure 6

- 1]. Do you explai n what a hot/cold start FTP is anywhere in the report?? p. 25 [Table 7

- 1]. What does the value in (...) mean?? Please explain. p. 35 [Table 7

- 4]. Explain what is in (....)? p. 39 [Table 7

- 6]. Explain what is in (......)? p. 46. “The BIC value for the first

- order autoregressive structure was 764.90.” Did you explain BIC?? If not then do so. p. 47 [Table 7

- 8]. What is Pr? Explain. p. 55 [Table 7

- 10]. Is there an explanation as to why these lines diverge and are not parallel?? Need some discussion here from the m odel. p. 56 [Table 7

- 11]. What is in (....)? p. 57 [Table 7

- 13]. Is “Probt” probability??

Hoekman:

p. 3, footnote: “The program has collected additional data that is being incorporated ...” p. 5, 5 th line: Insert period after “overall.” p. 6, footnote to Tab le ES

- 3: It is stated that reduction estimates were computed differently for Bag 1 THC and CH 4 because these clean

- out effects were not statistically significant. Table ES

- 1 shows that the clean

- out effects were also not statistically significant for Bag 1 NOx and NMHC. p. 6, next to last line: “... are consistent with those formed in the MSAT and Unicor studies ...” p. 17, Figure 6

- 1: The sulfur clean

- out cycle in the “short procedure” (blue

- colored box) should indicate 28 ppmS, not 30 ppmS. p. 29, 2 nd line: “ In addition, the results from the clean

- out data were used to supplement ...”Page 20

p. 30, 4 th line: “... representing the as received sulfur level ...” p. 32, 2 nd line: “... vehicles from the same vehicle family have the similar emission profiles.” p. 40, 4 th line: “Ho wever, the sulfur loading effect certainly varies by vehicle ...” p. 46: the term “BIC” is used without definition. The reader must refer to page 32 to see that BIC refers to the Schwarz Bayesian Criterion. It would be helpful to include a table of acron yms and abbreviations, as many rather obscure terms are used throughout this report

. p. 58, 1 st ¶, 5 th line: “... which is calculated after an iterative process ...” p. 59, Summary and Conclusions: the first bulleted finding should be clarified to indicate th at the stated NOx, NMHC, and PM reduction values were obtained using a clean

- out procedure with 28 ppmS. The second bulleted finding should be clarified to indicate that although clean

out with 5 ppmS fuel rather than 28 ppmS fuel further reduced NOx, NMHC , and CO, no effect was observed for PM.



## 4. References

U.S. EPA (2011). The Effects of Fuel Sulfur Level on Emissions from the In

- Use Tier 2 VehiclesPage 21 

## Appendix A: Resumes of Peer Reviewers

Tom Durbin

Research Engineer

University of California, Riverside Bourns College of Engineering CE

- CERT (Center for Environmental Research & Tech.)

Thomas D. Durbin is an associate research engineer in the vehicle emissions research group of CE

- CERT. Dr. Durbin conducts research in a broad range of topics related to vehicle emissions including particulate matter (PM) emissions, diesel, biodiesel, and gasoline fuels, portable emissions measurement systems (PEMS), and the unregulated species such as ammonia, N2O, and toxics. Dr

. Durbin conducts research in CE

- CERT’s state

- of

- of

- the

- art emissions test facilities including a heavy

- duty mobile emissions laboratory, a heavy

- duty engine dynamometer laboratory, and a light

- duty chassis dynamometer l aboratory. Dr. Durbin is also extensively involved with in

- field measurements of emissions from passenger cars, heavy

- duty trucks, and construction equipment. Prior to joining the vehicle emissions group, Dr. Durbin was involved in several other areas of r esearch at CE

- CERT including renewable energy and fuel sources and advanced vehicle technologies. Tom Durbin received his doctorate degree in Physics from the University of California, Riverside, in 1994 where the primary focus of his dissertation was the study of Si films and solid lubricants.



### Degrees

B.S. Physics 1988 ; UCR M.S. Physics 1990 ; UCR Ph.D. Physics 1994 ; UCR



### Research Area

Vehicle emissions



### Publications

Five Fuel

- Related Publications

• Durbin, T. D., J. R. Collins, H. A. Galdamez, J. M. No rbeck, M. R. Smith, R.D Wilson and T. Younglove. 2000. Evaluation of the Effects of Biodiesel Fuel on Emissions from Heavy

- Duty Non

Road Vehicles. South Coast Air Quality Management District, May.

• Durbin, T.D., J.W. Miller, B.B. Holden, and N.L. Helges on. 2003. The Effects of Biodiesel andPage 22



### Other Alternative Diesel Fuels on Emissions from Diesel Vehicles. California Air Resources Board’s and California Energy Commission’s Alternative Diesel Fuel Symposium, Sacramento, CA, August.

• Durbin, T.D. and J.M . Norbeck. 2002. The Effects of Biodiesel Blends and ARCO EC

- Diesel on Emissions from Light Heavy

- Duty Diesel Vehicles. Environ. Sci. & Technol. 36:1686

- 1691.

• Durbin, T. D., J. R. Collins, J. M. Norbeck and M. R. Smith. 2000. The Effects of Biodiesel, Biodiesel Blends, and a Synthetic Diesel on Emissions from Light Heavy

- Duty Diesel Vehicles. Environ. Sci. & Technol., 34, 349.

• Durbin, T. D., X. Zhu and J. M. Norbeck. 2003. The Effects of Diesel Particulate Filters and a
Low

- Aromatic, Low

- Sulfur Dies el Fuel on Emissions for Medium

- duty Diesel Trucks. Atmospheric Environment, vol. 37, 2105

- 2116.

Other Significant Publications

• Moosmüller, H.; Arnott, W.P.; Rogers, C.F.; Bowen, J.L.; Gillies, J.; Pierson, W.R.; Collins, J.F.; Durbin, T.D.; and Norb eck, J.M. (2001) Time Resolved Characterization of Diesel Particulate Emissions: 1. Instruments for Particle Mass Measurements. Environmental Science & Technology 35(4):781

- 787, 2/15.

• Cadle, S.H.; Mulawa, P.A.; Ragazzi, R.A.; Knapp, K.T.; Norbeck, J.;M .; Durbin, T.D.; Truex, T.J.; and Whitney, K.A.(1998) Exhaust Particulate Matter Emissions from In

- Use Passenger Vehicles Recruited in Three Locations

- CRC Project E

- 24. SAE Technical Paper No. 99FL

- 215.

• Rodriguez

- Forker, A.; Uihlein, J.P.; Segal, J.S.; Sverdrup, G.M.; Seymour, J.P.; Kinateder, J.G.; Pierce, A.; and Durbin, T.D. (1998) Fleet Test Using Butane and Propane Mixtures. SAE Technical Paper No. 982444.Page 23



### Dr. Ron Heck is currently an independent consultant. Prior to that, Ron was a research mana ger responsible for developing catalyst technology for Engelhard Corporation's worldwide customers in automotive catalyst. He has worked on the development of catalytic processes in SCR NOx, NSCR NOx, automotive catalyst, diesel catalyst, PremAir TM 

### catalys t systems, hydrogenation technology, ozone abatement, volatile organic compound abatement, ammonia oxidation, chemical feedstock purification and chemical synthesis. Ron is a member of American Men and Women of Science and Who's Who in Technology Today. H e is a recipient of the Forest R. McFarland award from the Society of Automotive Engineers for outstanding contributions to this professional society. Ron is an SAE Fellow in recognition of engineering creativity and contributions to the profession and th e public at large. He is co

- instructor for courses for the SAE in automotive emission control catalysis and diesel emission control catalysis. He was a member of the Scientific Advisory Board of the Strategic Environmental Research and Development Progra m for environmental studies in the Department of Defense. Ron has been involved in over 80 publications in commercial applications of catalysts and holds 36 U.S. patents on catalytic processes. He is the co

- author of the book entitled "Catalytic Air Poll ution Control: Commercial Technology" (1 st 

### , 2 nd 

### and 3 rd 

### edition) and is the former co

- editor of the NewsBrief section of Applied Catalysis B: Environmental. Ron and his former research team from Engelhard received the 2004 Thomas Alva Edison Patent Award f rom R&D Council of New Jersey for the invention of close coupled catalyst technology for ultra low emission gasoline vehicles. Ron received his B.S. in Chemical Engineering and his Ph.D. from the University of Maryland, and a M.A. in Theology from the Col lege of St. Elizabeth.Page 24



### S. KENT HOEKMAN Tel: 775

- 674

- 7065 Research Professor Fax: 775

- 674

- 7016 Desert Research Institute email: Kent.Hoekman@dri.edu Division of Atmospheric Sciences 2215 Raggio Parkway, Reno, NV 89512

- 1095

Education

Ph.D. 1980 Iowa State University, Ames, IA Organic Chemistry B.S. 1975 Calvin College, Grand Rapids, MI Chemistry

Professional Interests and Activities Dr. Hoekman is a Research Professor within the Division of Atmosphe ric Sciences at the Desert Research Institute (DRI). DRI is a statewide division of the Nevada System of Higher Education (NSHE) that pursues basic and applied environmental research on local, national, and international scales. His professional interests include environmental impacts of energy production, distribution, and use; development of renewable and sustainable energy systems; conversion of biomass to biofuels; air quality impacts of vehicle emissions; and impacts of advanced

- technology fuels and ve hicles on emissions and energy use. He is also interested in the interface between politics and environmental science, particularly in the areas of energy policy, renewable fuels, greenhouse gases, and climate change. In addition to his personal professi onal activities, Dr. Hoekman has provided leadership for DRI in the identification, protection, and licensing of intellectual property (IP) developed at the Institute. Dr. Hoekman was instrumental in establishing a joint Technology Transfer Office (TTO) be tween DRI and the University of Nevada, Reno (UNR). He currently serves as DRI’s Liaison to the TTO, and oversees the activities of this office on behalf of DRI. Dr. Hoekman has also served DRI by coordinating and promoting the Institution’s R&D portfolio in the field of renewable energy. He has led the effort to establish a Renewable Energy Center (REC) at DRI, and continues to provide leadership in this area. For further information about the DRI

- REC, please refer to its website at http://www.dri.edu/rec

. In addition, Dr. Hoekman is active in the scientific academic and business communities. He serves as a reviewer for numerous science and engineering journals, is a member of several professional societies, has ass isted in organizing scientific conferences, and contributes to the mentoring and advisement of graduate students at the University of Nevada in Reno. Currently, he serves as Associate Editor for the International Journal of Alternative Energy.

From 2001 to 2007, Dr. Hoekman served as Executive Director of DRI’s Division of Atmospheric Sciences (DAS). DAS consists of approximately 50 research faculty, along with 70 technologists, graduate students, post

- docs, and other support staff. The Division conducts fundamental and applied research around the world on topics pertaining to emissions, renewable energy, air pollution, meteorology, climatology, aerosol chemistry and physics, and other areas related to atmospheric science. DAS also serves as the institutio nal home for the Western Regional Climate Center, one of six NOAA

- funded regional climate centers in the U.S. As Director, Dr. Hoekman was responsible for all personnel, financial, organizational, and professional activities of Divisional operations. The D ivision’s scientific work is sponsored by over 100 federal, state, local, and private organizations that provide approximately $14 million per year in research grants and contracts. For more information about the Division and its activities, please refer t o its web site at http://www.das.dri.edu

.Page 25



### Prior to joining DRI in 2001, Dr. Hoekman spent over 20 years at Chevron, where his research focused on transportation fuels and their impacts on motor vehicle emissions and a ir quality. Experimental work included detailed characterization of exhaust emissions compositions from gasoline

- , diesel

- , and alcohol

- fueled vehicles. Laboratory studies were conducted to investigate how changes in fuel formulation could reduce vehicle emissions and improve ambient air quality. He has served on several technical committees representing the American Petroleum Institute (API), the Western States Petroleum Association (WSPA), the Coordinating Research Council (CRC), and other industry organ izations interested in fuels, emissions, atmospheric chemistry and air quality. Dr. Hoekman also has experience in environmental regulatory affairs pertaining to vehicles, fuels, emissions, air quality, and health effects. He has served in technical adviso ry roles to EPA and was a member of the California Air Resources Board (CARB) Research Screening Committee for five years. He served as a member of the Health Effects Institute’s (HEI) Special Committee on Emerging Technologies from 2001 through 2007.

P rofessional Experience

2007

– Present Research Professor, Division of Atmospheric Sciences, Desert Research Institute,

Reno, NV

2001

– 2007 Executive Director, Division of Atmospheric Sciences, Desert Research Institute,

Reno and Las Vegas, NV

1997

– 2 001 Senior Staff Scientist, Chevron Products Co., San Francisco and San Ramon, CA

1990

– 1996 Staff Scientist and Senior Staff Scientist, Chevron Research and Technology

Company, Richmond, CA

1980

– 1989 Research Chemist and Senior Research Chemist, Che vron Research and

Technology Company, Richmond, CA

Professional Memberships
American Association for the Advancement of Science (AAAS) American Chemical Society (ACS) Air and Waste Management Association (AWMA) Society of Automotive Engineers (SAE)

Award s/Honors Chevron Chairman’s Award (1984)

– Presented in recognition of diesel emissions research Horning Memorial Award (1985)

– Presented by the Society of Automotive Engineers Arch T. Colwell Merit Award (1985)

– Presented by the Society of Automotive En gineers Society of Automotive Engineer’s Award for Excellence in Oral Presentation (1993 and 1995) Recognition of Appreciation from the California Air Resources Board (2001)Page 26



### Peer

- Reviewed Publications Yan, W. and S.K. Hoekman (2012). Dust suppression with glycerin from biodiesel production: a review, J. Environ. Protection (in press). Hoekman, S.K. and C. Robbins (2012). Review of the effect of biodiesel on NOx emissions, Fuel Proc. Technol . doi:10.1016/j.fuproc.2011.12.036 (in press).
Hoekman, S.K., A. Bro ch, C. Robbins, E. Ceniceros, and M. Natarajan (2012). Review of Biodiesel Composition, Properties, and Specifications. Renewable and Sustainable Energy Reviews 16

, 143

- 169. Wang, X., C. Robbins, S.K. Hoekman, J.C. Chow, J.G. Watson, and D. Schuetzle (2011 ). Dilution Sampling and Analysis of Particulate Matter in Biomass

- Derived Syngas. Front. Environ. Sci. Engin. China 5 (3) 320

- 330. Robbins, C., S.K. Hoekman, E. Ceniceros, and M. Natarajan (2011). Effects of Biodiesel Fuels upon Criteria Emissions. Soc. A uto. Eng. Tech. Paper No. 2011

- 01

- 1943. Hoekman, S.K., A. Broch, and C. Robbins (2011). Hydrothermal Carbonization (HTC) of Lignocellulosic Biomass. Energy Fuels 25 (4) 1802

- 1810. Bruins, R., S.K. Hoekman, R. Efroymson, A.Aden, and A. Hecht (2010).Transpor tation Fuels for the 21 st 

### Century. EM Nov. 2010, 26

- 32. Hoekman, S.K., A. Broch, C. Robbins, and R. Purcell (2009). CO 2 

### Recycling by Reaction with Renewably

- Generated Hydrogen. International J. of Greenhouse Gas Control 4 , 44

- 50.
Hoekman, S.K., A.W. Gertl er, A. Broch, C. Robbins, and M. Natarajan (2009). Biodistillate Transportation Fuels 1

– Production and Properties. Soc. Auto. Eng. Tech. Paper No. 2009

- 01

- 2766. Robbins, C., S.K. Hoekman, A. Gertler, A. Broch, and M. Natarajan (2009). Biodistillate Trans portation Fuels 2

– Emissions Impacts. Soc. Auto. Eng. Tech. Paper No. 2009

- 01

- 2724. Broch, A., S.K. Hoekman, A. Gertler, C. Robbins, and M. Natarajan (2009). Biodistillate Transportation Fuels 3

– Life

- Cycle Impacts. Soc. Auto. Eng. Tech. Paper No. 2009

- 0 1

- 2768. Hoekman, S.K. (2009). Biofuels in the U.S.

– Challenges and Opportunities. Renewable Energy 34

, 14

- 22. Chow, J.C., S.K. Hoekman, J.M. Norbeck, K.N. Black, R.M. O’Keefe, D.L. Kopinski, M.P. Walsh, J.L. Suchecki, S.L. Altshuler, B. MacClarence, R.A. Harley, and D. Marrack (2001). Diesel Engines: Environmental Impact and Control. J. Air and Waste Management. Assoc

. 51

, 1258

- 1270. Hoekman, S.K., R.S. MacArthur, M. Naylor and J.A. Rutherford (1998). RVP Reduction for Control of Wintertime CO. Soc. Aut o. Eng. Tech. Paper No. 981373. Lanning, L.A., W.L. Clark, W.O. Siegl, S.K. Hoekman, R.M. Stanley and W.F. Biller (1997). CRC Hydrocarbon Emissions Analysis Round Robin Test Program, Phase II. Soc. Auto. Eng. Tech. Paper No. 971608.
Hoekman, S.K., J. Free l and R.S. MacArthur (1996). Reduced

- RVP Gasoline

– An Attractive Alternative to RFG. Soc. Auto. Eng. Tech. Paper No. 961281.Page 27



### Hoekman, S.K., R.M. Stanley, W.L. Clark, W.O. Siegl, A.M. Schlenker and W.F. Biller (1995). CRC Speciated Hydrocarbon Emissions A nalysis Round Robin Test Program. Soc. Auto. Eng. Tech. Paper No. 950780. Hoekman, S.K. and T.E. Jensen (1993). Methanol Vehicle Emissions Round Robin Test Program. Soc. Auto. Eng. Tech. Paper No. 932773. Hoekman, S.K. (1993). Improved Gas Chromatography Procedure for Speciated Hydrocarbon Measurements of Vehicle Emissions. J. Chromatog

., 639 , 239

- 253. Hoekman, S.K. (1992). Speciated Measurements and Calculated Reactivities of Vehicle Exhaust Emissions from Conventional and Reformulated Gasolines. Envir on. Sci. Technol

. 26 , 1206

- 1216. Croes, B.E., S.K. Hoekman and A. Guerrero (1991). Ozone

- Forming Potential of Emissions from Lower Aromatic Content Gasolines. Air and Waste Management Association Paper No. 91

- 107.8. Gething, J.A., S.K. Hoekman, A.R. Guerr ero and J.M. Lyons (1990). The Effect of Gasoline Aromatics Content on Exhaust Emissions: A Cooperative Test Program. Soc. Auto. Eng. Tech. Paper No. 902073. Eberhard, G.A., M. Ansari and S.K. Hoekman (1990). Emissions and Fuel Economy Tests of a Methanol Bus with a 1988 DDC Engine. Soc. Auto. Eng. Tech. Paper No. 900342. Beyaert, B.O., S.K. Hoekman, A.J. Jessel, J.S. Welstand, R.D. White and J.E. Woycheese (1989). An Overview of Methanol Fuel Environmental, Health, and Safety Issues. Summer Meeting of A merican Institute of Chemical Engineers, Philadelphia, PA, August 22, 1989. Eberhard, G.A., M. Ansari and S.K. Hoekman (1989). Emissions and Fuel Economy Test Results for Methanol

- and Diesel

- Fueled Buses. Air and Waste Management Association Paper 89

- 9.4

. Horn, J.C. and S. K. Hoekman (1989). Methanol

- Fueled Light

- Duty Vehicle Exhaust Emissions. Air and Waste Management Association Paper No. 89

- 9.3. Hoekman, S.K. and M.C. Ingham (1987). Measurement of PAH and Nitro

- PAH from a Heavy

- Duty Diesel Engine. Ai r Pollution Control Association Paper No. 87

- 1.4. Wall, J.C. and S.K. Hoekman (1984). Fuel Composition Effects on Heavy

- Duty Diesel Particulate Emissions. Soc. Auto. Eng. Tech. Paper No. 841364. Seizinger, D.E. and S.K. Hoekman (1984). Aromatic Measuremen ts of Diesel Fuel

- A CRC Round

Robin Study. Soc. Auto. Eng. Tech. Paper No. 841363. Barton, T.J., Hoekman, S.K., Burns, S.A. (1982). Comments on the Formation of Silanones in the Thermolysis of Hydridosilyl Peroxides. Organometallics 1 , 721

- 725.
Barton, T.J., Hoekman, S.K. (1980). Bis(trimethylsilyl)diazomethane, Trimethylsily trimethylgermyl diazomethane, and Bis(trimethylgermyl)diazomethane

– Synthesis and Chemistry of Quantitative Silene and Germene Precursors. J. Amer. Chem. Soc. 102 , 1584

- 1591.

Bart on, T.J., Hoekman, S.K. (1979). Convenient Synthesis of Trimethylsilyldiazomethane

– Silene Generator. Synthesis and Reactivity in Inorganic and Metal

- Organic Chemistry 9 , 297

- 300.Page 28



### Final Technical Reports Broch, A., and S.K. Hoekman. (2012) “Transportat ion Fuel Life Cycle Analysis: A Review of Indirect Land Use Change and Agricultural N 2 

### O Emissions.” CRC Final Report No. E

- 88

- 2, January 2012. (156 pp) Hoekman, S.K., C. Robbins, and X. Wang. (2011) “Dilution Sampling System for Biomass

- Derived Syngas.” Fi nal report to DOE under DE

- FG30

- 08CC00057, January 30, 2011. (51 pp) Hoekman, S.K., A. Broch, C. Robbins, and E. Ceniceros. (2011) “Investigation of Biodiesel Chemistry, Carbon Footprint and Regional Fuel Quality.” CRC Final Report No. AVFL

- 17a, January, 2 011. (256 pp) Hoekman, S.K. (2011) “Thermal Treatment of Biomass.” Final Report for Subtask 1.2 of the Nevada Renewable Energy Consortium, January, 2011. (30 pp) Hoekman, S.K. (2010) “Renewable Energy Center

– Desert Research Institute: Phase II.” Submitte d to NREL, October 13, 2010. (109 pp) Hoekman, S.K. (2010) DRI

- REC

- II Task 3A. “Enhance Biofuels/Biomass Testing Capabilities: Sampling and Analysis of Thermal Conversion Products.” Submitted to NREL, March 31, 2010. (17 pp)
Hoekman, S.K., C. Robbins, and X. Wang. (2010) “Gridley Biofuels Project Final Report.” Submitted to REII, July 21, 2010. (63 pp) Robbins, C., X. Wang, and S.K. Hoekman. (2010) “Recovery Act: Solar Reforming of Carbon Dioxide to Produce Diesel Fuel.” Submitted to REII, October 30, 2010. (24 pp) Hoekman, S.K., A.W. Gertler, A. Broch, and C. Robbins, (2009) “Investigation of Biodistillates as Potential Blendstocks for Transportation Fuels,” CRC Project No. AVFL

- 17, Report to Coordinating Research Council, June 2009 (289 pages)
Hoekman, S.K . (2008) “DRI

- REC Phase I Final Report,” submitted to NREL, under Subcontract No. ADO

- 5

- 44431

- 06, July 21, 2008. (18 pages) Hoekman, S.K., A. Broch, C. Robbins, and R. Purcell. (2008) “Renewable Energy Production via Carbon Capture and Recycling,” submitte d to RCO2, Nov. 3, 2008. (29 pages) Reinhardt, R., and S.K. Hoekman. (2007) “Hawaiian Island Gasoline Volatility Study: Analysis of 90 th 

### Percentile Maximum Temperature,” Submitted to Chevron Products Company, August 3, 2007. (34 pages).
Gertler, A. and S.K . Hoekman (2007) “Review of NAC 590.065: Overview and Implications of Gasoline Volatility Rule Change,” Final Report submitted to Nevada Department of Agriculture, February 14, 2007. (60 pages)Page 29 

## Appendix B : Conflict of Interest Statements

Conflict of Interest and Bias for Peer Review

Background

Identification and management of potential conflict of interest (COI) and bias issues are vital to the successes and credibility of any peer review consisting of external experts. The questionnaire that fol lows is consistent with EPA guidance concerning peer reviews. 2

. 

### Definitions

Experts in a particular field will, in many cases, have existing opinions concerning the subject of the peer review. These opinions may be considered bias, but are not necessari ly conflicts of interest.

Bias : For a peer review, means a predisposition towards the subject matter to be discussed that could influence the candidate's viewpoint.

Examples of bias would be situations in which a candidate:

1. Has previously expressed a position on the subject(s) under consideration by the panel; or

2. Is affiliated with an industry, governmental, public interest, or other group which has expressed a position concerning the subject(s) under consideration by the panel.

Conflict of Int erest : For a peer review, as defined by the National Academy of Sciences, 3 

### includes any of the following:

1. Affiliation with an organization with financial ties directly related to the outcome;

2. Direct personal/financial investments in the sponsoring organization or related to the subject; or

3. Direct involvement in the documents submitted to the peer review panel

... that could impair the individual's objectivity or create an unfair competitive advantage for the individual or organization.

2 U.S. EPA (200 9 ). Science Policy Council Peer Review Handbook

. OMB (2004). Final Information Quality Bulletin for Peer Review. 3 NAS (2003). "Policy and Procedu res on Committee Composition and Balance and Conflict or Interest for Committees Used in the Development of Reports" (www.nationalacademies.org/coi).Page 30



### Policy a nd Process

● Candidates with COI, as defined above, will not be eligible for membership on those panels where their conflicts apply.

● In 

### general, candidates with bias, as defined above, on a particular issue will be eligible for all panel memberships; however, extr eme biases, such as those likely to impair a candidate's ability to contribute to meaningful scientific discourse, will disqualify a candidate.

● Ideally, the composition of each panel will reflect a range of bias for a particular subject, striving for ba lance.

● Candidates who meet scientific qualifications and other eligibility criteria will be asked to provide written disclosure through a confidential questionnaire of all potential COI and bias issues during the candidate identification and selection p rocess.

● Candidates should be prepared, as necessary, to discuss potential COI and bias issues.

● All bias issues related to selected panelists will be disclosed in writing in the final peer review record.Page 31 

### Conflict of Interest and Bias Questionnaire

Fuel Sulfur Effects Report Peer Review

Instructions to Candidate Reviewers

1. Please check YES/NO/DON'T KNOW in response to each question.

2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the circumstances.

3. Please make a reasonable effort to answer accurately each question. For example, to the extent a question applies to individuals (or entities) other than you (e.g., spouse, dependents, or their employers), you should make a reasonable inquiry, such as emailing the q uestions to such individuals/entities in an effort to obtain information necessary to accurately answer the questions.

Questions

1. Are you (or your spouse/partner or dependents) or your current employer, an author, contributor, or an earlier reviewer of the document(s) being reviewed by this panel?

YES___ NO_ X

_ DON'T KNOW___

2. Do you (or you spouse/partner or dependents) or your current employer have current plans to conduct or seek work related to the subject of this peer review following the comple tion of this peer review panel?

YES___ NO___ DON'T KNOW_ X

_
3. Do you (or your spouse/partner or dependents) or your current employer have any known financial stake in the outcome of the review (e.g., investment interest in a business related to the sub ject of peer review)?

YES___ NO_ X

_ DON'T KNOW___

4. Have you (or your spouse/partner or dependents) or your current employer commented, reviewed, testified, published, made public statements, or taken positions regarding the subject of this peer review?

YES___ NO_ X

_ DON'T KNOW___Page 32



### 5. Do you hold personal values or beliefs that would preclude you from conducting an objective, scientific evaluation of the subject of the review?

YES___ NO_ X

_ DON'T KNOW___

6. Do you know of any reason that you might be unable to provide impartial advice or comments on the subject review of the panel?

YES___ NO_ X

_ DON'T KNOW___

7. Are you aware of any other factors that may create potential conflict of interest or bias issues for you as a member of the panel?

YES___ N O_ X

_ DON'T KNOW___

Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I further declare th at I have made reasonable effort and inquiry to obtain the information needed to answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change in circumstances that would require me to revise the answers that I have provide d.

Tom Durbin

______________ Name

11/14/2011____ Signature DatePage 33 

### Conflict of Interest and Bias Questionnaire

Fuel Sulfur Effects Report Peer Review

Instructions to Candidate Reviewers

1. Please check YES/NO/DON'T KNOW in response to each questi on.

2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the circumstances.

3. Please make a reasonable effort to answer accurately each question. For example, to the extent a question applies to individuals (or entities) other than you (e.g., spouse, dependents, or their employers), you should make a reasonable inquiry, such as emailing the questions to such individuals/entities in an effort to obtain information necessary to accurately answer the questions.

Questions

1. Are y ou (or your spouse/partner or dependents) or your current employer, an author, contributor, or an earlier reviewer of the document(s) being reviewed by this panel?

YES___ NO__+_ DON'T KNOW___

2. Do you (or you spouse/partner or dependents) or your curre nt employer have current plans to conduct or seek work related to the subject of this peer review following the completion of this peer review panel?

YES___ NO__+_ DON'T KNOW___

3. Do you (or your spouse/partner or dependents) or your current employer h ave any known financial stake in the outcome of the review (e.g., investment interest in a business related to the subject of peer review)?

YES___ NO __+_ DON'T KNOW___

4. Have you (or your spouse/partner or dependents) or your current employer commented

, reviewed, testified, published, made public statements, or taken positions regarding the subject of this peer review?

YES___ NO __+_ DON'T KNOW___Page 34



### 5. Do you hold personal values or beliefs that would preclude you from conducting an objective, scientifi c evaluation of the subject of the review?

YES___ NO__+_ DON'T KNOW___

6. Do you know of any reason that you might be unable to provide impartial advice or comments on the subject review of the panel?

YES___ NO __+_ DON'T KNOW___

7. Are you aware of a ny other factors that may create potential conflict of interest or bias issues for you as a member of the panel?

YES___ NO__+_ DON'T KNOW___

Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I further declare that I have made reasonable effort and inquiry to obtain the information needed to answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change in circumstances that would require me to revise the answers that I have provided.

Ronald M. Heck

____________ Name

__________________________ 1/21/2012

______ Signature DatePage 35 

### Conflict of Interest and Bias Qu estionnaire

Fuel Sulfur Effects Report Peer Review

Instructions to Candidate Reviewers

1. Please check YES/NO/DON'T KNOW in response to each question.

2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the circumstances.

3

. Please make a reasonable effort to answer accurately each question. For example, to the extent a question applies to individuals (or entities) other than you (e.g., spouse, dependents, or their employers), you should make a reasonable inquiry, such as e mailing the questions to such individuals/entities in an effort to obtain information necessary to accurately answer the questions.

Questions

1. Are you (or your spouse/partner or dependents) or your current employer, an author, contributor, or an earlie r reviewer of the document(s) being reviewed by this panel?

YES___ NO_ X

__ DON'T KNOW___

2. Do you (or you spouse/partner or dependents) or your current employer have current plans to conduct or seek work related to the subject of this peer review follow ing the completion of this peer review panel?

YES___ NO_ X

__ DON'T KNOW___

3. Do you (or your spouse/partner or dependents) or your current employer have any known financial stake in the outcome of the review (e.g., investment interest in a business rela ted to the subject of peer review)?

YES___ NO_ X

__ DON'T KNOW___

4. Have you (or your spouse/partner or dependents) or your current employer commented, reviewed, testified, published, made public statements, or taken positions regarding the subject of th is peer review?

YES___ NO_ X

__ DON'T KNOW___Page 36



### 5. Do you hold personal values or beliefs that would preclude you from conducting an objective, scientific evaluation of the subject of the review?

YES___ NO_ X

__ DON'T KNOW___

6. Do you know of any reason t hat you might be unable to provide impartial advice or comments on the subject review of the panel?

YES___ NO_ X

__ DON'T KNOW___

7. Are you aware of any other factors that may create potential conflict of interest or bias issues for you as a member of th e panel?

YES___ NO_ X

__ DON'T KNOW___

Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I further declare that I have made reasonable effort and inquiry to obtain the information needed to answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change in circumstances that would require me to revise the answers that I have provided.

S. Kent Hoekman

_____________ Name

__ ____ December 14, 2011____ Signature DatePage 37 

## Appendix C : Peer Review Charge

Sulfur in gasoline has long been known to reduce the efficiency of automobile exhaust emission aftertreatment syst ems. Some emission studies have suggested an increase in catalyst sensitivity to sulfur (i.e

. , binding to active catalytic sites) with increasing stringency of vehicle exhaust emission standards. This may be a result of higher catalytic efficiencies requ ired for compliance with recent emission standards. Though, historically, light

- duty vehicle emission standards have been high enough to mask any impact of fuel sulfur level as negligible. EPA, in promulgating its Tier 2 light

- duty vehicle emission stand ards, recognized the importance of fuel sulfur level. Reductions to new vehicle exhaust emission standards under the Tier 2 vehicle and fuels program were accompanied with corresponding reductions in fuel sulfur level to improve the cost and feasibility o f the vehicle technology changes required for compliance.

In the 2005/2006 timeframe, EPA’s Mobile Source Air Toxics/MSAT

- 2 study 4 

### examined the effects of fuel sulfur level on exhaust emissions from a test fleet of nine Tier 2

- compliant cars and trucks. The study examined four gasolines, without ethanol and blended in a step

- wise manner from a base fuel at 6ppm S to the final fuel at 32 ppm S. FTP

- weighted emission reductions related to fuel sulfur changes in this program for exhaust emissions were all

s tatistically significant (α = 0.9). However, due to specific catalyst prep procedures that compared a fully cleaned

- out catalyst with low sulfur fuel to a fully

- loaded catalyst with high sulfur fuel, these results were thought to represent a “worst case” only. The data, though, suggested reversible sulfur loading on Tier 2 vehicle catalysts and likely significant emission reductions through further fuel sulfur level control.

The test program described in this report used two fuels with properties identi cal to conventional federal certification gasoline, except that the “lower sulfur” fuel contained a nominal 5 ppm sulfur. The “higher sulfur” fuel had a level similar to the national average in

- use fuel at approximately 30 ppm sulfur. The types of data t argeted by the program were the level of “reversible” loading in catalysts found in

- use Tier 2 vehicles and the relative emission differences due to sulfur re

- loading for two different sulfur level fuels in the same vehicle. In order to generate these emi ssion data, privately

- owned in

- use vehicles were recruited for the study. It was not feasible however to damage or destroy catalysts to directly measure any sulfur loading and, so, the behavior of emissions relative to a baseline was used as a proxy for c atalyst sulfur loading.

The report to be reviewed contains information on the hypothesis, design, and execution of the test program as well as an in

- depth statistical analysis of the results. EPA is seeking the reviewers’ expert opinion on all aspects of the report and, in particular, the methodologies used to evaluate the effects of both “cleanout” and sulfur level on in

- use Tier 2 vehicle emissions and whether these techniques are likely to yield accurate results. EPA asks the reviewer to also consider the appropriateness of the statistical techniques used to analyze the data described in

4 ““Proposed Rule: Control of Hazardous Air Pollutants from Mobile Sources” Preamble and Regulations (publ ished March 29, 2006). See http://www.epa.gov/oms/toxics.htm

.Page 38



### this report, their appropriateness in the context of data accuracy and quality issues and the overall conclusions drawn in this report pertaining to reducing fuel sulf ur levels in the light

- duty fleet.

To that end, reviewers should address the following questions, specifically:

1. Was the imputation method used for replacement of measurements with low concentration reasonable? What other alternatives may have been better in this case? (Section 7.1)

2. Please comment on the use of mixed model in analyzing the data. Was the model fitting strategy in selecting the final model statistically sound? (Section 7.2)

3. Were the model assumptions for the covariance struc ture reasonable given the data? (Section 7.3)

4. Please comment on the methodology used in determining the in

- use sulfur effect for models with and without the sulfur and mileage interaction term. (7.3.3)

5. Is the interpretation of the mileage

- by

- sulf ur interaction term presented correctly? (Section 7.3.3)

6. Are the sensitivity analyses on effects of low

- level concentrations, imputed values, and influential vehicles sufficient as performed? Do the results from the sensitivity analyses provide addit ional support for the robustness of the conclusion? (Section 7.3.4)Page 39 

## Appendix D: Reviews

Comments on Fuel Sulfur Effects

January 25, 2012

RE: Peer review comments on in

- use sulfur effects study

Dear Mr. Menard:

Please find attached my comments on the document “The Effects of Fuel Sulfur Level on Emissions from the In

- Use Tier 2 Vehicles ”. I reviewed the document itself, as well as the underlying statistical concepts used in the analysis. I work as a research engineer for the University of Californ ia at Riverside, CE

- CERT and have no real or perceived conflict of interest related to this evaluation. I have considerable expertise in emissions testing and have conducted several major emissions test programs related to gasoline fuel properties, and the analysis of such data. Please let me know if you would like further information or have further questions relating to my comments, or would like to discuss the comments via a conference call.

Regards, Tom Durbin, Ph.D Research Engineer University of Cali fornia CE

- CERT Riverside, CA 92521Page 40 

### Comments on

US EPA Report “Effects of Fuel Sulfur Level on Emissions from the In

- Use Tier 2 Vehicles”

This report presents an interesting approach to estimating the real effects of fuel sulfur in the Tier 2 in

- use f

leet. The study was very extensive in terms of the number of vehicles and level of testing that was done, and the results of the study some potentially important impacts of differing in

- use sulfur levels. It would be interesting to evaluate these results f urther to determine the modeled impact of in

- use fuel sulfur levels on ambient quality. The analysis methodologies in the report appear to be statistically sound, and appropriate for repeated measures types of testing. The presentation of the data is relat ively extensive in terms of the descriptions of the statistical analysis procedures used. As such, the report would relatively straightforward to read for the some with a moderately strong background in statistics, but might a little too technical for some readers without a strong statistical background. Detailed comments related to this report are provided below in three different areas 1) general technical content 2) editorial comments, and 3) answers to assigned peer review questions.

1. Executive Sum mary. No significant comments outside of editorial comments provided below.

2. Introduction

– See editorial comments below. Also, it might be useful to discuss the Umicore study cited in the Executive summary, since usually the executive summary is more cond ensed than the Background.

3. Study Design

Was the fuel commercially available in the Ann Arbor area at the time of the study oxygenated or not. This is of relevance since the test fuel is non

- oxygenated.

30 ppm is given as the in

- use sulfur average. Is t his based on actual data of in

- use sulfur samples, or is this based on regulatory average requirements.

In discussing why the US06 x2 clean

- out cycle was used, it might be worth discussing the clean

- out procedures that were used in the MSAT and Umicore st udies..

4. Test Vehicle Selection, Recruitment, and Delivery

What is the average sulfur level for the as received vehicles.

Section 4.1 should specifically mention that 19 makes/models were utilized in the study.

Was any attempt made to characterize the v ehicle categories by other factors such as level of long term fuel trim, or whether the vehicle used a closed coupled or double catalyst or only an underbody, or catalyst metals.

What incentive was provided to owners to participate in the program?

– add t o section 4.2.Page 41



### More information should be given about the number of vehicles targeted for testing, and the number of vehicles in each vehicle make/model category. The first mention should be here, as opposed to in section 6.2. In that section, it talks ab out 5 vehicles for each make/model class. It would be useful to say the total number of vehicles tested at this point.

5. Test Fuel Specs and Procurement

Some more details should be included about the fuel selection would be useful. How does this fuel c ompare to typical in

- use fuel, in terms of say aromatics and RVP. Were oxygenates in use in the recruitment area? Or compared to averages of in

- use used throughout the US?

6. Test Procedures

It would be useful to add in fuel change points into the test procedure flow charts.

There is some lack of clarity in reading through this section that can be shown by looking at the last paragraph. The last paragraph indicates that only 4 of 19 family used the modified short procedure, whereas the first sentence o f the paragraph indicates that the change was made approximately halfway through the program. Then looking at Table 7.7, it indicates that between 2

- 4 vehicles in each make/model class were given the modified short procedure, whereas it seems to imply in t he paragraph that once the change was made, it was applied thereafter. Was the change just made for a subset of the vehicles tested in each vehicle category. Also, in the last sentence it indicates that the change in the number of vehicles providing “sulfu r level data” is seen in Table 7

- 7. Does this mean that the modified short vehicles were added to the “sulfur level data” as well as the Long or modified Long procedure vehicles.

Related to the above comment, why not simply just say at the end of each p rocedure description how many vehicles were tested on each procedure.

Figure 6

- 3 indicates that only the 28 ppm sulfur level data is used for the “clean out effect”, but the very first sentence under section 7 indicates that there is a set of “clean

- out” data at 5
ppm.

7. Data Analysis and Results

The comments for this section are provided in a combination of some comments here, which are more related to the presentation, flow, or missing information in the section. At the end of this document, additi on information is provided to address the questions specifically provided under the reviewer,

p. 21

– 3 rd 

### paragraph

– A statement should be added to answer the following question

– Why do the results of the Bag 2 NOx emissions have more substantial impli cations than those of other pollutants/bags?Page 42



### While it is useful to discuss the results at low concentrations, further information on the experimental methods might merit a section. For example, what analyzer are used? Can the detection limits of the analy zers be included in an Appendix? What methods/microbalance was used for the PM measurements, etc.

p. 24

– top of page. There should be a brief paragraph discussing Table 7

- 1 and the number of zero value measurements before discussing the imput value metho dologies.

P. 24

– last sentence

– It says that the statistical analyses were run both with and without imputed values. In terms of the “without” input value case, does that mean that the values were eliminated, were zeros used as the imput values, or wer e the original values used, whether they are positive or negative.

p. 25

– second paragraph

– A new terms vehicle “families” is introduced. Presumably this term is the same as the vehicle make/models used earlier. Presumably these also have the same engin e family and this should be included along with engine size, vehicle configuration, and weight.

p. 30

– There is no discussion on the symbols in the plots presumably the plots show the average and median inside the bar. Then the bars represent the 95% con fidence levels and the error bars the full data range. Then there are some other dots on the plot?

p. 30

– It is not clear what is meant by “Some vehicle families show the presence of within

family variability” Presumably this could also be due to differ ences in in

- use sulfur levels as well, since a handful of vehicles came in with in tank fuels with sulfur levels much higher than those for others. For the post

- cleanout variability, it seems like this should be independent of the in

- use sulfur loading. Co uld some of the variability be related to the condition of the vehicle? Vehicle family M504 seems to show a very variability post

cleanout.

P. 32

– Related to BIC criterion. It indicates that the BIC performs relatively better for small sample size settin gs. How would one define 5 here? Is the current experiment a small sample size?

Figure 7

- 4 does not appear to add significant value. Much of the same information is available in the box plots in Figure 7

- 5, and that is a little easier to read. It talks a bout the vehicles with similar emissions profiles being grouped. However, it is not immediately obvious what grouping were used for the Figures 7

- 4 and 7

- 5.

8. Summary and Conclusions

Under the first bullet point. Performing a clean

- out cycle with a 28 p pm fuel

The second bullet point could be clearer. For example: for a subset up vehicles tested on both 5 ppm and 28 ppm fuels, it was found that additional FTP composite reductions of 18%Page 43



### for NO x 

### , 9% for NMHC, and 8% for CO were for the 5 ppm fuel in comp arison with the 28 ppm fuel.

9. Grammar and other editorial things.

--- Executive summary

– the number of spaces between sentences differs from to in some cases 3 or even 4. This should be made consistent.

--- p. 2

– The end of the first sentence and sta rt of the second redundantly talk about catalyst efficiency.

--- p. 2

– The final sentence of the first paragraph should be split into two sentences.

--- p. 2

– Second to last sentence

-- rewrite... for a PZEV operated on a 3 ppm fuel compared to a 33 ppm fu el.

--- p. 3

– third sentence

– In response

.... To address this question, the EPA (spell out EPA first use)

--- p. 3

– last sentence

– split it

– performance . F or Example,

--- p. 4

– 2 nd 

### line

– fuel s (add s); Also split the second to last sentence.

--- p. 5

– need space for 5 ppm and 28 ppm, and period after “overall”. This sentence should probably also be split.

--- p. 5

– 3 rd 

### sentence

– rewrite

– a significant miles

- by

- sulfur interaction was not found from the model fitting....

--- p. 5

– last sentence

– re write

– In this case, the relatively differences with sulfur level varied as a function of mileage, so determination.....

--- p. 7

– 1 st 

### sentence

– “has long been shown” is ackward. Also in executive summary.

--- p. 8, 9, and 10

– comma after e.g. ,

--- p. 8

– comma after idles) , comma after in 2005 ,

--- p. 9

– 1 st 

### line

– spell out MSAT acronym; line 7

– word “specimens” is a bit ackward; 6 th 

### line form bottom “benefits of further sulfur control”

--- p. 11

– about line 10 comma after high temperatures), lin e 15 “conditions that are favorable” line 16 lines further down “A vehicle with a relatively..., reference 10 on the E

- 60 report is not referenced properly and should contain more author names.

--- p. 12

– 1 st 

### line operation , (add comma); 1 st 

### line of section 4.1 “ with the intent of being to be representative of the latest...”;

--- p. 12

– last sentence table 4

- 1 should not be underlined. This same changed should be incorporated throughout the report, as it appears there is something in the formatting underlini ng table and figure references in the text.

--- p. 13

– First line

– “Vehicles recruited for testing were targeted to have a mileage between 12,000 and 40,000 miles and an age of less than three years old.”

--- p. 14

– line 5 up front bottom

– exhaust l eaks , (add comma)

--- p. 16

– First full paragraph

– Following the fuel change ,

--- p. 16

– last sentence

– rewrite to “The Long and Short procedures are shown in Figure 6

- 1 and are discussed in greater detail below. The Long and Short procedures are iden tical in structure for the first six emissions tests.Page 44



### --- p. 17

– third line

– start a new paragraph about the Long procedure.

--- p. 20

– last sentence

– Split into two sentences.

--- p. 21

– 3 rd 

### paragraph

– detail

. but t The

--- p. 27

– 2 nd 

### paragraph lin e 4 “vehicles which was were “...., later in same paragraph Besides this , (add comma), 3 rd 

### paragraph line 2

– effects , (add comma)

--- p. 28

– line right after equation

- respectively , (add comma) 3 rd 

### paragraph

– line 2

– model , (add comma)

– line 4 structur e , (add comma)

--- p. 29

– 1 st 

### paragraph line 3 exchange “were” for was

- line 4

– levels , (add comma)

-

--- p. 32

– last paragraph line 3

– covariance , (add comma)

--- p. 33 line 6

– selected , (add comma)

--- p. 35

– last paragraph 1 st 

### line

– “dataset in clude d 17” ......

--- p. 38

– 2 nd 

### line from bottom

– Section 7.3.3 , (add comma)

--- p. 40

– 3rd line from bottom

– i.e. , (add comma)

--- p. 40

– 5th line from bottom

– rewrite

- considering each vehicle as a random effect s might be useful

--- p. 42

– The sentence that includes Figure 7

- 6 should be broken up into two sentences. Example: This is shown in figure 7

- 6, which shows the log

- transformed emissions from individual vehicles by sulfur level.
5 th 

### sentence from bottom

– add comma ... some vehicles , and s uggests....

--- p. 44

– 2 nd 

### line from bottom

– add comma ... vehicle to vehicle , and simple descriptive statistics...

--- p. 45

– 1 st 

### line

– rewrite .. a similar top

- down model fitting statistical approach was applied to the clean

- out data, as described in Sect ion 7.3.1.

--- p. 45

– 2 nd 

### sentence

– rewrite .. Furthermore, additional analysis was done that used a subset of the sulfur level dataset that isolates the emission measurements immediately following the clean

- out to address the effectiveness of the clea n

- out cycle in reducing emissions.

--- p. 46

– 2 nd 

### paragraph 4 th 

### sentence

– add comma ... irregularly , where

--- p. 46

– 3 rd 

### paragraph 2 nd 

### sentence

– ....exponentially with time , i.e., the variability....

---- p. 50

– 2 nd 

### to last line

– operation , (add comma) causing an increase in emissions

--- p. 58

– 1 st 

### paragraph

– line 3 through the presence, line 4 (RLD), (add comma), line 5 after an iterative...

--- p. 58

– 1 st 

### paragraph

– line 3 through the presence, line 4 (RLD), (add comma), line 5 after an iterativ e...

--- p. 59

– 2 nd 

### paragraph under “summary and conclusions”

- ...US06 cycles . This data was used to examine the existence...Page 45



### Peer Review Charge Questions

1. Was the imputation method used for replacement of measurements with low concentration reasonable? What other alternatives may have been better in this case? (Section 7.1)

The text talks about tailpipe emissions being greater than zero, while the actual question of relevance may be

– Are the tailpipe emissions greater than the ambient background

conce ntrations? (presuming a relatively clean background) If the vehicle’s emissions is below background levels, than it would not be making a additive contribution to pollution levels. In this case, you could get a distribution of both positive and negative va lues fluctuating around zero.

My concern with the imput value procedure is that it might introduce a slight positive bias. My preference would probably be to use zeros as the imput values, since this would represent a distribution of positive and negative values that would average to zero over a larger dataset. For the cases where the dilute exhaust concentrations are lower than the measured background. By eliminating, or essentially not allowing, negative or zero values, doesn’t this add a positive bias to the results. Or how do you ensure there is a not a positive bias.

p. 24

– first paragraph

– its states that a data point can either be deleted or replaced with an imput value. Couldn’t the value just simply be left as is? Also, if an input value is del eted for a particular bag, presumably no composite emissions would be available. Would the corresponding test be eliminated in its entirety?

p. 25 last line

– its unclear how an outlier could be an input value. Is this a case where the input value is a v ery low value? A description of what these values might be should be added to the text. Again, if the imput outlier value is eliminated, how does that impact the result of the test?

Can Table 7

- 2 be modified to include not only the number of outliers iden tified, but also the number that were eliminated? This should also be in the text in the paragraph for section 7.1.2.

2. Please comment on the use of mixed model in analyzing the data. Was the model fitting strategy in selecting the final model statisticall y sound? (Section 7.2)

The use of a mixed model is a relatively standard method to treat data analysis of emissions, and appears to be appropriate for the study design given here. The fitting strategy of using a saturated model, then developing the most o ptimal covariance structure, and then fitting the final model appears to be statistically sound. The step

- wise backward elimination approach for the “Sulfur Effects data” also appears to be statistically appropriate.Page 46



### One thought relating to the discussion of the univariate ANOVA and the multivariate MANOVA in section 7.2. Overall, it does not seem to add significant value to the report. Looking at the audience for this report, it would likely be composed of a mix of readers that may or may not have a stati stical background. For those with a statistical background, the additional information on the univariate ANOVA and the multivariate MANOVA would not provide significant value. For those that are not as familiar with statistical methods, the added informati on would likely be more confusing than clarifying. It is suggested that this information, although interesting, might be better placed in an Appendix.

3. Were the model assumptions for the covariance structure reasonable given the data? (Section 7.3)

Fo r the “Clean

- out data”, an unstructured covariance structure was initially used (which did not converge), and then a compound symmetry structure was selected. The compound symmetry structure is appropriate since the measurements from the same vehicle shoul d have a homogeneous variance, as included in the text. For the “Effect of Sulfur data, where multiple measurements are made at different mileage accumulations, the autoregression covariance structure is appropriate since the since the correlations between measurements is expected to decline as the measurements are further apart in terms of mileage accumulation, as mentioned in the text of the report.

4. Please comment on the methodology used in determining the in

- use sulfur effect for models with and with out the sulfur and mileage interaction term. (7.3.3)

Overall, the methodology for determining the in

- use sulfur effect for models with and without the sulfur and mileage interaction term appears to be sound. For cases where the sulfur level and mileage in teraction term was not significant, the sulfur effect, sulfur loading and associated percentage differences between the high and low sulfur fuels should be constant as a function of increasing mileage. Thus, using the differences in the least squares means from the final model and reverse transforming them could be used to quantify the percentage differences between high and low sulfur for the case where the sulfur level and mileage interaction term was not significant.

For cases where the sulfur level an d mileage interaction term is significant, the rate of sulfur loading between the low and high sulfur fuels would differ as a function of mileage, so a different approach would be needed. The methodology of using the in

- use emission level upon arrival (pre

- cleanout) from a larger clean

- out dataset and projecting it out to the mileage where the two lines intersect (in

- use equivalent loading) seems reasonable for estimating the in

- use sulfur level effect. Some clarification should be given on what is meant by the “larger clean

- out dataset” since it is not immediately obvious what data these actually are. Additionally, how do the incoming sulfur levels for the “larger clean

- out dataset” compare with average in

- use values for the US fleet, or the fleet in differ ent regions of the country.Page 47



### 5. Is the interpretation of the mileage

- by

- sulfur interaction term presented correctly? (Section 7.3.3)

As discussed under question #4, the interpretation of the mileage

- by

- sulfur interaction term does appear to be presented correctly. On page 48, when the interpretation of the interaction term is first being discussed, I would also add in something about the percentage differences in emissions being constant as a function of mileage, just to make things clearer. For example, for the last line of the paragraph just after Table 7

- 9:

In other words, the effect of high fuel sulfur on Bag 2 NO x 

### exists immediately after clean

- out and remains constant on a percentage basis during subsequent driving of a vehicle.

6. Are the sensiti vity analyses on effects of low

- level concentrations, imputed values, and influential vehicles sufficient as performed? Do the results from the sensitivity analyses provide additional support for the robustness of the conclusion? (Section 7.3.4)

Overall, the sensitivity analyses see to add additional support to the robustness of the conclusions. Here are a few thoughts on the sensitivity analyses.

If the sensitivity analyses were only done for the NO x 

### bag 2 emissions, it would be worth mentioning that th e NO x 

### bag 2 emissions showed a higher percentage of measurements with zero values than most other pollutant/bag combinations, as illustrated in Table 7

- 1.

It would be interesting to get some feel for the vehicles/data eliminated in the sensitivity analysi s for the low concentration measurements. Since these would be low emitting vehicles, it seems like they might have more robust and sulfur tolerant catalyst systems. This is consistent with the results showing an increase in the emissions reductions as the se data are pulled out.

Relating to the imput values, it would be interesting to see what the effect would be of making the imput values simply zero. Given the small differences between the model with and without the imput model, there would likely be on ly a minor impact, but this would add to the robustness of the results, although probably not change any conclusions.

For the influential vehicles, it is interesting to look at the vehicles selected. Vehicles 0007 and 0178 show a relatively large sulfur e ffect, whereas vehicle 0046 shows a slight reverse sulfur effect with tight variability. It is interesting that other vehicles with relatively strong sulfur effects, such as 0165, 0179, and 0011, did not have a strong influence. The result of this sensitiv ity analysis is reasonably intuitive, with the NOx differences between sulfur levels shrinking, which would be consistent with removing some vehicles with a stronger sulfur effect.Page 48 

### Report

– Review of The Effects of Fuel Sulfur Level on Emissions from th e In

- Use Tier 2 Vehicles

- DRAFT REPORT

Ronald Heck 2/3/2012

General discussion:

It is very important that the authors discuss right up front that they have taken an approach where the vehicles are considered as “black boxes”. This is a totally dif ferent approach than that adopted by the Umicore paper (Ball D., Clark D., Moser D. (2011). Effects of Fuel Sulfur on FTP NOx Emissions from a PZEV 4 Cylinder Application. SAE 2011 World Congress Paper 2011

- 01

- 0300. SAE International: Warrendale, PA.) In the “black box “ approach there is no consideration of the engine control strategies (rich or lean bias, cold start strategy, etc.), sensor responses (from aging), vehicle operation (e.g. fuel cut), catalyst technologies, catalyst configurations(close cou pled, underfloor, cell density, volume, etc.) See James M. Lyons, David Lax, and Steve Welstand, Investigation of Sulfur Sensitivity and Reversibility in Late

Model Vehicles, SAE 1999

- 01

- 3676 for work that takes a more phenomenological approach to sulfur effects. By the way I could not find this reference in the report. In the “black box” approach very little information is given besides the vehicle type, mileage, age, etc. Because of this lack of information it will be difficult for anyone to repeat t he authors’ experiments and challenge there results. In addition it will be difficult for the authors to explain data anomalies because there are no specifics to tease out a possible explanation. Also since there are so many uncontrolled variables, the s tatistical analysis may be limited since it is only statistics! This is not meant to say the work has no merit only to provide caution in the final conclusions and merits of too much statistics!

One concern I have is if some of the vehicles have a fuel cut strategies. If so, this could confound the results since in the US06 the catalyst with a fuel cut strategy will see lean excursions while a catalyst without will be stoichiometric during the cycle. Again since no information is given in the report on engine operation it is impossible to go back and uncouple this effect.

Sulfur poisoning is a complex phenomenon for catalysts and the catalyst manufacturers have found recipes to compensate for high and low sulfur levels (see Harold N. Rabinowitz, Samue l J. Tauster, Ronald M. Heck, The effects of sulfur & ceria on the activity of automotive Pd/Rh catalysts, Applied Catalysis A: General 212 (2001) 215

– 222). If the catalyst companies can design for Tier 2 vehicles at 30 ppm S, then this study may be in qu estion or to put in another way, maybe some of the vehicles you studied already have these types of technologies and that is why you found some responses that didn’t show an effect of S level!

The removal of sulfur from the catalyst needs a consistent st atement throughout the paper. You show Figure 2

- 1 from Ford but the text is not consistent in describing this removal which is due to temperature, air to fuel ratio, and sulfur level. Also the degree of removal isPage 49



### different for Pt, Pd and Rh and the oxyg en storage materials (Ce, Zr, La, etc.). So this discussion needs a little more technology.

In the report, there are a number of places where symbols are not explained in tables and in text. Also, I did not see an explanation about how one does a hot/c old start FTP. Please make sure this is explained in the text of the report.

At end of papers, please a list of references. I had to go through the entire paper many times to see if a reference was cited. This was painful!

Specific textual comments:

I am including the entire report as a Word document with 27 comments embedded in the
text.DRAFT Page 50 Specific Textual Comments Ron Heck
Page No. Report Text Comment

2 Fuel sulfur content has long been understood to affect the performance of emission after tre atment catalysts in light duty vehicles, where the sulfur and/or its oxides adsorb to the active precious metal sites, reducing the catalyst’s efficiency in destroying harmful pollutants. Fuel sulfur content has long been understood to affect the performan ce of emission after treatment catalysts in light duty vehicles, where the sulfur and/or its oxides adsorb to the active precious metal sites and oxygen storage materials

, reducing the catalyst’s efficiency in destroying harmful pollutants.

2 The quantity of sulfur present on the catalyst at any given time is a function of its temperature and the fuel sulfur level . . . The quantity of sulfur present on the catalyst at any given time is a function of its temperature, air to fuel ratio

, and the fuel sulfur level . . .

2 . . . with elevated catalyst temperature and lower fuel sulfur concentration both reducing sulfur loading. . . . with elevated catalyst temperature , rich of stoichiometric operation, and lower fuel sulfur concentration both reducing sulfur loading.

3 Test fuels were two non

- ethanol gasolines with properties typical of certification fuel, one at a sulfur level of 5 ppm and the other at 28
ppm. Is there any reason to expect different results with ethanol gasolines?

4 [Table ES

- 1] The clean

- out effect is not significant at α = 0.10 when no reduction estimate is provided. Need to explain what p

- value is in the text? Also what is in (?????)?

4 This indicates that the catalyst is not fully desulfurized, even after a clean out procedure, as long as there is sulfur in the fuel. Not sure you can make this conclusion. It may be that the sulfur in fuel equilibrates instantaneously and it is a concentration effect not a desulfurization. To prove this you would have to vary the desulfurization time and see where it reaches steady s tate.

5 [Table ES

- 2] Again what is p value and what is in (....)?

5 This analysis found highly significant reductions for several pollutants, as shown in Error! Reference source not found. ; reductions for Bag 2 NOx were particularly high, estimated at 59 percent between 28ppm and 5ppm overall This analysis found highly significant reductions for several pollutants, as shown in Error! Reference source not found. ; reductions fo r Bag 2 NOx were particularly high, estimated at 59 percent between 28ppm and 5ppm overall

. 5 Other results, such as Bag 1 hydrocarbons, did show a significant miles

- by

- sulfur interaction. Does this mean that the effect never equilibrates? Are you comfor table in saying this?DRAFT Page 51 Specific Textual Comments Ron Heck
Page No. Report Text Comment

7 The amount of sulfur retained by the catalyst is primarily a function of its operating temperature, the active materials and coatings used within the catalyst and the concentration of sulfur oxides in the incoming exhaust gases. Ag ain air to fuel ratio is important as you mention in the next sentence for reducing conditions.

8 However, the temperatures necessary to release sulfur oxides can also lead to thermal degradation of the catalyst over time. No where do you mention that nor mal operation of the vehicle, the catalyst is constantly being exposed to rich/lean condition from the control system using an oxygen sensor. So the catalyst is being regenerated in

- situ form the perturbation around stoichiometric.

11 The level of revers ible in

- use sulfur storage and release (or loading) within an exhaust catalyst system can be assessed by measuring emissions from the vehicle as received, performing a high speed, high load clean

- out cycle, then measuring emissions again. In describing the cleanout it is important to mention the temperature range and the air to fuel ratio. Also depending on the vehicle calibration the degree of richness and time at such could be different from vehicle to vehicle. How is this taken into account for the de gree of cleanout?

11 A vehicle with relatively high exhaust temperature at the catalyst location, and/or significant excess loading of certain platinum group metals (PGM) and other active materials in the catalyst may be relatively insensitive to sulfur l oading regardless of driver behavior. Up to now you have not mentioned the effect of S on the oxygen storage components which will affect NOx and lightoff in cold start. Is there a reason this is not mentioned?

12 This loading continues over time with ve hicle operation and can be observed as an increase in emissions (sometimes referred to in the auto industry as “NOx creep”). Is this due to PGM and/or oxygen storage component deactivation??/

13 Table 4

- 1 Tes t Vehicles Recruited If you look at the Umicore study you will see details about the e3mission control system (close coupled, underfloor, etc.) You give no such detail in this study. Also, they also show a plot of engine lambda during the FTP. This is i mportant background information to possibly explain outliers or unusual results. This study loses significant meaning without this information. You are treating the cars as black boxes and I think this is a mistake! Also, the oxygen sensor response is v ery important as well as it age. This is never mentioned in this entire study.

17 [Figure 6

- 1] Do you explain what a hot/cold start FTP is anywhere in the report??

25 [Table 7

- 1] What does the value in (...) mean?? Please explain.DRAFT Page 52 Specific Textual Comments Ron Heck
Page No. Report Text Comment

35 [Table 7

- 4] The cl ean

- out effect is not significant at α = 0.10 when no reduction estimate is provided. Explain what is in (....)?

38 Furthermore, the reduction in emissions from cleanout shown in Error! Reference source not found. would likely be larger if the low sulfur test fuel at 5 ppm had been used for the cleanout procedure and the tests immediately following the as

- received baseline emissions. This looks like a conjecture. Present evidence?

39 [Table 7

- 6] Explain what is in (......)?

40 For example, vehicle IDs 0011, 0022, and 017 8 clearly show large effect of fuel sulfur level on emissions while the effect is only marginal for vehicle IDs 0123 and 0264. Differences can be A/F ratio, close coupled catalyst, oxygen storage components, temperature history. This is why more informat ion on vehicle characteristics is important to explain unusual results.

46 The BIC value for the first

- order autoregressive structure was 764.90. Did you explain BIC?? If not then do so.

47 [Table 7

- 8] What i s Pr? Explain.

55 [Figure 7

- 10] Is there an explanation as to why these lines diverge and are not parallel?? Need some discussion here from the model.

56 [Table 7

- 11] What is in (....)?

57 [Table 7

- 13] Is “Probt” probability??

59

- 60 Comparing emissions immediately following the clean

- out procedur e on 5 vs. 28 ppm fuel, FTP composite NO x emissions were 18% lower, NMHC 9% lower, and CO 8% lower. To make this conclusion you need to age with 28, clean out with 28 and test with 5 and compare age with 5, cleanout with 5 and test with 5. I went back in text and really didn’t see this discussed in analysis. If I missed it OK otherwise need some discussion in analysis.DRAFT Page 53 

### Review of EPA Draft Report
The Effects of Fuel Sulfur Level on Emissions from the In

- Use Tier 2 Vehicles Report Dated: November, 2011
S . Kent Hoekman

This draft report describes a vehicle test program conducted to assess the emissions impacts of reducing gasoline sulfur levels in Tier 2 passenger cars and light

- duty trucks. The experimental test design was suitable for the intended purp oses, although clearly, other testing procedures could have been used as well. For example, the standard FTP test procedure was used, at 75 F, to determine the emissions results from each vehicle test. Some comments should be offered regarding the suitabil ity of the FTP driving cycle and a 75 F test temperature in providing representative/realistic operating conditions. For the in

- use vehicle fleet, a set of standardized test conditions is clearly necessary to obtain repeatable and statistically significant results. However, the FTP test is now over 35 years old, and 75 F is just one of many temperatures that could be used. The overall quantitative effect of fuel sulfur on emissions as determined in this study would be more convincing if it were shown that s imilar results apply to other realistic operating conditions.

The use of two, back

- to

- back US06 dynamometer cycles was an appropriate choice for catalyst regeneration. Although a somewhat more aggressive cycle might provide more complete desulfurization, potential damage to the vehicle was a significant concern, as explained in this report. Also, the approach of catalyst clean

- out, followed by repeated testing with a single fuel is a valid way to determine a new “equilibrium emissions level” at a specific fuel sulfur level. However, in real

- world driving, such an equilibrium level is never attained, because operating conditions are constantly changing, and fuel sulfur varies from tank

- to

- tank.

It is not clear why this report places so much emphasis on Bag 2 results

– especially Bag 2 NOx. Typically, Bag 2 emissions concentrations are extremely low from properly functioning Tier 2 vehicles. In fact, as shown in this study, it is often difficult to distinguish Bag 2 levels from background levels. Large perce ntage differences between two very small numbers may not be very meaningful. To help put these results in perspective, it would be useful to show the relevant certification emissions levels for these Tier 2 vehicles. Also, something should be said about th e impact of fuel sulfur reduction on fleet

- wide emissions, and how large a reduction this represents in comparison to the entire mobile

- source emissions inventory. In reality, this test program only captured a small portion of the entire in

- use fleet (only Tier 2 vehicles of model years 2007

- 2009).

Some historical perspective about the “FTP bag method” would be helpful here. This sampling and analysis method was developed about 40 years ago, to measure emissions from uncontrolled (or slightly controlled) vehicles. Prior to introduction of emissions control systems, Bag 2 emission levels were routinely 2 orders of magnitude higher than today. Under such conditions, the FTP bag method was perfectly suitable for the intended purpose. Now, with Bag 2 emission levels being nearly indistinguishable from background, a different method should be employed for quantifying vehicle emissions.

In retrospect, it seems unfortunate that only non

- ethanol gasolines were used, as such fuels are increasingly uncommon. It is stated that similar results would be expected from ethanol

- containingDRAFT Page 54 gasolines, but no data are provided to support this. If any subsequent testing has been done with ethanol

- containing gasolines, this should be mentioned.

Overall, the statistical model ing approach seems appropriate. As described in Section 7.2 (pages 26

- 28) the structure and limitations of the emissions dataset make the linear mixed model a good choice for analysis, as this better accommodates missing data, irregularly spaced measuremen ts, and within

- vehicle effects.

Also, the method used to impute emissions values in cases where measured levels were lower than background is probably the best that can be done. The approach of developing an imputation value based upon each specific vehic le family seems better than using a single, fleet

- wide imputation value. The fact that imputed values were applied to only about 20% of the vehicles provides some assurance that this imputation method did not excessively distort the results. This was furth er confirmed by performing the statistical analyses with and without inclusion of the imputed values. One additional test that could be useful is to conduct the statistical analyses using values of zero instead of the imputed emissions values.

A significa nt question about the statistical modeling approach is the failure to include any information about the vehicles’ catalyst formulations, or other catalyst properties. It is not surprising to observe that vehicle type (car vs. truck) was not a significant m odel term, but it is surprising that there is no term related to the emissions control systems used across the range of vehicles. Surely not all catalysts were the same, and different catalysts would be expected to respond to changes in sulfur levels in di fferent ways. This is an area that should be addressed. Inclusion of a catalyst term could result in quite different statistical models.

A number of other specific comments, questions, and suggestions regarding this draft report are offered below. The ord ering of these items is chronological, as they appear in the report.

On page 2 (1 st paragraph) the quantity of sulfur present on the vehicle’s catalyst is said to be a function of temperature and fuel sulfur level. Isn’t the catalyst formulation/metallurg y another important factor? (Catalyst formulation is mentioned on page 7.) What is known about the catalyst formulations used in each of the test vehicles of this study?

It is interesting to note that when reducing fuel sulfur from 28 ppm to 5 ppm, signif icant emissions reductions were observed for all pollutants except PM, (See Tables ES

- 2 and ES

3.) Is there a clear mechanistic reason for this lack of effect for PM? Are the catalyst systems used in Tier 2 vehicles ineffective in reducing PM (and PM precu rsors)? This lack of a PM effect should be mentioned along with the major findings that are shown in bullet form on page 6.

On page 7 (final paragraph) it is stated that the impact of fuel sulfur on emissions was considered negligible under the Tier 0, T ier 1, and NLEV programs. I don’t think this is true. The emissions impacts of fuel sulfur have been well known for a long time, but more severe sulfur reductions were not thought to be a cost

- effective emissions control approach until recently. To provide greater context, it would be useful to include a brief summary of gasoline regulations (including sulfur levels), and how they have evolved over the past 30

- years. Along with this, a history of LD vehicle emissions standards should also be presented.DRAFT Page 55 On page 8, various detrimental effects of hot/rich catalyst operation are described

– including catalyst degradation and increased emissions of PM, NMOG, and CO. However, nothing is said about fuel economy effects

– either here or elsewhere in the report. In general, fuel economy is a concern with any type of fuel or vehicle modification. This topic should at least be mentioned. In addition, the dataset generated in this study provides an excellent opportunity to evaluate the impacts of sulfur reduction on fue l economy, by analyzing CO 2 as another pollutant, along with CO, THC, NOx, and PM.

The 2005 MSAT study, which used nine Tier 2 vehicles, is described on pages 8

- 9. The emissions reductions upon lowering fuel sulfur from 32 to 6 ppm are said to be 33% for NOx, 11% for THC, 17% for CO, and 32% for CH 4 . Are these values simple averages of the nine vehicles, or were they computed by a more sophisticated method? Were similar statistical methodologies used to compute the fuel sulfur effects in both the MSAT stu dy and the present study? Were PM emissions also measured in the MSAT study?

When first describing the two test fuels on page 10, it would be helpful to refer the reader to Table 5

- 1 (page 15), which provides a more complete listing of fuel properties. Al so, did these fuels contain anything unusual in the way of additives; e.g. antioxidants, detergents, dyes, etc.?

In selecting the vehicles, was the ratio of LD trucks/LD passenger cars representative of the in

- use fleet? Table 4

- 1 (page 13) which lists th e 19 makes/models that were recruited, shows that the No. 1 U.S. sales rank vehicle is missing. Was this an oversight, or are the rankings shown in this table incorrect? Notice also that the Toyota Camry is shown as No. 23 in U.S. sales rank. This seems un believably low for what has traditionally been a “top seller.” Also, it would be useful to add a column to Table 4

- 1 to indicate the number of vehicles in each category that were tested in this study, comprising the total test fleet of 81. It might also be useful to indicate here the number of vehicles in each class that underwent the various test procedures: i.e. short procedure, long procedure, modified short procedure, and modified long procedure.

The modified long testing procedure shown in Figure 6

- 2 (page 19) is confusing. The blue box indicating the short procedure shows only two post cleanout FTPs, although the wording still indicates “triplicate FTPs at 28 ppmS.” Also, in describing the modified short procedure on page 20, it is said that “... the ch ange in the number of vehicles providing sulfur level data can be seen in Table 7

- 7 starting with Family ID N513.” It is not clear what this means. Does this table list the vehicle families in chronological testing order, so that the first 13 families list ed were tested using the standard short procedure, and the last 4 families listed were tested using the extended short test? It is also confusing that Tables 7

- 5 and 7

- 7 provide identical information.

How is NMOG different from NMHC? What analytical proce dures were used to measure these two pollutants?

Displaying the background and sample measurements by FTP Bag (as done in Figure 7

- 1 and Appendix B) is very instructive. However, only NOx and THC results are shown in this way; it would be useful to also i nclude CO and PM. It would also be helpful to draw ovals in Figure 7

- 1 to capture all background measurements for each vehicle family shown. AlthoughDRAFT Page 56 it is not possible to tell which sample points correspond to which background points, this graphical appro ach clearly illustrates the problem of very low emissions measurements relative to background in Bag 2. It also begs the question of why background levels of NOx are so variable. Figure 7

- 1 shows that these background concentrations varied by over an order of magnitude for many of the vehicle families tested.

In the determination of outlier data points (described on page 25) what is the rationale for choosing an outlier screening criterion value of ± 3.5 for the studentized residuals? Also, although Table 7

- 2 identifies the number of outlier points for each pollutant/bag, it doesn’t indicate how many of these outlier points were actually excluded from the statistical analyses.

As explained on page 26, log

- transformation of emissions measurements has common ly been used to analyze vehicle emissions data. However, many previous emissions modeling studies have utilized test fleets that included a variety of technology types having a wide range of emission levels. In the present study, only Tier 2 vehicles were used, and the emission levels did not vary drastically across the test fleet. Given this situation, is log

transformation still necessary (and helpful)?

On page 29 it is stated that the average starting odometer reading of the vehicles used to assess the clean

- out effect at 28 ppmS was 31,470 miles. Is this value the average of the 17 vehicle families shown in Table 7

- 3, or the average of all 81 vehicles? Similarly, is the ± value of 1,578 miles the standard deviation of the 17 vehicle families or the 81 i ndividual vehicles? (The same questions pertain to the vehicles used to assess the clean

- out effect at 5 ppmS, described on page 35; and the vehicles used to assess the sulfur effects, described on page 39.)

The box plots shown in Figure 7

- 2 (page 31) are quite informative in displaying the relative variances between the pre

- and post

- cleanout vehicle tests. It would be useful to show similar plots for other pollutants and Bags. Also, a legend should be included to explain the different symbols used in the se plots.

When describing the dependent variable ( Y i ) and effects ( X i and Z

i ) on pages 31

- 32, the reader should be reminded of the mixed model being used, shown as Equation 7

- 1 on page 28. Also, it is stated on page 32 that “The significance of between

- fa mily variation was observed graphically in Figure 7

- 2 ...” But this figure only show Bag 2 NOx results. Was similar between

- family variation observed with other Bags and pollutants?

Figure 7

- 3 (pages 37

- 38) is not easy for the reader to process. I realize t hat the data are lumped by emission level, but displaying the data in three separate charts, over two pages, is confusing and difficult to understand. It would be preferable to show these data on a single page (similar to Figure 7

- 2, page 31) or even in a single chart (similar to Figure 7

- 1, using a logarithmic scale). It would also be better to order the vehicles along the x

- axis in the same way they are listed in Table 7

- 5. Finally, it would be helpful to see more Bags/pollutants presented in this way

– n ot just Bag 2 NOx.

On page 38, the enhanced emissions reduction benefits of the 5 ppmS clean

- out are described as compared to the 28 ppmS clean

- out (comparing Tables 7

- 4 and 7

- 6). It isDRAFT Page 57 interesting to note that this does not apply to PM emissions. Some ex planation should be offered as to why PM emissions show such a different behavior. Also, the last sentence on page 38 mentions confirmation of results that have not even been presented yet. The placement of this sentence seems odd.

The same comments given above regarding data display in Figure 7

- 3 apply to Figure 7

- 4 (pages 41

- 42). The current grouping of vehicles shown in Figure 7

- 4 makes it very difficult to see the important point being made on page 40 that some vehicles within the same family had marke dly different emission profiles. This would be much easier to see if a single chart were used to display all vehicles

– similar in structure to Figure 7

- 1. Also, the box plots in Figure 7

- 5 (pages 43

- 44) should all appear on a single page, and should have a legend to explain the symbols being used. Further, it would be helpful to identify each vehicle ID number shown in Figure 7

- 5 by make/model; e.g. Vehicle ID 0003 is a Toyota Corolla. Again, additional figures should be shown for other Bags and pollutants

.
On page 46, the term “BIC” is used without definition. The reader must refer to page 32 to see that BIC refers to the Schwarz Bayesian Criterion. It would be helpful to include a table of acronyms and abbreviations, as many rather obscure terms are used throughout this report.

Figure 7

- 8 (page 50) is very helpful in convincingly demonstrating the reasonableness of the derived statistical model in predicting Bag 2 NOx emissions with both high and low sulfur levels. It would be useful to include similar p lots for other Bags/pollutants.

At the bottom of page 50, where Figure 7

- 9 is being discussed, it is stated that “the rate of sulfur loading is the same for both high and low sulfur levels.” What is the basis for this statement? Sulfur loading is not meas ured directly. Is NOx emission rate taken as a surrogate for sulfur loading? (This statement about sulfur loading is also given in the Summary and Conclusions Section on page 59.) Also, Figure 7

- 9 shows quite clearly that for some vehicles, not only are th e emission rates higher with the high sulfur fuel, but also the increase in emissions with mileage accumulation is higher. For example (see vehicles 0075, 0123, 0264, 0178, and 0179). This seems inconsistent with the statement that rate of sulfur loading i s the same for both high and low sulfur levels.

In Table 7

- 10 (page 53) the fixed effects in the NOx statistical model include only sulfur level and miles for all three FTP Bags. Yet the FTP Composite result includes a sulfur level by miles interaction te rm. This seems strange, since the FTP Composite emissions are simply calculated as a combination of the individual Bag emissions. Some explanation should be offered.

Pages 53

- 54 discuss the problem of comparing emissions from high and low sulfur levels wh en there is a sulfur level by mileage interaction. The approach taken to address this (as illustrated in Figure 7

- 10) seems rather arbitrary, although it may be as good as any other approach. However, to explore the impact of this selection, it might be go od to conduct some sensitivity analyses using other methods.DRAFT Page 58 On page 55, it is stated that “For all models except CO Bag 1, CO Bag 1

– Bag 3, and CH 4 Bag 1

– Bag 3, the reduction estimates are statistically significant ...” However, this is not true for the pollutants NOx + NMOG or PM.

Were the sensitivity analyses described in Section 7.3.4 (pages 56

- 59) only conducted for Bag 2 NOx? If so, this seems like a major limitation. Also, at the top of page 59 it is mentioned that even when removing the influenti al vehicles from the analysis, sulfur effect “is still highly significant.” What are the significance levels before and after removing these influential vehicles?

In the Summary and Conclusions Section (page 59) the first bulleted finding should be clarif ied to indicate that the stated NOx, NMHC, and PM reduction values were obtained using a clean

- out procedure with 28 ppmS. The second bulleted finding should be clarified to indicate that although clean

- out with 5 ppmS fuel rather than 28 ppmS fuel further reduced NOx, NMHC, and CO, no effect was observed for PM.

In discussing the sensitivity analyses within the Summary and Conclusions Section (page 60) it should be stated that these analyses were only performed for Bag 2 NOx emissions (if this is true).

A number of typographical and other errors were also noted:

Page 3, footnote: “The program has collected additional data that is being incorporated ...”

Page 5, 5 th line: Insert period after “overall.”

Page 6, footnote to Table ES

- 3: It is stated that reduc tion estimates were computed differently for Bag 1 THC and CH 4 because these clean

- out effects were not statistically significant. Table ES

- 1 shows that the clean

- out effects were also not statistically significant for Bag 1 NOx and NMHC.

Page 6, next to l ast line: “... are consistent with those formed in the MSAT and Unicor studies ...”

Page 17, Figure 6

- 1: The sulfur clean

- out cycle in the “short procedure” (blue

- colored box) should indicate 28 ppmS, not 30 ppmS.

Page 29, 2 nd line: “In addition, the results f rom the clean

- out data were used to supplement ...”

Page 30, 4 th line: “... representing the as received sulfur level ...”

Page 32, 2 nd line: “... vehicles from the same vehicle family have the similar emission profiles.”

Page 40, 4 th line: “However, the sulfur lo ading effect certainly varies by vehicle ...”

Page 58, 1 st ¶, 5 th line: “... which is calculated after an iterative process ...”