Page 1 MEMORANDUM

TO: William Charmley, Director, Assessment and Standards Division

FROM: Kent Helmer, Assessment and Standards Division

Office of Transportation and Air Quality, U.S. Environmental Protection Agency

SUBJECT: EPA Response to Comments on the peer review of The Effects of Fuel Sulfur Level on

Emissions from the In

‐ Use Tier 2 Vehicles (Fuel Sulfur Effects Report)

DATE: April 11, 2013

The three peer reviewers of the Fuel Sulfur Effects Report selected by SRA International were Dr.

Thomas Durbin (University of California, Riverside); Dr. Ronald Heck (Independent Consultant); and Dr.

S. Kent Hoekman (Desert Research Institute). EPA would like to extend its appreciation to all three

reviewers for their efforts in reviewing the report. The three reviewers brought useful and distinctive

views in response to the charge questions.

This memo includes a compilation of comments prepared by SRA International and responses and

actions in response to those comments from EPA. We have retained the organization reflected in SRA’s

compilation of the comments to aid the reader in moving from the SRA report to our responses. All

textual edits and corrections provided by the reviewers are indicated in strikethrough and bold type.



### 3.1 Technical Comments

The reviewers provided a significant number of specific technical comments and were generally

favorable in their reviews of the technical and statistical aspects of the Fuel Sulfur Effects Report

. This

section contains their comments and is divided into those that specifically address either questions or

requests contained in the peer review charge and additional technical comments that reviewers chose

to provide.



### 3.1.1 Charge Questions & Requests for Comment

In addition to encouraging reviewers to best apply their particular area(s) of expertise to review the

overall study, EPA drafted six questions or requests for comment to serve as a focus for the reviewers.

These were included in the peer review charge provided to the reviewers. In varying degree, the three

reviewers provided direct responses to the questions or requests for comments.

1. Was the imputation method used for replacement of measurements with low concentration

reasonable? What other alternatives may have been better in this case? (Section 7.1)

Durbin: The text talks about tailpipe emissions being greater than zero, while the actual question of

relevance may be

– Are the tailpipe emissions greater than the ambient background

concentrations? (presuming a relatively clean background) If the vehicle’s emissions is below

background levels, than it would not be making an additive contribution to pollution levels. In

this case, you could get a distribution of both positive and negative values fluctuating around

zero.Page 2 My concern with the imput value procedure is that it might introduce a slight positive bias. My

preference would probably be to use zeros as the imput values, since this would represent a

distribution of positive and negative values that would average to zero over a larger dataset.

For the cases where the dilute exhaust concentrations are lower than the measured

background. By eliminating, or essentially not allowing, negative or zero values, doesn’t this add

a positive bias to the results? Or how do you ensure there is a not a positive bias.

RESPONSE:

The main objective of the study was to examine the relative changes in emissions between

different fuel sulfur levels. Since the number of zero values was not significantly different

between the two sulfur levels, the presence of potential bias in the result is minimal.

p. 24

– first paragraph

– its states that a data point can either be deleted or replaced with an

imput value. Couldn’t the value just simply be left as is? Also, if an imput value is deleted for a

particular bag, presumably no composite emissions would be available. Would the

corresponding test be eliminated in its entirety?

RESPONSE:

Due to the highly

‐ skewed nature of the emissions data (log

‐ normal), it was necessary to apply

log

‐ transformation, and thus, the zero values could not be left as is. Zero values were

imputed rather than being removed from the dataset. Furthermore, since each bag was

treated separately, the imputation approach did not cause the entire test to be removed.

p. 25 last line

– it’s unclear how an outlier could be an imput value. Is this a case where the

imput value is a very low value? A description of what these values might be should be added to

the text. Again, if the imput outlier value is eliminated, how does that impact the result of the

test?

Can Table 7

‐ 2 be modified to include not only the number of outliers identified, but also the

number that were eliminated? This should also be in the text in the paragraph for section 7.1.2.

RESPONSES TO THE TWO COMMENTS ABOVE:

Since the imputed outliers that were removed represent less than 1 percent of the

measurements, the likelihood of introducing bias in the result is minimal. The description of

the imputed values that were identified as outliers was added to the report text and the

number of outliers that were removed from each bag was included in Table 7

‐ 2.

Hoekman: [T]he method used to impute emissions values in cases where measured levels were lower

than background is probably the best that can be done. The approach of developing an

imputation value based upon each specific vehicle family seems better than using a single, fleet

‐

wide imputation value. The fact that imputed values were applied to only about 20% of the

vehicles provides some assurance that this imputation method did not excessively distort the

results. This was further confirmed by performing the statistical analyses with and without

inclusion of the imputed values. One additional test that could be useful is to conduct the

statistical analyses using values of zero instead of the imputed emissions values.

RESPONSE:Page 3 EPA appreciates the comment. Additional sensitivity analysis using the zero values could not

be performed because the modeling approach being used is in log

‐ space.

1. Please comment on the use of mixed model in analyzing the data. Was the model fitting

strategy in selecting the final model statistically sound? (Section 7.2)

Durbin: The use of a mixed model is a relatively standard method to treat data analysis of emissions,

and appears to be appropriate for the study design given here. The fitting strategy of using a

saturated model, then developing the most optimal covariance structure, and then fitting the

final model appears to be statistically sound. The step

‐ wise backward elimination approach for

the “Sulfur Effects data” also appears to be statistically appropriate.

RESPONSE:

EPA appreciates the comment.

One thought relating to the discussion of the univariate ANOVA and the multivariate MANOVA

in section 7.2. Overall, it does not seem to add significant value to the report. Looking at the

audience for this report, it would likely be composed of a mix of readers that may or may not

have a statistical background. For those with a statistical background, the additional information

on the univariate ANOVA and the multivariate MANOVA would not provide significant value. For

those that are not as familiar with statistical methods, the added information would likely be

more confusing than clarifying. It is suggested that this information, although interesting, might

be better placed in an Appendix.

RESPONSE:

In consideration of readers without a statistical background, the detailed discussion of

univariate and multivariate analysis of variance, which is necessary to provide the readers

with the understanding of the reasoning behind taking the mixed model approach, was moved

to Appendix D.

Hoekman: Overall, the statistical modeling approach seems appropriate. As described in Section 7.2

(pp. 26

‐ 28) the structure and limitations of the emissions dataset make the linear mixed model a

good choice for analysis, as this better accommodates missing data, irregularly spaced

measurements, and within

‐ vehicle effects.

RESPONSE:

EPA appreciates the comment.

A significant question about the statistical modeling approach is the failure to include any

information about the vehicles’ catalyst formulations, or other catalyst properties. It is not

surprising to observe that vehicle type (car vs. truck) was not a significant model term, but it is

surprising that there is no term related to the emissions control systems used across the range

of vehicles. Surely not all catalysts were the same, and different catalysts would be expected to

respond to changes in sulfur levels in different ways. This is an area that should be addressed.

Inclusion of a catalyst term could result in quite different statistical models.

RESPONSE:Page 4 Additional text on this point was added to Section 4.1. The test fleet was chosen to be

representative of latest

‐ technology light duty vehicles being sold at the time the program was

launched. We did not attempt to analyze or model details of aftertreatment design specific to

each vehicle model such as catalyst position, precious metal types and quantities used, or

related engine control strategies such as timing advance at cold start or fuel cut during

deceleration. While these things undoubtedly influence the behavior of emissions and may

interact with the fuel sulfur effects being investigated, including them in an analysis requires

correctly assessing and parameterizing them for all vehicles in the study. Instead this

program’s aim was to characterize overall effects on emission inventories by observing the

aggregate behavior of a representative fleet of vehicles.

2. Were the model assumptions for the covariance structure reasonable given the data?

(Section 7.3)

Durbin: For the “Clean

‐ out data”, an unstructured covariance structure was initially used (which did not

converge), and then a compound symmetry structure was selected. The compound symmetry

structure is appropriate since the measurements from the same vehicle should have a

homogeneous variance, as included in the text. For the “Effect of Sulfur data, where multiple

measurements are made at different mileage accumulations, the autoregression covariance

structure is appropriate since the since the correlations between measurements is expected to

decline as the measurements are further apart in terms of mileage accumulation, as mentioned

in the text of the report.

RESPONSE:

EPA appreciates the comment.

3. Please comment on the methodology used in determining the in

‐ use sulfur effect for models

with and without the sulfur and mileage interaction term. (7.3.3)

Durbin: Overall, the methodology for determining the in

‐ use sulfur effect for models with and without

the sulfur and mileage interaction term appears to be sound. For cases where the sulfur level

and mileage interaction term was not significant, the sulfur effect, sulfur loading and associated

percentage differences between the high and low sulfur fuels should be constant as a function

of increasing mileage. Thus, using the differences in the least squares means from the final

model and reverse transforming them could be used to quantify the percentage differences

between high and low sulfur for the case where the sulfur level and mileage interaction term

was not significant.

For cases where the sulfur level and mileage interaction term is significant, the rate of sulfur

loading between the low and high sulfur fuels would differ as a function of mileage, so a

different approach would be needed. The methodology of using the in

‐ use emission level upon

arrival (pre

‐ cleanout) from a larger clean

‐ out dataset and projecting it out to the mileage where

the two lines intersect (in

‐ use equivalent loading) seems reasonable for estimating the in

‐ use

sulfur level effect. Some clarification should be given on what is meant by the “larger clean

‐ out

dataset” since it is not immediately obvious what data these actually are. Additionally, how do

the incoming sulfur levels for the “larger clean

‐ out dataset” compare with average in

‐ use values

for the US fleet, or the fleet in different regions of the country.Page 5 RESPONSE:

The report has been modified to address some of the comments. Although it is an interesting

question, comparing to the US fleet or the fleet in different regions of the country is outside

the scope of the current study.

4. Is the interpretation of the mileage

‐ by

‐ sulfur interaction term presented correctly? (Section

7.3.3)

Durbin: As discussed under question #4, the interpretation of the mileage

‐ by

‐ sulfur interaction term

does appear to be presented correctly. On p. 48, when the interpretation of the interaction term

is first being discussed, I would also add in something about the percentage differences in

emissions being constant as a function of mileage, just to make things clearer. For example, for

the last line of the paragraph just after Table 7

‐ 9: In other words, the effect of high fuel sulfur

on Bag 2 NO

x exists immediately after clean

‐ out and remains constant on a percentage basis

during subsequent driving of a vehicle.

RESPONSE:

EPA appreciates the comment. We have revised the report to clarify the interpretation.

5. Are the sensitivity analyses on effects of low

‐ level concentrations, imputed values, and

influential vehicles sufficient as performed? Do the results from the sensitivity analyses

provide additional support for the robustness of the conclusion? (Section 7.3.4)

Durbin: Overall, the sensitivity analyses see to add additional support to the robustness of the

conclusions. Here are a few thoughts on the sensitivity analyses.

If the sensitivity analyses were only done for the NO

x bag 2 emissions, it would be worth

mentioning that the NO

x bag 2 emissions showed a higher percentage of measurements with

zero values than most other pollutant/bag combinations, as illustrated in Table 7

‐ 1.

It would be interesting to get some feel for the vehicles/data eliminated in the sensitivity

analysis for the low concentration measurements. Since these would be low emitting vehicles, it

seems like they might have more robust and sulfur tolerant catalyst systems. This is consistent

with the results showing an increase in the emissions reductions as these data are pulled out.

Relating to the imput values, it would be interesting to see what the effect would be of making

the imput values simply zero. Given the small differences between the model with and without

the imput model, there would likely be only a minor impact, but this would add to the

robustness of the results, although probably not change any conclusions.

For the influential vehicles, it is interesting to look at the vehicles selected. Vehicles 0007 and

0178 show a relatively large sulfur effect, whereas vehicle 0046 shows a slight reverse sulfur

effect with tight variability. It is interesting that other vehicles with relatively strong sulfur

effects, such as 0165, 0179, and 0011, did not have a strong influence. The result of this

sensitivity analysis is reasonably intuitive, with the NOx differences between sulfur levels

shrinking, which would be consistent with removing some vehicles with a stronger sulfur effect.Page 6 Hoekman: Were the sensitivity analyses described in Section 7.3.4 (pp. 56

‐ 59) only conducted for Bag 2

NOx? If so, this seems like a major limitation. Also, at the top of p. 59 it is mentioned that even

when removing the influential vehicles from the analysis, sulfur effect “is still highly significant.”

What are the significance levels before and after removing these influential vehicles?

In discussing the sensitivity analyses within the Summary and Conclusions Section (p. 60) it

should be stated that these analyses were only performed for Bag 2 NOx emissions (if this is

true).

RESPONSE:

The report has been modified to address the comments above. Responses to some of

reviewers’ specific comments: The sensitivity analyses were conducted only for Bag 2 NOx,

because some of the issues that could potentially affect the mixed model results were most

prevalent in Bag 2 NOx. This was an effort to examine the robustness of the model results,

and since added value in additional sensitivity analyses for other pollutants and bags is likely

minimal, we do not consider this a limitation. As for the additional sensitivity analysis with

the inclusion of zero values, due to the highly

‐ skewed nature of the emissions data, it was

necessary to apply log

‐ transformation, and thus, the comparisons of models with and without

zero values could not be performed.



### 3.1.2 Other Technical Comments

All of the reviewers provided specific technical comments in addition to their responses to the specific

questions in the peer review charge.

Durbin:

[1] Study Design

 Was the fuel commercially available in the Ann Arbor area at the time of the study oxygenated

or not? This is of relevance since the test fuel is non

‐ oxygenated.

 30 ppm is given as the in

‐ use sulfur average. Is this based on actual data of in

‐ use sulfur

samples, or is this based on regulatory average requirements.

 In discussing why the US06 x2 clean

‐ out cycle was used, it might be worth discussing the clean

‐

out procedures that were used in the MSAT and Umicore studies.

RESPONSE:

The report has been modified to address these comments.

[2] Test Vehicle Selection, Recruitment, and Delivery

 What is the average sulfur level for the as received vehicles?

 Section 4.1 should specifically mention that 19 makes/models were utilized in the study.

 Was any attempt made to characterize the vehicle categories by other factors such as level of

long term fuel trim, or whether the vehicle used a closed coupled or double catalyst or only an

underbody, or catalyst metals?

 What incentive was provided to owners to participate in the program?

– add to section 4.2.Page 7  More information should be given about the number of vehicles targeted for testing, and the

number of vehicles in each vehicle make/model category. The first mention should be here, as

opposed to in section 6.2. In that section, it talks about 5 vehicles for each make/model class. It

would be useful to say the total number of vehicles tested at this point.

RESPONSE:

The report has been modified to address these comments.

[3] Test Fuel Specs and Procurement

 Some more details should be included about the fuel selection would be useful. How does this

fuel compare to typical in

‐ use fuel, in terms of say aromatics and RVP. Were oxygenates in use

in the recruitment area? Or compared to averages of in

‐ use used throughout the US?

RESPONSE:

The report has been modified to address these comments.

[4] Test Procedures

 It would be useful to add in fuel change points into the test procedure flow charts.

RESPONSE:

The report has been modified to address this comment.

 There is some lack of clarity in reading through this section that can be shown by looking at the

last paragraph. The last paragraph indicates that only 4 of 19 family used the modified short

procedure, whereas the first sentence of the paragraph indicates that the change was made

approximately halfway through the program. Then looking at Table 7.7, it indicates that

between 2

‐ 4 vehicles in each make/model class were given the modified short procedure,

whereas it seems to imply in the paragraph that once the change was made, it was applied

thereafter. Was the change just made for a subset of the vehicles tested in each vehicle

category? Also, in the last sentence it indicates that the change in the number of vehicles

providing “sulfur level data” is seen in Table 7

‐ 7. Does this mean that the modified short vehicles

were added to the “sulfur level data” as well as the Long or modified Long procedure vehicles?

 Related to the above comment, why not simply just say at the end of each procedure

description how many vehicles were tested on each procedure?

RESPONSE:

Some clarifying text was added to the report to address these comments. To answer specific

questions in the comment, the modified short procedure was introduced at a certain point in

the testing, after which all future recruits underwent either the long procedure (target one per

make/model) or the modified short (target four per make/model). The vehicle make/model

family was initiated in the sequence shown in Table 7.7, but it took longer for some than

others to work through the five replicate recruits and therefore some vehicles got out of

sequence with their replicate group. Thus, the modified short procedure would have been

implemented partway through the replicate group for some vehicle make/model families.Page 8 The modified short procedures provided sulfur level data together with the long procedures.

After seeing several initial long procedure vehicles showing an immediate effect of sulfur level,

often with little change over mileage accumulation, we decided to add the modified short

procedure simply to increase the number of sulfur level observations. Since the modeling used

mileage as a treatment parameter, this should not have had an impact on the analysis.

 Figure 6

‐ 3 indicates that only the 28 ppm sulfur level data is used for the “clean out effect”, but

the very first sentence under section 7 indicates that there is a set of “clean

‐ out” data at 5 ppm.

RESPONSE:

Both the long and modified

‐ short procedure vehicles had clean

‐ outs performed on 5 ppm fuel

for the second half of testing, after changing them over from 28 ppm. While we don’t have

the same type of clean

‐ out data as for 28 ppm, where it was performed without a fuel change,

we believe that some information could still be gleaned about a sort of “instantaneous” effect

of fuel sulfur level by comparing the emissions immediately following the cleanout on 5 vs. 28

ppm. We talked over various ways to describe the name and meaning of this part of the

analysis, and settled on the “clean

‐ out effectiveness” description seen in the report.

[5] Data Analysis and Results

 p. 21

– 3 rd paragraph

– A statement should be added to answer the following question

– Why do

the results of the Bag 2 NOx emissions have more substantial implications than those of other

pollutants/bags?

RESPONSE:

We have revised the report to address the comment.

 While it is useful to discuss the results at low concentrations, further information on the

experimental methods might merit a section. For example, what analyzer is used? Can the

detection limits of the analyzers be included in an Appendix? What methods/microbalance was

used for the PM measurements, etc.?

RESPONSE:

Additional information on analytical equipment was added to the report.

 p. 24

– top of p. There should be a brief paragraph discussing Table 7

‐ 1 and the number of zero

value measurements before discussing the imput value methodologies.

RESPONSE:

We have revised the report to address the comment.

 p. 24

– last sentence

– It says that the statistical analyses were run both with and without

imputed values. In terms of the “without” input value case, does that mean that the values were

eliminated, were zeros used as the imput values, or were the original values used, whether they

are positive or negative.

RESPONSE:Page 9 For the model without imputed values, the mixed model was re

‐ fit using a new dataset with

all imputed values removed, consisting only of actual measurements, as described in Section

7.3.4.

 p. 25

– second paragraph

– A new terms vehicle “families” is introduced. Presumably this term is

the same as the vehicle make/models used earlier. Presumably these also have the same engine

family and this should be included along with engine size, vehicle configuration, and weight.

RESPONSE:

We have modified the report to address this comment.

 p. 30

– There is no discussion on the symbols in the plots presumably the plots show the

average and median inside the bar. Then the bars represent the 95% confidence levels and the

error bars the full data range. Then there are some other dots on the plot?

RESPONSE:

The box

‐ plot was modified and the text describing the symbols in the plot is added to the

report.

 p. 30

– It is not clear what is meant by “Some vehicle families show the presence of within

‐ family variability” Presumably this could also be due to differences in in

‐ use sulfur levels as well,

since a handful of vehicles came in with in tank fuels with sulfur levels much higher than those

for others. For the post

‐ cleanout variability, it seems like this should be independent of the in

‐

use sulfur loading. Could some of the variability be related to the condition of the vehicle?

Vehicle family M504 seems to show a very variability post

‐ cleanout.

RESPONSE:

We agree that variability in the pre

‐ clean

‐ out emissions could be due to varying sulfur levels in

the tank at the time of recruitment. Variability of emission results following cleanouts,

especially in the sulfur

‐ level effect found in the study, suggests varying levels of catalyst

deterioration over the vehicle’s life, looser control of air

‐ fuel ratio during the test, or

differences in driver behavior from test to test.

 p. 32

– Related to BIC criterion. It indicates that the BIC performs relatively better for small

sample size settings. How would one define 5 here? Is the current experiment a small sample

size?

RESPONSE:

We do not consider the sample size of this study to be small. This study, by far, is the largest

study examining the effect of sulfur on emissions on Tier 2 vehicles. We used the BIC criterion,

because the sample sizes required by other selection criterions are much bigger.

 Figure 7

‐ 4 does not appear to add significant value. Much of the same information is available in

the box plots in Figure 7

‐ 5, and that is a little easier to read. It talks about the vehicles with

similar emissions profiles being grouped. However, it is not immediately obvious what grouping

were used for the Figures 7

‐ 4 and 7

‐ 5.

RESPONSE:Page 10 We generally agree with this comment. We have added a box

‐ plot in place of the original

plots.

[6] This report presents an interesting approach to estimating the real effects of fuel sulfur in

the Tier 2 in

‐ use fleet. The study was very extensive in terms of the number of vehicles and level

of testing that was done, and the results of the study some potentially important impacts of

differing in

‐ use sulfur levels. It would be interesting to evaluate these results further to

determine the modeled impact of in

‐ use fuel sulfur levels on ambient quality. The analysis

methodologies in the report appear to be statistically sound, and appropriate for repeated

measures types of testing.

RESPONSE:

EPA appreciates the comment. However, analyzing the impact of fuel sulfur level on ambient

air quality is beyond the scope of the report.

The presentation of the data is relatively extensive in terms of the descriptions of the statistical

analysis procedures used. As such, the report would relatively straightforward to read for the

some with a moderately strong background in statistics, but might a little too technical for some

readers without a strong statistical background.

RESPONSE:

In consideration of readers without a statistical background, the detailed discussion of

univariate and multivariate analysis of variance was moved to Appendix D.

Heck:

[1] It is very important that the authors discuss right up front that they have taken an approach

where the vehicles are considered as “black boxes”. This is a totally different approach than

that adopted by the Umicore paper (Ball D., Clark D., Moser D. (2011). Effects of Fuel Sulfur on

FTP NOx Emissions from a PZEV 4 Cylinder Application. SAE 2011 World Congress Paper 2011

‐ 01

‐

0300. SAE International: Warrendale, PA.) In the “black box “ approach there is no

consideration of the engine control strategies (rich or lean bias, cold start strategy, etc.), sensor

responses (from aging), vehicle operation (e.g. fuel cut), catalyst technologies, catalyst

configurations(close coupled, underfloor, cell density, volume, etc.) See James M. Lyons, David

Lax, and Steve Welstand, Investigation of Sulfur Sensitivity and Reversibility in Late

‐ Model

Vehicles, SAE 1999

‐ 01

‐ 3676 for work that takes a more phenomenological approach to sulfur

effects. By the way I could not find this reference in the report. In the “black box” approach

very little information is given besides the vehicle type, mileage, age, etc. Because of this lack of

information it will be difficult for anyone to repeat the authors’ experiments and challenge there

results. In addition it will be difficult for the authors to explain data anomalies because there

are no specifics to tease out a possible explanation. Also since there are so many uncontrolled

variables, the statistical analysis may be limited since it is only statistics! This is not meant to say

the work has no merit only to provide caution in the final conclusions and merits of too much

statistics!

[2] One concern I have is if some of the vehicles have a fuel cut strategies. If so, this could

confound the results since in the US06 the catalyst with a fuel cut strategy will see lean

excursions while a catalyst without will be stoichiometric during the cycle. Again since noPage 11 information is given in the report on engine operation it is impossible to go back and uncouple

this effect.

[3] Sulfur poisoning is a complex phenomenon for catalysts and the catalyst manufacturers have

found recipes to compensate for high and low sulfur levels (see Harold N. Rabinowitz, Samuel J.

Tauster, Ronald M. Heck, The effects of sulfur & ceria on the activity of automotive Pd/Rh

catalysts, Applied Catalysis A: General 212 (2001) 215–222). If the catalyst companies can

design for Tier 2 vehicles at 30 ppm S, then this study may be in question or to put in another

way, maybe some of the vehicles you studied already have these types of technologies and that

is why you found some responses that didn’t show an effect of S level!

RESPONSE TO THREE COMMENTS ABOVE:

EPA appreciates the comments. Additional text was added to the report to clarify the intent of

the study being to assess the aggregate behavior of a representative sample of the Tier 2 fleet.

While we agree that many catalyst and engine operation parameters interact with sulfur to

influence emissions and could be studied as part of an engineering analysis, attempting to

account for all of them across the test fleet would be very complex and unnecessary for our

purpose of assessing the impacts across a broad cross section of vehicles.

[4] The removal of sulfur from the catalyst needs a consistent statement throughout the paper.

You show Figure 2

‐ 1 from Ford but the text is not consistent in describing this removal which is

due to temperature, air to fuel ratio, and sulfur level. Also the degree of removal is different for

Pt, Pd and Rh and the oxygen storage materials (Ce, Zr, La, etc.). So this discussion needs a little

more technology.

RESPONSE:

We have modified the report to address this comment.

[5] [p. 3] “Test fuels were two non

‐ ethanol gasolines with properties typical of certification fuel,

one at a sulfur level of 5 ppm and the other at 28 ppm.”

– Is there any reason to expect

different results with ethanol gasolines?

RESPONSE:

We have no basis to expect that sulfur and ethanol would interact, and are unaware of any

studies examining this issue.

[6] [p. 4.] “This indicates that the catalyst is not fully desulfurized, even after a clean out

procedure, as long as there is sulfur in the fuel.”

– Not sure you can make this conclusion. It

may be that the sulfur in fuel equilibrates instantaneously and it is a concentration effect not a

desulfurization. To prove this you would have to vary the desulfurization time and see where

it reaches steady state.

RESPONSE:

We have modified the report to address this comment.

[7] [p. 5] “Other results, such as Bag 1 hydrocarbons, did show a significant miles

‐ by

‐ sulfur

interaction.”

– Does this mean that the effect never equilibrates? Are you comfortable in

saying this?Page 12 RESPONSE:

We feel it is useful to at least report the statistical findings from the dataset. To conclude that

the effect “never equilibrates” seems speculative.

[8] [p. 7] “The amount of sulfur retained by the catalyst is primarily a function of its operating

temperature, the active materials and coatings used within the catalyst and the concentration of

sulfur oxides in the incoming exhaust gases.”

– Again air to fuel ratio is important as you

mention in the next sentence for reducing conditions.

[9] [p. 8] “However, the temperatures necessary to release sulfur oxides can also lead to

thermal degradation of the catalyst over time.”

– Nowhere do you mention that normal

operation of the vehicle, the catalyst is constantly being exposed to rich/lean condition from

the control system using an oxygen sensor. So the catalyst is being regenerated in

‐ situ form

the perturbation around stoichiometric.

[10] [p. 11] “The level of reversible in

‐ use sulfur storage and release (or loading) within an

exhaust catalyst system can be assessed by measuring emissions from the vehicle as received,

performing a high speed, high load clean

‐ out cycle, then measuring emissions again.”

– In

describing the cleanout it is important to mention the temperature range and the air to fuel

ratio. Also depending on the vehicle calibration the degree of richness and time at such could

be different from vehicle to vehicle. How is this taken into account for the degree of

cleanout?

[11] [p. 11] “A vehicle with relatively high exhaust temperature at the catalyst location, and/or

significant excess loading of certain platinum group metals (PGM) and other active materials in

the catalyst may be relatively insensitive to sulfur loading regardless of driver behavior.”

– Up to

now you have not mentioned the effect of S on the oxygen storage components which will

affect NOx and lightoff in cold start. Is there a reason this is not mentioned?

RESPONSE TO FOUR COMMENTS ABOVE:

We have modified the report to address these comments.

[12] [p. 12] “This loading continues over time with vehicle operation and can be observed as an

increase in emissions (sometimes referred to in the auto industry as “NOx creep”).”

– Is this due

to PGM and/or oxygen storage component deactivation?

RESPONSE:

Yes, that is our assumption.

[13] [p. 13] Table 1

‐ 1 Test Vehicles Recruited

– If you look at the Umicore study you will see

details about the emission control system (close coupled, underfloor, etc.) You give no such

detail in this study. Also, they also show a plot of engine lambda during the FTP. This is

important background information to possibly explain outliers or unusual results. This study

loses significant meaning without this information. You are treating the cars as black boxes

and I think this is a mistake! Also, the oxygen sensor response is very important as well as it

age. This is never mentioned in this entire study.Page 13 RESPONSE:

We appreciate this point. The Umicore study was a detailed analysis of the behavior of a

single vehicle, while our study was intended to produce information about the fleet’s behavior

in general.

[14] [p. 38] “Furthermore, the reduction in emissions from cleanout shown in

... would likely be

larger if the low sulfur test fuel at 5 ppm had been used for the cleanout procedure and the

tests immediately following the as

‐ received baseline emissions.”

– This looks like a conjecture.

Present evidence.

RESPONSE:

The statement is supported by the additional reduction achieved by using the low sulfur fuel at

5 ppm for the cleanout procedure, presented in Table 7

‐ 6. It suggests that the cleanout

procedure using low sulfur fuel is more effective in reducing the sulfur loading.

[15] [p. 40] “For example, vehicle IDs 0011, 0022, and 0178 clearly show large effect of fuel

sulfur level on emissions while the effect is only marginal for vehicle IDs 0123 and 0264.”

– Differences can be A/F ratio, close coupled catalyst, oxygen storage components, temperature

history. This is why more information on vehicle characteristics is important to explain

unusual results.

RESPONSE:

We agree that some combination of these factors is likely the explanation for the difference.

As stated in an earlier response, engineering analysis of vehicle components was beyond the

scope of the study, so we are simply noting the difference in behavior.

[16] [pp. 59

‐ 60] “Comparing emissions immediately following the clean

‐ out procedure on 5 vs.

28 ppm fuel, FTP composite NO

x emissions were 18% lower, NMHC 9% lower, and CO 8%

lower.”

– To make this conclusion you need to age with 28, clean out with 28 and test with 5

and compare age with 5, cleanout with 5 and test with 5. I went back in text and really didn’t

see this discussed in analysis. If I missed it OK otherwise need some discussion in analysis.

RESPONSE:

We have modified the report to address this comment.

Hoekman:

[1] The experimental test design was suitable for the intended purposes, although clearly, other

testing procedures could have been used as well. For example, the standard FTP test procedure

was used, at 75  F, to determine the emissions results from each vehicle test. Some comments

should be offered regarding the suitability of the FTP driving cycle and a 75  F test temperature

in providing representative/realistic operating conditions. For the in

‐ use vehicle fleet, a set of

standardized test conditions is clearly necessary to obtain repeatable and statistically significant

results. However, the FTP test is now over 35 years old, and 75  F is just one of many

temperatures that could be used. The overall quantitative effect of fuel sulfur on emissions as

determined in this study would be more convincing if it were shown that similar results apply to

other realistic operating conditions.Page 14 RESPONSE:

We appreciate the points raised here, and agree that the FTP cycle conducted at 75F leaves

out a significant portion of in

‐ use operation. The largest (percent) effects of sulfur level were

found during warmed up operation, a mode that (eventually) occurs at all temperatures.

Thus, the results are applicable to a wide range of operation modes. The fact that the clean

‐

out was found to be less effective at higher sulfur levels (e.g., there is an instantaneous effect

of sulfur level) also suggests that benefits of lower sulfur would continue into more aggressive

modes.

[2] The use of two, back

‐ to

‐ back US06 dynamometer cycles was an appropriate choice for

catalyst regeneration. Although a somewhat more aggressive cycle might provide more

complete desulfurization, potential damage to the vehicle was a significant concern, as

explained in this report.

Also, the approach of catalyst clean

‐ out, followed by repeated testing with a single fuel is a valid

way to determine a new “equilibrium emissions level” at a specific fuel sulfur level. However, in

real

‐ world driving, such an equilibrium level is never attained, because operating conditions are

constantly changing, and fuel sulfur varies from tank

‐ to

‐ tank.

RESPONSE:

We appreciate the comments. While the commenter raises useful points for discussion, the

overarching point is that for this study a series of controlled conditions were chosen and used

consistently throughout data collection in order to conduct a credible experiment with

meaningful results.

[3] It is not clear why this report places so much emphasis on Bag 2 results

– especially Bag 2

NOx. Typically, Bag 2 emissions concentrations are extremely low from properly functioning Tier

2 vehicles. In fact, as shown in this study, it is often difficult to distinguish Bag 2 levels from

background levels. Large percentage differences between two very small numbers may not be

very meaningful. To help put these results in perspective, it would be useful to show the

relevant certification emissions levels for these Tier 2 vehicles. Also, something should be said

about the impact of fuel sulfur reduction on fleet

‐ wide emissions, and how large a reduction this

represents in comparison to the entire mobile

‐ source emissions inventory. In reality, this test

program only captured a small portion of the entire in

‐ use fleet (only Tier 2 vehicles of model

years 2007

‐ 2009).

RESPONSE:

We have modified the report to address this comment.

[4] Some historical perspective about the “FTP bag method” would be helpful here. This

sampling and analysis method was developed about 40 years ago, to measure emissions from

uncontrolled (or slightly controlled) vehicles. Prior to introduction of emissions control systems,

Bag 2 emission levels were routinely 2 orders of magnitude higher than today. Under such

conditions, the FTP bag method was perfectly suitable for the intended purpose. Now, with Bag

2 emission levels being nearly indistinguishable from background, a different method should be

employed for quantifying vehicle emissions.Page 15 RESPONSE:

We are aware of the challenges of quantifying FTP bag 2 emission levels for some pollutants

for some vehicles. However, the cold

‐ start FTP is still representative of a large fraction of

driving and provides measurable, relevant fuel effects results.

[5] In retrospect, it seems unfortunate that only non

‐ ethanol gasolines were used, as such fuels

are increasingly uncommon. It is stated that similar results would be expected from ethanol

‐

containing gasolines, but no data are provided to support this. If any subsequent testing has

been done with ethanol

‐ containing gasolines, this should be mentioned.

RESPONSE:

We are not aware of any studies examining the interaction of sulfur and ethanol, so the

assertion that they don’t interact is based on the opinion of emission control experts within

our office. We are interested to see any data related to this issue.

[6] On page 2 (1 st paragraph) the quantity of sulfur present on the vehicle’s catalyst is said to be

a function of temperature and fuel sulfur level. Isn’t the catalyst formulation/metallurgy another

important factor? (Catalyst formulation is mentioned on page 7.) What is known about the

catalyst formulations used in each of the test vehicles of this study?

RESPONSE:

We have modified the report to address this comment.

[7] It is interesting to note that when reducing fuel sulfur from 28 ppm to 5 ppm, significant

emissions reductions were observed for all pollutants except PM, (See Tables ES

‐ 2 and ES

‐ 3.) Is

there a clear mechanistic reason for this lack of effect for PM? Are the catalyst systems used in

Tier 2 vehicles ineffective in reducing PM (and PM precursors)? This lack of a PM effect should

be mentioned along with the major findings that are shown in bullet form on page 6.

RESPONSE:

We have modified the report to address this comment.

[8] On page 7 (final paragraph) it is stated that the impact of fuel sulfur on emissions was

considered negligible under the Tier 0, Tier 1, and NLEV programs. I don’t think this is true. The

emissions impacts of fuel sulfur have been well known for a long time, but more severe sulfur

reductions were not thought to be a cost

‐ effective emissions control approach until recently. To

provide greater context, it would be useful to include a brief summary of gasoline regulations

(including sulfur levels), and how they have evolved over the past 30

‐ years. Along with this, a

history of LD vehicle emissions standards should also be presented.

RESPONSE:

We have modified the report to address this comment.

[9] On page 8, various detrimental effects of hot/rich catalyst operation are described

– including catalyst degradation and increased emissions of PM, NMOG, and CO. However,

nothing is said about fuel economy effects

– either here or elsewhere in the report. In general,

fuel economy is a concern with any type of fuel or vehicle modification. This topic should at least

be mentioned. In addition, the dataset generated in this study provides an excellent opportunityPage 16 to evaluate the impacts of sulfur reduction on fuel economy, by analyzing CO 2 as another

pollutant, along with CO, THC, NOx, and PM.

RESPONSE:

Fuel economy is a consideration, but seems a step beyond the scope of the document. The

primary purpose here is to describe a study of the effects of sulfur on regulated emissions.

Implications of other vehicle

‐ related strategies for reducing emissions are only mentioned

briefly as background; that discussion is not meant to be comprehensive.

[10] The 2005 MSAT study, which used nine Tier 2 vehicles, is described on pages 8

‐ 9. The

emissions reductions upon lowering fuel sulfur from 32 to 6 ppm are said to be 33% for NOx,

11% for THC, 17% for CO, and 32% for CH 4

. Are these values simple averages of the nine

vehicles, or were they computed by a more sophisticated method? Were similar statistical

methodologies used to compute the fuel sulfur effects in both the MSAT study and the present

study? Were PM emissions also measured in the MSAT study?

RESPONSE:

We have modified the report to address this comment.

[11] When first describing the two test fuels on page 10, it would be helpful to refer the reader

to Table 5

‐ 1 (page 15), which provides a more complete listing of fuel properties. Also, did these

fuels contain anything unusual in the way of additives; e.g. antioxidants, detergents, dyes, etc.?

RESPONSE:

We have modified the report to address this comment.

[12] In selecting the vehicles, was the ratio of LD trucks/LD passenger cars representative of the

in

‐ use fleet? Table 4

‐ 1 (page 13) which lists the 19 makes/models that were recruited, shows

that the No. 1 U.S. sales rank vehicle is missing. Was this an oversight, or are the rankings shown

in this table incorrect? Notice also that the Toyota Camry is shown as No. 23 in U.S. sales rank.

This seems unbelievably low for what has traditionally been a “top seller.” Also, it would be

useful to add a column to Table 4

‐ 1 to indicate the number of vehicles in each category that

were tested in this study, comprising the total test fleet of 81. It might also be useful to indicate

here the number of vehicles in each class that underwent the various test procedures: i.e. short

procedure, long procedure, modified short procedure, and modified long procedure.

RESPONSE:

We have modified the report to address this comment.

[13] The modified long testing procedure shown in Figure 6

‐ 2 (page 19) is confusing. The blue

box indicating the short procedure shows only two post cleanout FTPs, although the wording

still indicates “triplicate FTPs at 28 ppmS.” Also, in describing the modified short procedure on

page 20, it is said that “... the change in the number of vehicles providing sulfur level data can be

seen in Table 7

‐ 7 starting with Family ID N513.” It is not clear what this means. Does this table

list the vehicle families in chronological testing order, so that the first 13 families listed were

tested using the standard short procedure, and the last 4 families listed were tested using the

extended short test? It is also confusing that Tables 7

‐ 5 and 7

‐ 7 provide identical information.Page 17 RESPONSE:

We have modified the report to address this comment.

[14] How is NMOG different from NMHC? What analytical procedures were used to measure

these two pollutants?

RESPONSE:

We have modified the report to address this comment.

[15] Displaying the background and sample measurements by FTP Bag (as done in Figure 7

‐ 1 and

Appendix B) is very instructive. However, only NOx and THC results are shown in this way; it

would be useful to also include CO and PM. It would also be helpful to draw ovals in Figure 7

‐ 1

to capture all background measurements for each vehicle family shown. Although it is not

possible to tell which sample points correspond to which background points, this graphical

approach clearly illustrates the problem of very low emissions measurements relative to

background in Bag 2. It also begs the question of why background levels of NOx are so variable.

Figure 7

‐ 1 shows that these background concentrations varied by over an order of magnitude

for many of the vehicle families tested.

RESPONSE:

There were no background PM measurements collected as part of the dataset (not a common

practice in LD gasoline testing). CO behavior would be similar to THC, with a relatively large

sample/background ratio in bag 1 and much smaller in bags 2

‐ 3. The points in Figure 7

‐ 1 are

arranged into vertical bands, with space between the vehicle groups; the ovals would seem

redundant.

[16] In the determination of outlier data points (described on page 25) what is the rationale for

choosing an outlier screening criterion value of ± 3.5 for the studentized residuals? Also,

although Table 7

‐ 2 identifies the number of outlier points for each pollutant/bag, it doesn’t

indicate how many of these outlier points were actually excluded from the statistical analyses.

RESPONSE:

We have modified the report to address this comment.

[17] As explained on page 26, log

‐ transformation of emissions measurements has commonly

been used to analyze vehicle emissions data. However, many previous emissions modeling

studies have utilized test fleets that included a variety of technology types having a wide range

of emission levels. In the present study, only Tier 2 vehicles were used, and the emission levels

did not vary drastically across the test fleet. Given this situation, is log

‐ transformation still

necessary (and helpful)?

RESPONSE:

The decision to apply log

‐ transformation was made based on preliminary descriptive statistics,

and the data plots, such as a box plot, which showed that the emissions measurements were

highly skewed even though the current study was limited to Tier 2 vehicles. Thus, the log

‐ transformation was necessary to normalize the distribution of the residual and to stabilize the

variance.Page 18 [18] On page 29 it is stated that the average starting odometer reading of the vehicles used to

assess the clean

‐ out effect at 28 ppmS was 31,470 miles. Is this value the average of the 17

vehicle families shown in Table 7

‐ 3, or the average of all 81 vehicles? Similarly, is the ± value of

1,578 miles the standard deviation of the 17 vehicle families or the 81 individual vehicles? (The

same questions pertain to the vehicles used to assess the clean

‐ out effect at 5 ppmS, described

on page 35; and the vehicles used to assess the sulfur effects, described on page 39.)

RESPONSE:

We have added text to clarify this point.

[19] The box plots shown in Figure 7

‐ 2 (page 31) are quite informative in displaying the relative

variances between the pre ‐ and post

‐ cleanout vehicle tests. It would be useful to show similar

plots for other pollutants and Bags. Also, a legend should be included to explain the different

symbols used in these plots.

[20] When describing the dependent variable ( Y i ) and effects ( X i and Z i ) on pages 31

‐ 32, the

reader should be reminded of the mixed model being used, shown as Equation 7

‐ 1 on page 28.

Also, it is stated on page 32 that “The significance of between

‐ family variation was observed

graphically in Figure 7

‐ 2 ...” But this figure only show Bag 2 NOx results. Was similar between

‐ family variation observed with other Bags and pollutants?

[21] Figure 7

‐ 3 (pages 37

‐ 38) is not easy for the reader to process. I realize that the data are

lumped by emission level, but displaying the data in three separate charts, over two pages, is

confusing and difficult to understand. It would be preferable to show these data on a single

page (similar to Figure 7

‐ 2, page 31) or even in a single chart (similar to Figure 7

‐ 1, using a

logarithmic scale). It would also be better to order the vehicles along the x

‐ axis in the same way

they are listed in Table 7

‐ 5. Finally, it would be helpful to see more Bags/pollutants presented in

this way

– not just Bag 2 NOx.

RESPONSE TO THREE COMMENTS ABOVE:

EPA appreciates the comments. We have incorporated many of the recommended edits

directly into the report. Also, we have included additional boxplots for other pollutants and

bags in the Appendix.

[22] On page 38, the enhanced emissions reduction benefits of the 5 ppmS clean

‐ out are

described as compared to the 28 ppmS clean

‐ out (comparing Tables 7

‐ 4 and 7

‐ 6). It is

interesting to note that this does not apply to PM emissions. Some explanation should be

offered as to why PM emissions show such a different behavior. Also, the last sentence on page

38 mentions confirmation of results that have not even been presented yet. The placement of

this sentence seems odd.

[23] The same comments given above regarding data display in Figure 7

‐ 3 apply to Figure 7

‐ 4

(pages 41

‐ 42). The current grouping of vehicles shown in Figure 7

‐ 4 makes it very difficult to see

the important point being made on page 40 that some vehicles within the same family had

markedly different emission profiles. This would be much easier to see if a single chart were

used to display all vehicles

– similar in structure to Figure 7

‐ 1. Also, the box plots in Figure 7

‐ 5

(pages 43

‐ 44) should all appear on a single page, and should have a legend to explain the

symbols being used. Further, it would be helpful to identify each vehicle ID number shown inPage 19 Figure 7

‐ 5 by make/model; e.g. Vehicle ID 0003 is a Toyota Corolla. Again, additional figures

should be shown for other Bags and pollutants.

[24] Figure 7

‐ 8 (page 50) is very helpful in convincingly demonstrating the reasonableness of the

derived statistical model in predicting Bag 2 NOx emissions with both high and low sulfur levels.

It would be useful to include similar plots for other Bags/pollutants.

RESPONSE TO THREE COMMENTS ABOVE:

We have modified the report to address these comments.

[25] At the bottom of page 50, where Figure 7

‐ 9 is being discussed, it is stated that “the rate of

sulfur loading is the same for both high and low sulfur levels.” What is the basis for this

statement? Sulfur loading is not measured directly. Is NOx emission rate taken as a surrogate for

sulfur loading? (This statement about sulfur loading is also given in the Summary and

Conclusions Section on page 59.) Also, Figure 7

‐ 9 shows quite clearly that for some vehicles, not

only are the emission rates higher with the high sulfur fuel, but also the increase in emissions

with mileage accumulation is higher. For example (see vehicles 0075, 0123, 0264, 0178, and

0179). This seems inconsistent with the statement that rate of sulfur loading is the same for

both high and low sulfur levels.

RESPONSE:

Yes, the changes in emission rates were considered as an indicator of the amount of sulfur

loading. In addition, the model showed that the changes in emission rates were constant on a

percentage basis between the two sulfur levels.

[26] In Table 7

‐ 10 (page 53) the fixed effects in the NOx statistical model include only sulfur level

and miles for all three FTP Bags. Yet the FTP Composite result includes a sulfur level by miles

interaction term. This seems strange, since the FTP Composite emissions are simply calculated

as a combination of the individual Bag emissions. Some explanation should be offered.

RESPONSE:

This report simply presents the findings from the data. The study wasn’t designed to evaluate

specific mechanisms, so trying to explain trends found in the data would seem speculative.

[27] Pages 53

‐ 54 discuss the problem of comparing emissions from high and low sulfur levels

when there is a sulfur level by mileage interaction. The approach taken to address this (as

illustrated in Figure 7

‐ 10) seems rather arbitrary, although it may be as good as any other

approach. However, to explore the impact of this selection, it might be good to conduct some

sensitivity analyses using other methods.

RESPONSE:

EPA appreciates the comment. Considerations were given to other methods to estimate the

in

‐ use sulfur effect in the report text. However, since the method of using the in

‐ use emission

level upon arrival was clearly superior to others, no additional sensitivity analyses were

conducted.Page 20 [28] On page 55, it is stated that “For all models except CO Bag 1, CO Bag 1

– Bag 3, and CH 4 Bag

1

– Bag 3, the reduction estimates are statistically significant ...” However, this is not true for the

pollutants NOx + NMOG or PM.

RESPONSE:

We have modified the report to address this comment.



### 3.2 Editorial Comments

All of the reviewers provided significant editorial comments. They provided suggestions for

strengthening the report through clarification and elaboration. In addition, the reviewers noted

typographical and formatting errors and textual omissions.

RESPONSE:

EPA appreciates the comments. We have incorporated many of the recommended edits

directly into the report. Lists of references and acronyms were included in the report as

suggested.

Durbin:

 It might be useful to discuss the Umicore study cited in the Executive summary, since usually the

executive summary is more condensed than the Background.

 Executive summary

– the number of spaces between sentences differs from to in some cases 3

or even 4. This should be made consistent.

 p. 2 The end of the first sentence and start of the second redundantly talk about catalyst

efficiency.

 p. 2 The final sentence of the first paragraph should be split into two sentences.

 p. 2 Second to last sentence

‐‐ rewrite... for a PZEV operated on a 3 ppm fuel compared to a 33

ppm fuel.

 p. 3 third sentence

– In response

.... To address this question, the EPA (spell out EPA first use)

 p. 3 last sentence

– split it

– performance

. F or Example,

 p. 4 2 nd line

– fuel s (add s); Also split the second to last sentence.

 p. 5 need space for 5 ppm and 28 ppm, and period after “overall”. This sentence should

probably also be split.

 p. 5 3 rd sentence

– rewrite

– a significant miles

‐ by

‐ sulfur interaction was not found from the

model fitting....

 p. 5 last sentence

– rewrite

– In this case, the relatively differences with sulfur level varied as a

function of mileage, so determination.....

 p. 7

– 1 st sentence

– “has long been shown” is awkward. Also in executive summary

 p. 8, 9, and 10

– comma after e.g.

,

 p. 8 comma after idles)

, comma after in 2005

,

 p. 9

– 1 st line

– spell out MSAT acronym; line 7

– word “specimens” is a bit awkward; 6 th line

form bottom “benefits of further sulfur control”

 p. 11

– about line 10 comma after high temperatures), line 15 “conditions that are favorable”

line 16 lines further down “A vehicle with a relatively..., reference 10 on the E

‐ 60 report is not

referenced properly and should contain more author names.Page 21  p. 12

– 1 st line operation

, (add comma); 1 st line of section 4.1 “with the intent of being to be

representative of the latest...”;

 p. 12

– last sentence table 4

‐ 1 should not be underlined. This same changed should be

incorporated throughout the report, as it appears there is something in the formatting

underlining table and figure references in the text.

 p. 13

‐ First line

– “Vehicles recruited for testing were targeted to have a mileage between

12,000 and 40,000 miles and an age of less than three years old.”

 p. 14

‐ line 5 up front bottom

– exhaust leaks

, (add comma)

 p. 16

‐ First full paragraph

– Following the fuel change

,

 p. 16

‐ last sentence

– rewrite to “The Long and Short procedures are shown in Figure 6

‐ 1 and

are discussed in greater detail below. The Long and Short procedures are identical in structure

for the first six emissions tests.

 p. 17

‐ third line

– start a new paragraph about the Long procedure

 p. 20

– last sentence

– Split into two sentences.

 p. 21

– 3 rd paragraph

– detail

. but t The

 p. 27

– 2 nd paragraph line 4 “vehicles which was were “...., later in same paragraph Besides this

, (add comma), 3 rd paragraph line 2

– effects

, (add comma)

 p. 28

– line right after equation

‐ respectively

, (add comma), 3 rd paragraph

– line 2

–

model

, (add comma)

– line 4 structure

, (add comma)

 p. 29 1 st paragraph line 3 exchange “were” for was ‐ line 4

– levels

, (add comma)

–

 p. 32

– last paragraph line 3

– covariance

, (add comma)

 p. 33 line 6

– selected

, (add comma)

 p. 35 last paragraph 1 st line

– “dataset include d 17”

......

 p. 38

– 2 nd line from bottom

– Section 7.3.3

, (add comma)

 p. 40

– 3rd line from bottom

– i.e.

, (add comma)

 p. 40

– 5th line from bottom

– rewrite

‐ considering each vehicle as a random effect s might be

useful

 p. 42

– The sentence that includes Figure 7

‐ 6 should be broken up into two sentences. Example:

This is shown in figure 7

‐ 6, which shows the log

‐ transformed emissions from individual vehicles

by sulfur level. 5 th sentence from bottom

– add comma

... some vehicles

, and suggests....

 p. 44

– 2 nd line from bottom

– add comma

... vehicle to vehicle

, and simple descriptive

statistics...

 p. 45

– 1 st line

– rewrite

.. a similar top

‐ down model fitting statistical approach was applied to

the clean

‐ out data, as described in Section 7.3.1.

 p. 45

– 2 nd sentence

– rewrite

.. Furthermore, additional analysis was done that used a subset of

the sulfur level dataset that isolates the emission measurements immediately following the

clean

‐ out to address the effectiveness of the clean

‐ out cycle in reducing emissions.

 p. 46

– 2 nd paragraph 4 th sentence

– add comma

... irregularly

, where

 p. 46

– 3 rd paragraph 2 nd sentence

– ....exponentially with time

, i.e., the variability....

 p. 50

– 2 nd to last line

– operation

, (add comma) causing an increase in emissions

 p. 58

– 1 st paragraph

– line 3 through the presence, line 4 (RLD), (add comma), line 5 after an

iterative...

 p. 58

– 1 st paragraph

– line 3 through the presence, line 4 (RLD), (add comma), line 5 after an

iterative...

 p. 59

– 2 nd paragraph under “summary and conclusions”

‐ ...US06 cycles

. This data was used to

examine the existence...Page 22  “Summary and Conclusions”

– Under the first bullet point. Performing a clean

‐ out cycle with a

28 ppm fuel

 The second bullet point could be clearer. For example: for a subset up vehicles tested on both 5

ppm and 28 ppm fuels, it was found that additional FTP composite reductions of 18% for NO

x ,

9% for NMHC, and 8% for CO were for the 5 ppm fuel in comparison with the 28 ppm fuel.

Heck:

 In the report, there are a number of places where symbols are not explained in tables and in

text. Also, I did not see an explanation about how one does a hot/cold start FTP. Please make

sure this is explained in the text of the report.

 At end of papers, please a list of references. I had to go through the entire paper many times to

see if a reference was cited. This was painful!

 p. 2. Fuel sulfur content has long been understood to affect the performance of emission after

treatment catalysts in light duty vehicles, where the sulfur and/or its oxides adsorb to the active

precious metal sites and oxygen storage materials

, reducing the catalyst’s efficiency in

destroying harmful pollutants.

 p. 2. The quantity of sulfur present on the catalyst at any given time is a function of its

temperature, air to fuel ratio

, and the fuel sulfur level

. . .

 p. 2.

. . . with elevated catalyst temperature

, rich of stoichiometric operation, and lower fuel

sulfur concentration both reducing sulfur loading.

 p. 4 [Table ES

‐ 1]. Need to explain what p

‐ value is in the text? Also what is in (?????)?

 p. 5. [Table ES

‐ 2]. Again what is p value and what is in (....)?

RESPONSE TO TWO COMMMENTS ABOVE:

The numbers in parenthesis represent the p

‐ value and the explanation is already clearly stated

in the table.

 p. 5. This analysis found highly significant reductions for several pollutants, as shown in Error!

Reference source not found. ; reductions for Bag 2 NOx were particularly high, estimated at 59

percent between 28ppm and 5ppm overall.

 p. 17 [Figure 6

‐ 1]. Do you explain what a hot/cold start FTP is anywhere in the report??

 p. 25 [Table 7

‐ 1]. What does the value in (...) mean?? Please explain.

RESPONSE:

The text was revised to explain the number in parenthesis.

 p. 35 [Table 7

‐ 4]. Explain what is in (....)?

 p. 39 [Table 7

‐ 6]. Explain what is in (......)?

RESPONSE TO TWO COMMMENTS ABOVE:

The numbers in parenthesis represent the p

‐ value and the explanation is already clearly stated

in the table.

 p. 46. “The BIC value for the first

‐ order autoregressive structure was 764.90.” Did you explain

BIC?? If not then do so.Page 23 RESPONSE:

The Schwarz Bayesian Criterion (BIC) was explained in detail earlier in Section 7.3.1. However,

we added a list of acronyms to assist the reader.

 p. 47 [Table 7

‐ 8]. What is Pr? Explain.

RESPONSE:

The heading “Pr > F” represents the p

‐ value associated with the F statistic of a given source.

The description of the heading was added in the table.

 p. 55 [Table 7

‐ 10]. Is there an explanation as to why these lines diverge and are not parallel??

Need some discussion here from the model.

RESPONSE:

The lines diverge because the model has a significant sulfur level and mileage interaction term

and this was explained in 2 nd paragraph of page 54.

 p. 56 [Table 7

‐ 11]. What is in (....)?

RESPONSE:

The numbers in parenthesis represent the p

‐ value and the explanation is already clearly stated

in the table

.

 p. 57 [Table 7

‐ 13]. Is “Probt” probability??

RESPONSE:

“Probt” represents the p

‐ value associated with the t

‐ test. The table heading was revised.

Hoekman:

 p. 3, footnote: “The program has collected additional data that is being incorporated ...”

 p. 5, 5 th line: Insert period after “overall.”

 p. 6, footnote to Table ES

‐ 3: It is stated that reduction estimates were computed differently for

Bag 1 THC and CH 4 because these clean

‐ out effects were not statistically significant. Table ES

‐ 1

shows that the clean

‐ out effects were also not statistically significant for Bag 1 NOx and NMHC.

RESPONSE:

It was necessary to use the least squares means for Bag 1 THC and CH4 because they had a

significant sulfur and mileage interaction term, while having not statistically significant clean

‐

out effect. However, for Bag 1 NOx and NMHC, since the interaction term was not significant,

adjusting for in

‐ use sulfur loading was not needed.

 p. 6, next to last line: “... are consistent with those formed in the MSAT and Unicor studies ...”

 p. 17, Figure 6

‐ 1: The sulfur clean

‐ out cycle in the “short procedure” (blue

‐ colored box) should

indicate 28 ppmS, not 30 ppmS.

 p. 29, 2 nd line: “In addition, the results from the clean

‐ out data were used to supplement ...”Page 24  p. 30, 4 th line: “... representing the as received sulfur level ...”

 p. 32, 2 nd line: “... vehicles from the same vehicle family have the similar emission profiles.”

 p. 40, 4 th line: “However, the sulfur loading effect certainly varies by vehicle ...”

 p. 46: the term “BIC” is used without definition. The reader must refer to page 32 to see that

BIC refers to the Schwarz Bayesian Criterion. It would be helpful to include a table of acronyms

and abbreviations, as many rather obscure terms are used throughout this report

.

 p. 58, 1 st ¶, 5 th line: “... which is calculated after an iterative process ...”

 p. 59, Summary and Conclusions: the first bulleted finding should be clarified to indicate that

the stated NOx, NMHC, and PM reduction values were obtained using a clean

‐ out procedure

with 28 ppmS. The second bulleted finding should be clarified to indicate that although clean

‐

out with 5 ppmS fuel rather than 28 ppmS fuel further reduced NOx, NMHC, and CO, no effect

was observed for PM.