Peer Review of the Effects of Fuel Sulfur Level 

on Emissions from the In-Use Tier 2 Vehicles 

Prepared for

Assessment and Standards Division

Office of Transportation and Air Quality

U.S. Environmental Protection Agency

Prepared by

Systems Research and Application Corporation

652 Peter Jefferson Parkway, Suite 300

Charlottesville, VA 22911

EPA Contract Number: EP-C-11-007

February 2012
Page 2

Peer Review of the Effects of Fuel Sulfur Level 

on Emissions from the In-Use Tier 2 Vehicles 

Table of Contents

Peer Review of the Fuel Sulfur Effects Report, Conducted by SRA International p. 3

1. Background p. 3

2. Description of Review Process p. 4

3. Compilation of Review Comments p. 4

4. References p. 20

Appendices

A. Resumes of Peer Reviewers p. 21

B. Conflict of Interest Statements p. 29

C. Peer Review Charge p. 37

D. Reviews p. 39
Page 3

TO: Kent Helmer, Connie Hart, U.S. Environmental Protection Agency, Office of 

Transportation and Air Quality (OTAQ)

FROM: Brian Menard, SRA International

DATE: February 23, 2012

SUBJECT: Peer Review of The Effects of Fuel Sulfur Level on Emissions from the In-Use Tier 2 

Vehicles (Fuel Sulfur Effects Report)

1. Background

Sulfur in gasoline has long been known to reduce the efficiency of automobile exhaust emission 

aftertreatment systems. Some emission studies have suggested an increase in catalyst sensitivity to 

sulfur (i.e., binding to active catalytic sites) with increasing stringency of vehicle exhaust emission 

standards. This may be a result of higher catalytic efficiencies required for compliance with recent 

emission standards. EPA, in promulgating its Tier 2 light-duty vehicle emission standards, recognized the 

importance of fuel sulfur level. Reductions to new vehicle exhaust emission standards under the Tier 2 

vehicle and fuels program were accompanied by corresponding reductions in fuel sulfur level to improve 

the cost and feasibility of the vehicle technology changes required for compliance.

In the 2005/2006 timeframe, EPA’s Mobile Source Air Toxics/MSAT-2 study1examined the effects of fuel 

sulfur level on exhaust emissions from a test fleet of nine Tier 2-compliant cars and trucks. The study 

examined four gasolines, without ethanol and blended in a step-wise manner from a base fuel at 6 ppm 

S to the final fuel at 32 ppm S. FTP-weighted emission reductions related to fuel sulfur changes in this 

program for exhaust emissions were all statistically significant (α = 0.9). Due to specific catalyst prep 

procedures that compared a cleaned-out catalyst with low sulfur fuel to a loaded catalyst with high 

sulfur fuel, these results were thought to represent a “worst case” only. The data, though, suggested 

reversible sulfur loading was occurring in Tier 2 vehicle catalysts and significant emission reductions 

were likely possible through further fuel sulfur level control. 

The test program described in this report used two fuels with properties similar to conventional federal 

certification gasoline, except that the “lower sulfur” fuel contained 5 ppm sulfur. The “higher sulfur” 

fuel had a level similar to the national average in-use fuel at approximately 30 ppm sulfur. The types of

data targeted by the program were the level of “reversible” loading in catalysts found in-use Tier 2 

vehicles and the relative emission differences due to sulfur re-loading for two different sulfur level fuels 

in the same vehicle. In order to generate these emission data, privately-owned in-use vehicles were 

recruited for the study. It was not feasible to damage or destroy catalysts to directly measure any sulfur 

loading and, so, the behavior of emissions relative to a baseline was used as a proxy for catalyst sulfur 

loading. 

EPA sought an expert peer review of the Fuel Sulfur Effects Report. This report documents the peer 

review. Section 2 of this memorandum describes the process for selecting reviewers, administering the

review process, and closing the peer review. Section 3 summarizes reviewer comments according to the 

 

1

““Proposed Rule: Control of Hazardous Air Pollutants from Mobile Sources” Preamble and Regulations (published 

March 29, 2006). See http://www.epa.gov/oms/toxics.htm.
Page 4

series of specific questions set forth in the peer review charge. The appendices to the memorandum 

contain the peer reviewers’ resumes, completed conflict of interest, and bias questionnaires for each 

reviewer, and the peer review charge letter. 

2. Description of Review Process

In October 2011, OTAQ contacted SRA International to facilitate the peer review of the Fuel Sulfur 

Effects Report. EPA provided SRA with a short list of subject matter experts from academia, consulting,

and industry to serve as a “starting point” from which to assemble a list of peer reviewer candidates. 

SRA selected three independent (as defined in Sections 1.2.6 and 1.2.7 of EPA’s Peer Review Handbook, 

Third Edition) subject matter experts to conduct the requested reviews. SRA selected subject matter 

experts familiar with statistical analysis and vehicle emissions. To ensure the independence and 

impartiality of the peer review, SRA was solely responsible for selecting the peer review panel. 

Appendix A of this report contains the resumes of the three peer reviewers. A crucial element in 

selecting peer reviewers was to determine whether reviewers had any actual or perceived conflicts of 

interest or bias that might prevent them from conducting a fair and impartial review of the Fuel Sulfur 

Effects Report. SRA required each reviewer to complete and sign a conflict of interest and bias 

questionnaire. Appendix B of this report contains an explanation of the process and standards for 

judging conflict and bias along with copies of each reviewer’s signed questionnaire.

SRA provided the reviewers a copy of the most recent version of the Fuel Sulfur Effects Report as well as 

the peer review charge containing specific questions EPA asked the reviewers to address. Appendix C of 

this report contains the memo to reviewers from SRA with the peer review charge.

SRA delivered the final review comments to EPA by the requested date. These reviews, contained in 

Appendix D of this report, include the reviewers’ response to the specific charge questions and any 

additional comments they might have had.

3. Compilation of Review Comments

The Fuel Sulfur Effects Report was reviewed by Dr. Thomas Durbin (University of California, Riverside); 

Dr. Ronald Heck (Independent Consultant); and Dr. S. Kent Hoekman (Desert Research Institute). 

Appendix A contains resumes for each of the reviewers. This section consolidates their comments. The 

complete comments of the three reviewers may be found in Appendix D. 

The reviewers were asked on the basis of their work experience and expertise to provide expert opinion 

on all aspects of the report. In particular, they were asked to assess the methodologies used to evaluate 

the effects of both “cleanout” and sulfur level on in-use Tier 2 vehicle emissions and to indicate whether 

these techniques are likely to yield accurate results. In addition, they were asked to consider the

appropriateness of the statistical techniques used to analyze the data described in this report, their 

appropriateness in the context of data accuracy and quality issues and the overall conclusions drawn in 

this report pertaining to reducing fuel sulfur levels in the light-duty fleet. Lastly, EPA provided a series of

specific questions for reviewers to address in their comments. 

The reviewers were in general agreement as to the importance of the fuel sulfur effects study and the 

significance of its results. In this section of the memorandum, their written comments are consolidated 

into three categories: (1) comments in direct response to questions set out in the peer review charge; 
Page 5

(2) other technical comments; and (3) editorial comments. All textual edits and corrections provided by 

the reviewers in Section 3.2 are indicated in strikethrough and bold type.

3.1 Technical Comments

The reviewers provided a significant number of specific technical comments and were generally 

favorable in their reviews of the technical and statistical aspects of the Fuel Sulfur Effects Report. This 

section contains their comments and is divided into those that specifically address either questions or 

requests contained in the peer review charge and additional technical comments that reviewers chose 

to provide.

3.1.1 Charge Questions & Requests for Comment

In addition to encouraging reviewers to best apply their particular area(s) of expertise to review the 

overall study, EPA drafted six questions or requests for comment to serve as a focus for the reviewers. 

These were included in the peer review charge provided to the reviewers. In varying degree, the three 

reviewers provided direct responses to the questions or requests for comments. 

1. Was the imputation method used for replacement of measurements with low concentration 

reasonable? What other alternatives may have been better in this case? (Section 7.1)

Durbin: The text talks about tailpipe emissions being greater than zero, while the actual question of 

relevance may be – Are the tailpipe emissions greater than the ambient background 

concentrations? (presuming a relatively clean background) If the vehicle’s emissions is below 

background levels, than it would not be making an additive contribution to pollution levels. In 

this case, you could get a distribution of both positive and negative values fluctuating around 

zero.

My concern with the imput value procedure is that it might introduce a slight positive bias. My 

preference would probably be to use zeros as the imput values, since this would represent a 

distribution of positive and negative values that would average to zero over a larger dataset. 

For the cases where the dilute exhaust concentrations are lower than the measured 

background. By eliminating, or essentially not allowing, negative or zero values, doesn’t this add 

a positive bias to the results? Or how do you ensure there is a not a positive bias.

p. 24 – first paragraph – its states that a data point can either be deleted or replaced with an 

imput value. Couldn’t the value just simply be left as is? Also, if an imput value is deleted for a 

particular bag, presumably no composite emissions would be available. Would the 

corresponding test be eliminated in its entirety? 

p. 25 last line – it’s unclear how an outlier could be an imput value. Is this a case where the 

imput value is a very low value? A description of what these values might be should be added to 

the text. Again, if the imput outlier value is eliminated, how does that impact the result of the 

test?

Can Table 7-2 be modified to include not only the number of outliers identified, but also the 

number that were eliminated? This should also be in the text in the paragraph for section 7.1.2.
Page 6

Hoekman: [T]he method used to impute emissions values in cases where measured levels were lower 

than background is probably the best that can be done. The approach of developing an 

imputation value based upon each specific vehicle family seems better than using a single, fleetwide imputation value. The fact that imputed values were applied to only about 20% of the 

vehicles provides some assurance that this imputation method did not excessively distort the 

results. This was further confirmed by performing the statistical analyses with and without 

inclusion of the imputed values. One additional test that could be useful is to conduct the 

statistical analyses using values of zero instead of the imputed emissions values.

2. Please comment on the use of mixed model in analyzing the data. Was the model fitting 

strategy in selecting the final model statistically sound? (Section 7.2)

Durbin: The use of a mixed model is a relatively standard method to treat data analysis of emissions, 

and appears to be appropriate for the study design given here. The fitting strategy of using a 

saturated model, then developing the most optimal covariance structure, and then fitting the 

final model appears to be statistically sound. The step-wise backward elimination approach for 

the “Sulfur Effects data” also appears to be statistically appropriate.

One thought relating to the discussion of the univariate ANOVA and the multivariate MANOVA 

in section 7.2. Overall, it does not seem to add significant value to the report. Looking at the 

audience for this report, it would likely be composed of a mix of readers that may or may not 

have a statistical background. For those with a statistical background, the additional information 

on the univariate ANOVA and the multivariate MANOVA would not provide significant value. For 

those that are not as familiar with statistical methods, the added information would likely be 

more confusing than clarifying. It is suggested that this information, although interesting, might 

be better placed in an Appendix.

Hoekman: Overall, the statistical modeling approach seems appropriate. As described in Section 7.2 

(pp. 26-28) the structure and limitations of the emissions dataset make the linear mixed model a 

good choice for analysis, as this better accommodates missing data, irregularly spaced 

measurements, and within-vehicle effects. 

A significant question about the statistical modeling approach is the failure to include any 

information about the vehicles’ catalyst formulations, or other catalyst properties. It is not 

surprising to observe that vehicle type (car vs. truck) was not a significant model term, but it is 

surprising that there is no term related to the emissions control systems used across the range 

of vehicles. Surely not all catalysts were the same, and different catalysts would be expected to 

respond to changes in sulfur levels in different ways. This is an area that should be addressed. 

Inclusion of a catalyst term could result in quite different statistical models.

3. Were the model assumptions for the covariance structure reasonable given the data? 

(Section 7.3)

Durbin: For the “Clean-out data”, an unstructured covariance structure was initially used (which did not 

converge), and then a compound symmetry structure was selected. The compound symmetry 

structure is appropriate since the measurements from the same vehicle should have a 

homogeneous variance, as included in the text. For the “Effect of Sulfur data, where multiple 

measurements are made at different mileage accumulations, the autoregression covariance 
Page 7

structure is appropriate since the since the correlations between measurements is expected to 

decline as the measurements are further apart in terms of mileage accumulation, as mentioned 

in the text of the report.

4. Please comment on the methodology used in determining the in-use sulfur effect for models 

with and without the sulfur and mileage interaction term. (7.3.3)

Durbin: Overall, the methodology for determining the in-use sulfur effect for models with and without 

the sulfur and mileage interaction term appears to be sound. For cases where the sulfur level 

and mileage interaction term was not significant, the sulfur effect, sulfur loading and associated 

percentage differences between the high and low sulfur fuels should be constant as a function 

of increasing mileage. Thus, using the differences in the least squares means from the final 

model and reverse transforming them could be used to quantify the percentage differences 

between high and low sulfur for the case where the sulfur level and mileage interaction term 

was not significant. 

For cases where the sulfur level and mileage interaction term is significant, the rate of sulfur 

loading between the low and high sulfur fuels would differ as a function of mileage, so a 

different approach would be needed. The methodology of using the in-use emission level upon 

arrival (pre-cleanout) from a larger clean-out dataset and projecting it out to the mileage where 

the two lines intersect (in-use equivalent loading) seems reasonable for estimating the in-use 

sulfur level effect. Some clarification should be given on what is meant by the “larger clean-out 

dataset” since it is not immediately obvious what data these actually are. Additionally, how do 

the incoming sulfur levels for the “larger clean-out dataset” compare with average in-use values 

for the US fleet, or the fleet in different regions of the country. 

5. Is the interpretation of the mileage-by-sulfur interaction term presented correctly? (Section 

7.3.3)

Durbin: As discussed under question #4, the interpretation of the mileage-by-sulfur interaction term 

does appear to be presented correctly. On p. 48, when the interpretation of the interaction term 

is first being discussed, I would also add in something about the percentage differences in 

emissions being constant as a function of mileage, just to make things clearer. For example, for 

the last line of the paragraph just after Table 7-9: In other words, the effect of high fuel sulfur 

on Bag 2 NOx exists immediately after clean-out and remains constant on a percentage basis

during subsequent driving of a vehicle.

6. Are the sensitivity analyses on effects of low-level concentrations, imputed values, and 

influential vehicles sufficient as performed? Do the results from the sensitivity analyses 

provide additional support for the robustness of the conclusion? (Section 7.3.4)

Durbin: Overall, the sensitivity analyses see to add additional support to the robustness of the 

conclusions. Here are a few thoughts on the sensitivity analyses.

If the sensitivity analyses were only done for the NOx bag 2 emissions, it would be worth 

mentioning that the NOx bag 2 emissions showed a higher percentage of measurements with 

zero values than most other pollutant/bag combinations, as illustrated in Table 7-1.
Page 8

It would be interesting to get some feel for the vehicles/data eliminated in the sensitivity 

analysis for the low concentration measurements. Since these would be low emitting vehicles, it 

seems like they might have more robust and sulfur tolerant catalyst systems. This is consistent 

with the results showing an increase in the emissions reductions as these data are pulled out. 

Relating to the imput values, it would be interesting to see what the effect would be of making 

the imput values simply zero. Given the small differences between the model with and without 

the imput model, there would likely be only a minor impact, but this would add to the 

robustness of the results, although probably not change any conclusions.

For the influential vehicles, it is interesting to look at the vehicles selected. Vehicles 0007 and 

0178 show a relatively large sulfur effect, whereas vehicle 0046 shows a slight reverse sulfur 

effect with tight variability. It is interesting that other vehicles with relatively strong sulfur 

effects, such as 0165, 0179, and 0011, did not have a strong influence. The result of this 

sensitivity analysis is reasonably intuitive, with the NOx differences between sulfur levels 

shrinking, which would be consistent with removing some vehicles with a stronger sulfur effect.

Hoekman: Were the sensitivity analyses described in Section 7.3.4 (pp. 56-59) only conducted for Bag 2 

NOx? If so, this seems like a major limitation. Also, at the top of p. 59 it is mentioned that even 

when removing the influential vehicles from the analysis, sulfur effect “is still highly significant.” 

What are the significance levels before and after removing these influential vehicles?

In discussing the sensitivity analyses within the Summary and Conclusions Section (p. 60) it 

should be stated that these analyses were only performed for Bag 2 NOx emissions (if this is 

true).

3.1.2 Other Technical Comments

All of the reviewers provided specific technical comments in addition to their responses to the specific 

questions in the peer review charge.

Durbin:

[1] Study Design

Was the fuel commercially available in the Ann Arbor area at the time of the study oxygenated 

or not? This is of relevance since the test fuel is non-oxygenated.

30 ppm is given as the in-use sulfur average. Is this based on actual data of in-use sulfur 

samples, or is this based on regulatory average requirements.

In discussing why the US06 x2 clean-out cycle was used, it might be worth discussing the cleanout procedures that were used in the MSAT and Umicore studies.

[2] Test Vehicle Selection, Recruitment, and Delivery

What is the average sulfur level for the as received vehicles?

Section 4.1 should specifically mention that 19 makes/models were utilized in the study.
Page 9

Was any attempt made to characterize the vehicle categories by other factors such as level of 

long term fuel trim, or whether the vehicle used a closed coupled or double catalyst or only an 

underbody, or catalyst metals?

What incentive was provided to owners to participate in the program? – add to section 4.2.

More information should be given about the number of vehicles targeted for testing, and the 

number of vehicles in each vehicle make/model category. The first mention should be here, as 

opposed to in section 6.2. In that section, it talks about 5 vehicles for each make/model class. It 

would be useful to say the total number of vehicles tested at this point.

[3] Test Fuel Specs and Procurement

Some more details should be included about the fuel selection would be useful. How does this 

fuel compare to typical in-use fuel, in terms of say aromatics and RVP. Were oxygenates in use 

in the recruitment area? Or compared to averages of in-use used throughout the US?

[4] Test Procedures

It would be useful to add in fuel change points into the test procedure flow charts.

There is some lack of clarity in reading through this section that can be shown by looking at the 

last paragraph. The last paragraph indicates that only 4 of 19 family used the modified short 

procedure, whereas the first sentence of the paragraph indicates that the change was made 

approximately halfway through the program. Then looking at Table 7.7, it indicates that 

between 2-4 vehicles in each make/model class were given the modified short procedure, 

whereas it seems to imply in the paragraph that once the change was made, it was applied 

thereafter. Was the change just made for a subset of the vehicles tested in each vehicle 

category? Also, in the last sentence it indicates that the change in the number of vehicles 

providing “sulfur level data” is seen in Table 7-7. Does this mean that the modified short vehicles 

were added to the “sulfur level data” as well as the Long or modified Long procedure vehicles?

Related to the above comment, why not simply just say at the end of each procedure 

description how many vehicles were tested on each procedure?

Figure 6-3 indicates that only the 28 ppm sulfur level data is used for the “clean out effect”, but 

the very first sentence under section 7 indicates that there is a set of “clean-out” data at 5 ppm. 

[5] Data Analysis and Results

p. 21 – 3

rd paragraph – A statement should be added to answer the following question – Why do 

the results of the Bag 2 NOx emissions have more substantial implications than those of other 

pollutants/bags?

While it is useful to discuss the results at low concentrations, further information on the 

experimental methods might merit a section. For example, what analyzer is used? Can the 

detection limits of the analyzers be included in an Appendix? What methods/microbalance was 

used for the PM measurements, etc.?

p. 24 – top of p. There should be a brief paragraph discussing Table 7-1 and the number of zero 

value measurements before discussing the imput value methodologies. 

p. 24 – last sentence – It says that the statistical analyses were run both with and without 

imputed values. In terms of the “without” input value case, does that mean that the values were
Page 10

eliminated, were zeros used as the imput values, or were the original values used, whether they 

are positive or negative.

p. 25 – second paragraph – A new terms vehicle “families” is introduced. Presumably this term is 

the same as the vehicle make/models used earlier. Presumably these also have the same engine 

family and this should be included along with engine size, vehicle configuration, and weight.

p. 30 – There is no discussion on the symbols in the plots presumably the plots show the 

average and median inside the bar. Then the bars represent the 95% confidence levels and the 

error bars the full data range. Then there are some other dots on the plot? 

p. 30 – It is not clear what is meant by “Some vehicle families show the presence of withinfamily variability” Presumably this could also be due to differences in in-use sulfur levels as well, 

since a handful of vehicles came in with in tank fuels with sulfur levels much higher than those 

for others. For the post-cleanout variability, it seems like this should be independent of the inuse sulfur loading. Could some of the variability be related to the condition of the vehicle? 

Vehicle family M504 seems to show a very variability post-cleanout.

p. 32 – Related to BIC criterion. It indicates that the BIC performs relatively better for small 

sample size settings. How would one define 5 here? Is the current experiment a small sample 

size? 

Figure 7-4 does not appear to add significant value. Much of the same information is available in 

the box plots in Figure 7-5, and that is a little easier to read. It talks about the vehicles with 

similar emissions profiles being grouped. However, it is not immediately obvious what grouping 

were used for the Figures 7-4 and 7-5.

[6] This report presents an interesting approach to estimating the real effects of fuel sulfur in 

the Tier 2 in-use fleet. The study was very extensive in terms of the number of vehicles and level 

of testing that was done, and the results of the study some potentially important impacts of 

differing in-use sulfur levels. It would be interesting to evaluate these results further to 

determine the modeled impact of in-use fuel sulfur levels on ambient quality. The analysis 

methodologies in the report appear to be statistically sound, and appropriate for repeated 

measures types of testing. The presentation of the data is relatively extensive in terms of the 

descriptions of the statistical analysis procedures used. As such, the report would relatively 

straightforward to read for the some with a moderately strong background in statistics, but 

might a little too technical for some readers without a strong statistical background.

Heck:

[1] It is very important that the authors discuss right up front that they have taken an approach 

where the vehicles are considered as “black boxes”. This is a totally different approach than 

that adopted by the Umicore paper (Ball D., Clark D., Moser D. (2011). Effects of Fuel Sulfur on 

FTP NOx Emissions from a PZEV 4 Cylinder Application. SAE 2011 World Congress Paper 2011-01-

0300. SAE International: Warrendale, PA.) In the “black box “ approach there is no 

consideration of the engine control strategies (rich or lean bias, cold start strategy, etc.), sensor 

responses (from aging), vehicle operation (e.g. fuel cut), catalyst technologies, catalyst 

configurations(close coupled, underfloor, cell density, volume, etc.) See James M. Lyons, David 

Lax, and Steve Welstand, Investigation of Sulfur Sensitivity and Reversibility in Late-Model 

Vehicles, SAE 1999-01-3676 for work that takes a more phenomenological approach to sulfur 

effects. By the way I could not find this reference in the report. In the “black box” approach 

very little information is given besides the vehicle type, mileage, age, etc. Because of this lack of 

information it will be difficult for anyone to repeat the authors’ experiments and challenge there 

results. In addition it will be difficult for the authors to explain data anomalies because there 
Page 11

are no specifics to tease out a possible explanation. Also since there are so many uncontrolled 

variables, the statistical analysis may be limited since it is only statistics! This is not meant to say 

the work has no merit only to provide caution in the final conclusions and merits of too much 

statistics!

[2] One concern I have is if some of the vehicles have a fuel cut strategies. If so, this could 

confound the results since in the US06 the catalyst with a fuel cut strategy will see lean 

excursions while a catalyst without will be stoichiometric during the cycle. Again since no 

information is given in the report on engine operation it is impossible to go back and uncouple 

this effect.

[3] Sulfur poisoning is a complex phenomenon for catalysts and the catalyst manufacturers have 

found recipes to compensate for high and low sulfur levels (see Harold N. Rabinowitz, Samuel J. 

Tauster, Ronald M. Heck, The effects of sulfur & ceria on the activity of automotive Pd/Rh 

catalysts, Applied Catalysis A: General 212 (2001) 215–222). If the catalyst companies can 

design for Tier 2 vehicles at 30 ppm S, then this study may be in question or to put in another 

way, maybe some of the vehicles you studied already have these types of technologies and that 

is why you found some responses that didn’t show an effect of S level!

[4] The removal of sulfur from the catalyst needs a consistent statement throughout the paper. 

You show Figure 2-1 from Ford but the text is not consistent in describing this removal which is 

due to temperature, air to fuel ratio, and sulfur level. Also the degree of removal is different for 

Pt, Pd and Rh and the oxygen storage materials (Ce, Zr, La, etc.). So this discussion needs a little 

more technology.

[5] [p. 3] “Test fuels were two non-ethanol gasolines with properties typical of certification fuel, 

one at a sulfur level of 5 ppm and the other at 28 ppm.” – Is there any reason to expect 

different results with ethanol gasolines?

[6] [p. 4.] “This indicates that the catalyst is not fully desulfurized, even after a clean out 

procedure, as long as there is sulfur in the fuel.” – Not sure you can make this conclusion. It 

may be that the sulfur in fuel equilibrates instantaneously and it is a concentration effect not a 

desulfurization. To prove this you would have to vary the desulfurization time and see where 

it reaches steady state.

[7] [p. 5] “Other results, such as Bag 1 hydrocarbons, did show a significant miles-by-sulfur 

interaction.” – Does this mean that the effect never equilibrates? Are you comfortable in 

saying this?

[8] [p. 7] “The amount of sulfur retained by the catalyst is primarily a function of its operating 

temperature, the active materials and coatings used within the catalyst and the concentration of 

sulfur oxides in the incoming exhaust gases.” – Again air to fuel ratio is important as you 

mention in the next sentence for reducing conditions.

[9] [p. 8] “However, the temperatures necessary to release sulfur oxides can also lead to 

thermal degradation of the catalyst over time.” – Nowhere do you mention that normal 

operation of the vehicle, the catalyst is constantly being exposed to rich/lean condition from 
Page 12

the control system using an oxygen sensor. So the catalyst is being regenerated in-situ form 

the perturbation around stoichiometric.

[10] [p. 11] “The level of reversible in-use sulfur storage and release (or loading) within an 

exhaust catalyst system can be assessed by measuring emissions from the vehicle as received, 

performing a high speed, high load clean-out cycle, then measuring emissions again.” – In 

describing the cleanout it is important to mention the temperature range and the air to fuel 

ratio. Also depending on the vehicle calibration the degree of richness and time at such could 

be different from vehicle to vehicle. How is this taken into account for the degree of 

cleanout?

[11] [p. 11] “A vehicle with relatively high exhaust temperature at the catalyst location, and/or 

significant excess loading of certain platinum group metals (PGM) and other active materials in 

the catalyst may be relatively insensitive to sulfur loading regardless of driver behavior.” – Up to 

now you have not mentioned the effect of S on the oxygen storage components which will 

affect NOx and lightoff in cold start. Is there a reason this is not mentioned?

[12] [p. 12] “This loading continues over time with vehicle operation and can be observed as an 

increase in emissions (sometimes referred to in the auto industry as “NOx creep”).” – Is this due 

to PGM and/or oxygen storage component deactivation?

[13] [p. 13] Table 3-1 Test Vehicles Recruited – If you look at the Umicore study you will see 

details about the e3mission control system (close coupled, underfloor, etc.) You give no such 

detail in this study. Also, they also show a plot of engine lambda during the FTP. This is 

important background information to possibly explain outliers or unusual results. This study 

loses significant meaning without this information. You are treating the cars as black boxes 

and I think this is a mistake! Also, the oxygen sensor response is very important as well as it 

age. This is never mentioned in this entire study.

[14] [p. 38] “Furthermore, the reduction in emissions from cleanout shown in Error! Reference 

source not found. would likely be larger if the low sulfur test fuel at 5 ppm had been used for 

the cleanout procedure and the tests immediately following the as-received baseline 

emissions.” – This looks like a conjecture. Present evidence.

[15] [p. 40] “For example, vehicle IDs 0011, 0022, and 0178 clearly show large effect of fuel 

sulfur level on emissions while the effect is only marginal for vehicle IDs 0123 and 0264.” –

Differences can be A/F ratio, close coupled catalyst, oxygen storage components, temperature 

history. This is why more information on vehicle characteristics is important to explain 

unusual results.

[16] [pp. 59-60] “Comparing emissions immediately following the clean-out procedure on 5 vs. 

28 ppm fuel, FTP composite NOx emissions were 18% lower, NMHC 9% lower, and CO 8% 

lower.” – To make this conclusion you need to age with 28, clean out with 28 and test with 5 

and compare age with 5, cleanout with 5 and test with 5. I went back in text and really didn’t 

see this discussed in analysis. If I missed it OK otherwise need some discussion in analysis.
Page 13

Hoekman:

[1] The experimental test design was suitable for the intended purposes, although clearly, other 

testing procedures could have been used as well. For example, the standard FTP test procedure 

was used, at 75 F, to determine the emissions results from each vehicle test. Some comments 

should be offered regarding the suitability of the FTP driving cycle and a 75 F test temperature 

in providing representative/realistic operating conditions. For the in-use vehicle fleet, a set of 

standardized test conditions is clearly necessary to obtain repeatable and statistically significant 

results. However, the FTP test is now over 35 years old, and 75 F is just one of many 

temperatures that could be used. The overall quantitative effect of fuel sulfur on emissions as 

determined in this study would be more convincing if it were shown that similar results apply to 

other realistic operating conditions.

[2] The use of two, back-to-back US06 dynamometer cycles was an appropriate choice for 

catalyst regeneration. Although a somewhat more aggressive cycle might provide more 

complete desulfurization, potential damage to the vehicle was a significant concern, as 

explained in this report.

Also, the approach of catalyst clean-out, followed by repeated testing with a single fuel is a valid 

way to determine a new “equilibrium emissions level” at a specific fuel sulfur level. However, in 

real-world driving, such an equilibrium level is never attained, because operating conditions are 

constantly changing, and fuel sulfur varies from tank-to-tank.

[3] It is not clear why this report places so much emphasis on Bag 2 results – especially Bag 2 

NOx. Typically, Bag 2 emissions concentrations are extremely low from properly functioning Tier 

2 vehicles. In fact, as shown in this study, it is often difficult to distinguish Bag 2 levels from 

background levels. Large percentage differences between two very small numbers may not be 

very meaningful. To help put these results in perspective, it would be useful to show the 

relevant certification emissions levels for these Tier 2 vehicles. Also, something should be said 

about the impact of fuel sulfur reduction on fleet-wide emissions, and how large a reduction this 

represents in comparison to the entire mobile-source emissions inventory. In reality, this test 

program only captured a small portion of the entire in-use fleet (only Tier 2 vehicles of model 

years 2007 - 2009).

[4] Some historical perspective about the “FTP bag method” would be helpful here. This 

sampling and analysis method was developed about 40 years ago, to measure emissions from 

uncontrolled (or slightly controlled) vehicles. Prior to introduction of emissions control systems, 

Bag 2 emission levels were routinely 2 orders of magnitude higher than today. Under such 

conditions, the FTP bag method was perfectly suitable for the intended purpose. Now, with Bag 

2 emission levels being nearly indistinguishable from background, a different method should be 

employed for quantifying vehicle emissions. 

[5] In retrospect, it seems unfortunate that only non-ethanol gasolines were used, as such fuels 

are increasingly uncommon. It is stated that similar results would be expected from ethanolcontaining gasolines, but no data are provided to support this. If any subsequent testing has 

been done with ethanol-containing gasolines, this should be mentioned.
Page 14

[6] On page 2 (1st paragraph) the quantity of sulfur present on the vehicle’s catalyst is said to be 

a function of temperature and fuel sulfur level. Isn’t the catalyst formulation/metallurgy another 

important factor? (Catalyst formulation is mentioned on page 7.) What is known about the 

catalyst formulations used in each of the test vehicles of this study?

[7] It is interesting to note that when reducing fuel sulfur from 28 ppm to 5 ppm, significant 

emissions reductions were observed for all pollutants except PM, (See Tables ES-2 and ES-3.) Is 

there a clear mechanistic reason for this lack of effect for PM? Are the catalyst systems used in 

Tier 2 vehicles ineffective in reducing PM (and PM precursors)? This lack of a PM effect should 

be mentioned along with the major findings that are shown in bullet form on page 6. 

[8] On page 7 (final paragraph) it is stated that the impact of fuel sulfur on emissions was 

considered negligible under the Tier 0, Tier 1, and NLEV programs. I don’t think this is true. The 

emissions impacts of fuel sulfur have been well known for a long time, but more severe sulfur 

reductions were not thought to be a cost-effective emissions control approach until recently. To 

provide greater context, it would be useful to include a brief summary of gasoline regulations 

(including sulfur levels), and how they have evolved over the past 30-years. Along with this, a 

history of LD vehicle emissions standards should also be presented.

[9] On page 8, various detrimental effects of hot/rich catalyst operation are described –

including catalyst degradation and increased emissions of PM, NMOG, and CO. However, 

nothing is said about fuel economy effects – either here or elsewhere in the report. In general, 

fuel economy is a concern with any type of fuel or vehicle modification. This topic should at least 

be mentioned. In addition, the dataset generated in this study provides an excellent opportunity 

to evaluate the impacts of sulfur reduction on fuel economy, by analyzing CO2 as another 

pollutant, along with CO, THC, NOx, and PM. 

[10] The 2005 MSAT study, which used nine Tier 2 vehicles, is described on pages 8-9. The 

emissions reductions upon lowering fuel sulfur from 32 to 6 ppm are said to be 33% for NOx, 

11% for THC, 17% for CO, and 32% for CH4. Are these values simple averages of the nine 

vehicles, or were they computed by a more sophisticated method? Were similar statistical 

methodologies used to compute the fuel sulfur effects in both the MSAT study and the present 

study? Were PM emissions also measured in the MSAT study?

[11] When first describing the two test fuels on page 10, it would be helpful to refer the reader 

to Table 5-1 (page 15), which provides a more complete listing of fuel properties. Also, did these 

fuels contain anything unusual in the way of additives; e.g. antioxidants, detergents, dyes, etc.?

[12] In selecting the vehicles, was the ratio of LD trucks/LD passenger cars representative of the 

in-use fleet? Table 4-1 (page 13) which lists the 19 makes/models that were recruited, shows 

that the No. 1 U.S. sales rank vehicle is missing. Was this an oversight, or are the rankings shown 

in this table incorrect? Notice also that the Toyota Camry is shown as No. 23 in U.S. sales rank. 

This seems unbelievably low for what has traditionally been a “top seller.” Also, it would be 

useful to add a column to Table 4-1 to indicate the number of vehicles in each category that 

were tested in this study, comprising the total test fleet of 81. It might also be useful to indicate 

here the number of vehicles in each class that underwent the various test procedures: i.e. short 

procedure, long procedure, modified short procedure, and modified long procedure.
Page 15

[13] The modified long testing procedure shown in Figure 6-2 (page 19) is confusing. The blue 

box indicating the short procedure shows only two post cleanout FTPs, although the wording 

still indicates “triplicate FTPs at 28 ppmS.” Also, in describing the modified short procedure on 

page 20, it is said that “... the change in the number of vehicles providing sulfur level data can be 

seen in Table 7-7 starting with Family ID N513.” It is not clear what this means. Does this table 

list the vehicle families in chronological testing order, so that the first 13 families listed were 

tested using the standard short procedure, and the last 4 families listed were tested using the 

extended short test? It is also confusing that Tables 7-5 and 7-7 provide identical information.

[14] How is NMOG different from NMHC? What analytical procedures were used to measure 

these two pollutants?

[15] Displaying the background and sample measurements by FTP Bag (as done in Figure 7-1 and 

Appendix B) is very instructive. However, only NOx and THC results are shown in this way; it 

would be useful to also include CO and PM. It would also be helpful to draw ovals in Figure 7-1 

to capture all background measurements for each vehicle family shown. Although it is not 

possible to tell which sample points correspond to which background points, this graphical 

approach clearly illustrates the problem of very low emissions measurements relative to 

background in Bag 2. It also begs the question of why background levels of NOx are so variable. 

Figure 7-1 shows that these background concentrations varied by over an order of magnitude 

for many of the vehicle families tested.

[16] In the determination of outlier data points (described on page 25) what is the rationale for 

choosing an outlier screening criterion value of ± 3.5 for the studentized residuals? Also, 

although Table 7-2 identifies the number of outlier points for each pollutant/bag, it doesn’t 

indicate how many of these outlier points were actually excluded from the statistical analyses.

[17] As explained on page 26, log-transformation of emissions measurements has commonly 

been used to analyze vehicle emissions data. However, many previous emissions modeling 

studies have utilized test fleets that included a variety of technology types having a wide range 

of emission levels. In the present study, only Tier 2 vehicles were used, and the emission levels 

did not vary drastically across the test fleet. Given this situation, is log-transformation still 

necessary (and helpful)?

[18] On page 29 it is stated that the average starting odometer reading of the vehicles used to 

assess the clean-out effect at 28 ppmS was 31,470 miles. Is this value the average of the 17 

vehicle families shown in Table 7-3, or the average of all 81 vehicles? Similarly, is the ± value of 

1,578 miles the standard deviation of the 17 vehicle families or the 81 individual vehicles? (The 

same questions pertain to the vehicles used to assess the clean-out effect at 5 ppmS, described 

on page 35; and the vehicles used to assess the sulfur effects, described on page 39.)

[19] The box plots shown in Figure 7-2 (page 31) are quite informative in displaying the relative 

variances between the pre- and post-cleanout vehicle tests. It would be useful to show similar 

plots for other pollutants and Bags. Also, a legend should be included to explain the different 

symbols used in these plots.

[20] When describing the dependent variable (Yi) and effects (Xi and Zi) on pages 31-32, the 

reader should be reminded of the mixed model being used, shown as Equation 7-1 on page 28. 
Page 16

Also, it is stated on page 32 that “The significance of between-family variation was observed 

graphically in Figure 7-2 ...” But this figure only show Bag 2 NOx results. Was similar betweenfamily variation observed with other Bags and pollutants?

[21] Figure 7-3 (pages 37-38) is not easy for the reader to process. I realize that the data are 

lumped by emission level, but displaying the data in three separate charts, over two pages, is 

confusing and difficult to understand. It would be preferable to show these data on a single 

page (similar to Figure 7-2, page 31) or even in a single chart (similar to Figure 7-1, using a 

logarithmic scale). It would also be better to order the vehicles along the x-axis in the same way 

they are listed in Table 7-5. Finally, it would be helpful to see more Bags/pollutants presented in 

this way – not just Bag 2 NOx.

[22] On page 38, the enhanced emissions reduction benefits of the 5 ppmS clean-out are 

described as compared to the 28 ppmS clean-out (comparing Tables 7-4 and 7-6). It is 

interesting to note that this does not apply to PM emissions. Some explanation should be 

offered as to why PM emissions show such a different behavior. Also, the last sentence on page 

38 mentions confirmation of results that have not even been presented yet. The placement of 

this sentence seems odd.

[23] The same comments given above regarding data display in Figure 7-3 apply to Figure 7-4 

(pages 41-42). The current grouping of vehicles shown in Figure 7-4 makes it very difficult to see 

the important point being made on page 40 that some vehicles within the same family had 

markedly different emission profiles. This would be much easier to see if a single chart were 

used to display all vehicles – similar in structure to Figure 7-1. Also, the box plots in Figure 7-5 

(pages 43-44) should all appear on a single page, and should have a legend to explain the 

symbols being used. Further, it would be helpful to identify each vehicle ID number shown in 

Figure 7-5 by make/model; e.g. Vehicle ID 0003 is a Toyota Corolla. Again, additional figures 

should be shown for other Bags and pollutants.

[24] Figure 7-8 (page 50) is very helpful in convincingly demonstrating the reasonableness of the 

derived statistical model in predicting Bag 2 NOx emissions with both high and low sulfur levels. 

It would be useful to include similar plots for other Bags/pollutants.

[25] At the bottom of page 50, where Figure 7-9 is being discussed, it is stated that “the rate of 

sulfur loading is the same for both high and low sulfur levels.” What is the basis for this 

statement? Sulfur loading is not measured directly. Is NOx emission rate taken as a surrogate for 

sulfur loading? (This statement about sulfur loading is also given in the Summary and 

Conclusions Section on page 59.) Also, Figure 7-9 shows quite clearly that for some vehicles, not 

only are the emission rates higher with the high sulfur fuel, but also the increase in emissions 

with mileage accumulation is higher. For example (see vehicles 0075, 0123, 0264, 0178, and 

0179). This seems inconsistent with the statement that rate of sulfur loading is the same for 

both high and low sulfur levels.

[26] In Table 7-10 (page 53) the fixed effects in the NOx statistical model include only sulfur level 

and miles for all three FTP Bags. Yet the FTP Composite result includes a sulfur level by miles 

interaction term. This seems strange, since the FTP Composite emissions are simply calculated 

as a combination of the individual Bag emissions. Some explanation should be offered.
Page 17

[27] Pages 53-54 discuss the problem of comparing emissions from high and low sulfur levels 

when there is a sulfur level by mileage interaction. The approach taken to address this (as 

illustrated in Figure 7-10) seems rather arbitrary, although it may be as good as any other 

approach. However, to explore the impact of this selection, it might be good to conduct some 

sensitivity analyses using other methods.

[28] On page 55, it is stated that “For all models except CO Bag 1, CO Bag 1 – Bag 3, and CH4 Bag 

1 – Bag 3, the reduction estimates are statistically significant ...” However, this is not true for the 

pollutants NOx + NMOG or PM.

3.2 Editorial Comments

All of the reviewers provided significant editorial comments. They provided suggestions for 

strengthening the report through clarification and elaboration. In addition, the reviewers noted 

typographical and formatting errors and textual omissions. 

Durbin:

It might be useful to discuss the Umicore study cited in the Executive summary, since usually the 

executive summary is more condensed than the Background. 

Executive summary – the number of spaces between sentences differs from to in some cases 3 

or even 4. This should be made consistent.

p. 2 The end of the first sentence and start of the second redundantly talk about catalyst 

efficiency.

p. 2 The final sentence of the first paragraph should be split into two sentences.

p. 2 Second to last sentence -- rewrite... for a PZEV operated on a 3 ppm fuel compared to a 33 

ppm fuel.

p. 3 third sentence – In response....To address this question, the EPA (spell out EPA first use)

p. 3 last sentence – split it – performance. For Example, 

p. 4 2nd line – fuels (add s); Also split the second to last sentence.

p. 5 need space for 5 ppm and 28 ppm, and period after “overall”. This sentence should 

probably also be split.

p. 5 3rd sentence – rewrite – a significant miles-by-sulfur interaction was not found from the 

model fitting....

p. 5 last sentence – rewrite – In this case, the relatively differences with sulfur level varied as a 

function of mileage, so determination.....

p. 7 – 1

st sentence – “has long been shown” is awkward. Also in executive summary

p. 8, 9, and 10 – comma after e.g.,

p. 8 comma after idles), comma after in 2005, 

p. 9 – 1

st line – spell out MSAT acronym; line 7 – word “specimens” is a bit awkward; 6th line 

form bottom “benefits of further sulfur control”

p. 11 – about line 10 comma after high temperatures), line 15 “conditions that are favorable” 

line 16 lines further down “A vehicle with a relatively..., reference 10 on the E-60 report is not 

referenced properly and should contain more author names.

p. 12 – 1

st line operation, (add comma); 1st line of section 4.1 “with the intent of being to be

representative of the latest...”;
Page 18

p. 12 – last sentence table 4-1 should not be underlined. This same changed should be 

incorporated throughout the report, as it appears there is something in the formatting 

underlining table and figure references in the text.

p. 13 - First line – “Vehicles recruited for testing were targeted to have a mileage between 

12,000 and 40,000 miles and an age of less than three years old.”

p. 14 - line 5 up front bottom – exhaust leaks, (add comma)

p. 16 - First full paragraph – Following the fuel change,

p. 16 - last sentence – rewrite to “The Long and Short procedures are shown in Figure 6-1 and 

are discussed in greater detail below. The Long and Short procedures are identical in structure 

for the first six emissions tests.

p. 17 - third line – start a new paragraph about the Long procedure

p. 20 – last sentence – Split into two sentences.

p. 21 – 3

rd paragraph – detail. but tThe

p. 27 – 2

nd paragraph line 4 “vehicles which was were “...., later in same paragraph Besides this,

(add comma), 3rd paragraph line 2 – effects, (add comma)

p. 28 – line right after equation - respectively, (add comma), 3

rd paragraph – line 2 –

model, (add comma) – line 4 structure, (add comma)

p. 29 1st paragraph line 3 exchange “were” for was - line 4 – levels, (add comma) –

p. 32 – last paragraph line 3 – covariance, (add comma)

p. 33 line 6 – selected, (add comma)

p. 35 last paragraph 1st line – “dataset included 17” ......

p. 38 – 2

nd line from bottom – Section 7.3.3, (add comma)

p. 40 – 3rd line from bottom – i.e., (add comma)

p. 40 – 5th line from bottom – rewrite - considering each vehicle as a random effects might be 

useful

p. 42 – The sentence that includes Figure 7-6 should be broken up into two sentences. Example: 

This is shown in figure 7-6, which shows the log-transformed emissions from individual vehicles 

by sulfur level. 5

th sentence from bottom – add comma ... some vehicles, and suggests.... 

p. 44 – 2

nd line from bottom – add comma ... vehicle to vehicle, and simple descriptive 

statistics...

p. 45 – 1

st line – rewrite .. a similar top-down model fitting statistical approach was applied to 

the clean-out data, as described in Section 7.3.1.

p. 45 – 2

nd sentence – rewrite .. Furthermore, additional analysis was done that used a subset of 

the sulfur level dataset that isolates the emission measurements immediately following the 

clean-out to address the effectiveness of the clean-out cycle in reducing emissions.

p. 46 – 2

nd paragraph 4th sentence – add comma ... irregularly, where

p. 46 – 3

rd paragraph 2nd sentence – ....exponentially with time, i.e., the variability.... 

p. 50 – 2

nd to last line – operation, (add comma) causing an increase in emissions 

p. 58 – 1

st paragraph – line 3 through the presence, line 4 (RLD), (add comma), line 5 after an

iterative...

p. 58 – 1

st paragraph – line 3 through the presence, line 4 (RLD), (add comma), line 5 after an

iterative...

p. 59 – 2

nd paragraph under “summary and conclusions” - ...US06 cycles. This data was used to 

examine the existence... 

“Summary and Conclusions” – Under the first bullet point. Performing a clean-out cycle with a 

28 ppm fuel
Page 19

The second bullet point could be clearer. For example: for a subset up vehicles tested on both 5 

ppm and 28 ppm fuels, it was found that additional FTP composite reductions of 18% for NOx, 

9% for NMHC, and 8% for CO were for the 5 ppm fuel in comparison with the 28 ppm fuel.

Heck:

In the report, there are a number of places where symbols are not explained in tables and in 

text. Also, I did not see an explanation about how one does a hot/cold start FTP. Please make

sure this is explained in the text of the report.

At end of papers, please a list of references. I had to go through the entire paper many times to 

see if a reference was cited. This was painful!

p. 2. Fuel sulfur content has long been understood to affect the performance of emission after 

treatment catalysts in light duty vehicles, where the sulfur and/or its oxides adsorb to the active 

precious metal sites and oxygen storage materials, reducing the catalyst’s efficiency in 

destroying harmful pollutants.

p. 2. The quantity of sulfur present on the catalyst at any given time is a function of its 

temperature, air to fuel ratio, and the fuel sulfur level . . .

p. 2. . . . with elevated catalyst temperature , rich of stoichiometric operation, and lower fuel 

sulfur concentration both reducing sulfur loading.

p. 4 [Table ES-1]. Need to explain what p-value is in the text? Also what is in (?????)?

p. 5. [Table ES-2]. Again what is p value and what is in (....)?

p. 5. This analysis found highly significant reductions for several pollutants, as shown in Error! 

Reference source not found.; reductions for Bag 2 NOx were particularly high, estimated at 59 

percent between 28ppm and 5ppm overall.

p. 17 [Figure 6-1]. Do you explain what a hot/cold start FTP is anywhere in the report??

p. 25 [Table 7-1]. What does the value in (...) mean?? Please explain.

p. 35 [Table 7-4]. Explain what is in (....)?

p. 39 [Table 7-6]. Explain what is in (......)?

p. 46. “The BIC value for the first-order autoregressive structure was 764.90.” Did you explain 

BIC?? If not then do so.

p. 47 [Table 7-8]. What is Pr? Explain.

p. 55 [Table 7-10]. Is there an explanation as to why these lines diverge and are not parallel?? 

Need some discussion here from the model.

p. 56 [Table 7-11]. What is in (....)?

p. 57 [Table 7-13]. Is “Probt” probability??

Hoekman:

p. 3, footnote: “The program has collected additional data that is being incorporated ...”

p. 5, 5th line: Insert period after “overall.”

p. 6, footnote to Table ES-3: It is stated that reduction estimates were computed differently for 

Bag 1 THC and CH4 because these clean-out effects were not statistically significant. Table ES-1 

shows that the clean-out effects were also not statistically significant for Bag 1 NOx and NMHC.

p. 6, next to last line: “... are consistent with those formed in the MSAT and Unicor studies ...”

p. 17, Figure 6-1: The sulfur clean-out cycle in the “short procedure” (blue-colored box) should 

indicate 28 ppmS, not 30 ppmS.

p. 29, 2nd line: “In addition, the results from the clean-out data were used to supplement ...”
Page 20

p. 30, 4th line: “... representing the as received sulfur level ...”

p. 32, 2nd line: “... vehicles from the same vehicle family have the similar emission profiles.”

p. 40, 4th line: “However, the sulfur loading effect certainly varies by vehicle ...”

p. 46: the term “BIC” is used without definition. The reader must refer to page 32 to see that 

BIC refers to the Schwarz Bayesian Criterion. It would be helpful to include a table of acronyms 

and abbreviations, as many rather obscure terms are used throughout this report.

p. 58, 1st ¶, 5th line: “... which is calculated after an iterative process ...”

p. 59, Summary and Conclusions: the first bulleted finding should be clarified to indicate that 

the stated NOx, NMHC, and PM reduction values were obtained using a clean-out procedure 

with 28 ppmS. The second bulleted finding should be clarified to indicate that although cleanout with 5 ppmS fuel rather than 28 ppmS fuel further reduced NOx, NMHC, and CO, no effect 

was observed for PM.

4. References

U.S. EPA (2011). The Effects of Fuel Sulfur Level on Emissions from the In-Use Tier 2 Vehicles
Page 21

Appendix A: Resumes of Peer Reviewers

Tom Durbin

Research Engineer

University of California, Riverside

Bourns College of Engineering

CE-CERT (Center for Environmental Research & Tech.)

Thomas D. Durbin is an associate research engineer in the vehicle emissions research group of 

CE-CERT. Dr. Durbin conducts research in a broad range of topics related to vehicle emissions 

including particulate matter (PM) emissions, diesel, biodiesel, and gasoline fuels, portable 

emissions measurement systems (PEMS), and the unregulated species such as ammonia, N2O, 

and toxics. Dr. Durbin conducts research in CE-CERT’s state-of-of-the-art emissions test facilities 

including a heavy-duty mobile emissions laboratory, a heavy-duty engine dynamometer 

laboratory, and a light-duty chassis dynamometer laboratory. Dr. Durbin is also extensively 

involved with in-field measurements of emissions from passenger cars, heavy-duty trucks, and 

construction equipment. Prior to joining the vehicle emissions group, Dr. Durbin was involved in 

several other areas of research at CE-CERT including renewable energy and fuel sources and 

advanced vehicle technologies. Tom Durbin received his doctorate degree in Physics from the 

University of California, Riverside, in 1994 where the primary focus of his dissertation was the 

study of Si films and solid lubricants. 

Degrees

B.S. Physics 1988; UCR 

M.S. Physics 1990; UCR 

Ph.D. Physics 1994; UCR 

Research Area

Vehicle emissions

Publications

Five Fuel-Related Publications

• Durbin, T. D., J. R. Collins, H. A. Galdamez, J. M. Norbeck, M. R. Smith, R.D Wilson and T. 

Younglove. 2000. Evaluation of the Effects of Biodiesel Fuel on Emissions from Heavy-Duty NonRoad Vehicles. South Coast Air Quality Management District, May. 

• Durbin, T.D., J.W. Miller, B.B. Holden, and N.L. Helgeson. 2003. The Effects of Biodiesel and 
Page 22

Other Alternative Diesel Fuels on Emissions from Diesel Vehicles. California Air Resources 

Board’s and California Energy Commission’s Alternative Diesel Fuel Symposium, Sacramento, 

CA, August. 

• Durbin, T.D. and J.M. Norbeck. 2002. The Effects of Biodiesel Blends and ARCO EC-Diesel on 

Emissions from Light Heavy-Duty Diesel Vehicles. Environ. Sci. & Technol. 36:1686-1691. 

• Durbin, T. D., J. R. Collins, J. M. Norbeck and M. R. Smith. 2000. The Effects of Biodiesel, 

Biodiesel Blends, and a Synthetic Diesel on Emissions from Light Heavy-Duty Diesel Vehicles. 

Environ. Sci. & Technol., 34, 349. 

• Durbin, T. D., X. Zhu and J. M. Norbeck. 2003. The Effects of Diesel Particulate Filters and a 

Low-Aromatic, Low-Sulfur Diesel Fuel on Emissions for Medium-duty Diesel Trucks. 

Atmospheric Environment, vol. 37, 2105-2116. 

Other Significant Publications

• Moosmüller, H.; Arnott, W.P.; Rogers, C.F.; Bowen, J.L.; Gillies, J.; Pierson, W.R.; Collins, J.F.; 

Durbin, T.D.; and Norbeck, J.M. (2001) Time Resolved Characterization of Diesel Particulate 

Emissions: 1. Instruments for Particle Mass Measurements. Environmental Science & 

Technology 35(4):781-787, 2/15. 

• Cadle, S.H.; Mulawa, P.A.; Ragazzi, R.A.; Knapp, K.T.; Norbeck, J.;M.; Durbin, T.D.; Truex, T.J.; 

and Whitney, K.A.(1998) Exhaust Particulate Matter Emissions from In-Use Passenger Vehicles 

Recruited in Three Locations-CRC Project E-24. SAE Technical Paper No. 99FL-215. 

• Rodriguez-Forker, A.; Uihlein, J.P.; Segal, J.S.; Sverdrup, G.M.; Seymour, J.P.; Kinateder, J.G.; 

Pierce, A.; and Durbin, T.D. (1998) Fleet Test Using Butane and Propane Mixtures. SAE Technical 

Paper No. 982444. 
Page 23

Dr. Ron Heck is currently an independent consultant. Prior to that, Ron was a research manager 

responsible for developing catalyst technology for Engelhard Corporation's worldwide customers 

in automotive catalyst. He has worked on the development of catalytic processes in SCR NOx, 

NSCR NOx, automotive catalyst, diesel catalyst, PremAirTM catalyst systems, hydrogenation 

technology, ozone abatement, volatile organic compound abatement, ammonia oxidation, chemical 

feedstock purification and chemical synthesis. Ron is a member of American Men and Women of 

Science and Who's Who in Technology Today. He is a recipient of the Forest R. McFarland 

award from the Society of Automotive Engineers for outstanding contributions to this 

professional society. Ron is an SAE Fellow in recognition of engineering creativity and 

contributions to the profession and the public at large. He is co-instructor for courses for the 

SAE in automotive emission control catalysis and diesel emission control catalysis. He was a 

member of the Scientific Advisory Board of the Strategic Environmental Research and 

Development Program for environmental studies in the Department of Defense. Ron has been 

involved in over 80 publications in commercial applications of catalysts and holds 36 U.S. 

patents on catalytic processes. He is the co-author of the book entitled "Catalytic Air Pollution 

Control: Commercial Technology" (1st, 2nd and 3rd edition) and is the former co-editor of the 

NewsBrief section of Applied Catalysis B: Environmental. Ron and his former research team 

from Engelhard received the 2004 Thomas Alva Edison Patent Award from R&D Council of 

New Jersey for the invention of close coupled catalyst technology for ultra low emission gasoline 

vehicles. Ron received his B.S. in Chemical Engineering and his Ph.D. from the University of 

Maryland, and a M.A. in Theology from the College of St. Elizabeth. 
Page 24

S. KENT HOEKMAN Tel: 775-674-7065

Research Professor Fax: 775-674-7016

Desert Research Institute email: Kent.Hoekman@dri.edu

Division of Atmospheric Sciences

2215 Raggio Parkway, Reno, NV 89512-1095

Education

Ph.D. 1980 Iowa State University, Ames, IA Organic Chemistry

B.S. 1975 Calvin College, Grand Rapids, MI Chemistry

Professional Interests and Activities

Dr. Hoekman is a Research Professor within the Division of Atmospheric Sciences at the 

Desert Research Institute (DRI). DRI is a statewide division of the Nevada System of Higher 

Education (NSHE) that pursues basic and applied environmental research on local, national, and 

international scales. His professional interests include environmental impacts of energy production, 

distribution, and use; development of renewable and sustainable energy systems; conversion of 

biomass to biofuels; air quality impacts of vehicle emissions; and impacts of advanced-technology 

fuels and vehicles on emissions and energy use. He is also interested in the interface between politics 

and environmental science, particularly in the areas of energy policy, renewable fuels, greenhouse 

gases, and climate change. 

In addition to his personal professional activities, Dr. Hoekman has provided leadership for 

DRI in the identification, protection, and licensing of intellectual property (IP) developed at the 

Institute. Dr. Hoekman was instrumental in establishing a joint Technology Transfer Office (TTO) 

between DRI and the University of Nevada, Reno (UNR). He currently serves as DRI’s Liaison to 

the TTO, and oversees the activities of this office on behalf of DRI.

Dr. Hoekman has also served DRI by coordinating and promoting the Institution’s R&D 

portfolio in the field of renewable energy. He has led the effort to establish a Renewable Energy 

Center (REC) at DRI, and continues to provide leadership in this area. For further information about 

the DRI-REC, please refer to its website at http://www.dri.edu/rec. 

In addition, Dr. Hoekman is active in the scientific academic and business communities. He 

serves as a reviewer for numerous science and engineering journals, is a member of several 

professional societies, has assisted in organizing scientific conferences, and contributes to the 

mentoring and advisement of graduate students at the University of Nevada in Reno. Currently, he 

serves as Associate Editor for the International Journal of Alternative Energy. 

From 2001 to 2007, Dr. Hoekman served as Executive Director of DRI’s Division of 

Atmospheric Sciences (DAS). DAS consists of approximately 50 research faculty, along with 70 

technologists, graduate students, post-docs, and other support staff. The Division conducts 

fundamental and applied research around the world on topics pertaining to emissions, renewable 

energy, air pollution, meteorology, climatology, aerosol chemistry and physics, and other areas 

related to atmospheric science. DAS also serves as the institutional home for the Western Regional 

Climate Center, one of six NOAA-funded regional climate centers in the U.S. As Director, Dr. 

Hoekman was responsible for all personnel, financial, organizational, and professional activities of 

Divisional operations. The Division’s scientific work is sponsored by over 100 federal, state, local, 

and private organizations that provide approximately $14 million per year in research grants and 

contracts. For more information about the Division and its activities, please refer to its web site at 

http://www.das.dri.edu.
Page 25

Prior to joining DRI in 2001, Dr. Hoekman spent over 20 years at Chevron, where his 

research focused on transportation fuels and their impacts on motor vehicle emissions and air quality. 

Experimental work included detailed characterization of exhaust emissions compositions from 

gasoline-, diesel-, and alcohol-fueled vehicles. Laboratory studies were conducted to investigate how 

changes in fuel formulation could reduce vehicle emissions and improve ambient air quality. He has 

served on several technical committees representing the American Petroleum Institute (API), the 

Western States Petroleum Association (WSPA), the Coordinating Research Council (CRC), and 

other industry organizations interested in fuels, emissions, atmospheric chemistry and air quality.

Dr. Hoekman also has experience in environmental regulatory affairs pertaining to vehicles, 

fuels, emissions, air quality, and health effects. He has served in technical advisory roles to EPA and 

was a member of the California Air Resources Board (CARB) Research Screening Committee for 

five years. He served as a member of the Health Effects Institute’s (HEI) Special Committee on 

Emerging Technologies from 2001 through 2007. 

Professional Experience 

2007 – Present Research Professor, Division of Atmospheric Sciences, Desert Research Institute, 

Reno, NV

2001 – 2007 Executive Director, Division of Atmospheric Sciences, Desert Research Institute, 

Reno and Las Vegas, NV 

1997 – 2001 Senior Staff Scientist, Chevron Products Co., San Francisco and San Ramon, CA

1990 – 1996 Staff Scientist and Senior Staff Scientist, Chevron Research and Technology 

Company, Richmond, CA

1980 – 1989 Research Chemist and Senior Research Chemist, Chevron Research and 

Technology Company, Richmond, CA

Professional Memberships

American Association for the Advancement of Science (AAAS)

American Chemical Society (ACS)

Air and Waste Management Association (AWMA)

Society of Automotive Engineers (SAE)

Awards/Honors

Chevron Chairman’s Award (1984) – Presented in recognition of diesel emissions research

Horning Memorial Award (1985) – Presented by the Society of Automotive Engineers

Arch T. Colwell Merit Award (1985) – Presented by the Society of Automotive Engineers

Society of Automotive Engineer’s Award for Excellence in Oral Presentation (1993 and 1995)

Recognition of Appreciation from the California Air Resources Board (2001)
Page 26

Peer-Reviewed Publications

Yan, W. and S.K. Hoekman (2012). Dust suppression with glycerin from biodiesel production: a 

review, J. Environ. Protection (in press).

Hoekman, S.K. and C. Robbins (2012). Review of the effect of biodiesel on NOx emissions, Fuel 

Proc. Technol. doi:10.1016/j.fuproc.2011.12.036 (in press).

Hoekman, S.K., A. Broch, C. Robbins, E. Ceniceros, and M. Natarajan (2012). Review of Biodiesel 

Composition, Properties, and Specifications. Renewable and Sustainable Energy Reviews 16, 

143-169.

Wang, X., C. Robbins, S.K. Hoekman, J.C. Chow, J.G. Watson, and D. Schuetzle (2011). Dilution 

Sampling and Analysis of Particulate Matter in Biomass-Derived Syngas. Front. Environ. 

Sci. Engin. China 5 (3) 320-330.

Robbins, C., S.K. Hoekman, E. Ceniceros, and M. Natarajan (2011). Effects of Biodiesel Fuels upon 

Criteria Emissions. Soc. Auto. Eng. Tech. Paper No. 2011-01-1943.

Hoekman, S.K., A. Broch, and C. Robbins (2011). Hydrothermal Carbonization (HTC) of 

Lignocellulosic Biomass. Energy Fuels 25 (4) 1802-1810.

Bruins, R., S.K. Hoekman, R. Efroymson, A.Aden, and A. Hecht (2010).Transportation Fuels for the 

21st Century. EM Nov. 2010, 26-32.

Hoekman, S.K., A. Broch, C. Robbins, and R. Purcell (2009). CO2 Recycling by Reaction with 

Renewably-Generated Hydrogen. International J. of Greenhouse Gas Control 4, 44-50.

Hoekman, S.K., A.W. Gertler, A. Broch, C. Robbins, and M. Natarajan (2009). Biodistillate 

Transportation Fuels 1 – Production and Properties. Soc. Auto. Eng. Tech. Paper No. 2009-

01-2766.

Robbins, C., S.K. Hoekman, A. Gertler, A. Broch, and M. Natarajan (2009). Biodistillate 

Transportation Fuels 2 – Emissions Impacts. Soc. Auto. Eng. Tech. Paper No. 2009-01-2724.

Broch, A., S.K. Hoekman, A. Gertler, C. Robbins, and M. Natarajan (2009). Biodistillate 

Transportation Fuels 3 – Life-Cycle Impacts. Soc. Auto. Eng. Tech. Paper No. 2009-01-2768.

Hoekman, S.K. (2009). Biofuels in the U.S. – Challenges and Opportunities. Renewable Energy 34, 

14-22.

Chow, J.C., S.K. Hoekman, J.M. Norbeck, K.N. Black, R.M. O’Keefe, D.L. Kopinski, M.P. Walsh, 

J.L. Suchecki, S.L. Altshuler, B. MacClarence, R.A. Harley, and D. Marrack (2001). Diesel 

Engines: Environmental Impact and Control. J. Air and Waste Management. Assoc. 51, 

1258-1270.

Hoekman, S.K., R.S. MacArthur, M. Naylor and J.A. Rutherford (1998). RVP Reduction for Control 

of Wintertime CO. Soc. Auto. Eng. Tech. Paper No. 981373.

Lanning, L.A., W.L. Clark, W.O. Siegl, S.K. Hoekman, R.M. Stanley and W.F. Biller (1997). CRC 

Hydrocarbon Emissions Analysis Round Robin Test Program, Phase II. Soc. Auto. Eng. 

Tech. Paper No. 971608.

Hoekman, S.K., J. Freel and R.S. MacArthur (1996). Reduced-RVP Gasoline – An Attractive 

Alternative to RFG. Soc. Auto. Eng. Tech. Paper No. 961281.
Page 27

Hoekman, S.K., R.M. Stanley, W.L. Clark, W.O. Siegl, A.M. Schlenker and W.F. Biller (1995). 

CRC Speciated Hydrocarbon Emissions Analysis Round Robin Test Program. Soc. Auto. 

Eng. Tech. Paper No. 950780.

Hoekman, S.K. and T.E. Jensen (1993). Methanol Vehicle Emissions Round Robin Test Program. 

Soc. Auto. Eng. Tech. Paper No. 932773.

Hoekman, S.K. (1993). Improved Gas Chromatography Procedure for Speciated Hydrocarbon 

Measurements of Vehicle Emissions. J. Chromatog., 639, 239-253.

Hoekman, S.K. (1992). Speciated Measurements and Calculated Reactivities of Vehicle Exhaust 

Emissions from Conventional and Reformulated Gasolines. Environ. Sci. Technol. 26, 1206-

1216.

Croes, B.E., S.K. Hoekman and A. Guerrero (1991). Ozone-Forming Potential of Emissions from 

Lower Aromatic Content Gasolines. Air and Waste Management Association Paper No. 91-

107.8.

Gething, J.A., S.K. Hoekman, A.R. Guerrero and J.M. Lyons (1990). The Effect of Gasoline 

Aromatics Content on Exhaust Emissions: A Cooperative Test Program. Soc. Auto. Eng. 

Tech. Paper No. 902073.

Eberhard, G.A., M. Ansari and S.K. Hoekman (1990). Emissions and Fuel Economy Tests of a 

Methanol Bus with a 1988 DDC Engine. Soc. Auto. Eng. Tech. Paper No. 900342.

Beyaert, B.O., S.K. Hoekman, A.J. Jessel, J.S. Welstand, R.D. White and J.E. Woycheese (1989). An 

Overview of Methanol Fuel Environmental, Health, and Safety Issues. Summer Meeting of 

American Institute of Chemical Engineers, Philadelphia, PA, August 22, 1989.

Eberhard, G.A., M. Ansari and S.K. Hoekman (1989). Emissions and Fuel Economy Test Results for 

Methanol- and Diesel-Fueled Buses. Air and Waste Management Association Paper 89-9.4.

Horn, J.C. and S. K. Hoekman (1989). Methanol-Fueled Light-Duty Vehicle Exhaust Emissions. Air 

and Waste Management Association Paper No. 89-9.3.

Hoekman, S.K. and M.C. Ingham (1987). Measurement of PAH and Nitro-PAH from a Heavy-Duty 

Diesel Engine. Air Pollution Control Association Paper No. 87-1.4.

Wall, J.C. and S.K. Hoekman (1984). Fuel Composition Effects on Heavy-Duty Diesel Particulate 

Emissions. Soc. Auto. Eng. Tech. Paper No. 841364.

Seizinger, D.E. and S.K. Hoekman (1984). Aromatic Measurements of Diesel Fuel - A CRC RoundRobin Study. Soc. Auto. Eng. Tech. Paper No. 841363.

Barton, T.J., Hoekman, S.K., Burns, S.A. (1982). Comments on the Formation of Silanones in the 

Thermolysis of Hydridosilyl Peroxides. Organometallics 1, 721-725.

Barton, T.J., Hoekman, S.K. (1980). Bis(trimethylsilyl)diazomethane, Trimethylsily trimethylgermyl 

diazomethane, and Bis(trimethylgermyl)diazomethane – Synthesis and Chemistry of 

Quantitative Silene and Germene Precursors. J. Amer. Chem. Soc. 102, 1584-1591.

Barton, T.J., Hoekman, S.K. (1979). Convenient Synthesis of Trimethylsilyldiazomethane – Silene 

Generator. Synthesis and Reactivity in Inorganic and Metal-Organic Chemistry 9, 297-300.
Page 28

Final Technical Reports

Broch, A., and S.K. Hoekman. (2012) “Transportation Fuel Life Cycle Analysis: A Review of 

Indirect Land Use Change and Agricultural N2O Emissions.” CRC Final Report No. E-88-2, 

January 2012. (156 pp)

Hoekman, S.K., C. Robbins, and X. Wang. (2011) “Dilution Sampling System for Biomass-Derived 

Syngas.” Final report to DOE under DE-FG30-08CC00057, January 30, 2011. (51 pp)

Hoekman, S.K., A. Broch, C. Robbins, and E. Ceniceros. (2011) “Investigation of Biodiesel 

Chemistry, Carbon Footprint and Regional Fuel Quality.” CRC Final Report No. AVFL-17a, 

January, 2011. (256 pp)

Hoekman, S.K. (2011) “Thermal Treatment of Biomass.” Final Report for Subtask 1.2 of the Nevada 

Renewable Energy Consortium, January, 2011. (30 pp)

Hoekman, S.K. (2010) “Renewable Energy Center – Desert Research Institute: Phase II.” Submitted 

to NREL, October 13, 2010. (109 pp)

Hoekman, S.K. (2010) DRI-REC-II Task 3A. “Enhance Biofuels/Biomass Testing Capabilities: 

Sampling and Analysis of Thermal Conversion Products.” Submitted to NREL, March 31, 

2010. (17 pp)

Hoekman, S.K., C. Robbins, and X. Wang. (2010) “Gridley Biofuels Project Final Report.” 

Submitted to REII, July 21, 2010. (63 pp)

Robbins, C., X. Wang, and S.K. Hoekman. (2010) “Recovery Act: Solar Reforming of Carbon 

Dioxide to Produce Diesel Fuel.” Submitted to REII, October 30, 2010. (24 pp)

Hoekman, S.K., A.W. Gertler, A. Broch, and C. Robbins, (2009) “Investigation of Biodistillates as 

Potential Blendstocks for Transportation Fuels,” CRC Project No. AVFL-17, Report to 

Coordinating Research Council, June 2009 (289 pages)

Hoekman, S.K. (2008) “DRI-REC Phase I Final Report,” submitted to NREL, under Subcontract No. 

ADO-5-44431-06, July 21, 2008. (18 pages)

Hoekman, S.K., A. Broch, C. Robbins, and R. Purcell. (2008) “Renewable Energy Production via 

Carbon Capture and Recycling,” submitted to RCO2, Nov. 3, 2008. (29 pages)

Reinhardt, R., and S.K. Hoekman. (2007) “Hawaiian Island Gasoline Volatility Study: Analysis of 

90th Percentile Maximum Temperature,” Submitted to Chevron Products Company, August 

3, 2007. (34 pages).

Gertler, A. and S.K. Hoekman (2007) “Review of NAC 590.065: Overview and Implications of 

Gasoline Volatility Rule Change,” Final Report submitted to Nevada Department of 

Agriculture, February 14, 2007. (60 pages)
Page 29

Appendix B: Conflict of Interest Statements

Conflict of Interest and Bias for Peer Review

Background

Identification and management of potential conflict of interest (COI) and bias issues are vital to 

the successes and credibility of any peer review consisting of external experts. The 

questionnaire that follows is consistent with EPA guidance concerning peer reviews.2

.

Definitions

Experts in a particular field will, in many cases, have existing opinions concerning the subject of 

the peer review. These opinions may be considered bias, but are not necessarily conflicts of 

interest.

Bias: For a peer review, means a predisposition towards the subject matter to be discussed that 

could influence the candidate's viewpoint. 

Examples of bias would be situations in which a candidate:

1. Has previously expressed a position on the subject(s) under consideration by the panel; or

2. Is affiliated with an industry, governmental, public interest, or other group which has 

expressed a position concerning the subject(s) under consideration by the panel.

Conflict of Interest: For a peer review, as defined by the National Academy of Sciences,3

includes any of the following:

1. Affiliation with an organization with financial ties directly related to the outcome;

2. Direct personal/financial investments in the sponsoring organization or related to the 

subject; or

3. Direct involvement in the documents submitted to the peer review panel... that could 

impair the individual's objectivity or create an unfair competitive advantage for the 

individual or organization.

 

2

U.S. EPA (2009). Science Policy Council Peer Review Handbook.

 OMB (2004). Final Information Quality Bulletin for Peer Review.

3

NAS (2003). "Policy and Procedures on Committee Composition and Balance and Conflict or Interest for Committees Used in 

the Development of Reports" (www.nationalacademies.org/coi).
Page 30

Policy and Process

● Candidates with COI, as defined above, will not be eligible for membership on those panels 

where their conflicts apply.

● In general, candidates with bias, as defined above, on a particular issue will be eligible for all 

panel memberships; however, extreme biases, such as those likely to impair a candidate's 

ability to contribute to meaningful scientific discourse, will disqualify a candidate.

● Ideally, the composition of each panel will reflect a range of bias for a particular subject, 

striving for balance.

● Candidates who meet scientific qualifications and other eligibility criteria will be asked to 

provide written disclosure through a confidential questionnaire of all potential COI and bias 

issues during the candidate identification and selection process.

● Candidates should be prepared, as necessary, to discuss potential COI and bias issues.

● All bias issues related to selected panelists will be disclosed in writing in the final peer 

review record.
Page 31

Conflict of Interest and Bias Questionnaire

Fuel Sulfur Effects Report Peer Review

Instructions to Candidate Reviewers

1. Please check YES/NO/DON'T KNOW in response to each question.

2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the 

circumstances.

3. Please make a reasonable effort to answer accurately each question. For example, to the 

extent a question applies to individuals (or entities) other than you (e.g., spouse, 

dependents, or their employers), you should make a reasonable inquiry, such as emailing 

the questions to such individuals/entities in an effort to obtain information necessary to 

accurately answer the questions.

Questions

1. Are you (or your spouse/partner or dependents) or your current employer, an author, 

contributor, or an earlier reviewer of the document(s) being reviewed by this panel?

YES___ NO_X_ DON'T KNOW___

2. Do you (or you spouse/partner or dependents) or your current employer have current 

plans to conduct or seek work related to the subject of this peer review following the 

completion of this peer review panel?

YES___ NO___ DON'T KNOW_X_

3. Do you (or your spouse/partner or dependents) or your current employer have any known 

financial stake in the outcome of the review (e.g., investment interest in a business related 

to the subject of peer review)?

YES___ NO_X_ DON'T KNOW___

4. Have you (or your spouse/partner or dependents) or your current employer commented, 

reviewed, testified, published, made public statements, or taken positions regarding the 

subject of this peer review?

YES___ NO_X_ DON'T KNOW___
Page 32

5. Do you hold personal values or beliefs that would preclude you from conducting an 

objective, scientific evaluation of the subject of the review?

YES___ NO_X_ DON'T KNOW___

6. Do you know of any reason that you might be unable to provide impartial advice or 

comments on the subject review of the panel?

YES___ NO_X_ DON'T KNOW___

7. Are you aware of any other factors that may create potential conflict of interest or bias 

issues for you as a member of the panel?

YES___ NO_X_ DON'T KNOW___

Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that 

no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I 

further declare that I have made reasonable effort and inquiry to obtain the information needed to 

answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change 

in circumstances that would require me to revise the answers that I have provided.

Tom Durbin______________

Name

11/14/2011____

Signature Date
Page 33

Conflict of Interest and Bias Questionnaire

Fuel Sulfur Effects Report Peer Review

Instructions to Candidate Reviewers

1. Please check YES/NO/DON'T KNOW in response to each question.

2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the 

circumstances.

3. Please make a reasonable effort to answer accurately each question. For example, to the 

extent a question applies to individuals (or entities) other than you (e.g., spouse, 

dependents, or their employers), you should make a reasonable inquiry, such as emailing 

the questions to such individuals/entities in an effort to obtain information necessary to 

accurately answer the questions.

Questions

1. Are you (or your spouse/partner or dependents) or your current employer, an author, 

contributor, or an earlier reviewer of the document(s) being reviewed by this panel?

YES___ NO__+_ DON'T KNOW___

2. Do you (or you spouse/partner or dependents) or your current employer have current 

plans to conduct or seek work related to the subject of this peer review following the 

completion of this peer review panel?

YES___ NO__+_ DON'T KNOW___

3. Do you (or your spouse/partner or dependents) or your current employer have any known 

financial stake in the outcome of the review (e.g., investment interest in a business related 

to the subject of peer review)?

YES___ NO__+_ DON'T KNOW___

4. Have you (or your spouse/partner or dependents) or your current employer commented, 

reviewed, testified, published, made public statements, or taken positions regarding the 

subject of this peer review?

YES___ NO__+_ DON'T KNOW___
Page 34

5. Do you hold personal values or beliefs that would preclude you from conducting an 

objective, scientific evaluation of the subject of the review?

YES___ NO__+_ DON'T KNOW___

6. Do you know of any reason that you might be unable to provide impartial advice or 

comments on the subject review of the panel?

YES___ NO__+_ DON'T KNOW___

7. Are you aware of any other factors that may create potential conflict of interest or bias 

issues for you as a member of the panel?

YES___ NO__+_ DON'T KNOW___

Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that 

no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I 

further declare that I have made reasonable effort and inquiry to obtain the information needed to 

answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change 

in circumstances that would require me to revise the answers that I have provided.

Ronald M. Heck____________

Name

__________________________ 1/21/2012______

Signature Date
Page 35

Conflict of Interest and Bias Questionnaire

Fuel Sulfur Effects Report Peer Review

Instructions to Candidate Reviewers

1. Please check YES/NO/DON'T KNOW in response to each question.

2. If your answer is YES or DON'T KNOW, please provide a brief explanation of the 

circumstances.

3. Please make a reasonable effort to answer accurately each question. For example, to the 

extent a question applies to individuals (or entities) other than you (e.g., spouse, 

dependents, or their employers), you should make a reasonable inquiry, such as emailing 

the questions to such individuals/entities in an effort to obtain information necessary to 

accurately answer the questions.

Questions

1. Are you (or your spouse/partner or dependents) or your current employer, an author, 

contributor, or an earlier reviewer of the document(s) being reviewed by this panel?

YES___ NO_X__ DON'T KNOW___

2. Do you (or you spouse/partner or dependents) or your current employer have current 

plans to conduct or seek work related to the subject of this peer review following the 

completion of this peer review panel?

YES___ NO_X__ DON'T KNOW___

3. Do you (or your spouse/partner or dependents) or your current employer have any known 

financial stake in the outcome of the review (e.g., investment interest in a business related 

to the subject of peer review)?

YES___ NO_X__ DON'T KNOW___

4. Have you (or your spouse/partner or dependents) or your current employer commented, 

reviewed, testified, published, made public statements, or taken positions regarding the 

subject of this peer review?

YES___ NO_X__ DON'T KNOW___
Page 36

5. Do you hold personal values or beliefs that would preclude you from conducting an 

objective, scientific evaluation of the subject of the review?

YES___ NO_X__ DON'T KNOW___

6. Do you know of any reason that you might be unable to provide impartial advice or 

comments on the subject review of the panel?

YES___ NO_X__ DON'T KNOW___

7. Are you aware of any other factors that may create potential conflict of interest or bias 

issues for you as a member of the panel?

YES___ NO_X__ DON'T KNOW___

Acknowledgment

I declare that the disclosed information is true and accurate to the best of my knowledge, and that 

no real, potential, or apparent conflict of interest or bias is known to me except as disclosed. I

further declare that I have made reasonable effort and inquiry to obtain the information needed to 

answer the questions truthfully, and accurately. I agree to inform SRA promptly of any change 

in circumstances that would require me to revise the answers that I have provided.

S. Kent Hoekman_____________

Name

______ December 14, 2011____

Signature Date
Page 37

Appendix C: Peer Review Charge

Sulfur in gasoline has long been known to reduce the efficiency of automobile exhaust emission 

aftertreatment systems. Some emission studies have suggested an increase in catalyst sensitivity 

to sulfur (i.e., binding to active catalytic sites) with increasing stringency of vehicle exhaust 

emission standards. This may be a result of higher catalytic efficiencies required for compliance 

with recent emission standards. Though, historically, light-duty vehicle emission standards have 

been high enough to mask any impact of fuel sulfur level as negligible. EPA, in promulgating its 

Tier 2 light-duty vehicle emission standards, recognized the importance of fuel sulfur level. 

Reductions to new vehicle exhaust emission standards under the Tier 2 vehicle and fuels 

program were accompanied with corresponding reductions in fuel sulfur level to improve the 

cost and feasibility of the vehicle technology changes required for compliance.

In the 2005/2006 timeframe, EPA’s Mobile Source Air Toxics/MSAT-2 study4examined the 

effects of fuel sulfur level on exhaust emissions from a test fleet of nine Tier 2-compliant cars 

and trucks. The study examined four gasolines, without ethanol and blended in a step-wise 

manner from a base fuel at 6ppm S to the final fuel at 32 ppm S. FTP-weighted emission 

reductions related to fuel sulfur changes in this program for exhaust emissions were all 

statistically significant (α = 0.9). However, due to specific catalyst prep procedures that 

compared a fully cleaned-out catalyst with low sulfur fuel to a fully-loaded catalyst with high 

sulfur fuel, these results were thought to represent a “worst case” only. The data, though, 

suggested reversible sulfur loading on Tier 2 vehicle catalysts and likely significant emission 

reductions through further fuel sulfur level control. 

The test program described in this report used two fuels with properties identical to conventional 

federal certification gasoline, except that the “lower sulfur” fuel contained a nominal 5 ppm 

sulfur. The “higher sulfur” fuel had a level similar to the national average in-use fuel at 

approximately 30 ppm sulfur. The types of data targeted by the program were the level of 

“reversible” loading in catalysts found in-use Tier 2 vehicles and the relative emission 

differences due to sulfur re-loading for two different sulfur level fuels in the same vehicle. In 

order to generate these emission data, privately-owned in-use vehicles were recruited for the 

study. It was not feasible however to damage or destroy catalysts to directly measure any sulfur 

loading and, so, the behavior of emissions relative to a baseline was used as a proxy for catalyst 

sulfur loading. 

The report to be reviewed contains information on the hypothesis, design, and execution of the 

test program as well as an in-depth statistical analysis of the results. EPA is seeking the 

reviewers’ expert opinion on all aspects of the report and, in particular, the methodologies used 

to evaluate the effects of both “cleanout” and sulfur level on in-use Tier 2 vehicle emissions and 

whether these techniques are likely to yield accurate results. EPA asks the reviewer to also 

consider the appropriateness of the statistical techniques used to analyze the data described in 

 

4

““Proposed Rule: Control of Hazardous Air Pollutants from Mobile Sources” Preamble and Regulations (published March 29, 

2006). See http://www.epa.gov/oms/toxics.htm.
Page 38

this report, their appropriateness in the context of data accuracy and quality issues and the overall 

conclusions drawn in this report pertaining to reducing fuel sulfur levels in the light-duty fleet. 

To that end, reviewers should address the following questions, specifically:

1. Was the imputation method used for replacement of measurements with low 

concentration reasonable? What other alternatives may have been better in this case? 

(Section 7.1)

2. Please comment on the use of mixed model in analyzing the data. Was the model fitting 

strategy in selecting the final model statistically sound? (Section 7.2)

3. Were the model assumptions for the covariance structure reasonable given the data? 

(Section 7.3)

4. Please comment on the methodology used in determining the in-use sulfur effect for 

models with and without the sulfur and mileage interaction term. (7.3.3)

5. Is the interpretation of the mileage-by-sulfur interaction term presented correctly? 

(Section 7.3.3)

6. Are the sensitivity analyses on effects of low-level concentrations, imputed values, and 

influential vehicles sufficient as performed? Do the results from the sensitivity analyses 

provide additional support for the robustness of the conclusion? (Section 7.3.4)
Page 39

Appendix D: Reviews

Comments on Fuel Sulfur Effects

January 25, 2012

RE: Peer review comments on in-use sulfur effects study

Dear Mr. Menard:

Please find attached my comments on the document “The Effects of Fuel Sulfur Level on 

Emissions from the In-Use Tier 2 Vehicles”. I reviewed the document itself, as well as the 

underlying statistical concepts used in the analysis. I work as a research engineer for the 

University of California at Riverside, CE-CERT and have no real or perceived conflict of interest 

related to this evaluation. I have considerable expertise in emissions testing and have conducted 

several major emissions test programs related to gasoline fuel properties, and the analysis of such 

data. Please let me know if you would like further information or have further questions relating 

to my comments, or would like to discuss the comments via a conference call.

Regards,

Tom Durbin, Ph.D

Research Engineer

University of California

CE-CERT

Riverside, CA 92521
Page 40

Comments on

US EPA Report “Effects of Fuel Sulfur Level on Emissions from the In-Use Tier 2 Vehicles”

This report presents an interesting approach to estimating the real effects of fuel sulfur in the Tier 

2 in-use fleet. The study was very extensive in terms of the number of vehicles and level of 

testing that was done, and the results of the study some potentially important impacts of differing 

in-use sulfur levels. It would be interesting to evaluate these results further to determine the 

modeled impact of in-use fuel sulfur levels on ambient quality. The analysis methodologies in 

the report appear to be statistically sound, and appropriate for repeated measures types of testing. 

The presentation of the data is relatively extensive in terms of the descriptions of the statistical 

analysis procedures used. As such, the report would relatively straightforward to read for the 

some with a moderately strong background in statistics, but might a little too technical for some

readers without a strong statistical background. Detailed comments related to this report are 

provided below in three different areas 1) general technical content 2) editorial comments, and 3) 

answers to assigned peer review questions. 

1. Executive Summary. No significant comments outside of editorial comments provided 

below.

2. Introduction – See editorial comments below. Also, it might be useful to discuss the 

Umicore study cited in the Executive summary, since usually the executive summary is 

more condensed than the Background. 

3. Study Design

Was the fuel commercially available in the Ann Arbor area at the time of the study 

oxygenated or not. This is of relevance since the test fuel is non-oxygenated.

30 ppm is given as the in-use sulfur average. Is this based on actual data of in-use sulfur 

samples, or is this based on regulatory average requirements.

In discussing why the US06 x2 clean-out cycle was used, it might be worth discussing the 

clean-out procedures that were used in the MSAT and Umicore studies..

4. Test Vehicle Selection, Recruitment, and Delivery

What is the average sulfur level for the as received vehicles.

Section 4.1 should specifically mention that 19 makes/models were utilized in the study.

Was any attempt made to characterize the vehicle categories by other factors such as level of 

long term fuel trim, or whether the vehicle used a closed coupled or double catalyst or only 

an underbody, or catalyst metals.

What incentive was provided to owners to participate in the program? – add to section 4.2.
Page 41

More information should be given about the number of vehicles targeted for testing, and the 

number of vehicles in each vehicle make/model category. The first mention should be here, 

as opposed to in section 6.2. In that section, it talks about 5 vehicles for each make/model 

class. It would be useful to say the total number of vehicles tested at this point.

5. Test Fuel Specs and Procurement

Some more details should be included about the fuel selection would be useful. How does 

this fuel compare to typical in-use fuel, in terms of say aromatics and RVP. Were 

oxygenates in use in the recruitment area? Or compared to averages of in-use used 

throughout the US?

6. Test Procedures

It would be useful to add in fuel change points into the test procedure flow charts.

There is some lack of clarity in reading through this section that can be shown by looking at 

the last paragraph. The last paragraph indicates that only 4 of 19 family used the modified 

short procedure, whereas the first sentence of the paragraph indicates that the change was 

made approximately halfway through the program. Then looking at Table 7.7, it indicates 

that between 2-4 vehicles in each make/model class were given the modified short 

procedure, whereas it seems to imply in the paragraph that once the change was made, it 

was applied thereafter. Was the change just made for a subset of the vehicles tested in each 

vehicle category. Also, in the last sentence it indicates that the change in the number of 

vehicles providing “sulfur level data” is seen in Table 7-7. Does this mean that the modified 

short vehicles were added to the “sulfur level data” as well as the Long or modified Long 

procedure vehicles. 

Related to the above comment, why not simply just say at the end of each procedure 

description how many vehicles were tested on each procedure.

Figure 6-3 indicates that only the 28 ppm sulfur level data is used for the “clean out effect”, 

but the very first sentence under section 7 indicates that there is a set of “clean-out” data at 5 

ppm. 

7. Data Analysis and Results

The comments for this section are provided in a combination of some comments here, which 

are more related to the presentation, flow, or missing information in the section. At the end 

of this document, addition information is provided to address the questions specifically 

provided under the reviewer, 

p. 21 – 3

rd paragraph – A statement should be added to answer the following question –

Why do the results of the Bag 2 NOx emissions have more substantial implications than 

those of other pollutants/bags?
Page 42

While it is useful to discuss the results at low concentrations, further information on the 

experimental methods might merit a section. For example, what analyzer are used? Can the 

detection limits of the analyzers be included in an Appendix? What methods/microbalance 

was used for the PM measurements, etc.

p. 24 – top of page. There should be a brief paragraph discussing Table 7-1 and the number 

of zero value measurements before discussing the imput value methodologies. 

P. 24 – last sentence – It says that the statistical analyses were run both with and without 

imputed values. In terms of the “without” input value case, does that mean that the values 

were eliminated, were zeros used as the imput values, or were the original values used, 

whether they are positive or negative.

p. 25 – second paragraph – A new terms vehicle “families” is introduced. Presumably this 

term is the same as the vehicle make/models used earlier. Presumably these also have the 

same engine family and this should be included along with engine size, vehicle 

configuration, and weight.

p. 30 – There is no discussion on the symbols in the plots presumably the plots show the 

average and median inside the bar. Then the bars represent the 95% confidence levels and 

the error bars the full data range. Then there are some other dots on the plot? 

p. 30 – It is not clear what is meant by “Some vehicle families show the presence of withinfamily variability” Presumably this could also be due to differences in in-use sulfur levels as 

well, since a handful of vehicles came in with in tank fuels with sulfur levels much higher 

than those for others. For the post-cleanout variability, it seems like this should be 

independent of the in-use sulfur loading. Could some of the variability be related to the 

condition of the vehicle? Vehicle family M504 seems to show a very variability postcleanout.

P. 32 – Related to BIC criterion. It indicates that the BIC performs relatively better for small 

sample size settings. How would one define 5 here? Is the current experiment a small 

sample size? 

Figure 7-4 does not appear to add significant value. Much of the same information is 

available in the box plots in Figure 7-5, and that is a little easier to read. It talks about the 

vehicles with similar emissions profiles being grouped. However, it is not immediately 

obvious what grouping were used for the Figures 7-4 and 7-5.

8. Summary and Conclusions

Under the first bullet point. Performing a clean-out cycle with a 28 ppm fuel

The second bullet point could be clearer. For example: for a subset up vehicles tested on 

both 5 ppm and 28 ppm fuels, it was found that additional FTP composite reductions of 18% 
Page 43

for NOx, 9% for NMHC, and 8% for CO were for the 5 ppm fuel in comparison with the 28 

ppm fuel.

9. Grammar and other editorial things.

--- Executive summary – the number of spaces between sentences differs from to in some 

cases 3 or even 4. This should be made consistent.

--- p. 2 – The end of the first sentence and start of the second redundantly talk about catalyst 

efficiency.

--- p. 2 – The final sentence of the first paragraph should be split into two sentences.

--- p. 2 – Second to last sentence -- rewrite... for a PZEV operated on a 3 ppm fuel compared 

to a 33 ppm fuel. 

--- p. 3 – third sentence – In response....To address this question, the EPA (spell out EPA 

first use)

--- p. 3 – last sentence – split it – performance. For Example, 

--- p. 4 – 2

nd line – fuels (add s); Also split the second to last sentence.

--- p. 5 – need space for 5 ppm and 28 ppm, and period after “overall”. This sentence should 

probably also be split.

--- p. 5 – 3

rd sentence – rewrite – a significant miles-by-sulfur interaction was not found from 

the model fitting....

--- p. 5 – last sentence – rewrite – In this case, the relatively differences with sulfur level 

varied as a function of mileage, so determination.....

--- p. 7 – 1

st sentence – “has long been shown” is ackward. Also in executive summary.

--- p. 8, 9, and 10 – comma after e.g., 

--- p. 8 – comma after idles), comma after in 2005, 

--- p. 9 – 1

st line – spell out MSAT acronym; line 7 – word “specimens” is a bit ackward; 6th

line form bottom “benefits of further sulfur control” 

--- p. 11 – about line 10 comma after high temperatures), line 15 “conditions that are

favorable” line 16 lines further down “A vehicle with a relatively..., reference 10 on the 

E-60 report is not referenced properly and should contain more author names.

--- p. 12 – 1

st line operation, (add comma); 1st line of section 4.1 “with the intent of being to 

be representative of the latest...”; 

--- p. 12 – last sentence table 4-1 should not be underlined. This same changed should be 

incorporated throughout the report, as it appears there is something in the formatting 

underlining table and figure references in the text. 

--- p. 13 – First line – “Vehicles recruited for testing were targeted to have a mileage between 

12,000 and 40,000 miles and an age of less than three years old.”

--- p. 14 – line 5 up front bottom – exhaust leaks, (add comma)

--- p. 16 – First full paragraph – Following the fuel change,

--- p. 16 – last sentence – rewrite to “The Long and Short procedures are shown in Figure 6-1 

and are discussed in greater detail below. The Long and Short procedures are identical in 

structure for the first six emissions tests. 
Page 44

--- p. 17 – third line – start a new paragraph about the Long procedure.

--- p. 20 – last sentence – Split into two sentences.

--- p. 21 – 3

rd paragraph – detail. but tThe

--- p. 27 – 2

nd paragraph line 4 “vehicles which was were “...., later in same paragraph 

Besides this, (add comma), 3rd paragraph line 2 – effects, (add comma)

--- p. 28 – line right after equation - respectively, (add comma) 3rd paragraph – line 2 –

model, (add comma) – line 4 structure, (add comma)

--- p. 29 – 1

st paragraph line 3 exchange “were” for was - line 4 – levels, (add comma) -

--- p. 32 – last paragraph line 3 – covariance, (add comma)

--- p. 33 line 6 – selected, (add comma)

--- p. 35 – last paragraph 1st line – “dataset included 17” ......

--- p. 38 – 2

nd line from bottom – Section 7.3.3, (add comma) 

--- p. 40 – 3rd line from bottom – i.e., (add comma) 

--- p. 40 – 5th line from bottom – rewrite - considering each vehicle as a random effects

might be useful

--- p. 42 – The sentence that includes Figure 7-6 should be broken up into two sentences. 

Example: This is shown in figure 7-6, which shows the log-transformed emissions from 

individual vehicles by sulfur level.

5

th sentence from bottom – add comma ... some vehicles, and suggests.... 

--- p. 44 – 2

nd line from bottom – add comma ... vehicle to vehicle, and simple descriptive 

statistics...

--- p. 45 – 1

st line – rewrite .. a similar top-down model fitting statistical approach was 

applied to the clean-out data, as described in Section 7.3.1. 

--- p. 45 – 2

nd sentence – rewrite .. Furthermore, additional analysis was done that used a 

subset of the sulfur level dataset that isolates the emission measurements immediately 

following the clean-out to address the effectiveness of the clean-out cycle in reducing 

emissions.

--- p. 46 – 2

nd paragraph 4th sentence – add comma ... irregularly, where

--- p. 46 – 3

rd paragraph 2nd sentence – ....exponentially with time, i.e., the variability.... 

---- p. 50 – 2

nd to last line – operation, (add comma) causing an increase in emissions 

--- p. 58 – 1

st paragraph – line 3 through the presence, line 4 (RLD), (add comma), line 5 

after an iterative... 

--- p. 58 – 1

st paragraph – line 3 through the presence, line 4 (RLD), (add comma), line 5 

after an iterative... 

--- p. 59 – 2

nd paragraph under “summary and conclusions” - ...US06 cycles. This data was 

used to examine the existence... 
Page 45

Peer Review Charge Questions

1. Was the imputation method used for replacement of measurements with low 

concentration reasonable? What other alternatives may have been better in this 

case? (Section 7.1)

The text talks about tailpipe emissions being greater than zero, while the actual question 

of relevance may be – Are the tailpipe emissions greater than the ambient background 

concentrations? (presuming a relatively clean background) If the vehicle’s emissions is 

below background levels, than it would not be making a additive contribution to pollution 

levels. In this case, you could get a distribution of both positive and negative values 

fluctuating around zero.

My concern with the imput value procedure is that it might introduce a slight positive 

bias. My preference would probably be to use zeros as the imput values, since this would 

represent a distribution of positive and negative values that would average to zero over a 

larger dataset. For the cases where the dilute exhaust concentrations are lower than the 

measured background. By eliminating, or essentially not allowing, negative or zero 

values, doesn’t this add a positive bias to the results. Or how do you ensure there is a not 

a positive bias.

p. 24 – first paragraph – its states that a data point can either be deleted or replaced with 

an imput value. Couldn’t the value just simply be left as is? Also, if an input value is 

deleted for a particular bag, presumably no composite emissions would be available. 

Would the corresponding test be eliminated in its entirety? 

p. 25 last line – its unclear how an outlier could be an input value. Is this a case where the 

input value is a very low value? A description of what these values might be should be 

added to the text. Again, if the imput outlier value is eliminated, how does that impact the 

result of the test?

Can Table 7-2 be modified to include not only the number of outliers identified, but also 

the number that were eliminated? This should also be in the text in the paragraph for 

section 7.1.2.

2. Please comment on the use of mixed model in analyzing the data. Was the model 

fitting strategy in selecting the final model statistically sound? (Section 7.2)

The use of a mixed model is a relatively standard method to treat data analysis of 

emissions, and appears to be appropriate for the study design given here. The fitting 

strategy of using a saturated model, then developing the most optimal covariance 

structure, and then fitting the final model appears to be statistically sound. The step-wise 

backward elimination approach for the “Sulfur Effects data” also appears to be 

statistically appropriate.
Page 46

One thought relating to the discussion of the univariate ANOVA and the multivariate 

MANOVA in section 7.2. Overall, it does not seem to add significant value to the report. 

Looking at the audience for this report, it would likely be composed of a mix of readers 

that may or may not have a statistical background. For those with a statistical 

background, the additional information on the univariate ANOVA and the multivariate 

MANOVA would not provide significant value. For those that are not as familiar with 

statistical methods, the added information would likely be more confusing than 

clarifying. It is suggested that this information, although interesting, might be better 

placed in an Appendix.

3. Were the model assumptions for the covariance structure reasonable given the 

data? (Section 7.3)

For the “Clean-out data”, an unstructured covariance structure was initially used (which 

did not converge), and then a compound symmetry structure was selected. The compound 

symmetry structure is appropriate since the measurements from the same vehicle should 

have a homogeneous variance, as included in the text. For the “Effect of Sulfur data, 

where multiple measurements are made at different mileage accumulations, the 

autoregression covariance structure is appropriate since the since the correlations between

measurements is expected to decline as the measurements are further apart in terms of 

mileage accumulation, as mentioned in the text of the report.

4. Please comment on the methodology used in determining the in-use sulfur effect 

for models with and without the sulfur and mileage interaction term. (7.3.3)

Overall, the methodology for determining the in-use sulfur effect for models with and 

without the sulfur and mileage interaction term appears to be sound. For cases where the 

sulfur level and mileage interaction term was not significant, the sulfur effect, sulfur 

loading and associated percentage differences between the high and low sulfur fuels 

should be constant as a function of increasing mileage. Thus, using the differences in the 

least squares means from the final model and reverse transforming them could be used to 

quantify the percentage differences between high and low sulfur for the case where the 

sulfur level and mileage interaction term was not significant. 

For cases where the sulfur level and mileage interaction term is significant, the rate of 

sulfur loading between the low and high sulfur fuels would differ as a function of 

mileage, so a different approach would be needed. The methodology of using the in-use 

emission level upon arrival (pre-cleanout) from a larger clean-out dataset and projecting 

it out to the mileage where the two lines intersect (in-use equivalent loading) seems 

reasonable for estimating the in-use sulfur level effect. Some clarification should be 

given on what is meant by the “larger clean-out dataset” since it is not immediately 

obvious what data these actually are. Additionally, how do the incoming sulfur levels for 

the “larger clean-out dataset” compare with average in-use values for the US fleet, or the 

fleet in different regions of the country. 
Page 47

5. Is the interpretation of the mileage-by-sulfur interaction term presented 

correctly? (Section 7.3.3)

As discussed under question #4, the interpretation of the mileage-by-sulfur interaction 

term does appear to be presented correctly. On page 48, when the interpretation of the 

interaction term is first being discussed, I would also add in something about the 

percentage differences in emissions being constant as a function of mileage, just to make 

things clearer. For example, for the last line of the paragraph just after Table 7-9:

In other words, the effect of high fuel sulfur on Bag 2 NOx exists immediately after 

clean-out and remains constant on a percentage basis during subsequent driving of a 

vehicle.

6. Are the sensitivity analyses on effects of low-level concentrations, imputed 

values, and influential vehicles sufficient as performed? Do the results from the 

sensitivity analyses provide additional support for the robustness of the 

conclusion? (Section 7.3.4)

Overall, the sensitivity analyses see to add additional support to the robustness of the 

conclusions. Here are a few thoughts on the sensitivity analyses.

If the sensitivity analyses were only done for the NOx bag 2 emissions, it would be worth 

mentioning that the NOx bag 2 emissions showed a higher percentage of measurements 

with zero values than most other pollutant/bag combinations, as illustrated in Table 7-1.

It would be interesting to get some feel for the vehicles/data eliminated in the sensitivity 

analysis for the low concentration measurements. Since these would be low emitting 

vehicles, it seems like they might have more robust and sulfur tolerant catalyst systems. 

This is consistent with the results showing an increase in the emissions reductions as 

these data are pulled out. 

Relating to the imput values, it would be interesting to see what the effect would be of 

making the imput values simply zero. Given the small differences between the model 

with and without the imput model, there would likely be only a minor impact, but this 

would add to the robustness of the results, although probably not change any conclusions.

For the influential vehicles, it is interesting to look at the vehicles selected. Vehicles 0007 

and 0178 show a relatively large sulfur effect, whereas vehicle 0046 shows a slight 

reverse sulfur effect with tight variability. It is interesting that other vehicles with 

relatively strong sulfur effects, such as 0165, 0179, and 0011, did not have a strong 

influence. The result of this sensitivity analysis is reasonably intuitive, with the NOx 

differences between sulfur levels shrinking, which would be consistent with removing 

some vehicles with a stronger sulfur effect.
Page 48

Report – Review of The Effects of Fuel Sulfur Level on Emissions 

from the In-Use Tier 2 Vehicles - DRAFT REPORT

Ronald Heck 2/3/2012

General discussion:

It is very important that the authors discuss right up front that they have taken an 

approach where the vehicles are considered as “black boxes”. This is a totally different approach 

than that adopted by the Umicore paper (Ball D., Clark D., Moser D. (2011). Effects of Fuel 

Sulfur on FTP NOx Emissions from a PZEV 4 Cylinder Application. SAE 2011 World Congress 

Paper 2011-01-0300. SAE International: Warrendale, PA.) In the “black box “ approach there is 

no consideration of the engine control strategies (rich or lean bias, cold start strategy, etc.), 

sensor responses (from aging), vehicle operation (e.g. fuel cut), catalyst technologies, catalyst 

configurations(close coupled, underfloor, cell density, volume, etc.) See James M. Lyons, 

David Lax, and Steve Welstand, Investigation of Sulfur Sensitivity and Reversibility in LateModel Vehicles, SAE 1999-01-3676 for work that takes a more phenomenological approach to 

sulfur effects. By the way I could not find this reference in the report. In the “black box” 

approach very little information is given besides the vehicle type, mileage, age, etc. Because of 

this lack of information it will be difficult for anyone to repeat the authors’ experiments and 

challenge there results. In addition it will be difficult for the authors to explain data anomalies 

because there are no specifics to tease out a possible explanation. Also since there are so many 

uncontrolled variables, the statistical analysis may be limited since it is only statistics! This is 

not meant to say the work has no merit only to provide caution in the final conclusions and 

merits of too much statistics!

One concern I have is if some of the vehicles have a fuel cut strategies. If so, this could 

confound the results since in the US06 the catalyst with a fuel cut strategy will see lean 

excursions while a catalyst without will be stoichiometric during the cycle. Again since no 

information is given in the report on engine operation it is impossible to go back and uncouple 

this effect.

Sulfur poisoning is a complex phenomenon for catalysts and the catalyst manufacturers 

have found recipes to compensate for high and low sulfur levels (see Harold N. Rabinowitz, 

Samuel J. Tauster, Ronald M. Heck, The effects of sulfur & ceria on the activity of automotive 

Pd/Rh catalysts, Applied Catalysis A: General 212 (2001) 215–222). If the catalyst companies 

can design for Tier 2 vehicles at 30 ppm S, then this study may be in question or to put in another 

way, maybe some of the vehicles you studied already have these types of technologies and that is 

why you found some responses that didn’t show an effect of S level!

The removal of sulfur from the catalyst needs a consistent statement throughout the 

paper. You show Figure 2-1 from Ford but the text is not consistent in describing this removal 

which is due to temperature, air to fuel ratio, and sulfur level. Also the degree of removal is 
Page 49

different for Pt, Pd and Rh and the oxygen storage materials (Ce, Zr, La, etc.). So this discussion 

needs a little more technology.

In the report, there are a number of places where symbols are not explained in tables and 

in text. Also, I did not see an explanation about how one does a hot/cold start FTP. Please make 

sure this is explained in the text of the report.

At end of papers, please a list of references. I had to go through the entire paper many 

times to see if a reference was cited. This was painful!

Specific textual comments:

I am including the entire report as a Word document with 27 comments embedded in the 

text.
DRAFT

Page 50

Specific Textual Comments

Ron Heck

Page 

No. Report Text Comment

2 Fuel sulfur content has long been understood to affect the 

performance of emission after treatment catalysts in light duty 

vehicles, where the sulfur and/or its oxides adsorb to the active 

precious metal sites, reducing the catalyst’s efficiency in destroying 

harmful pollutants.

Fuel sulfur content has long been understood to affect the performance of 

emission after treatment catalysts in light duty vehicles, where the sulfur 

and/or its oxides adsorb to the active precious metal sites and oxygen 

storage materials, reducing the catalyst’s efficiency in destroying harmful 

pollutants.

2 The quantity of sulfur present on the catalyst at any given time is a 

function of its temperature and the fuel sulfur level . . .

The quantity of sulfur present on the catalyst at any given time is a function 

of its temperature, air to fuel ratio, and the fuel sulfur level . . .

2 . . . with elevated catalyst temperature and lower fuel sulfur 

concentration both reducing sulfur loading.

. . . with elevated catalyst temperature , rich of stoichiometric operation,

and lower fuel sulfur concentration both reducing sulfur loading.

3 Test fuels were two non-ethanol gasolines with properties typical of 

certification fuel, one at a sulfur level of 5 ppm and the other at 28 

ppm.

Is there any reason to expect different results with ethanol gasolines?

4 [Table ES-1] The clean-out effect is not significant at α = 0.10 when 

no reduction estimate is provided.

Need to explain what p-value is in the text? Also what is in (?????)?

4 This indicates that the catalyst is not fully desulfurized, even after a 

clean out procedure, as long as there is sulfur in the fuel.

Not sure you can make this conclusion. It may be that the sulfur in fuel 

equilibrates instantaneously and it is a concentration effect not a 

desulfurization. To prove this you would have to vary the desulfurization 

time and see where it reaches steady state.

5 [Table ES-2] Again what is p value and what is in (....)?

5 This analysis found highly significant reductions for several 

pollutants, as shown in Error! Reference source not found.; 

reductions for Bag 2 NOx were particularly high, estimated at 59 

percent between 28ppm and 5ppm overall

This analysis found highly significant reductions for several pollutants, as 

shown in Error! Reference source not found.; reductions for Bag 2 NOx 

were particularly high, estimated at 59 percent between 28ppm and 5ppm 

overall.

5 Other results, such as Bag 1 hydrocarbons, did show a significant 

miles-by-sulfur interaction.

Does this mean that the effect never equilibrates? Are you comfortable in 

saying this?
DRAFT

Page 51

Specific Textual Comments

Ron Heck

Page 

No. Report Text Comment

7 The amount of sulfur retained by the catalyst is primarily a function 

of its operating temperature, the active materials and coatings used 

within the catalyst and the concentration of sulfur oxides in the 

incoming exhaust gases.

Again air to fuel ratio is important as you mention in the next sentence for 

reducing conditions.

8 However, the temperatures necessary to release sulfur oxides can also 

lead to thermal degradation of the catalyst over time.

No where do you mention that normal operation of the vehicle, the catalyst 

is constantly being exposed to rich/lean condition from the control system 

using an oxygen sensor. So the catalyst is being regenerated in-situ form 

the perturbation around stoichiometric.

11 The level of reversible in-use sulfur storage and release (or loading) 

within an exhaust catalyst system can be assessed by measuring 

emissions from the vehicle as received, performing a high speed, 

high load clean-out cycle, then measuring emissions again.

In describing the cleanout it is important to mention the temperature range 

and the air to fuel ratio. Also depending on the vehicle calibration the 

degree of richness and time at such could be different from vehicle to 

vehicle. How is this taken into account for the degree of cleanout?

11 A vehicle with relatively high exhaust temperature at the catalyst 

location, and/or significant excess loading of certain platinum group 

metals (PGM) and other active materials in the catalyst may be 

relatively insensitive to sulfur loading regardless of driver behavior.

Up to now you have not mentioned the effect of S on the oxygen storage 

components which will affect NOx and lightoff in cold start. Is there a 

reason this is not mentioned?

12 This loading continues over time with vehicle operation and can be 

observed as an increase in emissions (sometimes referred to in the 

auto industry as “NOx creep”).

Is this due to PGM and/or oxygen storage component deactivation??/

13 Table 4-1 Test Vehicles Recruited If you look at the Umicore study you will see details about the e3mission 

control system (close coupled, underfloor, etc.) You give no such detail in 

this study. Also, they also show a plot of engine lambda during the FTP. 

This is important background information to possibly explain outliers or 

unusual results. This study loses significant meaning without this 

information. You are treating the cars as black boxes and I think this is a 

mistake! Also, the oxygen sensor response is very important as well as it 

age. This is never mentioned in this entire study.

17 [Figure 6-1] Do you explain what a hot/cold start FTP is anywhere in the report??

25 [Table 7-1] What does the value in (...) mean?? Please explain.
DRAFT

Page 52

Specific Textual Comments

Ron Heck

Page 

No. Report Text Comment

35 [Table 7-4] The clean-out effect is not significant at α = 0.10 when 

no reduction estimate is provided.

Explain what is in (....)?

38 Furthermore, the reduction in emissions from cleanout shown in 

Error! Reference source not found. would likely be larger if the 

low sulfur test fuel at 5 ppm had been used for the cleanout procedure 

and the tests immediately following the as-received baseline 

emissions.

This looks like a conjecture. Present evidence?

39 [Table 7-6] Explain what is in (......)?

40 For example, vehicle IDs 0011, 0022, and 0178 clearly show large 

effect of fuel sulfur level on emissions while the effect is only 

marginal for vehicle IDs 0123 and 0264.

Differences can be A/F ratio, close coupled catalyst, oxygen storage 

components, temperature history. This is why more information on vehicle 

characteristics is important to explain unusual results.

46 The BIC value for the first-order autoregressive structure was 764.90. Did you explain BIC?? If not then do so.

47 [Table 7-8] What is Pr? Explain.

55 [Figure 7-10] Is there an explanation as to why these lines diverge and are not parallel?? 

Need some discussion here from the model.

56 [Table 7-11] What is in (....)?

57 [Table 7-13] Is “Probt” probability??

59-

60

Comparing emissions immediately following the clean-out procedure 

on 5 vs. 28 ppm fuel, FTP composite NOx emissions were 18% 

lower, NMHC 9% lower, and CO 8% lower.

To make this conclusion you need to age with 28, clean out with 28 and test 

with 5 and compare age with 5, cleanout with 5 and test with 5. I went 

back in text and really didn’t see this discussed in analysis. If I missed it 

OK otherwise need some discussion in analysis.
DRAFT

Page 53

Review of EPA Draft Report

The Effects of Fuel Sulfur Level on Emissions from the In-Use Tier 2 Vehicles

Report Dated: November, 2011

S. Kent Hoekman

This draft report describes a vehicle test program conducted to assess the emissions impacts of 

reducing gasoline sulfur levels in Tier 2 passenger cars and light-duty trucks. The experimental test 

design was suitable for the intended purposes, although clearly, other testing procedures could have 

been used as well. For example, the standard FTP test procedure was used, at 75 F, to determine 

the emissions results from each vehicle test. Some comments should be offered regarding the 

suitability of the FTP driving cycle and a 75 F test temperature in providing representative/realistic 

operating conditions. For the in-use vehicle fleet, a set of standardized test conditions is clearly 

necessary to obtain repeatable and statistically significant results. However, the FTP test is now over 

35 years old, and 75 F is just one of many temperatures that could be used. The overall quantitative 

effect of fuel sulfur on emissions as determined in this study would be more convincing if it were 

shown that similar results apply to other realistic operating conditions.

The use of two, back-to-back US06 dynamometer cycles was an appropriate choice for catalyst 

regeneration. Although a somewhat more aggressive cycle might provide more complete 

desulfurization, potential damage to the vehicle was a significant concern, as explained in this report.

Also, the approach of catalyst clean-out, followed by repeated testing with a single fuel is a valid way 

to determine a new “equilibrium emissions level” at a specific fuel sulfur level. However, in real-world 

driving, such an equilibrium level is never attained, because operating conditions are constantly 

changing, and fuel sulfur varies from tank-to-tank.

It is not clear why this report places so much emphasis on Bag 2 results – especially Bag 2 NOx. 

Typically, Bag 2 emissions concentrations are extremely low from properly functioning Tier 2 

vehicles. In fact, as shown in this study, it is often difficult to distinguish Bag 2 levels from 

background levels. Large percentage differences between two very small numbers may not be very 

meaningful. To help put these results in perspective, it would be useful to show the relevant 

certification emissions levels for these Tier 2 vehicles. Also, something should be said about the 

impact of fuel sulfur reduction on fleet-wide emissions, and how large a reduction this represents in 

comparison to the entire mobile-source emissions inventory. In reality, this test program only 

captured a small portion of the entire in-use fleet (only Tier 2 vehicles of model years 2007 - 2009).

Some historical perspective about the “FTP bag method” would be helpful here. This sampling and 

analysis method was developed about 40 years ago, to measure emissions from uncontrolled (or 

slightly controlled) vehicles. Prior to introduction of emissions control systems, Bag 2 emission levels 

were routinely 2 orders of magnitude higher than today. Under such conditions, the FTP bag method 

was perfectly suitable for the intended purpose. Now, with Bag 2 emission levels being nearly 

indistinguishable from background, a different method should be employed for quantifying vehicle 

emissions. 

In retrospect, it seems unfortunate that only non-ethanol gasolines were used, as such fuels are 

increasingly uncommon. It is stated that similar results would be expected from ethanol-containing 
DRAFT

Page 54

gasolines, but no data are provided to support this. If any subsequent testing has been done with 

ethanol-containing gasolines, this should be mentioned.

Overall, the statistical modeling approach seems appropriate. As described in Section 7.2 (pages 

26-28) the structure and limitations of the emissions dataset make the linear mixed model a good 

choice for analysis, as this better accommodates missing data, irregularly spaced measurements, 

and within-vehicle effects.

Also, the method used to impute emissions values in cases where measured levels were lower than 

background is probably the best that can be done. The approach of developing an imputation value 

based upon each specific vehicle family seems better than using a single, fleet-wide imputation 

value. The fact that imputed values were applied to only about 20% of the vehicles provides some 

assurance that this imputation method did not excessively distort the results. This was further 

confirmed by performing the statistical analyses with and without inclusion of the imputed values. 

One additional test that could be useful is to conduct the statistical analyses using values of zero 

instead of the imputed emissions values.

A significant question about the statistical modeling approach is the failure to include any information 

about the vehicles’ catalyst formulations, or other catalyst properties. It is not surprising to observe 

that vehicle type (car vs. truck) was not a significant model term, but it is surprising that there is no 

term related to the emissions control systems used across the range of vehicles. Surely not all 

catalysts were the same, and different catalysts would be expected to respond to changes in sulfur 

levels in different ways. This is an area that should be addressed. Inclusion of a catalyst term could 

result in quite different statistical models.

A number of other specific comments, questions, and suggestions regarding this draft report are 

offered below. The ordering of these items is chronological, as they appear in the report.

On page 2 (1st paragraph) the quantity of sulfur present on the vehicle’s catalyst is said to be 

a function of temperature and fuel sulfur level. Isn’t the catalyst formulation/metallurgy 

another important factor? (Catalyst formulation is mentioned on page 7.) What is known 

about the catalyst formulations used in each of the test vehicles of this study?

It is interesting to note that when reducing fuel sulfur from 28 ppm to 5 ppm, significant 

emissions reductions were observed for all pollutants except PM, (See Tables ES-2 and ES3.) Is there a clear mechanistic reason for this lack of effect for PM? Are the catalyst systems 

used in Tier 2 vehicles ineffective in reducing PM (and PM precursors)? This lack of a PM 

effect should be mentioned along with the major findings that are shown in bullet form on 

page 6. 

On page 7 (final paragraph) it is stated that the impact of fuel sulfur on emissions was 

considered negligible under the Tier 0, Tier 1, and NLEV programs. I don’t think this is true. 

The emissions impacts of fuel sulfur have been well known for a long time, but more severe 

sulfur reductions were not thought to be a cost-effective emissions control approach until 

recently. To provide greater context, it would be useful to include a brief summary of gasoline 

regulations (including sulfur levels), and how they have evolved over the past 30-years. 

Along with this, a history of LD vehicle emissions standards should also be presented.
DRAFT

Page 55

On page 8, various detrimental effects of hot/rich catalyst operation are described – including 

catalyst degradation and increased emissions of PM, NMOG, and CO. However, nothing is 

said about fuel economy effects – either here or elsewhere in the report. In general, fuel 

economy is a concern with any type of fuel or vehicle modification. This topic should at least 

be mentioned. In addition, the dataset generated in this study provides an excellent 

opportunity to evaluate the impacts of sulfur reduction on fuel economy, by analyzing CO2 as 

another pollutant, along with CO, THC, NOx, and PM. 

The 2005 MSAT study, which used nine Tier 2 vehicles, is described on pages 8-9. The 

emissions reductions upon lowering fuel sulfur from 32 to 6 ppm are said to be 33% for NOx, 

11% for THC, 17% for CO, and 32% for CH4. Are these values simple averages of the nine 

vehicles, or were they computed by a more sophisticated method? Were similar statistical 

methodologies used to compute the fuel sulfur effects in both the MSAT study and the 

present study? Were PM emissions also measured in the MSAT study?

When first describing the two test fuels on page 10, it would be helpful to refer the reader to 

Table 5-1 (page 15), which provides a more complete listing of fuel properties. Also, did 

these fuels contain anything unusual in the way of additives; e.g. antioxidants, detergents, 

dyes, etc.?

In selecting the vehicles, was the ratio of LD trucks/LD passenger cars representative of the 

in-use fleet? Table 4-1 (page 13) which lists the 19 makes/models that were recruited, shows 

that the No. 1 U.S. sales rank vehicle is missing. Was this an oversight, or are the rankings 

shown in this table incorrect? Notice also that the Toyota Camry is shown as No. 23 in U.S. 

sales rank. This seems unbelievably low for what has traditionally been a “top seller.” Also, it 

would be useful to add a column to Table 4-1 to indicate the number of vehicles in each 

category that were tested in this study, comprising the total test fleet of 81. It might also be

useful to indicate here the number of vehicles in each class that underwent the various test 

procedures: i.e. short procedure, long procedure, modified short procedure, and modified 

long procedure.

The modified long testing procedure shown in Figure 6-2 (page 19) is confusing. The blue 

box indicating the short procedure shows only two post cleanout FTPs, although the wording 

still indicates “triplicate FTPs at 28 ppmS.” Also, in describing the modified short procedure 

on page 20, it is said that “... the change in the number of vehicles providing sulfur level data 

can be seen in Table 7-7 starting with Family ID N513.” It is not clear what this means. Does 

this table list the vehicle families in chronological testing order, so that the first 13 families 

listed were tested using the standard short procedure, and the last 4 families listed were 

tested using the extended short test? It is also confusing that Tables 7-5 and 7-7 provide 

identical information.

How is NMOG different from NMHC? What analytical procedures were used to measure 

these two pollutants?

Displaying the background and sample measurements by FTP Bag (as done in Figure 7-1 

and Appendix B) is very instructive. However, only NOx and THC results are shown in this 

way; it would be useful to also include CO and PM. It would also be helpful to draw ovals in 

Figure 7-1 to capture all background measurements for each vehicle family shown. Although 
DRAFT

Page 56

it is not possible to tell which sample points correspond to which background points, this 

graphical approach clearly illustrates the problem of very low emissions measurements 

relative to background in Bag 2. It also begs the question of why background levels of NOx 

are so variable. Figure 7-1 shows that these background concentrations varied by over an 

order of magnitude for many of the vehicle families tested.

In the determination of outlier data points (described on page 25) what is the rationale for 

choosing an outlier screening criterion value of ± 3.5 for the studentized residuals? Also, 

although Table 7-2 identifies the number of outlier points for each pollutant/bag, it doesn’t 

indicate how many of these outlier points were actually excluded from the statistical 

analyses.

As explained on page 26, log-transformation of emissions measurements has commonly 

been used to analyze vehicle emissions data. However, many previous emissions modeling 

studies have utilized test fleets that included a variety of technology types having a wide 

range of emission levels. In the present study, only Tier 2 vehicles were used, and the 

emission levels did not vary drastically across the test fleet. Given this situation, is logtransformation still necessary (and helpful)?

On page 29 it is stated that the average starting odometer reading of the vehicles used to 

assess the clean-out effect at 28 ppmS was 31,470 miles. Is this value the average of the 17 

vehicle families shown in Table 7-3, or the average of all 81 vehicles? Similarly, is the ± 

value of 1,578 miles the standard deviation of the 17 vehicle families or the 81 individual 

vehicles? (The same questions pertain to the vehicles used to assess the clean-out effect at 

5 ppmS, described on page 35; and the vehicles used to assess the sulfur effects, described 

on page 39.)

The box plots shown in Figure 7-2 (page 31) are quite informative in displaying the relative 

variances between the pre- and post-cleanout vehicle tests. It would be useful to show 

similar plots for other pollutants and Bags. Also, a legend should be included to explain the 

different symbols used in these plots.

When describing the dependent variable (Yi) and effects (Xi and Zi) on pages 31-32, the 

reader should be reminded of the mixed model being used, shown as Equation 7-1 on page 

28. Also, it is stated on page 32 that “The significance of between-family variation was 

observed graphically in Figure 7-2 ...” But this figure only show Bag 2 NOx results. Was 

similar between-family variation observed with other Bags and pollutants?

Figure 7-3 (pages 37-38) is not easy for the reader to process. I realize that the data are 

lumped by emission level, but displaying the data in three separate charts, over two pages, is 

confusing and difficult to understand. It would be preferable to show these data on a single 

page (similar to Figure 7-2, page 31) or even in a single chart (similar to Figure 7-1, using a 

logarithmic scale). It would also be better to order the vehicles along the x-axis in the same 

way they are listed in Table 7-5. Finally, it would be helpful to see more Bags/pollutants 

presented in this way – not just Bag 2 NOx.

On page 38, the enhanced emissions reduction benefits of the 5 ppmS clean-out are 

described as compared to the 28 ppmS clean-out (comparing Tables 7-4 and 7-6). It is 
DRAFT

Page 57

interesting to note that this does not apply to PM emissions. Some explanation should be 

offered as to why PM emissions show such a different behavior. Also, the last sentence on 

page 38 mentions confirmation of results that have not even been presented yet. The 

placement of this sentence seems odd.

The same comments given above regarding data display in Figure 7-3 apply to Figure 7-4 

(pages 41-42). The current grouping of vehicles shown in Figure 7-4 makes it very difficult to 

see the important point being made on page 40 that some vehicles within the same family 

had markedly different emission profiles. This would be much easier to see if a single chart 

were used to display all vehicles – similar in structure to Figure 7-1. Also, the box plots in 

Figure 7-5 (pages 43-44) should all appear on a single page, and should have a legend to 

explain the symbols being used. Further, it would be helpful to identify each vehicle ID 

number shown in Figure 7-5 by make/model; e.g. Vehicle ID 0003 is a Toyota Corolla. Again, 

additional figures should be shown for other Bags and pollutants.

On page 46, the term “BIC” is used without definition. The reader must refer to page 32 to 

see that BIC refers to the Schwarz Bayesian Criterion. It would be helpful to include a table 

of acronyms and abbreviations, as many rather obscure terms are used throughout this 

report.

Figure 7-8 (page 50) is very helpful in convincingly demonstrating the reasonableness of the 

derived statistical model in predicting Bag 2 NOx emissions with both high and low sulfur 

levels. It would be useful to include similar plots for other Bags/pollutants.

At the bottom of page 50, where Figure 7-9 is being discussed, it is stated that “the rate of 

sulfur loading is the same for both high and low sulfur levels.” What is the basis for this 

statement? Sulfur loading is not measured directly. Is NOx emission rate taken as a 

surrogate for sulfur loading? (This statement about sulfur loading is also given in the 

Summary and Conclusions Section on page 59.) Also, Figure 7-9 shows quite clearly that for 

some vehicles, not only are the emission rates higher with the high sulfur fuel, but also the 

increase in emissions with mileage accumulation is higher. For example (see vehicles 0075, 

0123, 0264, 0178, and 0179). This seems inconsistent with the statement that rate of sulfur 

loading is the same for both high and low sulfur levels.

In Table 7-10 (page 53) the fixed effects in the NOx statistical model include only sulfur level 

and miles for all three FTP Bags. Yet the FTP Composite result includes a sulfur level by 

miles interaction term. This seems strange, since the FTP Composite emissions are simply 

calculated as a combination of the individual Bag emissions. Some explanation should be 

offered.

Pages 53-54 discuss the problem of comparing emissions from high and low sulfur levels 

when there is a sulfur level by mileage interaction. The approach taken to address this (as 

illustrated in Figure 7-10) seems rather arbitrary, although it may be as good as any other 

approach. However, to explore the impact of this selection, it might be good to conduct some 

sensitivity analyses using other methods.
DRAFT

Page 58

On page 55, it is stated that “For all models except CO Bag 1, CO Bag 1 – Bag 3, and CH4

Bag 1 – Bag 3, the reduction estimates are statistically significant ...” However, this is not 

true for the pollutants NOx + NMOG or PM.

Were the sensitivity analyses described in Section 7.3.4 (pages 56-59) only conducted for 

Bag 2 NOx? If so, this seems like a major limitation. Also, at the top of page 59 it is 

mentioned that even when removing the influential vehicles from the analysis, sulfur effect “is 

still highly significant.” What are the significance levels before and after removing these 

influential vehicles?

In the Summary and Conclusions Section (page 59) the first bulleted finding should be 

clarified to indicate that the stated NOx, NMHC, and PM reduction values were obtained 

using a clean-out procedure with 28 ppmS. The second bulleted finding should be clarified to 

indicate that although clean-out with 5 ppmS fuel rather than 28 ppmS fuel further reduced 

NOx, NMHC, and CO, no effect was observed for PM.

In discussing the sensitivity analyses within the Summary and Conclusions Section (page 

60) it should be stated that these analyses were only performed for Bag 2 NOx emissions (if 

this is true).

A number of typographical and other errors were also noted:

Page 3, footnote: “The program has collected additional data that is being incorporated ...”

Page 5, 5th line: Insert period after “overall.”

Page 6, footnote to Table ES-3: It is stated that reduction estimates were computed 

differently for Bag 1 THC and CH4 because these clean-out effects were not statistically 

significant. Table ES-1 shows that the clean-out effects were also not statistically significant 

for Bag 1 NOx and NMHC.

Page 6, next to last line: “... are consistent with those formed in the MSAT and Unicor 

studies ...”

Page 17, Figure 6-1: The sulfur clean-out cycle in the “short procedure” (blue-colored box) 

should indicate 28 ppmS, not 30 ppmS.

Page 29, 2nd line: “In addition, the results from the clean-out data were used to supplement 

...”

Page 30, 4th line: “... representing the as received sulfur level ...”

Page 32, 2nd line: “... vehicles from the same vehicle family have the similar emission 

profiles.”

Page 40, 4th line: “However, the sulfur loading effect certainly varies by vehicle ...”

Page 58, 1st ¶, 5th line: “... which is calculated after an iterative process ...”