# The Effects of Gasoline Sulfur Level on Emissions from Tier 2 Vehicles in the In

- Use Fleet

DraftNOTICE
This technical report does not necessarily represent final EPA decisions or positions. It is intended to present technical analysis of issues using data that are currently available. The purpose in the release of such reports is to facilitate the exchange of technical information and to inform the public of technical developments. Assessment and Standards Division Office of Transportation and Air Quality U.S. Environmental Protection Agency 

# The Effects of Gasoline Sulfur Level on Emissions from Tier 2 Vehicles in the In

- Use Fleet

Draft

E PA-420-D-13-003 April 20131 

### Table of Contents Acronyms ........................................................................................................................................ 2 1. Executive Summary ................................................................................................................. 3 2. Introduction ............................................................................................................................. 7 2.1. Background ...................................................................................................................... 7 2.2 Motivation for this Study ............................................................................................... 10 3. Study Design .......................................................................................................................... 10 3.1. Measurement of Reversible In

- Use Loading (Clean

- Out Effect) .................................. 11 3.2 Effect of Sulfur Level ..................................................................................................... 12 4. Test Vehicle Selection, Recruitment, and Delivery .............................................................. 12 4.1 Choice of Makes and Models ......................................................................................... 12 4.2 Vehicle Recruitment Criteria ......................................................................................... 13 4.3 Initial Checks and Test Vehicle Delivery ...................................................................... 14 5. Test Fuel Specs and Procurement .......................................................................................... 15 6. Test Procedures ...................................................................................................................... 16 6.1 Initial fu el exchange and vehicle prep ............................................................................ 16 6.2 Test procedure description ............................................................................................. 17 7. Data Analysis and Results ..................................................................................................... 22 7.1. Data Preparation ............................................................................................................. 23

7.1.1. Imputation of measurements with low concentration ............................................. 23 7.1.2. Detection of outliers ................................................................................................ 26 7.2. Modeling Methodology .................................................................................................. 27 7.3. Statistical Analysis and Results ..................................................................................... 29

7.3.1. Effect of clean -out at 28 ppm.................................................................................. 29 7.3.2. Effect of clean -out at 5 ppm.................................................................................... 34 7.3.3. Effect of Sulfur level ............................................................................................... 38 7.3.4. Sensitivity Analysis ................................................................................................ 53 8. Summary and Conclusions .................................................................................................... 56 9. References ............................................................................................................................. 59 Appendix A. Detail s of On -Road Mileage Accumulation Route ................................................. 62 Appendix B. Emission Concentration Plots.................................................................................. 64 Appendix C. Additional Information on Emission Measurements ............................................... 67 Appendix D. Discussion of U nivariate and M ultivariate A nalysis of V ariance ........................... 68 Appendix E. Plots of A ll Pollutants and B ags .............................................................................. 692 

### Acronyms

A/F Ratio Air

- to

- Fuel Ratio BIC Schwarz Bayesian Criterion CH4 Methane CO Carbon Monoxide CRC Coordinating Research Council CS Compound Symmetry Covariance EPA Environmental Protection Agency F Fahrenheit FID Flame Ionization Detector FTP Federal Test Procedure HC Hydrocarbons LOQ Limit of Quantification MIL Malfunction Indicator Lamp ML Maximum Likelihood MSAT Mobile Source Air Toxics NMHC Non

- Methane Hydrocarbons NMOG Non

- Methane Organic Gases NO x Oxides of Nitrogen NVFEL National Vehicle and Fuel Emission Laboratory PGM Platinum Group Metals PM Particulate Matter PPB Parts per Billion PPM Parts per Million PSI Pounds per Square Inch PZEV Partial Zero

- Emissions Vehicle REML Restricted Maximum Likelihood RLD Restricted Likelihood Distance RVP Reid Vapor Pressure SAS Statistical Analysis Systems THC Total Hydrocarbons UN Unstructured Covariance3 

### 1. **Executive Summary**

Fuel sulfur content has long been understood to affect the performance of emission aftertreatment catalysts in light duty vehicles , where t he sulfur and its oxides occupy active precious metal sites and oxygen storage materials , reducing the catalyst’s efficiency in destroyin g pollutants. Numerous studies have shown the direct impact of fuel sulfur levels above 30 ppm on emissions
, data that formed the basis of the sulfur controls in EPA’s Tier 2 rulemaking for light duty vehicle emissions, published in 2000. 1

Following the successful implementation of the Tier 2 sulfur standards, new research has focused on the emission reduction potential of lowering sulfur levels below 30 ppm, particularly
on Tier 2 and newer technology vehicles, under the hypothesis that increased reliance on the catalytic conver tor will result in a higher sensitivity to fuel sulfur content

. A 2005 study conducted jointly by EPA and several automakers found large decreases in NOx and HC emissions from vehicles meeting Tier 2 Bin 5 emission levels when operating on 6 ppm versus
32 ppm sulfur test fuel

. 2 In order to gain further understanding of how these emission reduction s would translate into the in

- use fleet
, EPA conduct ed the study described here to assess the state
of sulfur loading (poisoning) in typical in

- use vehicle s , as well as the effect of fuel sulfur level on these vehicles during subsequent mileage accumulation. It was designed to take into consideration what was known from prior studies on sulfur build

- up in catalysts over time and the effect of periodic regeneration events that can occur during higher speed and load operation in day- to -day driving.

The study sample described in this analysis consisted of 81 cars and light trucks recruited from owners in southeast Michigan, covering model years 2007

- 9 with approximately 20,000

- 40,000 odometer miles. The makes and models targeted for recruitment were chosen to be representative of high sales vehicles covering a range of types and sizes. Test fuels were two

non- ethanol gasolines with properties typical of certification fuel, one at a sulfur level of 5 ppm and the other at 28 ppm, the higher level chosen to be representative of retail fuel available to the

pu blic in the vehicle recruiting area. All emissions data was collected using the FTP cycle at a
nominal ambient temperature of 75°F.4 Using the 28 ppm test fuel, emissions data were collected from vehicles in their as

received state, and then following a high

- speed/load “clean

- out” procedure consisting of two
back

- to

- back US06 cycles intended to reduce sulfur loading in the catalyst. A statistical analysis of this data showed highly significant reductions in several pollutants including NO x and hydrocarbo ns ( Table ES -1 ), suggesting that reversible sulfur loading exists in the in

- use fleet and has a measurable effect on aftertreatment performance . F or example, Bag 2 N O x emissions dropped 32 percent between the pre- and post-cleanout tests on 28 ppm fuel.

Table ES -1 Average clean

- out effect on in

- use emissions using 28 ppm test fuel

NO x (p

- value) THC (p

- value) CO (p

- value) NMHC (p

- value) CH 4 (p

- value) PM (p

- value) Bag 1 – – 4.7% (0.0737) – – 15.4% (< 0.0001) Bag 2 31.9% (0.0009) 16.5% (0.0024) – 17.8% (0.0181) 15.3% (0.0015) – Bag 3 38.3% (<0.0001) 21.4% (<0.0001) 19.5% (0.0011) 27.8% (<0.0001) 12.0% (<0.0001) 24.5% (<0.0001)
FTP Composite 11.4% (<0.0001) 4.1% (0.0187) 7.6% (0.0008) 3.0% (0.0751) 6.9% (0.0003) 13.7% (<0.0001) Bag 1 – Bag 3 – – 4.2% (0.0714) – – – The clean

- out effect is not significant at α = 0.10 when no reduction estimate is provided.

To assess the impact of lower sulfur fuel on in

- use emissions, a representative subset of vehicles was kept to conduct testing on 28 ppm and 5 ppm fuel s with accumulated mileage. A first step in this portion of the study was to assess differences in the effectiveness of the clea nout procedure when done using different fuel sulfur levels. Table ES -2 presents a comparison of emissions immediately following (<50 miles) the clean

- out procedures at the low vs. high sulfur level. These results show significant emission reductions for the 5 ppm fuel relative to the 28 ppm fuel immediately after this clean

- out; for example, Bag 2 NO x emissions were 47 percent lower after a clean -out on the 5 ppm fuel vs. following the clean

- out on the 28 ppm fuel. This
indicates that either the catalyst is not fully desulfurized after a clean out procedure as long as there is sulfur in the fuel, or that there is an instantaneous effect of sulfur concentration in t he exhaust gas on the catalyst’s activity.5 Table ES -2 Percent reduction in emissions from 28 ppm to 5 ppm for the first three repeat FTP tests immediately following clean -out

NO x (p

- value) THC (p

- value) CO (p

- value) NMHC (p

- value) CH 4 (p

- value) PM
Bag 1 5.9% (0.0896) 5.4% (0.0118) 7.3% (0.0023) 4.6% (0.0465) 11.1% (<0.0001) – Bag 2 47.3% (0.0010) 40.2% (<0.0001) – 34.4% (0.0041) 53.6% (<0.0001) – Bag 3 51.2% (<0.0001) 35.0% (<0.0001) 10.1% (0.0988) 45.0% (<0.0001) 25.4% (<0.0001) – FTP Composite 17.7% (0.0001) 11.2% (<0.0001) 8.3% (0.0003) 8.8% (0.0003) 21.4% (<0.0001) – Bag 1 – Bag 3 – – 5.8% (0.0412) – – – The effectiveness of clean

- out cycle is not significant at α = 0.10 when no reduction estimate is provided

.

To assess the overall in

- use reduction between high and low sulfur fuel, a mixed model analysis of all data as a function of fuel sulfur level and miles driven after cleanout was performed. This analysis found highly significant reductions for several pollutants, as shown in
Table ES -3 ; the reductions for Bag 2 NOx were particularly high, estimated at 59 percent
between 28 ppm and 5 ppm. For most pollutants, the analysis suggested the effect of sulfur level didn’t depend on miles driven after the fuel change, and therefore the emission benefit of lower fuel sulfur occurred immediately and continued as miles were accumulated. Some results , such as Bag 1 hydrocarbons, did show a significant miles

- by

- sulfur interaction. In this case,
determination of a sulfur level effect for the in

- use fleet required estimation of a miles -equivalent level of sulfur loading, which can be gleaned from the cleanout results obtained from the baseline testing on the vehicles as

- received.6 Table ES -3 Summary of mixed model results for emission reductions from 28 to 5 ppm

sulfur, adjusted for in -use sulfur loading (mileage accumulation ) where appropriate

NO x (p

- value) THC (p

- value) CO (p

- value) NMHC (p

- value) CH 4 (p

- value) NO x +NMOG (p

- value) PM
Bag 1 10.7% (0.0033) 8.5% † (0.0382) 7.5% † (0.0552) 7.5% (< 0.0001) 13.9% † (< 0.0001) N/A – Bag 2 59.2% (< 0.0001) 48.8% (< 0.0001) – 44.8% † (0.0260) 49.9% (< 0.0001) N/A – Bag 3 62.1% (< 0.0001) 40.2% (< 0.0001) 20.1% (< 0.0001) 49.9% (< 0.0001) 29.2% (< 0.0001) N/A – FTP Composite 23.0% † (0.0180) 13.0% † (0.0027) 11.9% † (0.0378) 10.6% † (0.0032) 25.8% † (< 0.0001) 17.3% (0.0140) – Bag 1 – Bag 3 – 5.2% (0.0063) 4.3% (0.0689) 5.1% (0.0107) 4.6% (0.0514) N/A – † Model with significant sulfur and mileage interaction term. The effect of fuel s ulfur level is not significant at α = 0.10 when no reduction estimate is provided . For THC bag 1 and CH4 bag 1, because the effect of clean

- out was not statistically significant, the reduction estimates are based on the estimates of least squares means.

Major findings from this study include:

• Reversible sulfur poisoning is occurring in the in

- use fleet of Tier 2 vehicles and has a
measureable effect on emissions of NO x , hydrocarbons, and other pollutants of interest.

• The effectiveness of high speed/load procedures in restoring catalyst efficiency is limited when operating on higher sulfur fuel.

• Reducing fuel sulfur levels from 28 to 5 ppm produces significant reductions in
emissions of NO x , hydrocarbons, and other pollutants of interest from a broad range of
in

- use Tier 2 vehicles

.

• Bag 2 NOx effects , which are relatively large but based on low emission lev els, were found to be robust to sensitivity analyses related to influential vehicles and measurement uncertainty at low emission levels.7 

### 2. **Introduction**

**2.1. Background** 

Sulfur in gasoline has long been found to reduce the conversion efficiency of automotive
three

- way catalysts , with some studies suggesting an increase in catalyst sensitivity (in terms of percent conversion efficiency) to sulfur as tailpipe emission levels decrease

. 3 The Tier 2 light duty emission standards recognized the importance of sulfur and required lower gasoline sulfur
levels in accordance with the relative level of the new emission standards. Since that time, little work has been published on the effect of further reductions of fuel sulfur level
, especially on in

use vehicles that may have some degree of catalyst deterioration due to real world operation

. This study investigates the effects of lo w er sulfur gasoline on in

- use Tier 2 vehicles that have been operated on commercial gasoline (with relatively higher sulfur levels) under real world driving conditions.

Aftertreatment systems utilize catalysts containing precious metals and metal oxides to selectively oxidi ze hydrocarbons and reduce nitrogen oxides in the exhaust gases. Sulfur oxides from fuel combustion preferentially bind to active sites in the catalyst, inhibiting their ability to participate in the intended conversion reactions (sulfur contamination or poisoning) . The amount of sulfur retained by the catalyst is a function of t he type and arrangement of active materials and coatings used within the catalyst
, its operating temperature, as well as the concentration of sulfur
oxides and the air

- to

- fuel ratio of the exhaust gas. 4 , 5 Modern vehicle engines operate with rich

lean oscillations that maintain the proper oxidation-reduction condition of the catalyst, but under typical driving conditions there is a nonzero equilibrium level of sulfur retained. Regularly operating the cata lyst at a high temperature under net reducing conditions can release much the
sulfur oxides from the catalyst ( Figure 2-1 ) , and could minimize the effects of fuel sulfur on catalyst efficiency. However, producing these conditions at sustained and/or regular intervals
would accelerate thermal degradation of the catalyst and may also raise other challenges for emissions control and fuel economy.8 Figure 2-1 Catalyst- sulfur interaction by temperature and A/F ratio (Adapted from information from Ford Motor Company)

Additionally , it is not always possible to maintain these catalyst temperatures (e. g. , cold
weather
, idles)
, and the rich air

- to

- fuel ratios necessary can result in increased PM, NMOG and CO emissions, therefore reducing fuel sulfur levels has been the pri mary regulatory mechanism to minimize sulfur contamination of the catalyst and ensure optimum emissions performance
over the useful life of a vehicle

.

In 2005
, EPA and several automakers jointly conducted a program that examined the effects of sulfur, benzene, and volatility on gaseous emissions from a fleet of nine Tier 2 compliant vehicles (referred to as the Mobile Source Air Toxics or MSAT study 6 ) . The fuel
matrix consisted of four non-ethanol fuels blended in a stepwise manner, starting with a purchased base fuel containing the lowest level s of the three properties of interest. The base fuel 

### Hot

- Rich Hot

- Lean

Cold

- Lean Cold

- Rich 900 °C

5 00 °C Rich Air/Fuel Sulfur oxidation & sulfate decomposition

SO 2 release & conversion to SO 3 , H 2 SO 4



### Lean Air/Fuel Sulfate reduction & release as SO 2 , H 2 S

SO 2 adsorption & reduction to elemental S

(reversible poisoning) SO 2 adsorption & oxidation to SO 3 , SO 49 contained 6 ppm sulfur, and after additions of butane (to arrive at 9 psi RVP) and benzene (to
arrive at 1.0 vol. %), sul fur was increased to 32 ppm using a small amount of doping agent containing a sulfur compound. Thus, the sulfur effect could be deduced by comparing emission results between the final fuel and the one just before it in the blending sequence

. The test
veh icles were broken- in production samples owned by the manufacturers, which had emission catalysts and oxygen sensors bench

- aged to the equivalent of 12 0,000 miles. The dataset consisted of tests performed at four different labs, with three manufacturers te sting two of their own vehicles, and EPA testing the remaining three. Tests conducted on high

- sulfur fuel were preceded by a low

- speed cruise to simulate sulfur loading, and tests on low

- sulfur fuels were preceded by a high

- speed/load cycle used in previous programs performed by the Coordinating
Research Council. 7

Mixed model analyses were performed on the data using vehicle and lab as random effects and fuel properties as fixed effects. Reductions in FTP

- weighted emissions for 6 compared to 32 ppm sulfur were 33 percent for NOx , 11 percent for THC, 17 percent for CO,
and 32 percent for methane (all statistically significant at α = 0. 10 ). T hese data suggested that there were likely to be significa nt emission reductions possible with further reductions in gasoline sulfur level.

A study published in 2011 by Umicor e Autocat USA describes tests performed on a model year 2009 Chevrolet Malibu calibrated to PZEV levels (20 mg/mi NOx), intended to examine the hypothesis that very

- low emitting vehicles are more sensitive to fuel sulfur content. The study used two test fuels, at 3 and 33 ppm sulfur, combined with FTP, US06, and a high speed/load clean

- out cycle similar to one in the MSAT study described above, to examine the impact of sulfur on the underfloor catalyst efficiency during repeated FTP tests. The authors
obs erved consistent degradation of catalyst performance when repeat FTPs were performed using the 33 ppm fuel, versus none with the 3 ppm test fuel. Emission measurements showed a NOx reduction of 41 percent between averages of three replicate FTPs on low ve rsus high sulfur
level

. 810 

### 2.2 **Motivation for this Study**

The MSAT study described in the previous section motivated the design of a new study to further investigate the effects of lower sulfur gasoline on a larger number of vehicles. Specifically , we wanted to assess what the benefits of sulfur control might be under in

- use driving conditions for typical modern vehicles, as previous studies have generally not provided
data that can be utilized for this type of analysis in a straightforward manner

. Many studies have designs that look for a change in emissions at a single point in time shortly after a fuel sulfur change and/or a clean

- out cycle, while in fact catalysts of in

- use vehicles are normally operating at a point somewhere between being clean of sulfur (e.g.
, following high speed, high load and
therefore h igh catalyst temperature operation) and fully loaded with sulfur (e.g.
, following low speed, low load, cool catalyst temperature operation). Others have measured effects of aging catalysts on different fuel sulfur levels, which do not give any information on what happens when fuel sulfur is changed partway through a vehicle’s life. Thus, the design of this study sought to assess the impact of reversible sulfur loading on in

- use Tier 2 vehicles r ecruited from
their owners , a s well as the emission performan ce after a change to lower sulfur fuel and subsequent mileage accumulation

.



### 3. **Study Design** 

This section outlines the hypotheses and theoretical bases for the specific procedures that follow in the testing section.

In order to characterize the effect of sulfur on the in

- use fleet, vehicles were recruited
from their owners and tested in EPA’s National Vehicle and Fuel Emission L aboratory (NVFEL) in Ann Arbor, Michigan

. Given this arrangement, it was not feasible to d irectly measure sulfur loading in the catalysts themselves by means that involve d damagin g or destroying them.
Instead
, the behavior of emissions during various test procedures was used as a proxy for sulfur loading.11 The program design used two fuels differing only in sulfur level, with the higher level of 30 ppm chosen to match the current average sulfur limit promulgated along with the Tier 2
vehicle standards . All other fuel properties were controlled to be representative of conventional
commercia l gasoline (see Table 5 -1). i



### 3.1. **Measurement of Reversible In** 

**- Use Loading (Clean** 

**- Out**  **Effect )** 

The level of reversible in

- use sulfur storage (or loading) and release within a n exhaust catalyst system can be assessed by measuring emissions from the vehicle as received, performing

a high

- speed, high

- load clean

- out cycle, then measuring emissions again. This change in
emissions represents the amount of reversible deactivation relative to a “clean” baseline s tate. Prior to the start of this test program, we consulted previous studies as well a s vehicle calibration
and emission controls experts to select an appropriate clean

- out cycle for this test program

. An example of such a cycle is presented in Appendix C of the CRC E

- 60 emissions study report. 9
The US06 certification cycle, while not as extreme as the E

- 60 cycle, does provide similar
conditions th at are favorable for desulfurization (rich

- lean switching at high temperatures) and is commonly used in the auto industry to desulfurize catalysts prior to emissions testing

. Though
the E

- 60 cycle would be preferred, there were concerns that it would cause undesirable wear and

tear on recruited vehicles. As a compromise, two back

- to -back US06 cycles were selected for use in this test program.

The level of sulfur loading in a vehicle catalyst is primarily a function of the fuel sulfur level and catalyst temperature, the latter of which is a function of vehicle and catalyst design and
driver behavior. A vehicle with a relatively high exhaust temperature at the catalyst location
, and/or significant excess lo ading of certain platinum group metals (PGM) and other active
materials in the catalyst may be relatively insensitive to sulfur loading regardless of driver behavior. These effects of vehicle design were accounted for in the study by recruiting several

i At the time this study was designed, a non

- oxygenated gasoline blend was selected as being representative of in

use fuel as ethanol blends were not as ubiquitous as they are today. However, since sulfur’s effect on (non

- sulfur) emissions is understood to occur solely in the catalyst, we do not anticipate meaningful interaction with ethanol. Moreover, since sulfur was varied independently of other fuel properties, we do not anticipate that the relative difference in emission results would have been advers ely affected by this design choice.12 di fferent makes and models covering a range of engine and vehicle sizes. We attempted to
account for t he effects of the owner’s behavior by testing several samples of the same make and
model.



### 3.2 **Effect of Sulfur Level**

Following vehicle operation that raises catalyst temperature and removes sulfur, accumulation immediately resumes toward a higher equilibrium level as temperature declines under more typical (milder) driving conditions. This loading continues over a period of time with vehicle operation, and can be observed as an increase in emissions ( sometimes referred to as “NOx creep”). To capture this effect, repeated emission tests were performed following a
cleanout at two different fuel sulfur levels. Given that the exhaust stream contain s different concentrations of sulfur according to the fuel sulfur level, the rate of reloading is expected to be
lower for the lower sulfur fuel. Additionally, if an equilibrium loading (stable level of
emissions ) were to be reached after many miles of dri ving, this procedure investigates whether
this equilibrium loading is also lower for the lower sulfur fuel.



### 4. **Test Vehicle Selection, Recruitment, and Delivery** 

**4.1 Choice of Makes and Models** 

The test fleet was chosen to be representative of latest

- techn ology light duty vehicles being sold at the time the program was launched. The study did not attempt to analyze or model details of aftertreatment design specific to each vehicle model such as catalyst position, precious metal types and quantities used, o r related engine control strategies such as timing advance at cold start or fuel cut during deceleration . While these things undoubtedly influence the behavior
of emissions and may interact with the fuel sulfur effects being investigated, including them i n an analysis requires correctly assessing and parameterizing them for all vehicles in the study

. Instead this program’s aim wa s to characterize overall effects on emission inventories by observing the aggregate behavior of a representative fleet of vehicles.13 In terms of emissions standards, the test fleet was to conform on average to Tier 2 Bin 5 exhaust levels and employ a variety of emission control technologies . These goals were achieved by including a range of vehicle sizes, engine displacements, and manufacturers. Engine family sales data obtained from EPA certification and Wards databases w ere analyzed to generate a list of high

- sales vehicles as candidates for inclusion. Grouping sales data by engine family allowed additional transparency and flexibility in choosing test vehicles that represent a wider group with identical powertrains without targeting one specific make and model. The resulting target list of 19 vehicle model s for recruitment is shown in Table 4-1

.

Table 4-1 Test Vehicles Recruited

Make Brand Model Program ID Engine Size Engine Family Tier 2 Cert Bin

GM Chevrolet Cobalt CCOB 2.2L I4 8GMXV02.4025 5 GM Chevrolet Impala FFV CIMP 3.5L V6 8GMXV03.9052 5 GM Saturn Outlook SOUT 3.6L V6 8GMXT03.6151 5 GM Chevrolet Silverado FFV CSIL 5.3L V8 8GMXT05.3373 5 Toyota Toyota Corolla TCOR 1.8L I4 8TYXV01.8BEA 5 Toyota Toyota Camry TCAM 2.4L I4 8TYXV02.4BEA 5 Toyota Toyota Sienna TSIE 3.5L V6 8TYXT03.5BEM 5 Toyota Toyota Tundra TTUN 4.0L V6 8TYXT04.0AES 5

Ford Ford Focus FFOC 2.0L I4 8FMXV02.0VD4 4 Ford Ford Taurus FTAU 3.5L V6 8FMXV03.5VEP 5 Ford Ford Explorer FEXP 4.0L V6 8FMXT04.03DB 4 Ford Ford F150 FFV F150 5.4L V8 8FMXT05.44HF 8 Chrysler Dodge Caliber DCAL 2.4L I4 8CRXB02.4MEO 5 Chrysler Dodge Caravan FFV DCAR 3.3L V6 8CRXT03.3NEP 8 Chrysler Jeep Liberty JLIB 3.7L V6 8CRXT03.7NE0 5 Honda Honda Civic HCIV 1.8L I4 8HNXV01.8LKR 5 Honda Honda Accord HACC 2.4L I4 8HNXV02.4TKR 5 Honda Honda Odyssey HODY 3.5L V6 8HNXT03.54KR 5 Nissan Nissan Altima NALT 2.5L I4 8NSXV02.5G5A 5



### 4.2 **Vehicle Recruitment Criteria**

The aforementioned 19 makes /models selected for recruitment were targeted t o have a mileage between 12,000 and 40,000 miles and an age of less than three years old. The recruitment process used the State of Michigan vehicle owner registration database to identify
candidate vehicle owners wi thin 50 miles of NVFEL

. Solicitations were sent to owners of14 candidate vehicles to find willing participants who would allow their vehicles to be held and
tested for up to six weeks, in exchange for cash and/or loaner vehicle . If an owner agreed to participate, the vehicle w as scheduled for testing.



### 4.3 **Initial Chec ks and**  **Test Vehicle Delivery**

V ehicles were driven to the test facility either by their o wners or NVFEL personnel sent to retrieve them . Drivers were instructed to avoid hard accelerations and high speeds in an effort to preserve the “as

- received” state of the catalyst. Once in custody, vehicles were subjected to visual inspection for obvious damage or malfunction. The condition of the MIL (malfunction indicator, or “check engine”, lamp) was also noted, and if illuminated, the vehicle was not accepted for testing.

Upon receipt of the vehicle at NVFEL , the existing fuel was drained and a small sample

was analyzed for its sulfur content to screen for vehicles that had been operating on unusually high or low sulfur levels. Federal fuel sulfur standards applicable in the recruiting area specify a 30 ppm annual refinery average with an 80 ppm per

- gallon cap. Figure 4-1 shows the
distribution of fuel sulfur levels found in the tanks of test vehicles “as received”. The mean of
the fuel samples for which data were available was 25 ppm with standard deviation of 11.0. Fuel in the recruiting area typically contains 10% ethanol.

All vehicles were inspected for minor fuel and exhaust leaks , and were repaired
, as needed. However, damage to the catalyst, muffler, or sections of exhaust pipe disqualified the
vehi cle from inclusion in the program. F uel leaks were not repaired unless they present ed a safety issue

. A ll repairs were documented

. Gas caps and other fuel systems components were visually inspected.15 Figure 4-1 Fuel sulfur levels for test vehicles as received



### 5. **Test Fuel Specs and Procurement**

A non-ethanol, low-sulfur, certification

- grade gasoline was purchased in bulk from a commercial fuel blende r (Haltermann Solutions, Houston, TX ) and split into two underground
fuel tanks prior to start of the program . One tank was spiked with a small amount (on the order of a liter of volume) of dibutyl disulfide to increase the sulfur content from the as

- received 5 ppm to 28 ppm . Due to the small volume of sulfur agent added, changes in other fuel properties

were assumed to be negligible

. Other properties of the test fuels are listed in Table 5-1 , and fall in the range of what is encountered in typical commercial gasoline in the vehicle recruiting area

. Relative to national average 2010 summer conventional gasoline (much of which contained ethanol) as reported in the Auto Alliance U.S fuel survey database, these test fuels are somewhat higher in aromatics (by about 6 vol

. %), somewhat lower in olefins (by about 6 vol

. %)
, somewhat higher in T50 (by about 20°F), and slightly lower in T90 (by about 3°F).16 Table 5-1 Test Fuel Properties
Fuel Property ASTM Method Low S Test Fuel High S Test Fuel †
Sulfur D2622 5 ppm 28 ppm Benzene D 5769 0.3 4 Vol

. % 0.34 Vol

. %
Total Aromatics D 5769 31

. 2 Vol

. % 31.2 Vo l

. %
Olefins D1319 0.5 Vol

. % 0.5 Vol

. %
Saturates D1319 68.3 Vol

. % 68.3 Vol

. % Oxygenates D5599 0.0 Vol

. % 0.0 Vol

. %
T50 D86 221 °F 221 °F
T90 D86 31 7°F 31 7°F
RVP D5191 9.0 psi 9.0 psi

† S ulfur content wa s confirmed for the higher

- sulfur test fuel, while other properties were assumed to be the same as the base fuel given the small amount of dopant added.



### 6. **Test Procedures**

**6.1 Initial fuel exchange and vehicle prep** 

Fuel drains were generally done via the fue l rail by energizing the vehicle’s pump. Next, the key was turned to the “on” position, without starting the vehicle, for 30 seconds before keying off. This allowed the vehicle to register the new fuel level to trigger any fast fuel trim learning procedu res the vehicle may have. ii Next, the vehicle was filled to 50 percent capacity with test fuel. This fuel drain and fill process was then repeated a second time to ensure that the fuel system ha d been flushed fully prior to testing

.

Following the fuel change the vehicle was moved to the dynamometer to conduct an LA4 preconditioning drive cycle

. iii At the completion of this prep cycle , the vehicle was idled in park

ii Fuel trim refers to the ability of the engine control system to adapt to a change in fuel properties, which may affect vehicle or emissions performance. Some vehicles may accelerate this “learning” process if a fuel change is detected. iii The LA4 drive cycle covers the first 7.5 miles of the FTP test cycle, consisting of typical urban driving behavior. For more details on these test cycles see 40 CFR Part 86.17 for two minutes before engine shutdown , a step intended to ensure fuel trim data was updated for idle operation . The vehicle was then moved to a storage (“soak”) area using an electric towing
device for a 12 -36 hour holding period at approximately 70°F prior to testing.



### 6.2 **Test procedure description**

Vehicles recruited for testing in this p rogram fell into one of two test groups : “Long T est
P rocedure” vehicles or “Short Test Procedure” vehicles. For a given vehicle model class (or family ID) , one of the five vehicles was tested under the L ong procedure while the other four
were tested under the S hort procedure. The vehicle undergoing the L ong procedure was selected at random from willing participants in a given vehicle class. The L ong and S hort procedures are shown in Figure 6-1 and are discussed in greater detail below. They are identical in structure for
the first six emission tests

.18 Figure 6-1 Original Short ( S) and Long (L) P rocedure flow chartLONG PROCEDURE CONTINUES INTO THESE TESTSSHORT PROCEDURE

Pre

- clean

- out (as

- received) replicate FTPs at 28 ppm S

Post

- clean

- out replicate FTPs at 28 ppm S

Add'l replicates for long

- procedure vehicle at 28 ppm S, alternating cold start FTPs for data collection (numbered circles) with unsampled hot start FTPs used for mileage accumulation (blank circles) Sulfur clean

- out cycle, 2 x US06 at 28 ppm S

1

2

3

4

5

6

1

2

3

4

5

6 1

2

3
Data used for "clean

- out effect"

Data used for "sulfur level effect" Fuel change to high

- sulfur (28 ppm) test fuel

Data collection at 5 ppm S, alternating cold and hot start FTPs as above Fuel change to low

- sulfur (5 ppm) test fuel Sulfur clean

- out cycle, 2 x US06 at 5 ppm S19 All vehicles were tested over three cold

- start FTPs with high sulfur fuel to assess “as received” emissions levels prior to performing a sulfur clean

- out cycle (two back

- to

- back US06 cycles) to remove sulfur accumulated in

- use from the catalyst. iv Emissions were then sampled again over three cold

- start FTPs on the same high sulfur test fuel to assess the effect of the sulfur
clean

- out cycle on emissions. This data, collected on all vehicles, was used to deter mine the
“clean

- out” effect and is referred to as the clean

- out dataset in the analysis section of this report. Testing of a short procedure vehicle was complete at this point and the vehicle was returned to
its owner.

Long procedure vehicles continued testing for an additional length of time to determine the long term effects of both high and low sulfur fuel on emissions. Beginning with the fourth post- cleanout FTP test on high sulfur fuel, the test sequence began to alternate between sampled
, cold

- st art FTPs and un

- sampled hot- start FTPs (immediately following the cold

- start test) in order to accumulate additional miles between emission tests. Following the sixth sampled FTP, the fuel was changed to low sulfur test fuel and a clean

- out cycle was run to remove accumulated sulfur from the catalyst and establish a new “cleaned

- out” emissions baseline on low sulfur test fuel. Six additional FTPs (with un

- sampled FTPs in

- between) were conducted on low sulfur fuel. Together with the six high sulfur fuel t ests, this data is used to determine the “sulfur level effect,” which is the assessment of high and low sulfur fuel use starting from an “unloaded” catalyst.

V ehicles tested on the L ong procedure were run for approximately 100 miles of operation
on each fuel following a cleanout. Mid

- way through the test program, however, we were
concerned that 100 miles on each fuel may not be a sufficient amount of time to fully re

- load the catalyst to a level that is representative of relatively mild in

- use driving. Therefore, we modified
the L ong procedure to incorporate several 50- mile on

- road mileage accumulation intervals in

between emission tests on each fuel. This modified L ong procedure, called the L/M procedure
, allowed us to accumulate an additional 100 mile s on each fuel without increasing the number of

iv A cold

- start FTP refers to the FTP drive cycle being performed after the vehicle has been sitting with no engine starts for between 12

- 36 hours at approximately 70°F. This is useful because emissions are generally highest in the first few minutes of cold engine operation. A hot

- start FTP refers to a test driven within a short time after a previous test, such that the engine and aftertreatment are still well above room temperature. Hot

- start tests were only used for mileage accumulation and did not have emissions sampled.20 sampled emission tests . A flowchart of the modified L ong procedure (L/M) is shown in Figure
6-2

. Figure 6-2 Modified Long Procedure (L/M) flowchart

On

- road mileage accumulation for the L/M procedure was performed over a pre

- defined driving route using the same fuel used for emissions testing (either high or low sulfur). The route used was selected for its resemblance to the FTP in terms of speed and load distribution,LONG PROCEDURE CONTINUES INTO THESE TESTSSHORT PROCEDURE

4

5 Pre

- clean

- out ("as

- recieved") replicate FTPs at 28 ppm S

Post

- clean

- out replicates at 28 ppm S

Add'l replicates of cold start FTPs at 28 ppm S

Low sulfur replicates of cold start FTPs at 5 ppm S Sulfur clean

- out cycle, 2 x US06 at 28 ppm S

Sulfur clean

- out cycle, 2 x US06 at 5 ppm S 1

2

3

6

1

2 1

2

3 Data used for "clean

- out effect"

Data used for "sulfur level effect" ~50 Miles on

- road mileage accumulation

Add'l replicates of cold start FTPs at 28 ppm S

Low sulfur replicates of cold start FTPs at 5 ppm S 3

4

Low sulfur replicates of cold start FTPs at 5 ppm S 5

6 ~50 Miles on

- road mileage accumulation

~50 Miles on

- road mileage accumulation

~50 Miles on

- road mileage accumulation Fuel change to low

- sulfur (5 ppm) test fuel Fuel change to high

- sulfur (28 ppm) test fuel21 and was used as a way to rapidly accumulate miles over a representative test cycle without occupying a chassis dynamometer. Refer to Appendix A for more details on this drive route.

Approximately halfway through the data collection
, the Short procedure was extended to include a second clean

- out and subsequent set of emission tests performed on low sulfur fuel

. This change was made after a mid

- point analysis of available data suggested that sulfur level had an effect on emissions starting immediately after the cleanou t for some vehicles. v Figure 6-3
shows the modified S hort procedure containing two fuel sulfur levels, and the change in the number of vehicles providing the “ sulfur level ” data can be seen in Table 7-7 starting with Family ID N513

.

Figure 6-3 Modified Short procedure

v This information became the basis for the “clean

- out at 5ppm” conclusions.EXTENSIONS PROCEDURE Pre

- clean

- out (as

- received) replicate FTPs at 28 ppm S

Post

- clean

- out replicate FTPs at 28 ppm S

Low

- sulfur replicates for modified S procedure, FTPs at 5 ppm S Sulfur clean

- out cycle, 2 x US06 at 28 ppm S

1

2

3

1

2

3 1

2

3
Data used for "clean

- out effect"

Data used for "sulfur level effect" Sulfur clean

- out cycle, 2 x US06 at 5 ppm S Fuel change to low

- sulfur (5 ppm) test fuel Fuel change to high

- sulfur (28 ppm) test fuel22 

### 7. **Data Analysis and Results** 

The data generated by this test program will be discussed as three distinct but overlapping
datasets: “ clean -out at 28 ppm ” data , “ clean

- out at 5 ppm ” data, and “ sulfur level ” data. The
“clean

- out at 28 ppm” data consists of the measurements from original and modified ‘short’ procedures; “clean

- out at 5 ppm” data consists of a subset of original and modified ‘long’ procedures and modified ‘short procedures; “ sulfur level” data consists of the complete data from original and modified ‘long’ procedures and modified ‘short’ procedures. All vehicles
included in the “ sulfur level ” dataset had a subset of their tests included in the “ clean -out”
dataset s, by design.

P ollutants included in the analysis were total hydrocarbons (THC) as reported by the FID analyzer , carbon monoxide (CO), oxides of nitrogen ( NO x ), methane (CH 4 ), as well as

p articulate matter (PM) mass

. The following calculated emissions were also included: nonmethane hydrocarbons (NMHC, defined as THC minus methane ) and oxides of nitrogen plus

non- methane organic gases ( NO x + NMO G) , a measurement relevant in regulatory contexts

. vi Each bag, ‘Bag 1 minus Bag 3’, and the composites from the FTP test cycle were analyzed separately. The first bag captures the initial “cold s tart” , meaning the emissions produced when the vehicle is started after cooling to room temperature overnight. The “hot stabilized” operation is captured in the second bag (a portion of the test that begins after approximately 8 minutes of driving) , and t he emissions from “ hot- re start” are measured in the third bag (a repeat of the cold start bag, but after the vehicle has been turned off for only 10 minutes). The ‘Bag 1 minus Bag
3’ emission was used to isolate cold

- start emissions for each FTP test.

Th e statistical methodologies described in the following section were applied consistently in the analysis of all pollutants and all bags. However, the analysis of nitrogen oxides ( NO x ) from Bag 2 is presented in greater detail to assist the reader in understanding the analytical

vi NMOG refers to non

- methane organic gases. The flame ionization analyzer used to produce the THC result produces an inaccurate measurement of oxygenated emission species, which can be corrected using speciated result s for those compounds along with their analyzer response factors. The ratio of NMOG/NMHC can be predicted with good accuracy using the fuel ethanol content (zero in this case), which was the approach used here to generate the NMOG results. This method is described in Sluder S.C, West B.H., “NMOG Emissions Characterizations and Estimation for Vehicles Using Ethanol

- Blended Fuels”. Oak Ridge National Laboratories, 2011. Publication number ORNL/TM

- 2011/461

.23 approaches and illustrate the statistical methods being used. T he final model results from all other pollutants are also presented in this report.

The goals of the statistical analyses of the “ clean -out” and “ sulfur level ” data were to assess the reversible sulfur loading in the catalysts of the in

- use fleet, to examine the differences
in the effectiveness of the clean

- out cycle by comparing emissions immediately following the
clean

- out procedure at two sulfur levels, and to characterize the effects of fuel sulfur level on
emissions as mileage was accumulated

.



### 7.1. **Data Preparation**
Prior to proceeding with the statistical analyses, the issues associated with the d ataset, namely, very low emission measurements and outliers were examined. The following sections
describe how these issues were addressed.

**7.1.1. Imputation**  **of measurements with low concentration**

The graphical examination of both the “ clea n-out” and the “ sulfu r level ” dataset revealed the presence of very low emission measurements from some pollutants and bags including NO x Bag 2. Since uncertainty associated with these low measurements could potentially affect the outcome of the analysis, it was important to understand the measurement process and evalua te the impact of any uncertainty.

During emission testing, the vehicle exhaust stream is collected and diluted with background air to avoid water condensation and other stability problems. A small sample of this mixture flows into a collection bag for analysis after the test. The concentration of emission species in the bag is determi ned by flowing the contents through a properly

- calibrated gas analyzer and reading the result. This method provides a time

- weighted result via physical integration of the emission stream produced over the course of a transient driving cycle.

The measurement process has uncertainty associated with it as a result of the physics of mixing and sampling from a gas stream as well as noise in analyzer components such as24 optoelectronic detectors and signal amplifiers. This means that repeated measure ments taken under identical process conditions will produce a range of results, their average being the true (intended) response of the instrument and the range around it representing the measurement variability.

For the analyzers used in this program, the size of the measurement error is expected to
increase relative to the measured value as the concentration decreases. Moreover, t he dilute bag
method used here requires measurement of concentrations in both the sample and background bags, followed by a subtraction between the two, such that the net result contains variability from two measurements. To assess whether these issues may affect this dataset, we examined plots of the measured concentrations for each test by vehicle by pollutant and bag. Figure 7-1 shows the
B ag 2 NO x dataset for the vehicles providing the “ sulfur level” data , which contains a number of very low values, as well as tests where sample and background are of similar magnitude ( the vehicle codes refer to the Family IDs listed in Table 7 -7 ; additional plots are available in Appendix B ). Given these findings, we performed sensitivity analyses to evaluate the impact of these low emission measurements on the study results (presented in Section 7.3.4 ). Figure 7-1 Bag 2 NO x concentration measurements by vehicle

0.1

1 10 100 1000ppb NOx

Background Sample, High Sulfur Sample, Low Sulfur Bag 2 NOx concentrationM501-0023M502-0026M513-0089M513-0178M514-0010M515-0006M511-0179M515-0165M520-0022M512-0107M500-0003M503-0194M504-0021M505-0031M506-0123M507-0148M508-0075M509-0046M510-0264M514-0101M515-0007M515-0074M520-001125 When a dilute emission measurement is lower than the measured background level, the net result is reported as zero (this calculation is performed on a test

- by

- test basis)

. However, since it is unlikely that tailpipe emissions are truly zero during a test, it was assumed that a zero is a result of the actual emission level being smaller than the sum of the measurement errors occurring on the sample and background measurement s. The emission level was thus considered to be below the limit of quantification (LOQ), a level below which we are not confident in the accuracy of quantitative value.

In this situation, the data point can be left as zeroes, deleted
, or replaced with an imputed
value. However, because it was necessary to apply log -transformation , as described in more
detail in Section 7.2 , the zero values could not be left as is in the data. Table 7-1 summarizes the number of measurements with zero values and the percentages in parenthesis for each da taset. Given that observations below the LOQ appear to be randomly distributed across sulfur levels
and vehicles
, and since excluding such observations would result in reduced sample size, less statistical power, and larger standard errors, 10 they were imputed in the current analysis.

Since an imputation method involving each vehicle’s own longitudinal data would be superior to methods using no information about the vehicle, 11 a commonly -used single

imputation method, using half the minimum of a valid measurement from a given mileage bin fo r the vehicle with zero values
, was performed. This imputation method recognized the fact that emission measurements below the limit of quantification must be smaller than any quantified
value.

Although using vehicle

- specific imputation minimize s the likelihood of artificially reducing the natural variance of the data, there is a potential to inflate the reliability estimates as the number of imputed values increase. However, since the number of measurements with imputed values are less than 20 p ercent ( Table 7-1 ), we can expect good estimates of the reliability of measures. 12 Nonetheless, it is important to determine the effect of these imputed values on the resulting test statistics and corresponding conclusions. Thus, the results from the statistical analysis with and without the imputed values were compared once the model was finalized to assess the potential for introducing bias. This is discussed further in Section 7.3.4

.26 Table 7-1 Number of measurements with zero values

Clean

- out at 28 ppm data (N = 479) NO x THC CO NMHC CH4 PM Bag 1 0 1 (0.2%) 0 1 (0.2%) 1 (0.2%) 1 (0.2%) Bag 2 32 (6.7%) 6 (1.3%) 33 (6.9%) 32 (6.7%) 4 (0.8%) 2 (0. 4 %) Bag 3 0 1 (0.2%) 21 (4.4%) 35 (7.3%) 1 (0.2%) 1 (0.2%) FTP Composite 0 1 (0.2%) 0 1 (0.2%) 1 (0.2%) 2 (0.4%)

Bag 1

– Bag 3 0 1 (0.2%) 0 1 (0.2%) 1 (0.2%) 1 ( 0.2 %) Clean

- out at 5 ppm data (N = 132) NO x THC CO NMHC CH4 PM Bag 1 0 0 0 0 0 0 Bag 2 14 (10.6%) 2 (1.5%) 3 (2.3%) 5 (3.8%) 3 (2.3%) 0 Bag 3 2 0 1 (0.8%) 8 (6.1%) 0 0 FTP Composite 0 0 0 0 0 0

Bag 1

– Bag 3 0 0 0 0 0 0 Sulfur level data (N = 228) † NO x THC CO NMHC CH4 PM Bag 1 0 0 0 0 0 0 Bag 2 18 (7.9%) 2 (0.9%) 8 (3.5%) 9 (3.9%) 3 (1.3%) 2 (0.9%) Bag 3 3 (1.3%) 0 3 (1.3%) 6 (2.8%) 0 0 FTP Composite 0 0 0 0 0 0

Bag 1

– Bag 3 7 (3.1%) 0 1 (0.4%) 0 0 15 (6.6%) † The sulfur level data for NMHC Bag 3 had 215 measurements.

**7.1.2. Detection of outliers** 

Next, before proceeding to the full analysis, preliminary models were fit to detect

o utliers . The residual plots were visually inspected for outlying observations and the outliers were identified using the screening cri terion value of ± 3.5 for the externally studentized
residua ls. Generally, one can expect about 95% of the externally studentized residuals to be
within ± 3.5 standard deviations, and applying the ±3.5 criterion to detect potential outliers has been widely used in statistics

. When the outlying observation represen ted an actual measurement, it was examined to assess its validity. Since none of the outliers representing actual measurements showed a clear indication of measurement error, it was assumed that the outlying observations were real and thus they were included in the dataset for the analysis. However, there were instances where a very low imputed value was identified as an outlier. In27 such instances, the imputed values were removed from the dataset. Table 7-2 summarizes the
number of outliers as well as the number of imputed measurements that were removed in parenthesis.

Table 7-2 Number of outliers

Clean

- out at 28 ppm data (N = 479) NO x THC CO NMHC CH4 PM Bag 1 2 (0) 6 (1) 2 (0) 6 (1) 2 (1) 2 (0) Bag 2 3 (1) 3 (2) 6 (3) 1 (0) 6 (1) 5 (0) Bag 3 2 (0) 2 (0) 4 (2) 4 (0) 1 (0) 2 (0) FTP Composite 3 (0) 3 (0) 2 (0) 5 (0) 0 (0) 1 (0)

Bag 1

– Bag 3 10 (0) 5 (1) 2 (0) 6 (1) 5 (1) 7 (1) Clean

- out at 5 ppm data (N = 132) NO x THC CO NMHC CH4 PM Bag 1 1 (0) 1 (0) 1 (0) 2 (0) 1 (0) 1 (0) Bag 2 0 (0) 0 (0) 3 (1) 1 (0) 1 (0) 1 (0) Bag 3 2 (0) 1 (0) 1 (0) 1 (0) 0 (0) 0 (0) FTP Composite 1 (0) 0 (0) 1 (0) 1 (0) 0 (0) 1 (0)

Bag 1

– Bag 3 3 (0) 1 (0) 2 (0) 1 (0) 1 (0) 3 (0) Sulfur level data (N = 228) † NO x THC CO NMHC CH4 PM Bag 1 0 (0) 4 (0) 2 (0) 4 (0) 2 (0) 2 (0) Bag 2 0 (0) 1 (0) 4 (1) 3 (1) 2 (2) 1 (0) Bag 3 1 (0) 2 (0) 2 (2) 2 (2) 1 (0) 1 (0) FTP Composite 3 (0) 2 (0) 2 (0) 2 (0) 0 (0) 2 (0)

Bag 1

– Bag 3 2 (1) 2 (0) 2 (0) 2 (0) 1 (0) 4 (0) † The sulfur level data for NMHC Bag 3 had 215 measurements.



### 7.2. **Modeling Methodology**

The following section describes the statistical approaches and the model fitting methodologies applied in the analysis of all three datasets – “ clean -out at 28 ppm”, “clean

- out at 5 ppm”, and “ sulfur level” data.

First, the emission measurements were log

- transformed. In the current study, the
distributions of emis sions exhibited positive skew ness (log -normal), and thus, transforming emission measurements by the natural logarithm was necessary to stabilize the variance, to obtain a linear relationship between the mean of the dependent variable and the fixed and28 rand om effects, and to normalize the distribution of the residual. The log

- transformation of
emission measurements has been well

- established in previous studies analyzing vehicle
emissions data. 13
, 14
, 15

Both the “ clean -out” and “ sulfur level ” data are a classic example of a “ repeated
measures data ” where multiple measurements were taken from a single vehicle at different accumulated mileages. The conventional methods for analyzing “ repeated measures data ” are the univariate and multivariate analysis of variance (discussed further in Appendix D). However, the linear mixed model was selected for the analyse s of both “ clean -out” and “sulfur
level ” data for the following reasons: The mi xed model approach uses generalized least squares
to estimate the fixed effects
, which is considered superior to the ordinary least squares used by the univariate and multivariate procedure

. 15 It is a more robust and flexible procedure in modeling the covariance structures for repeated measures data and better accounts for within

vehicle mileage

- dependent correlations

. 13
, 14 In addition, the mixed model is capable of including vehicles with missing data and handling irregularly spaced measurements.

The MIXED procedure in the SAS 9.2 software package was used to fit the model. The mixed model is represented in Equation 7-1 as:

Y i = X i β + Z i u i + ε i Equation 7-1

where β and u i are fixed and random effects parameters, respectively , and ε i is the random residuals. The mixed model accounts for correlation in the data through the inclusion of random effects and modeling of the covariance structure. β represents parameters that are the same for
all vehicles and u i represents parameters that are allowed to vary over vehicles reflecting the natural heterogeneity in the vehicle fleet. In other words, the model considers the differences in the effect of sulfur level on emission for each vehicle. The distributional assumptions for the
mixed model are : u i is no rmal with mean 0 and variance G i ; ε i is normal with mean 0 and
variance R i ; the random components u i and ε i are independent.

In developing the mixed model, a top

- down model fitting strategy, similar to previously established methods, 16
, 17 was used. The first step was to start with a saturated model , including29 all candidate fixed effects to ensure unbiasedness of the fixed effect estimates. Next, we selected a model with the most optimal covariance structure
, which specifies the variation between
vehicles as well as the covariation between emission measurements at different accumulated mileages on the same vehicle. Finally, the fixed effects portion of the model was reduced to fit
the final model.



### 7.3. **Statistical Analysis and Results**

**7.3.1. Effect of clean** 

**- out at 28 ppm** 

The ma in objective of analyzing the “ clean -out at 28 ppm ” data was to assess the in

- use reversible sulfur loading in the catalysts at the fuel sulfur level of 28 ppm. In addition, the
results from the “ clean -out” data were used to supplement the “ sulfur level ” data in estimating
the differences of in

- use sulfur loading between high and low fuel sulfur levels
, as described in
Section 7.3.3

.

The “ clean -out at 28 ppm ” data consists of the as

- received emission measurements (pre

cleanout) and the measurements after the back

- to

- back US06 cycles (post

- cleanout) at the fuel sulfur level of 28 ppm from original and modified ‘short’ procedures . The change in emissions between pre - and post

- cleanout represents the amount of reversible in

- use sulfur loading at 28 ppm. The “ clean -out at 28 ppm” data at the time of the analysis comprised 17 vehicle families, for a total of 81 unique vehicles. Vehicles from the same engine family had the same engine size, vehicle configuration, and weight. The average starting odometer reading of 81 unique vehicles

was 32,508 ± 6,164 miles. Additional details of this test fleet are shown in Table 7-3 . A total of
479 measurements were taken – 242 pre-cleanout and 237 post- cleanout measurements.30 Table 7-3 Description of vehicles in the “ clean

- out at 28 ppm data ”

Vehicle Family ID Make Model Model Year Tier 2 Cert Bin Number of Vehicles Average Starting Odometer M500 Toyota Corolla 2007 5 5 32,520 M501 Ford Explorer 2007 4 5 33,548 M502 Dodge Caliber 2007 5 5 31,131 M503 Honda Odyssey 2007 5 5 35,893 M504 Saturn Outlook 2007 5 5 35,700 M505 Chevrolet Silverado 2007 5 5 37,342 M506 Nissan Altima 2007 5 5 32,227 M507 Ford Taurus 2007 5 5 29,407 M508 Dodge Caravan 2007 8 5 34,317 M509 Chevrolet Impala 2007 5 5 26,126 N510 Toyota Sienna 2007 5 5 30,939 N511 Chevrolet Cobalt 2008 5 6 28,106 N512 Jeep Liberty 2008 5 5 27,530 N513 Ford Focus 2008 4 5 26,843 N514 Honda Civic 2008 5 3 32,869 N515 Ford F150 2008 8 5 31,662 N520 Toyota Tacoma 2009 5 2 28,837

The box

- plot of the “ clean

- out at 28 ppm ” data ( Figure 7-2 ) for log

- transformed NO x bag 2 shows the spread of the data for each vehicle family by clean

- out status. The diamond and the
line inside the box represent the mean and the median, respectively. The box represents the interquartile range between 25 th and 75 th percentile and the error bars show the full data range.

The box -plot suggests that the variance in the pre

- cleanout group tends to be larger than the variance observed in the post

- cleanout group. Generally, there is a tendency for the pre

cleanout measurements
, represent ing the as

- received sulfur level
, to have higher NO x than the post- cleanout measurements. In addition, the plot illustrates that the emission profiles are substantially dif ferent between vehicle families. Some of the vehicle families also show the presence of within

- family variability , as demonstrated by each error bar, reflecting the influence
of individual drivin g patterns on in

- use sulfur loading

. These findings from the graphical examination of the data assisted in selecting the appropriate statistical model for the analysis

.31 Figure 7-2 Box-plot of vehicle famil ies by pre - and post -cleanout at 28ppm (NO x Bag 2)

The dependent variable ( Y i ) of the mixed model (Equation 7

- 1) was the natural logarithm of emissions. The fixed effect ( X

i ) included in the model was the cleanout status (pre - and post

cleanout), vehicle type (car vs. trucks), and the interaction term. The random effects ( Z i ) were each vehicle family in the study assuming that vehicles from the same vehicle family have similar emission profiles. For the degrees of freedom, the Satterthwaite appro ximation was used
for tests of fixed effects. 18

First, a test for the significance of the between

- family variation was performed for inclusion of the random intercept for each family using the likelihood ratio test. For all pollutants and bags, the result was significant and the random intercept for each family was included for all subsequent models. The significance of between family

- variation was observed graphically in Figure 7-2 and is also supported by recent automotive emissions studies. 19
, 2032 The covariance structure was estimated using restricted, or residual, maximum likelihood (REML) 21 and selected by comparing the Schwarz Bayesian Criterion (BIC) values for various potential covariance structures. BIC, considered a consistent criterion, accounts for the total
number of observations in the da taset and has generally been known to perform relatively better for small size settings. 22
, 23
, 24

In modeling the covariance structures, the variance associated with the pre

- cleanout was permitted to differ from the variance associated with the post

- cleanout. As an initial step, a saturated model with fixed effects was attempted to be fit with “unstructured” (UN) covariance
, which makes no assumptions regarding equal variances or correlations. One advantage of starting with the unstructured covariance is the fact that it allows selection of the most optimal covariance structure based on the data by visually examining the matrix for patterns. However, the model failed to converge and it was not possible to fit an unstructured covariance matrix. It may be explained by the fact that UN structure requires estimation of a large number of variance and covariance parameters , resulting in computational problems.

Since the “ clean -out” data involved three repeat tests of FTP for both pre - and post

cleanout without mileage accumulation, the covariance matrix of compound symmetry (CS), in Equation 7 -2 , was selected
, which assumes that the measurements from the same vehicle have
the ho mogeneous variance and the correlation among measurements is constant regardless of how far apart the measurements are.

R i = Var ( ε i ) = ⎣ ⎢ ⎢ ⎢ ⎡ σ 2 + σ 1 2 σ 1 2 σ 1 2

σ 1 2 σ 2 + σ 1 2 σ 1 2

σ 1 2

σ 1 2 σ 1 2

σ 1 2 σ 2 + σ 1 2

σ 1 2 σ 1 2

σ 1 2

σ 1 2

σ 2 + σ 1 2 ⎦ ⎥ ⎥ ⎥ ⎤ Equation 7-2

The covariance structure was modeled using a combination of compound symmetry
structure within families and a random effect between families . This combination structure specifies a n inter

- vehicle random effect of differences between families , and a correlation
structure within families that are the same between emission measurements

. It implies that the only aspect of the covariance between repeated measures is due to the vehicle contribution.33 Following the selection of structures for the random effects and the covariance structure
for the residuals, the fixed effects in the model were tested using the approximate F

- test with the Satterthwaite degrees of freedom. A non

- significant effect was removed from the model and the
reduced model was re

- fit and re

- tested until all fixed effects in the model were significant . The significanc e level of 10 percent (α = 0.1) was used to test the null hypothesis. Statistical hierarchy was kept in removing insignificant fixed parameters – first order terms were included when the second order term was significant.

We then performed the likelihoo d ratio test between the reference and the nested models using M aximum Likelihood (ML ) 21
, 24 as the estimation method to examine whether the model can be reduced further without influencing the model fit. To perform the likelihood ratio test, the

- 2 Res Log Likelihood Scores from two separate m odels were considered and the chi s quare test statistic was computed by subtracting the

- 2 Res Log Likelihood from the model with more parameters from the model with fewer . The Chi

- Square test statistic had the degrees of freedom equal to the differ ence o f the number of parameters. When this test statistic was significant, the model with the greater number of parameters was chosen. However, when the test statistic was not significant, there is no significant difference in the fit of the two models and the model with
the fewer number of parameters was selected

. Once the fixed effects were finalized based on the chi square test, no further simplification of the fixed

- ef fects component of the model was performed. For all pollutants and bags, only the cleanout effect was significant – the emissions did not differ significantly between cars and trucks.

The change in emissions between pre - (as

- received level) and post

- cleanout at 28 ppm was estimated using the differences of least squares means from the final model. The Tukey

Kramer adjustment was used in calculating the p

- values for the least squares me ans. The difference of least squares means between pre - and post

- cleanout were reverse

- transformed to grams per mile space to estimate the percent reduction in emissions. Table 7-4 shows the reductions in emissions for all pollutants between pre - and post

- cleanout at the fuel sulfur level of 28 ppm , reflecting the level of reversible sulfur loading in catalysts

. The statistical analysis of
the “ clean

- out at 28 ppm ” showed highly significant reductions in emissions for several34 pollutants including NO x and hydrocarbons. The findings here suggest there is a degradation of emission performance for Tier 2 vehicles in

- use due to sulfur loading in the catalyst
, and it has a measureable effect on aftertreatment performance

.

Table 7-4 Average emission reduction post-cleanout vs. pre- cleanout using 28 ppm test fuel

NO x (p

- value) THC (p

- value) CO (p

- value) NMHC (p

- v alue) CH4 (p

- value) PM (p

- value) Bag 1 – – 4.7% (0.0737) – – 15.4% (< 0.0001) Bag 2 31.9% (0.0009) 16.5% (0.0024) – 17.8% (0.0181) 15.3% (0.0015) – Bag 3 38.3% (<0.0001) 21.4% (<0.0001) 19.5% (0.0011) 27.8% (<0.0001) 12.0% (<0.0001) 24.5% (<0.0001) FTP Composite 11.4 % (<0.0001) 4.1 % (0.0 187 ) 7.6 % (0.0 008 ) 3. 0 % (0.0 751 ) 6.9 % (0.0 003 ) 13.7 % (<0.0001) Bag 1 – Bag 3 – – 4.2% (0.0714) – – –
The clean

- out effect is not significant at α = 0.10 when no reduction estimate is provided

.

**7.3.2. Effect of clean** 

**- out**  **at 5 ppm**

To study the differences in the effectiveness of the clean -out procedure between 28 ppm and 5 ppm fuel sulfur levels, a dataset was constructed using the subset of “ sulfur level ” data
which include the measurements from a subset of original and modified ‘long’ procedures and modified ‘short’ procedure with mileage accumulation less than 50 miles for both sulfur levels. This dataset essentially represents the first three repeat tests of FTP from each sulfur level following a high- speed/load “clea n-out” procedure consisting of two back- to

- back US06 cycles.

The dataset include d 17 vehicle families, 23 individual vehicles with 1 3 2 observations – 68 measurements from clean

- out at 28 ppm and 64 measurements from clean

- out at 5 ppm. The average star ting odometer reading of 23 unique vehicles was 31,869 ± 6,572 miles. Additional
details of the test fleet are shown in Table 7-5

.35 Table 7-5 Description of vehicles in the “ clean -out at 5 ppm data”

Vehicle Family ID Vehicle ID Make Model Model Year Tier 2 Cert Bin Number of Vehicles Average Starting Odometer M500 0003 Toyota Corolla 2007 5 1 33,122 M501 0023 Ford Explorer 2007 4 1 27,562 M502 0026 Dodge Caliber 2007 5 1 29,097 M503 0194 Honda Odyssey 2007 5 1 35,816 M504 0021 Saturn Outlook 2007 5 1 43,733 M505 0031 Chevrolet Silverado 2007 5 1 27,891 M506 0123 Nissan Altima 2007 5 1 39,936 M507 0148 Ford Taurus 2007 5 1 28,802 M508 0075 Dodge Caravan 2007 8 1 41,117 M509 0046 Chevrolet Impala 2007 5 1 37,734 N510 0264 Toyota Sienna 2007 5 1 38,464 N511 0179 Chevrolet Cobalt 2008 5 1 38,722 N512 0107 Jeep Liberty 2008 5 1 24,614 N513 0089, 0178 Ford Focus 2008 4 2 24,726 N514 0010, 0101 Honda Civic 2008 5 2 30,091
N515 0006, 0007, 0074, 0165 Ford F150 2008 8 4 29,738
N520 0011, 0022 Toyota Tacoma 2009 5 2 28,897

Figure 7-3 shows the box- plot of log- transformed emissions from B ag 2 NO x by vehicle and by clean

- out sulfur level at 28 ppm and 5 ppm. The diamond and the line inside the box represent the mean and the median, respectively. The box represents the interquartile range
between 25 th and 75 th percentile and the error bars show the full data range. The data generally
shows that the NO x emissions from clean

- out at 28 ppm fuel sulfur are higher compared to the
emissions from clean

- out at 5 ppm , suggesting that the fuel sulfur level might affect the
effectiveness of clean

- out cycle in reducing sulfur loading in the catalyst . Furthermore, the data illustrates that there are significant between - and within

- veh icle variability.36 Figure 7-3 Box-plot of v ehicle emissions by clean

- out sulfur level at 28 ppm and 5 ppm (NO x Bag 2)

The statistical approach described in Section 7.3.1 was applied in modeling the fixed
effects and the covariance structure. Table 7-6 summarizes the perc ent reduction in emissions from 28 ppm to 5 ppm fuel sulfur level only for the first three test replicates following the clean

out cycle. The results from bag 2 NO x , for example, shows that immediately after the clean -out, a reduction of 47 percent was achieved by lowering the fuel sulfur level. This suggests that the effectiveness of high

- temperature and high

- speed cleanout cycle to reduce catalyst sulfur loading
is limited by the level of sulfur in the fuel. Also of note here is the absence of any s tatistically significant PM reductions for these procedures; there is more discussion of PM effects in 7.3.3.37 Furthermore, the reduction in emissions from cleanout shown in Table 7-4 would likely be larger if the low sulfur test fuel at 5 ppm had been used for the cleanout procedure and the

tests that immediately follow ed the as

- received baseline emissions. This is demonstrated in Figure 7-4, which compares the pre

- cleanout ( as

- received ) emissions level to the emissions level
after the clean

- out using 28 ppm and 5 ppm fuel sulfur. The data generally shows that the lower fuel sulfur level increases the effectiveness of the cleanout cycle. In order to eliminate t he vehicle variability, only the pre

- cleanout results from 23 unique vehicles that received the cleanout at both 28 ppm and 5 ppm are included (rather than comparing to the pre -cleanout
results from 81 vehicles in Section 7.3.1 ).

Figure 7-4 Box- plot of vehicle emissions by pre

- cleanout and po st

- cleanout at 28 ppm and 5 ppm (NO x Bag 2)38 Table 7-6 Percent reduction in emissions from 28 ppm to 5 ppm for the first three repeat FTP tests immediately following clean -out

NO x (p

- value) THC (p

- value) CO (p

- value) NMHC (p

- value) CH4 (p

- value) PM ‡

Bag 1 5.9% (0.0896) 5.4% (0.0118) 7.3% (0.0023) 4.6% (0.0465) 11.1% (<0.0001) – Bag 2 47.3% (0.0010) 40.2% (<0.0001) – ‡ 34.4% (0.0041) 53.6% (<0.0001) – Bag 3 51.2% (<0.0001) 35.0% (<0.0001) 10.1% (0.0988) 45.0% (<0.0001) 25.4% (<0.0001) – FTP Composite 17.7% (0.0001) 11.2% ( <0.0001) 8.3% (0.0003) 8.8% (0.0003) 21.4% (<0.0001) – Bag 1 – Bag 3 – ‡
– ‡ 5.8% (0.0412) – ‡
– ‡
–
‡ Sulfur level not significant at α = 0.10

.

**7.3.3. Effect of Sulfur level** 

The “ sulfur level” data represents the emission measurements from the repeated FTP test cycles following the clean

- out. This dataset provided key information for assessing the in

- use effect of sulfur level on emissions over time as vehicles operate on fuels with different sulfu r levels. The dataset is comprised of emission measurements from two different fuel sulfur levels at 28 ppm and 5 ppm
, following vehicle operation that removed sulfur from the catalyst (clean

out procedure) . The “ sulfur level” data included all measureme nts from vehicles tested on both sulfur levels, for a total of 23 unique vehicles from 17 vehicle families ( Table 7-7 ) . The average starting odometer of 23 unique vehicles was 31,869 ± 6,572 miles. A total of 228 measurements
were taken from Long (S/L and L/M) and modified Short procedures – 114 measuremen ts each for both high and low fuel sulfur levels.39 Table 7-7 Description of vehicles in the “ sulfur level ” data

Vehicle Family ID Vehicle ID Make Model Model Year Tier 2 Cert Bin Number of Vehicles Average Starting Odometer M500 0003 Toyota Corolla 2007 5 1 33,122 M501 0023 Ford Explorer 2007 4 1 27,562 M502 0026 Dodge Caliber 2007 5 1 29,097 M503 0194 Honda Odyssey 2007 5 1 35,816 M504 0021 Saturn Outlook 2007 5 1 43,733 M505 0031 Chevrolet Silverado 2007 5 1 27,891 M506 0123 Nissan Altima 2007 5 1 39,936 M507 0148 Ford Taurus 2007 5 1 28,802 M508 0075 Dodge Caravan 2007 8 1 41,117 M509 0046 Chevrolet Impala 2007 5 1 37,734 N510 0264 Toyota Sienna 2007 5 1 38,464 N511 0179 Chevrolet Cobalt 2008 5 1 38,722 N512 0107 Jeep Liberty 2008 5 1 24,614 N513 0089, 0178 Ford Focus 2008 4 2 24,726 N514 0010, 0101 Honda Civic 2008 5 2 30,091
N515 0006, 0007, 0074, 0165 Ford F150 2008 8 4 29,738
N520 0011, 0022 Toyota Tacoma 2009 5 2 28,897

The box

- plot of the log

- transformed emissions from Bag 2 NO x “ sulfur level ” data ( Figure 7-5 ) shows the spread of the data for each vehicle and sulfur level across all mileages. The diamond and the line inside the box represent the mean and the median, respectively. The box represe nts the interquartile range between 25 th and 75 th percentile and the error bars show the full data range. Generally, there is a tendency for the vehicles running on high sulfur fuel to emit

more NO x than the vehicles running on low sulfur fuel. However, the effect of operation on higher sulfur fuel certainly varies by vehicle. For example, Toyota Corolla, Ford Focus, and Toyota Tacoma clearly show a large effect of fuel sulfur level on emissions while the effect is

more marginal for Nissan Altima and Toy ota Sienna. Furthermore, the “ sulfur level ” data illustrates the presence of substantial between

- vehicle variability
, and suggest ing that each
vehicle be considered as a random effect in constructing a statistical model. Although some of
the vehicle fami lies had multiple vehicles (i.e., vehicle family IDs N513, N514, N515, and N520), we did not consider each family as random effects since the vehicles from the same40 family had markedly different emission profiles, suggestive of considerable between

- vehicle variability even within the same family.

Also, the increase in emissions as vehicles accumulate mileage for the high sulfur level is more significant compared to the low sulfur level, contributing to the increased variance for
some vehicles
, and suggests that the rate of sulfur loading might be different for the two fuel
sulfur levels. This is shown in Figure 7-6 , which shows the log

- tran sformed emissions from individual vehicles by sulfur level. Thus, the interaction between sulfur level and the accumulated mileage was included in the statistical modeling of the data. These findings from the graphical examination of the data assisted in selecting the statistical modeling approach
discussed in this section

.

Figure 7-5 Box- plot of individual vehicles by sulfur level ( NO x Bag 2)41 We refrained from looking at the simple descriptive statistics, such as the mean and the standard deviation, to assess the relationship between the sulfur level and emissions even as a preliminary step, because reaching any conclusion from them can potenti ally be very misleading
due to the presence of repeated measurements, and both between

- vehicle and within

- vehicle variability. In addition, the mileage accumulations varied from vehicle to vehicle
, and simple descriptive statistics would not be able to ac curately capture the significant amount of variability
inherent in the dataset.

Figure 7-6 Log- transformed emissions from individual vehicles by sulfur level ( NO x Bag 2)

In analyzing the “ sulfur level ” data , a similar top- down model fitting statistical approach applied to the “ clean -out” data, as described in Section 7.3.1
, was applied to characterize the effects of fuel sulfur level on emissions as a function of accumulated mileages since cleanout. The dependent variable ( Y i ) was the natural logarithm of emissions. The fixed effects ( X

i ) included in the model were sulfur level, accumulated mileage, vehicle type, and the interaction42 terms. The random effects ( Z i ) were each vehicle in the study. The likelihood ratio test for the significance of between -vehi cle variation was statistically significant for all pollutants and bags, and thus, the random intercept for each vehicle was included in the model. The significance of
the between

- vehicle variation was observed graphically in Figure 7-5

.

All of the measurements on the same vehicle will have the same between

- vehicle errors;
their within

- vehicle errors will differ, and can be correlated within a vehicl e. The measurements from the same vehicle are correlated simply because they share common characteristics. Also, measurements on the same vehicle close in time are often more highly correlated than measurements far apart in time as observed in Figure 7-6 – the covariation within vehicles. Both
within - and between

- vehicle errors are assumed independent from vehicle to vehicle. Since measurements on different vehicles are assumed independent, the structure refers to the covariance pattern of measurements on the same subject. For most of these structures, the covariance between two measurements on the same vehicle depends only on the differences in mileage accumulation between measurements, an d the variance is constant over mileage.

The covariance structure was modeled by first fitting the “unstructured” (UN) covariance matrix with a saturated model including all fixed effects. For similar reasons provided in the analysis of the “ clean -out” data, the unstructured model failed to converge. Next, since emissions were measured irregularly
, where the mileage intervals between measurements are more or less unique to each vehicle, the spatial covariance structure, which allows for a continuous representation of mileage, was fit. However, the model failed to converge for the spatial covariance matrix as well. Thus, we proceeded to fit the compound symmetry (CS) which specifies that measurements at all mileage have the same variance, and that all measurements on the same vehicle have the same correlation. The BIC value for the compound symmetry was 803.36.

Lastly, the first -order autoregressive structure was modeled. It assumes that the variances are homogeneous and the correlations decline expo nentially with time , i.e., the variability in

m easured emissions is constant regardless of mileage for each vehicle and the measurements next to each other are more correlated than the measurements further apart. The BIC value for43 the first

- order autoregressive structure was 764.90. Based on the BIC value, the first -order autoregressive structure (Equation 7- 3) was selected as the covariance matrix.

R i = Var ( ε i ) = ⎣ ⎢ ⎢ ⎡ σ 2 σ 2 ρ σ 2 ρ 2

σ 2 ρ σ 2 σ 2 ρ

⋮
σ 2 ρ n−1 ⋮
σ 2 ρ n−2 ⋮
σ 2 ρ n−3 ⋯ σ 2 ρ n−1

... σ 2 ρ n−2
⋱ ⋮

... σ 2 ⎦ ⎥ ⎥ ⎤ Equation 7-3

A combination of first -order autoregressive structure within vehicles and a random effect
between vehicles was used to model the covariance structure which specifie d an inter

- vehicle
random effect of differences between vehicles , and a correlation structure within vehicles that decreases with increasing mileage lag between emission measurements

. Furthermore, the variance associated with the low sulfur level was permitted to differ from the variance associated with the high sulfur level. Since the first

- order autoregressive structure was selected due to limited available options
, we acknowledge that there might be some limitations inherent in the assumption of constant distance between two measurements. However, the estimates of fixed effects, such as the differences between sulfur level means, may be the same for diff erent covariance structures , differing only in the standard errors of these estimates.

Once the structures for the random effects and the covariance structure for the residuals were selected, the fixed effects in the model were tested using the approximate F

- test with the
Satterth waite degrees of freedom. The step

- wise backward elimination approach was used to

remove any non

- significant fixed effects (shown in red in Table 7-8 ), starting from the saturated
model. The significance level of 10% (α = 0.1) was used to test the null hypothesis while keeping statistical hierarchy

.44 Table 7-8 Type 3 Tests of Fixed Effects (NO x Bag2)

† slevel = sulfur level (high and low); miles = accumulated mileage since clean

- out; vehclass = vehicle type s (car and truck); ‡ Pr > F represents the p

- value associated with the F statistic;

Then, the likelihood ratio test using maximum likelihood was performed to examine if the model can be reduced further without compromising the model fit. For example, comparing model 4 and 5 ( Table 7-9 ), since the result of the likelihood ratio test was significant, we concluded that accumulated mileage indeed has an effect on B ag 2 NO x , after controlling for sulfur level, and thus, model 4 was selected as the final model.

Table 7-9 Likelihood ratio test for Bag 2 NO x model

Fixed effects in model

- 2 Res Log Likelihood p - value (χ 2 ) Model 4 slevel, miles 740.0 0.04713 Model 5 slevel 737.2

The final NO x B ag 2 model (model 4) had sulfur level and mileage terms as the fixed effects. Thus, the model finds that there is a statistically significant difference in emissions from high and low fuel sulfur levels that did not differ between vehicle types (car vs. truck) since sulfur level and vehicle type interaction term was not significant. Also, since the mileage term is significant, it can be concluded that the mileage accumulation after the clean

- out increases
emission s independent of the fuel sulfur level. However, since the sulfur level and the accumulated mileage interaction term was not significant, the model suggests that the rate of Effect † Num DF Den DF F Value Pr > F ‡

Model 1 slevel 1 93.8 5.32 0.0232 miles 1 197 0.85 0.3579 vehclass 1 23.1 0.11 0.7443 slevel * miles 1 85.3 2.36 0.1279 miles * vehclass 1 199 0.00 0.9591

Model 2 slevel 1 94.4 5.35 0.0229 miles 1 161 7.67 0.0063 vehclass 1 21 0.11 0.7468 slevel * miles 1 85.9 2.40 0.1247

Model 3 slevel 1 94.5 5.38 0.0226 miles 1 161 7.69 0.0062 slevel * miles 1 86.1 2.38 0.1263
Model 4 slevel 1 68 19.25 < 0.0001 miles 1 167 8.31 0.004545 sulfur loading does not vary by accumulated mileages after the clean

- out between high and low fuel sulfur levels. In other words, the effect of high fuel sulfur on Bag 2 NO x exists immediately
after clean

- out and remains essentially constant on a percentage basis, during subsequent driving
of a vehicle.

Figure 7-7 shows the data vs. predicted plots based on the final model for NO x Bag 2. There are two paired plots next to each other with the same vehicle ID showing emissions from both high and low sulfur. There are some instances (e.g., vehicle ID 0046) where the model overestimates the effect of sulfur by over

- predicting high sulfur and under

- predicting low sulfur. In contrast, there are other instances (e.g., vehicle ID 0178) where the model undere stimates the effect of sulfur by under

- predicting high sulfur and over

- predicting low sulfur. However, this is to be expected given the variability in the emission testing. In general, the model predictions are in agreement with the data.46 Figure 7-7 Data vs. predicted (log-transformed Bag 2 NO x )47 Furthermore, the one

- to

- one plot of data vs. model predictions in Figure 7-8 shows that the points generally lie close to the 1:1 line and has the adjusted R

- square of 0.72, demonstrating reasonable accuracy in model predictions for Bag 2 NO x .

Figure 7-8 Data vs. predicted (log- transformed NO x Bag 2)

Figure 7-9 presents the model predictions for individual vehicle by sulfur level for Bag 2
NO x in grams/mile. The model

- predicted values depicted in this figure were generated from
Model 4 in Table 7-8 for NO x bag 2. The model for B ag 2 NO x shows that sulfur loading resumes immediately after the clean

- out (at or near zero accumulated mileage) with high fue l sulfur level emitting significantly higher emissions compared to low fuel sulfur level. In addition, the sulfur loading continues with vehicle operation
, causing an increase in emissions.
The rate of sulfur loading was not statistically different betw een high and low sulfur levels on a percentage basis

.48 Figure 7-9 Model predictions for individual vehicle by sulfur level (NO x Bag 2)49 Table 7-10 summarizes the final models that were selected for all pollutants and bags, applying the same statistical methodology described for Bag 2 NO x . For models without the sulfur level and mileage interaction term, the change in emissions from reducing the fuel sulfur from 28 ppm to 5 ppm was estimated using the differences of least squares means from the final model, adjusting for other effects in the model. The Tukey

- Kramer adjustment was used in calculating the p

- values for the least squares means. The differences of least squares means between high and low fuel sulfur level were reverse

- transformed to estimate the percent
reduction in emissions ( Table 7-11 ). When the sulfur level and mileage interaction term is not significant, the percent differences in emissions between high and low fuel sulfur levels st ay constant across accumulated mileage after clean

- out (the sulfur loading curves for high and low sulfur are parallel) and thus, using the least squares means to quantify the reduction in emissions without considering the as- received in -use sulfur loading was sufficient.50 Table 7-10 Final models of all pollutants

Pollutant Bag Fixed Effects †

NO x Bag 1 slevel, miles Bag 2 slevel, miles Bag 3 slevel, miles FTP Composite slevel, miles, slevel * miles Bag 1

– Bag 3 slevel

THC Bag 1 slevel, miles, slevel * miles Bag 2 slevel, miles Bag 3 slevel, miles FTP Composite slevel, miles, slevel * miles Bag 1

– Bag 3 slevel

CO Bag 1 slevel, miles, slevel * miles Bag 2 slevel Bag 3 slevel, miles FTP Composite slevel, miles, slevel * miles Bag 1

– Bag 3 slevel

NMHC Bag 1 slevel, miles Bag 2 slevel, miles, slevel * miles Bag 3 slevel, miles FTP Composite slevel, miles, slevel * miles Bag 1

– Bag 3 slevel

CH 4 Bag 1 slevel, miles, slevel * miles Bag 2 slevel, miles Bag 3 slevel, miles FTP Composite slevel, miles, slevel * miles Bag 1

– Bag 3 slevel, miles

PM Bag 1

– Bag 2

– Bag 3

– FTP Composite

– Bag 1

– Bag 3 miles
† slevel = sulfur level (high and low); miles = accumulated mileage since clean

- out;

However, for models with significant sulfur level and mileage interaction term, the sulfur effect does depend on mileage accumulation and the rate of sulfur loading is different for high and low sulfur levels. Thus, the in

- use emission level upon arrival prior to clean

- out was necessary to accurately estimate the percent differences in emissions between high and low fuel
sulfur levels for in

- use operation. Without comparing the high sulfur loading curve to the level
of as

- received loading, the calculated emissions difference between high and low sulfur may be arbitrarily high or low based on the point at which the comparison is made. Comparing51 emissions immediately after clean

- out, for example, may not accurately represent the role of sulfur accumulation in the real world. Conversely, comparing emissions after a large number of miles accumulated on the FTP cycle (relatively mild driving) may overestimate the effect of loading. Comparing just the average results of high sulfur to the average results of low sulfur would bias the results depending on the number of tests included in the analysis.

Therefore, for cases where the sulfur level and mileage interaction term (slevel * miles) was significant, the in

- use emission level upon arrival (pre

- cleanout) from the “ clean -out” dataset was projected out to the sulfur level curve and the differences in emissions at the mileage where the two lines intersect (in

- use equivalent loading) were estimated as the in

- use sulfur level effect. For example, the NO x FTP composite model had a significant sulfur level and mileage interaction term and thus, the sulfur level curves diverge as the vehicle accumulates mileage. The mileage at which the pre

- cleanout level intersects the high sulfur curve was used to estimate the percent reduction in emissions from high to low fuel sulfur levels as illustrated in Figure
7-10

.52 Figure 7-10 Estimating in

- use sulfur effect based on clean

- out (NO x FTP composite)

Table 7-11 summarizes the percent reduction in emissions from 28 ppm to 5 ppm fuel sulfur for all pollutants and all bags. For all models except CO Bag 1, CO Bag1

- Bag3 , CH4 Bag1

- Bag3 , and PM, the reduc tion estimates are statistically significant at α = 0.05; CO Bag 1,
CO Bag1

- Bag3 , and CH4 Bag1

- Bag3 are significant at α = 0.10. The results suggest that significant reduction in emissions can be achieved by reducing the fuel sulfur levels from 28 to 5 ppm in the in- use fleet of Tier 2 vehicles.

Unlike the gaseous pollutants, t here was no effect of sulfur level found for PM. A likely explanation is that the majority of PM as measured in this program (that is, from normal

- emittin g Tier 2 vehicles operated at low and moderate loads) wa s soot produced shortly after cold start (bag 1)

. 25 O nce formed in the combustion chamber, oxidation of soot requires high temperatures, lean air

- fuel conditions , and residence time . Modern gasoline vehicles are calibrated to operate at or very near a stoichiometric fuel/air mixture over all operating modes,
and therefore destruction of soot b y the catalyst is minimal regardless of i t s relative efficiency.53 As a result, sulfur would not be expected to have a significant effect on directly

- emitted PM (other than very small amounts of sulfate). The clean

- out effect on PM was observed in the
init ial portion of the procedures but not later, suggesting again that it was not likely an effect of
sulfur on catalyst efficiency but something else related to release and accumulation of material typically measured as PM

. Since there were no analyses of PM composition in this program, we are not able to draw more definite conclusions here.

Table 7-11 Estimated reduction in in

- use emissions from 28 ppm to 5 ppm fuel sulfur

NO x (p

- value) THC (p

- value) CO (p

- value) NMHC (p

- value) CH4 (p

- value) NO x +NMOG (p

- value) PM ‡
Bag 1 10.7% (0.0033) 8.5% † (0.0382) 7.5% † (0.0552) 7.5% (< 0.0001) 13.9% † (< 0.0001) N/A – Bag 2 59.2% (< 0.0001) 48.8% (< 0.0001) – ‡ 44.8% † (0.0260) 49.9% (< 0.0001) N/A – Bag 3 62.1% (< 0.0001) 40.2% (< 0.0001) 20.1% (< 0.0001) 49.9% (< 0.0001) 29.2% (< 0.0001) N/A – FTP Composite 23.0% † (0.0180) 13.0% † (0.0027) 11.9% † (0.0378) 10.6% † (0.0032) 25.8% † (< 0.0001) 17.3% (0.0140) – Bag 1 – Bag 3 – ‡ 5.2% (0.0063) 4.3% (0.0689) 5.1% (0.0107) 4.6% (0.0514) N/A – † Model with significant sulf ur and mileage interaction term. ‡ Sulfur level not significant at α = 0.10. For THC bag 1 and CH4 bag 1, because the effect of clean

- out was not statistically significant, the reduction estimates are based on the estimates of least squares means.

**7.3.4. Sensitivity Analysis** 

A series of sensitivity analyses of the “ sulfur level ” data was performed to address some of the issues that might affect the mixed model results. They include the impacts of: low concentration measurements, censoring of measurements with zero values, and influential
vehicles. The sensitivity analyses were conducted only for Bag 2 NO x , since above mentioned issues pertain the most to Bag 2 NO x . For example, Bag 2 NO x showed a higher percentage of measurements with zero values than most other pollutant and bag combinations, as illustrated in
Table 7-1

.54 Effect of l ow concentration measurements

The issue of measurements with very low concentration from B ag 2 NO x has been
discussed in 7.1.1 . To address uncertainty of measurements from these very low

- emitting vehicles, we performed sensitivity analyses using two measurement concentration screening
levels: 100 ppb (based on the lower end of the instrument manufacturer’s stated calibration range for the emission analyzer), and 50 ppb (chosen at half the former limit). In each analysis, vehicles with all sample measurements falling below the screening level were removed, and models were refit. Results of these sensitivity analyses are provided in Table 7-12

.

Table 7-12 Results of sensitivity modeling analysis ( NO x Bag 2)

Model Description Vehicles Observations Model Estimate of Bag 2 NO x Reduction Final NO x bag 2 model 23 228 59.2% 50 ppb vehicle screen 17 174 60.5% 100 ppb vehicle screen 11 120 70.2%

In each of these sensitivity analyses, the sulfur level effect remained highly significant with p

- value < 0.0001 , suggesting a meaningful sulfur effect exists regardless of removal of the
lowest

- emitting vehicles. Thus, we conclude that the sulfur effect is considerably larger than the uncertainty or error associated with the measurements.

Effect of use of imputed values

In order to assess the impact of replacing censored values, models with and without imputed values for Bag 2 NO x were compared in Table 7-13

. For the model without imputed values, the mixed model was re

- fit using a new dataset with all imputed values removed, consisting only of actual measureme nts. Based on the examination of the estimates of fixed effects and the standard errors from both models, we concluded that the imputed values did not significantly bias the results. The percent reduction in emissions from 28 ppm to 5 ppm fuel
sulfur lev el was changed from 59.2% in model with imputed values to 57.8% in model without
them.55 Table 7-13 Impact of imputed values on final model (NO x Bag 2)

Effect of influential vehicles

As an additional test of robustness, we also looked at the impact of removing the
influential vehicles from the dataset. Influence can be broadly defined as the ability of a single or multiple vehicles to affect the resulting outcome through the presence or absence in the data. The influential vehicles can be identified by examining the restricted likelihood distance (RLD)
, which is calculated after an iterative process of refitting the model with and without each vehicle.

Figure 7-11 shows the restricted likelihood distance from the influential diagnostics where vehicle IDs 0007, 0046, and 0178 can be considered influential vehicles affecting both the fixed effects and covariance parameter estimates based on Cook’s D and COVRATIO estimates. Although we do not have specific grounds for excluding these vehicles from the mixed model analysis since the measurements from these vehicles did not fall into the category of neither low concentration measurements nor the outlying observations, these influential vehicles were
removed and the model for B ag 2 NO x was refit to examine the impacts from these vehicles. Effect Estimate StdErr DF tValue p - value
Model with imputed values slevel

- 0.8953 0.2040 68

- 4.39 < 0.0001 miles 0.0047 0.0016 167 2.88 0.0045 Model without imputed values slevel

- 0.8618 0.2001 64.1

- 4.31 < 0.0001 miles 0.0046 0.0016 157 2.82 0.005556 Figure 7-11 Influential diagnostics for NO x bag 2

The resulting model showed that the percent reduction in emissions from 28 ppm to 5 ppm was 50.9 percent , compared to the reduction of 59.2 percent from the final model. This analysis demonstrated that even whe n the influential vehicles are removed from the analysis, the reduction in emissions from reducing the fuel sulfur level from 28 ppm to 5 ppm is still highly significant with p

- value < 0.0001.



### 8. **Summary and Conclusions**

This study assessed the emission r eductions expected from in

- use Tier 2 light duty
vehicles with reduction in gasoline sulfur content. The test fleet consisted of light

- duty cars and trucks chosen to be representative of high sales models covering a range of types and sizes. Test
fuels w ere two non

- ethanol gasolines with properties typical of certification fuel, one at a sulfur content of 5 ppm and the other at 28 ppm.57 Using the high

- sulfur test fuel, emissions data were collected from vehicles in their as

received state, and then fol lowing a high

- speed/load “clean

- out” procedure consisting of two
back

- to

- back US06 cycles to examine the existence of reversible sulfur loading in the in

- use fleet. In addition, the differences in the effectiveness of the clean

- out procedure at reducing

e missions between the two fuel sulfur levels were assessed. Lastly, a representative subset of
vehicles performed additional test replicates alternated with mileage accumulation on both high
and low sulfur test fuels. This dataset was used to assess the d ifferences in the rate of sulfur reloading after clean

- out as a function of fuel sulfur level. Major findings from this study
include:

• Reversible sulfur loading is occurring in the in

- use fleet of Tier 2 vehicles and has a
measureable effect on emissions of NO x and other pollutants of interest. For example, by performing a clean

- out cycle on 28 ppm fuel, FTP composite NO x was reduced by 11%, NMHC by 3% . A PM reduction of 14% was found, but it is not clear that it was a sulfur effect given other results in the program.

• The effectiveness of high speed/load procedures in restoring catalyst efficiency is limited when using higher sulfur fuel . Comparing emissions immediately following the clean

out procedure on 5 vs. 28 ppm fuel, FTP composite NO x emissions were 18% lower, NMHC 9% lower, and CO 8% lower.

• Reducing fuel sulfur levels from 28 to 5 ppm is expected to achieve significant reductions
in emissions of NO x , hydrocarbons, and other pollutants of interest in the in

- use fleet

. For example, controlling for sulfur loading effects by mileage where appropriate, FTP composite NO x was 23% lower, NO x +NMOG 17% lower, and CO 12% lower. The
foll owing sensitivity analyses were performed for Bag 2 NOx and suggested the magnitude and statistical s i gnificance of the results are robust:

o With an d without data from very -low- emitting vehicles, which may contain larger
measurement error than data from vehicles with higher emission levels

o With and without inclusion of imputed values used in place of zero measurements

o With and without data from vehicles determined to be most influential on model fitting58 The overall results of this study are in agreement with other studies conducted using low sulfur gasoline in Tier 2 vehicles. The magnitude of NO x and HC reductions found in this study when switching from 28 ppm to 5 ppm fuel are consistent with those found in other studies done by the US EPA and automob ile and catalyst manufacturers. 26
, 27
, 2859 

### 9. **References**

1. The Regulatory Impact Analysis for the Tier2/Sulfur Final Rule, Chapter III and Appendix B, EPA 420-R-99-023.

2. The Regulatory Impact Analysis for the Control of Hazardous Air Pollutants from Mobile Sources Final Rule, Chapter 6, EPA 420-R-07-002.

3. Hochhauser A.M. (2008). Review of Prior Studies of Fuel Effects on Vehicle Emissions
, Report number E-84. Coordinating Research Council, Alpharetta, GA. Available at www.crcao.org.
4. Heck R., Farrauto R.J., Gulati, S.T. (2009). Catalytic Air Pollution Control: Commercial Technology (3 rd Ed.). John Wiley & Sons: Hoboken, NJ.

5. Eastwood P. (2000) Critical Topics in Exhaust Gas Aftertreatment (Engineering Design). Research Studies Press Ltd: Hertfordshire, England.

6. The Regulatory Impact Analysis for the Tier2/Sulfur Final Rule, Chapter III and Appendix B, EPA 420-R-99-023.

7. Durb in T.D., et al. (2003). The Effect of Fuel Sulfur on NH3 and Other Emissions from 2000-2001 Model Year Vehicles, Appendix C. Report number E-60. Coordinating Research Council, Alpharetta, GA. Available at www.crcao.org.

8. Ball D., Clark D., Moser D. (2011). Effects of Fuel Sulfur on FTP NOx Emissions from a PZEV 4 Cylinder Application. SAE 2011 World Congress Paper 2011-01-0300. SAE International: Warrendale, PA.

9. Durbin T.D., et al

. (2003). The Effect of Fuel Sulfur on NH3 and Other Emissions from 2000-2001 Model Year Vehicles, Report number E-60. Coordinating Research Council, Alpharetta, GA. Available at www.crcao.org.

10. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2nd Ed.). Hillsdale, NJ: Lawrence Erlbaum.

11. Engels, J.M.; Diehr, P. (2003). Imputation of missing longitudinal data: a comparison of methods. Journal of Clinical Epidemiology, 56 (2003), 968–976.

12. Downey R.G., & King C.V. (1998). Missing data in Likert ratings: a comparison of replacement methods. J Gen Psychol. 125:175–191.

13. Stephens, R.D. (1994). Remote sensing data and a potential model of vehicle exhaust emissions. Journal of the Air Waste Management Association, 44:1284-1292.60 14. Holmen, B.A., & Niemeier, D.A. (1998). Characterizing the effects of driver variability on real -world vehicle emissions. Transportation Research Part D: Transport and Environment , 3(2), 117-128.

15. Beydoun, M., & Guldmann, J. (2006). Vehicle characteristics and emissions: Logit and regression analyses of I/M data from Massachusetts, Maryland, and Illinois. Transportation Research Part D: Transport and Environment, 11(1), 59-76.

16. Diggle P.J. (1988). An approach to the analysis of repeated measures. Biometrics, 44:959-971.

17. Wolfinger, R.D. (1993). Covariance structure selection in general mixed models. Communications in Statistics, Simulation and Computation, 22(4):1079-1106.

18. Keselman, H.J., Algina, J., Kowalchuk, R.K., & Wolfinger, R.D. (1999). A comparison of recent approaches to the analysis of repeated measurements. British Journal of Mathematical and Statistical Psychology , 52:63-78.

19. Deaton, M.I., & Winebrake, J.J. (2000). The Use of Mixed Effect s ANCOVA to Characterize Vehicle Emissions Profiles. J. Transp. Stat. 3:49-64.

20. Winebrake, J.J., Deaton, M.L., Coburn, T.C., & Kelly, K.J. (2000). Statistical Analysis of Emissions and Deterioration Rates from In-Use, High Mileage CNG and Gasoline Ford Cro wn Victoria Taxicabs ; SAE Technical Paper Series No. 2000-01-1959; Society of Automotive Engineers: Warrendale, PA.

21. Verbeke, G., & Molenberghs, G. (2000). Linear Mixed Models for Longitudinal Data
, New York: Springer

- Verlag.

22. Bozdogan, H. (1987). Model selection and Akaike's information criterion (AIC): the general theory and its analytical extensions. Psychometrika, 52:345-370.

23. Little, R.C., Milliken, G.A., Stroup, W.W. & Wolfinger, R.D. (1996). SAS system for mixed models, Cary, NC: SAS Institute.

24. Wolfinger, R.D. (1993). Covariance structure selection in general mixed models. Communications in Statistics, Simulation and Computation, 22(4):1079-1106.

25. “Analysis of Particulate Matter Emissions from Light

- Duty Gasoline Vehicles in Kansas City” (2008), Chapter 8. EPA420-R-08-010.

26. Regulatory Impact Analysis for the Control of Hazardous Air Pollutants from Mobile Sources Final Rule, Chapter 6. EPA 420-R-07-002.61 27. Ball D., Clark D., Moser D. (2011). Effects of Fuel Sulfur on FTP NOx E missions from a PZEV 4 Cylinder Application. SAE 2011 World Congress Paper 2011

- 01

- 0300. SAE International: Warrendale, PA.

28. Shapiro, E. (2009). National Clean Gasoline, An Investigation of Costs and Benefits. Published by the Alliance of Automobile Manufacturers, Washington, DC.