Document ID: EPA-HQ-OAR-2009-0211-14019
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2010-10-13T04:00Z

Technical Memorandum

From:  Robert Anderson, Environmental Protection Agency

To:  Docket EPA-HQ-OAR-2009-0211

Date: 10/XX/2010

Subject:  Technical Summary of DOE Study on E15 Impacts On Tier 2
Vehicles

and Southwest Research Teardown Report 

The purpose of this technical memorandum is to: (1) outline in more
detail the research study design of the recently completed Department of
Energy (DOE) E0 and E15 catalyst durability test program (DOE V4 program
hereafter); (2) explain in more detail analyses that were used as a
basis for the partial waiver decision evaluating the data obtained from
the DOE V4 program; (3) summarize the results of the engine teardown
report that the Southwest Research Institute (SwRI) completed for DOE on
several vehicles tested in DOE V4 program; and, (4) explain how
non-methane organic gas (NMOG) emissions were estimated for use in the
analyses for the partial waiver decision.  

1 –Study Design – DOE V4 Program

The DOE V4 program was established to investigate the potential impacts
of gasoline-ethanol blended fuels containing greater than 10 volume
percent (vol%) on the durability of vehicle emissions control systems. 
The program began with a subcontracted effort at Southwest Research
Institute (SwRI) in San Antonio, Texas and was later expanded and
accelerated by placing additional subcontracts to perform portions of
the needed work at Transportation Research Center (TRC) in East Liberty,
Ohio, and Environmental Testing Corporation (ETC) in Aurora, Colorado. 
The SwRI and TRC subcontracts and technical work was managed by Oak
Ridge National Laboratory (ORNL), while the ETC subcontract and
technical work was managed by the National Renewable Energy Laboratory
(NREL).  The scope-of-work for each subcontract differed in some details
(such as the number of fuels being evaluated) but were generally the
same in terms of the methodology of the program.

Vehicle Selection and Matching

There were a number of relevant criteria that determined the selection
of vehicle models for the program.  These included whether a vehicle
model was Tier-2 compliant, manufacturer, sales/registration volumes,
and whether a vehicle model did or did not apply learned fuel trim (LFT
or non-LFT, respectively) at wide-open throttle (WOT).  Several studies
provided information that impacted selection:  EPA’s EPACT vehicle
study at SwRI, Coordinating Research Council’s (CRC) E87-1 study, and
DOE’s previous 16 vehicle screening study.  Based on the vehicle set
EPA used to represent the Tier 2 fleet in its EPAct program, DOE
instructed the national laboratories to use similar criteria in
selecting a vehicle set for long-term durability studies.  The make and
model of 18 of the 19 vehicles tested were identical to the makes and
models of the 19-vehicle EPAct test vehicle fleet.  However, at the
request of CRC, DOE substituted the Nissan Quest for a Toyota Sienna. 
DOE also choose to have the model years (MY) of the test fleet vehicles
range from MY2005-MY2009 instead of just using MY2008 vehicles that
EPAct utilized.  Additionally, the sample size and vehicle selection
methodology of this program conforms to EPA guidance necessary to
conduct a long-term durability test program.

Matched sets of vehicles were needed to accomplish the desired testing
program.  The vehicle sets were matched to prevent confounding of the
data by undesirable vehicle attribute changes.  Specifically, the engine
family, engine displacement, evaporative emissions control family, model
year, powertrain control unit calibration, axle ratios, wheel size, and
tire size were constrained to be identical within a vehicle set. 
Physical inspections of the vehicles to eliminate obvious problematic
vehicles (such as those with gross fluid leaks, obvious and excessive
body damage, etc) were also a part of the selection of vehicles for the
matched sets.  Odometer reading was also used to identify candidate
pre-owned vehicles with the goal of restricting the range of odometer
readings within a vehicle set to a maximum of 10,000 miles.  One vehicle
from each set was aged and tested on E0 and E15.



Table   SEQ Table \* ARABIC  1 . Vehicle attribute summary.

The assignment of a particular vehicle to a particular fuel was random
and was accomplished prior to any emissions tests being conducted on the
vehicles.  Table 1 provides information on the test vehicles in the
program.  Obtaining suitable matched sets of vehicles was challenging
for several of the older model year vehicles in the program.  For this
reason, there were a few instances where it was necessary to compromise
the 10,000 mile odometer range target in order to obtain an otherwise
suitably-matched set of vehicles.

Fuels and Blending

The fuels used at the several test sites were obtained commercially by
the subcontractors.  In all cases, for both emissions certification and
aging fuels, the ethanol-containing fuels were splash-blended.

Emissions and related tests were conducted using emissions certification
gasoline (E0) or  E15 blended from emissions certification gasoline and
denatured fuel-grade ethanol.  These batches were subsequently analyzed
to provide the information necessary to conduct the program.  These
analyses included:  the actual ethanol content of the batch; carbon;
hydrogen and oxygen fractions; and, density.  Whenever possible, large
enough batches to complete the several phases of the program were
produced to reduce potential batch-to-batch effects on the results of
the program.

Aging fuels were produced by blending denatured fuel-grade ethanol with
gasoline (E0) commercially available at retail stations, rather than
emissions certification gasoline, in order to control costs.  (Many
thousands of gallons of aging fuel were used in the program.)  Batches
of E15 used for vehicle aging were analyzed for assurance that they
contained the correct amount of ethanol.  These fuels were not, in
general, subjected to the same level of analysis that was required for
the emissions test fuels at these sites.

Emissions Test Protocol

The vehicles were subjected to an initial emissions test (Federal Test
Procedure (FTP)) to assure that they were emissions-compliant before
acceptance into the program.  Once in the program, vehicles were
subjected to emissions and related tests at the beginning of mileage
accumulation, at least one mid-mileage point (two points for vehicles at
ETC), and at the end of mileage accumulation.  At SwRI and TRC, the
acceptance tests also included tests at wide-open throttle to aid in
classifying the vehicles as either LFT or non-LFT vehicles.  At each
emissions test interval, duplicate FTP tests were conducted on each
vehicle using both E15 and E0.  (i.e. the E15 vehicle received duplicate
FTPs on both E15 and E0.)  Wide-open throttle tests were also conducted
at each emissions test interval with the same fuels as the FTP tests. 
The vehicles also underwent compression and leak-down checks on each
engine cylinder at each emissions interval.

For the pre-owned vehicles, the vehicles’ initial mileage was dictated
by market availability and varied considerably, with more recent model
years having a lower mileage at the beginning of the program.  Emissions
test and mileage accumulation intervals are based on the highest
odometer reading in a given vehicle set.  The new cars typically had a
few hundred miles of difference from the highest to lowest odometer
reading in a vehicle set.  However, the used cars had larger
differences, typically several thousand miles.  As an example, a set of
vehicles with mileages of 32,000, 35,000, and 38,000 would experience
end-of-program emissions tests as the highest mileage car reached
120,000 miles.  Thus, the two remaining vehicles in the set would
experience end-of-program emissions tests at lower mileages (114,000 and
117,000 miles respectively.)  Mid-aging test intervals were determined
in the same way.  In this way, the overall mileage accumulation
experienced by the vehicles in a given set was the same regardless of
the fuel being used.  This method was used because it allowed any fuel
effects that were identified to be characterized based on equivalent
mileage accumulation.  This same procedure was used with new cars as
well, though the mileage differences within a vehicle set are slight for
these vehicles compared with those typical of the used car sets.

Tier 2 compliant vehicles were driven up to their statutory full-useful
life (120,000 miles).  The initial mileages of the Tier 2 vehicles were
about 50,000 or lower, meaning that these vehicles were driven
approximately 70,000-120,000 miles during the program.  Emissions
intervals were determined based in part on the initial mileage of the
vehicles. 

New cars were first subjected to aging to an odometer reading of 4,000
miles on retail E0 to stabilize the engine and emissions control
systems, followed by the emissions test for acceptance into the program
and the initial emissions test sequence.  Following these tests, these
cars were subjected to mileage accumulation until approximately 60,000
miles on the odometer, at which point the first mid-aging emissions
tests were conducted.  This cycle was then repeated at approximately
90,000 miles for the vehicles under test at ETC for a second mid-aging
emissions test series.  At TRC and SwRI, the 90,000 mile emissions tests
were not conducted.  End-of-aging tests were conducted for all new
vehicles at odometer readings of 120,000 miles.  

Pre-owned vehicles were similarly subjected to initial acceptance tests,
then initial emissions tests, followed by mileage accumulation.  Vehicle
sets with nominally less than 70,000 miles at the start of the program
were subjected to mid-aging emissions tests at 95,000 miles, followed by
end-of-aging tests at 120,000 miles.  Vehicle sets with nominally more
than 70,000 miles were driven 25,000 miles prior to the mid-aging
emission test and an additional 25,000 miles to the end-of-program
emissions tests.  An exception to this schedule was the Nissan Quest,
for which the mid-aging emissions tests were dropped from the program at
DOE’s direction to accelerate completion of the program for this
vehicle model.

Mileage Accumulation

The standard road cycle (SRC) was used for all aging. The SRC is the
official EPA driving cycle used for aging in the whole vehicle exhaust
durability procedure.  This is a recommended EPA procedure that the
manufacturers regularly use for verifying full useful life emissions
capability.  It has an average speed of 46.3 miles per hour (mph) and a
maximum of 75 mph.  The Nissan Quest aging was changed part way through
aging to a series of steady speed laps on the test track at TRC at the
DOE’s direction to accelerate completion of this vehicle set.

Vehicles under test at TRC were aged using a closed test track.  At this
site, drivers followed the SRC as they drove the vehicles around the
track.  (Signs were installed at various locations around the track to
assist the drivers in following the SRC protocol.)  At ETC and SwRI, the
mileage accumulation was accomplished by using mileage accumulation
dynamometers (MADs).  When a MAD is used, a mechanical actuator is used
to manipulate the accelerator pedal and allow the vehicle to accelerate
and decelerate to match the desired driving profile.  Some mileage
accumulation at ETC was also conducted on emissions certification
dynamometers when sufficient MAD capacity was not available.

During mileage accumulation, the subcontractors followed manufacturer
guidelines for scheduled maintenance on each vehicle.  Oil changes were
scheduled so that they did not occur less than 500 miles prior to any
emissions tests.

2 – DOE V4 Waiver Analysis

Battelle Memorial Institute (Battelle) acted as a data collection,
dissemination, and analysis center for all three test sites under
contract to ORNL.  Battelle provided the data that were used in our
analyses.  These data included the emissions test results broken down by
bag and calculated composite values (computed according to 40 CFR
86.144-94).  These data included CO, NOx, and NMOG results; however, the
NMOG values used for these analyses are estimated NMOG results that were
necessary due to limitations of measurement equipment at two of the
three labs.  Part 4 of this technical memo has an in-depth discussion
about the background and method used for the estimated NMOG
calculations.  

Table 1 shows average FTP composite emissions results for NOx, CO, and
estimated NMOG for E0 and E15 vehicles tested on the fuel the vehicle
would be aged on (i.e. the vehicle to be aged on E15’s test results on
E15) at the beginning of testing.  Table 2 shows the average FTP
composite emissions results for the end of program testing (i.e. full
useful life mileage accumulation testing).

Table 1.  Start of Testing Average FTP Composite Emissions Test Results.

	CO	Nox	Est. NMOG

Vehicles 	E0	E15	E0	E15	E0	E15

Accord	0.22	0.12	0.017	0.014	0.031	0.018

Altima	0.46	0.38	0.030	0.023	0.066	0.050

Caliber	1.02	0.66	0.035	0.030	0.054	0.046

Camry	0.15	0.15	0.019	0.039	0.019	0.032

Caravan	1.61	1.16	0.052	0.024	0.044	0.041

Civic	0.52	0.23	0.007	0.024	0.026	0.019

Cobalt	0.62	0.49	0.035	0.027	0.039	0.041

Corolla	0.42	0.33	0.021	0.019	0.034	0.032

Explorer	1.06	1.00	0.010	0.009	0.051	0.051

F150	1.42	1.42	0.038	0.030	0.046	0.065

Focus	0.56	0.41	0.007	0.006	0.042	0.024

Impala	1.00	0.91	0.037	0.023	0.044	0.038

Liberty	0.86	0.78	0.011	0.007	0.049	0.053

Odyssey	0.23	0.18	0.009	0.010	0.021	0.025

Outlook	0.25	0.27	0.011	0.010	0.026	0.030

Quest	 	0.70	 	0.023	 	0.074

Silverado	0.60	0.76	0.027	0.027	0.044	0.044

Taurus	0.55	0.30	0.003	0.002	0.026	0.016

Tundra	0.69	0.88	0.020	0.029	0.048	0.072

Average	0.68	0.59	0.021	0.020	0.040	0.041

*Note: The E0 Nissan Quest data are omitted since the E0 vehicle failed
to complete the test program

**Note: E0 and E15 results are on different vehicles, so emission
differences are due to both fuel and vehicle differences.

Table 2.  End of testing Average FTP Composite Emissions Test Results

	CO	Nox	Est. NMOG

Vehicle	E0	E15	E0	E15	E0	E15

Accord	0.23	0.13	0.026	0.024	0.048	0.018

Altima	3.18	0.62	0.067	0.053	0.135	0.055

Caliber	2.60	3.61	0.078	0.059	0.083	0.077

Camry	0.29	0.25	0.053	0.052	0.032	0.035

Caravan	1.67	1.12	0.051	0.047	0.048	0.038

Civic	0.57	0.33	0.027	0.043	0.040	0.033

Cobalt	0.87	0.47	0.042	0.027	0.039	0.039

Corolla	0.55	0.57	0.061	0.047	0.048	0.053

Explorer	1.23	1.04	0.033	0.028	0.060	0.059

F150	3.25	2.23	0.078	0.060	0.077	0.090

Focus	1.10	0.67	0.113	0.062	0.032	0.028

Impala	1.13	1.44	0.038	0.039	0.040	0.047

Liberty	0.99	1.16	0.052	0.045	0.045	0.045

Odyssey	0.60	0.20	0.074	0.039	0.051	0.028

Outlook	1.34	0.43	0.047	0.016	0.049	0.034

Quest	 	1.02	 	0.040	 	0.069

Silverado	0.54	0.76	0.032	0.035	0.048	0.056

Taurus	0.53	0.45	0.010	0.013	0.029	0.028

Tundra	0.80	0.94	0.042	0.035	0.052	0.055

Average	1.19	0.92	0.051	0.040	0.053	0.047

*Note: The E0 Nissan Quest data are omitted since the E0 vehicle failed
to complete the test program

**Note: E0 and E15 results are on different vehicles, so emission
differences are due to both fuel and vehicle differences.

The first question EPA attempts to answer when evaluating fuel waivers
is whether the new fuel or fuel additive cause a vehicle to exceed full
useful life emissions standards.  With the data collected from Battelle
for all Tier 2 motor vehicles, the Agency compared the average composite
emission levels for each vehicle tested on E0 and E15 to the applicable
Tier 2 full useful life exhaust emissions standard.  Based on historical
analyses used to evaluate previous waivers, we treated these data as
binomial data on whether a vehicle had passed or failed applicable Tier
2 full useful life standards.  Vehicles that had emissions level greater
than the applicable Tier 2 full useful life exhaust emissions standard
were treated as a failure.  We then conducted several analyses that are
broken down into two distant types:  (1) analyses that compare the
pass/fail performance of the E0 and E15 vehicle test groups and (2)
analyses that compared the groups independently to an “anticipated
failure rate.” 

In the past, the Agency has generally performed analyses for fuel or
fuel additive waiver requests evaluating two questions.  The second
question EPA looks at to evaluate a fuel waiver is whether the new fuel
or fuel additive cause an adverse effect on the emissions of regulated
pollutants that might contribute to vehicles exceeding their standards
over their full useful life?  For this we assessed the rate of emission
deterioration of these vehicles operating on E15 in comparison to those
operating on E10 to determine any statistically significant differences.

Further elaboration of statistical methods and results are discussed
below.

Pass/Fail Analyses Comparing E0 and E15

	We conducted two kinds of statistical analyses that either compared the
pass/fail performance of the E15 vehicle test group to E0 vehicle test
group or that compared the E15 and E0 test group separately to an
anticipated failure rate.  Table 2 shows how the Tier 2 vehicles that
were aged on E0 performed when tested after FUL mileage accumulation
relative to the applicable Tier 2 exhaust emissions standard while Table
3 shows how the Tier 2 vehicles performed on E15.  

Table 2. E0 FUL Results Compared to Tier 2 Standards

Year	Model	LFT@WOT	Nox	NMOG	CO

2007	Accord	N	Pass	Pass	Pass

2006	Silverado	Y	Pass	Pass	Pass

2008	Altima	N	Pass	Fail	Pass

2008	Taurus	Y	Pass	Pass	Pass

2007	Caravan	N	Pass	Pass	Pass

2006	Cobalt	N	Pass	Pass	Pass

2007	Caliber	N	Fail	Pass	Pass

2009	Civic	N	Pass	Pass	Pass

2009	Explorer	Y	Pass	Pass	Pass

2009	Corolla	Y	Pass	Pass	Pass

2009	Liberty	N	Pass	Pass	Pass

2005	Tundra	Y	Pass	Pass	Pass

2006	Impala	Y	Pass	Pass	Pass

2005	F150	Y	Pass	Pass	Pass

2006	Quest	N	N/A	N/A	N/A

2009	Outlook	Y	Pass	Pass	Pass

2009	Camry	Y	Pass	Pass	Pass

2009	Focus	Y	Fail	Pass	Pass

2009	Odyssey	N	Pass*	Pass	Pass

*Denotes that average of emissions tests were below applicable FUL
standard, but had at least one test value above the applicable FUL
standard.

Table 3. E15 FUL Results Compared to Tier 2 Standards

Year	Model	LFT@WOT	Nox	NMOG	CO

2007	Accord	N	Pass	Pass	Pass

2006	Silverado	Y	Pass	Pass	Pass

2008	Altima	N	Pass	Pass	Pass

2008	Taurus	Y	Pass	Pass	Pass

2007	Caravan	N	Pass	Pass	Pass

2006	Cobalt	N	Pass	Pass	Pass

2007	Caliber	N	Pass	Pass	Pass

2009	Civic	N	Pass	Pass	Pass

2009	Explorer	Y	Pass	Pass	Pass

2009	Corolla	Y	Pass	Pass	Pass

2009	Liberty	N	Pass	Pass	Pass

2005	Tundra	Y	Pass	Pass	Pass

2006	Impala	Y	Pass	Pass	Pass

2005	F150	Y	Pass	Pass	Pass

2006	Quest	N	Pass	Pass	Pass

2009	Outlook	Y	Pass	Pass	Pass

2009	Camry	Y	Pass	Pass	Pass

2009	Focus	Y	Fail	Pass	Pass

2009	Odyssey	N	Pass	Pass	Pass

	First, the Agency compared the two groups using Fisher’s Exact Test. 
Fisher's exact test is a statistical test used to determine if there are
nonrandom associations between two categorical variables like the
pass/fail binomial data across two test groups that we are looking at in
this program.  Table 4 is a 2 X 2 contingency table tabulating the
results of this analysis test program for NOx.  Since the E15 vehicle
test group performed similarly to the E0 test group, we cannot reject
the hypothesis that the two vehicle test groups had different results. 
Since E15 performed better for CO and estimated NMOG, the results for
those pollutants are similar for NOx.  It should also be noted that
since the test subjects in this test are assumed to be independent, the
Nissan Quest that completed testing on E15 was included in the analysis.
 However, its inclusion does not impact the results.  Ultimately,
concerning direct comparisons of the pass/fail performance of the E0 and
E15 vehicle groups, it is unlikely that any statistical analysis
comparing the proportion of vehicles that failed in each group would
show that E15 performed worse than E0.  This is due to the fact that
either fewer vehicles failed for a particular pollutant (e.g. NMOG and
NOx) on E15 compared to E0 or that the two groups performed identically
(e.g. CO).

Table 4. Fisher's Exact Probability Test for NOx

	 	Pass	Fail	     Total

E0	16	2	18

E15	18	1	19

Total	35	2	37

p=	0.374

The Agency also decided to compare the failure rate of the E15 group
independently to an assumed “anticipated failure rate.” Vehicle
testing experience has shown that various factors, such as testing
variability and differences in vehicle performance across models lead to
a small number of anticipated emissions failures across the national
fleet.  For purposes of assessment in this waiver, we determined our
anticipated failure rate experienced in the In-use Verification Program
(IUVP).  We use this anticipated failure rate to conduct a binomial test
to evaluate the hypothesis of whether E15 had a higher failure rate than
the anticipated failure rate of 5% for each pollutant with a 90%
confidence level.  The binomial test is an exact test of the statistical
significance of deviations from a theoretically expected distribution of
observations into two categories (in this case pass/fail).   If E15 had
no effect on the proportion of vehicles expected to fail, the number of
vehicles should be approximately 5%.  In this case, the low number of
vehicles aged on E15 that failed to meet the emissions standards at the
end of the test program indicates that we cannot reject the hypothesis
that the rate of failure experienced in the test program was greater
than 5% for NOx, CO, and estimated NMOG (p=0.245 for NOx, p=0.623 for
estimated NMOG, and p=0.623 for CO).  Therefore, we cannot conclude that
E15 caused a significant number of vehicles to fail meet their emissions
standards when compared to the failure rate experienced as part of the
IUVP test program.

Emission Deterioration with E0 and E15 blends

 

	We performed a statistical analysis of the data from the DOE catalyst
durability test program in order to assess whether the emission
deterioration rates were any different between the matched pairs of
vehicle aged to full useful life mileage on E0 and E15 respectively. 
Since 1) not all vehicles started the test program with the same
mileage, and 2) deterioration over time, particularly at low mileage, is
extremely nonlinear, it was necessary to normalize the data by excluding
mileage points less than nominally 10,000 miles.  Since the E0 Nissan
Quest was not able to complete the program, the E15 Nissan Quest could
not be included in this deterioration analysis. 

	We chose a statistical model in the SPSS™ suite of statistical
software for this analysis which had the following characteristics:

Utilized the natural logarithm of emissions,

Was linear in nature, 

Allowed each individual vehicle to have its own base emission level, 

Allowed each matched set of vehicles (i.e., the two F150’s) to have a
unique rate of emissions deterioration aside from an effect of ethanol
blend percentage, and

Allowed a single offset to the emission deterioration rate of all the
vehicles to represent the impact of E15, in order to determine the
average impact of E15 on emission deterioration.

The decision to analyze the natural logarithm of emissions, as opposed
to emissions, was due to the range of base emissions observed across the
test vehicles.  NOx emissions at the lowest mileages tested varied by a
factor of twelve, while NMOG emissions varied by a factor of four.  An
analysis of deterioration of absolute emissions in terms of g/mi would
have placed great weight on those vehicles with the highest base
emissions and essentially ignored those with the lowest emission levels.
 In contrast, an analysis of the natural logarithm of emissions
essentially measures deterioration in terms of the percent of base
emissions.  Thus, each vehicle is provided roughly equal weight in this
type of analysis.  This approach is also consistent with those used in
EPA’s analyses of emission data in its development of both the Complex
Model and the EPA Predictive Models.  

	The decision to utilize a linear model was necessary due to the fact
that we only had emission data at two mileage points for a number of the
test vehicles.  Also, extensive experience with emission deterioration
has demonstrated that such deterioration tends to be linear in nature
once the catalyst is past its “green” stage and in the absence of
some type of component failure.  As will be seen below, it is difficult
to discern a statistically significant difference between a simple,
linear deterioration rate.  Discerning statistically significant
difference between non-linear rates would be even more difficult.

	Vehicles of different model types obviously have different emission
levels, as described above.  However, even within a matched pair of
vehicles of the same model type, base emissions vary for two reasons. 
One, no two vehicles’ components are exactly the same and
manufacturing variability exists, as well.  Also, the pre-owned vehicles
which were part of the test program were driven differently prior to
their purchase for the program.  Thus, the degree of emission
deterioration upon entrance into the test program were likely different.
 For example, eight of the E0-aged vehicles had lower base NMOG
emissions than its matching E15-aged vehicle, while eight had higher
base NMOG emissions.

Two, the other reason that the emissions of matched vehicles differ is
the immediate impact of ethanol blend percentage on emissions, any
effect on emission deterioration aside.  These two effects (vehicle to
vehicle and fuel) cannot be separated in this type of test program,
since different vehicles had to be used for aging on the different
ethanol blends.  Thus, deterioration in emissions must be determined
primarily using each individual vehicle’s data (i.e., relative to each
vehicle’s baseline emission level).  

We took a slightly different approach towards emission deterioration
rate.  In our statistical model, we allowed each group of identical
vehicle models, but not each individual vehicle, to have its own rate of
emissions deterioration.   Like base emissions, individual vehicle’s
emission deterioration rates can differ.  They will differ more across
vehicle type and model than they will between ostensibly identical
vehicles (i.e., those of the same make and model).  (In this data set,
average deterioration rates can vary by an order of magnitude between
different model types.)  Still, the emissions from even identical
vehicles will tend to deteriorate at different rates.  However, we could
not allow each vehicle in the modeling to have its own deterioration
rate, as we would then have no comparison left upon which to base the
effect of ethanol blend content on deterioration.   Therefore, we
essentially assumed that any differences in the underlying emission
deterioration rate between vehicles within a model type would average
out across the test program.  The fact that all vehicles were aged using
the same durability driving cycles during the program supports this
assumption.

The net result was a statistical model with the following variables:

A coefficient for each vehicle (treating the two vehicles of the same
model type as individual vehicles with independent coefficients)

A coefficient for each model type by the number of miles it had been
aged on either E0 or E15 (this mileage was almost always identical for
the two blends) 

A coefficient for the number of miles any vehicle had been aged on E15

This last coefficient was not vehicle specific and therefore represents
the overall impact of E15 on emission deterioration relative to E0. 
Tables 2, 3, and 4 present the specific coefficients of the linear
models developed for the natural logarithm of NOx, NMOG and CO
emissions, respectively.

Table 2.  Model of NOx Emission Deterioration

	Base Emission Level (ln(emissions(g/mi))	Model Type Deterioration Rate
(change in ln(emissions(g/mi)) per 1000 miles

Vehicle Model	EO	E15

	2007 Honda Accord	-3.978	-4.108	0.003273

2008 Nissan Altima	-3.409	-3.69	0.006937

2007 Dodge Caliber	-3.314	-3.674	0.009208

2009 Toyota Camry	-3.769	-3.882	0.007135

2007 Dodge Caravan	-3.228	-3.631	0.004621

2009 Honda Civic	-3.915	-3.707	0.003623

2006 Chevrolet Cobalt	-3.248	-3.827	0.004818

2009 Toyota Corolla	-3.068	-3.402	0.001796

2009 Ford Explorer	-3.540	-3.948	0.001839

2005 Ford F150	-3.452	-3.66	0.01195

2009 Ford Focus	-3.320	-3.836	0.008952

2006 Chevrolet Impala	-3.061	-3.399	-0.00308

2009 Jeep Liberty	-3.943	-4.241	0.008561

2009 Honda Odyssey	-3.306	-4.049	0.006325

2009 Saturn Outlook	-3.749	-4.732	0.005077

2006 Chevrolet  Silverado	-3.566	-3.631	0.003667

2008 Ford Taurus	-5.994	-5.776	0.01569

2005 Toyota Tundra	-3.685	-3.704	0.006862

	Change in Deterioration Rate due to E15 aging

0.00125

Table 3.  Model of NMOG Emission Deterioration

	Base Emission Level (ln(emissions(g/mi))	Model Type Deterioration Rate
(change in ln(emissions(g/mi)) per 1000 miles

Vehicle Model	EO	E15

	2007 Honda Accord	-3.348	-3.861	0.002422

2008 Nissan Altima	-2.660	-3.019	0.003436

2007 Dodge Caliber	-2.946	-2.992	0.007783

2009 Toyota Camry	-3.508	-3.398	0.000817

2007 Dodge Caravan	-3.110	-3.233	0.001839

2009 Honda Civic	-3.451	-3.604	0.001919

2006 Chevrolet Cobalt	-3.340	-3.166	0.000669

2009 Toyota Corolla	-2.837	-2.548	-0.00172

2009 Ford Explorer	-3.106	-2.979	0.002885

2005 Ford F150	-3.004	-2.799	0.007143

2009 Ford Focus	-3.510	-3.305	-0.00129

2006 Chevrolet Impala	-3.229	-3.194	0.000973

2009 Jeep Liberty	-2.851	-2.533	-0.00299

2009 Honda Odyssey	-3.478	-3.85	0.0034

2009 Saturn Outlook	-3.115	-3.332	0.000658

2006 Chevrolet  Silverado	0.082	-3.058	0.003436

2008 Ford Taurus	-3.825	-3.996	0.002629

2005 Toyota Tundra	-3.095	-2.898	0.002404

	Change in Deterioration Rate due to E15 aging

-0.00114

Table 4.  Model of CO Emission Deterioration

	Base Emission Level (ln(emissions(g/mi))	Model Type Deterioration Rate
(change in ln(emissions(g/mi)) per 1000 miles

Vehicle Model	EO	E15

	2007 Honda Accord	-3.377	-3.99	0.002451

2008 Nissan Altima	-2.689	-3.163	0.003453

2007 Dodge Caliber	-2.979	-3.133	0.007827

2009 Toyota Camry	-3.531	-3.539	0.00077

2007 Dodge Caravan	-3.140	-3.373	0.001829

2009 Honda Civic	-3.471	-3.741	0.001832

2006 Chevrolet Cobalt	-3.367	-3.303	0.000645

2009 Toyota Corolla	-2.868	-2.69	-0.00175

2009 Ford Explorer	-3.126	-3.114	0.002829

2005 Ford F150	-3.032	-2.937	0.007082

2009 Ford Focus	-3.545	-3.456	-0.00117

2006 Chevrolet Impala	-3.256	-3.337	0.000987

2009 Jeep Liberty	-2.879	-2.667	-0.00306

2009 Honda Odyssey	-3.511	-4.001	0.003464

2009 Saturn Outlook	-3.147	-3.481	0.000698

2006 Chevrolet  Silverado	0.081	-3.197	0.00343

2008 Ford Taurus	-3.856	-4.12	0.002521

2005 Toyota Tundra	-3.123	-3.036	0.002376

	Change in Deterioration Rate due to E15 aging

-0.00109

 

Table 5 summarizes the average emission deterioration rates on the E0
and E15 blends.  The average emission deterioration rate for the E0
blend is the average of the vehicle specific deterioration rates shown
in Tables 2, 3, and 4 above, respectively, for NOx, NMOG and CO
emissions.  The average emission deterioration rate for the E15 blend is
the average deterioration rate for the E0 blend plus the change in
deterioration rate to due to E15 aging shown in these same tables.

Table 2.  Emission Deterioration on E0 and E15 Blends (ln(emissions per
1000 miles)

	NOx	NMOG	CO

E0 deterioration rate	0.0060	0.0020	0.0020

E15 deterioration rate	0.0072	0.0009	0.0009

The results of the statistical analysis shown in Table 5 indicate that
the rate of deterioration in NMOG and CO emissions decreased slightly on
average, while that for NOx emissions increased slightly.  However, none
of the three impacts were statistically significant deterioration at the
90% confidence level.   Thus, due to the variability in the effect
across the various test vehicles, we cannot confidently reject the
hypothesis that the emission deterioration rates on both blends are the
same.  In other words, there is a significant chance that the average
impacts observed are the result of the randomness in the data.  This
conclusion is supported by the fact that the average changes in NMOG/CO
and NOx emission deterioration rates went in opposite directions.  If
the catalysts had in fact been deteriorating faster with E15, then all
emissions should have deteriorated consistently.  Therefore, the
catalyst durability test program results also support the conclusion
that E15 will not contribute to Tier 2 vehicles exceeding their emission
standards over their full useful life.

3 – Analysis of SwRI Powertrain Component Inspection Data 

Background

Southwest Research Institute released a report , (SwRI®) Project
08-58845 Status Report, 

"Powertrain Component Inspection from Mid-Level Blends Vehicle Aging
Study”,  that details the results of their Teardown measurements of
the engines from the vehicles that they aged as part of the DOE Catalyst
Durability Program. 

Summary

Six pairs of engines and fuel system components were disassembled and
analyzed for signs of wear and material compatibility problems after E0
and E15 mileage accumulation.   Vehicles were matched pairs of used
vehicles from MY2006 to MY2008 with mileage ranging from 10k to 50k
miles prior to the program.   Miles of test program aging ranged from
68K to 102K miles depending on the initial mileage. 

The following powertrain components were analyzed:

evaporative system for leaks

evaporative canister working storage capacity

intake valve deposits

valves and camshafts for wear

fuel pump flow performance

Fuel injector flow rate analysis

Engine oil chemical analysis for wear particles

Intake valve seals were visually inspected.

Making any conclusive statements about the inspection data is limited by
the lack of baseline, “before”, measurements prior to the aging
program’s start, and the aging process itself.   So it is not the most
rigorous for engine components and materials.  Furthermore, the engines
were run near continuously with few starts or soaks.  Nevertheless, It
would only be noteworthy if significant component degradation did occur.
 

Nonetheless, looking at the following observations, the measurement
results show no systemic trends pointing to greater wear or degradation
caused by fuel-related differences.  The one very consistent trend was a
greater buildup of valve deposits with E15 use –possibly due to
detergent dilution when displacing 15% of the gasoline with Ethanol.   

Types of analysis performed

Evaporative Emissions System Integrity Check—Evaporative  system leak
checked using a pressure test and smoke.  All vehicles passed the tests.
 

Evaporative Canister Butane Working Capacity– Measure each
canister’s working storage capacity (maximum amount of fuel vapor that
can be absorbed).  With no measurements taken before aging,  we can only
make relative comparisons between the canisters from E0 aged vehicles to
canisters from matching  E15 aged vehicles.   When comparing the 6
Canisters aged on E15 to the matching 6 aged on E0, there was no trend
indicating any difference between them.   The variations in results was
uniformly distributed.  No statistically significant change in
evaporative canister working capacity is suggested with this small data
set.

Intake Valve Deposits -  Quantified in milligrams by weighing the valves
before and after cleaning them.  All of the E15 engines have a
consistent and often very significant increase in intake valve deposits.
 41 of the 44 (95%) intake valves from E15 aged engines had more
deposits than the valves from  the E0 aged engines.  77% of the E15
valves had more than  twice the deposits as their corresponding E0
valves. This could be the result of having 15% less detergent in the
fuel as the Ethanol displaced the detergent-enriched gasoline. 

Valvetrain / Camshaft Measurements--  The following measurements (cam
lobe, valve seat and valve stem height) are intended to check if there
is any noticeable increase in wear on the camshafts and valve seats due
to the E15 fuel usage.  Only limited observations are possible from the
data since there were no baseline  “before” measurements prior to
the aging program’s start.  Nonetheless, looking at the observations
following below, the measurement results show no systemic trends
pointing to greater wear caused by the E15 exposure.    

Cam Lobe Wear- The overall cam lobe height was measured for any
indications of increased wear with E15 fuel compared to the E0 aging. 
As stated in the SWRI report, “Because the cams were not measured at
the beginning of the program, and due to normal part-to-part variation,
the measurements are potentially inconclusive with respect to any
fuel-related differences in cam wear”.   With no “before”
measurements, there are no actual wear numbers to compare. Nonetheless,
by comparing the final lobe dimensions of the 12 pairs of camshafts
(intake plus exhaust) there are two observations.   

There is no trend showing smaller E15 lobes.  Looking at each matched
pair, only 2 of the 12 E15 aged camshafts have smaller dimensions than
the corresponding E0 cam.  

The typical differences were less than 0.02mm (0.0008”).  This is very
minimal and may be due more to the variation of the initial cam grinding
process than wear.   

Valve Seat Width and Valve Surface Contour – 3 different dimensions
were measured on the valve seat surface for indications of valve seat
recession.  Without initial measurements, results aren’t conclusive
since we don’t know how much wear was due to the aging program versus
prior aging on unknown fuels.

  

With that limitation in mind and focusing the peak wear depth on the
valve seat contact point (the report labeled this the “Z” term): 

Comparing the matched sets of valves, all E15 engines showed similar or
less wear than their corresponding E0 valves, except for the Silverado.

Intake valve seat wear for both E0 and E15 aged engines was 1 to 8
microns or, more importantly,  less than 0.0001”.   

Exhaust valve seat wear was typically 8 to 25 microns with the Silverado
approaching 30 microns (0.0012”).   It is normal for Exhaust valves to
wear more than intake valves. 

 Such low wear values indicate none of the valves experienced
significant wear on this the particular aging cycle.   Field vehicles
with high mileage can have  more than ten times this amount of wear. 

Valve Stem Height  - valve stem height increase can be an indicator of
valve seat recession.  Unfortunately, it is variable with casting and
machining variances.  

As stated in the SWRI report: “Because valve stem heights at the
beginning of the program are not known, and because of variations in
actual cylinder head geometry and valve length, it is difficult to draw
any conclusions about valve seat wear.”

There is no consistent difference between the E0 and E15 engines when
comparing valve heights between corresponding pairs of engines. 

The exception is the Accord exhaust valves which are uniformly taller on
the E0 engine.  This is likely a manufacturing difference.   

Several E15 engines had 1 valve noticeably taller than the rest. There
is no way to determine if this is manufacturing variability or valve
recession. 

ASTM D5185 Analyses of Engine Oil Drain Samples –

The difference in metal levels between the E0 and E15 related oil
samples do not show any trends indicating that E15 increases internal
engine wear relative to E0 fuel.

As stated in the SWRI report: “There do not appear to be any unusually
high wear metal levels in any of the engine oil drains.”  

Fuel Pump Evaluations - Fuel pump flow rate was measured at pressures
above and below the mean expected pressure specified by the
manufacturerSome pumps exhibited different flow rates but without
measurements before the start of aging,  we can not distinguish between
part-to-part variation and deterioration. None of the flow rate curves
looked significantly different between the two aging fuels in the
matched pairs.

As stated in the SWRI report: “While pump characteristics prior to the
vehicle aging program are unknown, there do not appear to be any serious
differences between the RE0 fuel pumps and the RE15 fuel pumps.”  

Fuel Injector flow rate analysis-  SwRI flow tested the injectors at 3
duty cycle settings.   They concluded all injectors fall within 3% of
the mean for each engine application.   It appears these were static
flow tests which are less sensitive to pintle characteristics.  Dynamic
testing would be more appropriate for detecting any pintle mechanical
degradation.   While the test results indicate no adverse effects
related to ethanol exposure, a more appropriate measurement would allow
a definitive conclusion.  

4 – Estimated NMOG Calculation

The term “non-methane organic gases” (NMOG) describes the sum of
hydrocarbon emissions other than methane that may or may not contain
oxygen, a set that is inclusive of many compounds that are both directly
harmful to health as well as contribute to smog and ozone pollution. 
Oxygen-containing species (referred to as oxygenated hydrocarbons, or
OHCs) are typically a very small fraction of NMOG emissions for
non-oxygenated fuels, but can become a significant portion for a
gasoline-ethanol blended fuel.  As such, NMOG is the basis for Tier 2
emissions standards, and also for evaluating emissions related to the
waiver decision.

Generation of NMOG results from an emission test is complicated by the
fact that the type of hydrocarbon analyzer typically used in emission
test cells produces an incomplete account of OHCs.  Thus, the OHCs must
be measured by other methods and combined with the non-methane
hydrocarbon (NMHC or NONMHC) to properly account for all NMOG emissions.

During the planning phase of the program, EPA and DOE staff agreed that
NMOG emissions would be computed as NONMHC plus at least methanol,
ethanol, formaldehyde, and acetaldehyde.  Other alcohol and/or carbonyl
species may be included, but typically don’t change the result
significantly since they are emitted in much smaller amounts than these.
 

As described in Section 1 above, DOE used three commercial emission
testing companies to produce emission results for this waiver: Southwest
Research Institute (SwRI), Transportation Research Corporation (TRC),
and Environmental Testing Corporation (ETC).  Between the three labs,
there were differences in both how OHCs were measured and how NMOG was
calculated, complicating the task of aggregating all results into a
consistent form necessary for evaluation of the data.  Of the three
labs, SwRI measured the required OHCs for all tests, in a way consistent
with typical best practices, and computed NMOG accordingly.  ETC did not
produce useable OHC results for all compounds for all tests, and thus
were not able to compute NMOG as defined for all tests due to the level
of uncertainty in the emission measurement.  TRC utilized an approach
similar to ETC at the start of testing, but then switched part-way
through the test program to measure OHC emissions similar to SwRI.

Due to the inconsistencies in data, DOE (in agreement with EPA)
performed calculations on the contractor-reported values to produce NMOG
estimates for all tests on a consistent basis.  The first step was to
produce NONMHC from test-cell-reported values (that contained incomplete
accounting for oxygenated species) for all 3 laboratories that were all
measured on a consistent basis.  Attached to this memo is a document
entitled “Derivation of Mass-Correction Method for NMOG Determination,
August 20, 2010” that describes this calculation in detail.

The second step was to adjust the NONMHC results to reflect the presence
of OHCs in NMOG emissions.  Examining data from SwRI, the NMOG to NONMHC
ratio was found to have a linear correlation with fuel ethanol content. 
A linear fit equation was generated for this ratio as a function of fuel
ethanol content for composite (FTP weighted) emission results.  This
equation is given here, solved for NMOG, where ETOH is defined as the
fuel ethanol content in volume percent:

 

To avoid inconsistent treatment of vehicles and fuels between the three
labs, this equation was used to generate estimated NMOG results for all
tests in the program.  This approach is sufficiently accurate to address
the question of whether a vehicle’s emissions deterioration is worse
with E15, since this question involves looking at rates of change, and
tests at all mileage points are consistently treated by the ratio.  We
believe it is also sufficiently accurate to estimate actual NMOG
measurements, as the difference between estimated and actual NMOG
measurements, where data was available, is very small.

 EPA selected vehicles for its EPAct study based on several criteria. 
The Agency selected vehicles based on MY engine family sales data since
there are usually multiple models to choose from for each engine family.
 Additionally, EPA focused on high volume sellers since, by definition,
high volume sellers are representative and eases recruitment.

 See EPA Docket #EPA-HQ-OAR-2009-0211-2559.2, API Technology Committee
Meeting, Chicago, 6/4/08.

 The number of vehicles in a matched set varied during the test program
according to the number of fuels being targeted for test.  In some cases
four ethanol blend levels were tested (E0, E10, E15, and E20), while in
other cases a subset of these fuels were tested.  Since the waiver
request is for E15, this analysis focuses on those vehicles that were
aged on E15 compared to those vehicles that were aged on E0.

 The average differences in these results are not statistically
significant (directional paired t-test, 90% confidence level or Sign
Test) for NOx and estimated NMOG.  CO results were statistically
significantly lower (p=.018) for E15 when compared to E0 at the 90%
confidence level for the paired t-test.  It should be noted that these
analyses do not take into account the differences that arise from
vehicle-to-vehicle variation within a specific vehicle model.  These
analyses were not conducted for FUL test data since aging effects may
impact the results.

 These values assumed the applicable rounding procedures for
certification as outlined in 40 CFR 86.1838-01.

 In previous waivers, the Agency has conducted analyses that compared
the failure rates of test vehicle fleets to an expected failure rate.  

 For a more formal discussion of Fischer’s Exact Test see S. Siegel,
Nonparametric Statistics, New York: McGraw-Hill, 1956: 96.

 With sufficient sample sizes, analyses using more continuous
distributions may be used.  However, given the low sample size in this
case, it is reasonable to use Fisher’s exact test in lieu of those
methods.  Additionally, given the similar performance of the E0 and E15
vehicle test groups, it is unlikely that any such methods would
demonstrate the E15 group performed worse than the E0 group for Tier 2
vehicles when looking strictly at whether vehicles passed or failed FUL
certification standards.

 For a more formal discussion of the binomial test see M. Hollander and
D. Wolfe, Nonparametric Statistical Methods, 2nd, New York:
Wiley-Interscience, 1999: 20.

 Using the binomial test for 19 vehicles compared to an expected
probability of 0.05, two vehicles would have had to have failed for E15
to fail this test.

 The instantaneous exhaust emissions impact of E15 compared to E0 is
discussed in great detail in section IV.? of the waiver document.

 The Agency has typically used a confidence level of 90% in CAA section
211(f)(4) waiver requests instead of the more conventional 95%
confidence level.  We feel that the 90% confidence level increases the
likelihood that increases in deterioration would be statistically
significant and therefore would be more conservative in this case. 
However, these differences are also not statistically significant at the
95% confidence level.

Draft – Deliberative – Internal Use Only		Page   PAGE  1  of  
NUMPAGES   19