Sierra Instruments Model BG - 3 vs. CVS Multiple Engine Correlation Study Forward: In 2006, Sierra Instruments Inc. recognized that demonstration of correlation of the Model BG
- 3 transient partial flow sampling system to full

- flow CVS systems for particul ate matter (PM) sampling and gravimetric measurement, per Parts 86, 1065, et al., was a necessary step toward acceptance of the unit as a certification

- grade device . With input from technical partner Caterpillar Inc., a series of correlation tests were de fined. In all, 120 pairs of simultaneously obtained test data were taken at West Virginia University (WVU) and at Caterpi llar Inc i n Part 1065
- compliant CVS

- equipped test cells
. The testing was conducted between March, 2008 and April, 2009. The engines and after

- treatment systems used are shown below in Table 1. Testing was performed on both on

- highway and non

- road engines on the heavy

- duty FTP or non

- road tr ansient cycle (NRTC) as required for certification of the engine types tested. In order to prov ide a representative mix of EGR and non

- EGR equipped engines from various manufacturers, engines from John Deere, Cummins and Caterpillar were tested, producing cycle PM levels ranging from approximately 0.5 to 100 milligrams/hp

- hr. Manufacturer Model Test Lab Use Cycle PM Limit Emissions abatement Cummins ISX 565 WVU On
- road FTP 2004 EGR Deere 6081 WVU Off

- road NRTC Tier IV CCRT (added for test) Cat C
- 11 Cat On
- road FTP 2007 CCRT Cat C
- 9 Cat Off

- road NRTC Tier IV CCRT (added for test) Cat C
- 9 Cat Off

- road NRTC N/A CCRC w/ Bypass (added for test) Cat C
- 13 Cat On
- Road FTP 2007 Production DPF Table 1 , Test Plan Summary for BG

- 3

– CVS Correlation Study The Deere engine was a Tier II engine equip ped with CCRT to achieve Tier IV NRTC cycle PM li mits . The Caterpillar C

- 11 was a MY2004 engine equipped with a CCRT and which is currently the heavy duty diesel industry round

- robin test engine. The Caterpillar C
- 9 engine is used as a mule by Caterpillar for test measurement method development at thei r engine development center . The C

- 9 wa s also equipped with CCRT technology and included an engineered bypass for a portion of the tests. The bypass was added specifically to evaluate the correlation of the BG

- 3 to the CVS at emissions levels approaching the 2007/2010 on

- highway engine PM emissions limits. The testing included a 10x repetit ion of the heavy duty diesel engine transient cycle certification procedure. O ne cold and one hot test cycle were obtained per day for a minimum of 10 cold

- hot repeat s per engine. These results were weighted as per 40CFRPart 86 requirements for engine certification (1/7*Cold Result, g/hp

- hr + 6/7*Hot Result, g/hp

- hr = Weighted Cycle Emissions. g/hp

- hr). Paired

- t and F

- test statistical analysis was subsequently perfo rmed on the weighted test results to determine acceptability at 90% confidence level per Part 1065.12. Maintaining consistency between BG

- 3 and CVS specific to critical sampling system setup parameters such as filter face velocity, dilution ratio, residen ce time, filter temperature and dilution air temperature was actively pursued to maintain the technical integrity of the correlation study. All aforementioned parameters were maintained within 40CFR Part 1065

- required limits. Correlation Test ing Results The initial phase of testing was performed at the West Virginia University Center for Alternate Fuels, Engines and Emissions, Morgantown, West Virginia (WVU). Graphs # 1 and # 2, below, denote cold

- hot cycle

- weighted results on the Cummins and Deere engine s, respectively for run days 1

- 10 . It should be noted that post
- test analysis of the Cummins engine day #5 gaseous emissions results showed significant within

- day increases in the hot test cycle hydrocarbon (+26.2%) and carbon monoxide (+3.7%) emissions ( directional indicators for PM), relative to results from Day 4 and Day 6, that were reflected in the BG

- 3 results (+12.7%) but not from the CVS (+1.37%)
. A weighing error specific to the CVS filter is suspected. This analysis q uestions only the CVS resu lt from Day 5 of this series. The graph below includes the Day 5 results for completeness. All engines in this program were able to complete their prescribed test cycles in accordance with the speed and torque regression statistics required by Part 1065 and Part 86. BG
- 3 Standard Error of Estimate (%SEE) results were well below the 3.5% limit required by Part 1065 for all runs at both WVU and at Caterpillar
. BG-3 vs CVS Cold-Hot Results, Cummins Engine, FTP Cycle

0 20 40 60 80 100 120

1 2 3 4 5 6 7 8 9 10 Run Set #PM, mg/hp-hr CVS, mg/hp-hr BG-3, mg/hp-hr

Graph 1
- BG
- 3 vs. CVS Corre lation Results, Cummins ISX 565 On
- Highway Engine, WVU DataBG-3 vs CVS Cold-Hot Results, Deere Tier II Engine w/ DPF, NRTC Cycle

0 0.5

1 1.5

2 2.5

3

1 2 3 4 5 6 7 8 9 10 Run Pair #PM, mg/hp-hr CVS, mg/hp-hr BG-3, mg/hp-hr

Grap h 2
- BG
- 3 vs. CVS Correlation Results, Deere 8.1L Engine w/ CCRT , WVU Data The second phase of correlation testing was performed at the Caterpillar Technical Center, Mossville, Illinois, in a CVS test cell with a fully 1065 compliant particulate sampling system. The C

- 11 industry (EPA

- EMA) round robin engine with CCRT was at Caterpillar for inter

- lab comparison testing at that time. It was considered an excellent candidate for BG

- 3
- CVS correlation testing due to the number of labs having already tested t he engine, including EPA. Graph 3 shows the cold

- hot weighted results from the C
- 11 tests at Caterpillar. The variability in results shown in Graphs 2, 3 , 4 and 6 are dominated by uncertainties in the filter weighing process , coupled with engine and CCRT
- based day

- to
- day variability. This level of variability is an expected outcome when measuring PM , using approved gravimetric methods , from engine s operating at between 5 and 20 % of the 2007 PM standard . There were no emissions system or engine setup cha nges made during the course of the test and test filters were re

- weighed to verify PM mass results. Table 2 compares BG
- 3 and Caterpillar CVS data against results obtained by participants in the EPA

- EMA round

- robin inter

- lab comparison program. This cha rt includes the coefficient of variance (1std dev/mean) relative to the 2007/2010 particulate sta ndard. This comparison underscored the correlation for both sample mean and variance values from the Caterpillar CVS and BG
- 3 versus a broader population of t est labs from testing a common engine and after

- treatment package.BG-3 vs. CVS, Cold-Hot Weighted Results, C-11 R-R Engine, FTP Cycle

0.0 0.5 1.0 1.5 2.0 2.5 3.0

1 2 3 4 5 6 7 8 9 10 Run #PM, mg/hp-hr CVS, mg/hp-hr BG-3, mg/hp-hr

Graph 3
- BG
- 3 vs. CVS Correlation Results, C

- 11 Engine w/ CCRT, Caterpillar Data

Lab Cold/Hot PM, g/hp hr COV Expressed as % of 2010 PM limit A .0012 7.0 B .0007 1.0 C .0010 3.0 D* .003 10.13 A2** .0031 20.2 E .0007 4.49 F .0007 2.08 Cat CVS .0010 4.58 BG
- 3 .00115 7.66
*

- Lab used incorrect filter media **Reran using denormalized FTP test cycle; labs listed after lab A2 ran the denormalized cycle Table 2
- Comparison of BG

- 3 and Caterpillar CVS mean and % coefficient of variance (COV) results with test labs participating in the C

- 11 round

- robin engine study.BG-3 vs. CVS Cold-Hot Weighted Results, Tier III C-9 w/ DPF

0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 1.80 2.00

1 2 3 4 5 6 7 8 9 10 Run #PM, mg/hp-hr CVS, mg/hp-hr BG-3, mg/hp-hr

Graph 4
- BG
- 3 v s. CVS Correlation Results, C

- 9 Engine w/ CCRT, Caterpillar Data During testin g, questions regard ing the BG

- 3 to CVS correlation at PM levels approaching the 2007/2010 standard were raised, essentially due to the disparity in PM levels generated by the selected engines relative to both the on

- highway 2007/2010 and non - road Tier IV emissions standards
. The results found on Gr aph 5 were obtained during BG

- 3 to CVS correlation testing from a C
- 9 with an engineered 17.25mm fixed bypass installed around the CCRT, selected to yield an emissions level of approximately 85% of the 2007/2010 standard. The engi ne was operated on the NRTC for this correlation exercise. Comparing the paired results from this test sequence show ed that 9 out of 10 cold

- hot BG

- 3 results are within less than 10% of the corresponding CVS result . Reviewing this same paired dataset usi ng a tighter comparison standard, 8 out of 10 of the cold

- hot BG

- 3 results were withi n 6% of their CVS cold

- hot cohort s . This agreement underscores the correlation integrity across the anticipated range of PM levels to be certified (or re

- certified) using Part 86 or Part 1065 methods on NRTC or FTP cycles. Graph 6 summarized cold

- hot results obtained on the 2008 Caterpillar C

- 13 engine. As previously stated, elevated levels of variability from both measurement systems are expected when sampling particula te matter at 5% of the 2007/2010 PM emissions limit. In the case of the C

- 13, net filter loadings of less than 5 micrograms were routinely measured from both the CVS and BG

- 3.BG-3 vs. CVS Cold-Hot Weighted Results, C-9 with Bypassed DPF, NRTC

0.00 2.00 4.00 6.00 8.00 10.00 12.00

1 2 3 4 5 6 7 8 9 10 Run NumberPM, mg/hp-hr CVS, mg/hp-hr BG-3, mg/hp-hr

Graph 5
- BG
- 3 v s. CVS Correlation Results, C

- 9 Engine w/ partially bypassed CCRT, Caterpillar Data
BG-3 vs. CVS Cold-Hot Weighted Results, C-13 Engine, FTP Cycle

0.00 0.20 0.40 0.60 0.80 1.00 1.20

1 2 3 4 5 6 7 8 9 10 Run #PM, mg/hp-hr CVS, mg/hp-hr BG-3, mg/hp-hr

Graph 6
- BG
- 3 v s. CVS Correlation Results, 2008 C

- 13 Engine, Caterpillar DataIn conclusion, based on test results from this initiative, the Sierra BG

- 3 passes paired

- t and F

- test results for equivalent means and equivalent var iances, respectively for cold

- hot cycle w eighted transient test results per Part 1065.12, on engines ranging from approximately 1/20 to 10x the applicable limit for 2007 on

- road and Tier IV off

- road engine certifications. Further, based on test results fr om an engine with a CCRT and engineered partial bypass, the Sierra BG

- 3 passes paired

- t and F

- test results for equivalent means and equivalent variances, respectively for cold

- hot cycle weighted transient test results per Part 1065.12 at PM emissions level s of approximately 85% of the applicable limit for 2010 on

- road, and approximately 60% of the Tier IV off

- road standard.