# PM

- PEMS Measurement Allowance Determination

Draft Final Report

Prepared by:

Imad A. Khalek and T h o m as L. Bougher Robert L. Mason and Janet P. Buckingham Southwest Research Institute

May 17, 2010

Prepared for:
U.S. Environmental Protection Agency California Air Resources Board Engine Manufacturers Associationii SwRI Project No. 14936.12 FORWARD

Th e PM

- PEMS measurement allowance program was performed by the Department of Emissions Research and Development under Mr. Jeff White, Director. Dr. Imad Khalek, Program Manager, was the Principal Investigator and Project Manager
, Mr. T h om as Bougher, Research Engineer, was the Project Leader , and Mr. Daniel Preece, Research Assistant, was the laboratory technical assistant. Dr. Bob Mason, Institute Analyst , was the Principal Co

Investigator responsible for statistical analysi s, and Ms. Janet Buckingham, Staff Analyst, was the Project Leader/Statistics. Other SwRI Emissions R&D staff with contribution to the project

were Mr. Michael Feist, Senior Research En gineer, Mr. Richard Mechler , Senior Research Technologist
, Mr. Don Parker
, Senior Technician, Mr. Jose Sosa, Principal Technician, Mr. Keith Echtle , Laboratory Assistant Manager , and Mr. Ernie Krueger, Laboratory Manager. Additional SwRI assistance during Environmental Testing was provided by Rick Pit man, Senior Engineering Technologist, Mr. Mike Negrete, Senior Technician, Mr. David Smith, Staff Technician, Mr. Herbert Walker, Senior Engineering Technologist, and Mr. Eric Dornes, Principal Engineer.

This w ork was performed under E nvironmental Protection Agency (EPA) Work Assignments 2

- 7, 3

- 7, 4

- 7, and 2

- 12, under SwRI Project Numbers 12859.07, 13749.07, 14658.07, and 14936.12, respectively. The original EPA Work Assignment Manager was Mr. Matt Spears. After March of 2009, the EPA Work Assignment Manager became Mr. Chris Laroo. This work started in June of 2007 and will end by June 26, 2010. Testing started in June of 2008 and ended in September of 2009.

Fundi ng for this work was provided by U.S EPA
, Engine Manufacturers Association (EMA), and the California Air Resources Board (CARB). Funding by EMA and CARB was provided directly to EPA in support of this work

.
In

- kind engine and technical support were provided by Volvo Powertrain . In

- kind PM

- PEMS and techn ical support were provided by AVL, Horiba, and Sensors. SwRI acknowledges the following individuals for their laboratory technical and logistical support:
Mr. Jeff Saxon and Mr. Steve Trevitz, from Volvo Powertrain
Mr. Scott Porter, Dr. Mike Akard and Dr. Qiang Wei, from Horiba
Mr. Bill Silvis, Mr. Siegfried Roeck, Dr. Wolfgang Schindler, Dr. Roland Wanker, Dr. Michael Arndt, Mr. P.J. Pankratz, and Ms. Sarah Kingham, from AVL
Mr. Andrew Reading, Dr. David Booker, Dr. Atul Shah, Mr. Carl Ensfield, Mr. Tim Bo ttomley, and Mr. Kevin Bouma, from Sensorsiii SwRI Project No. 14936.12 A measurement allowance steering committee (SC) composed of EMA, EPA, CARB, and PM

PEMS manufacturer members met on a regular basis throughout the entire project to discuss the progress made and make recommendati ons . The SC has contributed significantly to this project. SwRI acknowledges the following SC members for their active participation : Mr. Rey Agama, Caterpillar Dr. Mike Akard, Horiba Dr. Dipak Bishnu, CARB Dr. David Booker, Sensors Dr. Bruce Cantrell, EPA Consultant Mr. Tim French, EMA Dr. Bob Giannelli, EPA Mr. Denny Hao, PACCAR Dr. Kent Johnson, CE

- CERT Mr. Craig Kazmierczak , Detroit Diesel Mr. John Kegebein, John Deere Mr. Tom Kramer, Navistar Mr. Chris Laroo, EPA Mr. Hector Maldonado, CARB Mr. Bill Martin, Cummins Dr. Shirish Shimpi, Cummins Mr. Bill Silvis, AVL Ms. Carole Smith, Isuzu Mr. Matt Spears, EPA Mr. Steve Trevitz, Volvo Powertrain Dr. Qiang Wei, Horibaiv SwRI Project No. 14936.12 EXECUTIVE SUMMARY

The United States Environmental Protection Agency (EPA), Engine Manufacturers Association (EMA) and California Air Resources Board (CARB) agreed to pursue an experimental data driven program to establish a measurement allowance (MA) for in

- use particulate matter (PM) testi ng using PM portable emissions measurement systems ( PM

- PEMS). The MA is a brake

specific PM emission s error associated with using in

- use PM

- PEMS equipment compared to the laboratory reference filter method

. If the MA error is a positive value above zero, a fter EPA

rou nding method, it will increase the EPA in

- use not

- to

- exceed (NTE) standard by the rounded value. If the error is negative or zero, it will not contribute to any changes to the in

- use standard.

The measurement allowanc e steering committee (SC) accepted the following PM

- PEMS to be part of the MA program:

**•** Sensors Particle Pro portional Micro Diluter (PPMD). A PM

- PEMS that uses proportional dilution and a series of 8 quartz crystal microbalances to measure total (solid plus volatile) PM. The total P M is measured as a single flow

- weighted value for a NTE event.

**•** Horiba Transient Particulate Matter (TRPM) . A PM

- PEMS that uses proportional dilution, a real time electrical aerosol detector, and an integrated filter sample to report total PM. The total PM is measured as a single flow

- weighted value for a NTE event. This instrument can report real time total PM, but was not used as such on this program.

**•** AVL mic ro

- soot sensor (MSS). A n instrument that uses constant dilution and a photo acoustic detector to m easure soot or the eleme ntal carbon portion of PM. S oot is measured in real time during a NTE event.

The SC agreed that only the PPMD by Sensors and TRPM by Horiba will be used for the official determination and validation of the measurement allowance generated because both are designated as complete PEMS and both measure total (solid plus volatile) PM , as required by US EPA to be valid PM

- PEMS. The SC also agreed that only the PM

- PEMS that produces the lowest positive 95 th

**•** Laboratory steady

- state (SS) and transient engine NTE testing using PM

- PEMS and CVS filter measurement during SS testing only. The SS was used to capture bias, compared to the CVS, and the transient was used to capture precision only since there is no reference method to measure PM during a short NTE event. A 2007 heavy

- duty percentile measurement allowa nce , based on Sensors or Horiba only
, will be chosen for in

- use validation by CE

- CERT due to funding limitation . The third instrument, the MSS by AVL, was not a complete PM

- PEMS and it was always used in conjunction with the PPMD on this program, based on an agreement reached between Sensors and AVL. The SC agreed that the MSS will not be con sidered as an option for official measurement allowance determination, unless both the PPMD and the TRPM fa iled validation.

The PM

- PEMS

- MA project included four main el ements:v SwRI Project No. 14936.12 diesel Mack MP7, by Volvo Powertrain, was used to condu ct the engine experiments. The engine was configured with a bypass around the diesel particle filter (DPF) to provide PM concentration levels similar to those expected at the NTE threshold limits between 0.02 g/hp

- hr and 0.03 g/hp

- hr.

**•** Environmental testin g such as the effect of shock and vibration, pressure, temperature and relative humidity, and electric noise on PM

- PEMS precision.

**•** Monte Carlo simulation to determine a brake

- specific measurement allowance value using error surfaces generated from the PM

- PEMS laboratory and environmental testing, and from error surfaces generated during the gaseous PEMS program

o A total of 141 Reference NTEs were provided by EMA as an input to the model for calculating ideal brake specific emissions, prior to the Monte Carl o simulation

**•** Model validation using data generated from in

- use PM

- PEMS testing by CE

- CERT

Three methods are used to determine in

- use NTE brake

- specific PM emissions:

**•** Method 1 f( PPPP � � � � � , torque i , speed i , exhaust

- flow i

**•** Method 2 f( PPPP � � � � � , torque )

i , speed i , exha ust

- and fuel ECM, G - flow i

**•** Method 3 f(PM )

i , torque i , speed i , and fuel ECM , G - flow i

All methods require engine control module (ECM) broadcasted torque and speed. In addition, Method 1 requires measured exhaust flow but not fuel flow; Method 2 requires measured exhaust flow, ECM broadcasted fuel flow, and gas

- based fuel flow; Method 3 is similar to Method 2, but it does not require exhaust flow. Besides real time PM measurement, a ll methods can use a single flow

- weighted PM measurement for a NTE , except Method 3, where real time PM measurement is required. Thus, Method 1 and Method 2 were applied to all three PM

- PEMS, but Method 3 was only applied to the AVL MSS.
Compared to Horiba TRPM, the Sensors PPMD, as shown in Table 1, produced t he lowest positive 95 )

th percentile measurement allowance of 0.0061 g/hp

- hr for a NTE threshold level of 0.02 g/hp

- hr, using calculation Method 2. The Horiba TRPM produced a measurement allowance value of 0.0100 g/hp

- hr. Thus, the PPMD was selected by the SC for in

- use tes ting by CE

- CERT for Monte Carl o model validation

. Although not accepted as a PM

- PEMS, the AVL MSS produced the lowest measurement allowance of essentially zero. T he SC agreed to include the AVL MSS during in

- use validation because i t was used in conjunction with the PPMD during the labora tory portion of the testing.vi SwRI Project No. 14936.12

TABLE 1 . MEASRUEMENT ALLOWA NCE BASED ON 0.02 G/ HP

- HR THRESHOLD
Measurement Errors at NTE Threshold, g/hp

- hr

PEMS Method 1 Method 2 Method 3

AVL 0.0001

- 0.0005

- 0.0005

Horiba 0.0109 0.0100 n/a

Sensors 0.0069 0.0061 n/a

Fi gures 1 and 2 show examples of validation plots for Sensors PPMD using Method 1 and AVL MSS using Method 3. The dots on each of the two figures represent the in

- use delta BSPM between PEMS and CE

- CERT on the y

- axis versus CE

- CERT BSPM on the x

- axis (Ideal BSPM) . Similarly, the lines represents the 5 th and 95 th percentile errors produced by Monte

Carlo simulation based on laboratory testing versus a reference NTE BSPM value on the x

- axis (Ideal BSPM)

. To pass model validation, ≤ 10% of the in

- use NTE delta brake

- specific PM (BSPM) between the PEMS and the C E

- CERT (dots on Figures 1 and 2) must reside outside the 95 th percentile and 5 th percentile lines.

The Sensors PPMD fai led validation using Method 1 and Method 2 , as shown in Table 2. For Method 1, 32 percent of the data were below the 5 th percentile, and for Method 2, 34 percent of the data were below the 5 th percentile. The failure was mainly due to negative bias. The e xact cause for negative bias is unknown, but could be related to special environmental conditions experienced during in

- use testing that the Sensors PPMD did not account for properly.
The AVL MSS passed validation using Method 3, as shown in Table 3, where less than 10 percent of the data were outside the 5 th and 95 th percentiles , with the majority of the failed points being above the 95 th percentile. Method 2 failed validation b y one percentage point, and Method 1 failed validation by having18 percent of the data outside the 5 th and 95 th percentiles, with the majority of failed p oints being above the 95 th percentile.vii SwRI Project No. 14936.12 FIGURE 1 . IN

- USE VALIDATION PLOT FOR SENSORS PPMD USI NG METHOD 1. (Y AXIS IS THE DIFFE RENCE BETWEEN PEMS A ND IDEAL BSPM. IDEAL BSPM IS THE LABORATORY CV S BSPM FOR THE 5 TH AND 95 TH AND THE CE

- CERT BSPM FOR THE DOTS)viii SwRI Project No. 14936.12 FIGURE 2 . IN

- USE VALIDATION PLOT FOR AVL MSS USING ME THOD 3. (Y AXIS IS THE DIFFERENCE BE TWEEN PEMS AND IDEAL BSPM. IDEAL BSPM IS THE LABORATORY CVS BSPM FOR THE 5 TH AND 95 TH AND THE CE

- CERT BSPM FOR THE DOTS)ix SwRI Project No. 14936.12 TABLE 2 . SENSORS PPMD VALIDATION RESULTS

Method 1 Method 2

Total # CE

- CERT Points 217 217

# CE

- CERT Points within Ideal BSPM Range 210 210

# CE

- CERT Points above 95th Percentile 0 1

# CE

- CERT Points below 5th Percentile 68 70

# CE

- CERT Points between 5th and 95th Percentiles 142 139

% CE

- CERT points that did not validate 32.38 33.81

TABLE 3 . AVL MSS VALIDATION RESULTS

Method 1 Method 2 Method 3

Total # CE

- CERT Points 271 271 271

# CE

- CERT Points within Ideal BSPM Range 263 263 263

# CE

- CERT Points above 95th Percentile 47 28 23

# CE

- CERT Points below 5th Percentile 2 2 3

# CE

- CERT Points between 5th and 95th Percentiles 222 233 237

% CE

- CERT points that did not validate 18.08 11.41 9.89x SwRI Project No. 14936.12 The measurement allowan ce program was concluded by the SC because one of the instrument s , namely the AVL MSS, passed its own validation criteria using Method 3. However , the MSS as used in this program is not accepted as a PM

- PEMS by EPA , and the measurement allowance generated based on the performance of this instrument is not an official par t of the measurement allowance.

The final official measurement allowance accepted by the SC was based on the Sensors PPMD. Further investigation of why the PPMD did not validate the model i s being investigated outside this pr ogram, and the details are expected to be part of the CE

- CERT Fi nal Report on in

- use validation testing. As for the status of the Horiba T RPM, it is understood that EPA may consider approving this system, although it did not go through validation. As for the status of the AVL MSS, it is understood that EPA may consider approving this system along with a required filter box to collect total PM to span the real time soot signal produced by the photo acoustic sensor.xi SwRI Project No. 14936.12xii SwRI Project No. 14936.12 Table of Contents

Forward

................................ ................................ ................................ ................................ ........... ii Executive summary

................................ ................................ ................................ ........................ iv Table of Conte nts

................................ ................................ ................................ ........................ xii List of Figures

................................ ................................ ................................ ............................... xv List of Tables

................................ ................................ ................................ ............................... xxi 1 Introduction

................................ ................................ ................................ ......................... 1 2 Modeling Approac h

................................ ................................ ................................ ............ 2 2.1 Purpose of Model

................................ ................................ ................................ .................. 2 2.2 Model Improvement

................................ ................................ ................................ .............. 3 2.3 Monte Carlo Simulation Approach

................................ ................................ ....................... 4 2.4 Calculation Methods

................................ ................................ ................................ ............. 4 2.4.1 Calculation Method 1

– “Exhaust Flow

- Torque

- Speed” Method

................................ .... 5 2.4.2 Calculation Method 2

– “Exhaust and Fuel Flow

- Torque

- Speed” Method

..................... 6 2.4.3 Calculation Method 3

– “Fuel Flow

- Torque

- Speed” Method

................................ .......... 7 2.5 Reference NTE Events

................................ ................................ ................................ .......... 8 2.6 Error Surface Generation

................................ ................................ ................................ .... 14 2.7 Error Surface Sampling and Interpolation

................................ ................................ .......... 22 2.8 Brake

- Specific Emissions Calculations

................................ ................................ .............. 25 2.9 Convergence and Number of Trials

................................ ................................ .................... 28 2.10 Simulation Output

................................ ................................ ................................ .......... 29 2.11 Step

- by

- Step Simulation Example

................................ ................................ ................. 29 2.12 Measurement Allowance Generation

................................ ................................ ............. 33 2.12.1 Regression Method

................................ ................................ ................................ ......... 33 2.12.2 Median Method

................................ ................................ ................................ .............. 34 2.13 Model Validation

................................ ................................ ................................ ............ 35 3 Part 1065 PEMS and Laboratory Audit

................................ ................................ ............ 39 3.1 1065 PEMS Audit

................................ ................................ ................................ ............... 39 3.1.1 Horiba Flow Audits

................................ ................................ ................................ ........ 40 3.1.2 Exhaust Flow

................................ ................................ ................................ .................. 45 4 Engine Dynamometer Testing and Results

................................ ................................ ....... 47 4.1 Testing Objective

................................ ................................ ................................ ................ 47 4.2 Experimental Setup

................................ ................................ ................................ ............. 48 4.2.1 Engine and Sampling System

................................ ................................ ......................... 48 4.2.2 Sensors PPMD

................................ ................................ ................................ ............... 52 4.2.3 AVL MSS

................................ ................................ ................................ ....................... 53 4.2.4 Horiba TRPM

................................ ................................ ................................ ................. 55 4.3 Bypass Mixing Verification

................................ ................................ ................................ 56 4.4 PEMS Loss Corrections

................................ ................................ ................................ ...... 59 4.4.1 Sensors PPMD Loss Correction

................................ ................................ ..................... 59 4.4.2 AVL MSS Loss Correction

................................ ................................ ............................ 59 4.4.3 AVL MSS Total PM Correction

................................ ................................ .................... 60 4.5 Steady State Testing Procedure

................................ ................................ .......................... 60 4.6 Data Yield During Steady State Testing

................................ ................................ ............. 69 4.6.1 Data Yield During Steady State Testing

................................ ................................ ........ 70 4.7 Accounting for CVS Variability During Steady State Testing

................................ ........... 73xiii SwRI Project No. 14936.12 4.8 Steady State Testing Results

................................ ................................ ............................... 75 4.9 Steady State PM Error Surfaces

................................ ................................ .......................... 87 4.10 Transient Engine Results

................................ ................................ ................................ 92 4.11 CE

- CERT Mobile Lab Correlation

................................ ................................ .............. 104 4.12 Investigation of DPF Regeneration

................................ ................................ .............. 107 4.13 Investigation of Storage and Release

................................ ................................ ........... 109 4.14 Engine Manufacturers Torque and Fuel Error Surfaces

................................ ............... 112 5 Environmental Testing and Results

................................ ................................ ................ 114 5.1 Reference Measurement Testing

................................ ................................ ....................... 114 5.2 Pressure Chamber Testing

................................ ................................ ................................ 120 5.3 Temperature & Humidity Chamber Testing

................................ ................................ ..... 129 5.4 Electromagnetic and Radio Frequency Interference Screening

................................ ........ 138 5.4.1 Bulk Current Injection

................................ ................................ ................................ .. 140 5. 4.2 Radiated Immunity

................................ ................................ ................................ ....... 147 5.4.3 Electrostatic Discharge

................................ ................................ ................................ . 150 5.4.4 Conducted Transients

................................ ................................ ................................ ... 151 5.5 Vibration Testing

................................ ................................ ................................ .............. 153 6 Modeling results

................................ ................................ ................................ .............. 162 6.1 Convergence Results from MC Runs

................................ ................................ ............... 163 6.2 Sensitivity Based on Bias and Variance

................................ ................................ ........... 168 6.3 Validation Results

................................ ................................ ................................ ............. 183 6.4 Meas urement Error Allowance Results

................................ ................................ ............ 194 7 Summary

................................ ................................ ................................ ......................... 215 7.1 Monte Carlo Model

................................ .............................. Error! Bookmark not defined. 7.2 Engine Experiments and Results

................................ ......... Error! Bookmark not defined. 7.2.1 Steady

- State

................................ ................................ ..... Error! Bookmark not defined. 7.2.2 Transient

................................ ................................ .......... Error! Bookmark not defined. 7.2.3 CE

- CERT and SwRI Correlation

................................ .... Error! Bookmark not defined. 8 References

................................ ................................ ................................ ....................... 215 Appendix A: PEMS O peration Log

................................ ................................ ............................ 217 Appendix B: crystal ball output file descriptions

................................ ................................ ....... 229 Appendix C: Monte Carlo Spreadsheet Computations

................................ ............................... 241 Appendix D: Steering Committee Meeting Minutes

................................ ................................ .. 263xiv SwRI Project No. 14936.12xv SwRI Project No. 14936.12 LIST OF FIGURES

FIGURE 1. IN

- USE VALIDATION PLOT FOR SENSORS PPMD USING M ETHOD 1. (Y AXIS IS THE DIFFERENCE BETWE EN PEMS AND IDEAL BSPM. IDEAL BSPM IS THE LA BORATORY CVS BSPM FO R THE 5 TH AND 95 TH AND THE CE

CERT BSPM FOR THE DO TS)

................................ ................................ ................................ .............................. VII FIGURE 2. IN

- USE VALIDATION PLOT FOR AVL MSS USING ME THOD 3. (Y AXIS IS T HE DIFFERENCE BETWEE N PEMS AND IDEAL BSPM. IDEA L BSPM IS THE LABORA TORY CVS BSPM FOR THE 5 TH AND 95 TH AND THE CE

- CERT BSPM FOR THE DOTS)

................................ ................................ ................................ ................................ ...... VIII FIGURE 3. METHOD 1 IDEAL BSPM VALUES FO R REFERENCE NTE EVEN TS

................................ ................................ 11 FIGURE 4. ERROR SUR FACE CONSTRUCTION: PEMS VS. LABORATORY RESULTS

................................ ...................... 17 FIGURE 5. ERROR SUR FACE CONSTRUCTION: (PEMS

- LAB) VS. LABORATORY RESULTS

................................ ........... 18 FIGURE 6. ERROR SUR FACE CONSTRUCTION: ERROR AT VARIABILITY INDEX FOR 5 TH AND 95 TH PERCENTILES VS. LABORATORY RESULTS

................................ ................................ ................................ ................................ ....... 19 FIGURE 7. TRUNCATED NORMAL DISTRIBUTION PERCENTILES

................................ ................................ .................. 20 FIGURE 8. ERROR SUR FACE CONSTRUCTION: E RROR AT VARIABIL ITY INDEX FOR 1ST AN D 99TH PERCENTILES V S. LABORATORY RESULTS

................................ ................................ ................................ ................................ ....... 22 FIGURE 9. TRUNCATED STANDARD NORMAL AT 1 ST AND 99 TH PERCENTILES A ND UNIFORM PROBABILI TY DENSITY FUNCTIONS

................................ ................................ ................................ ................................ ........................ 23 FIGURE 10. STEADY

- STATE PM ERROR SURFA CE FOR AVL WITH EXAM PLE SAMPLING FOR A R EFERENCE NTE EVENT

................................ ................................ ................................ ................................ ................................ ........... 25 FIGURE 11. OVERVIEW OF MONTE CARLO SIMUL ATION FOR BSPM

................................ ................................ .......... 30 FIGURE 12. ERROR SU RFACES INCLUDED IN M ONTE CARLO SIMULATIO N

................................ ................................ 32 FIGURE 13. LINEAR R EGRESSION FIT TO 5TH AND 95TH PERCENTILE DELTAS

................................ ........................... 38 FIGURE 14. LOESS RE GRESSION FIT TO 5TH AND 95TH PERCENTILE DELTAS

................................ ............................. 39 FIGURE 15. LINEARITY CHECK ON PEMS

- 1 EXHAUST FLOW DURIN G STEADY STATE ENGIN E TESTING

...................... 45 FIGURE 16. LINEARITY CHECK ON PEMS

- 2 EXHAUST FLOW DURIN G STEADY STATE ENGIN E TESTING

...................... 46 FIGURE 17. LINEARITY CHECK ON PEMS

- 3 EXHAUST FLOW DURIN G STEADY STATE ENGIN E TESTING

...................... 47 FIGURE 18. VOLVO MP7 INSTALLED IN A CVS T EST CELL

................................ ................................ ............................. 48 FIGURE 19. DPF BYPAS S WITH DOC

................................ ................................ ................................ ............................. 49 FIGURE 20. SCHEMATIC OF ENGINE DYNAMOMETE R EXPERIMENTAL SETUP

................................ ............................ 50 FIGURE 21. PPMD INST ALLED IN THE TEST CE LL

................................ ................................ ................................ ......... 52 FIGURE 22. THE AVL M SS

................................ ................................ ................................ ................................ ............. 53 FIGURE 23. THE PROBE AND TPA FOR THE HORI BA OBS

- TRPM

................................ ................................ .................. 54 FIGURE 24. THE HORIB A OBS

- TRPM AND OBS

- 2200

................................ ................................ ................................ ... 56 FIGURE 25. EXPERIMEN TAL SETUP FOR MIXING VERIFICATION

................................ ................................ .................. 57 FIGURE 26. MIXING VE RIFICATION SAMPLE LO CATIONS

................................ ................................ ............................ 57 FIGURE 27. MIXING VE RIFICATION RESULTS

................................ ................................ ................................ ............... 58 FIGURE 28. SPEED AND LOAD FOR THE 80 POIN TS CYCLE

................................ ................................ ........................... 61 FIGURE 29. FINAL SIX STEADY STATE MODES

................................ ................................ ................................ .............. 63 FIGURE 30. EXAMPLE O F STEADY STATE CYCLE

................................ ................................ ................................ ........... 65 FIGURE 31. STEADY ST ATE SAMPLE TIMES

................................ ................................ ................................ .................. 66 FIGURE 32. CVS FILTE R WEIGHT GAIN FOR ST EADY STATE TESTING

................................ ................................ ........... 67 FIGURE 33. STEADY ST ATE EXHAUST PM CONCE NTRATION (μG/MOL)

................................ ................................ ...... 68 FIGURE 34. STEADY ST ATE BRAKE

- SPECIFIC PM, C VS FILTER (MG/HP

- HR)

................................ ................................ . 69 FIGURE 35. NUMBER OF VALID DATA POINTS FO R STEADY STATE TESTI NG

................................ ............................... 70 FIGURE 36. HORIBA SA MPLE FLOW TO THE FIL TER AND DILUTION FLO W WHILE COMPRESSOR S TOPS

................... 72xvi SwRI Project No. 14936.12 FIGURE 37. SENSORS E FM ZERO DURING STEAD Y STATE CYCLE

................................ ................................ ................. 73 FIGURE 38. HORIBA

- 1 ST EADY STATE PM CONCEN TRATION DELTAS

................................ ................................ ......... 76 FIGURE 39. SENSORS

- 1 STEADY STATE PM CO NCENTRATION DELTAS

................................ ................................ ....... 76 FIGURE 40. AVL PM CO NCENTRATION DELTAS F OR STEADY STATE TEST ING ON PEMS 1

................................ .......... 77 FIGURE 41. HORIBA

- 2 STEADY STATE PM CO NCENTRATION DELTAS

................................ ................................ ......... 78 FIGURE 42. SENSORS

- 2 STEADY STATE PM CO NCENTRATION DELTAS

................................ ................................ ....... 78 FIGURE 43. AVL

- 2 STEADY STATE PM CO NCENTRATION DELTAS

................................ ................................ ................ 79 FIGURE 44. HORIBA

- 3 STEADY STATE PM CO NCENTRATION DELTAS

................................ ................................ ......... 80 FIGURE 45. SENSORS

- 3 STEADY STATE PM CO NCENTRATION DELTAS

................................ ................................ ....... 80 FIGURE 46. AVL

- 3 STEADY STATE PM CO NCENTRATION DELTAS

................................ ................................ ................ 81 FIGURE 47. STEADY ST ATE CONCENTRATION DE LTAS FOR HORIBA

................................ ................................ ............ 82 FIGURE 48. STEADY ST ATE CONCENTRATION DE LTAS FOR S ENSORS

................................ ................................ ......... 82 FIGURE 49. STEADY ST ATE CONCENTRATION DE LTAS FOR AVL

................................ ................................ .................. 83 FIGURE 50. STEADY ST ATE HORIBA PEMS PM C ONCENTRATION VERSUS THE LABORATORY REFER ENCE

................. 84 FIGURE 51. STEADY ST ATE SENSORS PEMS PM CONCENTRATION VERSUS THE LABORATORY REFER ENCE

............... 85 FIGURE 52. STEADY ST ATE AVL PEMS PM CONC ENTRATION VERSUS THE LABORATORY REFERENCE

....................... 86 FIGURE 53. LINEAR RE GRESSION CORRELATION BETWEEN PEMS AND LAB

................................ ............................... 87 FIGURE 54. STEADY ST ATE CONCENTRATION DE LTAS FOR HORIBA

................................ ................................ ............ 88 FIGURE 55. STEADY ST ATE CONCENTRATION DE LTAS FOR SENSORS

................................ ................................ ......... 88 FIGURE 56. STEADY ST ATE CONCENTRATION DE LTAS FOR AVL

................................ ................................ .................. 89 FIGURE 57. FINAL STE ADY STATE PM ERROR S URFACE

– HORIBA

................................ ................................ .............. 90 FIGURE 58. FINAL STE ADY STATE PM ERROR S URFACE

- SEN SORS

................................ ................................ ............. 91 FIGURE 59. FINAL STE ADY STATE PM ERROR S URFACE

– AVL

................................ ................................ ..................... 91 FIGURE 60. REPEAT EN GINE SPEED TRACES FO R NTE TRANSIENT CYCL E

................................ ................................ .... 93 FIGURE 61. REPEAT EN GINE TORQUE TRACES F OR NTE TRANSIENT CYC LE

................................ ................................ 94 FIGURE 62. REPEAT AV L SOOT CONCENTRATION TRACES FOR NTE TRANS IENT CYCLE

................................ .............. 95 FIGURE 63. AVL SOOT CONCEN TRATION DURING NTE T RANSIENT CYCLES, EVE NTS 20

- 23

................................ ....... 96 FIGURE 64. COMPARISO N OF AVL AND HORIBA REAL TIME SIGNALS DU RING TRANSIENT CYC LE

............................. 97 FIGURE 65. CVS BSPM AND CORRECTION FACTO R FOR TRANSIENT CYCL E

................................ ................................ 98 FIGURE 66. HORIBA C ONCENTRATION DELTAS FOR TRANSIENT ENGINE TESTING

................................ .................... 99 FIGURE 67. SENSORS C ONCENTRATION DELTAS FOR TRANSIENT ENGINE TESTING

................................ ................. 100 FIGURE 68. AVL CONCE NT RATION DELTAS FOR TR ANSIENT ENGINE TESTI NG

................................ ......................... 101 FIGURE 69. FINAL HOR IBA TRANSIENT PM ERR OR SURFACE

................................ ................................ .................... 102 FIGURE 70. FINAL SEN SORS TRANSIENT PM ER ROR SURFACE

................................ ................................ .................. 103 FIGURE 71. FINAL AVL TRANSIENT PM ERROR S URFACE

................................ ................................ ........................... 103 FIGURE 72. EXHAUST C ONFIGURATION FOR CE

- CERT CORRELATION

................................ ................................ ....... 104 FIGURE 73. BRAKE

- SPECIFIC PM RESULTS FROM CE

- CERT CORR ELATION

................................ ................................ 106 FIGURE 74. CVS FILTE R WEIGHT GAIN DURING TUNNEL BLANKS

................................ ................................ ............. 107 FIGURE 75. BRAKE SPE CIFIC PM EMISSIONS D URING ACTIVE REGENER ATION

................................ ......................... 108 FIGURE 76. TOTAL EXH AUST NUMBER CONCENTR ATION DURING STORAGE AND RELEASE

................................ .... 110 FIGURE 77. BRAKE

- SPECIFIC PM EMISSION S DURING STORAGE AND RELEASE INVESTIGATIO N

.............................. 111 FIGURE 78. OEM SUPPLIED TOR QUE ERRORS

................................ ................................ ................................ ........... 112 FIGURE 79. OEM SUPPL IED FUEL FLOW ERRORS

................................ ................................ ................................ ....... 113 FIGURE 80. TARGET DI LUTION RATIO AND PM LEVEL PROFILE FOR EN VIRONMENTAL TESTING

............................. 115 FIGURE 81. EXPERIMEN TAL SETUP FOR ENVIRO NMENTAL TESTING

................................ ................................ ........ 116 FIGURE 82. HORIBA EN VIRONMENTAL BASELINE MEASUREMENTS

................................ ................................ ......... 117xvii SwRI Project No. 14936.12 FIGURE 83. SENSORS E NVIRONMENTAL BASELIN E MEASUREM ENTS

................................ ................................ ....... 118 FIGURE 84. AVL ENVIR ONMENTAL BASELINE ME ASUREMENTS

................................ ................................ ............... 118 FIGURE 85. REFERENCE MSS ENVIRONMENTAL BA SELINE MEASUREMENTS

................................ ........................... 119 FIGURE 86. REFERENCE AVL VERSUS PEMS AVL FOR ENVIRONMENTAL BA SELINE

................................ .................. 120 FIGURE 87. ORIGINAL AND REVISED PROFILE FOR ALTITUDE TESTING

................................ ................................ ..... 121 FIGURE 88. ALTITUDE TESTING CH AMBER

................................ ................................ ................................ ................ 122 FIGURE 89. PEMS INST ALLED IN THE ALTITUD E CHAMBER

................................ ................................ ....................... 123 FIGURE 90. HORIBA EN VIRONMENTAL PRESSURE MEASUREMENTS

................................ ................................ ........ 124 FIGURE 91. SENSORS E NVIRONMENTAL PRESSUR E MEASUREMENTS

................................ ................................ ...... 125 FIGURE 92. AVL ENVIR ONMENTAL PRESSURE ME ASUREMENTS

................................ ................................ .............. 126 FIGURE 93. AVL PRESS URE MEDIAN VERSUS MA D

................................ ................................ ................................ .... 127 FIGURE 94. HORIBA PR ESSURE MEDIAN VERSUS MAD

................................ ................................ ............................. 127 FIGURE 95. SENSORS P RESSURE MEDIAN VERSU S MAD

................................ ................................ ........................... 128 FIGURE 96. FINAL ERR OR SURFACE FOR ENVIR ONMENTAL PRESSURE AV L PM CONCENTRATION

.......................... 129 FIGURE 97. TEMPERATU RE AND HUMIDITY PROF ILE FOR ENVIRONMENTA L TESTING

................................ ............ 130 FIGURE 98. PEMS INSTALLED IN TH E TEMPERATURE AND HU MIDITY CHAMBER

................................ ..................... 131 FIGURE 99. THE PM GE NERATOR INSTALLED OU TSIDE THE TEMPERATUR E AND HUMIDI TY CHAMBER

................. 132 FIGURE 100. HORIBA E NVIRONMENTAL TEMPERA TURE MEASUREMENTS

................................ .............................. 133 FIGURE 101. SENSORS ENVIRONMENTAL TEMPER ATURE MEASUREMENTS

................................ ............................ 134 FIGURE 102. AVL ENVI RONMENTAL TEMPERATUR E MEASUREMENTS

................................ ................................ .... 135 FIGURE 103. HORIBA T EMPERATURE MEDIAN VE RSUS MAD

................................ ................................ ................... 136 FIGURE 104. SENSORS TEMPERATURE MEDIAN V ERSUS MAD

................................ ................................ ................. 136 FIGURE 105. HORIBA T EMPERATURE AND HUMID ITY MEDIAN VERSUS MA D

................................ ......................... 137 FIGURE 106. FINAL ER ROR SURFACE ENVIRONM ENTAL TEMPERATURE AN D HUMIDITY AVL PM

........................... 138 FIGURE 107. AVL PEMS IN THE RADIATION CHA MBER FOR EMI AND RFI TESTING

................................ .................. 140 FIGURE 108. BULK CURRENT INJECTI ON PROBE

................................ ................................ ................................ ........ 141 FIGURE 109. HORIBA P EMS SETUP FOR BULK C URRENT INJECTION

................................ ................................ ......... 142 FIGURE 110. SENSORS SETUP DURING BULK CU RRENT INJECTION TEST ING

................................ ............................ 143 FIGURE 111. AVL SETU P DURING BULK CURREN T INJECTION TESTING

................................ ................................ ..... 143 FIGURE 112. HORIBA F LOW DISTURBANCE FROM BULK CURRENT INJECTI ON

................................ ......................... 144 FIGURE 113. HORIBA E XHAUST FLOW NOIS E ON ANALOG CABLE DU RING BULK CURRENT IN JECTION

.................. 145 FIGURE 114. BCI NOIS E ON AVL ANALOG OUTP UT CABLE

................................ ................................ ........................ 147 FIGURE 115. HORIBA P EMS SETUP DURING RAD IATED IMMUNITY TESTI NG

................................ ........................... 148 FIGURE 116. SENSORS PEMS SETUP DURING RA DIATED IMMUNITY TEST ING

................................ ......................... 149 FIGURE 117. AVL PEMS SETUP DURING RADIATE D IMMUNITY TESTING

................................ ................................ .. 149 FIGURE 118. HORIBA D ILUTION RATIO FLUCTUATIONS DURING RADIATED IMMUNITY

................................ .......... 150 FIGURE 119. ELECTROS TATIC DISCHARGE SIMU LATOR

................................ ................................ ............................. 151 FIGURE 120. SENSORS SETUP DURING CONDUCT ED TRANSIENTS TESTIN G

................................ ............................. 152 FIGURE 121. AVL SETU P DURING CONDUCTED T RANSIENTS TESTING

................................ ................................ ..... 152 FIGURE 122. SENSORS PEMS MOUNTED FOR TRA NSVERSE HORIZONTAL V IBRATION

................................ ............. 154 FIGURE 123. SENSORS PEMS MOUNTED FOR LONGITUDINAL HOR IZONTAL VIBRATION

................................ ......... 154 FIGURE 124. SENSORS PEMS MOUNTED FOR TRA NSVERSE 45° VIBRATIO N

................................ ............................. 155 FIGURE 125. SENSORS PEMS MOUNTED FOR LON GITUDINAL 45° VIBRAT ION

................................ ......................... 155 FIGURE 126. SENSORS PEMS MOUNTED FOR VER TICAL VIBRATION

................................ ................................ ........ 156 FIGURE 127. SENSORS PEMS MOUNTED FOR 45° VERTICAL VIBRATION

................................ ................................ .. 156 FIGURE 128. POWER SP ECTRAL DENSITY FOR V ERTI CAL VIBRATION TESTIN G

................................ ......................... 157xviii SwRI Project No. 14936.12 FIGURE 129. POWER SP ECTRAL DENSITY FOR H ORIZONTAL VIBRATION TESTING

................................ ................... 158 FIGURE 130. HORIBA P EMS VIBRATION POSITI ONS

................................ ................................ ................................ .. 159 FIGURE 131. SENSORS TOTAL FLOW DURING VI BRATION TESTING

................................ ................................ .......... 160 FIGURE 132. AVL PEMS VIBRATION POSITIONS

................................ ................................ ................................ ......... 161 FIGURE 133. AVL SOOT MEASUREMENT NOISE DU RING VIBRATION TESTI N G

................................ ......................... 162 FIGURE 134. CONVERG ENCE FOR AVL METHOD 1 AS A PERCENT OF BS PM THRESHOLD

................................ ....... 164 FIGURE 135. CONVERG ENCE FOR AVL METHOD 2 AS A PERCENT OF BS PM THRESHOLD

................................ ....... 165 FIGURE 136. CONVERG ENCE FOR AVL METHOD 3 AS A PERCENT OF BS PM THRESHOLD

................................ ....... 165 FIGURE 137. CONVERG ENCE FOR HORIBA METH OD 1 AS A PERCENT OF BSPM THRESHOLD

................................ . 166 FIGURE 138 . CONVERGENCE FOR H ORIBA METHOD 2 AS A PERCENT OF BSPM THRE SHOLD

................................ . 167 FIGURE 139. CONVERG ENCE FOR SENSORS MET HOD 1 AS A PERCENT O F BSPM THRESHOLD

............................... 167 FIGURE 140. CONVERG ENCE FOR SENSORS MET HOD 2 AS A PERCENT O F BSPM THRESHOLD

............................... 168 FIGURE 141. BOX PLO T OF ERROR SURFACE S ENSITIVITY BASED ON BIAS AND VARIANCE FO R AVL BSPM METHOD 1

................................ ................................ ................................ ................................ ................................ ....... 174 FIGURE 142. BOX PLO T OF ERROR SURFACE S ENSITIVITY BASED ON BIAS AND VARIANCE FO R AVL METHOD 2

.... 175 FIGURE 143. BOX PLO T OF ERROR SURFACE S ENSITIVITY BASED ON BIAS AND VARIANCE FO R AVL METHOD 3

.... 176 FIGURE 144. BOX PLO T OF ERROR SURFACE S ENSITIVITY BASED ON BIAS AND VARIANCE FO R HORIBA BSPM METHOD 1

................................ ................................ ................................ ................................ ........................ 177 FIGURE 145. BOX PLOT OF ERROR SURFAC E SENSITIVITY BASED ON BIAS AND VARIANCE FOR HORIBA BSPM METHOD 2

................................ ................................ ................................ ................................ ........................ 178 FIGURE 146. BOX PLO T OF ERROR SURFACE S EN SITIVITY BASED ON BI AS AND VARIANCE FOR SENSORS BSPM METHOD 1

................................ ................................ ................................ ................................ ........................ 179 FIGURE 147. BOX PLOT OF ERROR SURFACE SEN SITIVITY BASED ON BI AS AND VARIANCE FOR SEN SORS BSPM METHOD 2

................................ ................................ ................................ ................................ ........................ 180 FIGURE 148. VALIDAT ION PERCENTILES FOR THE 141 REFERENCE NT E EVENTS FOR AVL MET HOD 1

.................... 184 FIGURE 149. VALIDAT ION PERCENTILES FOR THE 141 REFERENCE NT E EVENTS FOR AVL MET HOD 2

.................... 185 FIGURE 150. VALIDAT ION PERCENTILES FOR 141 REFERENCE NTE EV ENTS FOR AVL METHOD 3

........................... 185 FIGURE 151. VALIDAT ION PERCENTILES FOR 141 REFERENCE NTE EV ENTS FOR HORIBA METH OD 1

..................... 186 FIGURE 152. VALIDAT ION PERCENTILES FOR 141 REFERENCE NTE EV ENTS FOR HORIBA METH OD 2

..................... 1 86 FIGURE 153. VALIDAT ION PERCENTILES FOR 141 REFERENCE NTE EV ENTS FOR SENSORS MET HOD 1

................... 187 FIGURE 154. VALIDAT ION PERCEN TILES FOR 141 REFERE NCE NTE EVENTS FOR S ENSORS METHOD 2

................... 187 FIGURE 155. VALIDAT ION 95TH PERCENTILE BSPM DELTAS LOESS FI T FOR SENSORS METHOD 1

.......................... 189 FIGURE 156. VALIDAT ION 5TH PERCENTILE B SPM DELTAS LOESS FIT FOR SENSORS METHOD 1

............................ 190 FIGURE 157. VALIDAT ION 95TH PERCENTILE BSPM DELTAS LOESS FI T FOR SENSORS METHOD 2

.......................... 190 FIGURE 158. VALIDAT ION 5TH PERCENTILE B SPM DELTAS LOESS FIT FOR SENSORS METHOD 2

............................ 191 FIGURE 159. VALIDAT ION 95TH PERCENTILE BSPM DELTAS LOESS FI T FOR AVL METHOD 1

................................ ... 191 FIGURE 160. VALIDAT ION 5TH PERCENTILE BSPM DELTAS LOESS FI T FOR AVL METHOD 1

................................ ..... 192 FIGURE 161. VALIDAT ION 95TH PERCENTILE BSPM DELTAS LOESS FI T FOR AVL METHOD 2

................................ ... 192 FIGURE 162. VALIDAT ION 5TH PERCENTILE B SPM DELTAS LOESS FIT FOR AVL METHOD 2

................................ ..... 193 FIGURE 163. VALIDAT ION 95TH PERCENTILE BSPM DELTAS LOESS FI T FOR AVL METHOD 3

................................ ... 193 FIGURE 164. VALIDAT ION 5TH PERCENTILE B SPM DELTAS LOESS FIT FOR AVL METHOD 3

................................ ..... 194 FIGURE 165. REGRESS ION PLOT OF 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR AVL METHOD 1

..... 195 FIGURE 166. REGRESS ION PLOT OF 95 TH PERCENTILE DELTA BSP M VER SUS IDEAL BSPM FOR A VL METHOD 2

..... 197 FIGURE 167. REGRESS ION PLOT OF 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR AVL METHOD 3

..... 198 FIGURE 168. REGRESS ION PLOT OF 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR HORIBA METHOD 1

................................ ................................ ................................ ................................ ................................ ......... 200xix SwRI Project No. 14936.12 FIGURE 169. REGRESS ION PLOT OF 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR HORIBA METHOD 2

................................ ................................ ................................ ................................ ................................ ......... 201 FIGURE 170. REGRESS ION PLOT OF 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR SENSORS METHOD 1

................................ ................................ ................................ ................................ ................................ ......... 203 FIGURE 171. REGRESS ION PLOT OF 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR SENSORS METHOD 2

................................ ................................ ................................ ................................ ................................ ......... 204 FIGURE 172. VALIDAT ION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON T HE SIMULATION MODEL FOR BSPM SENSORS METHOD 1 WIT H NO REGEN

................................ ................................ ................................ ............ 207 FIGURE 173. VALIDAT ION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON T HE SIMULATION MODEL FOR BSPM SENSORS METHOD 2 WIT H NO REGEN

................................ ................................ ................................ ............ 208 FIGURE 174. VALIDAT ION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON T HE SIMULATION MODEL FOR BSPM AVL METHOD 1 WITH NO REGEN

................................ ................................ ................................ ..................... 209 FIGURE 175. VALIDAT ION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON T HE SIMULATION MODEL FOR BSPM AVL METHOD 2 WITH NO REGEN

................................ ................................ ................................ ..................... 210 FIGURE 176. VALIDAT ION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON T HE SIMULATION MODEL FOR BSPM AVL METHOD 3 WITH NO REGEN

................................ ................................ ................................ ..................... 211 FIGURE 177. VALIDAT ION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON T HE SIMULATION MODEL FOR BSPM AVL METHOD 1 WITH RE GEN

................................ ................................ ................................ ........................... 212 FIGURE 178. VALIDAT ION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON T HE SIMULATION MODEL FOR BSPM AVL METHOD 2 WITH RE GEN

................................ ................................ ................................ ........................... 213 FIGURE 179. VALIDAT ION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON T HE SIMULATION MODEL FOR BSPM AVL METHOD 3 WITH RE GEN

................................ ................................ ................................ ........................... 214 FIGURE 180. SENSITI VITY CHART FOR AVL D ELTA PM METHOD 1

................................ ................................ ........... 240 FIGURE 181. BRAKE

- SPECIFIC PM BY METHO D 1

................................ ................................ ................................ ...... 257xx SwRI Project No. 14936.12xxi SwRI Project No. 14936.12 LIST OF TABLES

TABLE 1. MEASRUEMENT ALLOWANC E BASED ON 0.02 G/HP

- HR THRESHOLD

................................ ......................... VI TABLE 2. SENSORS PPM D VALIDATION RESULTS

................................ ................................ ................................ .......... IX TABLE 3. AVL MSS VAL IDATION RESULTS

................................ ................................ ................................ ..................... IX TABLE 4. PM

- PEMS USED ALONG WITH SERIAL NUMBER

................................ ................................ ............................. 2 TABLE 5. REFERENCE NTE EVENTS AND IDEAL BSPM EMISSIONS

................................ ................................ ................ 9 TABLE 6. DESCRIPTIV E STATISTICS FOR BSP M EMISSIONS FOR 141 REFERENCE NTE EVENTS

................................ ... 11 TABLE 7. INPUT PARA METERS FOR REFERENCE NTE EVENTS

................................ ................................ ..................... 13 TABLE 8. ERROR SURF ACES FOR MONTE CARLO SIMULATION

................................ ................................ .................. 15 TABLE 9. ERROR SURF ACES USED FOR COMPUT ING BRAKE

- SPECIFIC PM EMISSION S BY THREE CALCULATI ON METHODS

................................ ................................ ................................ ................................ ........................... 27 TABLE 10. EXAMPLE OF SELEC TION OF MEASUREMENT ALLOWANCE AT 0.02 G/ HP

- HR NTE THRESHOLD FOR THE AVL PEMS

................................ ................................ ................................ ................................ ........................... 35 TABLE 11. SUMMARY OF PART 1065 A UDITS

................................ ................................ ................................ .............. 40 TABLE 12. LINEARITY VERIFICATIONS FOR HO RIBA PEMS

................................ ................................ ........................... 41 TABLE 13. LINEARITY VERIFICATIONS FOR SE NSORS PEMS

................................ ................................ ......................... 43 TABLE 14. LINEARITY VE RIFIC ATIONS FOR AVL PEMS

................................ ................................ ................................ . 44 TABLE 15. LIST OF DP F BYPASS CONFIGURATI ONS

................................ ................................ ................................ ...... 50 TABLE 16. SAMPLE ORD ER FOR STEADY STATE CYCLE TESTING

................................ ................................ .................. 64 TABLE 17. DATA YIELD BY EACH PM

- PEMS

................................ ................................ ................................ .................. 71 TABLE 18. TEST PROCE DURE FOR CE

- CERT CORRELATION

................................ ................................ ........................ 105 TABLE 19. PM EMISSIO NS RESULTS FROM ACTI VE DPF REGENERATION

................................ ................................ .. 108 TABLE 20. AVERAGE BR AKE

- SPECIFIC EMISSIONS D URING STORAGE AND RE LEASE CYCLE

................................ ...... 111 TABLE 21. OEM ERROR SURFACE DELTAS FOR T ORQUE AND FUEL FLOW

................................ ................................ 113 TABLE 22. SUMMARY O F NUMBER OF REFERENC E NTES MEETING 2% CO NVERGENCE

................................ ......... 168 TABLE 23. ERROR SUR FACE SENSI TIVITY TO BIAS AND V ARIANCE FOR 141 REFE RENCE NTE EVENTS FOR AVL BSPM METHOD 1

................................ ................................ ................................ ................................ ........................ 170 TABLE 24. ERROR SUR FACE SENSITIVITY TO BIAS AND VARIANCE FOR 141 REF ERENCE NTE EVENTS FO R AVL BSPM METHOD 2

................................ ................................ ................................ ................................ ....................... 170 TABLE 25. ERROR SUR FACE SENSITIVITY TO BIAS AND VARIANCE FO R 141 REFERENCE NTE EVENTS FOR AVL BSPM METHOD 3

................................ ................................ ................................ ................................ ........................ 171 TABLE 26. ERROR SUR FACE SENSITIVITY TO BIAS AND VARIANCE FO R 141 REFERENCE NTE EVENTS FOR HORIBA BSPM METHOD 1

................................ ................................ ................................ ................................ ............. 171 TABLE 27. ERROR SUR FACE SENSITIVITY TO BIAS AND VARIANCE FO R 141 REFERENCE NTE EVENTS FOR HORIBA BSPM METHOD 2

................................ ................................ ................................ ................................ ............. 172 TABLE 28. ERROR SUR FACE SENSITIVITY TO BIAS AND VARIANCE FO R 141 REFERENCE NTE EVENTS FOR SENSORS BSPM METHOD 1

................................ ................................ ................................ ................................ ............. 172 TABLE 29. ERROR SUR FACE SENSITIVITY TO BIAS AND VARIANCE FO R 141 REFERENCE NTE EVENTS FOR SENSORS BSPM METHOD 2

................................ ................................ ................................ ................................ ............. 173 TABLE 30. SUMMARY O F ERROR SURFACE SENS ITIVITIES TO BIAS AN D VARIANCE FOR BSPM METHOD 1

............ 181 TABLE 31. SUMMARY O F ERROR SURFACE SENS ITIVE TO BIAS AND VA RIANCE FOR BSPM METH OD 2

.................. 182 TABLE 32. SUMMARY O F ERROR SURFACE SENS ITIVE TO BIAS AND VA RI ANCE FOR BSPM METHOD 3

.................. 183 TABLE 33. LOESS SMO OTHING PARAMETERS FO R VALIDATION PERCENT ILES

................................ ........................ 188 TABLE 34. MEASUREME NT ERROR AT THRESHOL D FOR BSPM USING REG RESSION AND MEDIAN M ETHODS FOR AVL METHOD 1

................................ ................................ ................................ ................................ ................. 196xxii SwRI Project No. 14936.12 TABLE 35. MEASUREME NT ERROR AT THRESHOL D FOR BSPM USING REG RESSION AND MEDIAN M ETHODS FOR AVL METHOD 2

................................ ................................ ................................ ................................ ................. 197 TABLE 36. MEASUREME NT ERROR AT THRESHOL D FOR BSPM USING REG RESSION AND MEDIAN M ETHODS FOR AVL METHOD 3

................................ ................................ ................................ ................................ ................. 199 TABLE 37. MEASUREME NT ERROR AT THRESHOL D FOR BSPM USING REG RESSION AND MEDIAN M ETHODS FOR HORIBA METHOD 1

................................ ................................ ................................ ................................ .......... 200 TABLE 38. MEASUREME NT ERROR AT THRESHOL D FOR BSPM USING REG RESSION AND MEDIAN M ETHODS FOR HORIBA METHOD 2

................................ ................................ ................................ ................................ .......... 202 TABLE 39. MEASUREME NT ERROR AT THRESHOL D FOR BSPM USING REG RESSION AND MEDIAN M ETHODS FOR SENSORS METHOD 1

................................ ................................ ................................ ................................ ........ 203 TABLE 40. MEASUREME NT ERROR AT THRESHOL D FOR BSPM USING REG RESSION AND MEDIAN M ETHODS FOR SENSORS METHOD 2

................................ ................................ ................................ ................................ ........ 205 TABLE 41. BSPM MEAS UREMENT ERROR IN PER CENT OF NTE THRESHOL D BY PEMS AND CALCUL ATION METHOD

................................ ................................ ................................ ................................ ................................ ......... 205 TABLE 42. MEASUREME NT ALLOWANCE AT NTE THRESHOLD BY EMISSIO NS FOR METHOD 2

............................... 206 TABLE 43. SUMMARY OF BSPM MODE L VALIDATION RESULTS

................................ ................................ ............... 214 TABLE 44. SIMULATIO N VARIABLES

................................ ................................ ................................ .......................... 230 TABLE 45. DESCRIPTI VE STATISTICS FOR SI MULATION VARIABLES

................................ ................................ .......... 236 TABLE 46. ERROR SURF ACES USED IN SIMULAT ION

................................ ................................ ................................ .. 246 TABLE 47. METHODS W ORKSHEET PARAMETER C OLUMN DESCRIPTIO NS

................................ .............................. 2501 SwRI Project No. 14936.12 1 INTRODUCTION
The U.S. EPA, EMA, and CARB agreed to pursue an experimental data driven program to determine PEMS bias and precision measurement errors expected in in

- use NTE testing and compliance before enforcement. The idea is to comb ine these errors into a measurement allowance and add it to the EPA in

- use regulato ry limit to increase it to a new threshold level that the engine manufacturers must comply with. The gaseous PEMS measurement allowance program was completed by SwRI in Apri l of 2007, and this work focuses on the PM

- PEMS measurement allowance.

To determine all bias and precision errors associated with in

- use PM

- PEMS measurement, it will take a very extensive set of experimental data and engines to cover all engine steady

- stat e and transient NTE operation s as well as different environmental conditions and configurations that may influence the measurement. Instead of focusing on a wide matrix of experimental data, the SC appro ach was to : a) Perform repeats on a series of laboratory on a six NTE steady

- state tests using the CVS PM filter method and the PM

- PEMS, and establish a bias error surface for each the PEMS tested b) Perform repeats on a series of laboratory NTE transient cycles containing thirty 32 seconds NTE events, and use the PM

- PEMS measurement to produce precision error surface for each of the PEMS tested c) Perform a series of environmental tests that includes electromagnetic and radio frequency interferences, shock and vibration, pressure, and temperature and humidity, and produce an error surface, if any, for each PEMS associated with each parameter d) Use torque and f uel flow error surfaces provided by the engine manufacturers e) Use other error surfaces established during the gaseous measurement allowance program f) Use Monte

- Carlo simulation based on a set of 141 Ideal brake

- specific PM (BSPM) reference NTE events to predi ct the error distribution at each reference NTE g) Use the 95 th percentile and 5 th

h) Perform actual in

- u se testing with the PM

- PEMS using the CE

- CERT trailer to determine whether or not the data generated in

- use validates the model. percentile of the error distribution at each Ideal BSPM reference NTE level as the uppe r and low boundary of the deltas between PM

- PEMS BSPM and Ideal BSPM.

A total of nine PM

- PEMS, three from each manufacturer, were used on this progr am. The PM

PEMS, shown in Table 4 , included the Sensors PPMD, Horiba TRPM, and AVL MSS. Each set of three PM

- PEMS, one PM

- PEMS from each manufacturer, was tested simultaneously in parallel using a series of steady

- s tate and transient NTE engine experiments. After the completion of steady

- state testing and transient testing, one PM

- PEMS from each manufacturer was used on a series of environmental test conditions that included electromagnetic and radio frequency interf erences; shock and vibration, atmospheric pressure, and temperature and humidity.2 SwRI Project No. 14936.12 TABLE 4

. PM

- PEMS USED ALONG WITH SERIAL NUMBER
PEMS Name PEMS Serial Number Horiba 1 10107

- 01 Horiba 2 10107

- 02 Horiba 3 10107

- 03 Sensors 1 E08

- PD03 Sensors 2 G08

- PD02 Sensors 3 A08

- PD03

AVL 1 346 AVL 2 472 AVL 3 273

To validate the model, a series of in

- use tests with the PM

- PEMS were performed by CE

- CERT using the CE

- CERT emissions trailer.
This report describes:

**•** Model approach used to perform the modeling portion of this work.

**•** Engine experiment s with the PM

- PEMS including steady

- state and transient NTE PM results

**•** Environmental setup and PM results

**•** Measurement allowance produced by the model for each of the three PM

- PEMS provided by each manufacturer

**•** Model validation using the PM

- PEMS selected for in

- use testing

2 MODELING APPROACH

2.1 PURPOSE OF MODEL
The desire for this program was to generate BSPM measurement allowances based on rigorous statistical methods applied to a large body of data. At the same time, it was desirable to exclude outlier data caused by extreme measurement errors which were not considered representative of normal in

- use operations. A direct approach could have been to test PEMS against some kin d of mobile laboratory reference (such as the CE

- CERT Mobile Emission Laboratory) on a large number of vehicles, and quantify errors directly. However, such an approach would have been expensive in terms of both time and funding.

Given these factors, the Steering Committee ultimately elected to use a simulation approach in order to generate the BSPM measurement allowances, similar to what was done in the gaseous in

- use emissions testing program [1] . In this approach, the Steering Committee defined all of the expected sources of PEMS measurement errors based on existing in

- use testing expertise and3 SwRI Project No. 14936.12 understanding of how the PEMS functioned. Each of these errors was quantified using a series of controlled laboratory experiments, each designed to isolate erro rs related to a single error source. The results of each experiment would essentially be an empirical model of a given source of measurement error. In this report, these error models are referred to as error surfaces. It is important to note that each o f these error surfaces represents an incremental error of PEMS measurement, as compared to an associated laboratory reference measurement.

2.2 MODEL IMPROVEMENT
Several improvements to the execution of the Monte Carlo simulation model were implemented to impro ve the efficiency and post

- processing of the simulation runs. Eight macros were written to perform various tasks and are summarized in the section below.

Macro 1 : Controls batch processing and allows the ability to run several simulations back

to

- back (ba tch mode). Reads each reference NTE event data, number of trials, and the number of reference NTE events in the batch run. Calls other macros.

Macro 2: Clears and deletes extra rows in error model ( see Appendix D for a detailed description ) . C alculates i deal PM emissions for each calculation method

.
Macro 3: Checks error surfaces turned ‘off’ in simulation run and clears unused error surface charts and Excel tabs for calculation speedup ( see Appendix D for a detailed description )

.
Macro 4: Controls Crystal Ball run preferences
, runs Crystal Ball simulation for the given NTE Event, and controls Crystal Ball creation of Report and Extract data files . No longer store s each trial i c value (40,000

– 65 ,000 values) . Reduced the number of sensitivity chart s created. Only stores BS emissions in g/ hp

- hr units. This r educed EXTRACT and REPORT files from 133 MB to 17 MB per NTE event ( ≈ 87% reduction)

.
Macro 5: Reads EXTRACT file, controls calculation of 5 th , 50 th and 95 th percentiles from full model and cal ls Macro 6.

Macro 6: Compute s 5 th , 50 th and 95 th percentile delta BS emissions and confidence limits from order statistics from the Full model. These values are used in the measurement allowance plots and to check for convergence.

Macro 7: Reads Extract f ile Validation data, computes 5 th , 50 th and 95 th percentiles for the Validation model (calls Macro 6)

.
Macro 8: Reads REPORT file, selects and formats sensitivity data for Assumption

sens itivities greater than 5 percent or less than

- 5 percent contribution to variance and stores in sensitivity file. Create s sensitivity file with “important” error surfaces

.4 SwRI Project No. 14936.12 The development of the macros resulted in substantial improvements in the simulation run

- time and post

- processing of the extract files. The estimated r e duction in model simulation run

- time and post processing was approximately 80 to 85 percent of the time used in the gaseous PEMS program. The developed macros eliminated the need to manual ly post

- process the files as had been done in the gaseous PEMS prog ram . The batch processing allowed up to 20 reference NTE events to be simulated in a single Excel run. The size of the EXTRACT and REPORT files was greatly reduced by only sto ring needed variables and delta PM emissions.

2.3 MONTE CARLO SIMULATI ON APPROACH

T he error surfaces representing incremental error s of PEMS measurement were programmed into a computer model which employed Monte Carlo random sampling methods to simulate the combined effects of all of these sources of error on the final measured brake

- spe cific value. An ideal reference NTE data set for a given test event was run through the Model, and all the various errors were applied to that data set in a randomly chosen manner. Brake

- specific PM emission values were then calculated for both the ideal and error

- applied data sets, which were compared to yield a final measurement error. The process was repeated thousands of times, with many different ideal data sets, to generate a large, robust data set which was evaluated to determine a final set of co mbined measurement errors. These final errors, referred to in this report as deltas, were generated for the PM pollutant for each calculation method and three difference PEMS model units, for a final set of seven deltas; Methods 1

- 3 for the AVL PEMS unit, Methods 1

- 2 for the Horiba PEMS unit and Methods 1

- 2 for the Sensors PEMS unit. A complete description of the Monte Carlo methodology and of the model is given in Section 2.3 of this report.

2.4 CALCULATION METHODS
Calculations must be performed on the recorded data to determine brake

- specific PM emission values in accordance with methods outlined in 40 CFR Part 1065 Subparts G and J. The symbolic notation given in the formulas shown later in this section is fully described in 40 CFR Part 1065 Subpart K [2]

. 40 CFR Part 1065 allows for the use of any of three different calculation methods in order to determine brake

- specific emission values from in

- use test data. T he basic calculation of brake

specific emissions requires three main inputs as follows:

BBBB EEEEEEEEEEEEEEEEEE = MM WW WWWW = CC CCCC CC CC xx FFFF FF PP5 SwRI Project No. 14936.12 The three calculation methods vary somewhat in the means used to d etermine either the Flow component or the Work component of this calculation. Each of the three methods is summarized below. Because each method relies on different inputs, it is possible that each method of calculation will react differently to various measurement errors. Therefore, measurement allowances must be examined independently for each method. However, according to the Test Plan methodology, only one of the three calculation methods would be selected to generate the final measurement allowance s. The selection methodology is outlined later in this introduction under the Measurement Allowance Generation section.

2.4.1 Calculation Method 1

– “ Exhaust Flow

- Torque

- Speed ” Method
Calculation Method 1 is analogous to the method used by most dynamometer labo ratories, and relies on direct input of both exhaust flow and torque. In the case of exhaust flow, this is the flow rate measured by the same form of exhaust flow meter. The Sensors and AVL PEMS relied on the Sensors exhaust flow meter (EFM) while the Ho riba PEMS had a tail pipe adapter (TPA) it employed for exhaust flow measurement. Work is not measured directly, but is instead calculated using ECM broadcast engine speed and ECM broadcast engine torque. While engine speed is directly measured by the eng ine ECM, ECM broadcast torque is an estimate based on a variety of other parameters, therefore, torque cannot be directly verified during in

- use testing. A simplified formula for this method is:

h oo 1 = ∑ gg ∑

The more complete formula used for Method 1 is as follows:

For all PM PEMS :

PM m



## ( **) (**  **) (**  **)** 

# ∑ **∑**

**= =**  

###  ** **  ** **  ** ∆** 

**⋅ **  ** **  ** **  ** ∆**  ** **  ** **  ** **  ** **  ** **  ** ** 

**= ⋅**  N

i i i N

i i PM

PM 

### t m N T rpm Speed t s mol n mol

g m hr kW g e

1 1



### 3600 * 1000 * 60 * 14159

. 3 * 2 * * *

* /  is a flow weighted particulate matter exhaust concentration in g/mol

Where f or AVL:6 SwRI Project No. 14936.12 PM m

1

1 

### *

* N

i i i PM N

i i 

### g mol mPM n t mol s g m mol mol n t s **=**

**=** 

###   **   ∗ ∆**  **    **  **    **  ** **

**=  **  **   **  **  ∆**  **   **  **   **  

# ∑ **∑** 

 is computed numerically as follows,

It should be noted that calculation Method 1 is directly dependent on the accuracy of both the exhaust flow meter and the torque estimation, as well as on the measurement of gaseous concentration.

2.4.2 Calculation Method 2

– “ Exha ust and Fuel Flow

- Torque

- Speed” Method
This calculation is designated solely for in

- use testing, and is designed to minimize the effect of errors related to the accuracy of the exhaust flow measurement. The Method 2 calculation adjusts the exhaust flow measurement by a ratio of the CO 2 - based fuel flow to the ECM reported fuel flow. This means that although the flow meter must be linear, it does not necessarily have to be accurate. In addition, Method 2depends on the ECM broadcasted torque and speed, and on the ratio of fuel flo w calculated from the carbon balance using gaseous measurement over the fuel flow broadcasted by the ECM. 2 depends on the ECM broadcasted torque and speed. A simplified version of this method can be expressed as:

h oo 2 = ∑ gg ∑ � CCCC 2 ffff EE ffff × WW WWWW �

The more complete formula for Method 22 using PM as an example, is:7 SwRI Project No. 14936.12 For all PM PEMS :

PM m



## ( **)**

**( )**  **( )**  **( )**  

# ( **)**

**( )**  **( )**  1

6 2 2 

### * * /

* *10 % % *10 *

*

* *2*3.14159 60*1000*3600 i

N

PM i i PM

ii i C
fuel fueli

i i 

### g mol m nt mol s e g kW hr

mol n xTHC ppm xCO xCO t M s

w g m s Speed rpm T N m **=**

**− −**



###   **  ∆**  **   **  **   ** 

**⋅ =  **  **   **  **  ++ ∆**  **   **  **   **  **   **  **   ** 

**⋅  **  ** ** 

# ∑ 



 1 N

i **=** 

# ∑ is a flow weighted particulate matter exhaust concentration in g/mol

Where f or AVL:

PM m

1

1 

### *

* N

i i i PM N

i i 

### g mol mPM n t mol s g m mol mol n t s **=**

**=** 

###   **   ∗ ∆**  **    **  **    **  ** **

**=  **  **   **  **  ∆**  **   **  **   **  

# ∑ **∑** 

 is computed numerically as follows,

It should be noted that, as mentioned earlier, Method 22 is not subject to accuracy errors for the exhaust flow measurement, although that measurement must still be linear for the method to function properly.

2.4.3 Calculation Method 3

– “Fuel Flow

- Torque

- Speed ” Method
Method 3 does not use direct measurement of exhaust flow, but relies on a carbon balance and ECM broadcast fuel rate to determine mass. The work term for Method 3 is determined iden tically to the work term for Method 1; using the ECM broadcast values for engine speed and torque to calculate work. Method 3 entirely circumvents the use of an exhaust flow meter, but for the HDIUT program, EPA must approve the use of Method 1 for a give n test and manufacturer. A simplified version of Method 1 may be expressed as:8 SwRI Project No. 14936.12 h oo 3 = ∑ � gg × EE ffff CCCC 2 ffff �
∑ WW WWWW

The mor e complete formula for Method 3 is:
For AVL Only:



## ( **)**

**( )**  **( )**  **( )**  

# ( **) (**  **) (**  **)** 6 2 1 2

1 

### /

* * * *10 % % *10

* *2*3.14159* 60*1000*3600 i PM

ifue l N fuel PM i C i i

N i i

i 

### e g kW hr

g m w g s m t mol M xTHC ppm xCO xCO

Speed rpm T N m t **− −**

**=**

**=** 

### ⋅ = **   **  **   **  **   **  **∆  **  **+ +  **  **  ⋅ ∆**  **   **  ** ** 

# ∑

**∑** 

Where:



## ( **)** 

# ( **) (**  **) (**  **) (**  **)** 

# ( **)**

**( )**  **( )**  **( )**  

# ( **)** 6 2 1 2

6 2 1 2 

### / *

* * *10 % % *10

* * *10 % % *10 i

i i ifue l N fuel

i

C i i

PM
ifue l N fuel

i

C i i 

### g mPM g mol m w s t M xTHC ppm xCO xCO g m mol g m w s t M xTHC ppm xCO xCO **− −**

**=**

**− −**

**=** 

###   **   **  **   **  **∆ + +**  **   ** 

**=  **  **   **  **   **  **   **  **∆ + +**  ** ** 

# ∑

**∑** 



It should be noted that Method 3 is not subject to exhaust flow measurement accuracy errors, but also that this method is wholly dependent on ECM broadcast values for both mass and work determination.

2.5 REFERENCE NTE EVENTS

The reference data set to which all t he simulated errors were applied represented engine operations over a wide range of NTE events. This reference data set was generated from collections of real

- world PEMS data sets. Parameters in the reference data set were scaled in9 SwRI Project No. 14936.12 order to exercise the model through a more appropriate range of parameters (i.e. concentrations, flows, ambient conditions, etc.). In this scaling process, care was taken to maintain the dynamic characteristics of the reference data set.

The Monte Carlo simulation model was r un on a set of 141 reference NTE events collected from a number of sources. Five engine manufacturers provided a total of 97 events; 10 reference NTE events came from each of the three engines tested in the lab during the transient testing; and 14
events came from the pre

- pilot CE

- CERT data. The gaseous measurement allowance program used 195 reference NTE events, but 54 of the events were repeats in terms of speed and torque but with different gaseous emissions [1] . These events were removed and PM concen tration estimations were generated for the remaining 141 events. Before and a fter errors were applied in the Monte Carlo simulation for each of these reference NTE events.

NTE brake

- specific emissions results were calculated for PM us ing each of the three agreed

upon NTE calculation methods. Th e three different BS emission s calculation methods referred to in this test plan are :

1. Method #1: Exhaust Flow

- Torque

- Speed Method 2. Method #2: Exhaust and Fuel Flow

- Torque

- Speed Method 3. Method #3: Fuel Flow

-- Torque

- Speed M ethod

Table 5 lists the number of NTE events obtained from each data source and the BSPM emissions calculated using Method 1. The PM concentration used to calculate the Reference NTE PM emissions was devel oped generated by SwRI based on the Volvo engine used in this study. A simple model was developed to predict the PM concentration based on the speed, torque, and throttle position of the engine. The model was initially designed to match the actual PM emis sions over a short transient cycle for which data had been recorded. These emissions have been computed with no error values added to the input parameters. For this report, emissions with no errors added will be labeled the “ideal” emissions. In contras t, the emissions with errors added through the Monte Carlo simulation will be labeled emissions “with errors”.

TABLE 5

. REFERENCE NTE EVENTS AND IDEAL BSPM EMISS IONS

Source Number of NTE Events BSPM
g/hp

- hr

Min Max

International 19 0.01146 0.03069

DDC 18 0.01303 0.0346310 SwRI Project No. 14936.12 Caterpillar 20 0.01097 0.01789

Cummins 20 0.01561 0.03253

Volvo 20 0.01541 0.02701
Engine #1 10 0.01215 0.04066
Engine #2 10 0.01675 0.04872
Engine #3 10 0.01099 0.03669

Pre

- Pilot 14 0.01375 0.03561

When the ideal brake

- specific emission values were calculated for the various reference NTE events, it was noted that these ideal emission values were frequently different from one calculation method to another. While it was recognized that this was a realistic outcome, the Steering Committee felt that these discrepancies might introduce an unintended bias into the results of the Monte Carlo simulation. Therefore, the Steering Committee directed SwRI to adjust the NTE reference event data in ord er to align the brake

- specific emission levels from all the calculation methods

. The adjustment was performed by first assuming that the Method 1 result was the desired value, and that the other two calculation methods would be aligned to that result. Thi s meant that torque, speed, and exhaust flow values were not changed. The next step of the alignment process was to adjust CO 2 values for the NTE event, in order to line up the Method 2 PM result with the Method 1 value. This was done by using a single multiplier on all CO 2

The distribution of the ideal BSPM emissions data for the 141 reference NTE events was presented to the Steering Committee. It was noted that very few reference NTE events were at or below the BSPM threshold (0.02 g/hp

- hr). Thus, the reference NTE events were adjusted to produce more events with ideal BSPM values below 0.02 g/hp

- hr. The original and revised ideal BSPM distributions are depicted in values for the NTE event in question. Finally the fuel rate values and alignments were adjusted to bring Method 3 in line with Method 1 and 2.

Figure 3 . Note that the BSPM emission has data values spread above and below the corresponding NTE threshold.11 SwRI Project No. 14936.12 FIGURE 3 . METHOD 1 IDEAL BS PM VALUES FOR REFERE NCE NTE EVENTS

Table 6 provides a summary of some descriptive statistics for the reference NTE data set for BSPM emissions.

TABLE 6

. DESCRIPTIVE STATISTI CS FOR BSPM EMISSION S FOR 141 REFERENCE NTE EVENTS

Descripti ve Statistic BS PM
g/ hp

- hr

Minimum 0.010974

Maximum 0.048717

Mean 0.020753

Median 0.019101

Standard Deviation 0.00693212 SwRI Project No. 14936.12 The parameter data provided in each reference NTE event was on a second

- by

- second basis with a minimum of 30 seconds and a maximum of 300 seconds. The input parameters required for the BS PM emissions calculation methods and the Monte Carlo simulation are listed in Table 7

. An Excel file with a specific input format structure was used to standardize the format of the input files. Since the total hydrocarbons (THC) was selected as an input parameter, NMHC was computed as THC*0.98.13 SwRI Project No. 14936.12 TABLE 7 . INPUT PARAMETERS FOR REFERENCE NTE EV ENTS

Variable Number Input Variable Units Description

1 NTE Event Number integer All reference NTE events must be identified by an NTE number (e.g., 001). 2 NTE Source alphanumeric The source of the NTE event is the company, organization and/or lab that created the event data. 3 Engine Make alphanumeric Engine Make 4 Engine Model alphanumeric Engine Model 5 Engine Displacement L Engine Displacement (L) 6 Date mm/dd/yyyy The day the NTE event data was created (mm/dd/yyyy). 7 Time Stamp hh:mm:ss.s Time in seconds. Each reference NTE must contain second-bysecond data only. 8 Wet CO2 % CO2 (%) 9 Wet CO % CO (%) 10 Wet kNO ppm NO (ppm) with intake air-humidity correction 11 Wet kNO2 ppm NO2 (ppm) with intake air-humidity correction 12 Wet THC ppm THC (ppm) 13 Exhaust Flow Rate scfm Exhaust flow rate (scfm) To compute the % of PEMS flowmeter maximum flowrate we will need to know what size flowmeter was used for each NTE event. Enter either 3, 4, or 5 to represent the following flowmeters and maximum flow rates: 3 = 3 inch EFM with maximum flow rate = 600 scfm 4 = 4 inch EFM with maximum flow rate = 1100 scfm 5 = 5 inch EFM with maximum flow rate = 1700 scfm 15 Speed rpm Engine speed (rpm) 16 Low Speed, nlo rpm To compute the % of normalized speed we will need nlo and nhi for the engine computed as follows: nlo (rpm) = lowest speed below max power at which 50% max power
occurs

18 Fuel Rate L/sec Fuel rate (L/sec) To compute the % of maximum fuel rate we will need the max fuel rate of the engine for each NTE event. Max fuel rate (L/sec) 20 Derived Torque N

· m Torque (N

· m) 21 Peak Torque N

· m To compute the % of maximum torque we will need the peak torque of the engine for each NTE event Peak torque (N

· m) 22 Flow-weighted Average PM Concentration for Methods 1 & 2 μg/mol Flow-weighted average PM concentration, flow-weighted by the exhaust flow. Values were calculated based on a predictive model developed using transient and steady-state experimental data. 23 Flow-weighted Average PM Concentration for Method 3 μg/mol Flow-weighted average PM concentration, flow-weighted by the exhaust flow. Values were calculated based on a predictive model developed using transient and steady-state experimental data. 19 Max Fuel Rate L/sec nhi (rpm) = highest speed above max power at which 70% max power occurs 14 Flowmeter Diameter 3, 4, or 5 (inches)

17 High Speed, nhi rpm14 SwRI Project No. 14936.12 2.6 E RROR SURFACE GENERATION
During the discussions held at several Steering Committee meetings, 33 error surfaces were identified and considered for inclusion in the Monte Carlo simulation model. 25 of these error surfaces were the same surfaces used in the gaseous emissions in

- use testing program. Of the remaining eight error surfaces, two were discarded (Delta PM EMI/RFI and Delta PM Vibration). The six new error surfaces for the PM program were PM Steady

- State (SS), PM Transient, PM Atmospheric Pressure, PM Ambient Temperature, Torque Engine Manufacturers and Fuel Rate Engine Manufacturers. This resulted in a final total of 31 erro r surfaces that were incorporated into the Model. These individual error surfaces encompassed a wide variety of

error sources.
Table 8 lists th e error surfaces examin ed during the study with the surfaces excluded by the Steering Committee designated in italics. All remaining ones were implemented in the simulation model . Each error surface was assigned a number for easy identification.15 SwRI Project No. 14936.12 TABLE 8 . ERROR SURFACES FO R MONTE CARLO SIMULA TION

For each of the measurement errors defined in Table 8
, an error surface was created and used in the Monte Carlo simulation. Each error surface represented an additive error

— or a subtractive error if the sign wa s negative

— relative to the referen ce parameter value to which it was applied. Figure 4 through Figure 6 serve a s a hypothetical example of ho w these error surfaces were Component # Test Source Error Surface Description 1. Delta PM 1 Engine Dyno Delta PM SS AVL, Horiba and Sensors 2 Engine Dyno Delta PM Transient AVL, Horiba and Sensors 3 Environ Delta PM EMI/RFI Deleted by Steering Committee 4 Environ Delta PM Atmospheric Pressure AVL 5 Environ Delta PM Ambient Temperature AVL 6 Environ Delta PM Vibration Deleted by Steering Committee 2. Delta CO 7 Engine Dyno Delta CO SS Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 10 Environ Delta CO Atmospheric Pressure Same as Gaseous Study 11 Environ Delta CO Ambient Temperature Same as Gaseous Study 3. Delta NMHC 13 Engine Dyno Delta NMHC SS Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% NMHC = 0.98*THC 14 Engine Dyno Delta NMHC Transient Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 16 Environ Delta NMHC Atmospheric Pressure Same as Gaseous Study 17 Environ Delta NMHC Ambient Temperature Same as Gaseous Study 19 Environ Delta Ambient NMHC Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 4. Delta Exhaust Flow 20 Engine Dyno Delta Exhaust Flow SS Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 21 Engine Dyno Delta Exhaust Flow Transient Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 22 Engine Dyno Delta Exhaust Flow Pulsation Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 23 Engine Dyno Delta Exhaust Flow Swirl Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 25 Environ Delta Exhaust EMI/RFI Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 27 Environ Delta Exhaust Temperature Same as Gaseous Study 28 Environ Delta Exhaust Pressure Same as Gaseous Study 5. Delta Torque 29 Engine Dyno Delta Dynamic Torque Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 30 Engine Dyno Delta Torque DOE Testing (Interacting Parameters Test) Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 31 Engine Dyno Delta Torque Warm-up (Interacting Parameters Test) Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 32 Engine Dyno Delta Torque Humidity/Fuel (Independent Parameters Test) Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 34 Engine Dyno Delta Torque Interpolation Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 35 Engine Manuf Delta Torque Engine Manuf New 6. Delta Fuel Rate 42 Engine Manuf Delta Fuel Engine Manuf New 7. Delta Speed 43 Engine Dyno Delta Dynamic Speed Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 8. Delta Fuel Rate 44 Engine Dyno Delta Dynamic Fuel Rate Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th%

9. Delta CO 2 45 Engine Dyno Delta CO 2 SS Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 46 Engine Dyno Delta CO 2 Transient Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 49 Environ Delta CO 2 Ambient Temperature Same as Gaseous Study Measurement Error Surfaces and Deltas Used in BSPM Calculations16 SwRI Project No. 14936.12 created for every measurement error. Details on the construction of each error surface used in the simulation are provided in Section 2.6

. The example illustrated in Figure 4 through Figure 6 represent s the error surface for steady

- state bias and precision PM c oncentration errors for an individual PEMS unit . The plots sho wn correspond to hypothetical PM emissions concentration data acquired in the laboratory

.

Figure 4 PEMS vs. Laboratory Nominal Results

was constructed from raw data a cquired from steady

- state engine lab tests with the PEMS at repeat testing at various concentration levels (PM μg/mol ). The plot pools all bias and precision errors for the PEMS tested f or all steady

- state modes. A nominal target of 1010 repeat measurements of PM

Figure 4 were taken on each PEMS unit for each value of the corresponding average lab PM values (i.e., lab nominal value). The 11 0 PEMS measurements were plotted against the corresponding measurements using lab oratory equipment. Shown in are th e 5 th , 50 th

, and 95 th percentiles correspondin g to the distribution of these 11 0 observations u sing the PEMS at each average PM concentration level (note that the distribution of data a t each PM level may not represent a normal distribution). Since the 50 th

Figure 4 percentile s do not lie on the dashed (diagonal) line of perfect agreement, the data suggest that there is a bias error between the PEMS and lab results. In essence this graph summar izes the statistical distribution measured by the PEMS at each concentration level sampled. The example plot in shows only 6 discrete average PM concen tration levels (ranging from 10

- 6 0 μg/mol ). However, the actual number of discrete concentration levels was determined using the total number of operating conditions actually run for al l the tests on the engine . In the s ection on Steady

- State R epeat Engine Testing and Error Surfaces it is reported that 66 operating modes condition s from an initial number of 8 0 8 0 operating conditions were selected for con struction of the steady

- state PM error surface. From these 6 operating modes several discret e PM concentration levels were defined which were used in the error surface plot s for the Monte Carlo simulation.6

.17 SwRI Project No. 14936.12 FIGURE 4 . ERROR SURFACE CON STRUCTION: PEMS VS. LABORATORY RESULTS

Figure 5 (PEMS

– Laboratory) Deltas vs. Lab
illustrates the “error band” measured during testing. This plot was created by first subtracting the individual “lab nominal” PM value from th e corresponding individual PEMS PM measurement for each test run. This difference was defined as the “delta” error. Second, these “PEMS

- Laboratory” delta errors were poole d at each average lab nominal PM value to obtain the 5 th , 50 th , and 95 th Figure 5 percenti le values displayed in

. Therefore, the plot represents the average PM lab nominal at 6 discrete concentration levels versus the percentiles of the delta errors computed from the PEMS and laboratory individual test results. Error Surface for SS PM Concentration

18 26 38 45 56 69

13 24 36 42 53 64

8 19 33 39 51 61

0 10 20 30 40 50 60 70 80

0 10 20 30 40 50 60 70 80 PM μg/mole (lab,mean)PM μg/mole (PEMS)

diagonal 95th percentile 50th percentile (median) 5th percentile18 SwRI Project No. 14936.12 FIGURE 5 . ERROR SURFACE CON STRUCTION: (PEMS

- LAB) VS. LABORATORY RESULTS

This step normalized the plot in Variability Index vs. (PEMS

– Laboratory) Deltas and Lab Nominal
Figure 5 using what is called a “variability index (i c )”. This index represented the value randomly drawn by the Mont e Carlo simulation in order to select a given error level. For the 5 th and 95 th percentile of the truncated normal i t was allowed to vary from

– 1 to +1 , respectively . The likelihood of “i c ” being any value between

– 1 through +1 was specified by a “probab ility density function (PDF)” assigned to i c. In the case of this example, i c. was assumed to vary according to a standard normal (i.e, bell

- shaped) distribution during the Monte Carlo simulations. This was because it was beli eved that the distribution o f PM errors due to steady

- state bias and precision would be centered about the 50 th

Figure 5 percentile of the full range of conditions measured. Each set of data for each lab “setpoint” average (i.e., lab nominal value) in was normalized by aligning the corresponding 5 th Figure 5 percentile error from

with i c = - 1, the 50 th percentile error with i c = 0, and the 95 th percentile error with i c Error Surface for SS PM Concentration

8

6 8

5 6 9

3 4 6

2 3 4

-2 -1 3

-1 1 1

-4 -2 0 2 4 6 8 10

0 10 20 30 40 50 60 70

PM μg/mole (lab,mean)PM μg/mole (PEMS) - PM μg/mole (lab)

95th percentile 50th percentile (median) 5th percentile

= +1.19 SwRI Project No. 14936.12 These values were then plotted in Figure 6 , where the y

- axis is the variability index, the x

- ax is is the average lab nominal PM value, and the z

- axis is the delta PM value. Notice that, when using this normalization approach, the 5 th , 50 th , and 95 th Figure 5 percentile values remain equivalent between and Figure 6

. This development of the error surfaces from the lab data was the procedure used in the gaseous emissions in

- use testing program.

FIGURE 6 . ERROR SURFACE CON STRUCTION: ERROR AT VARIABILITY INDEX FOR 5 TH AN D 95 TH For the PM measurement allowance program it was decided by the Steering Committee to expand the tails of the truncated normal distribution in the error surface formation in order to allow larger PM deltas to be sam pled during the Monte Carlo simulation. Instead of truncating the lower tail at the 5 PERCENTILES VS. LABO RATORY RESULTS

th percentile, it was moved to the 1 st percentile. Likewise on the upper tail, the 95 th percentile was moved to the 99 th 8 6 8 6 9

3 4 6 2 3 4

- 2

- 1 3

- 1 1 1

- 1.5

- 1 - 0.5 0 0.5

1 1.5

0 10 20 30 40 50 60 70ic_SS_PM

PM μ g/mole (lab,mean) Error Surface for SS PM Concentration

95th percentile 50th percentile (median) 5th percentile Error Surface: z

- axis = ΔSS_PM_μg/mole

5

percentile. Since the original truncated normal distribution was defined by a mean = 0 and a standard deviation = 0.60795, the resulting indices20 SwRI Project No. 14936.12 corresponding to the 1 st and 99 th Figure 7 percentiles are

- 1.4143 and +1.1413, respectively, as illustrated in

.

FIGURE 7 . TRUNCATED NORMAL DISTRIBUTION PERCENT ILES

Finally, the 5 th and 95 th percentile (PEMS

– LAB) delta values at each lab nominal value were used to compute the corresponding values of the truncated normal for the 1 st and 99 th percentiles. To redefine the error surface delta at the 1 st percentile, the standard deviation for t he normal distribution below the 50 th

Using the mean = 50 percentile is defined as follows:

DD DD 1 = DD VV 50 h − DD VV 5 h 1

. 6449

th percentile delta and the standard deviation computed abov e, the 1 st - 3

- 2

- 1 0 1 2 3 

### Monte Carlo Truncated Normal Sampling Distribution

N(0,0.60795) - 1.4143

Area= 0.05 Area = 0.01 1.0

- 1.0

5th 95th 1.4143

1st 99th Percentiles **→**

percentile can be found using the Excel NORMINV function: NORMINV(0.01,mean,standard21 SwRI Project No. 14936.12 deviation 1st ). Similarly, to redefine the error surface delta at the 99 th percentile, the standard deviation for the normal distribution above the 50 th

Using the mean = 50 percen tile is defined as follows:

DD DD 99 h = DD VV 95 h − DD VV 50 h 1

. 6449

th percentile delta and the standard deviation computed above, the 99 th percentile can be found using t he Excel NORMINV function: NORMINV(0.99,mean,standard deviation 99th Figure 6 ). Taking the data for the error surface in and redefining it to include the 1 st and 99 th Figure 8 percentile truncated normal values results in the error surface displayed in . All the error surfaces carried over from the gaseous program that were sampled using a truncated normal were redefined at the 1 st and 99 th percentiles. Those error surfaces that were sampling using the uniform distribution remained unchanged. Error surfaces such as the one presented in Figure 6 are the error deltas the Monte Carlo simulation program used during calculation o f the BSPM emissions “with errors”

.22 SwRI Project No. 14936.12 FIGURE 8 . ERROR SURFACE CONSTRUCTION : ERROR AT VARIABILI TY INDEX FOR 1ST AND 99TH PER CENTILES VS. LABORAT ORY RESULTS

2.7 ERROR SURFACE SAMPLI NG AND INTERPOLATION
The error model used two different proba bility density functions to sample the error surfaces, depending upon whic h experimental parameter the surface represented. To sample error surfaces that were generated from the lab test results (Section on Engine Dynamometer Laboratory Testing ), and the applicable environmental test results , the model used a truncated standar d normal PDF because these tests were designed to evenly cover the full, but finite, range of engine operation and ambient conditions. To sample error surfaces that were generated from the pressure and temperature envir onmental test results (Section on En vironmental Chamber Testing ), the model used a uniform PDF because these tests were already designed to cover the typical range and frequency of the respective conditions. Both of these sampling distributions

are depicted in Figure 9

. 10.1 6.8 8.8 7.2 11.1

3 4 6 2 3 4

- 4.1

- 3.1 1.8

- 2.2 0.2

- 0.2

- 2 - 1.5

- 1 - 0.5 0 0.5

1 1.5

2

0 10 20 30 40 50 60 70ic_SS_PM

PM μ g/mole (lab,mean) Error Surface for SS PM Concentration

99th percentile 50th percentile (median) 1st percentile Error Surface: z

- axis = ΔSS_PM_μg/mole

6.223 SwRI Project No. 14936.12

FIGURE 9 . TRUNCATED STANDAR D NORMAL AT 1 ST AND 99 TH

When using the truncated standard normal PDF (see PERCENTILES AND UNIFORM PROBABIL ITY DENSITY FUNCTION S

Figure 9 ), the Monte Carlo model sampled normal deviates that ranged between

- 1 .4143 and +1 .4143

. These were used as the i c values defined in the section on Error Surfaces

. Similarly, th e pressure and temperature environmental tests used a uniform PDF to sample test time, from which calculated errors were used. All temperature error surfaces related to the four emissions were sampled uniformly from 1 to 1080 minutes while the error surfaces related to the pressure were sampled uniformly from 1 to 720 minutes. Exhaust flow error surface for temperature was sampled uniformly from 1 to 478 minutes while the exhaust flow for pressure was sampled uniforml y from 1 to 360 minutes. The errors from all the other tests were aligned with the truncated standard normal PDF such that each of the 50 th percentile error values at each of the tested signal magnitudes was centered at the median (i.e. , 0 value) of the P DF, and the 1st and 99 th percentile error values at each of the tested signal magnitudes were aligned with the extreme negative (i c = - 1 .4143 ) and positive (i c

Each error surface was sampled along its i = +1 .4143 ) edges of the PDF, respectively.

c axis (y

- a xis) once per trial for a reference NTE event simulation. Hence, every error surface had a separate randomly selected i c for each trial. Since each reference NTE event contained second

- by

- second parameter data, except for PM for the Sensors PPMD and Hori ba TRP M PEMS, the error surface was sampled at a given i c on the y

axis a nd at the several selected parameter value s on the x

- axi s that corresponded to each second Probability Density Functions for Sampling Error Surfaces Once Per NTE Event

- 1.41 0.00 1.41

0 1 2 3 4 5 Relative Probability i c Lab Tests, Normal, SD=0.60795, truncate @

- 1.4143 & 1.4143 Environmental Tests, Uniform

Note: A non

- truncated normal distribution with SD=0.60795 has P values of 0.01 and 0.99 at i c = - 1.4143 and i c =+1,4143 respectively.24 SwRI Project No. 14936.12 of the reference NTE event. The sampled error value was determined for the given second and parameter along the error axis (z

- axis) at the intersection of the i c value and the p arameter value from the reference NTE event. This was accomplished by taking each second in the reference NTE event and finding the two adjacent x

- axis values from the error surface between which to linearly interpolate to obtain the error surface x

- value. Each second in the reference NTE event was linearly interpolated with the same i c

Figure 10 value for a particular trial at the error surface x

- value. If any of the sampled lab n ominal values (PM, NMHC, CO, Speed, Fuel Rate, etc.) exceeded the upper or lower limits of the parameter error surface, the value of the closest endpoint of the error surface was assigned to them.
depicts an example of the error surface sampling using a steady

- state PM error surface containing 10 lab nominal PM x

- axis values. For this particular trial, the randomly selected i c is

- 0.5. The e xample reference NTE event is noted by the symbol ‘*’ and it plotted at i c = - 0.5 for each second in the NTE event.25 SwRI Project No. 14936.12 FIGURE 10 . STEADY

- STATE PM ERROR SURFA CE FOR AVL WITH EXAM PLE SAMPLING FOR A REFER ENCE NTE EVENT

2.8 BRAKE

- SPECIFIC EMISSIONS C ALCULATIONS
Errors from Sections 4, 5, and 6 were combined by adding all of the sampled error s once per trial for each reference NTE event simulation . For example, in order to assess the errors in PM concentration by calculation Method #1, several error surfaces were sampled and added to the corresponding parameter in the Method #1 calculation and the resulting BSPM “with errors” was computed. The errors used in this calculation for the Horiba and Sensors are the following ( note that the corresponding error surface number s are provided in the subscript s ):

PM μg/mol ‘with errors’ = PM μg/mol_ reference + **∆** PM

 μg/mol _1 +

**∆** PM μg/mol  _2 - 2 - 1.5

- 1 - 0.5

0 0.5

1 1.5

2

0 50 100 150 200 250 300 350 400 450ic_SS_AVL

PM μg/mol (lab,nom) SS Error Surface for AVL PM Concentration

99th percentile 50th percentile (median) 1st percentile

ic =

- 0.5 NTE Event Error Surface: z

- axis = ΔSS_PM_AVL_μgmol26 SwRI Project No. 14936.12 Exhaust Flow % ‘with errors’ = Exhaust Flow % reference +

**∆** Exhaust Flow % 20 **∆** Exhaust Flow _% 21 +

**∆** Exhaust Flow % 22 + **∆** Exhaust Flow % 23 +

**∆** Exhaust Flow % 25 + **∆** Exhaust Flow % 27 +

**∆** Exhaust Flow % 28

Torque % ‘with errors’ = Torque % reference +

**∆** Torque

 % 29 + **∆** Torque % 30 +

**∆** Torque % 31 + **∆** Torque % 32 +

**∆** Torque % 34 + **∆** Torque % 35

Speed % ‘with errors’ = Speed % reference + **∆** Speed % 43

w here,

**∆** 1 ,2 = PM concentration errors due to steady

- state and transient errors,

**∆** 20,21 = exhaust flow errors due to steady

- state and transient errors,

**∆** 22,23 = exhaust flow errors due to pulsation and swirl,

**∆** 25 = exhaust flow errors due to ambient temperature
,
**∆** 27,28 = exhaust flow errors due to temperature and pressure,

**∆** 29 = torque errors due to dynamic torque,

**∆** 30,31 = torque errors due to DOE and warm

- up,

**∆** 32 = torque errors due to interacting parameters humidity and fuel,

**∆** 34,35 = torque errors due to interpolation and engine manufacturers,27 SwRI Project No. 14936.12 **∆** 43

Using the formulas for the c alculation methods , the BSPM for Method #1 was computed without errors (“ideal”) and then with all the errors applied as outlined above. = speed errors due to dynamic speed

Table 9 lists all error surfaces used by each calculation method for th e PM emissions.

TABLE 9 . ERROR SURFACES US ED FOR COMPUTING BRA KE

- SPECIFIC PM EMISSIONS BY THREE C ALCULATION METHODS Component # Error Surface Method 1 Method 2 Method 3 1. Delta PM 1 Delta PM SS    2 Delta PM Transient    4 Delta PM Atmospheric Pressure    5 Delta PM Ambient Temperature    2. Delta CO 7 Delta CO SS   10 Delta CO Atmospheric Pressure   11 Delta CO Ambient Temperature   3. Delta NMHC 13 Delta NMHC SS   NMHC = 0.98*THC 14 Delta NMHC Transient   16 Delta NMHC Atmospheric Pressure   17 Delta NMHC Ambient Temperature   19 Delta Ambient NMHC   4. Delta Exhaust Flow 20 Delta Exhaust Flow SS   21 Delta Exhaust Flow Transient   22 Delta Exhaust Flow Pulsation   23 Delta Exhaust Flow Swirl   25 Delta Exhaust EMI/RFI   27 Delta Exhaust Temperature   28 Delta Exhaust Pressure   5. Delta Torque 29 Delta Dynamic Torque   30 Delta Torque DOE Testing (Interacting Parameters Test)   31 Delta Torque Warm-up (Interacting Parameters Test)   32 Delta Torque Humidity/Fuel (Independent Parameters Test)   34 Delta Torque Interpolation   35 Delta Torque Engine Manuf   6. Delta Fuel Rate 42 Delta Fuel Engine Manuf  7. Delta Speed 43 Delta Dynamic Speed   8. Delta Fuel Rate 44 Delta Dynamic Fuel Rate  9. Delta CO 2 45 Delta CO 2 SS   46 Delta CO 2 Transient   49 Delta CO 2 Ambient Temperature  28 SwRI Project No. 14936.12 2.9 CONVERGENCE AND NUMB ER OF TRIALS
Since the Test Plan did not include a provision for convergence criteria, the Steering Committee was tasked to develop a convergence method. The main goal was to define how many simulation trials at a given reference NTE event were required to estimate the 95 th percentile BSPM emission differences with a given precision. Although the Crystal Ball software contained precision control options, the method used to compute a confidence interval on percentiles was based on an analytical bootstrapping method which was not adequately documented. Thus, an independent convergence method was proposed and accepted by the Steering Committee.

A nonparametric statistical technique [3] was proposed which defined a 90% confidence interval for the 95 th

1. Run the Monte Carlo simulation for N trials. percentile of the BSPM emissions differences for an individual reference NTE simulation. If the width of the 90% confidence interval was less than 1% of the BSPM emissions threshold, then convergence was met. The following steps define the convergence method:

2. Order the BS emissions differences from smallest to largest. 3. Identify the trial number at the lower end of the 90% confidence interval

n lower 

### N N * 05

. 0 * 95

. 0 645

. 1 * 95

. 0

**−** =

4. Identify the trial number at the upper end of the 90% confidence interval

n upper 

### N N * 05

. 0 * 95

. 0 645

. 1 * 95

. 0 **+** =

5. Compute (BSPM differe nce value at n upper )

– (BSPM difference value at n lower 6. If the result in (5) < 1% of the BSPM emissions NTE threshold then convergence is met. ).

7. The BSPM threshold was defined as 0.02 g/hp

- hr. Thus, 1% of the threshold was 0.0002 g/hp

- hr.

The Screening C ommittee agreed to the proposed convergence criteria outlined above. During the initial simulation runs for 20 reference NTE events
, convergence was not met at the 1 percent criteria level until 60,000 trials were run. This only applied to the AVL PEMS unit at each of the three calculation methods. The Horiba and Sensors units only reached convergence at the 1 percent criteria for approximately half of the 20 reference NTE events simulations. Upon examination of the distributions of the delta BSPM emis sions generated from the simulations, some of the distributions were positively skewed which would make convergence very difficult at the 1 percent level. T his information was presented to the Steering Committee wherein a29 SwRI Project No. 14936.12 decision was made to relax the co nvergence level to 2 percent or higher, depending on the outcomes of the simulations.
In summary, the 141 reference NTE events were run at 40,000 trials and convergence was checked. If the width of the confidence interval on the 95 th

2.10 SIMULATION OUTPUT percentile delta BSP M emission was approaching 2 percent of the threshold, then the simulation was continued for up to 65,000 trials.

During the simulation of a reference NTE event , differences between the BSPM emissions “with errors” and the ideal BSPM emis sions were obtained by each o f the three PEMS model units and each of the three applicable calculation methods. These differences w ere computed thousands of times (once per trial) until the model converged

. Then the 95 th percentile difference value was determined for each reference NTE event’s distributions of BS differences for the PM emission s for all three PEMS units and applicable calculation method s

. The output from the Crystal Ball simulation for each reference NTE event was saved in two separate Excel files: an EXTRACT and a REPORT file. The EXTRACT file contained descriptive statistics on all differences computed for BSPM emissions by all three calculation methods, percentiles (0%, 5%, 10%,...95%, 100%) of the diff erences in BSPM emissions, sensitivity data for all error surfaces, and differences in BSPM emissions computed at each trial in the simulation.
The REPORT file contained a summary of the differences in the BSPM emissions for all three PEMS units and applic able calculation methods including descriptive statistics, the number of trials, a frequency histogram of the differences in BSPM emissions, and percentiles (0%, 5%, 10%,...95%, 100%) of the differences in BSPM emissions. Also included were descriptive
stat istics on each i c

2.11 STEP

- BY

- STEP SIMULATION EXAM PLE distribution sampled for each error surface. Lastly, sensitivity charts for the differences in BSPM emissions for the three PEMS unit and applicable calculation methods were stored. These charts provided information on how much each err or surface influenced the differences computed between the BSPM emissions “with errors” and the ideal BSPM emissions.
A more detailed description of the Crystal Ball output files can be found in Appendix B.

In order to clari fy the simulation process the following step

- by

- step summary is provided. This example assumes that a single reference NTE event was simulated for the BSPM difference computations. Figure 11 provides an overview of the simulation process.30 SwRI Project No. 14936.12 FIGURE 11 . OVERVIEW OF MONTE CARLO SIMULATION FOR BSPM

STEP 1 Enter the reference NTE input parameters into the Monte Carlo (MC) simulation model. The se include the emissions concentration s , exhaust flow, torque, speed and fuel rate data used in all three calculation methods.

STEP 2 Compute the “ideal” BS PM by all three PEMS model unit and applicable calculation methods from the reference NTE event.

S TEP 3 Set

- up the Monte Carlo simulation parameters in Crystal Ball. An Excel spreadsheet model was developed for use with Crystal Ball MC software for error analysis of brake

- specific emissions. Crystal Ball is a graphically

- oriented forecasting and simulation software that runs on Microsoft **** Windows and Excel. The simulations run in this program used Crystal Ball Version 11.1.1 and were run on PCs configured with a Pentium 4 CPU, 3.39 GHz, 3.50 GB RAM, 232 GB hard drive and Windows XP operating syst em. Microsoft **** Excel 2003 SP was the spreadsheet software.31 SwRI Project No. 14936.12 The options exercised in running Crystal Ball included the following:

**•** Number of trials = 40,000

o If convergence was not met at the 2% criteria then # trials = 65,000

**•** Monte Carlo sampling method wi th random initial seeds

**•** Normal speed run mode

**•** Suppress chart windows (fastest run time)

The Excel spreadsheet is in a modular structure following the specified model outline, and it makes provisions for the three identified calculation modules. Input cel ls to the model are clearly identified to facilitate any revisions that may become necessary for users who want to exercise the model with other Monte Carlo software such as @Risk or newer versions of Crystal Ball . The spreadsheet was tested with controlle d test cases of simplified input distributions with the Crystal Ball add

- on to confirm correct model implementation in accordance with this test plan. At least one typical analysis was run as an additional confirmation, and two independent checks were mad e on the ideal emissions by other SwRI staff. A complete description of the spreadsheet computations is contained in Appendix C

STEP 4 Execute a single MC trial by randomly generating a separate i c for each error surface used in the three calculations.

STEP 5 For each second in the reference NTE event, interpolate the **∆** error for all error surfaces at the input parameter values and the randomly generated i c Figure 12

. illustrates all the error surfaces available and where the corresponding **∆** errors are added. The numbers in parentheses represent the error surface number in the Monte Carlo simulation.32 SwRI Project No. 14936.12 FIGURE 12 . ERROR SURFACES IN CLUDED IN MONTE CARLO SIMUL ATION

STEP 6 Compute one BSPM “with errors” for the given MC trial by adding all the **∆** error values to the reference NTE data and then calculating the BSPM by all three PEMS units and applicable calculation methods.

STEP 7 Compute BSPM difference for the current trial:

BSPM emission “with errors”

– “Ideal” BSPM emission

STEP 8 Environ
chambers

Engine dyno

Environ
chambers Engine dyno

PM **∆** PM SS_ppm (1) **∆** PM TR_ppm (2)

**∆** PM press_ppm (4) **∆** PM temp_ppm (5)

**Ε** xhflow **∆** Exhflow SS (20) **∆** Exhflow TR (21) **∆** Exhflow pulsation (22) **∆** Exhflow swirl (23)

**∆** Exhflow EMI/RFI (25)

**∆** Exhflow temp (27)

**∆** Exhflow pressure (28) Speed **∆** Speed Dynamic (43)

**∆** Fuel Rate Dynamic (44) Fuel Rate

Torque **∆** Torque Dynamic (29)

**∆** Torque warmup (31)
**∆** Torque humidity (32)

**∆** Torque engine

- manuf (35)Engine dyno

**∆** F uel Rate engine

- manuf (42) Engine dyno

CO **∆** CO SS_ppm (7)

**∆** CO press_ppm (10) **∆** CO temp_ppm (11)

Engine dyno

NMHC **∆** NMHC SS_ppm (13) **∆** NMHC TR_ppm (14)

**∆** NMHC press_ppm (16) **∆** NMHC temp_ppm (17)

**∆** NMHC ambient (19) Engine dyno CO 2 **∆** CO2 SS_ppm (45)
**∆** CO2 TR_ppm (46)

**∆** CO2 temp_ppm (49) Environ
chambers

Environ
chambers **∆** Torque DOE (30)

**∆** Torque interpolation (34)

Repeat Steps 4

- 7 until the number of trials is met.33 SwRI Project No. 14936.12 STEP 9 Check the differences in BSPM for all three PEMS units and applicable calculation methods to be certain that the convergence criteria are met. If convergence is met for all three calculation methods, continue to Step 10. Otherwise, return to Step 4 and run the Monte Carlo simulation for an additional 25,000 trials until the total number of trials i s 65,000.

STEP 10 Select the 95 th percentile from the distribution of BSPM differences for each of the three PEMS units and applicable calculation methods. Store the ideal BSPM and the 95 th percentile BSPM differences for computing the measurement allowa nce.

STEP 11

2.12 MEASUREMENT ALLOWANC E GENERATION Repeat Steps 1

- 10 for each reference NTE event.

The generation of a set of measurement allowances represented the final outcome of this program. The Test Plan provided a methodology by which all of the data from the millions of Model simulation runs would be collected and analyzed statistically, in ord er to generate a set of potential measurement allowances for each PEMS model unit, one for each of the calculation methods. The Test Plan then outlined a specific method by which the final set of allowances would be chosen from among deltas generated for each of the three calculation methods. The assumption made by the Test Plan, was that the final outcome of all previous efforts would be a set of validated potential measurement allowance values for each PEMS unit. Each potential allowance was expressed as a percentage of its associated BSPM NTE threshold. These measurement allowances were computed by a regression method or a median method as described below.

2.12.1

This method involved determining the correlation between the 95 Regression Method

th percentile d ifferences versus the ideal emission values for the reference NTE dataset. For each combination of PEMS units and calculation method , a least squares linear regression of the 95 th percentile differences versus the ideal emissions results was computed. If the R 2 value from the regression model was greater than 0.85 and the SEE ( standard error of the estimate or root

- mean

- squared

- error) was less than 5 percent of the median ideal BS emissions, then the linear regression equation was used to determine the me asure ment allowance for that PEMS unit and calculation method. To determine the measurement allowance the NTE threshold was used to predict the measurement allowance from the regression model. The measurement allowance was then expressed as a percentage of the NTE BSPM threshold value (0.02 g/hp

- hr).34 SwRI Project No. 14936.12 2.12.2
If the linear regression did not pass the aforementioned criteria for the R Median Method

2 and SEE statistics, then the median value of the 95 th percentile differences from the 141 reference NTE events was used as the single measurement allowance for a combination of emissions and calculation method. The measurement allowance was then expressed as a percentage of the NTE threshold value.

After all 95 th

The anticipated outcome from the model runs, analysis, and validation efforts can be represented as a table similar to the one shown in percentile distributions were evaluate d, there were seven measurement allow ances corresponding to the combinations of the three PEMS units and the applicable calculation methods.

Next , the calculation method with the minimum normalized PM value will be chosen and the corresponding normalized PM value wil l be selected as the best measurement allowance for PM, assuming it validates. This PM measurement allowance would be the very last value added to the actual brake

- specific NTE PM threshold for a given engine, based on actual family emissions limit, milea ge, model year, etc. Note that if any measuremen t allowance was determined to have a value less than zero, then that measurement allowance was set equal to zero.
The BSPM NTE threshold used for this program was 0.02 g/hp

- hr. This NTE threshold was determ ined by EPA and approved by the Steering Committee during the generation of the Test Plan.
These threshold values are of critical importance to the program, as they provide the basis for the scaling of measurement allowances, the assessment of model conver gence, and a variety of other calculations performed during this program. The general philosophy of the Test Plan was to determine measurement allowances based on errors at these emission levels, especially in the case of any errors that scaled with emiss ion level.

Table 10 . The table illustrates both the model outcome, and the process for selecting the final measurement allowance values for the AVL PEMS.35 SwRI Project No. 14936.12 TABLE 10 . EXAMPLE OF SELECTI ON OF MEASUREMENT AL LOWANCE AT 0.02 G/HP

- HR NTE THRESHOLD FOR THE AVL PEMS
All owance at R espective NTE threshold (%)

Calc. Method  Method 1

Exhaust Flow Torque

- Speed Method 2

Exhaust and Fuel Flow Torque

Speed Method 3 Fuel Flow Torque

Speed

BS PM 38 % 18 % 20 %

Selected M ethod  Exhaust and Fuel Flow Torque

- Speed M ethod

The intent of the final selection process was to choose the smallest of the three normalized PM values for the final measurement allowance. At that point, the percentages given for the chosen calculation method would be applied to the BSPM NTE threshold value in order to generate the final additive, brake

- specific measurement allowances.
An implicit assumption of the process, as described in the Test Plan, was that the values produced by the model for all three calculation methods would be successfully validat ed. In the event that this did not occur, it would be necessary for the Steering Committee to determine a valid alternate course of action, in order to determine the final measurement allowance values.
The final model run and the selection and generation of measurement allowances are described fully in Section 6.2 of this report, including the final allowances approved by the Steering Committee.

2.13 MODEL VALIDATION

For reasons discussed earlier, the measurement allowances were generated using a Monte Carlo computer model. As with all simulations, it is vital that such a model be validated through comparison with real experimental data. In this case, the Measurement Al lowance model needed to be validated against a data set generated through actual in

- use field testing. Because the model generates an incremental error in comparison to a Laboratory Reference, a suitable in

- use reference measurement was needed for compari son to the PEMS measurements. The Steering Committee determined that the CE

- CERT Mobile Emission Laboratory, operated by the University of California

- Riverside, would be an appropriate reference for validation of the model

- based in

- field testing.36 SwRI Project No. 14936.12 To ensu re that the validation was not disturbed by some inherent bias between the SwRI Reference Laboratory and the CE

- CERT MEL validation reference, a correlation exercise was performed between the two laboratories, prior to the start of on

- road validation effor ts. The CE

CERT MEL was brought to SwRI’s laboratory facilities in San Antonio, Texas, and a side

- by

side correlation test was run. During this test, exhaust from the same test engine was alternately routed to the measurements systems of both SwRI and CE

- CERT. This was done repeatedly over the course of three days of testing. The data was then supplied to the Steering Committee, in order to allow for a determination to be made that correlation between the facilities was acceptable for the purposes of va lidation of the model.
After the correlation exercise was completed, a 2007 test truck wit h a Cummins engine was procured by CE

- CERT for use in this validation exercise. In addition, two Sensors PEMS used at SwRI during the program were also delivered to CE

- CERT. A third PEMS unit of the same type was provided by Sensors. The steering committee allowed Sensors to provide a similar model with some small hardware upgrades for testing. CE

- CERT then conducted a series of on

road test runs ove r various driving routes in California, which were designed to take the test truck through a wide range of environmental and ambient conditions. During these tests, simultaneous measurements were made with the PEMS and the MEL in order to generate a valid ation data set. This formed the primary validation set for the model.

Because the CE

- CERT MEL does not readily incorporate a means of direct torque measurement on a vehicle, the on

- road validation data set could not be used to validate model errors associ ated with broadcast torque.

The difference between the PEMS results and the CE

- CERT trailer results will be compared to the error predicted by the Monte Carlo model. To validate the Monte Carlo m odel
, data must be run through the model and the model resu lts must predict the actual test results within a reasonable level of accuracy.
Validation will be based on the following procedure. For each reference NTE event, the Monte Carlo model will be used to generate the 5 th and 95 th percentiles of the simulated distribution of the brake

- specific PM emission differences. In order to obtain simulations representing similar conditions to those obtained on

- road, some error surfaces may need to be suppressed in the simulations since not all of them may be applicable to the on

- road conditions. The choice of which error surfaces to suppress would need to be made by the Steering Committee.
Next, the 5 th and 95 th delta percentiles obtained from the above simulations will be separately fit to a line or curve using two ch osen methods: a linear regression procedure and a local regression (loess) technique [4] . Depending on which of the resulting two fits is best for each set of data (i.e., either for the 5 th percentile deltas or the 95 th percentile deltas )
, the resulting l ine or curve will be used as one of the lower or upper limits for the on

- road data.37 SwRI Project No. 14936.12 To determine the best fit for a given set of delta percentiles (i.e., 5 th or 95 th ), a simple regression line initially will be fit to the data. If a least squares linear regression of the 5 th or 95 th percentile deltas versus the idea l PM emission has an r 2 greater than 0.85 and an SEE less than 5 percent of the median ideal PM emissions , then the regression line will be used. If this set of criteria is not met, then a loess fit will be used

. Since a loess regression requires the selection of a smoothing parameter [5] to smooth the data, the chosen smoothness parameter should balance the residual sum of squares against the smoothness of the fit.

The on

- road de lta errors, obtained from the results of collecting data on several NTE events during on

- road operations, will be plotted on a graph containing the 5 th and 95 th

A hypothetical example of a validation plot is giv en in percentile delta limits determined from the regression fits chosen above. The graph will consi st of a plot of delta PM versus ideal PM. The number of on

- road points outside these limits will be determined and expressed as a percentage of the total number on on

- road data points. If this number does not exceed 10% of the total number of on

- road data , the simulation data will be considered to be valid.
Figure 13 and Figure 14 . The plots shown correspond to gaseous emissions concentration data that were collected in the prior PEMS study

. Figure 13 contains the 5 th and 95 th validation limits for NO X Figure 14 data determined by fitting a linear regression model to the simulated data for both limits. contains the 5 th and 95 th validation limits for NO X data determined by fitting a loess model to the simulated data for both limits

.38 SwRI Project No. 14936.12 FIGURE 1 3 . LINEAR REGRESSIO N FIT TO 5TH AND 95T H PERCENTILE DELTAS39 SwRI Project No. 14936.12

FIGURE 14 . LOESS REGRESSION FIT TO 5TH AND 95TH PERCENTILE DELTAS

Validation of the model was assessed independently for the PM pollutant for each of the three PEMS model units, an d for each of the applicable three calculation methods. A full description of the validation efforts, including the data analysis methodology and the results of PM validation for each PEMS unit by all three calculation methods is given in Section 2.4
, with the exception of the CE

- CERT on road validation testing.

3 PART 1065 PEMS AND L ABORATORY AUDIT

3.1 1065 PEMS AUDIT
Before the start of engine testing, each PEMS was r equired to pass the verifications set forth in CFR Part 1065. Because the measurement of PM is a non

- standard process there are no audits specified on the actual PM measurement, but the instruments still needed to pass the necessary flow, temperature, and pressure audits. Table 11 lists the Part 1065 audits performed on the lab and on the PEMS.40 SwRI Project No. 14936.12 TABLE 11 . SUMMARY OF PART 10 65 AUDITS Description CFR Reference Lab PEMS
Linearity 1065.307 x 1 x 2
Torque Meter 1065.310 x Pressure, Temperature, Dewpoint 1065.315 x

Fuel Flow 1065.320 x

Intake Flow 1065.325 x

CVS Verification 1065.341 x

PM Balance Verification 1065.390 x 1 Linearity for lab performed on flow meters, torque meter, pressures, and temperatures
2 Linearity for PEMS performed on flow meters

Since no analyzer verifications were performed, the most critical audits on the PEMS were the linearity verifications of flow measurements. Because all three types of PEMS use flow measurements to calculate a dilution ratio, the accuracy of the flow measurements is directly related to the accuracy of the reported PM emissions. The AVL and Horiba PEMS measure their total and dilution flow to calculate a dilution ratio while the S ensors PEMS measures the dilution and sample flow. The following equation is used to calculate the dilution ratio:

DD RR = TT FFFF FF SS FFFF FF = TT FFFF FF TT FFFF FF − DD FFFF FF

3.1.1 Horiba Flow Audits
The Horiba PEMS has four flow measurements in the system: dilution flow, DCS flow, make

- up air flow, and total flow. During typical operation all flows but the dilution flow are held constant. The dilution flow rate is varied to sample proport ionally from the raw exhaust based on changes in exhaust flow. Although the CFR requires a ten point linearity verification on all flow
measurements [ 2 ] this was not feasible on the DCS and make

- up flow, which are both held constant at 2 lpm and cannot eas ily be varied. The steering committee elected to perform a spot check on the DCS flow to ensure its accuracy and verify the filter flow which includes both the total and make

- up air flow. The filter flow is equal to the difference between the total flow an d make

- up air flow. Because the filter flow was designed to operate at a constant flow of approximately 28 lpm it was not practical or logical to verify the flow measurement over 10 even41 SwRI Project No. 14936.12 points down to zero flow as recommended in CFR Part 1065.307. After d iscussing the issue with Horiba and the steering committee it was decided to verify the filter flow over a range of plus and minus ten percent of its operating target, using eleven steps. The maximum range that the dilution flow could vary was approximatel y 21 to 31 lpm; the dilution flow was verified over this range in eleven even steps. The flow audits were initially performed with a TSI flowmeter, but this was replaced with a bubble flowmeter when it was discovered that the accuracy of the TSI flowmeter degraded as the flow deviated from atmospheric pressure. The results from the linearity ve rifications are shown in Table 12 for the Horiba PEMS.

TABLE 12 . LINEARITY VERIFICA TIONS FOR HORIBA PEM S

Horiba

- 1 narrowly missed the passing the linearity verification for total flow with a slope of 0.97. This check was repeated several times without passing. However, the error as a percent of Verification Description Intercept Slope SEE R 2

Measured 0.29 0.98 0.05 1.000 Linearity Criteria 0.31 0.98-1.02 0.62 0.99 Pass / Fail Pass Pass Pass Pass

Measured 0.07 0.97 0.03 1.000 Linearity Criteria 0.30 0.98-1.02 0.60 0.99 Pass / Fail Pass Fail Pass Pass

Measured 0.08 1.00 0.03 0.999 Linearity Criteria 0.31 0.98-1.02 0.61 0.99 Pass / Fail Pass Pass Pass Pass

Measured 0.18 0.98 0.03 1.000 Linearity Criteria 0.30 0.98-1.02 0.60 0.99 Pass / Fail Pass Pass Pass Pass

Measured 0.05 1.02 0.04 1.000 Linearity Criteria 0.31 0.98-1.02 0.62 0.99 Pass / Fail Pass Pass Pass Pass

Measured 0.01 0.99 0.04 0.999 Linearity Criteria 0.30 0.98-1.02 0.60 0.99 Pass / Fail Pass Pass Pass Pass Dilution Flow

Filter Flow Filter Flow Horiba1

Horiba2

Horiba3 Dilution Flow Filter Flow Dilution Flow42 SwRI Project No. 14936.12 point was better than 0.5 percent for all eleven points. This result is a problem with applying the linearity criteria from Part 1065 to a flow is not verified over the range from zero to full scale. Given the excellent agreement on a point

- by

- point basis, the steering committee elected to proceed without taking corrective action.
The linearity verification results for the Sensors PEMS are shown in Table 13

.43 SwRI Project No. 14936.12 TABLE 13 . LINEARITY VERIFICA TIONS FOR SENSORS PE MS
Verification Description Intercept Slope SEE R 2

Measured -0.01 1.00 0.00 1.000 Linearity Criteria 0.02 0.98-1.02 0.04 0.99 Pass / Fail Pass Pass Pass Pass

Measured 0.05 0.98 0.08 0.999 Linearity Criteria 0.09 0.98-1.02 0.17 0.99 Pass / Fail Pass Pass Pass Pass

Measured -0.04 1.01 0.01 1.000 Linearity Criteria 0.05 0.98-1.02 0.09 0.99 Pass / Fail Pass Pass Pass Pass

Verification Description Intercept Slope SEE R 2

Measured 0.01 1.00 0.01 0.999 Linearity Criteria 0.01 0.98-1.02 0.03 0.99 Pass / Fail Pass Pass Pass Pass

Measured 0.07 1.01 0.05 0.999 Linearity Criteria 0.07 0.98-1.02 0.13 0.99 Pass / Fail Pass Pass Pass Pass

Measured 0.02 1.01 0.06 0.998 Linearity Criteria 0.04 0.98-1.02 0.08 0.99 Pass / Fail Pass Pass Pass Pass

Verification Description Intercept Slope SEE R 2

Measured 0.01 1.00 0.01 0.999 Linearity Criteria 0.01 0.98-1.02 0.02 0.99 Pass / Fail Pass Pass Pass Pass

Measured 0.01 1.00 0.05 1.000 Linearity Criteria 0.07 0.98-1.02 0.14 0.99 Pass / Fail Pass Pass Pass Pass

Measured 0.01 0.96 0.01 1.000 Linearity Criteria 0.02 0.98-1.02 0.04 0.99 Pass / Fail Pass Fail Pass Pass Major Dilution Flow

Sample Flow

Major Dilution Flow

Minor Dilution Flow Minor Dilution Flow Sensors2 Sensors1

Sensors3 Sample Flow

Major Dilution Flow

Minor Dilution Flow

Sample Flow44 SwRI Project No. 14936.12 The Sensors PEMS had the capability of performing a self

- audit using 1065 criteria. An external TSI flowmeter was provided as part of the Sensors equipment and its measurements were recorded by the Sensors software to determine perform linearity verifications on the dilution and sample flows. The total dilution flow is calculated by the addition of the major and minor dilution flows, so these two measurements are audited independently. The Sensors PEMS was able to pass all but the minor dilution flow for Sensors

- 3. This verification was performed repeatedly without passing. Since the total dilution flow would stil l pass a linearity verification in this case, no further action was taken. In the case of both the Horiba and the Sensors, the absolute accuracy of the flows are important in determining the mass of PM emitted. Because the AVL system measures the soot conc entration in real time using a constant dilution ratio, only the accuracy of the dilution ratio matters when calculated the soot mass. For this reason the dilution ratio was audited instead of the total and dilution flow rates which are used to calculate t he dilution ratio. It was also not possible to vary the total flow, which is held constant during normal operation. Although the dilution ratio was maintained at a constant of 5 throughout official testing, a six point check was performed ranging from 2 to 6 in steps of 1. Table 14 shows the results for the linearity verifications for the AVL units.

TABLE 14 . LINEARITY VERIFIC ATIONS FOR AVL PEM S

The MSS dilution ratio was initially verified using both flow measurements as well as CO 2 measurements. Because the AVL PEMS had an internal CO 2 measurement it would have been possible to use a CO 2 span bottle to verify the dilution ratio, unfortunate ly it was not possible to span the CO 2 Table 14 sensor (ie a dilution ratio of 1). The results shown in were generated Verification Description Intercept Slope SEE R 2

Measured 0.01 0.96 0.01 1.000 Linearity Criteria 0.07 0.98-1.02 0.15 0.99 Pass / Fail Pass Fail Pass Pass

Verification Description Intercept Slope SEE R 2

Measured 0.03 0.97 0.02 1.000 Linearity Criteria 0.07 0.98-1.02 0.14 0.99 Pass / Fail Pass Fail Pass Pass

Verification Description Intercept Slope SEE R 2

Measured 0.05 1.02 0.00 1.000 Linearity Criteria 0.07 0.98-1.02 0.14 0.99 Pass / Fail Pass Pass Pass Pass AVL1

AVL2

AVL3 Flow Based Dilution Ratio

Flow Based Dilution Ratio

Flow Based Dilution Ratio45 SwRI Project No. 14936.12 using TSI flowmeters to measure the total and di lution flow and calculate the dilution ratio in the same manner as the PEMS. AVL

- 1 and AVL

- 2 were both unable to pass the slope criteria but were within three percent of point across the six point verification. The steering committee agreed to accept the d ilution ratio accuracy tolerance of three percent.

3.1.2 Exhaust Flow
Official linearity verifications were not conducted on the PEMS exhaust flow meters at SwRI. The steering committee decided that a calibration from the manufacturer was sufficient so long as the flow meter was within five percent of the lab during engine t esting. The three flow meters tested from Horiba and the three from Sensors were all within found to be within five percent of the lab measurement. An unofficial linearity verification was performed using the data from the steady state engine testing. Each steady state measurement was considered a data point which resulted in between 64 and 84 data points per exhaust flow meter. The linearity plots are shown in Figure 15 for PEMS

- 1, Figure 16 for PEMS

- 2, and Figure 17 for PEMS

- 3.

FIGURE 15 . LINEARITY CHECK ON PEMS

- 1 EXHAUST FLOW DURIN G STEADY STATE ENGINE TESTING 

### y = 0.977x

- 16.188 R2 = 0.998 y = 0.995x

- 15.427 R2 = 0.999

400 500 600 700 800 900 1000 1100 1200

400 500 600 700 800 900 1000 1100 1200PEMS Exhaust Flow (m3/hr, 20°C)



### Lab Exhaust Flow (m3/hr, 20 ° C)

Horiba

- 1 Sensors

- 14 6 SwRI Project No. 14936.12

FIGURE 16 . LINEARITY CHECK ON PEMS

- 2 EXHAUST FLOW DURIN G STEADY STATE ENGINE TESTING 

### y = 0.995x

- 17.619 R2 = 1.000 y = 1.021x

- 37.608 R2 = 0.997

400 500 600 700 800 900 1000 1100 1200

400 500 600 700 800 900 1000 1100 1200PEMS Exhaust Flow (m3/hr, 20°C)



### Lab Exhaust Flow (m3/hr, 20 ° C)

Horiba

- 2 Sensors

- 247 SwRI Project No. 14936.12

FIGURE 17 . LINE ARITY CHECK ON PEMS

- 3 EXHAUST FLOW DURIN G STEADY STATE ENGINE TESTING Each of the six exhaust flow meters was able to pass the standard error, slope, and correlation coefficient criteria for a raw exhaust flow measurement specific in CFR Part 1065.307 howe ver only one was able to pass the intercept criteria (Sensors

- 3). This is likely due in part to the fact that the measurements did not extend below 470 m3/hr making it more difficult to pass an intercept criteria that assumes evenly spaced data points exte nded down to zero. Treating the exhaust flow measurement during engine testing as a linearity verification was for informational purposes only.

4 ENGINE DYNAMOMETER T ESTING AND RESULTS

4.1 TESTING OBJECTIVE
The purpose of the engine dynamometer testing was to characterize the bias and precision errors of the PEMS during steady state and transient engine operation. During steady state engine operation the PEMS measurements were compared with that of the CVS filter

- based PM measurement to characterize the bias in each of the three PEMS. The transient engine testing was used to determine the precision of each PEMS by quantifying the variability of the PEMS measurement over a series of repeated transient NTE events. 

### y = 0.986x

- 13.106 R2 = 0.999 y = 0.987x + 3.218 R2 = 0.996

400 500 600 700 800 900 1000 1100 1200

400 500 600 700 800 900 1000 1100 1200PEMS Exhaust Flow (m3/hr, 20°C)



### Lab Exhaust Flow (m3/hr, 20 ° C)

Horiba

- 3 Sensors

- 348 SwRI Project No. 14936.12 4.2 EXPERIMENTAL SETUP

4.2.1 Engine and Sampling System

The test plan initially called for dynamometer testing to be performed on two different engines, but funding constraints reduced testing to a single engine. Initial test cell work was performed using a 6.4 liter light heavy

- duty diesel engine provided by Navis tar. The engine used to generate all official steady state and transient data was a 2007 Volvo MP7 provided by Volvo Powertrain. The engine had a displacement of 10.8 liters and wa s rated at 280 kilowatts ( 375 horsepower )

. The engine wa s equipped with a variable geometry turbocharger (VGT) and a water

- cooled high pressure exhaust gas recirculation (EGR) loop. The engine intake system was connected to a
test

- cell water

- cooled intercooler. The engine is shown in Figure 18

.

FIGURE 18 . VOLVO MP7 INSTALLE D IN A CVS TEST CELL

The engine wa s also equipped with a close

- coupled diesel oxidation catalyst (DOC) and a diesel particulate filter (DPF) c ombination. For the purpose of producing higher PM emission levels a bypass was created around the aftertreatment system to allow an adjustable amount of49 SwRI Project No. 14936.12 exhaust flow around the DPF. A DOC was added to the bypass so that all of the PM in the exhaust would pass through an oxidation catalyst simulating a scenario of a cracked DPF. A picture of the bypass is shown in Figure 19

.

FIGURE 19 . DPF BY PASS WITH DOC

The original stock aftertreatment was located in the main leg of the exhaust, while a separate catalyst was procured for the bypass leg. The DOC was 76 millimeters (six inches) in diameter with a length of 152 millimeters (12 inches). Three butterfly valves were placed in the exhaust system to control the amount of exhaust passing through each leg. The DPF wa s regenerated via

an exhaust fuel injection system. For all testing the bypass was open to some degree, however, the bypass leg was clos ed when active regenerations were performed on the DPF. The DPF bypass went through multiple iterations until the proper PM level was achieved during steady state testing. A PM level of 0.025 g/hp

- hr was easily obtainable during transient cycles, however i t was extremely difficult to obtain this same PM level during steady state engine operation. Table 15 lists the five different configurations of the bypass that were t ested.50 SwRI Project No. 14936.12 TABLE 15 . LIST OF DPF BYPASS CONFIGURATIONS

Iteration Pipe Diameter Inlet Probe DOC Diameter Outlet Probe

1 3 No 3 1", Upstream

2 3 No 3 1", Downstream

3 3 3 in , Upstream None 1", Downstream

4 3 3 in , Upstream 6 1", Downstream

5 (Final) 4 3 in , Upstream 6 3", Downstream

In the final configuration (iteration 4), two butterfly valves in the main leg of the exhaust were completely closed with only a one or two millimeter gap between the valve and the exhaust pipe. This not only forced a majority of the exhaust through the bypass, but significantly raised the exhaust backpressure. Based on measurements upstream and downstream of the bypass with the AVL PEMS it was estimated that well over 50 percent of the exhaust wa s routed through the bypass in the final configuration.
A diagram of the bypass system is shown in Figure 20

.

FIGURE 20 . SCHEMATIC OF ENGINE DYNAMOMETER E XPERIMENTAL SETUP51 SwRI Project No. 14936.12 The schematic is not to scale so that some distances may appear incorrectly. There are more than 10 diameters between the mixing point of the bypass and the first PEMS sampling position for the Sensors Semtech PPMD. Ad ditionally there is approximately 10 diameters in between each of the PEMS sampling locations so that any flow disturbances caused by a different pitot tube or sample probe should not affect the other PEMS. The portion of exhaust between the first and the last sampling location is insulated to minimize the cooling of the exhaust and thermophoretic deposition of particles. The last sampling probe is that of the Horiba OBS

- TRPM which is just upstream of the entrance into the CVS tunnel. The gaseous PEMS units (Sensors Semtech DS and Horiba OBS

- 2200) were used only for data acquisition and trigger signals. No PEMS gaseous emissions were recorded during this program. The Semtech DS was used with both the Sensors PPMD and the AVL MSS while the Horiba OBS

- 2200 was used in conjunction with the Horiba OBS

- TRPM.
The outlet of the aftertreatment was routed to a constant volume sampling (CVS) tunnel for emissions measurement. The CVS consists of a positive displacement pump and an upstream heat exchanger; it was mainta ined at a nominal flow rate of 3,740 m3/hr ( 2,200 scfm ) for this testing. The dilution air is extracted through a filter pack from a temperature and humidity controlled area. The portion of the CVS tunnel that is exposed to the test cell environment upst ream of the exhaust is insulated to prevent heating of the dilution air from the test cell. The particulate matter samples were extracted from the CVS as shown in Figure 20

. Intake air flow was measured by a laminar flow element (LFE) with a maximum flow of 1,700 m3/hr ( 1,000 scfm )

. The LFE was oriented so that there we re 10 diameters of straight pipe before the inlet and after the outlet to minimize flow disturbances. The fuel flow was measured by a Micro

- Motion flow meter. The addition of intake air flow and fuel flow was used to determine the exhaust flow using the equations from CFR Part 89. The exhaust flow was used to calculate the exhaust PM concentration from the CVS filter as well as a check on the PEMS exhaust flow
meters. A raw CO 2 analyzer was used in combination with the exhaust flow measurement to calculate a raw carbon balance fuel flow which wa s compared wi th the measured fuel flow and the dilute carbon balance fuel flow as a quality check.
The PM sampling system consisted of a 47mm Whatman Teflo filter, a fine metal screen backing, and a plastic filter cartridge. The total flow is operated at a nominal flow of 57 slpm (2 scfm) which results in a filter face velocity of approximately 100 cm/s. The nominal dilution flow is 28 slpm (1 scfm) resulting in a dilution ratio of 2 and a sample flow of 28 slpm (1 scfm). The system was heated to 47°C and had an approxi mate residence time of 0.8 seconds. A cyclone with a 2.5 micron cutpoint at 17 slpm was positioned just downstream of the sample probe.52 SwRI Project No. 14936.12 4.2.2 Sensors PPMD
The PPMD was the PEMS unit installed closest to the outlet of the bypass, approximately 15 diameters downs tream. An experiment was conducted to ensure the flow was fully mixed at this location as will be discussed later. The PPMD was installed in the horizontal orientation as shown in Figure 21

.

FIGURE 21 . PPMD INSTALLED IN THE TEST CELL The PPMD was equipped with a long straight pipe containing its exhaust flow meter (EFM) just upstream of the 90 degree elbow from which the sample is extracted. T he dilution in the Sensors system takes place inside the instrument. The most common installation of the PPMD is in the vertical position, although it requires only the rotation of the moisture traps to properly operate the PPMD in the horizontal orientati on. The Sensors PEMS was equipped with two stages of dilution known as MPS1 and MPS2. Although some preliminary testing was conducted using both stages of dilution, the steering committee decided to use only a single stage of dilution for all official test s. The PPMD is a proportional sampling system that varies its dilution ratio inversely with exhaust flow to maintain a minimum dilution ratio of 6 at the maximum exhaust flow rate of an engine. The PPMD measures PM using a Quartz Crystal Microbalance (QCM)
, which charges the particles using a corona needle, deposits the particle on a Quartz Crystal, and then measures the change in frequency of the crystal to determine the mass deposited. The PPMD is a batch sampling device meaning it does not report PM conc entration in real

- time but53 SwRI Project No. 14936.12 instead reports a single mass value per event. Because each crystal requires a pre and post frequency measurement to determine mass, a total of eight crystals are included to allow for continuous operation by switching crystals. Crystal sampling begins as soon as the engine enters the NTE zone and stops as soon as the engine exits the NTE zone. One of the of the eight crystals was used as a reference crystal to adjust the measurements for changes in temperature and pressure. This left up to seven crystals available for measurement although it was common to have one or two crystals not working on any given test. The timing of the samples during testing was designed around having a minimum of five working crystals available for measu rement. The PPMD was included in the measurement allowance program because inertial microbalances had already been approved for PEMS applications in 40 CFR Part 1065.

4.2.3 AVL MSS
The AVL Micro Soot Sensor (MSS, also known as the Photo Acoustic Soot Sensor or P ASS) was installed downstream of the PPMD in the middle of the vertical portion of the exhaust pipe leading to the CVS tunnel. The MSS is connected to the tunnel via a 3 meter heated sample line which was maintained at 150°C. The mixing of dilution air tak es place in the dilution box just upstream of the sample probe so that the dilute sample is transported through the sample line. The MSS is shown in Figure 22

.

FIGU RE 22 . THE AVL MSS54 SwRI Project No. 14936.12 The MSS consists of two boxes shown in the above figure. The top box is the measuring unit which contains the resonance chamber for the soot measurement. The bottom box is the conditioning unit which contains the sample and dilution pumps as well flow controllers. The dilution pump is optional as the MSS can also provide dilution air via an external input of 300 kPa of compressed air. The steering committee requested that the MSS operate using its internal dilutio n pump, since this is the way it would operate during in use testing. The MSS measures soot by heating the elemental carbon using a laser. When the soot is heated it emits a sound wave that is detected by a microphone. The MSS can report soot concentration on a 1Hz basis and uses a constant dilution ratio, which was set at 5 for all official testing. Because the MSS measures only soot and not total PM, it was included in this program as a partial participant. If both the Sensors and Horiba units could not c omplete the measurement allowance program it was to be considered for in

- use. The AVL system does not have its own gaseous PEMS; instead it sends its concentration signal to the Sensors Semtech DS where the necessary gaseous emissions and J1939 broadcast i nformation is recorded. For all testing in this program a single Semtech DS was used to record the signals from both the Sensors Semtech PPMD as well as the AVL MSS. The probe for the Horiba OBS

- TRPM is located approximately 1.65 meters (5 feet) downstream of the MSS. The Horiba system contains two separate exhaust pieces that are each about 150 mm (6 inches long). The upstream portion is a tail pipe adapter (TPA) which is a pitot tube exhaust flow measurement. The downstream pipe contained the probe for th e PM sampling. The TPA and sampling probe can be seen in Figure 23

.

FIGURE 23 . THE PROBE AND TPA FOR THE HORIBA OBS

- TRPM55 SwRI Project No. 14936.12 4.2.4 Horiba TRPM
The Horiba system is a proportional sampling device that varies its dilution ratio in the same

man ner as the PPMD. The dilution air is introduced just downstream of the probe before the heated sample line. The point of dilution can be seen in Figure 23 where the t hree stainless steel lines converge into the stainless steel cylinder. The OBS

- TRPM uses a TSI EAD (referred to here as a DCS) real time particle instrument to measure the particle concentration on a second by second basis and collects PM on a gravimetric filter simultaneously. The filter weight gain is used to provide a calibration constant to the real time particle signal and apportion the PM mass appropriately. The DCS instrument measures continuously, but the filter is designed to sample during valid NT E event operation. The filter sampling begins after five seconds in the NTE zone and will continue for a minimum sample time of 30 seconds even if the engine is no longer in the NTE zone. Because the EPA’s PM standard is based on gravimetric filter analysi s, the Horiba system was included in the program. The OBS

- TRPM is comprised of several different boxes including the heated enclosure (HE), the diffusion charge sensors (DCS), the electrical enclosure (EE), and the mechanical enclosure (ME). The HE contain s the 47mm filter holder, and the DCS is the real time particle sensor. Dilution air was provided using a commercially available oil

- less compressor. The Horiba OBS

- 2200 gaseous PEMS was used to log the ECM J1939 broadcast, measure the exhaust flow, and pr ovide an NTE trigger to the TRPM to start filter sampling. The combined TRPM

- 2200 system contains a total of 6 boxes plus an external compressor. The components of the Horiba system are depicted in Figure 24

.56 SwRI Project No. 14936.12 FIGURE 24 . THE HORIBA OBS

- TRPM AND OBS

- 2200
The TRPM used the same 47mm Whatman teflo filter, metal screen, and plastic cartridge as the CVS system. All weighing and conditioning of both the CVS and the TRPM filters was conducted in the SwRI filter weighing room. The filter weighing room is maintained at a temperature of 22 ± 1°C with a de wpoint of 9.5 ± 1°C in accordance with CFR Part 1065.190. Filters were stabilized in the weighing environment for at least 1 hour prior to both the initial and final weights. Each filter was weight was determined by the average of three weights on a scale with a resolution of 0.1 μg.

4.3 BYPASS MIXING VERIFI CATION
The flow from the DPF bypass was reintroduced into the main exhaust stream using a 76 mm (3 inch) probe facing downstream with an orifice near the tip of the probe to promote mixing. Testing was cond ucted to ensure the exhaust flow was fully mixed prior to the first sampling location, which was occupied by the Sensors PEMS. Two probe orientations were created at the spot where the Sensors sample was extracted. One of the probe orientations was paralle l to the upstream exhaust elbow and one was perpendicular to the elbow. The orientation of the AVL sample probe is shown in Figure 25

.57 SwRI Project No. 14936.12 FIGURE 25 . EXPERIMENTAL SETUP FOR MIXING VERIFICAT ION At each of the probe orientations the MSS was used to measure the exhaust soot concentration using a variable length probe that could traverse the length of the 127 mm (5 inch) exhaust pipe. There were five sample locations for each orientation as shown in Figure 26

.

FIGURE 26 . MIXING VERIFICATIO N SAMPLE LOCATIONS58 SwRI Project No. 14936.12 Each of the positions were one inch apart, with position 3,8 being at the center of the exhaust pipe. For a measurement, the probe was started at location 1, moving down to 5 and then back to 1. The probe was then moved to position 6 moving down to positio n 10 and back to position 6. The MSS measurement was recorded for 80 seconds at each position, with the average of the last 30 seconds used for comparison. The steady state modes with the highest and lowest exhaust flow rates were chosen to perform the mix ing verification, to ensure proper mixing over the entire range of test conditions.
The mixing verification was performed several times with different bypass configurations with similar results in all cases. Only the results from the final mixing verifica tion are presented since the other results from different configurations are not relevant to the data in this report. The data from the final verification is presented in Figure 27

.

FIGURE 27 . MIXING VERIFICATIO N RESULTS The data is presented as the ratio of the average concentration at each location to the centerline concentration. The data was normalized in this way not only to remove some of the engine variability but also so that the two exhaust flow conditions with different soot concentrations could be compared on the same plot. Each data point represents the average of two samples taken at each location. The highest deviation from the centerline concentration was four percent at location 10, low flow which was considered acceptable, especially given that no trends were observed in the data. A paired t

- test using equal variance was also performed on the data relative 0.0 0.2 0.4 0.6 0.8 1.0 1.2

1 2 3 4 5 6 7 8 9 10Concentration/Centerline Concentration

High Flow Low Flow59 SwRI Project No. 14936.12 to the centerline p osition for the parallel orientation using the high and low flow. Locations 6 th r ough 10 passed the t

- test, except for location 10 at low flow, although the difference in the mean value between locations 10 and 7 was less than 4 percent. The steering commi ttee was satisfied with the mixing results and directed SwRI to move forward with the program without any further modifications for the bypass setup.

4.4 PEMS LOSS CORRECTION S
Each PEMS manufacturer was given the opportunity to correct their final PM measurement to account for various particle losses encountered during the sampling process. Any loss correction had to be presented to the steering committee for approval before it was implemented in the program. Sensors and AVL both chose to implement los s corrections, while Horiba declined stating that the losses in their system were already comparable to the losses in a CVS system.

4.4.1 Sensors PPMD Loss Correction
Sensors conducted work under a separate project at SwRI to experimentally assess the losses in the PPMD. David Booker presented the results of this work along with the proposed Sensors loss corrections at the meeting on December 10 th

4.4.2 AVL MSS Loss Correction , 2008 at SwRI. The final PPMD loss correction included thermophoretic, electrostatic, and CVS loss factors. The CVS l oss correction factor was meant to estimate the particle losses in the CVS system, since this is the standard to which the PPMD is compared. Although typical loss correction factors will increase the estimated PM concentration, the CVS correction factor ac tually decreased the PPMD estimated concentration. A total CVS loss of 15 percent was estimated by Sensors based on general experience rather than specific data. The total loss correction was estimated to increase the PM concentration by 5 to 10 percent wh en including the electrostatic and thermophoretic loss factors. The proposed loss corrections were accepted by the steering committee and implemented in the Sensors PPMD post processor. All Sensors data in this report includes these correction factors unle ss otherwise stated.

The proposed AVL loss correction was presented at the meeting on November 12 th



## ( **) (**  **) (**  **) (**  **)** dt t T L t q t t mss M
ex ex soot **+ ∆**  **+ =**  

## ∫ 1 ) ( , 2008 at SwRI. The loss correction implemented by AVL was intended to correct for the thermophoretic losses in the system and is bas ed off a paper by Stratmann et al [6] . The equation for the loss correction is shown below:



## ( **) **  ** **  ** **  ****

**+ <** 

**=** otherwise 300 150 150 T if 0 )/ b(T- a T L60 SwRI Project No. 14936.12 The magnitude of the correction is temperature dependent and was estimated to be approximately 10 percent in most case s. This correction was accepted by the steering committee and implemented in the AVL Concerto post processor.
At the meeting at SwRI on December 10 th

4.4.3 AVL MSS Total PM Correction , 2008, AVL stated that their loss correction was currently capped at a maximum loss of 25 percent regardless of the calculated value. AVL requested approval to remove this limitation and allow the equations output to be the correction, regardless of its m agnitude. This change was accepted by the steering committee.

The AVL Concerto post processor includes a function that estimates the total PM based on the measured soot, exhaust temperature, and total hydrocarbon concentration, along with a number of adjustable input parameters including the volume of the catalyst, the light off temperature of the catalyst and the sulfur content of the fuel. Although this calculation was not approved for use with the official measurement allowan ce data, a portion of the AVL data was processed to examine the results from the total PM model. All AVL data presented in this report refers to the soot concentration corrected for losses unless otherwise stated.

4.5 STEADY STATE TESTING PROCEDURE
Originally the test plan called for two different engines at three different emission levels of PM: DPF out, 0.02 g/hp

- hr level, and a 0.03 g/hp

- hr level. The DPF out level is simply whatever the emissions happen to be with no bypass which was well below the 2007 sta ndard of 0.01 g/hp

- hr. The 0.02 and 0.03 g/hp

- hr levels would be set by adjusting the DPF bypass to produce the corresponding brake

- specific PM number from the CVS filter. Due to funding limitations the testing was reduced to a single engine at a single em ission level. Because 0.03 g/hp

- hr will be used for the first year of compliance testing and 0.02 g/hp

- hr is used with the following years, it was important to investigate the performance of the PEMS covering these levels . For this reason an average of the two threshold PM levels, 0.025 g/hp

- hr, was used as the target.
The objective of the steady state testing was to evaluate the bias and precision of the PEMS using 180 data points for each PEMS manufacturer. The 30 points consists of six steady state modes of engine operation (6), 10 repeats (10), one emission level (1), one engine (1), and three different PEMS units (3), 6*10*1*1*3=180. A PM steady state error surface, 

# ( **)** g SS PM mol Δ m , was developed for each PEMS manufacturer so that there are t hree steady state PM error surfaces for use with calculation methods 1 and 2. For calculation method 3, the AVL MSS will have a unique SS PM 

### Δ m based on the calculations for method 3. As mentioned previously, the Sensors and Horiba PEMS will only use methods 1 and 2. To pick six steady engine modes for steady state testing screening tests were performed using the 80 points Cummins cycle and measuring the PM levels with the AVL MSS and the TSI Engine Exhaust Particle Sizer (EEPS). The EEPS61 SwRI Project No. 14936.12 primarily provides size distribution information, but mass concentration can be inferred using an assumed density. Since the AVL is the only of the three instruments that can report a real time mass concentration without further efforts such as filter wei ghing and post processing it was chosen to perform the screening work. The Cummins cycle steps through 80 steady state engine modes as a transient cycle. The engine is stepped through 10 different speeds and eight engine loads at each of the selected speed s. The minimum speed of the cycle is the minimum NTE speed, and the minimum torque is 30 percent of the torque at the given speed meaning that the cycle effectively maps the NTE zone. Figure 28 shows the speed and torque points of the cycle.

FIGURE 28 . SPEED AND LOAD FOR THE 80 POINTS CYCLE The engine remained at each of the points for 88 seconds with a one second transition between modes. Because the speed and load were only incrementally changed in between each mode, it was assumed that each point would stabilize relatively quickly. The fac t that this cycle was used only as a method of screening also contributed to the decision to change modes quickly. Even with such short modes, the total cycle length was 2 hours. To allow for purging, and calibrating the gas analyzers, the cycle was divide d into two 40 points cycles each lasting one hour. The gas analyzers were necessary because this cycle was also used to tabulate ECM fuel rate errors for this error surface.
The PM emissions were estimated by measuring each of the 80 modes with the EEPS a nd MSS, then choosing two of the modes to perform a filter measurement and compare the ratio of the filter measurement to the EEPS and MSS as an estimate of the CVS filter BSPM at each of the 

### 0 20 40 60 80 100 120

0 20 40 60 80 100 120Engine Torque (%)



### Engine Speed (%)62 SwRI Project No. 14936.12 80 modes. Although this method is not highly accurate, it provid ed a way to quickly obtain rough estimates of the engine PM levels over a wide range of speed and torque conditions. Of the initial 80 points, only 12 were estimated to produce brake

- specific PM of greater than 0.02 g/hp

- hr. The final six modes were chosen with the intent of covering as much of the NTE zone as possible while still maintaining high BSPM levels, and a range of PM exhaust concentrations. Several iterations of adjusting the bypass, taking filter measurements, and narrowing down the number of po ints occurred before the steering committee approved the final six points for steady state testing. Originally the DPF bypass was adjusted to produce the 0.025 g/hp

- hr of PM based on a filter measurement during a short version of the NTE transient cycle. T he PM emissions at steady state were much lower at the same bypass setting. Since it was desirable to conduct the steady state testing at the same PM levels as the transient testing, it was necessary to adjust the system to allow a much greater amount of t he flow through the bypass. Six points were chosen out of the twelve to provide a range of PM concentration, exhaust flow rates, and engine operating conditions. The six points that were chosen are shown in Figure 29 along with the NTE

zone.63 SwRI Project No. 14936.12 FIGURE 29 . FINAL SIX STEADY S TATE MODES Although the MSS was used to screen the 80 points, the actual concentration at each position was verified with CVS filter measurements before selecting the points. The exhaust PM concentration was calculated by multiplying the CVS PM concentration by the C VS dilution ratio. The CVS dilution ratio was calculated by dividing the average CVS flow rate by the average exhaust flow rate. The CVS dilution ratio ranged from 3 to 7.5 resulting in an overall dilution ratio of 6 to 15 when including the secondary PM f ilter dilution.
The steady state testing was conducted as a modal transient cycle with each mode repeated twice for a total of 12 modes per cycle. 6 different cycles were created with the order of the modes randomized in each cycle. Table 3 lists the samp le order of the modes in the 6 steady state cycles. 

### 0 500 1000 1500 2000 2500

500 1000 1500 2000 2500Engine Torque (Nm)



### Engine Speed (rpm)
Lug Curve NTE S Line NTE T

- S Line NTE P

- S Line SS Points Mode 1 Mode 2

Mode 3

Mode 4 Mode 5 Mode 664 SwRI Project No. 14936.12 TABLE 16 . SAMPLE ORDER FOR S TEADY STATE CYCLE TE STING

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6
Sample 1 3 1 3 3 4 6
Sample 2 6 4 1 4 2 1
Sample 3 5 3 6 1 1 4
Sample 4 2 6 2 2 5 3
Sample 5 4 5 5 6 3 5
Sample 6 1 2 4 5 6 2
Sample 7 5 2 4 6 5 4
Sample 8 2 1 2 4 3 1
Sample 9 1 4 3 1 4 6
Sample 10 3 5 1 2 6 3
Sample 11 6 6 5 5 2 5
Sample 12 4 3 6 3 1 2

The steady state testing was conducted as a ramped modal cycle with the engine remaining at each operating condition for three minutes before the start of sampling. An external trigger from the lab was provided to each of the PEMS and the CVS filter system so that sampling would begin simultaneously for all instruments. The engine then remained at the operating condition for 5 seconds after the end of sampling to ensure that no delay in the end of sampling by any of the PEMS caused part of the transition pe riod to be captured as a sample. The engine remained at the condition for five seconds after sampling had finished to ensure all systems had finished sampling before the operating condition changed. The order of the modes was randomized and each mode was r epeated twice within a single cycle for a total of 12 data points for each cycle.65 SwRI Project No. 14936.12 Six different cycles were created, which would create a total of 72 data points. Although the target was only 60 valid points for each set of PEMS, in practice several cycles had to be repeated to collect enough valid data. An example of one of the steady state cycles is shown in Figure 30

.

FIGURE 30 . EXAMPLE OF STEADY STATE CYCLE To ensure an accurate filter weight, a filter weight gain of 100 μg was targeted for each mode. By ensuring that this amount of material is collected on the filter for each test, the weighing variability and tunnel background contributi on could be minimized. To produce a similar filter weight gain at six different steady state modes with different mass rates, the sampling time was adjusted for each mode to meet this target. The sample time ranged from 50 seconds to 245 seconds. Since the sampling time for each mode was different, the total length of time spent at each mode was different as well. The sample time for each mode for each round of the PEMS is shown in Figure 31

. 

### 0 500 1000 1500 2000 2500

0 500 1000 1500 2000 2500 3000 3500 4000Torque (Nm), Engine Speed (rpm)



### Time (sec)
Speed Torque66 SwRI Project No. 14936.12 FIGURE 31 . STEADY STATE SAMPL E TIMES Figure 32 shows the average filter weight gain for all six modes for each of the three PEMS. 

### Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6
PEMS 1 245 70 125 190 160 100 PEMS 2 240 50 100 175 90 60 PEMS 3 240 50 100 175 90 60 0 50 100 150 200 250Sample Time (sec)67 SwRI Project No. 14936.12 FIGURE 32 . CVS FILTER WEIGHT GAIN FOR STEADY STAT E TESTING Before presenting any of the PM concentration data it should be mentioned the choice of units in this work. The test plan called for the PM concentration to be calculated in terms of μg/mol. Although it is an uncommon unit for describing mass concentration it was considered for fundamental to use mol for volume rather than m3 in which case a standard reference condition must be defined. For reference, 1 mg/m3 is equal to 24.55 μg/mol. In several cases the values in mg/m3 are provided in parenthesis for reference, but all officia l data was calculated and plotted using μg/mol.
The preliminary steady state results from PEMS 1 were presented at the December 11 th

Figure 33 , 2008 meeting at SwRI. After reviewing the first set of steady state data, the steering committee felt that the concentrati ons from the six steady state points were not effectively covering the desired range. Five of the modes are clustered between 115 and 161 μg/mol ( 4.8 and 6.7 mg/m3 ) with the remaining mode at 325 μg/mol ( 13.5 mg/m3 ) . At the recommendation of the steering c ommittee the bypass setting was slightly adjusted for PEMS 2 and PEMS 3 in an attempt to fill in some of the region between 161 and 325 μg/mol . Due to a shift in the engine out PM emissions, it was possible to increase the concentration for modes 2, 3, 5, and 6 while maintaining the same levels for modes 1 and 4. In fact, the dampers were actually adjusted to flow less exhaust through the bypass indicating that the engine out PM had not only changed relatively between operating conditions but increased over all. shows the median CVS filter PM concentration for each of the three sets of PEMS. 

### 0 20 40 60 80 100 120 140

Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6CVS Filter Weight Gain (μg)



### PEMS 1 PEMS 2 PEMS 368 SwRI Project No. 14936.12 FIGURE 33

. STEADY STATE EXHAUST PM CONCENTR ATION (μG/MOL) The mode with the highest concentration, mode 2, increased up to nearly 423 μg/mol while the lowest concentration increased from 115 μg/mol on mode 5 to 132 μg/mol on mode 4. It is interesting to note that mode 5 and 6 increased in concentra tion by 87 and 64 percent, respectively while mode 4 increased by less than 2 percent. This increase in concentration came while opening the valve in the DPF leg of the exhaust, thereby increasing the flow of exhaust through the DPF. Modes 2 and 3 both sh ifted downwards between PEMS 2 and PEMS 3 without any change in the exhaust valve positions. Figure 34 shows the brake

- specific PM values as measured by the lab reference for all three sets of PEMS. 

### 0 50 100 150 200 250 300 350 400 450 500

Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6PM Concentration, CVS Filter (μg/mol)



### PEMS1 PEMS2 PEMS369 SwRI Project No. 14936.12 FIGURE 34 . STEADY STATE BRAKE

- SPECIFIC PM, CVS FIL TER (MG/HP

- HR)
T he brake

- specific PM ranged from 15.7 mg/hp

- hr to 43.5 mg/hp

- hr.

4.6 DATA YIELD DURING ST EADY STATE TESTING
The test plan called for a minimum of 10 valid data points for each mode for each PEMS, allowing for a minimum data set of 30 for each 5 th and 95 th 

### 0 5 10 15 20 25 30 35 40 45 50

Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6Brake-Specific PM, CVS Filter (mg/hp-hr)



### PEMS1 PEMS2 PEMS3

per centile delta that was generated. In reality the target of 10 data points per PEMS per mode was not met in all cases due to additional points that were invalidated by post processing software that had been updated after the testing had been completed. Whil e very few data points were removed during post processing for the Horiba and AVL PEMS, Sensors supplied SwRI with several new post processors after the completion of testing that invalidated a significant portion of the Sensors data. Since the information on what criteria would invalidate the data was not available at the time of testing, it was not possible to know how many additional tests would be required to achieve the necessary number of data points. The Sensor’s post processor was revised to include some points that were deemed valid data but had been excluded by the post processor. The final set of data for the Sensors PEMS had between 28 and 34 points per mode.70 SwRI Project No. 14936.12 FIGURE 35 . NUMBER OF VALID DA TA POINTS FOR STEADY STATE TESTI NG The total number of possible data points was either 40 or 41 depending on the mode. Samples were taken 41 times at each mode, but modes 2, 3, 4, and 6 each had one point excluded due to mishandling of the CVS filter.

4.6.1 Data Yield During Steady State Test ing

Table 17 shows the steady

- state data yield by each of the PM

- PEMS relative to the possible data yield obtained by the CVS. A total of 29 steady state cycles were conducted for all three PEMS with each of the six steady state modes repeated twice in each cycle for a total of 12 data points per cycle. 21 of these cycles were considered valid tests from the perspective of the function of the cycle command, NTE externa l trigger, filter sampling, and at least one or more of the PEMS capturing valid data. During these valid tests four data points were missed due to a mishandling of the CVS filter, but the rest were considered valid data points from the perspective of the lab measurements. Data from the PEMS was removed for a variety of reasons including problems with the datalogging, sampling, exhaust flow measurement, mechanical failures, and filter handling. Appendix A contains a complete list of the reasons data was exc luded, but several notable problems will be discussed here. 

### 0 5 10 15 20 25 30 35 40 45

Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6Number of Valid Data Points



### Horiba Sensors AVL71 SwRI Project No. 14936.12 TABLE 17 . DATA YIELD BY EACH PM

- PEMS

Possible Horiba Sensors

- NewPP Sensors

– Revised PP AVL

Mode 1 41 37 22 34 40

Mode 2 41 36 25 29 33

Mode 3 41 36 25 32 33

Mode 4 41 35 24 28 35

Mode 5 41 37 17 31 37

Mode 6 41 37 23 30 34

Total 246 218 136 184 212

The Horiba system requires an external source of compressed air capable of supplying approximately 30 lpm at 400 kPa or higher. A commercially available oil

- less compressor was provided by Horiba for use with the system. Unfortunately this compressor had a tendency to stop working on quite a few occasions during testing. Three different compressors of the same model wer e provided and each experienced this problem. It was believed that the compressor was overheating and shutting off to protect itself, although changes in the test cell temperature did not seem to influence its performance. The compressor would begin to wor k again after 10 to 15 minutes presumably once it had cooled off. If the compressor stopped working while the Horiba system was in operation, the system lost all of its dilution air once the small air tank had been depleted. This resulted in an undiluted e xhaust stream being sampled onto the filter which would quickly overload the filter at the PM concentration levels used in this work not to mention fail proportionality required of the Horiba system. If the compressor stopped working at any point during an official test, the Horiba data for that test was voided. Figure 36 shows an example of a steady state cycle where the compressor stopped working.72 SwRI Project No. 14936.12 FIGURE 36 . HORIBA SAMPLE FLOW TO THE FILTER AND DI LUTION FLOW WHILE COMPRESSOR STO PS A problem occurred with Sensors 1 involving the auto zero function of its exhaust flow meter. Every hour the Sensors system would attempt to zero the exhaust flow meter by switching the pressure transducers to ambient for a period of time less than a minu te. On the Sensors 1 the solenoid switching the pressure transducers from exhaust measurement to zero was not working properly causing the zero function to occur while pressures were being measured from the exhaust. Because the steady state cycle lasted lo nger than one hour, this would cause an erroneous exhaust flow measurement on the last two modes of each steady state cycle. Figure 37 shows an example of the EFM zer o problem during a test. 

### - 5

0 5 10 15 20 25 30 35

0 1000 2000 3000 4000Flow (slpm)



### Time (sec)
Sample Flow to Filter Dilution Flow73 SwRI Project No. 14936.12 FIGURE 37 . SENSORS EFM ZERO D URING STEADY STATE C YCLE The EFM auto zero function was disabled for the remaining tests on Sensors 1, and for Sensors 2 and 3.

4.7 ACCOUNTING FOR CVS V ARIABILITY DURING ST EADY ST ATE TESTING
The steady state data has variability due to the PEMS, CVS, and test article. By computing a paired difference for each data point, the test article variability is removed from the data. This concentration delta still contains variability assoc iated with the PEMS and the CVS. The data collected in this program does not allow for independent assessment of the PEMS and CVS variability, but by assuming a CVS variability it becomes possible to assign the remaining variability to the PEMS.

The follo wing procedure for estimating the PEMS and CVS variability was proposed by Bill Martin at the December 12 th 2008 meeting at SwRI and was accepted by the steering committee.

1. Since test article variation is the same for each individual observation by th e CVS ( x PEMS,i ) and PEMS ( x CVS ,i ) (

, , i CVS i PEMS i x x

**− =**  **∆** ), compute the paired differences,

. 

### 0 200 400 600 800 1000 1200 1400 1600

0 1000 2000 3000 4000Exhaust Flow (kg/hr)



### Time (sec)

Sensors Lab EFM Zero74 SwRI Project No. 14936.12 These paired differences (i.e., delta values or concentration deltas) contain random variation from the CVS, random variation from the PEMS, and a mean offset between CVS and PEMS (bias error).

2. Divide the entire set of delta values into j = 1 to M su bsets based on the values of χ CVSi . The data sets are not subdivided by engine operating mode or PEMS serial number, but only the level of the reference concentration.

3. For each subset, there are i = 1 to N j

j , 50 **∆** values. The median and the MAD are used as the descriptive statistics.

4. Calculate the median delta value, , the median absolute deviation of the i 

### ∆ values,

j 

### MAD , and the estimate of the standard deviation of the CVS random error, j CVSrandom 

### SD

, .

j N i i j 

### median

, 1 , 50 

### ) (

**=** 

### ∆ **= ∆** 

j N i j ij j median MAD

, 1 , 50 ) (

**= ∆** 

**− ∆**  **=**


# ∑ **∑ =** 

**= ⋅**  **=** j

j N

i i CVS j N

i i filter CVS j j CVSrandom x N L N g SD

1 ,

1 , , , 1 1 5 **μ**

**∑**

**⋅ ∑**  **=**

**=**

**=** j

j N

i i CVS N

i i filter CVS j CVSrandom x L g SD
1 ,

1 , , , 5 **μ**

j CVS j filter CVS j CVSrandom x L g SD

,

, , , 5

**⋅**

**= μ** 

where i filter CVS L

, , is the PM sample filter loading. 5 μg is the assumed CVS variability as proposed by the steering committee based on a nominal filter loading of 100 μg.75 SwRI Project No. 14936.12 For each subset j, calculate a corrected delta for each i 

### ∆ value, in subset j, as follows:

j ij j j j CVSrandom j j ij ij j CVSrandom j

MAD SD MAD SD MAD

50 , 50 2 2

, 2

, 50 2

, 2

else ) 45495

. 0 ( ) ( then 45495

. 0 if

**∆ =**  **∆ ′**  **∆ +**  **⋅ −** 

**⋅ ∆** 

**− ∆**  **= ∆**  **′ ⋅**  **>**

Note that this approach correctly passes through any significant offsets observed in the data. These offsets should be passed through even if they persist only for a subset of data, such as a given mode.

5. The entire set of corrected delta values is t hen to be used to establish the error surface for the steady

- state data. The 5 th , 50 th , and 95 th delta values are used to establish the 1 st , 50 th , and 99 th

4.8 STEADY STATE TESTING RESULTS percentile values which are the inputs to the Monte Carlo model.

All steady state data presented has already been corrected for the steady state variability as mentioned above. The steady state concentration deltas from PEMS 1 for Horiba, Sensors, and AVL, respectively are shown in Figure 38
, Figure 39 , and Figure 40

.76 SwRI Project No. 14936.12 FIGURE 38 . HORIBA

- 1 STEADY STATE PM CO NCENTRATION DELTAS

FIGURE 39 . SENSORS

- 1 STEADY STATE PM CO NCENTRATION DELTAS



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Horiba 50th % Horiba 95th % Horiba 5th %



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Sensors 50th % Sensors 95th % Sensors 5th %77 SwRI Project No. 14936.12

FIGURE 40 . AVL PM CONCENTRATI ON DELTAS FOR STEADY STATE TESTING ON PEMS 1 Each point on the x

- axis represents the median exhaust PM concentration from the CVS filter for a single mode. The y

- axis represents the 5 th , 50 th , and 95 th

Figure 41 percentile of the deltas, PEMS

– Lab. As mentioned previously there was a c lear gap in the data between 162 μg/mol and 325 μg/mol which represents a large portion of the target concentration range for a 0.025 g/hp

- hr level.

, Figure 42 , and Figure 43 show the deltas for PEMS 2 where three of the modes were be tween 208 and 267 μg/mol. 

### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

AVL 50th % AVL 95th % AVL 5th %78 SwRI Project No. 14936.12 FIGURE 41 . HORIBA

- 2 STEADY STATE PM CO NCENTRATION DELTAS

FIGURE 42 . SENSORS

- 2 STEADY STATE PM CO NCENTRATION DELTAS



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Horiba 50th Horiba 95th Horiba 5th



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 100 200 300 400PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Sensors 50th % Sensors 95th % Sensors 5th %79 SwRI Project No. 14936.12 FIGURE 43 . AVL

- 2 STEADY STATE PM CONCENTRATION DEL TAS
Horiba

- 1 and Horiba

- 2 performed similarly with mode 2 showing a significant negative bias, mode 4 showing a positive bias and the other four modes closer to zero. Sensors

- 2 showed a much greater negative bias than Sensors

- 1. Sensors

- 1 had a 50 th percentile of between 0 and

- 26 μg/mol for five of the six modes while Sensors

- 2 was between

- 32 and

- 104 μg/mol for the 50 th percentile for the same five modes. In addition, mode 2 had a 5 th percentile of

- 340 μg/mol at a reference concentra tion of 442 μg/mol. For the same mode on Sensors

- 1 the 5 th percentile was

- 157 μg/mol at a reference concentration of 326 μg/mol. AVL

- 2 was lower than AVL

- 1 with 50 th percentiles between

- 34 μg/mol and

- 70 μg/mol. The 50 th

Figure 44 percentiles for AVL

- 1 were betwee n 13 μg/mol and

- 32 μg/mol. No indication of the changes in performance for these PEMS were discovered through the recommended checks and audits. The PM deltas for Horiba

- 3, Sensors

- 3, and AVL

- 3 are shown in
, Figure 45 , and Figure 46 respectively. 

### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

AVL 50th % AVL 95th % AVL 5th %80 SwRI Project No. 14936.12 FIGURE 44 . HORIBA

- 3 STEADY STATE PM CO NCENTRATION DELTAS

FIGURE 45 . SENSORS

- 3 STEADY STATE PM CO NCENTRATION DELTAS



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Horiba 50th Horiba 95th Horiba 5th

- 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Sensors 50th % Sensors 95th % Sensors 5th %81 SwRI Project No. 14936.12 FIGURE 46 . AVL

- 3 STEADY STATE PM CO NCENTRAT ION DELTAS
Horiba

- 3 and Sensors

- 3 are within the same ranges of their first two instruments, while AVL

- 3 produced an extremely high bias at the highest concentration (mode 2). The 50 th

When the data from all three PEMS is considered, the range of concentrations from 5 mg/m3 to 18 mg/m3 is covered, although there is still a majority of the data located between 5 and 7 mg/m3 as a result of t he data from PEMS 1. The individual deltas for the Horiba, Sensors, and AVL are shown in percentile of mode 2 for AVL

- 3 was 84 μg/mol, while it was 13 μg/mol an d

- 62 μg/mol for AVL

- 1 and AVL

2, respectively.

Figure 47
, Figure 48 , and Figure 49 , respectively. Please note that each point represents a single measurement, not pooled data. 

### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

AVL 50th % AVL 95th % AVL 5th %82 SwRI Project No. 14936.12 FIGURE 47 . STEADY STATE CONCE NTRATION DELTAS FOR HORIBA

FIGURE 48 . STEADY STATE CONCE NTRATION DELTAS FOR SENSORS



### - 400 - 300 - 200 - 100 0 100 200

0 100 200 300 400 500PM Concentration Delta (μg/mol)



### Lab PM Concentration Reference (μg/mol)

Horiba

- 1 Horiba

- 2 Horiba

- 3



### - 400 - 300 - 200 - 100 0 100 200

0 100 200 300 400 500PM Concentration Delta (μg/mol)



### Lab PM Concentration Reference (μg/mol)

Sensors

- 1 Sensors

- 2 Sensors

- 383 SwRI Project No. 14936.12 FIGURE 49 . STEADY STATE CONCE NTRATION DELTAS FOR AVL
B ecause the reference concentration of several of the modes were similar, the steering committee decided to group the data by reference concentration rather than by operating mode. The data was split up into groups of approximately 20 data points based on r eference concentration for developing the steady state PM error surface. The Horiba data was grouped into 11 sets, the Sensors data into 9 sets, and the AVL data into 10 sets.
Figure 50
, Figure 51 , and Figure 52 show PEMS PM concentration plotted against the PM concentration determined by the reference laboratory filter method for steady

- state NTEs. These plots mainly show the qualitative PEMS to PEMS scatter relative to the filter method.
Figure 53 shows a linear regression between each set of PM

- PEMS and the CVS filter method. Good correlation wa s observed between the MSS and l ab , with as coefficient of determination ( R 2 

### - 400 - 300 - 200 - 100 0 100 200

0 100 200 300 400 500PM Concentration Delta (μg/mol)



### Lab PM Concentration Reference (μg/mol) AVL

- 1 AVL

- 2 AVL

- 3

) of 0.84 and a slope of 0.89. The slope suggests that the MSS PM concentrat ion is 11 percent lower than that determined by the lab. This trend is expected since the MSS is measuring soot and the lab reports total PM. The slope seemed to be high because the MSS PM concentration in the range between 15 and 20 mg/m3 was higher than that of the lab.84 SwRI Project No. 14936.12 The line a r regression between the Horiba TRPM and the lab resulted in R 2 of 0.55 and a slope of 0.86 , indicating some correlation. After further investigation, it was recognized that the weak correlation was due to Mode 4 (high speed, lig ht load) of the SS testing, with concentration leve ls of about 5 mg/m3. By removing this mode from the data , a R2 of 0.85 was obtained and the slope was moved from 0.86 to 0.88. It is likely that Mode 4, could have resulted in overestimation of PM due to n anoparticle formation with the Horiba dilution system, although CVS testing at this condition did not show a nanoparticle mode. Based on the slope of the correlation, the Horib a PM concentration was 14 percent lower than that reported by the lab.

The line ar regression between the Sensors PPMD and the lab results in R2 of 0.34 and a slope of 0.64. The weak correlation was due to data scatter . Except in the narrow range between 5 and 7 mg/m3, the Sensors PPMD showed underestimated PM. Based on the slope, the Sensors PPMD PM concentration was 36 percent lower than that reported by the lab.

FIGURE 50 . STEADY STATE HORIB A PEMS PM CONCENTRAT ION VERSUS THE LABORATORY REFERENCE 

### 0 50 100 150 200 250 300 350 400 450 500

0 100 200 300 400 500PEMS PM Concentration (μg/mol)



### Lab PM Concentration Reference (μg/mol)

Horiba

- 1 Horiba

- 2 Horiba

- 3 Lab Reference85 SwRI Project No. 14936.12 FIGURE 51 . STEADY STATE SENSORS PEMS PM CONC ENTRATION VERSUS THE LABORATORY REFERENCE 

### 0 50 100 150 200 250 300 350 400 450 500

0 100 200 300 400 500PEMS PM Concentration (μg/mol)



### Lab PM Concentration Reference (μg/mol)

Sensors

- 1 Sensors

- 2 Sensors

- 3 Lab Reference86 SwRI Project No. 14936.12 FIGURE 52 . STEADY STATE AVL P EMS PM CONCENTRATION VERSUS THE LABORATORY REFERENCE 

### 0 50 100 150 200 250 300 350 400 450 500

0 100 200 300 400 500PEMS PM Concentration (μg/mol)



### Lab PM Concentration Reference (μg/mol) AVL

- 1 AVL

- 2 AVL

- 3 Lab Reference87 SwRI Project No. 14936.12 FIGURE 53 . LINEAR REGRESSION CORRELATION BETWEEN PEMS AND LAB

4.9 STEADY STATE PM ERRO R SURFACES
Figure 54
, Figu re 55 , and Figure 56 , and show the original concentration deltas that were presented to the steering committee on April 2 nd , 2009 at SwRI. The Sensors data includes some additional points that were added later due to changing the error tolerances of their post processor. Each marker on the plot represents one of the j = 1 to M subsets of data divided up by concentration value. Each plot shows both the 5 th and 95 th percentiles which are based on the actual data. The error surfaces for the Monte Carlo model were based on the 1 st and 99 th percentiles which were extrapolated from the 5 th and 95 th percentiles assuming a normal distribution of the data.88 SwRI Project No. 14936.12 FIGURE 54 . STEADY STATE CONCE NTRATION DELTAS FOR HORIBA

FIGURE 55 . STEADY STATE CONCE NTRATION DELTAS FOR SENSORS



### - 200 - 150 - 100

- 50 0 50 100

0 100 200 300 400 500 600PM Concentration Delta (μg/mol)



### Lab Reference Median PM Concentration (μg/mol)

5th % 95th % 50th %

- 400 - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100

0 100 200 300 400 500 600PM Concentration Delta (μg/mol)



### Lab Reference Median PM Concentration (μg/mol)

5th % 95th % 50th %89 SwRI Project No. 14936.12 FIGURE 56 . STEADY STATE CONCE NTRATION DELTAS FOR AVL

The steering committee elected to smooth the error surface by not including some of the data points that were within the envelope of surrounding points. A similar decision was made on some of the error sur faces included in the gaseous measurement allowance program. 

### - 150 - 100

- 50 0 50 100 150

0 100 200 300 400 500 600PM Concentration Delta (μg/mol)



### Lab Reference Median PM Concentration (μg/mol)

5th % 95th % 50th %90 SwRI Project No. 14936.12 Figure 57
, Figure 58 , and Figure 59 show the final steady state error surfaces for Horiba, Sensors, and AVL, respectively. The data presented in these plots has already been processed to remove the CVS variability and adjusted to the 1 st and 99 th percentile from the 5 th and 95 th percentile. The lines on the plot represent the final error surfaces as approved by the steering committee. The Horiba and AVL error surfaces were accepted by the steering committee during the conference call on June 29 th

, 2009. Some of the Sensors data was reprocessed with different tolerances to increase the data yield, so the Sensors error surface was not accepted by the steering committee until the July 14 th , 2009 meeting in Indianapolis.

FIGURE 57 . FINAL STEADY STATE PM ERROR SURFACE

– HORIBA 

### - 400 - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150 200

0 100 200 300 400 500Corrected PM Delta (μg/mol)



### CVS Filter PM Concentration (μg/mol)

Delta 1st % Delta 50th % Delta 99th %91 SwRI Project No. 14936.12 FIGURE 58 . FINAL STEADY STATE PM ERROR SURFACE

- SENSORS

FIGURE 59 . FINAL STEADY STATE PM ERROR SURFACE

– AVL 

### - 400 - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150 200

0 100 200 300 400 500Corrected PM Delta (μg/mol)



### CVS Filter PM Concentration (μg/mol)

Delta 1st % Delta 50th % Delta 99th %



### - 400 - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150 200

0 100 200 300 400 500Corrected PM Delta (μg/mol)



### CVS Filter PM Concentration (μg/mol)

Delta 1st % Delta 50th % Delta 99th %92 SwRI Project No. 14936.12 With a few exceptions the PEMS generall y exhibited a low bias, with the majority of the 50 th percentile deltas falling below zero. The Horiba and Sensors both showed a degree of level dependence with the negative bias generally increasing for higher PM concentrations. At a lab reference concent ration of 413 μg/mol the Horiba 50 th percentile was

- 92 μg/mol and the Sensors 50 th percentile was

- 177 μg/mol at a lab reference concentration of 403 μg/mol. The AVL 50 th percentile was generally level independent and remained between

- 8 and

- 58 μg/mol fo r the entire range of concentrations tested. While the 1 st percentile for the AVL also remained relatively constant throughout the concentration range, the 99 th percentile jumped to a much higher value at high concentration. The AVL 99 th percentile jumped from 25 μg/mol at 291 μg/mol to 165 μg/mol at 402 μg/mol. The cluster of positive deltas that caused this large increase in the 99 th

4.10 TRANSIENT ENGINE RES ULTS percentile data is due solely to mode 2 with PEMS 2. The majority of the positive deltas observed for th e Horiba were due to mode 4 which was a high speed light load condition. Since it is understood that the DCS (EAD) real time particle sensor in the Horiba system is more sensitive to smaller particles it was assumed that may have a larger number of sub

- 50 nm particles than the other 5 modes tested, causing more of the filter mass to be attributed to mode 4.

The transient engine testing was designed to characterize the precision error of the PEMS. An transient cycle consisting of 30 NTE events was repeated multiple times and any differences in the PEMS measurements was attributed to PEMS variability under the assumption that the engine operation and PM emissions remained constant. No lab reference value was captured on an individual N TE event basis, because it was not possible to measure 30 NTE events with filters for each transient cycle. Instead filter measurements were taken for each integrated cycle as a general indicator of the PM emissions of the engine. The cycle used in this te sting was developed using the cycle generator from the gaseous measurement allowance program. In the gaseous program the cycle generator was used to develop 20 unique cycles with different orders of NTE events and different transitions between events. The steering committee elected to generate one single cycle for the current work that would be used for all transient testing. Additional time was added in between NTE events to ensure that the PPMD could sample throughout the cycle using without missing any N TE events under the assumption of 7 working crystals (6 for sample and one reference crystal). In addition several short non

- valid NTE events were added to the cycle to challenge the PEMS for measurements of non

- valid NTE events.
The DPF bypass was adjuste d to produce an integrated cycle BSPM of approximately 0.03 g/hp

hr based on the CVS filter. Instead of readjusting the bypass for each set of PEMS, the exhaust valves were set to the same position each time. The NTE events were all approximately 34 second s in duration. The original events from the cycle generator were each 32 seconds, however the Sensors PEMS would occasionally see these events as shorter than 30 seconds and exclude them. Many of the NTE events contained extreme accelerations and decelerat ions stopping just short of the lower boundaries of the NTE window. Considerable time was spent to ensure that93 SwRI Project No. 14936.12 the J1939 signal remained in the NTE window during events, although the engine performance shifted slightly and occasionally an event was invalid ated. If any NTE event did no remain in the NTE windows for at least 30 seconds it was not included in the data for the transient error surface. Figure 60 shows repea t engine speed traces for the transient cycle. The first and last official cycle conducted with each PEMS is shown.

FIGURE 60 . REPEAT ENGINE SPEE D TRACES FOR NTE TRA NSIENT CYCLE A total of 16 cycles were run for PEMS1, 17 for PEM S2, and 18 for PEMS2. Figure 61 shows the repeat torque traces for the transient cycle. The COV on cycle work was 0.7 percent over 51 cycles. 0 5 10 15 20 25 30 35

600 800 1000 1200 1400 1600 1800 2000 2200

0 500 1000 1500 2000NTE Event NumberEngine Speed (rpm)

Time (sec)

PEMS1

- 1 PEMS1

- 16 PEMS2

- 1 PEMS2

- 17 PEMS3

- 1 PEMS3

- 18 Event Number94 SwRI Project No. 14936.12

FIGURE 61 . REPEAT ENGINE TORQ UE TRACES FOR NTE TR ANSIENT CYCLE The exhaust concentration of soot, as measured by the AVL PEMS is shown in Figure 62

. 0 5 10 15 20 25 30 35

- 200

200 600 1000 1400 1800 2200

0 500 1000 1500 2000NTE Event NumberEngine Torue (Nm)

Time (sec)

PEMS1

- 1 PEMS1

- 16 PEMS2

- 1 PEMS2

- 17 PEMS3

- 1 PEMS3

- 18 Event Number95 SwRI Project No. 14936.12 FIGURE 62 . REPEAT AVL SOOT CO NCENTRATION TRACES F OR NTE TRANSIENT CYCLE The measured exhaust concentrations of soot ranged from 5 to 3800 μg/mol (0.2 to 160 mg/m3) during NTE operations, with concentrations of 5 to 14 μg/mol (0.2 to 0.6 mg/m3 ) outside of the NTE window. The widest dynamic range observed during an NTE event was for event 9 where the initial spike in soot concentration was measured as high as 3800 μg/mol before falling down to 55 μg/mol near the end of the event. Figure 63 gives a closer look at events 20 through 23 to show the variability of the real

- time signal over the three sets of PEMS. 

### 0 5 10 15 20 25 30 35

0 500 1000 1500 2000 2500 3000 3500 4000

0 500 1000 1500 2000NTE Event NumberExhaust Soot Concentration (μg/mol)



### Time (sec) PEMS3

- 18 PEMS3

- 1 PEMS2

- 17 PEMS2

- 196 SwRI Project No. 14936.12

FIGURE 63 . AVL SOOT CONCENTRATION DURING NTE TRANSIENT CYCLES , EVENTS 20

- 23 It should be noted that the AVL PEMS is shown here only because it reports the measured soot concentration on a second by second basis without additional processing. The Horiba PEMS also measures the concentration on a second by second basis, although it is being used in this program as a batch sampler rather than a second by second instrument. 

### 18 20 22 24 26 28 30

0 100 200 300 400 500 600 700 800 900 1000

1440 1490 1540 1590 1640NTE Event NumberExhaust Soot Concentration (μg/mol)



### Time (sec) PEMS3

- 18 PEMS3

- 1 PEMS2

- 17 PEMS2

- 197 SwRI Project No. 14936.12

FIGURE 64 . COMPARISON OF AVL AND HORIBA REAL TIME SIGNALS DURING TRANSIENT CYCLE There were major differences on some of the peak concentrations most notably NTE event 9, where the measured AVL concentration was nearly four times higher than the measured Horiba concentration. With no reference, the purpose of this figure is only to sho w that there are differences without trying to quantify the accuracy. The Sensors PEMS performs its measurements as a batch sampler so that there was no real

- time exhaust concentration reported.
To properly quantify the precision of the PEMS it is importa nt that the reference value remain constant otherwise changes in the source PM emissions will be attributed to the PEMS as measurement variability. In examining the CVS filter results, it was clear that the engine PM emissions varied somewhat during testin g. After reviewing the transient PEMS data at the January 28 th , 2009 meeting at SwRI, the steering committee requested that the idea of a correction to the PEMS data be applied to account for changes in the engine performance. A correction factor based on the cycle integrated CVS BSPM was presented to the committee during the April 2 nd

CC = ∑ CC NN

= 1 NN ∙ CC = CC CC VV The correction factor is multiplicative and applied to the PEMS data in the following manner: , 2009 meeting at SwRI and was accepted for use on all of the transient data. The correction factor was calculated as follows: 0 20 40 60 80 100 120 140 160 180

0 500 1000 1500 2000Exhaust Concentration (mg/m3)

Time (sec)

AVL

- Soot Horiba

- PM98 SwRI Project No. 14936.12 PP PP

, , jj = CC ∙ PP PP

, , jj The CVS BSPM along with the correction factor is shown in Figure 65

.

FIGURE 65 . CVS BSPM AND CORRE CTION FACTOR FOR TRA NSIENT CYCLE The correction factor ranged from 0.78 to 1.24. There were two cycles in which the CVS filter measurement was void; in these cases the correction factor was set to one. The CVS brake specific PM ranged from 34.4 mg/hp

- hr to 21.7 mg/hp

- hr with an average of 27.2 mg/hp

- hr. The COV was 11.4 percent when cycles for all three PEMS are included. T he COV was 7.2, 6.0, and 4.8 percent for cycles from PEMS1, 2, and 3, respectively.
While processing some of the AVL data and comparing to the lab , it was discovered that the incorrect CVS BSPM value had been used for one of the cycles. The value of 31.9 m g/hp

- hr from cycle 1956 had also been used for cycle 1965 instead of the correct value of 28.4 mg/hp

- hr. The entire set of transient CVS data was scrutinized again and no additional errors were found. This erroneous correction factor was included in the fi nal transient error surface included in the model. The transient error surface was found to shrink for all PEMS by between 1 and 2 percent with the correction applied. Given the small change in the outcome no action was taken to correct this mistake. The t ransient error surfaces presented in this report are those used in the Monte Carlo model. 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4

15 20 25 30 35 40 45 50 55

0 10 20 30 40 50 60Correction FactorCVS Brake-Specific PM (mg/hp-hr)

Transient Cycle Number

CVS Filter BSPM Correction Factor99 SwRI Project No. 14936.12 Figure 66 , Figure 67 , and Figure 68 show the concentration deltas for Horiba, Sensors, and AVL, respectively. In the transient testing there was no lab reference to create deltas. Rather the delta was calculated from the 50 th p ercentile of the lab data as follows:

Delta i,j = CF i * mPM i,j – 50 th i
Where mPM i,j is the average flow

- weighted PM concentration for the j th repeat of the i th NTE
event. 50 th i is the 50 th percentile of mPM for the i th NTE event. CF i is the previously descr ibed correction factor to adjust for engine variability.

FIGURE 66 . HORIBA CONCENTRAT ION DELTAS FOR TRANS IENT ENGINE TESTING 

### - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400PM Concentration Delta (μg/mol)



### Median Flow

- Weighted PM Concentration (μg/mol)

5th

- 50th Percentile 95th

- 50th Percentile100 SwRI Project No. 14936.12 FIGURE 67 . SENSORS CONCENTRAT ION DELTAS FOR TRANS IENT ENGINE TESTING 

### - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400PM Concentration Delta (μg/mol)



### Median Flow

- Weighted PM Concentration (μg/mol)

5th

- 50th Percentile 95th

- 50th Percentile101 SwRI Project No. 14936.12 FIGU RE 68 . AVL CONCENTRATION DELTAS FOR TRANSIENT ENGINE TESTING
The 5 th and 95 th percentiles of the Horiba and AVL PEMS were bounded by plus and minus 50 μg/mol. The Sensors PEMS had slightly larger deltas extending just beyond 100 μg/mol for the 95 th percentile and

- 150 μg/mol for the 5 th percentile. The bias from the data was remove d by subtracting the 50 th from the 5 th and 95 th percentiles.
The steady state error surface was designed to quantify the accuracy of the PEMS while the transient error surface was designed to quantify the precision of the PEMS. However, a portion of the PE MS precision error is inherently captured in the steady state error surface. At the meeting on July 15 th , 2009 in Indianapolis four different approaches were presented to the steering committee for removing the steady state contribution to the precision er ror from the transient error surface. The steering committee elected to proceed with approach 3; for simplicity the other three approaches will not be described.
In this approach, the data from all three PEMS from the same manufacturer were pooled togethe r (ie Sensors1, Sensors2, and Sensors3). This resulted in a total of 18 steady state data points and 90 transient data points. A median and a MAD value were calculated for each of the 108 data points and used to calculate a MAD relative error, transient ef fect, root mean squared or MAD re,tr,rms . The MAD re,tr,rms 

### - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400PM Concentration Delta (μg/mol)



### Median Flow

- Weighted PM Concentration (μg/mol)

5th

- 50th Percentile 95th

- 50th Percentile

is defined as follows:102 SwRI Project No. 14936.12 MM

, , = � 1 NN � � MM

, MM

, � 2 NN

= 1 − 1 NN � � MM

, MM

, � 2 NN

= 1
Where tr denotes transient and ss denotes steady state. Using an rms value eliminates the need for estimating the steady state variability at each of 90 transient data points. The error surface was defined as the 90 percent confidence interval around zero (or no bias) as:

5 h PP = − 1

. 65 ∙ PPPP ∙ MM

, , 95 h PP = 1

. 65 ∙ PPPP ∙ MM

, , Effectively the error surface is a straight line with a slope of +/

- 1.65·MAD re,tr,rms Figure 69 and an intercept of zero. The transient error surface is shown in for Horiba, Figure 70 for Sensors, and Figure 71 for AVL.

FIGURE 69 . FINAL HORIBA TRANS IENT PM ERROR SURFAC E - 120

- 80 - 40

0 40 80 120

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Median Flow Weighted PM Concentration ( μ g/mol)

1st Percentile 99th Percentile103 SwRI Project No. 14936.12 FIGURE 7 0 . FINAL SENSORS TRAN SIENT PM ERROR SURFA CE

FIGURE 71 . FINAL AVL TRANSIENT PM ERROR S URFACE - 120

- 80 - 40

0 40 80 120

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Median Flow Weighted PM Concentration ( μ g/mol)

1st Percentile 99th Percentile

- 120

- 80 - 40

0 40 80 120

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Median Flow Weighted PM Concentration ( μ g/mol)

1st Percentile 99th Percentile104 SwRI Project No. 14936.12 The median flow weighted PM concentration on the x

- axis of each graph is extended to the highest measured concentration observed by any of the PEMS during transient testing

. The slope of the transient PM error surface is 0.083 (1 .41) for Horiba, 0.228 (3.27) for Sensors, and 0.099 (1.66) for AVL. The 1 st percentile for the PEMS are

- 34,

- 92, and

- 40 μg/mol at the maximum PM concentrations observed by each of the PEMS during steady state testing (around 400 μg/mol) for the Horiba, Sensors, and AVL systems. The 1 st

4.11 CE

- CERT MOBILE LAB CORR ELATION percentiles during steady state testing at the same PM level were

- 177,

- 366, and

- 110 μg/mol for the Horiba, Sensors, and AVL, respectively. Because the steady state error surface was much larger than the transient it ten ded to dominate the model results as discussed later.

The test planned called for the validation of the model to be performed by the mobile emissions laboratory (MEL) operated by the University of Califor nia at Riverside Bour ns College of Engineering Center for Environmental Research and Technology (CE

- CERT) . The mobile laboratory consisted of a full

- flow CVS system inside the trailer of a Class A truck. The mobile lab was capable of measuring gaseous emissions and filter base d PM. The mobile lab was arrived at SwRI on April 9 th

Figure 72 , 2009 to compare brake

- specific PM emissions and ensure that the reference during in

- use validation is similar to the reference used during laboratory testing. The CE

- CERT MEL was parked behind the SwRI test cell and an exhaust transfer line was fabricated to allow the MEL to measure the full engine exhaust in the same way as the SwRI test cell. Because the CVS measurement technique requires the full flow of engine exhaust for emissions measurement, the exhaust system was designed in such a way that the exhaust pipe could easily be switched between the SwRI CVS and the MEL CVS using the same length and geometry of exhaust pipe. shows the exhaust system used for the CE

- CERT correlation.

FIGURE 72 . EXHAUST CONFIGURAT ION FOR CE

- CERT CORRELATION To CE

- CERT CVS Tunnel To SwRI CVS Tunnel105 SwRI Project No. 14936.12 The MEL was set to the same CVS flow, secondary dilution ratio, and filter face velocity a s the SwRI CVS to make the measurements as close as possible. One key factor that could not be controlled was the dilution air for the MEL CVS. The SwRI CVS was able to draw flow that is conditioned and maintained between 20 and 30oC. The MEL CVS drew its dilution air from the ambient which prevented the control of temperature and relative humidity of the dilution air. Each sampling system was conditioned by sampling for 10 hours during steady state DPF engine operation at a high exhaust temperature. Active DPF regeneration occurred during this conditioning period. A total of 16 short NTE transient cycles were conducted using each CVS system. The short NTE transient cycle was a modified version of the NTE transient cycle used for official transient PEMS test ing. The short cycle included 16 of the original 30 NTE events and lasted 755 seconds compared to 2130 seconds for the full NTE cycle. Table 18 shows the order of test ing.

TABLE 18 . TEST PROCEDURE FOR CE

- CERT CORRELATION

A manually triggered active DPF regeneration was performed before the start of each set of four test cycles to maintain a similar PM loading level on the DPF. Each day the test order was switched so that the SwRI tests were first on days 1 and 3 and the CE

- CERT tests were first on days 2 and 4. Each test was conducted as a hot

- start with a 20 minute hot soak in between test cycles or in between the DPF regeneration and the first test cycle. Figure 73 shows the brake

specific PM results from the 16 cycles.106 SwRI Project No. 14936.12 FIGURE 73 . BRAKE

- SPECIFIC PM RESULTS FROM CE

- CERT CORRELATION The data from test 9 for CE

- CERT was removed due to a filter weight that was deemed to be an outlier. The average SwRI BSPM was 0.0287 g/hp

- hr with a COV of 5.2 percent based on 16 repeats. The average CE

- CERT BSPM was 0.0265 g/hp

- hr with a COV of 3.5 percent based on 15 repeats. The reported CE

- CERT emissions were on average 7.7 p ercent lower than the SwRI reported emissions. The average reported BSCO 2

A series of tunnel blanks were measured from both systems over sample periods of 15, 30, and 60 minutes. by CE

- CERT was 2.6 percent lower than average SwRI reported value. One possible source of discrepancy between the two systems was the heat loss in the exhaust pipe prior to its entra nce into the CVS. The SwRI system was completely sheltered within the test cell, while the CE

- CERT exhaust pipe was protruding outside in such a way that it was expose to wind and ambient temperature effects. This could have resulted in a higher thermophor etic deposition of particles inside the exhaust pipe resulting in lower emissions for the CE

- CERT system. However, the test plan stated that agreement within ten percent was considered sufficient, so the correlation was considered complete and the issue

wa s not investigated further.

Figure 74 shows the filter weight gains as a function of sample time. 

### 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31Brake-Specific PM (g/hp-hr)



### SwRI CE

- CERT Day 1 Day 2 Day 3 Day 4107 SwRI Project No. 14936.12 FIGURE 74 . CVS FILTER WEIGHT GAIN DURING TUNNEL B LANKS The filter weight gains during tunnel blank operation were similar for the SwRI and CE

- CERT tunnels. The weight gains ranged from 1.9 to 12.6 μg and generally decreased as the sampling time increased. The filter weight gains during the correlation testing were greater than 300 μg making any differences in the tunnel blanks insignificant.

4.12 INVESTIGATION OF DPF REGENERATION
The steering commit tee requested that screening tests be performed on measurements of active DPF regeneration. Although the majority of the time active DPF regeneration is excluded from in

- use measurement as the regulations are currently written there are certain situations where regeneration could be included in a valid NTE event. In addition, if the Horiba system were to trigger filter sampling during DPF regeneration this would still be included in their calibration factor. So even if all NTE events containing active DPF r egeneration were considered invalid, the measurement accuracy of the Horiba system during valid NTE events could still be affected through the filter calibration. For this testing the engine was operated at a medium speed medium load condition at steady st ate and the DPF was allowed to accumulate PM until the ECM automatically triggered a regeneration. The valves in the bypass were closed forcing a large majority of the exhaust through the DPF although the bypass was not completely sealed. At the point when the ECM indicated it was preparing for an active regeneration the PEMS and CVS filter were triggered to sample. The PEMS sampled 40 seconds on, five seconds off throughout 

### 0 2 4 6 8 10 12 14

0 15 30 45 60 75Filter Weight Gain (μg)



### Sample Time (min)

SwRI CE

- CERT108 SwRI Project No. 14936.12 the regeneration, while the CVS filter sampled continuously. Table 19 lists the brake

- specific PM results of the PEMS and the CVS.

TABLE 19 . PM EMISSIONS RESUL TS FROM ACTIVE DPF R EGENERATION

It is likely that only a small portion of the emissions during the regeneration even included elemental carbon as shown by the fact that the AVL PEMS measured only three percent of the emissions measured by the CVS filter. The Horiba and Sensors PEMS both measured BSPM values on the sam e order as the CVS filter although the event

- by

- event emissions were much different. The EEPS also sampled continuously from the CVS through a long residence time secondary dilution tunnel with a nominal secondary dilution ratio of 2. The EEPS number

conce ntration was converted to a mass concentration assuming spherical particles with a density of 1 g/cm3. A comparison of the event

- by

- event emissions of the three PEMS and the EEPS are shown in Figure 75

.

FIGURE 75 . BRAKE SPECIFIC PM EMISSIONS DURING ACT IVE REGENERATION Although the average brake

- specific PM value from the Sensors system was very close to the lab value it is unlikely that the Sensors instrument was properly capturing the behavior of the regeneration. The event

- by

- event emissions from the Horiba system agr eed well with the EEPS CVS Filter Horiba Sensors AVL No. of Samples 1 36 31 35 Avg BSPM, mg/hp-hr 6.8 4.6 7.1 0.2



### 0 2 4 6 8 10 12 14 16 18

0 20 40 60 80 100Brake-Specific PM (mg/hp-hr)



### Regeneration Time (%)

Horiba Sensors AVL EEPS109 SwRI Project No. 14936.12 real

- time measurement. Comparing the Horiba measurements to the CVS filter and the EEPS indicate that it was more accurate than the other PEMS at measuring the regeneration emissions. However, one aspect of the Horiba measurement tha t was not captured in this experiment, and that is the change in sensitivity of the DCS particle instrument to different size particles. The filter calibration constant for the regeneration event indicated that the DCS was approximately 15 times more sensi tive to the particles emitted during the active regeneration compared to the particles emitted during steady state engine testing. This means that if a steady state engine cycle was sampled onto the same filter as the DPF regeneration the system would be m uch less accurate. The Horiba system would over predict the emissions during the regeneration and under predict the emissions during normal engine operation. These findings were presented to the steering committee at the December 11 th

4.13 INVESTIGATION OF STO RAGE AND RELEASE , 2008 meeting in San Antonio. Although this could be a major issue in the accuracy of the measurement, due to budget limitation, the steering co mmittee did not add DPF regeneration testing to the program given th at the regulated aspect of regeneration during in

- use testing was vague . No data f rom the DPF regeneration investigation was included in the model.

At the same time of the DPF regeneration investigation, the steering committee also request that screening tests be performed in the are a of storage and release of nanoparticles in the aftertreatment system. It has commonly been observed that volatile material emitted by the engine at low temperature will deposit on the DPF only to be released quickly when the engine exhaust temperature cl imbs. Three different DPF loading conditions were tested:

**•** Low idle: 650 rpm, 65 Nm

**•** Medium idle: 1200 rpm, 135 Nm

**•** High idle: 1800 rpm, 135 Nm
idling times of 20, 60, and 90 minutes were tested at each condition. Following the period of low temperature loa ding, the engine was immediately brought to a high temperature condition to promote the release of the stored particles. Each of the three idling conditions were tested with two high temperature conditions:

**•** Peak torque: 1200 rpm, 2170 Nm

**•** Near rated power: 1800 rpm, 1425 Nm
These tests were conducted as a screening exercise with measurements by the EEPS to determine which combination produced the largest release of nanoparticles to use for the PEMS testing. However none of the tested conditions resulted in any significant particle emissions on a number or mass basis. To ensure that the EEPS was not missing something that might have been captured by one of the PEMS, two tests were run with the PEMS: low idle to peak torque and high idle to peak torque. Only t he low idle test is shown here because the results are very similar.110 SwRI Project No. 14936.12 The engine was allowed to idle for one hour before going to peak torque for ten minutes, this process was repeated three times consecutively for a cycle length of 3.5 hours. The PEMS only sampled during the peak torque portion of the cycle with the same 40 seconds sample, five seconds off cycling used in the DPF regeneration study.

FIGURE 76 . TOTAL EXHAUST NUMB ER CONCENTRATION DUR ING STORAGE AND RELEASE A spike of just over 2.0E6 particles/cm3 was observed during the transition from idle to peak torque although this appears to be due to acceleration and a possible slight misalignment between the dilution ratio and EEPS measurement rather than a release of particles from the aftertreatment. The spike was less than 1.0E6 for the second and third transitions to peak torque. Figure 77 shows the brake

- specific PM emissions measured by the three PEMS during the peak torque portion of the storage and release cycle. All samples are taken during the three repeats of the ten minutes at peak torque and no emissions from the idle portion of the cycle are represented. 

### 0.0E+00 5.0E+05 1.0E+06 1.5E+06 2.0E+06 2.5E+06

0 500 1000 1500 2000 2500

0 2000 4000 6000 8000 10000 12000Exhaust Number Concentration (part/cm3)Engine Speed (rpm), Torque (Nm)



### Time (sec)
Speed RPM Torque N

- m EEPS111 SwRI Project No. 14936.12 FIGURE 77 . BRAKE

- SPECIFIC PM EMISSION S DURING STORAGE AND RELEASE INVESTIGATION It is unclear why the Sensors was reporting emissions much higher than the other two PEMS. A single CVS filter was sampled for the entire ten minutes at peak to rque for each repeat for a total of three filter measurements. The BSPM from the CVS filter was only between 0.3 and 0.4 mg/hp

- hr for each of the three repeats. This was lower than any of the PEMS and an order of magnitude lower than the Sensors PEMS. The three negative emissions were attributed to the same crystal which was unusually noisy during the test. A comparison of the average BSPM values for the cycle are shown in Table 20

. TABLE 20 . AVERAGE BRAKE

- SPECIFIC EMISSIONS D URING STORAGE AND RELEASE CYCLE

The effect of storage and release on the PEMS emissions was not clear because of the inability to generate a significant release of nanopart icles, but it was evident that investigating this phenomenon using this experimental configuration was not worthwhile. These findings were presented at the December 11 th 

### - 4.0 - 2.0

0.0 2.0 4.0 6.0 8.0 10.0 12.0

0 10 20 30 40Brake-Specific PM (mg/hp-hr)



### Sample Number

Horiba Sensors AVL 1st Repeat 2nd Repeat 3rd Repeat

, 2008 meeting in San Antonio and the steering committee declined to pursue any further work in this area. No data from the storage and release investigation was included in the model. CVS Filter Horiba Sensors AVL No. of Samples 3 39 36 39 Avg BSPM, mg/hp-hr 0.38 0.54 2.97 0.41112 SwRI Project No. 14936.12 4.14 ENGINE MANUFACTURERS TORQUE AND FUEL ERRO R SURFACES
The OEM supplied torque error surface was updated from the gaseous PEMS program and the OEM supplied BSFC error surface was replaced with a fuel flow error surface. Five different engine manufacturers supplied data from 61 different engines. In addition data was used from the four engines tested in the ACES program for a total of 2,099 data points from 65 eng ines. The ECM torque deltas were normalized by the maximum ECM torque. The torque deltas are shown in Figure 78

.

FIGURE 78 . OEM SUPPLIED TO RQUE ERRORS The errors as a percentage of the maximum ECM torque are relatively constant throughout the entire measured range indicating that the error as a percentage of point would increase as the absolute torque decreases. The much smaller data set used for the gaseous measurement allowance program showed constant errors as a percentage of point rather than a percentage of maximum. The plot of ECM fuel flow errors as a percentage of maximum ECM fuel flow is shown in Figure 79

. 

### - 20% - 15% - 10%

- 5%

0% 5% 10% 15% 20%

0.0 0.2 0.4 0.6 0.8 1.0 1.2(ECM Torque-Lab Torque)/Max ECMTorque



### ECM Torque/Max ECM Torque113 SwRI Project No. 14936.12 FIGURE 79 . OEM SUPPLIED FUEL FLOW ERRORS The fuel flow errors were also constant in relation to max fuel flow rate above 20 percent fuel flow. Since the fuel flow is e xpected to remain well above 20 percent during NTE event operation, the smaller errors at low fuel flows were not considered important. Although the OEM supplied torque and fuel flow error surfaces were created as percentages of point for the gaseous

measu rement allowance, these surfaces were generated as percentages of maximum for the current program. The 1 st , 50 th , and 99 th Table 21

. OEM Error Surface Deltas for Torque and Fuel Flow percentiles were calculated for torque and fuel flow and sampled in the model using a normal distribution. The error surface deltas a re shown in

TABLE 21 . OEM ERROR SURFACE DELTAS FOR TORQUE AN D FUEL FLOW 

### - 30% - 20% - 10% 0% 10% 20% 30%

0.0 0.2 0.4 0.6 0.8 1.0 1.2(ECM Fuel-Lab Fuel)/Max ECM Fuel



### ECM Fuel/Max ECM Fuel

1st, % point 50th, % point 99th, % of point Torque -7.6 -1.7 4.1 Fuel Flow -4.5 -0.1 4.9 Percentiles Parameter114 SwRI Project No. 14936.12 5 ENVIRONMENTAL TESTIN G AND RESULTS
A series of tests were conducted to characterize the PEMS response under in

- use conditions such as changes in pressure, temperature, and humidity, as well as their response to electromagnetic and radio frequency interference and shock and vibration.
Initially the test plan called for the EPA PM Generator system to be used as the PM source for all environmental testing. The PM generator is capable of producing soot, volatile hydrocarbons, and sulfuric acid to simulate the PM emitted from a diesel engine. However, after the system was operated at SwRI it was discovered that the PM generator was too tall to fit into the altitude chamber. It was likely that it would not be possible to operate the PM generator under atmospheric conditions while the output was subject to a constantly varying pressure. The diffusion rate of the hy drocarbon vials is pressure dependant so it would not have been possible to maintain a constant PM source while varying the pressure inside of the chamber. SwRI proposed using a Jing mini

- CAST soot generator in place of the PM generator. The soot generator is only a fraction of the size and much easier to operate compared to the PM generator. The steering committee agreed to allow the soot generator to be used for the altitude testing but requested that the PM generator be used for the temperature and humid ity testing.
It was decided to operate the Electromagnetic Interference / Radio Frequency Interference (EMI/RFI) and shock and vibration testing as screening. In this case screening testing meant that the PEMS were operated while sampling zero air to look for potential problems. The results from the screening testing would then be presented to the steering committee to decide whether to proceed with official testing to generate an error surface. A good portion of the motivation to conduct the EMI/RFI and v ibration testing as screening was the result of the finding of the gaseous measurement allowance program that in most cases a failure mode of the PEMS was observed only as a malfunction of the system in which it could no longer operated. It was not commonl y observed that the accuracy of the PEMS was affected while it continued to measure without detected problems.
SwRI’s Mechanical and Material Engineering Division (Division 18) performed the environmental testing. The altitude, temperature and humidity t esting was performed by Rick Pitman and Mike Negrete. The EMI/RFI testing was performed by David Smith and Herbert Walker. The shock and vibration testing was performed by David Smith, Mike Negrete, and Mark Orlowski.

5.1 REFERENCE MEASUREMEN T TESTING
An eigh t hour baseline measurement was performed for comparison to the eight hour environmental tests. Unlike the gaseous measurement allowance program it was not possible to compare the accuracy of the PM

- PEMS over the measurement period, only the variability. T he115 SwRI Project No. 14936.12 PM and soot generators provide a particle source, but the correct concentration of the source was unknown.
During the eight hours of testing the PM concentration and dilution ratio was cycled to allow a more accurate assessment of the PEMS performance over a range of operating conditions. Three PM levels and four dilution ratios were sampled for a total of 12 test conditions. Figure 80 shows the schedule of target PM concentration and dilution ratio for one hour of environmental testing.

FIGURE 80 . TARGET DILUTION RA TIO AND PM LEVEL PRO FILE FOR ENVIRONMENTAL TESTIN G

Each level was sampled at dilution ratios of 6, 12, 20, and 30 for the Ho riba and Sensors PEMS. The AVL PEMS was maintained at its constant dilution ratio of 5. The PEMS were maintained at a target dilution ratio for five minutes. With 35 seconds remaining, the sample trigger was enabled for 30 seconds. This allowed for four mi nutes and 25 seconds for stabilization, 30 seconds for sampling, and five seconds after sampling to ensure sampling on all PEMS had stopped before the target dilution ratio was changed. The PEMS were cycled through the four dilution ratios at a single PM c oncentration level before the process was repeated at the next concentration level. It took one hour to cycle through each combination of dilution ratio and PM concentration. This profile was repeated eight times for a total test time of eight hours. The

s chematic showing the experimental setup can be seen in 

### 0 1 2 3 4

0 5 10 15 20 25 30 35

0 10 20 30 40 50 60Target PM LevelTarget Dilution Ratio



### Time (min)

Target DR Target PM116 SwRI Project No. 14936.12 FIGURE 81 . EXPERIMENTAL SETUP FOR ENVIRONMENTAL TE STING

The particles from the soot generator passed through a Krypton

- 85 neutralizer to bring the charge of the particles t o a distribution similar to engine exhaust. A large volume was placed downstream of the neutralizer to make minimize the pressure fluctuations observed by the soot generator and also to smooth out any changes in concentration from the generator. One of the AVL PEMS units was placed outside of the environmental chamber upstream of the orifice and overflow so that it was isolated from the chamber conditions. This PEMS served as a reference to verify that the soot concentration from the generator was stable. T he AVL unit was chosen because it could provide a real time measurement of the soot concentration. This was not a guarantee that the total PM concentration from the generator was steady, but it typically the volatile emissions from the generator would not fluctuate significantly without some change in the soot concentration.

The mini

- CAST soot generator was used as the particle source for the baseline testing. The number mean diameter was approximately 70 nm based on measurement with the EEPS. The three con centration levels were nominal concentration levels of 1, 3, and 5 mg/m3 with approximately 30 percent organic carbon based on the OC/EC measurement.117 SwRI Project No. 14936.12 FIGURE 82 . HORIBA ENVIRONMENT AL BASELINE MEASUREM ENTS



### - 20 0 20 40 60 80 100 120 140 160 180 200

0 20 40 60 80 100Measured PM Concentration (μg/mol)



### Observation Number118 SwRI Project No. 14936.12 FIGURE 83 . SENSORS ENVIRONMEN TAL BASELINE MEASURE MENTS

FIGURE 84 . AVL ENVIRONMENTAL BASELINE MEASUREMENT S 

### - 20 0 20 40 60 80 100 120 140 160 180 200

0 20 40 60 80 100Measured PM Concentration (μg/mol)



### Observation Number

- 20 0 20 40 60 80 100 120 140 160 180 200

0 20 40 60 80 100Measured PM Concentration (μg/mol)



### Observation Number119 SwRI Project No. 14936.12

FIGURE 85 . REFERENCE MSS ENVI RONMENTAL BASELINE M EASUREMENTS Although the soot c oncentration from the generator was likely stable based on the measurements from both AVL PEMS it is not clear whether the total PM concentration remained constant throughout the test. The Horiba PEMS exhibited a clear downwards trend in measured
concentra tion as the test progressed. The reported PM concentration from the Sensors PEMS appeared to be somewhat independent of the actual PM concentration sampled. The Sensors data was too scattered to either confirm or disconfirm the PM trend observed in the Hor iba data. In addition to the general downwards trend of the concentration, the Horiba data appeared to suggest that the accuracy of the dilution ratio was playing a role in the measurement. In observations 25 through 96, the reported concentration decrease d each time the dilution ratio target increased. This suggests that the Horiba PEMS is has either a positive error on lower dilution ratios or a negative error on higher dilution ratios.

The PEMS and reference AVL were both able to resolve the differences between the three PM levels clearly even showing similar responses to small changes in concentration. Figure 86 depicts the relationship between the AVL reference measurement and the AVL PEMS
measurement. 

### 0 20 40 60 80 100 120 140

0 20 40 60 80 100Measured PM Concentration (μg/mol)



### Observation Number

Ref MSS PEMS MSS120 SwRI Project No. 14936.12 FIGURE 86 . REFERENCE AVL VERS US PEMS AVL FOR ENVI RONMENTAL BASELINE
The correlation coefficient between the two measurem ents is excellent, better than 0.99. The slope of 0.78 is likely to be a combination of difference in response of the two instruments and line losses between the two points of measurement.

5.2 PRESSURE CHAMBER TES TING
The SwRI altitude chamber is capable of s imulating altitudes up to 19.8 km. The chamber is 1.5 m in diameter and 2.1 m tall. Typically the simulated altitude chamber at SwRI is operated only under vacuum to simulate altitudes greater than that of San Antonio. The gaseous measurement allowance tes t plan called for pressures up to 101.87 kPa or 45 meters below sea level. The elevation of San Antonio is approximately 240 meters above sea level with a barometric pressure near 99 (98.4 based on altitude) kPa. In the gaseous measurement allowance progra m, the altitude chamber underwent significant alterations to achieve positive pressures and still many problems were encountered. The SwRI engineer in charge of the altitude chamber requested that only negative pressures be tested to preserve the integrity of their test equipment. Simply changing the positive pressures to ambient would have resulted in a large portion of the testing conducted at normal atmospheric pressure. Instead a slight negative pressure of 94.3 kPa (610 m, 2,000 ft) was repeated twice, once at 1.8 hours and once at 7.2 hours. Figure 87 shows the 

### y = 0.7762x R2 = 0.9991

0 20 40 60 80 100 120

0 20 40 60 80 100 120 140AVL PEMS Concentration (μg/mol)



### AVL Reference Concentration (μg/mol)121 SwRI Project No. 14936.12 original pressure profile from the test plan as well as the modified profile that was used during testing

.

FIGURE 87 . ORIGINAL AND REVIS ED PROFILE FOR ALTIT UDE TESTING Significant efforts were devoted to ensuring that a stable PM source could be generated that was insensitive to pressure. Because the soot generator contains an open flame operating at atmospheric pressure, the properties of the flame and hence the particle generation tended to change with the pressure of the outlet. By placing an orifice in the transfer line between the soot generator and the pressure chamber it was p ossible to operate the soot generator at a higher pressure and maintain a constant pressure through adjustment of an overflow valve upstream of the orifice. Adjustments were only necessary during the pressure ramps to 82.7 kPa and 90.0 kPa. The valve was a djusted to maintain a constant pressure upstream of the orifice. Figure 88 shows the setup outside of the altitude chamber. 

### - 800

- 300

200

700

1200

1700 80 85 90 95 100 105 110

0 1 2 3 4 5 6 7 8Altitude (m)Pressure (kPa)



### Time (hr)
Original Revised122 SwRI Project No. 14936.12 FIGURE 88

. ALTITUDE TESTING CHA MBER During the first practice run the AVL PEMS 3 blew a fuse in its measurement unit. The test was stopped and the fuse was replaced, but immediately blew again indicating an electrical failure within the unit. The unit was replaced wi th AVL 3 and testing continued with the problem not observed again. Due to space constraints, the Semtech DS and Horiba OBS

- 2200 gaseous PEMS were located outside of the environmental chamber. Since the purpose of these devices was only communications, it was not considered necessary to test them inside the chamber. The external compressor used for the Horiba dilution air was also installed outside the chamber. A large compressor was supplied by SwRI to provide oil

- less dilution air. The steering committee had requested that the Horiba supplied compressor should be used but then agreed to allow the replacement compressor after it was determined that the original compressor could not operate for the entire eight hours without shutting off. A picture of the PEMS installed in the altitude chamber is shown in Figure 89

.123 SwRI Project No. 14936.12 FIGURE 89 . PEMS INSTALLED IN THE ALTITUDE CHAMBER As mentioned previously, analysis of the environmental data was more difficult for the PM

PEMS program compared to the gaseous PEMS program due to the lack of a known reference concentration. For this reason, only the variability of the PEMS measurement in comparison to its avera ge was compared. The individual data points were plotted as well as the average levels to show how far each measurement deviated from the average. Figure 90
, Figure 91 , and Figure 92 show the concentration measurements by the Horiba, Sensors, a nd AVL respectively.124 SwRI Project No. 14936.12 FIGURE 90 . HORIBA ENVIRONMENT AL PRESSURE MEASUREM ENTS
The Horiba concentration measurements were not consistent for different dilution ratios. In the baseline test it was observed that increasing the dilution ratio in the range of 6, 12, 20, and 30 tended to decrease the reported concentration. In the pressure test, it appears that the measured concentration increases as the dilution ratio increases. The largest deviations from the average occurred during obs ervations 53

- 60 while the pressure was at its lowest indicating that the ambient pressure does have some effect on the accuracy of the Horiba reported concentration. 

### 80 82 84 86 88 90 92 94 96 98 100

0 20 40 60 80 100 120 140

0 20 40 60 80 100Chamber Pressure (kPa)PM Concentration (μg/mol)



### Observation Number
Avg Horiba Horiba Pressure125 SwRI Project No. 14936.12 FIGURE 91 . SENSORS ENVIRONMEN TAL PRESSURE MEASURE MENTS
The Sen sors data exhibited a large amount of variability with some data more than a factor of two higher and lower than the average. With that amount of scatter it was difficult to visually discern an effect of pressure on the measurement. 

### 80 82 84 86 88 90 92 94 96 98 100

0 50 100 150 200 250

0 20 40 60 80 100Chamber Pressure (kPa)PM Concentration (μg/mol)



### Observation Number
Avg Sensors Sensors Pressure126 SwRI Project No. 14936.12 FIGURE 92 . AVL ENVIRONMENTAL PRESSURE MEASUREMENT S The AVL data was grouped tightly around the average except during the low pressure excursion around observances 53

- 60. Because of the excellent repeatability of the measurement, the effect of pressure on the measurement was readily apparent. As the ambient pressure decreased, the measurement decreased as well.
To determine whether an error surface would be generated for each PEMS, the MAD from the baseline was compared to the MAD of the pres sure test. If the MAD of the pressure test was greater, an error surface was generated to present to the steering committee. The plots of median versus MAD shown in Figure 93
, Figure 94 , and Figure 95 were presented to the steering committee at the meeting in Indianapolis on July 15 th 

### 80 82 84 86 88 90 92 94 96 98 100

0 10 20 30 40 50 60 70 80

0 20 40 60 80 100Chamber Pressure (kPa)PM Concentration (μg/mol)



### Observation Number

Avg AVL AVL Pressure

, 2009.127 SwRI Project No. 14936.12 FIGURE 93 . AVL PRESSURE MEDIA N VERSUS MAD

FIGURE 94 . HORIBA PRESSURE ME DIAN VERSUS MAD 

### y = 0.022x

- 0.124 R2 = 0.999 y = 0.047x + 0.539 R2 = 0.413

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0

0 20 40 60 80 100 120MAD PM (μg/mol)



### Median PM Concentration (μg/mol)

AVL

- Baseline AVL

- Pressure

y = 0.073x + 5.936 R2 = 0.427
y = 0.121x

- 4.006 R2 = 0.991

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0

0 20 40 60 80 100 120MAD PM (μg/mol)



### Median PM Concentration (μg/mol)

Horiba

- Baseline Horiba

- Pressure128 SwRI Project No. 14936.12 FIGURE 95 . SENSORS PRESSURE M EDIAN VERSUS MAD The MAD values were higher for the baseline compared to the pressure test for both Horiba and Sensors indicating that no additional variability in the measurement could be attributed to changes in pressure using this technique. No environmental error surface was calculated for either Horiba or Sensors. The AVL MAD values were higher for the pressure test compared to the baseline, so an error surface was developed. It should be noted that the var iability of the AVL is much lower for the AVL compared to the Horiba and Sensors, however because of the high precision of the AVL measurements during the baseline it was still possible to discern the added variability due to changes in ambient pressure. T he environmental pressure error surface was calculated using the same pooled rms technique that was used for the transient error surface. Figure 96 shows the error sur face that was generated for environmental pressure on the AVL PM. The error surface was accepted for use by the steering committee during the July 15 th 

### y = 0.155x + 15.834 R2 = 0.756

y = 0.111x + 15.755 R2 = 0.609

0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0

0 20 40 60 80 100 120 140MAD PM (μg/mol)



### Median PM Concentration (μg/mol)

Sensors

- Baseline Sensors

- Pressure

, 2009 meeting in Indianapolis.129 SwRI Project No. 14936.12 FIGURE 96 . FINAL ERROR SURFAC E FOR ENVIRONMENTAL PRESSURE AVL PM CONCENTRATION This error surface was similar in magnitude to the AVL transient PM error surface, however it was much smaller than the AVL steady state PM error surface.

5.3 TEMPERATURE & HUMIDI TY CHAMBER TESTING
The temperature and humidity chamber testing was designed to characterize the effects of ambient temperature and humidity on the accuracy of PM measurement. The testing chamber has the capability of independently controlling the temperature and moi sture content of the air in the range of typically observed levels from ambient conditions. The temperature and humidity profile from the gaseous program was modified based on data acquired during CE

- CERT testing. At the June 12 th

Figure 97 , 2008 meeting in Madison data was presented by Kent Johnson indicating that temperatures near the PEMS instruments could be above 60°C during in

- use testing. With this in mind the steering committee elected to add temperatures as high as 50°C to the profile, where the original pro file had a maximum temperature of just over 30°C. The original and modified temperature and humidity profiles are shown in

. - 12

- 8 - 4 0 4 8 12

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Median Flow Weighted PM Concentration ( μ g/mol)

1st Percentile 99th Percentile130 SwRI Project No. 14936.12

FIGURE 97 . TEMPERATURE AND HU MIDITY PROFILE FOR E NVIRONMENTAL TESTING The temperature chamber was unable to control the humidity under a temperature of 5°C so the humidity was uncontrolled during the portion of the cycle where the temperature was between 2°C an d 5°C. The moisture content was quite small during this portion of the cycle, so this was not considered to be a significant issue. Figure 98 shows the PEMS in the tem perature and humidity chamber. 

### 0 10 20 30 40 50 60 70 80 90 100

0 10 20 30 40 50 60

0 1 2 3 4 5 6 7 8Relative Humidity (%)Temperature (°C)



### Time (hr)
Original Temp Modified Temp Original RH Modified RH131 SwRI Project No. 14936.12 FIGURE 98 . PEMS INSTALLED IN THE TEMPERATURE AND HUMIDITY CHAMBER The compressor pictured was provided by SwRI for use during environmental testing. The original compressor supplied by Horiba was mu ch smaller, but also had difficulty completing eight hours of operation without shutting off. The compressor was included in the temperature chamber so that the dilution air for the Horiba system would be affected by the same changes in temperature and hum idity that the dilution air of the Sensors and AVL systems would experience. The PM sample was transported into the chamber using a heated sample line maintained at 60°C. The temperature was set slightly above the maximum temperature of the chamber so that a constant temperature in the sample could be maintained throughout the test. There was a small portion of the end of the transfer line that was not heated, but it was extensively insulated to minimize temperature effects. The EPA PM generator was used in conjunction with the soot generator in this work to provide the particle source. The experimental setup was shown previously in Figure 81 . The PM generator is shown in Figure 99

.132 SwRI Project No. 14936.12

FIGURE 99 . THE PM GENERATOR I NSTALLED OUTSIDE THE TEMPERATURE AND HUMIDITY CHAMBER The Horiba and Sensors gaseous PEMS were i nstalled outside of the chamber along with the PM generator. The PM generator is designed to add volatile hydrocarbons, sulfuric acid, and water vapor to an elemental carbon source. An oxidation catalyst removed any volatile from the soot generator before volatile was added from the diffusion vial ovens. Unfortunately both the sulfur oven and the syringes injecting water malfunctioned during testing and neither were able to be quickly repaired. For the official temperature testing the particle source consis ted of elemental carbon from the mini

- CAST and volatile hydrocarbon from the PM generator.133 SwRI Project No. 14936.12 FIGURE 100 . HORIBA ENVIRONMENT AL TEMPERATURE MEASU REMENTS The Horiba data showed a significant amount of variability although it is unclear from this graph whether the variability is directly related to the changing temperature. For example, the measurements between observation 25 and 35 when the temperature is b elow 10°C tend to be below the cycle average. However, the second time the temperature drops below 10°C around observation 70, the measurements are above the cycle average. The temperature measurements for the Sensors PEMS are shown in Figure 101

. 

### 0 10 20 30 40 50 60

0 20 40 60 80 100 120 140 160 180 200

0 20 40 60 80 100Chamber Temperature (°C)PM Concentration (μg/mol)



### Observation Number
Avg Horiba Horiba Temperature134 SwRI Project No. 14936.12 FIGURE 101 . SENSORS ENVIRONMEN TAL TEMPERATURE MEAS UREMENTS The Sensors data displayed a high degree of variation from the mean with no clear tren d related to temperature. No valid data was captured for the last hour of operation because the Sensors PES was unable to maintain the crystals at a temperature of 50°C when the chamber temperature was above 47°C. 

### 0 10 20 30 40 50 60

0 50 100 150 200 250

0 20 40 60 80 100Chamber Temperature (°C)PM Concentration (μg/mol)



### Observation Number
Avg Sensors Sensors Temperature135 SwRI Project No. 14936.12 FIGURE 102 . A VL ENVIRONMENTAL TEM PERATURE MEASUREMENT S The AVL data exhibited excellent repeatability in comparison to the other two PEMS. Any temperature dependence by the AVL PEMS was extremely small. The temperature data was compared in the same manner as the pressu re data with the MAD values plotted in relation to the median. 

### 0 10 20 30 40 50 60

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100Chamber Temperature (°C)PM Concentration (μg/mol)



### Observation Number

Avg AVL AVL Temperature136 SwRI Project No. 14936.12 FIGURE 103 . HORIBA TEMPERATURE MEDIAN VERSUS MAD

FIGURE 104 . SENSORS TEMPERATUR E MEDIAN VERSUS MAD 

### y = 0.073x + 5.936 R2 = 0.427 y = 0.084x + 3.106 R2 = 0.767

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0

0 20 40 60 80 100 120 140 160MAD PM (μg/mol)



### Median PM Concentration (μg/mol)

Horiba

- Baseline Horiba

- Temperature

y = 0.155x + 15.834 R2 = 0.756
y = 0.040x + 22.144 R2 = 0.446
0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0

0 20 40 60 80 100 120 140 160MAD PM (μg/mol)



### Median PM Concentration (μg/mol)

Sensors

- Baseline Sensors

- Temperature137 SwRI Project No. 14936.12

FIGURE 105

. HORIBA TEMPERATURE A ND HUMIDITY MEDIAN V ERSUS MAD The MAD of the Horiba baseline is slightly higher than during the temperature test meaning that an error surface was not generated in this case. The Sensors and AVL MAD was higher for the baseline at lower concentrations, but higher for the temperature data for higher concentrations. An attempt was made to calculate an error surface for Sensors, but the pooled rms technique resulted in a slightly higher value for the baseline indicating that an error surfac e was not necessary. The error surface calculated for AVL using the same technique is shown in 

### y = 0.022x

- 0.124 R2 = 0.999 y = 0.006x + 1.072 R2 = 0.962

0.0 0.5 1.0 1.5 2.0 2.5

0 20 40 60 80 100 120 140 160MAD PM (μg/mol)



### Median PM Concentration (μg/mol)

AVL

- Baseline AVL

- Temperature138 SwRI Project No. 14936.12

FIGURE 106 . FINAL ERROR SURFAC E ENVIRONMENTAL TEMP ERATURE AND HUMIDITY AVL PM
The PM median concentrations tested were between 25 and 135 μg/mol. It was necessary to extend the error surface out to nearly 500 μg/mol to encompass the range of concentrations encountered during engine testing. The steering committee elected to cap the error surface at plus and minus 5.3 μg/mol because it w as unclear whether the errors would continue to increase outside of the concentrations observed in the temperature and humidity testing. Extending a straight line out to the median concentration of 481 μg/mol would have resulted in a 5 th

5.4 ELECTROMAGNETIC AND RADIO FREQUENCY INTE RFERENCE SCREENING percentile of 18.9 μg/mol.

The electromagnetic inference (EMI) and radio frequency interference (RFI) testing was conducted as a screening exercise without the use of PM source. During the gaseous measurement allowance program the majority of the problems encountered caused a malfunction of the PEMS to the point where it would no longer operate. The main purpose of the environmental testing was not to test the durability of the PEMS, but to quantif y any errors that might occur during the operation of a PEMS that would cause a measurement error. The same series of tests from the gaseous EMI/RFI testing were conducted as a series of individual screening tests using only HEPA filtered air. Based on the screening results it remained a possibility to conduct a full test cycle using a particle source to generate an error surface. - 12

- 8 - 4 0 4 8 12

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Median Flow Weighted PM Concentration ( μ g/mol)

1st Percentile 99th Percentile139 SwRI Project No. 14936.12 By providing filtered room air to the PEMS it would still have been possible to detect a wide range of possible measurement acc uracy issues, although some problems may only present themselves when a particle sample is present. To shorten the test time, the PEMS were triggered continuously so that time in between tests could be minimized. This meant that the Horiba system was conti nuously sampling on a filter, and the Sensors system was continuously sampling on a crystal. The Horiba system was able to sample continuously for eight hours or more on the filter since no particle source was loading on the filter. However, the Sensors sy stem had a maximum sa mple time that it was adjusted so that it would cycle through the crystals every 120 seconds. This was desirable since it exercised the operation of all eight crystals however some issues surfaced due to this testing technique. The pro blems included: both high voltage power supplies turning on at the same time, two crystals sampling at the same time, no crystal sampling even though one was available. Each of these problems occurred infrequently, and it was believed that the problems wer e a result of leaving the trigger on continuously for long periods of time since none of these issues were observed during engine testing. These were functionality issues that did not appear to have any impact on the accuracy of the measurement.

The EMI an d RFI testing was conducted in a radiation chamber with walls covered with large cones of carbon impregnated foam designed to absorb radiation and minimize reflections. Four standard Society of Automotive Engineers (SAE) tests were conducted: Bulk Current Injection, Radiated Immunity, Electrostatic Discharge, and Conducted Transients.
The Sensors and AVL PEMS were both designed to run off the vehicle’s 24 Volt electrical system so both systems were powered using a 12 V automotive battery that was continuall y charged using a 120 VAC charger supplied by Sensors. The Sensors PEMS was a 12V system so it connected directly to the battery. The AVL system is powered by 120 Volts 60 Hz AC power so a commercially available inverter was provided by AVL to convert the 12 volts DC into 120 volts AC. The Horiba system was designed to operate using a generator, therefore it was still powered using the 120 VAC wall outlets.140 SwRI Project No. 14936.12 FIGURE 107 . AVL PEMS IN THE RA DIATION CHAMBER FOR EMI AND RFI TESTING

5.4.1 Bulk Current Injection

SAE test J1113/4 Immunity to Radiated Electromagnetic Fields

– Bulk Current Injection was performed to determine the effect of electromagnetic radiation on the electrical cables of the PEMS. The specifications used are detailed in Re gion 2, Class B of the J1113/4 test protocol. A calibrated current probe was place around the electrical cable and used to inject RF current into the cable. Figure 108 shows a cable running through the bulk current injection probe.141 SwRI Project No. 14936.12 FIGURE 108 . BULK CURRENT INJEC TION PROBE The probe was positioned at 120 mm, 450 mm, and 750 mm from the cable connector to test the cable three times. For each tes t, the frequency of the current was stepped from 1 MHz to 400 MHz using the following step sizes.

1 MHz to 10 MHz

– 1 MHz step size

10 MHz to 200 MHz

– 10 MHz step size

200 MHz to 400 MHz

– 20 MHz step size
These were the maximum allowed step sizes acco rding to the SAE protocol. As with the gaseous program a 5 second dwell time was used to ensure the electromagnetic field had stabilized before switching to the next frequency. The probe was calibrated to deliver 60 milliamps of current as specified in the test procedure. Figure 109 shows the bulk current injection probe being used to test a cable on the Horiba PEMS.142 SwRI Project No. 14936.12 FIGURE 109 . HORIBA PEMS SE TUP FOR BULK CURRENT INJECTION 

# BCI Probe Test Cable143 SwRI Project No. 14936.12 FIGURE 110 . SENSORS SETUP DURI NG BULK CURRENT INJE CTION TESTING

FIGURE 111 . AVL SETUP DURING B ULK CURRENT INJECTIO N TESTING144 SwRI Project No. 14936.12 The cables tested on the Horiba system were th e sample line temperature, sample line heater power, heated enclosure heater power, sample trigger and exhaust flow signal, ethernet connection from the DCS to the electrical enclosure, ethernet connection from the TRPM to 2200, DCS 12 volt power, and two AC power cords. There were two main problems discovered when the bulk current injection was applied to the cables of the Horiba PEMS. When the line supplying power to the heater in the heated enclosure was probed, the reported total flow of the system drop ped started to rapidly fluctuate around the frequency of 10 MHz. This frequency was manually repeated after the sweep and the same behavior occurred. Figure 112 shows the response of several of the Horiba signals to the current injection.

FIGURE 112 . HORIBA FLOW DISTUR BANCE FROM BULK CURR ENT INJECTION The noise in the system appeared to have originated with several of the temperature an d pressure measurements including the filter face temperature and total flow temperature shown here. The total flow temperature and pressures began to fluctuate causing fluctuations in the total flow measurement, which in turn caused the dilution flow to f luctuate in an effort to maintain proportionality. This problem was only observed on the cable supplying heater power to the heated enclosure box. 

### 0 10 20 30 40 50 60 70 80

0 5 10 15 20 25 30 35

0 500 1000 1500 2000Temperature (°C)Flow (slpm)



### Time (sec)

OB

- PFSS_Total_1 [L/min] OB

- PFSS_Dil_1 [L/min]
OB

- PFSS_Filter_T [degC] OB

- PFSS_Tot_T [degC] 10 MHz During Sweep 10 MHz Repeated145 SwRI Project No. 14936.12 The communications between the laptop and the PEMS was disrupted when the Ethernet cables were probed. Sever al times the communications dropped out completely. The problem was intermittent around 40 to 50 MHz and 140 to 160 MHz.
The final major problem experienced by the Horiba system during BCI was a large amount of noise in the exhaust flow signal when the an alog signal cable between the OBS

- 2200 and OBS

TRPM was probed. The exhaust flow is measured by the Horiba gaseous PEMS OBS

- 2200. The OBS

- 2200 outputs the exhaust flow measurement as an analog voltage which is then read by the OBS

- TRPM Over frequencies bet ween 1 and 40 MHz the exhaust flow signal read as high as 3,000 kg/hr on the OBS

- TRPM while the measured value on the OBS

- 2200 was less than 10 kg/hr. The exhaust flow signal during the BCI sweep is shown in Figure 113

.

FIGURE 113 . HORIBA EXHAUST FLO W NOISE ON ANALOG CA BLE DURING BULK CURRENT INJECTI ON It was clear that the problem involved only the transmission of the exhaust flow signal as an analog voltage between the two systems, not the actual measurement of the exhaust flow. For in use testing this problem would be easily detectable when the two exhaust flow data from the two different files are compared; however, the TRPM relies on the analog exhaust flow signal for adjusting its dilution ratio for proportional sampling. Errors in the exhaust flow signal would cause the TRPM to lose its proportionality, and therefore any data collected with the same filter sampling would be voided. 

### - 500 0 500 1000 1500 2000 2500 3000 3500

0 200 400 600 800 1000Exhaust Flow (kg/hr)



### Time (sec)

OBS

- TRPM OBS

- 2200146 SwRI Project No. 14936.12 The c ables tested on the Sensors system were the communication cable from the PPMD to DS, and the DC power. As mentioned previously, many of the problems that were encountered with the Sensors PEMS happened intermittently and could not be reproduced at any spec ific frequencies. It was assumed that problems involving crystal sampling switching, high voltage power supplies, and fluctuations in the corona current were not induced by the bulk current injection. However, the Sensors system did experience significant communication problems when the serial cable between the PPMD and DS units was probed. The communication was repeatedly disrupted over the entire frequency range causing a loss of data. In nearly every case the communication was recovered once the radiatio n subsided allowing normal operation to continue.
The cables tested on the AVL system were the communications cable between the measuring and conditioning unit, the analog output to the Semtech DS, and two AC power cords. The BCI induced a significant amou nt of noise on the analog signal cable between the AVL MSS and the Sensors Semtech DS. The AVL system can log its own data, but the data used officially in this program and during in use testing is the data logged by the Semtech DS. Two signals are carried on the analog output wire: the measured concentration and the dilution ratio. These two numbers are multiplied together for the final reported concentration. The measured concentration did not see significant noise, but the dilution ratio signal was stron g influenced by radiation in the range of 1

- 3 MHz and 40 MHz. A comparison of the MSS measured dilution ratio and the dilution ratio recorded by the Semtech DS is shown in Figure 114

.147 SwRI Project No. 14936.12 FIGURE 114 . BCI NOISE ON AVL A NALOG OUTPUT CABLE The dilution ratio doubled from 5 to 10 during current injection at 1

- 3 MHz. The data in the plot was recorded during the frequency sweep, and the problem was re plicated manually afterwards. Since the true dilution ratio was still recorded in the MSS log file, the correct reported concentration could be recovered after testing.

5.4.2 Radiated Immunity
The response of the PEMS to continuous narrowband electromagnetic fie lds was measured using SAE test J1113/21 Electromagnetic Compatibility Measurement Procedure for Vehicle Components

– Part 21: Immunity to Electromagnetic Fields, 10 kHz to 18 GHz, Absorber

- Lined Chamber. The exact specifications of the tests performed wer e drawn from Region 2, Class B of the J1113/21 protocol. Several different antennae were used to generated electromagnetic radiation over the frequency range of 10 kHz to 1 GHz. The electromagnetic susceptibility experts at SwRI recommended ending the test at a frequency of 1 GHz rather than 18 GHz due to the very small probability of detecting any susceptibility above 1 GHz. This same approach was used in the gaseous PEMS program. The carbon impregnated foam walls of the radiation chamber was designed to a bsorb any radiation so that the PEMS would only see the direct radiation generated by the antenna. The following step sizes were used during the frequency sweeps:

**•** 10 kHz to 100 kHz

– 10 kHz step size 

### 0 2 4 6 8 10 12

0 100 200 300 400 500Dilution Ratio



### Time (sec)
Semtech DS Analog DR AVL MSS DR 1

- 3 MHz

40 MHz148 SwRI Project No. 14936.12 **•** 100 kHz to 1 MHz

– 100 kHz step size

**•** 1 MHz to 10 MHz

– 1 MHz step size

**•** 10 MHz to 200 MHz

– 2 MHz step size

**•** 200 MHz to 1 GHz

– 20 MHz step size

The SAE standard field intensity of 50 volts/meter was used with both vertical and horizontal electromagnetic radiation orientations.

FIGURE 115 . HORIBA PEMS SETUP DURING RADIATED IMMU NITY TESTING14 9 SwRI Project No. 14936.12

FIGURE 116 . SENSORS PEMS SETUP DURING RADIATED IMMU NITY TESTING

FIGURE 117 . AVL PEMS SETUP DUR ING RADIATED IMMUNIT Y TESTING

The Horiba PEMS experienced several problems related to the radiated immunity test including a loss of communications between the laptop and the instrument. This problem occurred at several different frequencies for both horizontal and vertical polarizatio ns. Because the data is150 SwRI Project No. 14936.12 logged on the laptop any loss of communication is also a loss of data. This would result in a voided test since the filter calibration relies on collecting data the entire time the filter is sampling. A problem was also observed wit h the control of the dilution ratio. During EMI and RFI testing the Horiba dilution ratio was maintained constant at 6.

FIGURE 118 . HORIBA DILUTION RA TIO FLUCTUATIONS DUR ING RADIATED IMMUNITY
The dilution ratio varied between 338 and

- 938 but only a smaller portion of the graph is shown for more detail. The external flowmeter on the end of the sample flow indicated that these rapid changes in flow were real and not just reported. The problem wi th the exhaust flow analog output signal first encountered during the bulk current injection testing was also observed during radiated immunity.
The AVL system reported a supply voltage error in the range of 200 MHz to 1 GHz of horizontal radiation. The i nverter powered down at 300 MHz shutting down the system completely.

5.4.3 Electrostatic Discharge

SAE test J1113/13 Electromagnetic Compatibility Measurement Procedure for Vehicle Components

— Part 13: Immunity to Electrostatic Discharge was performed to test the PEMS response to Electrostatic Discharges (ESDs) on surfaces and connectors. The exact procedure used was taken from Region 2, Class B of the J1113/13 standard. Approximately 40 ESDs were 

### - 100

- 80 - 60 - 40 - 20 0 20 40 60 80 100

0 500 1000 1500 2000 2500 3000 3500Dilution Ratio



### Time (sec)

Measured Dilution Ratio Setpoint Dilution Ratio 50

- 70 MHz Vertical 120

- 150 MHz Vertical151 SwRI Project No. 14936.12 supplied to each PEMS at various connector ports, exposed screws, a nd general surfaces. The test included both direct contact discharges as well as indirect discharges with the electrostatic discharge gun placed near the surface of interest. The ESD gun is shown in Figure 119

.

FIGURE 119 . ELECTROSTATIC DISC HARGE SIMULATOR The ESD gun was calibrated using an electrostatic voltmeter to deliver 4000 volts per discharge. Neither the Horiba nor the AVL PEMS exhibi ted any susceptibility to the ESD tests. The Sensors unit shut off completely with an ESD near the auxiliary connector port to which an external flow meter connects for audit purposes. This problem was confirmed with a second ESD. No issues possibly relate d to measurement accuracy were found with any of the PEMS.

5.4.4 Conducted Transients
The response of the PEMS to voltage disturbances in the 12 volt power supply cable was checked using SAE test J1113/11 Immunity to Conducted Transients on Power Leads. The tes ts were conducted using specifications found in Region 2, Class B of the J1113/11 protocol. A Schaffner NSG 5000 Automotive Electronics Test System was installed in between the 12 volt power supply and the PEMS. The Schaffner Electronics Test System delive red voltage perturbations to the PEMS of varying magnitudes and durations. The voltage spikes ranges from

- 200 to 100 volts and lasted anywhere between 250 ns and 200 ms. The tests included quick voltage recovery, slow voltage recovery, repeated voltage bu rsts, and load dump. The conducted transients tests were not performed on the Horiba PEMS, because the Horiba system was152 SwRI Project No. 14936.12 intended for use with an external generator and does not use 12 volt power. The Schaffner Test System can be seen in Figure 120 with the Sensors PEMS and Figure 121 for the AVL PEMS.

FIGURE 120 . SENSORS SETUP DURI NG CONDUCTED TRANSIE NTS TESTING

FIGURE 121 . AVL SETUP DURING C ONDUCTED TRANSIENTS TESTING The Sensors PEMS powered down during a

- 100 volt spike with quick recovery and again powered down when the magnitude of the spike was reduced to

- 50 volts and then

- 25 volts. The slow recovery voltage spikes also caused the PEMS to power down with the shortest dwell time of 40 ms causing the unit to shut down. Longer dwell times were not tested.153 SwRI Project No. 14936.12 The quick recov ery voltage spike caused the AVL inverter to power down, although it appeared to be working properly when it was restarted. The inverter shut down and started smoking during the slow recovery voltage spikes. The inverter would no longer work, and the test was not repeated with another inverter to prevent further damage.
The findings from the EMI and RFI testing were presented to the steering committee during the May 20 th

5.5 VIBRATION TESTING , 2009 in San Antonio. The steering committee declined to pursue further testing to deve lop any EMI and RFI error surfaces. The steering committee requested that Horiba investigate the issue of bulk current injection noise in the analog exhaust flow cable and that Sensors investigate the issue of exhaust flow errors during radiated immunity t esting.

The vibration testing was conducted as a screening exercise similar the method used for the EMI and RFI testing. The PEMS were operated on HEPA filtered room air while being subjected to vibration. The response of the PEMS to the screening exercise was used to determine whether further tested was needed to generate an error surface for vibration. Each PEMS was mounted on an Unholtze

- Dickie Shaker Table which was capable of vibrating the PEMS on all three axes separately by adjustin g the orientation of the PEMS. The system used a large table to vibrate horizontally. By rotating the orientation of the PEMS, this table was able to simulate longitudinal and transverse horizontal vibration. The shaker was rotated into a vertical position and a smaller platform was attached to provide vertical vibration. Due to considerations for non

- road in

- use testing, the steering committee requested that the PEMS also be rotated at a 45 degree incline on each axis as well. The horizontal vibration plat form was large enough to include all of the pieces any one type of PEMS at a time. The vertical platform and 45 degree stand were each capable of installing only a single piece of the instrumentation at a time.154 SwRI Project No. 14936.12 FIGURE 122 . SENSOR S PEMS MOUNTED FOR T RANSVERSE HORIZONTAL VIBRATION

FIGURE 123 . SENSORS PEMS MOUNT ED FOR LONGITUDINAL HORIZONTAL VIBRATION 

# Direction of vibration155 SwRI Project No. 14936.12 FIGURE 124 . SENSORS PEMS MOUNT ED FOR TRANSVERSE 45 ° VIBRATION

FIGURE 125 . SENSORS PEMS MOUNT ED FOR LONGITUDINAL 45° VIBRATION 

# Direction of vibration



# Direction of vibration156 SwRI Project No. 14936.12 FIGURE 126 . SENSORS PEMS MOUNT ED FOR VERTICAL VIBR ATION

FIGURE 127 . SENSORS PEMS MOUNT ED FOR 45° VERTICAL VIBRATION 

# Direction of vibration

Direction of vibration157 SwRI Project No. 14936.12 The po wer spectral density (PSD) from the Mil Standard 810, US Highway Truck Vibration Exposure was used in this testing. To prevent damage to the PEMS, the PSD was only operated at 25%, 50%, and 75% of the energy specified in the Mil Standard 810. After a few t ests at the 75% level, the energy levels were reduced to 10%, 25%, and 50% to maintain the integrity of the instruments. In addition each energy level was only tested for 5 minutes to minimize the chances of damaging the instruments. Figure 128 shows the PSD used for vertical vibration testing and Figure 129 shows the PSD used for horizontal testing.

FIGURE 128 . POWER SPECTRAL DEN SITY FOR VERTICAL VI BRATION TESTING158 SwRI Project No. 14936.12 5.5.1.1 FIGURE 129 . POWER SPECTRAL D ENSITY FOR HORIZONTA L VIBRATION TESTING
Because the OBS

- TRPM included three separate boxes, some of the vibration testing had to performed on individual boxes. The DCS, HE, and MEEE were all able to fit on the horizontal vibration table however only a sing le box could fit at a time on the 45 degree angle stand. The DCS and MEEE were tested on this stand; the HE box was not tested at 45 degrees, because it was felt that no errors would be detected in a box housing only a filter holder and a cyclone. A partic le source would need to be present to detect problems in this portion of the system. The vertical vibration platform was able to hold both the DCS and the MEEE, so these two boxes were tested simultaneously while the HE was again not tested. Any piece of t he system that was not being tested on the vibration stand was positioned directly next to the stand so that all flow and electrical cables could still connect to the system normally. The159 SwRI Project No. 14936.12 FIGURE 130 . HORIBA PEMS VIBRAT ION POSITIO NS The Horiba PEMS experienced a mechanical failure during testing at the 75% energy level of the transverse horizontal vibration. A bracket holding a filter inside the Mechanical Enclosure (ME) box broke causing a flow line to become blocked. The test was stopped and the bracket was replaced with one from another unit. The testing was resumed until it was discovered that one of the pressure transducers used to measure the flow was not working properly causing an error in the reported flow. This problem was believed to have originated from the broken bracket because the line that was closed off was attached to this pressure transducer. The pressure transducer was replaced with one from the other unit and the system functioned properly. All future tests were conducted at 10%, 25%, and 50% energy for all PEMS to prevent further damage. The only other damage incurred by the Horiba was a rubber foot on the bottom of one of the boxes was sheared off. The measured dilution flow exhibited a higher degree of fluctuat ions during the vibration testing than was normally observed although the differences were not significant.
The Sensors PEMS was operated without the exhaust flow meter and sample elbow attached to the unit. As can be seen in Figure 127 , the two pieces were removed to allow a HEPA filter to be placed on the inlet. Although it would have been desirable to leave these pieces attached it was considered more important to p rovide clean air to ensure a zero particle level. The Sensors160 SwRI Project No. 14936.12 PEMS experienced no mechanical failures during vibration testing although several functionality issues did occur. The exhaust flow measurement became noisier with vibration although the magnitud e of the noise was relatively small. In the EMI and RFI testing values as high as 1000 kg/hr were observed, but in vibration testing the exhaust flow never went above 50 kg/hr.
The Sensors PEMS experienced problems maintaining its total flow rate during v ibration in the vertical direction. This problem was related to the automatic drain valves on the moisture traps in the dilution flow line. These traps are designed to open automatically when enough downwards force is applied from accumulated moisture in t he reservoir, however the vertical vibration was causing the valves to repeatedly open allowing a portion of the flow to escape. Figure 131 shows an example of the tot al flow dropping while experiencing vertical vibration.

FIGURE 131 . SENSORS TOTAL FLOW DURING VIBRATION TES TING It was possible to test both of the AVL boxes simultaneously for the horizontal and vertical vibration, however they were tested separately for the 45 degree angle vibration. 

### 0 2 4 6 8 10

200 400 600 800 1000 1200 1400 1600Total Flow (slpm)



### Time (sec) None 10% 25% 50% None161 SwRI Project No. 14936.12 FIGURE 132 . AVL PEMS VIBRATION POSITIONS No mechanical problems were encountered with the AVL unit during vibration testing although the measurement did exhibit vibration induced noise. The measured soot concentration fluctuated with increasing amplitude as the vibration increased for all orienta tions tested. The peak measurement recorded was about plus and minus 0.2 mg/m3 with more typical values swinging between

- 0.1 mg/m3 and 0.1 mg/m3. A typical measurement is shown in Figure 133

.162 SwRI Project No. 14936.12

FIGURE 133 . AVL SOOT MEASUREME NT NOISE DURING VIBR ATION TESTING This particular data was from the 45 degree angle longitudinal vibration. The noise in the measurement has no bias indicating that it will average out over larger periods of time. The 30 second average is shown since this is likely the measurement error that would be observed during an NTE event. The maximum error of the 30 second average was

- 0.009 mg/m3 which is relatively insignificant bas ed on PM concentrations expected at the threshold. However, it is unclear from this testing whether the noise could be greater if a particle source was present. The results from the vibration testing were presented to the steering committee at the Septembe r 22 nd

6 MODEL ING RESULTS , 2009 meeting in Riverside. The steering committee declined to perform additional tests to create an error surface for vibration.

The main objective of this portion of the project was to use Monte Carlo techniques (e.g. random sampling) in an error model to simulate the combined effects of all the agreed

- upon sources of PEMS error incremental to lab error on the components of the brake

- specific (BS) PM emissions . This was accomplished by creating “error surfaces” for the Monte Carlo simulation to sample, based upon the results of a variety of lab experiments. The constructed model was simulated for thousands of trials (i.e., iterations) using data taken from a reference data set of 141 unique NTE events. The model results were used to determine the brake

- specific additive 

### - 0.20 - 0.15 - 0.10 - 0.05

0.00 0.05 0.10 0.15 0.20

0 200 400 600 800 1000 1200Reported Soot Concentration (mg/m3)



### Time (sec)
Concentration dil. corrected [mg/m3] 30 Sec Avg 75% 25% 50% None163 SwRI Project No. 14936.12 measurement a llowances for PM by three different calculation methods for three different PEMS model units.

The error surfaces were generated from the results of each of the engine dynamometer and environmental chamber lab oratory test s . The engine

- lab

- test error surfaces covered the domain of error versus the magnitude of the signal to which the error was to be applied (i.e., 1 st to 99 th

6.1 CONVERGENCE RESULTS FROM MC RUNS percentile error vs. concentration, flow, torque, etc.). The environmental

- test error surfaces for shock and vibration, and electromagnetic and radio frequenc y interference (EMI/RFI) was not included because no error surfaces were generated . The environmental test error surfaces for pressure and temperature were characteristica lly different because they covered the domain of the environmental

- test cycle time versus the magnitude of the signal to which the error was to be applied (i.e., error at a selected time vs. concentration).

This section contains a summary of the checks to determine if the convergence criteria were met for the simulation runs. Section 2.9 on Convergence and Number of Trials contains a detailed description of the convergence methodology and the procedures followed to check for convergence for the reference NTE event trials. This procedure was applied to the simulation data obtained for each of the three PEMS units and all ap plicable calculation methods.
Figure 134 through Figure 140 contain plots o f the 90% confidence interval widths at the 95 th

Table 22 percentile delta differences (expressed as a percent of the BSPM emissions NTE threshold) versus the ideal BSPM emissions for the 141 individual reference NTE events. This is done for each of the three PEMS units and the applicable calculation methods. A summary of the results is given in . Of interest was whether or not the simulations converged within 1% of t he threshold value. As can be seen in the plots, a majority of the reference NTE events did not converge within 1% of the BSPM threshold. However, a majority did converge within 2% of the threshold. For the three PEMS units and calculation methods, the maximum percent of the confidence interval widths ranged from 1.82% for the AVL Method 1 to 2.76% for Sensors Method 1. Upon examination of the delta BSPM distributions for the various reference NTE events, those that had a high percentage above the BSPM threshold at the 95 th percentile generally had low input PM concentration levels.164 SwRI Project No. 14936.12 FIGURE 134 . CONVERGENCE FOR A VL METHOD 1 AS A PER CENT OF BSPM THRESHOLD 

### 0.00 0.40 0.80 1.20 1.60 2.00

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

## AVL Method 1

0.00 0.40 0.80 1.20 1.60 2.00 2.40

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

### AVL Method 2165 SwRI Project No. 14936.12 FIGURE 135 . CONVERGENCE FOR A VL METHOD 2 AS A PERCENT OF BSPM THRESHOLD

FIGURE 136 . CONVERGENCE FOR A VL METHOD 3 AS A PER CENT OF BSPM THRESHOLD 

### 0.00 0.40 0.80 1.20 1.60 2.00 2.40

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

### AVL Method 3166 SwRI Project No. 14936.12 FIGURE 137 . CONVERGENCE FOR H ORIBA METHOD 1 AS A PERCENT OF BSPM THRESHOLD 

### 0.00 0.40 0.80 1.20 1.60 2.00 2.40 2.80

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

## Horiba Method 1

0.00 0.40 0.80 1.20 1.60 2.00 2.40 2.80

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

### Horiba Method 2167 SwRI Project No. 14936.12 FIGURE 138 . CONVERGENCE FOR H ORIBA METHOD 2 AS A PERCENT OF BSPM THRESHOLD

FIGURE 139 . CONVERGENCE FOR S ENSORS METHOD 1 AS A PERCENT OF BSPM THRESHOLD 

### 0.00 0.40 0.80 1.20 1.60 2.00 2.40 2.80 3.20

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

## Sensors Method 1168 SwRI Project No. 14936.12 FIGURE 140 . CONVERGENCE FOR S ENSORS METHOD 2 A S A PERCENT OF BSPM THRESHOLD

TABLE 22 . SUMMARY OF NUMBER OF REFERENCE NTES ME ETING 2% CONVERGENCE

PEMS uNIT mETHOD mIN MAX # nTES WITH in 2 % Convergence % NTEs within 2% CoNvergence A VL 1 0.4467 1.8211 141 100% 2 0.4446 1.9766 141 100% 3 0.4240 1.9215 141 100% H oriba 1 0.4158 2.5663 129 91% 2 0.4414 2.5447 131 93% Sensors 1 0.7883 2.7664 125 87% 2 0.8613 2.6544 131 93%

6.2 SENSITIVITY BASED ON BIAS AND VARIANCE
This section contains a summary of the error surfaces that contributed the most to the bias of the generated BS emissions. The sensitivity charts developed in Crystal Ball help identify the error surfaces (assumptions) that are sensitive to changes in vari ation with respect to their effect on the three delta BS emissions. Another type of sensitivity examined in this study was 

### 0.00 0.40 0.80 1.20 1.60 2.00 2.40 2.80

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

### Sensors Method 2169 SwRI Project No. 14936.12 concerned with the effects of potential “bias” in error surfaces and their effects on the forecast values. In order to study these effects a new error surface assumption was added to the MS simulation model for each of the original 31 error surfaces.
This assumption was sampled as a discrete binary distribution (i.e., on or off) during the simulation. For each trial of the simulation , 31 original error surfaces and 31 ‘on/off’ error surfaces were sampled according to their defined sample distribution. If the ‘on/off’ error surface produced an ‘off’ condition, the delta emissions from that particular error surface were not added to th e BS emissions computations for the BS emissions ‘with errors’. Similarly, if the ‘on/off’ error surface produced an ‘on’ condition, the delta emissions from that particular error surface were added to the BS emissions calculations.
During every trial of the simulation, the exclusions due to the ‘off’ conditions resulted in various combinations of the error surface delta emissions being added to the BS emissions ‘with errors’ computations. Over the course of a MC simulation with thousands of trials, the s ensitivity of a particular error either ‘on’ or ‘off’ was assessed by examining the change in the forecast delta emission. Therefore, in a single MC simulation of a reference NTE event sensitivities due to variance and/or bias were explored.

Simulation re sults from the reference NTE events produced sensitivity values for all 95th percentile delta emissions by all three PEMS units and applicable calculation methods

. Table 23 through Table 29 summarize the error surfaces in which either the contribution

- to

- variance normalized sensitivity value or the ‘on/off’ bias check for the error surface was at least 5% in magnitude compared to all the other error surfaces. If the label in the error surface contains the words ‘Delta’ then it represents a check for bias; otherwise, the error surface indicates a check for variance

. Table 23 through Table 25 list the sensitivity and bias descriptive statistics for the delt a BSPM emissions for the AVL PEMS for Methods 1, 2 and 3, respectively. For all three methods, the largest mean normalized variance was from the bias effect due to error surface #1, SSPM.170 SwRI Project No. 14936.12 TABLE 23 . ERROR SURFACE SEN SITIVITY TO BIAS AND VARIANCE FO R 141 REFERENCE NTE EVENTS FOR AVL BSPM METHOD 1
Error Surface # Error Surface # Ref NTE Events Avg Contribution to Normalized Variance, % Min Contribution, % Max Contribution, %
1 SS PM 141 11.34 7.59 35.25 2 TR PM 4 7.00 6.42 7.34 20 SS Exhaust Flow 4 5.57 5.32 6.05
31 Torque Warm

up 10

- 6.64

- 11.22

- 5.32
35 Torque Engine Manuf 1

- 6.27

- 6.27

- 6.27
Delta Exhaust Flow Pulsation 3 5.78 5.56 5.99

Delta SS PM 141

- 74.37

- 83.67

- 21.02

TABLE 24 . ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR AVL BSPM
Error Surface # METHOD 2

Error Surface # Ref NTE Events Avg Contribution to Normalized Variance, % Min Contribution, % Max Contribution, %
1 SS PM 141 21.48 7.44 60.56 2 TR PM 38 9.20 5.53 12.31 31 Torque Warm

up 8

- 6.82

- 10.77

- 5.24
35 Torque Engine Manuf 1

- 6.12

- 6.12

- 6.12
45 SS CO 4 2 - 6.89

- 7.82

- 5.32 Delta SS PM 141

- 63.27

- 83.77

- 20.95171 SwRI Project No. 14936.12 TABLE 25 . ERROR SURFACE SEN SITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR AVL BSPM METHOD 3
Error Surface # Error Surface # Ref NTE Events Avg Contribution to Normalized Variance, % Min Contribution, % Max Contribution, %
1 SS PM 141 20.91 7.71 60.50 2 TR PM 36 9.27 5.17 12.52 31 Torque Warm

up 9

- 6.62

- 10.77

- 5.14
35 Torque Engine Manuf 1

- 6.12

- 6.12

- 6.12
45 SS CO 4 2 - 6.96

- 7.89

- 5.40 Delta SS PM 141

- 64.04

- 83.76

- 20.93

Table 26 through Table 27 list the sensitivity and bias descriptive statistics for the delta BSPM emissions for Methods 1 and 2, respectively. For both methods, the highest mean normalized variances were from the bias and variance due to error surface #1, SS PM.

TABLE 26 . ERROR SURFACE SEN SITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR HORIB A BSPM METHOD 1
Error Surface # Error Surface # Ref NTE Events Avg Contribution to Normalized Variance, % Min Contribution, % Max Contribution, %
1 SS PM 138 41.86 6.65 80.39 2 TR PM 10 5.84 5.04 7.18 20 SS Exhaust Flow 26 8.94 5.00 7.80
31 Torque Warm

up 100

- 8.34

- 15.96

- 5.08
35 Torque Engine Manuf 29

- 5.72

- 8.07

- 5.00
Delta Exhaust Flow Pulsation 31 6.67 5.07 10.01

Delta SS PM 83

- 32.05

- 83.42 73.96172 SwRI Project No. 14936.12 TABLE 27 . ERROR SURFACE SEN SITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR HORIBA BSPM METH OD 2
Error Surface # Error Surface # Ref NTE Events Avg Contribution to Normalized Variance, % Min Contribution, % Max Contribution, %
1 SS PM 138 39.55 6.99 80.30 2 TR PM 10 5.89 5.00 7.42 31 Torque Warm

up 95

- 8.19

- 15.04

- 5.03
35 Torque Engine Manuf 24

- 5.72

- 7.65

- 5.08
42 Fuel Rate Engine Manuf 7 5.52 5.07 6.34
45 SS CO 54 2 - 6.60

- 9.83

- 5.09 Delta SS PM 89

- 42.60

- 83.42 75.48

Table 28 through Table 29 list the sensitivity and bias descriptive statistics for the delta BSPM emissions for the Sensors PEMS for Methods 1 and 2, respectively. For both methods, the highest mean normalized variance was from the bias and variance due to error surface #1, SSPM.

TABLE 28 . ERROR SURFACE SEN SITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR SENSORS BSPM MET HOD 1
Error Surface # Error Surface # Ref NTE Events Avg Contribution to Normalized Variance, % Min Contribution, % Max Contribution, %
1 SS PM 138 46.92 6.57 78.61 2 TR PM 100 8.48 5.00 12.31 20 SS Exhaust Flow 8 6.11 8.06 7.56
31 Torque Warm

up 30

- 8.28

- 11.57

- 5.02
35 Torque Engine Manuf 10

- 5.60

- 6.06

- 5.14
Delta Exhaust Flow Pulsation 9 7.01 5.56 8.79

Delta SS PM 120

- 31.19

- 86.58 75.35173 SwRI Project No. 14936.12 TABLE 29 . ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR SENSORS BSPM MET HOD 2
Error Surface # Error Surface # Ref NTE Events Avg Contribution to Normalized Variance, % Min Contribution, % Max Contribution, %
1 SS PM 138 45.92 6.68 77.78 2 TR PM 101 8.48 5.03 12.52 31 Torque Warm

up 24

- 7.84

- 12.01

- 5.08
35 Torque Engine Manuf 5

- 5.61

- 6.06

- 5.01
42 Fuel Rate Engine Manuf 4 6.18 5.72 6.53
45 SS CO 10 2 - 6.84

- 9.77

- 5.15 Delta SS PM 127

- 28.96

- 86.67 75.75

The contribution to normalized variance and bias sensitivities from Table 23 through Table 29

are illustrated pictorially as box plots in Figure 141 through Figure 147 for BSPM by PEMS unit for Methods 1, 2 and 3. Only the error surfaces with at leas t 35 of the 141 reference NTE events (1/4 of the events) are included as box plots. The mean normalized variance for each of the plotted error surfaces is noted by a “+” symbol in the boxes. The error surface with the largest mean normalized variance is plotted at the left of the chart. The error surface with the second largest mean normalized variance is plotted second from the left, and so on. Figure 141 and Figure 142 demonstrate the high sensitivity to the negative bias for error surface #1, PM SS. Figure 145 and Figure 146 show a large variance effect due to PM SS.174 SwRI Project No. 14936.12 FIGURE 141 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR AVL BSPM METHOD 1175 SwRI Project No. 14936.12 FIGURE 142 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR AVL METHOD 2176 SwRI Project No. 14936.12 FIGURE 143 . BOX PLOT OF ERROR SURFACE SENSIT IVITY BASED ON BIAS AND VARIANCE FOR AVL METHOD 3177 SwRI Project No. 14936.12 FIGURE 144 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR HOR IBA BSPM METHOD 1178 SwRI Project No. 14936.12 FIGURE 145 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR HOR IBA BSPM METHOD 2179 SwRI Project No. 14936.12 FIGURE 146 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR SENSORS BSPM MET HOD 1180 SwRI Project No. 14936.12 FIGURE 147 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR SENSORS BSPM MET HOD 2181 SwRI Project No. 14936.12 TABLE 30 . SUMMARY OF ERROR SURFACE SENSITIVITIE S TO BIAS AND VARIANCE FOR BSPM ME THOD 1

Method 1

AVL Horiba Sensors

Error Surface # Error Surface # NTE Events Avg Contribution to Normalized Variance, % # NTE Events Avg Contribution to Normalized Variance, % # NTE Events Avg Contribution to

Normalized Variance, %

1 SS PM 141 11.34 138 41.86 138 46.92

2 TR PM 4 7.00 10 5.84 100 8.48

20 SS Exhaust Flow 4 5.57 26 8.94 8 6.11

31 Torque Warm

- up 10

- 6.64 100

- 8.34 30

- 8.28

35 Torque Engine Manuf 1

- 6.27 29

- 5.72 10

- 5.60

Delta Exhaust Flow

Pulsation 3 5.78 31 6.67 9 7.01

Delta SS PM 141

- 74.37 83

- 32.05 120

- 31.19182 SwRI Project No. 14936.12 TABLE 31 . SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND VARIANCE FOR BSPM ME THOD 2
Method 2

AVL Horiba Sensors

Error Surface # Error Surface # NTE Events Avg Contribution to Normalized Variance, % # NTE Events Avg Contribution to Normalized Variance, % # NTE Events Avg Contribution to

Normalized Variance, %

1 SS PM 141 21.48 138 39.55 138 45.92

2 TR PM 38 9.20 10 5.89 101 8.48

31 Torque Warm

- up 8

- 6.82 95

- 8.19 24

- 7.84

35 Torque Engine Manuf 1

- 6.12 24

- 5.72 5

- 5.61

42 Fuel Rate Engine Manuf 7 5.52 4 6.18

45 SS CO2 4

- 6.89 54

- 6.60 10

- 6.84

Delta SS PM 141

- 63.27 89

- 42.60 127

- 28.96183 SwRI Project No. 14936.12 TABLE 32 . SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND VARIANCE FOR BSPM METHOD 3
Method 3

AVL

Error Surface # Error Surface # NTE Events Avg Contribution to

Normalized Variance, %

1 SS PM 141 20.91

2 TR PM 36 9.27

31 Torque Warm

up 9

- 6.62

35 Torque Engine Manuf 1

- 6.12

45 SS CO2 4

- 6.96

Delta SS PM 141

- 64.04

6.3 VALIDATION RESULTS
This section contains a summary of the model validation results, Section 2.13 on Validation contains a more detailed description of the validation methodology utilized both in the simulation and in the on

- road data collection efforts.
During the Monte Carlo simulation of the 141 reference NTE events some of the error surfaces were excluded in the computation of the BS emissions ‘with errors’ so that the simulation represented conditions used in collecting the on

- road data. The error surfaces excluded were torque errors (# 29

- 32, 34, 35), fuel rate engine manufacturers (#42), dynamic speed (#43) and dynamic fuel rate (#44). For each reference NTE event, the difference in BSPM emissions was computed as

delta BSPM = BSPM with “Validation error”

– “Ideal” BSPM.

These delt a BSPM emissions were computed for each of the three PEMS units and all applicable calculation methods. The 5 th , 50 th and 95 th percentiles were identified from the distributions of184 SwRI Project No. 14936.12 the delta BSPM emissions during the Monte Carlo simulation using the valid ation error surfaces only. Figure 148 through Figure 154 depict the validati on percentiles for the AVL PEMS unit for methods 1, 2 and 3, respectively. Similar validation plots for the Horiba PEMS unit are illustrated in Figure 151 and Figure 152 for methods 1 and 2, respectively. Sensors PEMS validation plots for methods 1 and 2 are shown in Figure 153 and Figure 154

.

FIGURE 148 . VALIDATION PERCEN TILES FOR THE 141 RE FERENCE N TE EVENTS FOR AVL METHO D 1 - 0.020 - 0.015 - 0.010 - 0.005 0.000 0.005

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation AVL 1 5th Validation AVL 1 50th Validation AVL 1 95th185 SwRI Project No. 14936.12 FIGURE 149 . VALIDATION PERCEN TILES FOR THE 141 RE FERENCE NTE EVENTS FOR AVL METHO D 2

FIGURE 150 . VALIDATION PERCEN TILES FOR 141 REFERE NCE NTE EVENTS FOR AVL METHOD 3 - 0.020 - 0.015 - 0.010 - 0.005 0.000 0.005

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation AVL 2 5th Validation AVL 2 50th Validation AVL 2 95th

- 0.020 - 0.015 - 0.010 - 0.005 0.000 0.005

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation AVL 3 5th Validation AVL 3 50th Validation AVL 3 95th186 SwRI Project No. 14936.12

FIGURE 151 . VALIDATION PERCEN TILES FOR 141 REFERE NCE NTE EVENTS FOR HORIBA METHOD 1

FIGURE 152 . VALIDATION PERCEN TILES FOR 141 REFERE NCE NTE EVENTS FOR HORIBA METHOD 2 - 0.015 - 0.010 - 0.005 0.000 0.005 0.010 0.015 0.020 0.025

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation Horiba 1 5th Validation Horiba 1 50th Validation Horiba 1 95th

- 0.015 - 0.010 - 0.005 0.000 0.005 0.010 0.015 0.020

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation Horiba 2 5th Validation Horiba 2 50th Validation Horiba 2 95th187 SwRI Project No. 14936.12

FIGURE 153 . VALIDATION PERCEN TILES FOR 141 REFERE NCE NTE EVENTS FOR SENSORS METHOD 1

FIGURE 154 . VALIDATION PERCEN TILES FOR 141 REFERE NCE NTE EVENTS FOR SENSORS METHOD 2 - 0.035 - 0.030 - 0.025 - 0.020 - 0.015 - 0.010 - 0.005 0.000 0.005 0.010 0.015

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation Sensors 1 5th Validation Sensors 1 50th Validation Sensors 1 95th

- 0.035 - 0.030 - 0.025 - 0.020 - 0.015 - 0.010 - 0.005 0.000 0.005 0.010 0.015

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation Sensors 2 5th Validation Sensors 2 50th Validation Sensors 2 95th188 SwRI Project No. 14936.12 The 5 th and 95 th percentiles of the validation delta BSPM were separately fit for the AVL and Sensors PEMS units using a simple linear regression model. However, the criteria for accepting the linear fits was not met for any of the calculations methods for these two PEMS units. Thus, loess regression fits were used to determine the best functional representation for the 5 th and 95 th delta BSPM based on the validation simulation modeling. These loess fits for the 95 th and 5 th Figure 155 percentiles for the Sensors unit methods 1 an d 2 can be found in through Figure 158
, respectively. The loess f its for the 95 th and 5 th Figure 159 percentiles for the AVL units methods 1, 2 and 3 can be found in through Figure 162 , respectively. The loess smoothing parameters for the regression fits are listed in Table 33

. TABLE 33 . LOESS SMOOTHING P ARAMETERS FOR VALIDA TION PERCENTILES
PEMS Method 5 th 95 Percentile th

Sensors Percentile

1 0.290 0.570

2 0.290 0.570

AVL 1 0.294 0.755

2 0.294 0.777

3 0.294 0.777189 SwRI Project No. 14936.12

FIGURE 155 . VALIDATION 95TH PERCENTILE BSPM DELT AS LOESS FIT FOR SENSORS METHOD 1190 SwRI Project No. 14936.12 FIGURE 156 . VALIDATION 5TH PE RCENTILE BSPM DELTAS LOESS FIT FOR SENSORS METHOD 1

FIGURE 157 . VALIDATION 95TH P ERCENTILE BSPM DELTA S LOESS FIT FOR SENSORS METHOD 2191 SwRI Project No. 14936.12 FIGURE 158 . VALIDATION 5TH PE RCENTILE BSPM DELTAS LOESS FIT FOR SENSORS METHOD 2

FIGURE 159 . VALIDATION 95TH P ERCENTILE BSPM DELTA S LOESS FIT FOR AVL METHOD 1192 SwRI Project No. 14936.12 FIGURE 160 . VALIDATION 5TH PE RCENTILE BSPM DELTAS LOESS FIT FOR AVL METHOD 1

FIGURE 161 . VALIDATION 95TH P ERCENTILE BSPM DELTA S LOESS FIT FOR AVL METHOD 2193 SwRI Project No. 14936.12 FIGURE 162 . VALIDATION 5TH PE RCENTILE BSPM DELTAS LOESS FI T FOR AVL METHOD 2

FIGURE 163 . VALIDATION 95TH P ERCENTILE BSPM DELTA S LOESS FIT FOR AVL METHOD 3194 SwRI Project No. 14936.12 FIGURE 164 . VALIDATION 5TH PE RCENTILE BSPM DELTAS LOESS FIT FOR AVL METHOD 3

6.4 MEASUREMENT ERROR AL LOWANCE RESULTS
This section contains a summary of the measurement error allowance results using both a regression method and a median method to determine the measurement allowance

. Section 2.12

on Measurement Allowance contains a detailed description of the methodology followed in determining these values. This procedure was applied to the simulation data for all 141 reference NTE events obtained for all three calculation methods for the AVL PEMS and for calculations methods 1 and 2 for the Horiba and Sensors PEMS.
Figure 165 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for AVL Method 1. Included in the plot is the equation for the fitted regression line, and the R

- square (R 2 ) value and roo t mean square error (RMSE) value for the regression fit. The two symbols in the plot represent reference NTE events where there was a dominant bias effect due to the SSPM error surface (diamond symbol) or there was a dominant variance effect due to the SS PM error surface (square symbol). The R

square value indicates that 47.95% of the variation in the 95 th percentile BSPM values is explained by the ideal BSPM values for the AVL Method 1 data. The RMSE value of 0.0008 displays the size of the estimated st andard deviation of the predicted 95 th percentile BSPM values.195 SwRI Project No. 14936.12 Table 34 includes a comparison of the results of the regression method based on Fi gure 165 and the median method as described in the Section 2.10 on Measurement Allowance . Under the heading of “Regression Method” in the table, it is shown that only the R

- square criterion was not met by the data. Thus, the Median Method must be used. Under the heading “Median Method” in the table, the measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSPM values, is 0.661% when expressed as a percent of the threshold of 0.02 g/hp

- hr.

FIGURE 165 . REGRESSION PLOT O F 95 TH 

### y = 0.116x

- 0.002 R2 = 0.4795 RMSE = 0.0008

- 0.0020 - 0.0010 0.0000 0.0010 0.0020 0.0030 0.0040 0.0050

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM g/hp-hr



### Ideal BSPM, g/hp

- hr 

## AVL Method 1

Delta SSPM ic SSPM

PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR AVL M ETHOD 1196 SwRI Project No. 14936.12 TABLE 34 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIA N METHODS FOR AVL ME THOD 1 Regression Method Median Method
R 0.4795

2 Did Not Meet Criteria RMSE (SEE) 0.0008 Met Criteria 5% Median Ideal 0.0191007 Predicted 95 th 0.0003399 % Delta at Threshold Median 95 th 0.0001322 % Delta
Measurement Error @ Threshold = 0.02 1.6993% Measurement Error @ Threshold = 0.02 0.661%

Figure 166 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for AVL Method 2. The R

- square value indicates that 46.02% o f the variation in the 95 th

Table 35 percentile BSPM values is explained by the Ideal BSPM values for the AVL Method 2 data. The RMSE value is 0.0008.
includes a comp arison of the results of the regression method based on Figure 166 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- square criterion for using this method is not met by the data. Thus, the Median Method m ust be used. Under the heading “Median Method” in the table, the measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSPM values, is

- 2.375% when expressed as a percent of the threshold value of 0.02.197 SwRI Project No. 14936.12

FIGU RE 166 . REGRESSION PLOT O F 95 TH

TABLE 35 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIA N METHODS FOR AVL ME THOD 2 PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR AVL M ETHOD 2

Regression M ethod Median Method
R 0.4602

2 Did Not Meet Criteria RMSE (SEE) 0.0008 Met Criteria 5% Median Ideal 0.0191007 Predicted 95 th

- 0.0002124 % Delta at Threshold Median 95 th

- 0.0004751 % Delta Measurement Error @ Threshold = 0.02 - 1.0618% Measurement Error @ Threshold = 0.02 - 2.375%

Figure 167 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for AVL Method 3. The R

- square value indicates that 46.20% o f the variation in the 95 th 

### y = 0.0996x

- 0.0022 R2 = 0.4602 RMSE = 0.0008

- 0.0020 - 0.0010 0.0000 0.0010 0.0020 0.0030 0.0040

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr



### Ideal BSPM, g/hp

- hr 

## AVL Method 2

Delta SSPM ic SSPM

percentile BSPM values is explained by the Ideal BSPM values for the AVL Method 3 data. The RMSE value is 0.0008.198 SwRI Project No. 14936.12 Table 36 includes a comparison of the results of the regression method based on Figure 167 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- square criterion for using this method is not met by the data. Thus, the Median Method m ust be used. Under the heading “Median Method” in the table, the measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSPM values, is

- 2.383% when expressed as a percent of the threshold value of 0.02.

FIGUR E 167 . REGRESSION PLOT O F 95 TH 

### y = 0.1001x

- 0.0022 R2 = 0.462 RMSE = 0.0008

- 0.0020 - 0.0010 0.0000 0.0010 0.0020 0.0030 0.0040

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr



### Ideal BSPM, g/hp

- hr 

## AVL Method 3

Delta SSPM ic SSPM

PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR AVL M ETHOD 3199 SwRI Project No. 14936.12 TABLE 36 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIA N METHODS FOR AVL ME THOD 3 Regression Me thod Median Method
R 0.4620

2 Did Not Meet Criteria RMSE (SEE) 0.0008 Met Criteria 5% Median Ideal 0.0191007 Predicted 95 th

- 0.0002118 % Delta at Threshold Median 95 th

- 0.0004766 % Delta Measurement Error @ Threshold = 0.02 - 1.0592% Measurement Error @ Threshold = 0.02 - 2.383%

Figure 168 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for Horiba Method 1. The R

- square value indicates that 26.57 % of the variation in the 95 th

Table 37 percentile BSPM values is explained by the Ideal BSPM values for the Horiba Method 1 data. The RMSE value is 0.0041.

incl udes a comparison of the results of the regression method based on Figure 168 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- square and the RMSE criteria for using this method were not met by the data. Thus, the M edian Method must be used. Under the heading “Median Method” in the table, the measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSPM values, is 54.379 % when expressed as a percent of the threshold value o f 0.02.200 SwRI Project No. 14936.12

FIGURE 168 . REGRESSION PLOT O F 95 TH

TABLE 37 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIA N METHODS FOR HORIBA METHOD 1 PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR HORIB A METHOD 1

Regression Method Median Method
R 0.2657

2 Did Not Meet Criteria RMSE (SEE) 0.0041 Did Not Meet Criteria 5% Median Ideal 0.0191007 Predicted 95 th 0.0102566 % Delta at Threshold Median 95 th 0.0108759 % Delta Measurement Error @ Threshold = 0.02 51.2831% Measurement Error @ Threshold = 0.02 54.379%

Figure 169 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for Horiba Method 2. The R

- square value indicates that 25.97 % of the variation in the 95 th 

### y = - 0.3555x + 0.0174 R2 = 0.2657 RMSE = 0.0041

- 0.0050 0.0000 0.0050 0.0100 0.0150 0.0200 0.0250

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr



### Ideal BSPM, g/hp

- hr 

## Horiba Method 1

Delta SSPM ic SSPM

percentile BSPM values is explained by the Ideal BSPM values for the Horiba Method 2 data. The RMSE value is 0.0041.201 SwRI Project No. 14936.12 Table 38 includes a comparison of the results of the regression method based on Figure 169 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- sq uare and the RMSE criteria for using this method were not met by the data. Thus, the Median Method must be used. Under the heading “Median Method” in the table, the measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSPM values, is 50.079 % when expressed as a percent of the threshold value of 0.02.

FIGURE 169 . REGRESSION PLOT O F 9 5 TH 

### y = - 0.3515x + 0.0165 R2 = 0.2597 RMSE = 0.0041

- 0.0050 0.0000 0.0050 0.0100 0.0150 0.0200

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr



### Ideal BSPM, g/hp

- hr 

## Horiba Method 2

Delta SSPM ic SSPM

PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR HORIB A METHOD 2202 SwRI Project No. 14936.12 TABLE 38 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIA N METHODS FOR HORIBA METHOD 2 Regression Method Median Method
R 0.2597

2 Did Not Meet Criteria RMSE (SEE) 0.0041 Did Not Meet Criteria 5% Median Ideal 0.0191007 Predicted 95 th 0.0094282 % Delta at Threshold Median 95 th 0.0100158 % Delta Measurement Error @ Threshold = 0.02 47.1408% Measurement Error @ Threshold = 0.02 50.079%

Figure 170 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for Sensors Method 1. The R

- square value indicates that 45.49 % of the variation in the 95 th

Table 39 percentile BSPM values is explained by the Ideal BSPM values for the Sensors Method 1 data. The RMSE value is 0.0029.
includes a comparison of the results of the regression method based on Figure 170 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- square an d the RMSE criteria for using this method were not met by the data. Thus, the Median Method must be used. Under the heading “Median Method” in the table, the measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSPM values, is 34.361 % when expressed as a percent of the threshold value of 0.02.203 SwRI Project No. 14936.12

FIGURE 170 . REGRESSION PLOT O F 95 TH

TABLE 39 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIA N METHODS FOR SENSOR S METHOD 1 PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR SENSO RS METHOD 1

Regression Method Median Method
R 0.4549

2 Did Not Meet Criteria RMSE (SEE) 0.0029 Did Not Meet Criteria 5% Median Ideal 0.0191007 Predicted 95 th 0.0066785 % Delta at Threshold Median 95 th 0.00687227 % Delta Measurement Error @ Threshold = 0.02 33.3924% Measurement Error @ Threshold = 0.02 34.361%

Figure 171 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for Sensors Method 2. The R

- square value indicates that 45.03 % of the variation in the 95 th 

### y = - 0.3866x + 0.0144 R2 = 0.4549 RMSE = 0.0029

- 0.0150 - 0.0100 - 0.0050 0.0000 0.0050 0.0100 0.0150

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr



### Ideal BSPM, g/hp

- hr 

## Sensors Method 1

Delta SSPM ic SSPM

percentile BSPM values is explained by the Ideal BSPM values for the S ensors Method 1 data. The RMSE value is 0.0030.204 SwRI Project No. 14936.12 Table 40 includes a comparison of the results of the regression method based on Figure 171 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- square and the RMSE criteria for using this method were not met by the data. Thus, the Median Method must be used. Under the heading “Median Method” in the table, the measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSPM values, is 30 .285 % when expressed as a pe rcent of the threshold value of 0.02.

FIGURE 171 . REGRESSION PLOT O F 95 TH 

### y = - 0.3875x + 0.0137 R2 = 0.4503 RMSE = 0.0030

- 0.0150 - 0.0100 - 0.0050 0.0000 0.0050 0.0100 0.0150

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr



### Ideal BSPM, g/hp

- hr 

## Sensors Method 2

Delta SSPM ic SSPM

PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR SENSO RS METHOD 2205 SwRI Project No. 14936.12 TABLE 40 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIAN METHODS F OR SENSORS METHOD 2 Regression Method Median Method
R 0.4503

2 Did Not Meet Criteria RMSE (SEE) 0.0030 Did Not Meet Criteria 5% Median Ideal 0.0191007 Predicted 95 th 0.0059333 % Delta at Threshold Median 95 th 0.0060569 % Delta Measurement Error @ Threshold = 0.02 29.6663% Measurement Error @ Threshold = 0.02 30.285%

Table 41 contains a summary of the measurement error values contained in Table 34 through Table 40 . The values are categorized by PEMS unit and by calculation method.

TABLE 41 . BSPM MEASUREMENT ERROR IN PERCENT OF NTE THRESHOLD BY PEMS AND CALCULATION METHOD
Measurement Errors (%) at Respective NTE Threshold

PEMS Method 1 Exhaust Flow Torque

- Speed Method 2 Exhaust and Fuel

Flow Torque

Speed Method 3 Fuel Flow Torque

- Speed

AVL 0.661

- 2.375

- 2.383

Horiba 54.379 50.079 n/a

Sensors 34.361 30.285 n/a

Table 42 includes in the measurement allowance selected based on the minimum normalized PM. The AVL was not used in the measurement allowance determination because the AVL at the start of the program was not accepted as an official PEMS, and the measurement Steering Committee had decided that the measurement allowance would only be based on the Sensors or the Horiba PEMS

.206 SwRI Project No. 14936.12

TABLE 42

. MEASUREMENT ALLOWANC E AT NTE THRESHOLD B Y EMISSIONS FOR METHOD 2

PEMS Method 2 Measurement Error % NTE Threshold g/hp

- hr Measurement Allowance,

g/hp

- hr

Sensors 30.285 0.02 0.00605

On

- road PM emissions were gathered from selected routes driven to collect emissions data with a CE

- CERT trailer and a PEMS installed on the tractor pulling the trailing. For each on

- road NTE event, a delta BSPM emissions value was computed as follows:

Delta BSPM = PEMS BSPM

– CE

- CERT BSPM.
These differences were compute d for the BSPM emissions for each PEMS unit tested and their corresponding calculation methods. The on

- road delta BSPM emissions were calculated for the The majority of CE

- CERT validation data were produced without any diesel particle filter (DPF) active regeneration “no regen”. Some data were produced during DPF active regeneration “regen” and those were used for information only, as the lab validation data did not include DPF active regeneration events. AVL methods 1, 2 and 3 including three individual units (#2, #3 and #4) and 271 NTE events. This data set was computed as “no regen”. In addition, AVL units #2, #3 and #4 collected 21 NTE event BSPM differences under the “regen” scenario. These “regen” data were not used in the validation decision but are presented for information purposes only. The second PEMS unit tested for validation was the Sensors. The on

- road delta BSPM emissions were calculated for Sensors methods 1 and 2 also using three individual units in the “no regen” scenario and resulte d in 217 NTE events.
The loess regression fits in Figure 172 through Figure 175 and the Sensors CE

- CERT BSPM differences collected on

- road were plotted in order to determine if the simulation method validated. If the number of CE

- CERT delta BSPM values does not exceed 10% of the total number of on

- road NTE events collected, the n the simulation method would be considered valid. Figure 172 represents the validation plot for the BSPM method 1 analysis for Sensors. Note that 7 of the 217 on

- road NTE events were either below or above the range of the ideal BSPM and were excluded from the validation percentage calculation. Therefore, 68 of the 210 CE

- CERT NTE events (32.38%) fell below the simulation model 5 th percentile based on the loess reg ression. Thus, the model was not considered valid for the BSPM Sensors Method 1.207 SwRI Project No. 14936.12 FIGURE 172 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON

THE SIMULATION MODEL FOR BSPM SENSORS MET HOD 1 WITH NO REGEN Figure 173 represents the validation plot for the BSPM method 2 analysis for Sensors. Note that again 7 of the 217 on

- road NTE events were either below or above the range of the ideal BSPM and were ex cluded from the validation percentage calculation. Therefore, 71 of the 210 CE

CERT NTE events (33.81%) fell above the simulation model 95 th percentile or below the simulation model 5 th percentile based on the loess regressions. Thus, the model was not

c onsidered valid for the BSPM Sensors Method 2.208 SwRI Project No. 14936.12 FIGURE 173 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON

THE SIMULATION MODEL FOR BSPM SENSORS MET HOD 2 WITH NO REGEN

Figure 174 represents the validation plot for the BSPM method 1 analysis for AVL with ‘no regen’. In this case 8 of the 271 on

- road NTE events were either below or above the range of the ideal BSPM and were excluded fr om the validation percentage calculation. Therefore, 49 of the 263 CE

- CERT NTE events (18.63%) fell above the simulation model 95 th percentile or below the simulation model 5 th percentile based on the loess regressions. Thus, the model was not considered valid for the BSPM AVL Method 1.209 SwRI Project No. 14936.12 FIGURE 174 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON THE SIMULATION MODEL FOR BSPM AVL METHOD 1 WITH NO REGEN

Figure 175 represents the validation plot for the BSPM method 2 analysis for AVL with ‘no regen’. Again in this case 8 of the 271 on

- road NTE events were either below or above the range of the ideal B SPM and were excluded from the validation percentage calculation. Therefore, 30 of the 263 CE

- CERT NTE events (11.41%) fell above the simulation model 95 th percentile or below the simulation model 5 th percentile based on the loess regressions. Thus, the model was not considered valid for the BSPM AVL Method 2.210 SwRI Project No. 14936.12 FIGURE 175 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON THE SIMULATION MODEL FOR BSPM AVL METHOD 2 WITH NO REGEN

Figure 176 represents the validation plot for the BSPM method 3 analysis for AVL with ‘no regen’. Again in this case 8 of the 271 on

- road NTE events were either below or above the range of the ideal BSPM and we re excluded from the validation percentage calculation. Therefore, 26 of the 263 CE

- CERT NTE events (9.89%) fell above the simulation model 95 th percentile or below the simulation model 5 th percentile based on the loess regressions. Thus, the model was c onsidered valid for the BSPM AVL Method 3 since the number of CE

- CERT NTE events outside the 5 th and 95 th percentile loes s regression was less than 10%.211 SwRI Project No. 14936.12 FIGURE 176 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON THE SIMULATION MODEL FOR BSPM AVL METHOD 3 WI TH NO REGEN

Figure 177 through Figure 179 illustrate the validation plots for the AVL methods 1, 2, and 3, respectively, based on the ‘regen’ computation. These were not included in the validation analysis but are provided for information purposes only.212 SwRI Project No. 14936.12 FIGURE 177 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON THE SIMULATION MODEL FOR BSPM AVL METHOD 1 WITH REGEN213 SwRI Project No. 14936.12 FIGURE 178 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON THE SIMULATION MODEL FOR BSPM AVL METHOD 2 WITH REG EN214 SwRI Project No. 14936.12 FIGURE 179 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON THE SIMULATION MODEL FOR BSPM AVL METHOD 3 WITH REGEN

TABLE 43 . SUMMARY OF BSPM M ODEL VALIDATION RESU LTS

PEMS Unit Method 1 Exhaust Flow Torque

Speed Method 2 Exhaust and Fuel Flow Torque

- Speed Method 3 Fuel Flow Torque

Speed
Sensors “no regen” No No No
AVL “no regen” No No Yes215 SwRI Project No. 14936.12 7 SUMMARY
A series of engine experiments and environmental tests were performed on three PM

- PEMS that included the Sensors PPMD, Horiba TRPM, and AVL MSS. The Sensors PPMD and Horiba TRPM were treated as official PEMS, and only those were used for official results and determination of next steps. The AVL MSS were used in conjunction with the S ensors PPMD based on an agreement reached between Sensors and AVL. Error surfaces were developed based on experimental work to determine PM

- PEMS bias and precision errors using Monte Carlo simulation model. The output of the model was to determine the erro r distribution at a set of reference NTE events, compared to their ideal value. The PM

- PEMS with the lowest 95 th percentile error that is greater than zero was selected for in

- use validation testing of the model.

The PM

- PEMS that was selected for in

- use va lidation was the Sensors PPMD. The PPMD produced a 95 th percentile measurement allowance error of 0.006 g/hp

- hr at a threshold NTE limit of 0.02 g/hp

- hr using Method 2, compared to the 0.01 g/hp

- hr that was produced by the Horiba TRPM. As for the AVL MSS, the instrument produced zero measurement allowance, but its measurement allowance value was not officially used because the MSS only measures the carbon fraction of PM, compared to the required total (solid plus volatile) PM measured by the other two PM

- P EMS. However, because the AVL MSS was used in conjunction with the PPMD, the SC agreed to include it during in

- use validation testing. Due to funding limitation, the Horiba TRPM was not included in in

- use validation testing.

Based on in

- use validation tes ting, the Sensors PPMD failed validation because 32 percent and 34 percent of the data produced in

- use were below the 5 th percentile of the validation window, using Method 1 and Method 2, respectively. The SC agreed during the development of the Test Plan that less than 10 percent of the data need to be outside the validation window to pass validation.
As for the AVL MSS, i t passed validation using Method 3 by having 10 percent of the data outside the validation window, with the majority of the data being higher the 95 th percentile. Method 2 failed by two percentage points and Method 1 failed by 8 percentage points.

Because the MSS Method 3 passed validation based on its own validation window, the SC concluded the measurement allowance program, and accepted the measurement allowance based on the Sensors PPMD as the final PM

- PEMS measurement allowance.

8 REFERENCES

1. “Determination of PEMS Measurement Allowance for Gaseous Emissions Regulated Under the Heavy

- Duty In

- Use Testing Program: Revised Final Report,” EPA420

- R

- 08

- 005, February 2008.216 SwRI Project No. 14936.12 2. “Electronic Code of Federal Regulations, Title 40 Protection of the Environmen t, Vol. 15 Part 1065,” http://www.ecfr.gpoaccess.gov , 2009.

3. Conover, W.J., Practical Nonparametric Statistics , John Wiley & Sons, 1971

4. Cleveland, W.S. , “Robust Locally

- Weighted Regression and Smoothing Scatterplots.” Journal of the American Statistical Association, 1979, 74, pp. 829

- 836.

5. Hurviuch, C.M., Simonoff, J.S., and Tsai, C.L., “Smoothing Parameters Selection in Nonparametric Regression Using and Improved Akaike Information Criterion,” Journal of the Ro yal Statistical Society B, 1998, 60, pp.271

- 293.

6. Stratmann, F., Otto, E., and Fissan, H., “Thermophoretical and Diffusional Particle Transport in a Cooled Laminar Tube Flow.” Journal of Aerosol Science, March 28th, 1994, vol. 25; pp. 1359

- 1365.217 SwRI Project No. 14936.12 APPENDIX A : PEMS OPERATION LOG218 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

2/25/2008 Horiba Lost communication with TRPM laptop on a regular basis Standard ethernet cable does not fit properly into this laptop A industrial grade ethernet cable was provided by Horiba which solved this problem

2/28/2008 Horiba 1 Could not control dilution ratio if external flow meter is not connected Unknown Horiba fixed the problem

2/28/2008 Horiba 2 Java software does not display the measured values that are in the Labview software Unknown Software update fixed the problem

3/3/2008 Horiba 2 Dilution air flow not stable Dilution air pressure too high Performed a dilution air flow adjustment per Horiba

3/4/2008 Sensors 1 Lookup table for MPS2 repeatedly failed Unknown Sensors said the criteria is too stringent and it is fine as long as it visually looks good

3/4/2008 Horiba 1 Dilution Ratio control is still somewhat erratic Bad PID
constants Adjusted PID constants to new values suggested by Horiba

3/14/2008 Horiba 1 Pressure transducer Pt1 would not respond to calibration Pressure transducer was broken Replaced with new part from Horiba

3/21/2008 Horiba 1,2,&3 Unable to pass Part 1065 sample flow linearity verification Sample flow is an inferred not measured value Linearity verification performed on dilution and total flow at MASC request

3/24/2008 Horiba 1,2,&3 Sample flow failed Horiba check with provided external flow
meter Flow calibrations needed updating Flow coefficients updated for all three units

3/26/2008 Horiba 1,2,&3 Unable to pass Part 1065 dilution flow linearity verification Dilution flow coefficients needed updating Performed "Dilution Ratio Accuracy Adjustment" as instructed by Horiba219 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

4/14/2008 Horiba 3 Java software freezes when loading Wrong parameters set in a config file AVL programmed an offset into their software to account for this, laterit appeared to be a grounding problem in the Semtech DS

4/22/2008 Horiba 3 Java softare is not communicating with the Labview software Improper software configuation Parameters adjusted to fix the problem

6/13/2008 Horiba 2200 Horiba system is unable to log the ISO-1576 ECM broadcast from the International Engine Software did not have this capability International engine wasreplaced with a heavyduty Volvo engine that uses J1939 broadcast

6/19/2008 Horiba 1 TRPM software not reading the same exhaust flow as OBS-2200 software Calibration coefficient is wrong Manually adjust the calibration coefficient to get the readings to match

6/28/2008 Sensors 1 Sample valve for crystal remains in "transient" state every time it attempts to sample Stepper motor attempting to turn too quickly (not enough torque) Sensors readjusted the stepper motor speed

7/31/2008 Horiba 2 The OBS-2200 software would not trigger the OBS-TRPM to start sampling during an NTE Connector wired incorrectly Reduced the speed of the stepper motor

8/8/2008 Horiba 2 Dilution flow is too low Replaced with connector from second unit

8/12/2008 Horiba 2200 OBS-2200 software unable to read the reference torque value from J1939 broadcast Unable to enter the proper data bit location Perform dilution flow adjustment using internal pressure regulator

8/14/2008 Sensors 2 Could not get any of the crystals to os cillate Power supply in CQCM head was likely burnt out Enter the reference torque value manually

8/15/2008 Horiba 2200 OBS-2200 laptop would not boot up Unknown Hard drive was placed in another identical laptop, broken laptop was shipped back to Horiba220 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

8/18/2008 Sensors 1 Bypass flow was not updating

8/21/2008 Sensors 1 Crystal frequencies, corona currents, and voltages were dropping
out High temperature causing communication problems Attempted to reduce test cell temperature, but problem persisted

8/26/2008 AVL 1 Unit switched into Zero Check in the middle of the test Uknown, possibly a pressure out of limit Unit switched back into sample after approx. 30 seconds

8/27/2008 Horiba 1 OBS-2200 laptop failed, would no longer boot Laptop broken New laptop provided by Horiba

8/27/2008 Horiba 1 OBS-2200 laptop would not read J1939 Unknown Sent back to Horiba for repair

8/27/2008 Horiba 1 TPA would not zero Zero function tied to gas analyzers, looking for gas flow Use the CAL function instead of the ZERO to just zero TPA

8/27/2008 Horiba 1 TRPM Java software unable to log exhaust flow (Labview works) Error in software Installed new version of Java software

8/27/2008 Horiba 1 EAD check will fail repeatedly Tolerances are too tight for test cell operation, tolerances relaxed. Ignored tolerances set in software for check

8/27/2008 AVL 1 Analog output signal to MSS would clip at 2 mg/m3 range Soot concentration too high Output switched to 0-10 mg/m3, DR setpoint increased from 3 to 6

8/27/2008 AVL 1 DR would oc casionally stop controlling Too much moisture in the system Unit purged for moisture overnight, and firmware upgraded

8/27/2008 Sensors 1 PPMD internal temperature was out of limit Test cell temperature too hot A new back panel was installed with two fans to promote cooling

8/27/2008 Sensors 1 Bypass flow from TSI flowmeter was not reading in software Faulty com cable Replaced TSI cable221 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

8/27/2008 Sensors 1 CQCM communication dropped out during testing CQCM power supply failed Power supply replaced

8/27/2008 Sensors 1 CQCM communication dropped out during testing Unknown CQCM comm chip replaced with a newer model

8/27/2008 Sensors 1 Corona needle high voltage erratic Unknown Fixing the comm issue resolved this problem

8/27/2008 Sensors 1 PPMD communication was dropping out High internal temperature Directed cooling air at PPMD

9/10/2008 Horiba 1 Compressor tripped breaker, turned off AVL unit Combining AVL and compressor on same 15A circuit was too much current Horiba compressor moved to dedicated 15A circuit

9/10/2008 Horiba 1 Make up air flow was 0.7 lpm instead of 2.3 lpm PID constants incorrect Modified PID constants

9/12/2008 AVL 1 Analog output signal to MSS would still briefly clip at 10 mg/m3 range Soot concentration too high Logarithmic analog output added as a firmware update

9/12/2008 Sensors 1 Negative emissions reported even at high emission levels on NTE High dilution, crystal saturation (no grease), crystal stabilization Do not use MPS2, grease crystals, wait longer for PPMD to warm up

9/18/2008 Sensors 1 The bypass flow would increase when crystal 1 on PPMD1 would sample Crystal may be installed backwards so that it is always sampling Set crystal 1 to be the reference crystal

9/18/2008 Sensors 1 Sample flow drifting during steady state engine operation Temperature estimate is based on mixing of two flows, if the estimate is off temperature will change the flow Adjusted the parameters in the temperature estimate222 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

9/18/2008 Sensors 1 No corona current was measured for two crystals Crystals were shorted to ground Crystals were replaced

9/18/2008 Sensors 1 High voltage reading was low on two crystals Corona needles were too close to the crystals Needles repositioned

9/20/2008 Sensors 1 Sample flow
measurement was too low Sample flow TC was in excess flow return, which had low flow due to connection to CVS TC moved into the main exhaust

9/26/2008 AVL 1 Error: MFD Temperature out of Spec Test cell temperature too hot Repos itioned chiller air to blow on the AVL unit

9/30/2008 Sensors 1 Lower than expected emissions for the PPMD Inaccurate sample flow temp causing low sample flow
meas, corona needles not positioned properly Repos ition the sample flow thermocouple, adjust corona needle

10/9/2008 Sensors 1 PPMD sampling delayed several seconds after trigger Delay in communications Increased residcence time inside PPMD up to 3 seconds

10/9/2008 Sensors 1 Inaccuracies in flow
measurement System uses an assumed inlet pressure for MPS2 Added pressure measurement downstream of MPS1 to account for changes in pressure due to MPS2

10/9/2008 Sensors 1 Crystal sampling as soon as it becomes available, including in an NTE event Software logic Software modified so that an available crystal waits for the start of the next NTE event

10/9/2008 Horiba 1 Compressor supplying dilution air stops working in middle of test (2 units) Unit shutting off due to overheat protection Cool air provided to the compressor (temporary solution)223 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

10/9/2008 AVL 1 MSS concentration reporting too high in Sensors software Sensors not correcting concentration from 0°C to 20°C (~8%) Semtech DS software updated to include the log range and volume correction

10/10/2008 Sensors 1 Bypass flow too low Flow leaking through the carbon filter connection Tightened the carbon filter

10/15/2008 Sensors 1 PPMD emission results were inconsistent Booker had decreased the crystal flow from 0.4 to 0.2 slpm to increase loading time Final crystal sample flow set at 0.5 slpm

10/15/2008 Sensors 1 PPMD emissions were lower than expected Uknown Crystal sensitivity adjusted from 125 hz/μg to 100 h z/μg (increasing sensitivity by 25%)

10/15/2008 Sensors 1 PPMD wouldn't sample when external trigger activates on Semtech DS Unknown Powered off hardware and laptop and restarted, problem was resolved

10/16/2008 Sensors 1 PPMD unable to communicate with DS Unknown Triggered the PPMD manually through Host software (unofficial testing) 10/28/2008 Horiba 1 Dilution flow inaccurate Unknown Recalibrated VFM

11/3/2008 Sensors 1 PPMD block pressure low Uknown, block pressures were activated in the software Aborted cycle, powered down the PPMD and restarted, problem was fixed

11/4/2008 Sensors 1 Multiple crystals stopped oscillating, when one was enabled another would disable Grease loading slightly too high? Re cleaned and greased crystals

11/4/2008 Horiba 1 Unable to maintain setpoint for total flow near end of the cycle High filter loading Adjusted the valve on the total flow pump to allow more flow224 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

11/5/2008 Horiba 1 Sample flow accuracy is out of spec Caused by inaccurate dilution flow measurement (cause of this uknown) Recalibrated VFM

11/6/2008 Horiba 1 Dilution flow is still inaccurate Unknown Recalibrated VFM

11/7/2008 Sensors 1 MPS Dilution Flow Major audit could not pass Overheated None, cycle voided

11/7/2008 Horiba 1 External compressor stopped working Overheated None, cycle voided

11/7/2008 Horiba 1 External compressor stopped working Overheated None, cycle voided, installed 2nd compressor in parallel

11/12/2008 Sensors 1 PPMD results not included in the DS results file Descrepancies in the time stamps Never resolved

11/12/2008 Horiba 1 Dilution compressor still stoppi ng sometimes Overheating Connected two compressors in parallel, so the 2nd will run if the first stops

11/12/2008 Sensors 1 Slope on flow for daily audits sometimes fails to 0.97 or 1.03 Monthly tolerances too tight for daily checks Tolerances relaxed slightly for daily checks to save time

11/12/2008 Horiba 1 Sample flow check is sometimes not within tolerance Monthly tolerances too tight for daily checks Tolerances relaxed slightly for daily checks to save time

11/24/2008 Horiba 1 Sample flow accuracy is out of spec Inaccurate dilution flow
measurement Recalibrated VFM

11/25/2008 AVL 1 MSS failed the external DR audit repeatedly Unknown - all internal checks passed Requirement of performing external audit on daily basis was removed225 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

12/1/2008 Sensors 1 EFM reading inaccurate at the end of each test Solenoid not switching properly during the 1 hour autozero, EFM zeroing while open to flow Solenoid fixed, auto zero disabled?

12/1/2008 Horiba 1 Newest compressor that was shipped still stopping sometimes Overheating Continued to use two compressors in parallel, a new compressor was shipped from Horiba

12/1/2008 Horiba 1 Post processor file size too big for Excel (>200 MB) Steady state data processed at 1Hz instead of 10Hz Continued to use two compressors in parallel

12/11/2008 Horiba 1 DCS signal flat lined during testing Uknown Problem did not occur again when system was restarted

12/15/2008 Horiba 1 TRPM data filed was not saved by Java software Unknown None, cycle was void

12/16/2008 Horiba 1 External compressor stopped working Overheated None, cycle voided

12/18/2008 Horiba 1 External compressor stopped working Overheated None, cycle voided

1/14/2009 Horiba 1 Sample flow accuracy is out of spec Caused by inaccurate dilution flow measurement (cause of this uknown) Recalibrated VFM

1/14/2009 Horiba 1 Dilution flow inaccurate Unknown Recalibrated VFM

1/26/2009 Sensors 2 Semtech DS lost communication with laptop Unknown Couldn't monitor data, but data was still recorded on compact flash card

1/28/2009 Horiba 2 TPA switched from measure to standby at the start of the test Software glitch? Did not occur again226 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

1/28/2009 Horiba 2 TRPM would switch out of filter sample mode once 30 seconds had elapsed Faulty software logic Software modified so that filter will never switch out of sample mode while in an NTE event

1/28/2009 Horiba 2 Filter sampling was beginning 10 sec after start of NTE instead of 5 OBS-2200 is delayed 5 seconds in its response Horiba chose not to make any changes

1/30/2009 AVL 2 Error: No dilution air available On-board pump was leaking Pump was replaced with new part shipped from AVL

2/2/2009 Sensors 2 Semtech DS lost communication with laptop, could not
reconnect Unknown Powered down unit, recovered data the next day

2/5/2009 Sensors 2 Semtech DS was losing communication with laptop Unknown Couldn't monitor data, but data was still recorded on compact flash card

2/18/2009 Horiba 2 External compressor stopped working Overheated None, cycle voided

3/2/2009 Horiba 3 Software zero pressure transducer function would not work after repeated attempts Unknown Shut down equipment attempted it again several hours later and it worked

3/3/2009 Horiba 3 External compressor stopped working Overheated None, cycle voided

3/5/2009 Sensors 3 Semtech DS lost communication with laptop Unknown Couldn't monitor data, but data was still recorded on compact flash card

3/6/2009 Horiba 3 External compressor stopped working Overheated None, cycle voided

3/23/2009 Horiba 2200 OBS-2200 software locked up during test Unknown Software worked when rebooted, but data for the cycle was lost

3/24/2009 Sensors DS Couldn't connect to the Semtech DS repeatedly LAN circuit board was likely damaged Unit sent back to Sensors for repair227 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

3/25/2009 Sensors 3 PPMD wouldn't sample when NTE trigger activates on the Semtech DS Unknown Turning the unit off and back on fixed the problem

4/3/2009 AVL 1 There is a voltage offset between what the AVL unit outputs and the Semtech DS reads Unknown Switched constants

4/13/2009 AVL 2 Power inverter shut down during radiated immunity
test EMI/RFI Test Replaced bypass pump

4/15/2009 Sensors 2 Communication with the laptop dropped out repeatedly during bulk current injection EMI/RFI Test Head sent back for repair

4/20/2009 Sensors 2 Communication with the laptop dropped out during radiated immunity EMI/RFI Test Replaced with bracket from other unit

4/21/2009 AVL 2 Power inverter shut down during conducted transient test EMI/RFI Test No corrective action taken

4/21/2009 AVL 2 Power inverter shut down, then start smoking during conducted transient EMI/RFI Test Unit was shipped back to AVL, replaced with other unit, problem was not observed again

4/21/2009 Sensors 2 PPMD shut down completely several times during conducted transient tests EMI/RFI Test No corrective action taken

4/24/2009 Horiba 3 Flow was erratic during bulk current injection test EMI/RFI Test No corrective action taken

4/27/2009 Horiba 3 Exhaust flow was reading very high during bulk current injection test EMI/RFI Test No corrective action taken

4/28/2009 Horiba 3 Lost communication with laptop during radiated immunity test EMI/RFI Test Inverter replaced with backup, test was not repeated228 SwRI Project No. 14936.12 Date PEMS Description Reason Solution

4/28/2009 Horiba 3 Exhaust flow was reading very high during radiated immunity test EMI/RFI Test No corrective action taken

5/15/2009 AVL 3 AVL PEMS blew a fuse when trying to pow er up after 1st pressure test Uknown Replaced with AVL 2

7/15/2009 Sensors 3 PPMD reported a barometric pressure increase when altitude chamber was at vacuum Constants in the software were backwards for the barometric pressure No corrective action taken

7/15/2009 Sensors 3 PPMD could not maintain a bypass flow of 4 slpm Bypass pump was dying No corrective action taken

7/21/2009 AVL 2 Soot concentration reading erratic during vibration (all orientations) Vibration Rebooted hardware to
reconnect

7/22/2009 Horiba 3 L-bracket holding filter in ME box broke during vibration testing Vibration Did not continue with
more extreme tests to prevent damage

7/22/2009 Horiba 3 Total Pi pressure transducer would not read correctly after problem with L-bracket occurred Vibration Test Pressure transducer was replaced

7/24/2009 Sensors 2 PPMD could not maintain total flow due to moisture traps opening Vibration Test No corrective action taken229 SwRI Project No. 14936.12 APPENDIX B : CRYSTAL BALL OUTPUT FILE DESCRIPTIONS230 SwRI Project No. 14936.12 EXTRACT DATA FILES

1.0 Simulation Variables

The simulation variables listed in Table 44 were extracted at the completion of the Monte Carlo simulation run for each reference NTE event. Crystal Ball classifies v ariables into two categories: assumptions and forecasts. Assumptions are the estimated inputs into the simulation model such as the variability indices used to sample each error surface. Assumption variables in this study are identified by an “i c ” at th e beginning of the variable name, or by “Delta” at the beginning of the variable name. The “i c

TABLE 44 . SIMULATION VARIAB LES ” variables are the simulation error model inputs such as

“ 01_ic_SS_PM ”. The “Delta” variables serve as switches that turn a given error surface on or off in the model, e.g. “ Delta PM SS ”. The Delta switch variables when turned on and off during a simulation are applied in post

- simulation analysis to determine sensitivity of results to the particular error surfaces. Forecasts are values calculated by a foreca st formula in the spreadsheet cells. Examples of forecast variables used in this study are “ 001AVL_ D ePM (g/hp

- hr), Method 1 ” and “ 005AVL_Valid D ePM (g/hp

- hr), Method 2 ”.

Variable Name Description

001AVL_ D ePM (g/hp

- hr), Method 1 MC Delta PM Method 1 for AVL PEMS

001Horiba_ D ePM (g/hp

- hr), Method 1 MC Delta PM Method 1 for Horiba PEMS

001Sensors_ D ePM (g/hp

- hr), Method 1 MC Delta PM Method 1 for Sensors PEMS

002AVL_ D ePM (g/hp

- hr), Method 2 MC Delta PM Method 2 for AVL PEMS

002Horiba_ D ePM (g/hp

- hr), Method 2 MC Delta PM Method 2 for Horiba PEMS

002Sensors_ D ePM (g/hp

- hr), Method 2 MC Delta PM Method 2 for Sensors PEMS231 SwRI Project No. 14936.12 003AVL_ D ePM (g/hp

- hr), Method 3 MC Delta PM Method 3 for AVL PEMS

004AVL_Valid D ePM (g/hp

- hr), Method 1 Validation MC Delta PM Method 1 for AVL PEMS

004Horiba_Valid D ePM (g/hp

- hr), Method 1 Validation MC Delta PM Method 1 for Horiba PEMS

004Sensors_Valid D ePM (g/hp

- hr), Method 1 Validation MC Delta PM Method 1 for Sensors PEMS

005A VL_Valid D ePM (g/hp

- hr), Method 2 Validation MC Delta PM Method 2 for AVL PEMS

005Horiba_118_Valid D ePM (g/hp

- hr), Method 2 Validation MC Delta PM Method 2 for Horiba PEMS

005Sensors_Valid D ePM (g/hp

- hr), Method 2 Validation MC Delta PM Method 2 for Sensors PEMS

006AVL_Valid D ePM (g/hp

- hr), Method 3 Validation MC Delta PM Method 3 for AVL PEMS

01_ic_SS_PM Random Sampling Variability Index for SS PM Error Surface, applied to AVL, Horiba and Sensors PEMS

02_ic_TR_PM Random Sampling Variability Index for Transient PM Error Surface, applied to AVL, Horiba and Sensors PEMS

04_ic_Atm.Pres_PM_AVL Random Sampling Variability Index for PM Atmospheric Pressure Error Surface, applied to AVL PEMS

05_ic_Amb.Temp_PM_AVL Random Sampling Variability Index for PM Temperature Error Surface, applied to AVL PEMS
07_ic_SS_CO Random Sampling Variability232 SwRI Project No. 14936.12 Index for SS CO

10_ic_Pressure_CO Random Sampling Variability Index for CO Pressure

11_ic_Temperature_CO Random Sampling Variability Index for CO Temperature

13_ic_SS_NMHC Random Sampling Variability Index for SS NMHC

14_ic_TR_NMHC Random Sampling Variability Index for Transient NMHC

16_ic_Pressure_NMHC Random Sampling Variability Index for NMHC Pressure

17_ic_Temperature_NMHC Random Sampling Variability Index for NMHC Temperature

19_ic_NMHC_Ambient Random Sampling Variability Index for Ambient NMHC

20_ic_SS_flow Random Sampling Variability Index for SS Exhaust Flow

21_ic_TR_Flowrate Random Sampling Variability Index for Transient Exhaust Flow

22_ic_Pu lsation_flow Random Sampling Variability Index for Exhaust Flow Pulsation

23_ic_Swirl_flow Random Sampling Variability Index for Exhaust Flow Swirl

25_ic_Radiation_Exhaust Flow Random Sampling Variability Index for Exhaust Flow EMI/RFI Radiation

27_ic_Temperature_Exhaust Flow Random Sampling Variability Index for Exhaust Flow Temperature
28_ic_Pressure_Exhaust Flow Random Sampling Variability233 SwRI Project No. 14936.12 Index for Exhaust Flow Pressure

29_ic_TR_Torque Random Sampling Variability Index for Dynamic Torque

30 _ic_Torque_DOE Random Sampling Variability Index for Torque Design of Experiments Testing

31_ic_Torque_Warm Random Sampling Variability Index for Torque Warm

- up

32_ic_Torque_IP Random Sampling Variability Index for Torque Independent Parameters Humidity and Fuel

34_ic_Torque_Interpolation Random Sampling Variability Index for Torque Interpolation

35_ic_Torque_Engine Manufacturers Random Sampling Variability Index for Torque Engine Manufacturers

42_ic_Fuel _Engine Manufacturers Random Sampling Variability Index for Fuel Engine Manufacturers

43_ic_TR_Speed Random Sampling Variability Index for Dynamic Speed

44_ic_TR_Fuel Rate Random Sampling Variability Index for Dynamic Fuel Rate

45_ic_SS_CO2 Random Sampling Variability Index for SS CO2

46_i c_TR_CO2 Random Sampling Variability Index for Transient CO2

49_ic_Temperature_CO2 Random Sampling Variability Index for CO2 Temperature

Delta PM SS Model switch controlling PM SS error surface application234 SwRI Project No. 14936.12 Delta PM Transient Model switch controlling PM Transient error surface application

Delta PM Atmospheric Pressure Model switch controlling PM Atmospheric Pressure error surface application

Delta PM Ambient Temperature Model switch controlling PM Ambient Temperature error surface application

Delta CO SS Model switch controlling CO SS error surface application

Delta CO Atmospheric Pressure Model switch controlling CO Atmospheric Pressure error surface application

Delta CO Ambient Temperature Model switch controlling CO Ambient Temperature error surface application

Delta NMHC SS Model switch controlling NMHC SS error surface application

Delta NMHC Transient Model switch controlling NMHC Transient error surface application

Delta NMHC Atmospheric Pressure Model switch controlling NMHC Atmo spheric Pressure error surface application

Delta NMHC Ambient Temperature Model switch controlling NMHC Ambient Temperature error surface application

Delta Ambient NMHC Model switch controlling Ambient NMHC error surface application

Delta Exhaust Flow SS Model switch controlling Exhaust Flow SS error surface application

Delta Exhaust Flow Transient Model switch controlling Exhaust Flow Transient error surface235 SwRI Project No. 14936.12 application

Delta Exhaust Flow Pulsation Model switch controlling Exhaust Flow Pulsation erro r surface application

Delta Exhaust Flow Swirl Model switch controlling Exhaust Flow Swirl error surface application

Delta Exhaust EMI/RFI Model switch controlling Exhaust EMI/RFI error surface application

Delta Exhaust Temperature Model switch controlling Exhaust Temperature error surface application

Delta Exhaust Pressure Model switch controlling Exhaust Pressure error surface application

Delta Dynamic Torque Model switch controlling Dynamic Torque error surface application

Del ta Torque DOE Testing Model switch controlling Torque DOE Testing error surface application

Delta Torque Warm

- up Model switch controlling Torque Warm

- up error surface application

Delta Torque Humidity Model switch controlling Torque Humidity error surface application

Delta Torque Interpolation Model switch controlling Torque Interpolation error surface application

Delta Torque Engine Manuf Model switch controlling Torque ( Engine Manuf acturer) error surface application

Delta Fuel Engine Man uf Model switch controlling Fuel ( Engine Manuf acturer) error surface application236 SwRI Project No. 14936.12 Delta Dynamic Speed Model switch controlling Dynamic Speed error surface application

Delta Dynamic Fuel Rate Model switch controlling Dynamic Fuel Rate error surface applica tion

Delta CO2 SS Model switch controlling CO2 SS error surface application

Delta CO2 Transient Model switch controlling CO2 Transient error surface application

Delta CO2 Ambient Temperature Model switch controlling CO2 Ambient Temperature error surface application

2.0 Statistics

Descriptive statistics summarizing the values obtained during a single reference NTE event simulation are provided in Table 45

.

TABLE 45 . DESCRIPTIVE STATI STICS FOR SIMULATION VARIABLES
Statistic Definition

Trials Number of times the simulation was repeated

Mean Arithmetic average

Median The value midway between the smallest value and the largest value

Mode Value that occurs most often

Standard Deviation Measurement of variability of a distribution. The square root of the variance237 SwRI Project No. 14936.12 Variance The average of the squares of the deviations of a number of values from their

mean

Skewness A measure of the degree of deviation of a distribution from the norm of a symmetric distribution

Kurtosis A measure of the degree of peakedness of a distribution
Coefficient of Variability Standard deviation/Mean

Minimum Smallest value

Maximum Largest va lue
Range Width Largest value

– smallest value

Mean Standard Error Standard deviation of the distribution of possible sample means

3.0 Percentiles

Percentiles are the probability of achieving values below a particular percentage in the following increments: 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, and 100%. Percentiles are computed for each of the simulation var iables described in Table 44

.

4.0 Sensitivity Data
Sensitivity data are provided by computing the rank correlation coefficient for all error surfaces and all simulat ion variables. The EXTRACT data file contains the absolute value of the rank correlation. In post

- simulation processing, values of control variable Delta PM SS in the simulation results were applied to dichotomize the data.

5.0 Trial Values

The value for all simulation variables is provided at each trial of the simulation.238 SwRI Project No. 14936.12 REPORT FILES

1.0 Report Summary

This section includes the simulation start date and time, stop date and time, number of trials run, sampling type (Monte Carlo), random seed used, and run statistics.

2.0 Forecasts

Descriptive statistics, percentiles, and a frequency histogram are provided for forecast variables 001AVL_ D ePM (g/hp

- hr), Method 1 through 006AVL_Valid DePM (g/hp

- hr), Method 3 (see Table 44 ).

3.0 Assumptions

Descriptive statistics, percentiles, distribution parameters, and a distribution chart are provided for assumption variables 01_ic_SS_PM through 49_ic_Temperature_CO2 (see Table 44 ).

4.0 Sensitivity Charts

Sensitivity charts are provided for forecast variables 001AVL_ D ePM (g/hp

- hr), Method 1 through 006AVL_Valid D ePM (g/hp

- hr), Method 3 (see Table 44 ). Crystal Ball calculates sensitivity by computing rank correlation coefficients between every assumption (error surface) and forecast (delta BSPM emi ssions) while the simulation is running. Positive rank correlations indicate that an increase in the assumption is associated with an increase in the forecast. The larger the absolute value of the rank correlation the stronger the relationship.239 SwRI Project No. 14936.12 The sen sitivity charts developed during the MC simulation are displayed as ‘Contribution to Variance” charts which are calculated by squaring the rank correlation coefficients for all assumptions used in a particular forecast and then normalizing them to 100%. Figure 180 displays a sensitivity chart for the AVL delta PM Method #1. The assumptions with the highest contribution to variance (in absolute value) are plotted at th e top of the chart. This example shows a typical dominant effect of the PM SS error surface represented by the 79.5% negative effect of Delta PM SS. As seen in the example in Figure 180 , as you increase the SS PM there is an increase in the delta PM Method #1 values represented by the 7.6% positive effect of ic_SS_PM, and as you increase the torque warm

- up there is a decrease in the delta PM Method #1 values represent ed by the 2.8% negative effect of ic_Torque_Warm. Only the top eight assumptions are plotted in this sensitivity chart.240 SwRI Project No. 14936.12 FIGURE 180 . SENSITIVITY CHART FOR AVL DELTA PM MET HOD 1241 SwRI Project No. 14936.12 APPENDIX C : MONTE CARLO SPREADSH EET COMPUTATIONS242 SwRI Project 14936.12 1.0 DESCRIPTION OF ASSUM PTIONS
The following assumptions were made in running the Monte Carlo model:

**•** Only one reference NTE event can be run at a time through the Monte Carlo simulation workbook . However, NTE event cases can be stacked and run in a batch mode.

**•** Uniform (1 second in duration) time steps are used in the reference NTE events.

**•** Standard format and engineering units for reference NTE data established for the proje ct are observed, and applied to the reference NTE event before the NTE event is entered in the Error Model workbook for Monte Carlo simulation.

**•** Any wet

– dry matter conversions, if not negligible, have been performed on the appropriate reference NTE event values before the reference NTE event was entered in the Error Model workbook for Monte Carlo simulation. No wet

– dry conversions are performed in the workbook.

**•** Any reference NTE event normalizations to produce similar emission s brake

- specific results from the three emission s calculation methods have been appropriately performed before the reference NTE event was entered in the Error Model workbook for Monte Carlo simulation. No normalizations among the three methods are performed in the workbook.

**•** PM e mission s models fo r three calculation methods are computed for the AVL PEMS. Only calculation methods 1 and 2 are computed for the Horiba and Sensors PEMS.

**•** Error surface models and supporting data were approved by the Steering Committee

.

**•** The error model spreadsheet has been correctly implemented, and its interaction with Monte Carlo tools like Crystal Ball is correctly understood.

**•** Random number generation by a Monte Carlo tool like Crystal Ball is correct

.243 SwRI Project 14936.12 **•** Convergence of the completed MC simulation was processed and checked outside of the simulation workbook. Benchmark checks on the convergence calculations were made using a SAS ****

2.0 WORKSHEET DESCRIPTIO NS

2.1 Macro Description
The Macro can be viewed in the Excel sp readsheet ‘Batch Control’ with the menu selections Tools>Macros>Macro1>Edit. The purpose of Macro1 is to control NTE event batch processing of stacked cases. For each NTE event case processed, the macro expedites clear ing extra cells below the reference NTE e vent in the spreadsheet ‘Error Model’ Methods worksheet and delete s extra rows in the Delta error worksheet s. The macro also performs Mode 0 calculation s and stores resultant 'ideal emissions' values for application in subsequent Monte Carlo simulati on

.

The user must begin with the starter version of the ‘Error Model’ Excel file which has 300 rows of equations in columns X

- CF and in rows 52

- 351 in the Methods worksheet

. The starter spreadsheet a lso has 300 rows of equations below charts in columns B

– F , or B

– L, in applicable Delta worksheets. The user (when not under automatic batch control) c op ies the reference NTE e vent into columns A

– V , row 52 and down
, in the Methods worksheet

. It is can then be c onfirm ed that cell J45 in the Meth ods worksheet displays the correct number of rows of the reference NTE e vent.

Macro execution can be accomplished through the menu selections Tools>Macros>Macro1>Run. N ote that t his macro clears cells without deleting rows in the Methods worksheet , and deletes rows in the Delta worksheet s. T his macro will not work if the reference NTE e vent has only one row. For a reference NTE e vent with exactly two rows, this macro will corrupt the second "check" values in columns B

- F type Delta worksheets. C heck

va lues are not used in the simulation, but are provided as a diagnostic aid

. Apply th e macro for reference NTE events with no more than 300 rows. computer program.

The reader can follow the description of execution that follows by viewing the macro and observing the commen t rows provided throughout the macro text. In execution, the macro first reads the contents of J45 in the Methods worksheet. It uses the number of rows in the reference244 SwRI Project 14936.12 NTE event defined by J45 to determine how many rows to clear and delete in the spread sheet. It checks that the number of rows is between 2 and 299, inclusive. It will also execute correctly for 300 rows.

Next, the macro clears cell contents in columns X

– CF below the reference NTE event in the Methods worksheet. Note the macro, as w ritten, will not execute properly if the starter spreadsheet has been revised with row insertion or deletion in certain areas of the spreadsheet. As written, the macro initiates in cell X52, counts down through the NTE Event rows, and clears contents in t he range from there in column X through cell CF351.

Next, the macro deletes extra rows below the reference NTE event, where applicable, for example in Delta worksheet 07. For Delta worksheet 07 it initiates in cell B79 and counts down through the rows of the reference NTE event to the first row to be deleted. It selects the range of rows from there down through row 378, deletes the rows, copies some equations and a value to the last row in the range the charts use, and returns the cursor to cell F68 l eaving the display more or less centered on the charts in the worksheet.

Subsequently, the macro performs similar operations in other Delta worksheets; however, the initiating cell and final row differ among the worksheets. The Delta worksheets processe d in this way are 7, 10, 11, 16, 17, 20, 21, 22, 23, 29, 30, 43, 44, 45, 46 and 49.

Following the row deletion operations in the Delta worksheets, or directly when the reference NTE event has 300 rows, the macro prepares for the Mode 0 (ideal emissions) calculation. First, in the Methods worksheet it copies the equations in row 52, columns X through CF, to the last row in the reference NTE event. This clears any errors introduced in the last row; however, it assumes that row 52 is correct. The last cel l in column AC ( **∆** t) is cleared for aesthetics, since the **∆** t values are not applied in the model calculations.

The Mode 0 calculation is performed by the macro by changing the value in cell A6 of the Summary worksheet to 0. Then in the Methods worksheet, the values from cells CU22 through CU30 are pasted (values only) to cells O22 through O30 where they are referenced by formulas during Monte Carlo simulation. The macro changes the value of A6 in the Summary worksheet to 2 in preparation for the Monte Ca rlo simulation, and moves the cursor to cell CT18 of the Methods worksheet.245 SwRI Project 14936.12 Additional comments regardi ng the macro operation are presen ted in the following section descriptions of the model spreadsheet.

2.2 Worksheet 1: ErrorControl

The ErrorControl worksheet of the Er ror Model workbook implements 31 logic switch functions. The user enters a numerical “1” in column A D in each row corresponding to error surfaces to be included in the calculation. A numerical “0” is applied to error surfaces to be

ex cluded in the calculation. Corresponding random variables for error

- surface on

- off switch random effects sensitivity modeling are implemented in column W under Crystal Ball control.
Error surfaces are numbered 1 through 49 . The numbered error surfaces ar e defined in columns A

– C, and information pertinent to their usage is presented in columns E

– V of the worksheet. Column E displays warning messages when an unusual value is monitored in column D.

The control switch elements in the worksheet are delib erately placed on rows in the worksheet corresponding to the error surfaces to expedite equation checking in the Methods worksheet where the control switch variables are applied in conjunction with error surfaces from the correspondingly numbered “Delta” w orksheets
The numbered error surfaces and time alignment controls that have been implemented are defined in the following Table 46

.246 SwRI Project 14936.12 TABLE 46 . ERROR SURFACES USE D IN SIMULATION

Component No. Error Surface

Delta PM 1 Delta PM SS

2 Delta PM Transient

4 Delta PM Atmospheric Pressure

5 Delta PM Ambient Temperature

Delta CO 7 Delta CO SS

10 Delta CO Atmospheric Pressure

11 Delta CO Ambient Temperature

Delta NMHC

NMHC = 0.98*THC 13 Delta NMHC SS

14 Delta NMHC Transient

16 Delta NMHC Atmospheric Pressure

17 Delta NMHC Ambient Temperature

19 Delta Ambient NMHC

Delta Exhaust Flow 20 Delta Exhaust Flow SS

21 Delta Exhaust Flow Transient

22 Delta Exhaust Flow Pulsation

23 Delta Exhaust Flow Swirl

25 Delta Exhaust EMI/RFI

27 Delta Exhaust Temperature

28 Delta Exhaust Pressure
Delta Torque 29 Delta Dynamic Torque

30 Delta Torque DOE Testing (Interacting Parameters Test)

31 Delta Torque Warm

- up247 SwRI Project 14936.12 (Interacting Parameters Test)

32 Delta Torque Humidity / Fuel (Independent Parameters Test)

34 Delta Torque Interpolation

35 Delta Torque Engine Manuf acturers

Delta Fuel 42 Delta Fuel Engine Manufacturers
Delta Speed 43 Delta Dynamic Speed

Delta Fuel Rate 44 Delta Dynamic Fuel Rate

Delta CO 45 2 Delta CO 2 SS

46 Delta CO 2 Transient

49 Delta CO 2 Ambient Temperature

The thirty

- one (31 ) error surfaces that have been implemented are included or excluded by the controls num bered 1

– 49 identified in Table 46

. When all 31 error controls are on (included in calculation), the sum of column D in the worksheet ErrorControl is 31.

2.3 Worksheet 2: Summary
The Summary worksheet in the Error Model wor kboo k comprises input mode control i n rows 4

– 10 and output summary i n rows 88 and 119

. Other rows in this worksheet are available for diagnostic purposes.
The calculation mode control is accomplished with cell A6 where the user normally confirms that a numerical value of “2” is designated. Mode 2 designates emission s calculation with all errors applied. Mode 1 corresponds to a calculation of emissions with all errors appl ied except environmental errors . Mode 0 designates an “ideal” emission s calculat ion with no errors applied

. I n Monte Carlo error model si mulation performed in this study M ode 2 w as used.
Mode 0 is used prior to Monte Carlo simulation to generat e the “ideal” emissions for a given reference NTE e vent. The Mode 0 values are calculated by entering a value of “ 0 ” in cell A6. The Mode 0 calculation and subsequent s toring of the “ideal” emission s resul ts may be accomplished manually (as described above) or by exercising a provided macro . The macro248 SwRI Project 14936.12 automatically sets the value in cell A6 to zero, calculates and saves the “ideal” emission s values, and returns the value in A6 to “ 2 ” in preparation for the Monte Carlo simulation. The locations where the reference NTE e vent must be entered manually, and the locations where the “ideal” emissions must be saved (done automatically if the macro is used) are described in the Methods worksheet section.
Mode 1 in cell A6 is not typically used but can be applied for diagnostic p urposes.
The output su mmary section of the Summary worksheet i n rows 88 and 119 presents numerically and descriptively labeled outputs of the emission s and emission s error calculations.
In the output summary, the cells that are highlighted in turquoise c olor are designated by Crystal Ball as “Forecast” (or output) random variables

.
A total of 14 outputs (“Forecasts”) a re designated in the Summary worksheet rows 88 and 119 covering the number of output values from PM emission , three calculation methods (Methods 1, 2 and 3) for the AVL PEMS and two methods (Methods 1 and 2) for the Horiba and Sensors PEMS , for the full error model and for the validation model (designated Valid in Summary worksheet variable labels). All of these “Forecasts” are provided i n units of grams/hp

hr. This variety of calculations was accomplished with the Methods worksheet

.
2.4 Worksheet 3: Methods

The Methods worksheet of the Error Model workbook comprises the following areas:

**•** Notes and diagnostic guides are located principa lly in rows 1

– 21 in columns A

– CF, continuing on row 5 through column DD.

**•** Reference NTE event data are located in rows 35

- 351 of columns A

– W. Actual reference NTE e vent data must be entered manually (or automatically under batch control) starting on row 52 in columns A

– V. Cell W52 data must be entered, and is provided for special case study where the Method 3 flow

- weighted PM concentration may differ from Methods 1 and 2. One to 300 rows of reference NTE e vent data are allowed. Uniform (one se cond interval) time steps are assumed represented by the reference NTE data.

**•** Parameters calculated are located in rows 35

– 351 of columns X

– CF. The number of rows of these parameter equations must match the number of rows in the reference NTE249 SwRI Project 14936.12 e vent. Excess cells in these columns may be cleared manually or automatically during execution of the macro.

**•** Mode 0, Ideal Emissions for this reference NTE e vent are stored in column O rows 22

– 30 (either manually or automatically by the macro). Related data o n the same rows are located in columns CT

– DD.

**•** Input i c random variable distributions (Crystal Ball uses the terminology “Assumptions” for these inputs) are located in rows 26

– 32 of columns AG

- CC

.

**•** Emission s calculations by three methods are located in rows 6

- 81 of columns CH

– CJ (Method 1), CL

– CN (Method 2) and C P (Method 3). This part of the worksheet calculates full model and validation model

.

2.5 Methods Worksheet: Notes and Diagnostic Guide

In r ows 1

– 22 for columns A

– CF, several descriptive labels and references are defined for use in navigating through the worksheet. R ow 5
, column s A

- DD , contain s column identification numbers referenced in rows 7 through 22 (depending on the column). For example, in column H the values 65 on ro w 8 indicate s that the values in column H (rows 52 and follow ing rows) are applied in column 65 (BM) labeled on row 5. If the user scrolls to cell BM52 it is observed that the spreadsheet formula in the cell refer s to values from column H. The informatio n in t he notes and diagnostic guide was not applied by the spreadsheet in any of t he emissions calculations. It wa s included with the intent to simplify diagnostics by providing informati on on locations where spreadsheet values were applied elsewhere in t he spreadsheet. Outside the areas indicated above, some other notes, comments and diagnostic guides may be found in other areas of the spreadsheet.

2.5.1 Methods Worksheet: Reference NTE Event

The reference NTE e vent used in the simu lation was entered in rows 35

- 351 of columns A

– W. Actual reference NTE e vent data must be entered manually starting on row 52 in columns A

– V. Cell W52 data must be entered, and is provided for special case study where the Method 3 flow

- weighted PM concentration may dif fer from Methods 1 and 2. A minimum of one and a maximum of 300 rows of reference NTE e vent data are allowe d. Equal time steps ( 1250 SwRI Project 14936.12 second intervals) are assumed in the reference NTE data r ows . The standard format and engineering units of reference NTE ev ent data established for this project must be observed. These are described in the column headings on rows 47

– 5 1, columns A

– V

.

2.5.2 Methods Worksheet: Parameters
Parameters applied in the three emission s methods are calculated in rows 35

– 351 of

c olumns X

– CF. The number of rows of these parameter equations must match the number of rows in the reference NTE e vent. Excess cells in these columns may be cleared manually or automatically during execution of the macro.
The formulas applied i n rows 52 and down in columns X

– CF have been produced by normal edit

- copy (typically of row 52 in these columns) and edit

- paste to rows 53 and following rows in these columns. The **∆** t values displayed in column AC are not used in any calculation, but are displ ayed so a user can confirm uniform reference NTE e vent time sampling. The last cell in column AC can be cleared (done automatically by the macro). Note that excess cells in these columns must be cleared, and row deletion operations should not be applied since this would affec t other areas in the Methods worksheet.
Certain sums are performed in several columns over the Parameter rows (range of the reference NTE event) . These are accomp lished i n row 46 in columns AI, AW, AX, BO, BQ, BU and C A

. Flow

- wei ghted PM concentration SS and TR errors are consolidated, with and without environmental errors, in the area of cells AN40:AB47. Certain constants applied in the calculation are stored in cells AW42, BC42, BI42, BP40 and BP42. Other constants or conversi on factors are incorporated numerically in spreadsheet formulas. Typical of these is “0.01” to convert a percentage to a fraction.
Specific parameters or variables are calculated in the various columns for application in all three methods, full model and validation model. Table 47 lists the parameters used in the Methods worksheet, the columns where they are computed and a brief description of the parameters.

TABLE 47 . METHODS WORKSHEET PARAMET ER COLUMN DESCRIPTIO NS
Methods Worksheet Parameters Column Descriptions

Subject Column Description

Engine operating state percentages X

– AB Convert NTE Event variables to percentages: speed, torque, fuel rate, exhaust flow

**∆** Time AC Displays **∆** t between NTE Event rows251 SwRI Project 14936.12 NMHC AD Calculate NMHC ppm as 0.98 of THC ppm

Fuel Rate AE Calculate fuel rate g/s based on fuel density of 851 g/L

Exhaust Flow Calculations AF Convert exhaust flow SCFM to mol/s

AG Sum exhaust flow errors from Delta tabs 20, 21, 22, 23, 25 27 and 28 expressed in % of mol/s maximum. Respective ErrorControl tab switches are applied.

AH Convert the total exhaust flow error in % of maximum mol/s to mol/s

AI Add the mol/s exhaust flow error to the exhaust flow in mol/s. Mode control logic is applied.

Speed with error AJ Add engine speed error from Delta tab 43 expressed as % of engine range converted to engine speed in rpm. Mode control logic and ErrorControl switch are applied.

Fuel rate with error AK Combine Delta tab 42 Fuel (engine manufacturer) with fuel rate from Delta tab 44 expressed as % of maximum fuel rate converted to g/s to engine fuel rate in g/s. Mode control logic and ErrorControl switch are applied.

Torque AL Sum torque errors from Delta tabs 29, 30, 31, 32, 34 expressed as % of peak torque, and from Delta tab 35 expressed as % of NTE point torque converted to % of peak torque. ErrorControl switches are applied.

AM Add the total torque error expressed as % of peak torque converted to N·m to engine torque in N·m. Mode control logic is applied.

PM, μg/mol AN42, AN43 Sum PM SS and TR errors from Delta tabs 1 and 2 for AVL PEMS expressed as μg/mol, AN 42 for Methods 1 and 2, AN43 for Method 3. ErrorControl switches are applied.

AO42 Sum PM SS and TR errors from Delta tabs 1 and 2 for Horiba PEMS expressed as μg/mol, for Methods 1 and 2. ErrorControl switches are applied.

AP42 Sum PM SS and TR errors from Delta tabs 1 and 2 for Sensors PEMS expressed as μg/mol, for Methods 1 and 2. ErrorControl switches are applied.

AQ42, AQ43 Sum AVL flow

- weighted PM concentration to all errors except environmental PM errors, AQ 42 for Metho ds 1 and 2, AQ43 for Method 3. Mode control logic is applied.

AR42 Sum Horiba flow

- weighted PM concentration to all errors except environmental PM errors. Mode control logic is applied.

AS42 Sum Sensors flow

- weighted PM concentration to all errors except environmental PM errors. Mode control logic is applied.252 SwRI Project 14936.12 AQ46, AQ47 Sum AVL flow

- weighted PM concentration to all errors including environmental PM errors Delta tabs 4 and 5, AQ 46 for Methods 1 and 2, AQ47 for Method 3. Mode control logic is applied.

AR46 Same formula as AR42 as no environmental PM error model applied to Horiba flow

- weighted PM concentration.

AS46 Same formula as AS42 as no environmental PM error model applied to Sensors flow

- weighted PM concentration.

Speed

· Torque AW Form product of Speed (rpm, all errors case, column AJ) and Torque (N

· m, all errors case, column AM) for application in Methods 1 and 3. Convert rpm to radians/sec with 2 **π** radians/revolution, minutes to seconds with 60sec/min, N

· m/sec to watt hr with 3600J oules/watt hr, and watt to kw with 1000w/kw.

AX Form product of Speed (rpm, no errors for validation case, column O) and Torque (N

· m, no errors for validation case, column T) for application in Methods 1 and 3. Convert rpm to radians/sec with 2 **π** radians/ revolution, minutes to seconds with 60sec/min, N

· m/sec to watt hr with 3600Joules/watt hr, and watt to kw with 1000w/kw.

CO and **∆** CO, % AY Sum environmental CO errors including errors from Delta tabs 10 and 11.

AZ Sum other CO errors. Error from Delta tab 7 is the only one developed.

BA Add the total CO errors expressed as % to engine CO in %. Mode control logic is applied.

NMHC and **∆** NMHC, ppm BE Sum environmental NMHC errors including errors from Delta tabs 16, 17 and 19.

BF Sum other NMHC errors including errors from Delta tabs 13 and 14.

BG Add the total NMHC errors expressed as ppm to engine NMHC in PPM. Mode control logic is applied.

CO 2 and **∆** CO 2 BK , % Sum environmental CO 2 errors. Error from Delta tab 49 is the only one developed.

BL Sum other CO 2 errors including errors from Delta tabs 45 and 46.

BM Add the total CO 2 errors expressed as % to engine CO 2 in %. Mode control logic is applied.

Exhaust Flow

·
[ NMHC + ( CO+CO 2 BO

) ] / [fuel mass flow rate / Speed · Torque ] Form product of hydrocarbons fraction plus CO and CO 2 fractions (all errors case, columns BG, BA and BM) and exhaust flow (mol/s, column AI) divided by ratio of fuel rate (g/s, all errors case, column AK) to speed

- torque product (all errors case, column AW) for application in PM Method 2.253 SwRI Project 14936.12 BQ Form product of NMHC fraction plus CO and CO 2 fractions (all errors case, columns BG, BA and BM) and exhaust flow (mol/s, column AI) divided by ratio of fuel rate (g/s, no errors case, column AE) to speed

torque product (no errors case, column AX) for application in PM Method 2 validation.

NMHC + ( CO+CO 2 BS ) Form sum of NMHC fraction plus CO and CO 2 fractions (all errors case, columns BG, BA and BM) for application in Method 3.

Fuel Rate / [ NMHC + ( CO+CO 2 BU ) ] Form quotient, Fuel Rate (g/s, all errors case, column AK) divided by sum of NMHC fraction plus CO and CO 2 fractions (all errors case, column BS) for application in Method 3.

Fuel Rate / [ NMHC + ( CO+CO 2 CA ) ] Form quotient, Fuel Rate (g/s, no errors case, column AE) divided by sum of NMHC fraction plus CO and CO 2 fractions (all errors case, column BS) for application in Method 3 validation.

2.5.3 Methods Worksheet: Mode 0 Ideal Emissions

For the reference NTE e vent in rows 52 and down in columns A

– V, an ideal emissions value must be calculated and stored for application in the emissions difference calculations. The ideal case can be calculated either manually
, or automatically by the macro. Following the calculation, the ideal values are stored by edit

- copy edit

- paste

- special

- values operation to the cells in column O, rows 22

– 30 . The manual operations described below are performed automatically by the macro, if executed, after manually entering the reference NTE e vent.

After manually entering the reference NTE e vent to be simulated and checking that the

n umber of rows of equations in the Parameters section matches the rows in the reference NTE

e vent, a numerical “0” can be enter ed in cell A6 of the Summary worksheet. The Methods worksheet should have calculated Mode 0 results using the reference NTE e vent. If error messages like “#VALUE or #DIV/0!” are displayed, there is probably still a mismatch between the rows of the reference NTE e vent and Parameter equations. When calculated properly (with 0 in Summary A6), the val ues displayed in the Methods w orksheet columns CU, CV and DB will be e qual on each of the rows 22

– 30 . The values in column CX are not yet equal (unless previously calculated and stored for this reference NTE e vent) because they reflect the values stored in Methods worksheet column O , rows 22

– 30 . The next manual step is to edit

- copy column CU, rows 22

- 30 , and store the values by edit

- paste

- special

- values in column O, rows 22

- 30. Now in rows 22

- 30 the columns CU , CV, CX and DB should be equal. The final step is
to return to Summary worksheet cell A6 and change the value from 0 to 2. At this point the spreadsheet could be run in Monte Carlo simulation to produce properly sampled values. Howev er, if the user desires to monitor charts provided in the Delta worksheet s during th e simulation, further row

- matching to the reference NTE e vent is re quired in most of the Delta worksheets

.254 SwRI Project 14936.12 The manual operations described in the previous paragraph are inte nded to explain how the Mode 0 ideal e mission s are calculated and stored for use i n the Monte Carlo simulation when **∆** emission s values are calculated using the ideal emission s results stored in O22

– O30 . The reference NTE e vent must be entered with an operation such as a manual edit

- copy and edit

paste or edit

- paste

- special

- values oper ation. The macro automatically performs the mode 0 calculation, stores the mode 0 results in O22

– O30 , and changes Summary A6 back to mode 2.

The macro also deletes extraneous rows fr om all the appropriate Delta worksheet s so the charts therein displa y properly. It is important to copy the reference NTE e vent into a fully ‘loaded’ starter file with equations filled on 300 rows in the Parameters area, and with full 300 row complement of equation

- rows in each of the appropriate Delta worksheets for the macro to modify the spreadsheet properly.

2.5.4 Methods Worksheet: Input ic Random Variable Distributions

Probability distribution parameters are applied, and simulation trial values of the inputs are generated in rows 26

– 32 of columns AG

- CC . Rows 26 and 27 are used to input distribution parameters . R ows 28 and 29 contain descriptive labels brough t from the appropriate Delta worksheet. R ow 30 is an information

- only number
, row 31 contains the name label applied in Monte Carlo simulation to the in put i c , and row 32 is where the Monte Carlo simulation tool places generated randomly

- sampled values during simulation. The values in row 32 are referenced by for mula in the respective Delta worksheet s where they are used fo r interpolation on the error su rfaces.

The Monte Carlo tool in Crystal Ball uses the terminology “Assumptions” for these inputs. Two distribution forms are applied: truncated normal (Gaussian), and discrete uniform. For the normal distribution, the applied standard deviation is i n row 27. In Crystal Ball, the standard deviation cell on row 27 and the label cell on row 31 were referenced by equation in the Crystal Ball assumption setup window, the mean was input as 0, and the distribution was truncated at

- 1 .414319083 and at +1. 4143 19083. Since all the truncated normal i c distributions are identical (although the sampled trial values from each will be random in the Monte Carlo simulation), the Crystal Ball define

- copy data and define

- paste data operations were applied to define the truncated normal distributions for other i c

For the discrete uniform distributions, the minimum d iscrete value (1 in all cases) was applied i n row 26, the maximum discrete value was applied i n row 27 and the other row descriptions are the same as before. Again, one of these inputs was setup with Crystal Ball variables once the first one had been defined.255 SwRI Project 14936.12 “ define assumption ” and then applied with Crystal Ball define

- copy data and define

- paste data operations to other i c cells on row 32 where a discrete uni form distribution wa s applied. When Crystal Ball “Assumptions” were defined, Crystal Ball colored each input cell bright green
During a Monte Carlo simulation, the Monte Carlo tool (e.g. Crystal Ball) placed a numerical value in each of the i c cells on r ow 3 2. Then the spreadsheet wa s exercised to perform int erpolations in all the Delta worksheet s. The resulting error sample values for the entire reference NTE e vent wer e returned to the Methods worksheet Para meters area, and then the Methods worksheet E mission Calculations section computes **∆** emissions using three methods, full model and validation to generate one set of the 14 output values described in the Summary section. The simulation tool stores the set of random input values from row 32 as well as the output values in a n Excel data base from which the corresponding sets of values can later be extracted. Once each trial wa s completed, the simulation tool randomly sampled a second set of input values from the respective probabilit y distributions, placed the values i n the cells on row 32, exercised the spreadsheet again, stored the inpu t and output values, and went to a thi rd trial, etc. Typically 4 0,000 to 65 ,000 trials, depending on the reference NTE e vent, were used in this project with this Error Model workbook.
Note that there are three ways the user can control the effect of the i c values in the emission s calculation s:
Mode control in Summary A6, Include / exclude switches in ErrorControl column A D, and Specification of input random variables (“Assumptions”) and t heir probability distributions i n the Methods worksheet row 32.

These three ways of controlling the i c

1. Mode 0

- no errors included va lues are independent , but the effects are i nterdependent as follows. Mode control determines what categories of errors are added into the calculations. Mode controls categories of errors are classified as :

2. Mode 1

- “all ” but ‘environmental’ errors included 3. Mode 2

- “all” erro rs added into the calculations
“All” in this context represent s those error surfaces turned on by the switches in the ErrorControl worksheet . The input random variable distribution controls the distribu tion of the sampled i c values applied during Monte Carlo simulation for the several Delta error surfaces. Mode and ErrorControl switches must be appropriately turned on for the effects of the sampled i c

2.5.5 Methods Worksheet: Emission Calculations values to be included in the emissions difference re sults. These controls affect the calculations in the Methods worksheet Parameters and Emission Calculations sections.256 SwRI Project 14936.12 In the ar ea of rows 6

- 81 of columns CH

– CP the brake

- specific emissions and **∆** emissions calculations are performed using the variables and parameters generated in the Parameters section. Three sets of columns , structured similarly, calculate the full model, validation model, time alignment and drift correction for the following met hods:

1. Method 1 calculations are applied in columns CH

– CJ,

2. Method 2 calculations are applied in columns CL

– CN, and

3. Method 3 calculations are in columns CP

.

Columns CH

- CJ for Method 1 are typical of the methods where the structure is the same, bu t the formulas are a little different. Column CH performs the PM emission calculations for the AVL PEMS
, column CI performs for the Horiba PEMS and column CJ for the Sensors PEMS

. The structure of the three columns is the same. Formulas implemented in t he three columns are the same, but the equations implementing the formulas apply variables and parameters appropriate to the respective PEMS

. As an example of the calculation for PM Method 1 we will examine column CH in detail. The full model calculation was accomplished in cells CH48

– CH54 . The ideal emissions result

wa s brought into the area by equation in CH51. Full model PM emissions ( e PM) in g/kw

- hr

were calculated in CH54. Cells CH55

– CH59 are information

- only diagnostic aids. The full model M ethod 1 result in CH54 is calculated by the formula s in Figure 1.

PM 

### m



## ( **) (**  **) (**  **)** 

# ∑ **∑**

**= =**  

###  ** **  ** **  ** ∆** 

**⋅ **  ** **  ** **  ** ∆**  ** **  ** **  ** **  ** **  ** **  ** ** 

**= ⋅**  N

i i i N

i i PM

PM 

### t m N T rpm Speed t s mol n mol

g m hr kW g e

1 1



### 3600 * 1000 * 60 * 14159

. 3 * 2 * * *

* /  is a flow weighted particulate matter exhaust concentration in g/mol257 SwRI Project 14936.12 FIGURE 181 . BRAKE

- SPECIFIC PM BY METHO D 1

In the formula for the full model mode 2, delta error values sampled from the Delta worksheet s 1 , 2, 4 and 5 have been added in PM 

### m

**•** 2 0 Delta Exhaust Flow SS . Similarly, delta error values sampled from Delta worksheets 20

- 23, 25, 27 and 28 have been added to the ex haust flow, delta errors sampled from worksheets 29

- 32 and 34

- 35 were added to torque, and worksheet 43 deltas were added to speed. The **∆** t values are equal (1 second) and therefore cancel out of the equation.

The validation model calculation was accomplished in the cells CH79

– CH8 1. Validation model PM emissions (g/kW

- hr) wa s calculated in cell CH81.

Calculations for PM by Method 1 described above for the AVL PEMS in column CH are similar for the Horiba and Sensors PEMS by Method 1 in columns C I and CJ, respectively. Similar calculations for PM by Method 2 are presented in columns CL

– CN, and by Method 3 in column CP.

2.6 Worksheet 4: Constants and Equations

The Constants&Eqns tab was strictly a snapshot of equations used in the brake

- speci fic emissions calculations. It displayed the equations and constants implemented in spreads heet formulas of the Methods worksheet

2.7 Worksheet 5: SS PM Error Surface

The 7 Delta C O SS worksheet is a typical Delta worksheet . Its functional structure, formulas, charts and operation are very similar to the following worksheets:

**•** 22 Delta Exhaust Flow Pulsation

**•** 23 Delta Exhaust Flow Swirl

**•** 30 Delta Torque DOE Testing

**•** 45 Delta CO2 SS

With minor changes in charts and structure, its function, formulas and ope ration are also similar to the following worksheets:258 SwRI Project 14936.12 **•** 1 AVL Delta PM SS

**•** 1 Horiba Delta PM SS

**•** 1 Sensors Delta PM SS

**•** 2 AVL Delta PM Transient

**•** 2 Horiba Delta PM Transient

**•** 2 Sensors Delta PM Transient

**•** 4 AVL Delta PM AtmosPressure

**•** 5 AVL Delta PM Ambient Temp

**•** 13 Delta NMHC SS

**•** 14 Delta NMHC Transient

**•** 19 Delta Ambient NMHC

**•** 21 Delta Exhaust Flow Transient

**•** 25 Delta Exhaust EMI

- RFI

**•** 29 Delta Dynamic Torque

**•** 31 Delta Torque Warm

- up

**•** 32 Delta Torque Humidity

**•** 34 Delta Torque Interpolation

**•** 35 Delta T orque Engine Manuf

**•** 42 Delta Fuel Engine Manuf

**•** 43 Delta Dynamic Speed

**•** 44 Delta Dynamic Fuel Rate

**•** 46 Delta CO2 Transient
The following provides a brief summary of the 7 Delta CO SS worksheet :

**•** Rows 1

– 7 contain descriptive information about the error surface implemented in the worksheet.

**•** Rows 8

- 42 present the error surface in columns A

– L. Other columns
, M

– W , on these rows generate a lookup table used with an interpolation routine.

**•** Figures A, B and C follow.

**•** R ows 76

– 379 calculate the **∆** C O SS err or values for each row of the reference NTE event. These values were returned to the Methods tab Parameters section.

The following paragraphs describe in further detail functions in the 7 Delta CO SS worksheet :

Data from the error surface (rows 13

– 42 , columns A

– L, in this Delta worksheet ) must be entered in sorted order (sorted on Lab Nominal column C in ascending order) for proper operation of the x

- lookup

- interpolation function. The three f igures chart the error function. Figure A , in similar De lta tabs, may plot several data sets versus the x

- value, Lab Nominal259 SwRI Project 14936.12 (column C). Figure A y

- values are C O % Lab Nominal (column C), and may also plot 99 th percentile , 50 th perc entile (median) and 1 st percentile.
Related error surface data are plotted in Figure B. Figure B plots several data sets versus the same x

- value, Lab Nominal (column C). Figure B y

- values are the difference, C O % (PEMS)

– C O % (lab, nom). The differences plotted may not correspond exactly to the values shown in Figure A because o f the statistical procedure applied in calculating the differences shown in Figure B. Figure B plots the 99 th percentile (column I), the 50 th percenti le (median) (column H) and the 1 st percentile (column G). In addition to the error surface data, Figure B also shows the interpolation line designated i c = xx (column V), and the reference NTE e vent values on the interpolation line (column F rows 80 through end of the reference NTE e vent versus Lab Nominal x

- values in column B rows 80 through end of the refe rence NTE e vent). When i c = +1 .414319083 , the interpolation line plots on the 9 9 th percentile. When i c = 0, the interpolation line plots on the 50 th percentile. When i c = - 1 .414319083 , the interpolation line plots on the 1 st percentile. The referenc e NTE e vent always plots on the interpolation line, with points at the x

values in the reference NTE e vent.

The error surface data were also plotted in the format o f Figure C. Again the x

- axis was the same Lab nominal (column C). This time the y

- axis data are the i c values. Thus, the 99 th percentile plots at +1 .414319083 , the 50 th percentile plots at 0 and the 1 st percentile plots at

- 1 .414319083 . The interpolation line plots at the value of i c

Now consider inner rows 13

– 41 in the look

- up table in columns T

– W. Column T is a repetition of the x

- value from column C. Column U calculates a row

- to

- row **∆** for the x

- values in column T for use in interpolation. Column V computes the interpolation line linearly interpolated according to the value of i , and the reference NTE e vent plots on the interpolation line at the x

- values in the reference NTE e vent. If appropriate value labels were displayed in Figure C, the values would represent t he error surface plotted on a z

- axis above the two

- dimensional x

- y plane. These error surface values are displayed graphically in Figure B.

c between the median and the 99 th percentile if i c > 0 (on median if i c = 0 and on 99 th percentile if i c = +1 .414319083 ); and b etween the median and the 1 st percentile if i c < 0 (on median if i c = 0 and on 1 st percentile if i c = - 1 .414319083 ). Only one i c value (from cell E80) is applied in this calculation of the interpolation line. The Microsoft Excel vertical lookup function VLOOKUP is applied to the table in rows 12

– 42 in columns T

– W. This is done in rows 80 and down in column F. Because of the way the VLOOKUP function operates, the first row cells T12 and V12, and the last row cell W42 (all three cells distinguished by darker line borders) contain formulas or values different from the formulas of the inner rows. The formula in cell T12 assures that the lookup function can always find an x

- value in its table. The formula in V12 and the value in W12 assure that the inte r polation in cells F80 to the end of the reference NTE event data returns the nearest **∆** CO SS value on the interpolation line if the x

value is outside the range of the error surface lab nominal values.260 SwRI Project 14936.12 Before going to the interpolation accomplished in F80 and down, consider briefly the formulation on rows 12

– 43 in columns O

– R. This formulation considers one x

- value from the reference NTE e vent, the first one, in cell B80 and selects the two adjacent rows in the error surface between which to interpolate on the B80 x

- value. The result is formed on row 43 in these columns and then the “check” cell G80 accomplishes the i c controlled interpolation. This provides an alternative calculation check on one row in the reference NTE e vent.
Now consider the interpolation for each point in the reference NTE e vent. Column B, row 80 and down, brings the lab nominal x

- value from the Methods worksheet reference NTE

e vent. For this Delta worksheet, that x

- value is C O % (lab,nom ). The out

- of

- range flags are information

- only indicating points in the reference NTE e vent with x

- value out of the range of the error surface lab nominal. The i c value for this De lta worksheet wa s brought in to cell E80 from the Methods wor ksheet i c area. Each point in the reference NTE event wa s interpolated with the same i c value, but with its own x

- value. Recalling that the interpolation line in column V was computed with this one i c value, the x

- interpolation between the appropriate tw o adjacent rows in the error surface can now be accomplished. This requires using the x

- value on each row in column B, B80 and down, in the VLOOKUP function, and performing the required calculation using the looked

- up values and deltas from the look

- up ta ble. The calculation is done with the formulas in cell F80 and down. The values computed in column F, cell F80 and down through the reference NTE e vent, could be considered elements of a column matrix or vector, and are returned to the Methods worksheet Parameters section.

In the Monte Carlo simulation, t he Methods worksheet combines this reference NTE

e vent result vector from the 7 Delta C O SS worksheet with similar results from ot her error surfaces, calculates **∆** emissions by three methods, full model an d validation to produce a set of 14 output values (“Forecasts” in Crystal Ball terminology) described in the Summary worksheet section. This was done having input i c values (including one i c value for this Delta C O SS) all chosen by random sample from the appropriate truncated normal or uniform distribution as explained in the Methods worksheet section. Then another sample set of randomly sampled values was input (only one i c value coming to this Delta function again). The reference NTE event C O SS vecto r wa s recomputed with the one new i c

The number of rows in the Delta worksheet reference NTE e vent area (rows 80 and down) shou ld match the number of rows in the reference NTE event applied in the Methods worksheet for proper function of Figures B and C. The starter spreadsheet has been set up with the range of charted reference NTE e vent series extending through row 379 in this Delta worksheet . Th e balance of the spreadsheet sho uld calculate correctly when a reference NTE value, returned to Methods worksheet and ano ther set of 14 output values was produced. This process wa s repeated many times until a statistical convergence criterion, d escribed in Section 2, wa s satisfied. Typically, 40,000 to 65 ,000 sets of input values and 14 output values were produced to satisfy the convergence criterion with this Error Model spreadsheet.261 SwRI Project 14936.12 e vent is properly entered in the Methods tab and Parameters formulas properly aligned, although figures like B and C will not display properly until the last row of the reference NTE e vent is coincident with the end of the range of the charted reference NTE e vent series. This could be done manually in each Delta worksheet wh ere needed, however, the macro wa s designed to convert the fully ‘loaded’ starter workbook after the referen ce NTE e vent wa s entered in the Methods worksheet . The macro uses the row count in the reference NTE e vent, al igns formulas in the Methods worksheet Parameters area, and eliminates extra rows in the reference NTE e vent

ar ea of each appropriate Delta works heet . Again, the macro will do the operations correctly only on a fully ‘loaded’ starter workbook set up with 300 row s of formulas in the Methods worksheet Parameter ar ea, and in each of the Delta worksheet s using the reference NTE event.262 SwRI Project 14936.12263 SwRI Project 14936.12 APPENDIX D : STEERING COMMITTEE M EETING MINUTES264 SwRI Project 14936.12265 SwRI Project 14936.12 PM Measurement Allowance Steering Committee Meeting
Meeting at CE

- CERT
April 23, 24, and 25, 2008

Future Meetings:

1. May 15

- 16 meeting, SwRI 2. June 12 and 13 at for PM

- PEMS and 10 and 11 for EMTC, ACS. 3. July 29 and 30, San Antonio, SwRI 4. August 28, 29, San Antonio, SwRI

On

- Road PM PEMS Discussion and Action Items

CO 2

1. Use CO

Activities

2 data provided by CE

- CERT during the gaseous program for CO 2

Test Plan Activities validation, and share the information during the May 15

- 1 6 meeting at SwRI.

a. Explain in test plan why Horiba can’t use Method 3 b. Replace BSFC in the equation of Method 2 with (fuel flow/torque *speed). Essentially, remove BSFC error surface

c. Show how (mp bar) for the AVL is calculated d. For AVL Method 3, use a different error surface for PM.

e. For reference NTEs, use the existing reference NTEs, and PM concentrations to produce different concentration from DPF out to threshold levels. f. Check section 2.3 with Bob Mason to make sure it is correct g. Change the example to PM in the test plan and give an appropriate PM concentration range per mole basis.266 SwRI Project 14936.12 h. Reexamine the text to explain the new equations better. No second by second for AVL i. Change Figure 8 in the test plan to reflect the fuel flow inclusio n, and remove BSFC error. j. Change Figure 8 to update all errors that are required k. State the time and date by when the model could be available to the group, assuming no last minute changes are required.

General Notes:

1. For AVL, use a 3 to 1 dilution r atio and change the selectable range if needed but not the dilution ratio. 2. For the NTE windowing, use the EEPS to determine the windowing sensitivity, assuming a maximum of 6 seconds delay at the beginning and end. 3. Measure the CVS dilution air temperatur e as close as close as possible to the exhaust and dilution air mixing point. This may require insulation of SwRI CVS system. 4. Use Whatman Teflon membrane, 2 micrometer for the entire program 5. Use the fine ambient backup screen instead of the coarse or diesel backup screen 6. List all the error surfaces in the test plan 7. Plan Training and Comissioning of the PM Generator 8. If 32 seconds NTE is not sufficient for the PM

- PEMS, extend the length 9. Use 10 % difference as an acceptable difference between SwRI and CE

- CERT

1065 Audit (For reference see SwRI presentation)

TSI Flow Meters267 SwRI Project 14936.12 **•** Using reference flow meters like the TSI flow meter as a transfer standards is okay, if independently checked via a master flow meter

Sensors PPMD

a. Verify that the external and by pass flow TSI flow meter certificates are valid for duration of testing b. Verify that the data published on their individual certificates meet the 1065 linearity thresholds

Horiba TRPM

**•** Use total flow and dilution air flow for Horiba and make sure they meet 1065 linearity verifications.

AVL MSS

**•** Meeting +/

- 3 % per point on dilution ratio is acceptable.

Vibration

Perform a frequency sweep with moderate amplitude and log the real time signals from the PEMS using a fixed PM generator level. If changes occur, design a frequency cycle around that frequency range to test for delta changes in PM with and without vibration. Obtain approval on the frequency cycle selected from the MASC before proceeding. If no changes are observed during a frequency sweep, th ere is not a need to test for vibration.

Use two orientations vertical and horizontal for the PPMD. If there is enough space on the vibration table at SwRI, test all PEMS units at once. If not, use them one by one.

EMI268 SwRI Project 14936.12 Use EMI on and off, and screen th e real time response of the instrument at a specific concentration level. If a response is observed during EMI on/off switching, the MASC will decide on what the next steps are.

Environmental Chamber:

1. Temperature and Humidity cycle will be made availab le by Matt Spears 2. Use a high velocity fan to blow over the 4 inch inlet section of the PPMD 3. PM Generator Setup and Verification

4. How to do the environmental pressure test?

a) Use three different levels concentration at the 0.02 and 0.03 g/hp

- hr b) Run a long baseline of 8 hours using three different toggeled PM concentration levels. For baseline, toggle the process five times, and use that as a baseline, followed by the continuation of the baseline. c) Run the temperature and humidity test using the sa me toggeling profile. For temperature, humidity, and pressure, start with a baseline with five repeats, and then kick off the environmental cycle. d) A toggeling of zero, mid, and high is on the order of 15 minutes.

There is a possibility of eliminating the zero and add one concentration level, and randomly sample from all the deltas generated. We need to talk to Bob Mason about this.

Use just one of each of the PEMS for environmental activities.

Finally, changes were made to the test plan during discussio ns. Also, SwRI presented that attached document on the test plan. More test plan discussions will take place in the next meeting on May 15 and 16, at SwRI.269 SwRI Project 14936.12270 SwRI Project 14936.12 PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI
May 15

- 16, 2008

Day 1

1. Tim French indicates that there are several issues came up with the PM

- PEMS onboard testing performed at CE

- CERT before the CE

- CERT meeting was presented.

2. Kent’s presentation, posted on FTP website:

Some highlights, but see presentation for more details:

a. Pr oblems with valve timing opening and closing consistency and long delays over 3 seconds. b. There was one crystal that was very noisy but the software did not reject it, and reported data using it.

c. A drop in frequency but an increase in mass deposit. (This is a non

- issue) d. Semtech DS 10[1].09 SP2 b5. This beta J1939 includes filtering and was able to include NTEs.

e. Post test J1939 filtering versus real time J1939 filtering. Post test captured different start up NTE than the real time filtering one. Essentially, post processing software needs to be scrutinized. f. NTE yield produced by PPMD was low. Some of the valid NTE that was captured in PPMD post processing was rejected, after carefully reviewing the data. g. Others. See presentation.

3. SwRI update:

a. Horiba linearit y check was resolved by using the Gilibrator directlty b. Engine A, B, C, and D 80 steady

- state testing was complete

c. Engine A was tested for 40 SS points in sub

- NTE runs d. International engine was also tested

e. All above will be posted on the website after carefu l review271 SwRI Project 14936.12 4. Janet and Bob Presentation, please see website.

Bill Martin questioned the idea of excluding the 95 th to 100 th

5. General Discussion and Action Items: data twice. One in the error surfaces, and the other in the measurement allowance. He was concerned whether such practice will lead to truncation of the distribution. Janet will show in the next meeting that such practice will not lead to any truncation o f data or it will effect the shape of the error distribution.

a. Find ways to load up the can bus during the actual testing to simulate real world operation by making the can bus communication busy. b. Keep using the gase ous error surfaces for Method 2 and Method 3 on the PM

PEMS.

c. Environmental Chamber:

1. Use a 9 minute cycle for the environmental chamber and go into three concentration levels, 3 minutes per level. Sample for a period of 35 seconds from each level. 2. Use fi ve levels of dilution ratio ranging from 6 to 50. 3. Check with Sensors to see if they accept an exhaust flow meter analog input.

For Model output, keep the 95 th as the measurement allowance. However, if it did not validate consider the potential of using o ther than the 95 th

Day 2: , if the MASC reaches an agreement on that.272 SwRI Project 14936.12 1. Move forward with the International Engine 2. Communicate via J1939 to load the engine can bus during testing. Use only the J1939 communication with the heavy

- heavy duty diesel engine 3. Accept the idea that Horiba will use test cell engine speed and load analog output signals 4. The right to remove outliers using good engineering judgement. 5. Starting on June 2 nd 6. Spend one week of commissioning before we start. with commissioning

Plot:

x

- axi s percent of max torque or fuel rate

y

- axis absolute difference over max torque or fuel rate

Matt’s Discussion:

Model Validation Testing at CE

- CERT

a. Number of engines/vehicles b. Number of bypass conditions, at least two, maybe 3

c. Number of PEMS: at least t wo of each, highly desirable three d. Number of NTE events: total 100

- 200 per PEMS

e. Number of route repeats: 1 to 3 f. Types of NTE events i. CE

- CERT limits 1. Minimum filter loading: 50 micrograms

Gaseous plus PM. But PM set priorities. Sensors goes first.

NTE Route Bypass Repeat273 SwRI Project 14936.12 Events
Horiba plus AVL 30

- 50 Palm Springs 1 One

- Run

Sensors plus AVL 30

- 50 Palm Springs 1, review One

- Run

Horiba plus AVL 30

- 50 San Diego 1 or change One

- Run

Sensors plus AVL 30

- 50 San Diego 1 One

- Run

Horiba plus AVL 30

- 50 Baker 1 One

- Run

Sensors plus AVL 30

- 50 Baker 1 One

- Run274 SwRI Project 14936.12 PM Measurement Allowance Steering Committee Meeting
Meeting in Madison, Wisconsin

June 12

- 13, 2008

The meeting started by reviewing and approving the last meeting minutes.

PPMD Commissioning

a. Following the meeting minutes, Imad Khalek gave a presentation on the status of the PPMD commissioning at SwRI. The presentation was sent to the MASC, but was not posted on the website.

b. David Booker also gave a presentation on the SwRI commissioning activities. The presentation was sent to the MASC, but was not posted on the website at the request of Sensors.

c. As a result of these two presentations, the MASC decided to give Sensors a chance to fix some of the problems encountered and com e back to SwRI for additional commissioning during the week of June 16. Also, the MASC requested that a conference call should take place on Friday, June 20, and SwRI should give a status update on whether or not the Semtech

- DS/PPMD issues were fixed to a satisfactory level so the program can proceed.

d. The MASC also made the following points on the PPMD:

i. It is up to Sensors to decide on the quartz crystal equilibration time, after a crystal goes into a invalid NTE window. ii. It is requested that when all cry stals are locked out, and one of them becomes available during an NTE, the crystal should wait until the current NTE terminates, and a new NTE starts before it samples from an NTE.275 SwRI Project 14936.12 PM

- PEMS Engine Selection

Imad Khalek pointed out to the MASC the fact that the PM

- PEMS program is moving forward with some deficiencies related to the Horiba system inability to communicate with the Navistar engine ECM ISO protocol. It was recommended to the MASC that a heavy

- heavy duty diesel engine be installed in the tes t cell first so the Horiba system can communicate with the engine ECM using the J1939 protocol. In addition, this will give a chance for Horiba to upgrade their system in preparation for the Navistar engine after a heavy heavy

- duty diesel engine.

EMA agr eed to take a look at the possibility of providing a heavy heavy

- duty diesel engine in a timely manner. EMA agreed to make a final decision on this issue by the June 20 conference call.

Horiba Concern

Horiba was concerned about the fact that Sensors brou ght a new model of the PPMD to be used on the PM

- PEMS

- MA program. They wanted to have a chance to update their system as well.

The MASC decided to give Horiba a chance to work and upgrade their systems. It was decided that SwRI should ship back to Horiba one of their system present at SwRI, and give them until July 14 to ship back the system.

Horiba was also given until September 15 to upgrade their system so it can communicate with the Navistar ECM protocol using ISO

- 15765.

Bill Martin made comments about the difficulty he sees in the Horiba system meeting 1065 requirements. Matt indicated that EPA would enter an allowance for Horiba specifically, at the time of Direct Final Rule (DFR), through the alternate procedure approval.276 SwRI Project 14936.12 AVL Presentation

Bill Silvis presented results on the MSS, where a compensation algorithm is added to account for organic carbon and sulfate. No copy of the presentation was given to SwRI for distribution. Matt Spears was not convinced that such compensation will be acceptable for EPA approval due to the significant correction required and due to the MSS principle itself. SwRI also had some technical reservation about the process due to its technical complexity. Essentially, the problem is not trivial and more thorough work an d understanding is still needed.

AVL was encouraged to continue working on this issue and refine it. It is understood that they will submit the compensating algorithms prior to the start of steady

- state testing. They may also submit their compensating al gorithms at any time for the MASC to have analyzed by SwRI or CE

- CERT.

Rey Agama suggested that time would be the best window of opportunity for AVL to be included via DFR through alternate procedure approval.

SwRI Test Cell

The MASC requested the following:

**•** SwRI should install all PM

- PEMS as close as practically possible to the entry of the CVS. This will reduce any particle losses between the point of measurement among the PM

- PEMS and also relative to the CVS. Furthermore, this will minimize the backpressure experienced by the PPMD, by shortening the length of exhaust piping present downstream of the PPMD, prior to entry to the CVS. The target backpressure is (

- 1 to +4 kpa)277 SwRI Project 14936.12 **•** SwRI should try to maintain temperature of 35°C ± 5°C in the vicinity of the PM

- PEMS inside the test cell.

CE

- CERT Presentation

Kent Johnson gave a presentation on the temperature distribution on various location of the MEL such as behind the CAB, under MEL trail, and under passenger door. Kent’s presentation is posted on th e website. One of intriguing highlights of the presentation is that the temperature profile can reach as high as 60 °C to 90 °C during in

- use in the vicinity of the PM

- PEMS. This triggers the idea of changing the environmental temperature profile that was used during the gaseous PEMS program that was based on ambient temperature. Matt spears will be modifying the existing environmental chamber temperature profile, taking into account Kent’s finding.

Budget

CE

- CERT presented their budget with some options. The overall impression was that the total budget for both SwRI and CE

- CERT is beyond the funding level available for this program. Below are some of the options entertained that will be discussed during the next meeting at SwRI starting on July 21 st

1. Red uce the scope of work by cutting the number of routes and the number of PEMS used by CE

- CERT .

2. Reduce the scope of work at SwRI by reducing the number of engines to be tested from two engines to one engine 3. Increase the overall budget by eliminating this year EMA pilot program requirement, and add funding to the MA program instead, assuming that the funding will be cost shared among EMA, EPA, and CARB278 SwRI Project 14936.12 Additional Comments and Action Items:

1. If PPMD and OBS200, both, resulted in negative allowance, e.g.

- 0.01 to

- 0.02 g/hp

- hr, the MA will be zero, essentially one MA for the entire program and for all the PM

- PEMS used. Under such circumstances, both instruments will be allowed to be used with a zero MA. 2. Keep aware of alignment issues. One may want to inve stigate the brake

- specific emission value reported by PM

- PEMS by shifting the numerator and denominators relative to one another and relative to absolute time. This could be done with initial transient test results with bypass. 3. Sensors will provide a VI to simulate the exhaust flow rate so we can exercise the multiple dilution ratios in the environmental chamber. 4. SwRI will need to resubmit some budget options of doing the work with one engine versus two engines.279 SwRI Project 14936.12 PM Measurement Allowance Steering Commi ttee Meeting
Meeting at SwRI, San Antonio
July 21

- 23, 2008

July 21

The meeting started at 1:00 PM. It started by reviewing and approving the previous minutes, and also by discussing future meeting schedules, which were already sent by EMA as future meeting notices. The next meeting is scheduled for August 27 (starts at 2:00 PM), 28, and 29 (ends at 2:00 PM).

Imad Khalek started a presentation, posted on FTP website , on project status update. He showed a comparison between laboratory

- based exhaust f low and the 4

- inch Sensors exhaust flow that was used on the Navistar engine. The slope of the correlation was too high at 1.27. As a result of the discussions, the MASC decided on the following with the current 5

- inch exhaust flow meters used with the Mac k engine, which the first engine to be used as a part of the official measurement allowance work.

**•** Check laboratory

- based exhaust flow rate with Horiba and Sensors 5

- inch exhaust flow meters. If a problem is obvious or the slope of the correlation differs by more than +/

- 0.05 from a slope of 1, then send the flow meters back to the manufacturers to check on the calibration.

After finishing a part of the presentation, Janet Buckingham and Bob Mason showed up for a scheduled presentation at 3:00 PM. The pre sentation is posted on the FTP website and addressed the double truncation issue raised by Bill Martin at the 5 th and 95 th

a. The 95 percentile. The presentation is posted on the FTP website. Based on the presented work, the following was agreed upon:

th b. The 5 perc entile is still desired for the measurement allowance th and 95 th are still acceptable to bound the validation range.280 SwRI Project 14936.12 c. There was still a remaining unresolved issue about where you assign the

- 1, 0,1 on the error surfaces for the delta change between la b and PEMS. It was decided that this issue should be addressed during the last day of the meeting, but that was never brought up again. There was a proposal by Bill Martin to fit both sides of the error distribution independently using a normal distributio n fit. The median of such distribution will be the median based on previous practice along with 95 th for one side and 5 th

July 22

A significant part of the day was allocated for bu dget discussion. The different budget scenarios are posted in the FTP website . Below are some of the highlights:

For budget cutting purposes, one engine will be used for the measurement allowance at SwRI. In addition, work will be performed with one bypass setting that gives brake specific PM emissions levels between the two thresholds of 0.02 and 0.03 g/hp

- hr. For CE

- CERT, choose the PEMS with the lowest positive measurement allowance. If one PEMS clearly shows a lower measurement allowance than the rest, and both are positive, pick the one with the low measurement allowance for the rest of the validation testing.
If one allowance is positive by one PEMS, and if the other one is negative, then, in principle, choose the positive allowance if it slight ly positive. (no clear cut agreement yet).

After the budget discussion, Tim French from EMA walked the group (via phone) through the EMA proposal, posted on FTP website , to substitute year 1 pilot program and to also provide some supplemental funding on the order of $200 K to inject more funding into the measurement allowance program. The overall EMA budget funding was projected to be on the order of $500 K to $700 K. Most or all the proposed activities will be performed in

- use by hiring a third party tha t is not part of EMA and does not belong to one of the engine manufacturer to conduct the testing. As a result of the EMA proposal the following will take place: for the other side. When each side of the distribution is fitted using the above boundary conditions, one can then expand the data pic ked from the error surfaces to cover 1 % to 99 % of the data or even 0.1 % to 99.9 %. No decision has been made on this issue yet. This topic will require more discussion during the next meeting.

**•** Matt Spears will speak with EPA upper

- management to consider the EMA proposal in the context of the one year pilot program.

**•** EPA will also make a final determination on whether to use the shortened version of the measurement allowance program. E.g. one versus two engines, by

- pass, no

- bypass, etc...281 SwRI Project 14936.12 After the budget discussions, Imad Khalek continue d his presentation that was started on the first day. The work showed the different torque and fuel errors surfaces determined between the laboratory and the engine ECM public broadcast. The data covered a total of five engines (four heavy

- heavy and one li ght

- heavy) that included a DDC Series 60, CAT C13, Cummins ISM, Mack MP7, and Navistar 6.4 liter engine.

Also the work covered sub

- NTE fuel flow errors. Based the sub

- NTE results, it was decided that for forced NTEs, if the engine operation falls below N TE, use the last BSFC value observed within the NTE. Use that only for Method 2 calculations.

After this presentation, Imad Khalek refreshed the memory of the group by giving a status update presentation, posted on the FTP website , based on the last comm issioning work done at SwRI. The presentation was posted on the FTP website. After that, Kent Johnson from CE

- CERT shared some observations about the Sensors data produced by SwRI during commissioning. There was no conflict between SwRI and CE

- CERT reporti ng on the results. Kent’s Word document that was shared with the group will be posted at the FTP website when it becomes available to SwRI.

July 23, 2008

The focus of this day was on the Test Plan, particularly the environmental testing. However, at the beginning of this day, the idea brought by SwRI earlier of adding short NTE windows for the laboratory transient testing, along with some low or medium idle operation prior to the NTE transient cycle was discussed.

**•** It was decided that Short NTE windows s hould be added to the thirty 32

- seconds NTE cycle, in the inter time. A total of 10 short NTEs will be added as follows:

1. 4 five seconds NTEs, with two back to back within an inter

- time 2. 3 10 seconds NTEs 3. 3 18 seconds NTEs282 SwRI Project 14936.12 Position two five seconds NTEs back to back in the inter

- NTEs. For Horiba, they need about 2 to 3 seconds to exit an NTE. Thus, place the short NTEs at least five seconds after the end of a valid NTE to avoid continuous sampling with Horiba.

**•** On the conditioning prior to the start of a transient NTE, start with five to 10 minutes at medium idle time before the NTE transient cycle to observe if there is an effect on the laboratory PM emissions results. If there is and effect, then propose an idle time or something similar prior to startin g the NTE transient cycle. (Note that before each NTE transient cycle, the plan is to force

- regenerate the DPF first, medium speed idle will then be added after the forced regeneration).

EMI/RFI/Shock &Vibration

1. For EMI/RFI, expose the PM PEMS to EMI/RFI using HEPA filtered air, similar to the gaseous program. Then decide with the MASC after reviewing the results, what is the next step. As an option, we could use the PM generator to perform this work. One PEMS from each manufacturer will be used for thes e activities. 2. Vibration and Orientation (non

- road only)

a. On the orientation, ask PEMS manufacturers on the worst orientation scenario postion. Use such orientation at 45 o

b. Perform a frequency sweep, with a very moderate amplitude, then share with MASC to decide on how to go forward. Eric should propose the sweep frequency, amplitude and duration. Do two repeats. One of each instrument will be used. This is only for nonroad. Outside the scope of the program. with the appropriate orientation, and survey the worst case scenario for all gaseous and for PPMD, and TRPM.

3. As for shock, ask Eric about a recommended on

- highway profile for shock.

Temperature/Humidity chamber and Pressure Chamber

For both temperature and humidity and pressure chamber work, use three PM concentration levels around the two threshold levels such as 4000, 5000, and 6000 μg/m3. Use a total of four dilution ratios of 6, 12, 20 and 30. At each dilution ratio, stabilize for 4.5 minutes at each concentration level and measure for 30 seconds b efore you go to the next concentration level to do the same. E.g.283 SwRI Project 14936.12 1. Set the dilution ratio to 6, stabilize at 4000 μg/m3 for a 4.5 minutes 2. Measure PM using PEMS for 30 seconds 3. Move to next concentration level of 5,000 μg/m3 and stabilize for 4.5 minutes 4. Measure PM using PEMS for 30 seconds 5. Move to the next concentration level of 6,000 μg/m3 6. Measure PM using PEMS for 30 seconds 7. Repeat 1 through 6 at different dilution ratio

A spreadsheet on PM concentration and loading was originally made by Matt Spears d uring the meeting and was very slightly modified by Imad is posted to the FTP website

As a result of the proposed scenario above, Sensors PPMD will require a secondary dilution using MPS2 to perform an 8 hours of activities using the above scenarios. In c ase of Horiba, either we need to allow higher dilution ratio than those adheres to 1065 or we need to allow more loading on the filter past a 0.4 mg. More discussions on this will need to take place during the next meeting.

The PM cycle to run by the PM g enerator contains five 15 minutes cycle to be used as a baseline. E.g. Run the first five 15 minutes at normal temperature, use those as a baseline line, then proceed for the rest of the day to capture an eight hours of similar repeats. Do the same thing p rior to starting the Temperature/Humidity profile and well as the Pressure Chamber profile . One of the remaining issues that has not been resolved around this topic is how do we capture the baseline information with the Horiba system without the need to ch ange the filter. More discussion on this subject is needed during the next meeting.

New Temperature Profile

Take the mean of CE

- CERT Cab minus ambient and add it to original temperature profile, and solve for new temperature and humidity profile that ma intains derived from the real world data.

Perform a Fourier transform on the CE

- CERT Cab data, eliminate frequency content that are similar to the base ambient profile, and use a magic synthesizer to superimpose the frequency on top of the new temperature profile. Matt Spears is assigned to do this task.284 SwRI Project 14936.12 Pressure Chamber

We resolve the logistical issue of the pressure chamber work by placing the PM generator inside the chamber. Matt will need to send me some dimension on the PM generator to see if it fi ts in the Chamber. Other gases as well as 30 amp circuits should all be accommodated.

For Horiba, heat trace the segment of the transition to 250 o C, similar to the PM generator outlet temperature for both the Horiba and AVL.

Use each instrument separat ely for these tests.285 SwRI Project 14936.12 PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
August 27,28, & 29, 2008

Meetings

October: 9,10

- at EPA
Nov: 12,13, and 14

- at SwRI
Dec.: 10,11,12

- at SwRI

Discussion Points and Decisions Made

**•** For the error distribution choose the 1 st and 99 th at

- 1 and 1, based on a normal distribution fitting to extrapolate down to the 1 st and 99 th

**•** Recheck the 4 inch flowmeter calibration that was on the International engine . For the height of the median, if there is a discontinuity, choose the average or randomly pick one or the other.

**•** For Horiba, filter loading is allowed to go beyond 400 microgram up to 700 microgram or beyond as long as the flow is controlled.

**•** For Horiba, do the five cycle baseline at the beginning for humidity and temperature and do it at the end for the pressure. This will require changing the filter. For the pressure, if it can be done within an hour than do it at the beginning, otherwise, do it at the end.

**•** Which of the three units are acceptable to be used in the pilot program:

 PPMD is approved

 Horiba might be approved286 SwRI Project 14936.12  AVL will have to make the case at EPA to see if approved

 EMA would like to know what instrument would be acceptable for pilot one program

 Can EMA combine pilot effort s within companies

 EMA will fund the additional funding required that will be required to do the intermediate testing that involves:

**•** One bypass setting, one engine at SwRI

**•** Three PEMS, one manufacturer with CE

- CERT

**•** We should move forward on RMI, RFI, and vi bration sweep, as soon as we start the official testing

**•** Due to spikes, we may revisit the post processing of the Horiba results, especially during the NTE transient test. As of right now, Horiba first correlates the entire EAD signal (including spike) wit h the filter weight (not including spikes), then apply the relationship for the NTE window portion of the cycle.

SwRI Presentation and Action Items

During the meeting, SwRI gave a status update via presentation, see enclosed presentation (Update 6), on the PPMD, MSS, and TrPM. SwRI, also discussed the bypass tuning for using 10 NTE transient cycle, 80 points steady

- state using MSS, and projected filter mass concentration, using a relationship between the filter and the MSS. In addition, the bypass mixin g was shown qualitatively. Furthermore, the transient concentration trace with the MSS was presented for the 10 NTE transient cycle.

As a result of the work reported several action items were born:

**•** Make sure that the filter

- based steady

- state projected concentration is consistent with the filter

- based measured for the six steady

- state points selected. Thus, perform an experiment to determine the filter

- based measured concentration for the six points selected and share the results with MASC

**•** Instead of showing a qualitative results on the mixing, provide some quantitative assessment such as a T

- test.

**•** Based on the high spikes observed with the MSS, check if the PPMD results in a low bias due to the fact that it may be missing the early spike. For these ex periments, pick the transient NTE window with the highest spike and create a cycle that consists of 10 repeats of that same window. During this exercise, use only three working crystals and287 SwRI Project 14936.12 vary the PPMD trigger into the NTE with time advancement of 1,2 an d 3 seconds, time delay of 1 second, no time delay, and ECM trigger

The above work and the problems encountered were presented to the MASC via two conference calls that were done on September 12 and September 30, 2008. The presentations are also enclosed as Update 7, and Update 8.288 SwRI Project 14936.12 PM Measurement Allowance Steering Committee Meeting
Meeting at EPA, Ann Arbor
October 9 and 10, 2008

SwRI gave a status update on the progress of testing. See enclosed presentation.

There was a concern about the predicted lasting time of the PPMD in the field, which was predicted to be on the order of 1 hour, using an average NTE threshold of 0.02 g/hp

- hr. There was a desire to extend the running time of the PPMD to at least 2 hours. This is indeed met by the instrument if one takes into account overall collection efficiency of the instrument. In the example given, it was assumed that the collection efficiency was a 100 %, where in reality it was on the order of 50 %. Thus the one hour of lasting time reported is in reality two hours.

A decision was made to move forward with testing using only the MPS1, as shown below, after final commissioning of the PPMD by Sensors. Actually, David Booker from Sensors flew to SwRI late on October 9 to be at SwRI on October 10.

1. Test on MP S1 only, as its currently configured at one microgram. 2. Okay to use external trigger for steady

- state testing 3. For steady

- state testing:

a. Target 100 microgram on the filter b. 50 microgram for TRPM

c. 0.66 microgram for PPMD

Do not clean crystals until it is apparent that the next run will likely overload the filter.

The second day of the meeting was spent at Sensors. Matt Spears gave a presentation on the PM generator. Later, he showed the PM generator setup, and explained to the group the various elements o f the PM generator and the equipment used. A copy of Matt’s presentation is enclosed. Also, a copy of Matt’s note is shown below, particularly to 1065 PEMS changes:289 SwRI Project 14936.12 Matt’s Note:

1. Review minutes from last meeting 2. Discuss 1065 changes required for PM PEMS 3. Update from SwRI on recent activities 4. Update from PEMS manufacturers regarding recent phone conference 5. Friday afternoon @ Sensors

a. SUN conference presentation b. PM Generator 6. Discuss 1065 changes required for ** PEMS field testing only**

a. Dilution Air i. Temperatur e control 1. 1065 = 25±5 ° C 2. PM PEMS= no direct feedback control from a dilution air temperature measurement required

-- use good engineering judgment; if directly and actively controlled then target 25 ° C . ii. You may use a fixed molar mass of the diluted exhaust mixture for all PEMS field testing, as determined by engineering analysis. b. “Filter” i. Media 1. PTFE membrane or TX

- 40 ii. Face velocity 1. 1065= target near 100 cm/s actual, unless overloading 2. PM PEMS

a. Flow

- through media: (10 to 100) cm/s actual, which can be verifie d by engineering analysis b. Non

- flow through media: no specification iii. Temperature 1. 1065 is 47±5 ° C 2. PM PEMS target (42 to 52) at all times, with a minimum tolerance of 32 ° C and a maximum tolerance of 62 ° C, where the tolerances apply only during filter samplin g. iv. Conditions during mass determination 1. 1065=see subpart B 2. PM PEMS

a. If mass is not determined in

- situ

— i.e. within the PEMS

— then the sample collection media must be pre and post analyzed according to 1065.190x. b. If mass is determined in

- situ, follow .195.

c. In subpart J, have no requirement to hold to dewpoint specs for in

- situ analyzers.290 SwRI Project 14936.12 c. Absolute reference for inertial balance i. Current status: QCM OEM stated specs are assumed. ii. For 1065 measurement allowance audit we had Sensors verify frequency measurement circ uit. iii. No immediate solution available d. Cleanup 915 table for inertial batch PM analyzers: no freq, or rise/fall time specs. Recommend a process for determining noise, accuracy, and repeatability

e. 1065 Subpart J needs to state that field testing applies at any ambient temperature, pressure and humidity, unless otherwise specified in the standard setting part. f. State that EPA approves of electrostatic deposition technique for PM collection. Must meet 95% collection efficiency, as stated by the manufacturer. g. Overa ll PEMS test requirement should be reread and edited to be applicable to batch analyzers. For example describe how to use a combination of steady

- state and transient test modes to determine accuracy and repeatability separately; like what we’re doing in t he measurement allowance. h. 1065.308

- 09: also required for continuous PM analyzers

— read and edit accordingly i. 1065 clarify that options after 400 ug loading are optional j. Clarify whether or not ambient air may be used for zero air for PEMS, including for hangu p check. k. Drift: allow any drift that doesn’t affect your ability to demonstrate compliance with the applicable standard. 7. Timetable

a. Next meetings 8. Test Plan

- documentation reflecting the latest agreements

a. Validation

— get from CE

- CERT b. Modeling 9. For PEMS testing set dilution ratio based on manufacturers literature regarding maximum exhaust flow. You may also use other manufacturer information to perform an engineering analysis to estimate the maximum. 10. There will be no dilution ratio verification.291 SwRI Project 14936.12 PM Measure ment Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
November 12 and 14, 2008

Flow Alignment in PEMS

In case of continuous sampling, use a step function to measure the delay from the probe to the instrument.

In case of batch sampling, use the geometry to account for time delay from the probe to the batch sampler

The above is done to align the flow or to account for time delay.

Leave alone any time alignment between exhaust flow and ECM torque and speed. Use the ECM torque an d speed to determine the integral over the NTE.

Loss Correction

a. The principle of PM loss corrections for PM PEMS is agreed upon by the steering committee b. EMA desires a legal construct in 1065 for allowing the use of PM loss corrections i. Open up 1065.295 to allow more types of compensating algorithms, based upon other variables ii. Utilize Subpart J overall approval test to validate entire PM PEMS, including its loss corrections.292 SwRI Project 14936.12 The PEMS manufacturers will be allowed to use compensation algorithms.

**•** Horiba decided to use no particle loss algorithm.

**•** AVL presented a loss algorithm to correct for thermophoresis. The loss correction is already defined and will be implemented via a post processor provided by AVL

**•** Sensors plans to correct for particle loss and wil l share the process with MASC during the next meeting

As of today, no filter data can be shared with Sensors unless the loss correction is shared with the MASC.

Test Matrix, DPF out

Full day no QCM cleaning or filter changing

Storage, high speed, l ight load
Release at peak torque, run for 20 minutes
Store again at high speed, light load
Release at rated power

Do another storage at low idle, equivalent

Cycle the PEMS every one minute to sample into NTE.
Cycle the PEMS every 32 seconds293 SwRI Project 14936.12 5 second dwe ll time in between the NTEs

PEMS Daily Checks Tolerance

Slope of 0.96 is acceptable by Sensors
For Horiba, 3% on filter flow, and 3 percent at a dilution ratio of 5, and 5 percent at a dilution ratio of 15, and Mike will reconfirm.
We will get a feedback from AVL on the accepted tolerance for a dilution ratio of 5.f

Milestones

Milestone for the Model dry run
PM generator milestone for commissioning, week of January 5
Milestone for the fuel flow error surface delivery by the engine manufacturers (EMA wi ll target the end of February for these data to be available. The eight or nine engines that are available now will be used for the dry run).

Fixed date for the delivery of environmental chamber kit by the PEMS manufacturer.
Ship it on the 19 th of Januar y by Sensors and Horiba.

We will schedule EMI, RFI, on the 26 th

Wednesday of January, and shock and vibration on the following week.

Matt Spears’ Note:294 SwRI Project 14936.12 1. Upcoming meetings, December, January, and March all at SwRI

a. December 10

- 12 at SwRI (10 th b. January 28 2pm start, 12 2pm close) th

– 30 th (28 th 2pm start, 30 th

c. March 18 2pm close) th

– 20 th (18 th 2pm start, 20 th 2. Review October meeting minutes 2pm close)

3. EPA / SwRI / CE

- CERT PM filter round

- robin

a. Initial results 4. Temperature

- Humidity test cycle

a. EPA cycle b. SwRI addition of CE

- CERT frequency content 5. Status and progress at SwRI since last meeting

a. Update on mission time projections i. Relook at projections to see if it is possible to collect 20 failed NTE events ( 0.02 g/bhp

- hr) in one mission b. Test plan review

c. Decision s for MASC 6. Budget update

a. Arrangements for EPA/EMA/ARB

Thursday

7. PEMS PM loss algorithms

a. AVL b. Sensors

c. Horiba 8. Test plan development

a. Inclination discussion 9. Continue if necessary: Status and progress at SwRI since last meeting

a. Decisions for MASC

Friday

10. Continue PM PEMS related Part 1065 changes 11. Other meetings during our November meetings

a. Wednesday i. Wrap

- up by 5:30pm due to another room reservation b. Thursday i. 12:00pm EMA, Rey Agama

– break for lunch at 12:00pm ii. 1:00pm

– 2:00pm, Shirish Shimpi, continue meetin g
c. Friday i. Matt Spears 10:00am to 10:45am, continue meeting 12. Loss corrections resolutions

a. The principle of PM loss corrections for PM PEMS is agreed upon by the steering committee, including EMA, EPA, ARB. b. EMA desires a legal construct in 1065 for allowing th e use of PM loss corrections295 SwRI Project 14936.12 i. Open up 1065.295 to allow more types of compensating algorithms, based upon other variables. ii. EPA agrees that PM loss corrections will not be applied to certification testing. If in the future EPA desires to apply PM loss corre ctions to certification testing, such a provision would be proposed as part of a notice of proposed rulemaking because EPA acknowledges that such a change would cause a change in the stringency of the certification standard. iii. EPA will be the approving body with respect to PM PEMS PM loss correction. 1. May make case

- by

- case approvals, based upon specific PM PEMS manufacturer circumstances, such as, but not limited to submitted, models, theory, validation data, or even simply the magnitude of the correction. 2. May utilize Subpart J overall approval test to validate entire PM PEMS, including its loss corrections. 3. May develop (with consultation with EMA) other procedures for codification within Part 1065. 13. Sensors will provide an advance copy of Sensors’ December pres entation, requesting EMA question consolidation ahead of meeting.296 SwRI Project 14936.12 PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio

December 10

- 12, 2008

SwRI presented the steady

- state testing results, and the storage and release and regeneration results. The presentations were posted on the FTP site.

Sensors presented their approach to particle loss correction. A presentation is available at the FTP site.

Craig Kazmierczak presented some in

- use PM

- PEMS work that was done on one of D DC trucks using the PM

- PEMS equipment. The work was done by Sensors and presented by Craig.

The MASC requested that the reference crystal be used during SwRI laboratory activities, if that is to be used on the road.

The testing done by SwRI so far is acc eptable. For the particular PPMD used, disable the reference crystal, and use a working crystal to be a reference. Essentially, operate with six working crystals, and use one for reference.

Sensors intends to use the reference crystal for correction, but they are going to use a logic to decide whether or not it will be used in the post processor.

The post processor should be available before January 23, 2009. The post processor should include any loss correction intended or any reference crystal correctio n model, remove all Part 1065 excursions, and all other miscellaneous items that will make a measurement invalid.297 SwRI Project 14936.12 In the current AVL post processor, thermophoretic loss was capped at 25 %. AVL wants to change that to include the full range of the model, a nd remove the 25 % cap. Both corrected and uncorrected data will be provided in the output of the post processor.

We agreed to use the paired analysis for steady

- state and for model validation.
We agreed to use 5 % COV at a 1 sigma standard deviation for the CVS at 100 microgram filter loading.

The above subject was tabled for further discussion on how to subtract the CVS error contribution. Bill Martin will send his proposal to Bob and Imad

.

How to account for a regeneration in the field:

NTE event > 30 seconds to be valid

1

- Discrete

2

- Triggered by ECM

3

- The regen is defined between two regen flags

4

- If the regen occurs during an NTE event, and the length between two consecutive NTE flag x 2 is shorter than the NTE event, then the regeneration is coun ted

5

- If the regen occurs before an NTE event, and end outside the NTE event, the NTE will count.

Tim French mentioned that the subject of regeneration and its inclusion in NTE need to be discussed in a different forum.298 SwRI Project 14936.12 Blow

- By and how it will be compu ted for the measurement allowance. Next meeting

As of now, the results reported for storage and release and for regeneration will not impact the MA program.

Matt will share the filter results during the next meeting.299 SwRI Project 14936.12 PM Measurement Allowance St eering Committee Meeting
Meeting at SwRI, San Antonio
January 28

- 30, 2009

The last minutes was reviewed and approved. The scheduling for future meetings and conference calls was as follows:

Conference Call, Thursday, Feb. 26, 2009, 9:00 am Central, length will be decided on Friday.

Next MA

- SC, April 1 st (2:00 to 6:00), 2 nd (9:00 to 5:00) and 3 rd (8:00 to 2:00), San Antonio.

Week of the 18 th

**•** Some of the in

- use testing performed b y Ce

- Cert will include regeneration events. No action will be taken on modifying the test plan at SwRI. The test plan will remain the same and it will not include active regeneration. of May for EMTC and MASC (May 20, 21, 22, same as the April meeting)

**•** Bill Martin explained his proposal on the CVS variability addition to S S delta data. Bill Martin presented the analysis on how to assign the CVS variability into the delta between PM

- PEMS and CVS in order to shrink the 95 th and 5 th A copy of his write

up is on the FTP website

**•** SwRI presented work on the progress made. A copy of the presentations is posted on the FTP website

**•** Rey Agama presented an argument about using standard deviation instead of MAD for the data analysis. Matt Spears suggested that the MAD should be used, and if no300 SwRI Project 14936.12 validation was observed at the end of the p rogram, other possibilities can be considered. The group agreed to move forward with this approach. Rey asked that we apply a normality test on the data, and he will consult with Bob Mason on that. This is in relation to applying a MAD or SD for the model. Bob Mason will present some material on the normality criteria during the next meeting.

**•** The contribution of blow

- by will be a constant based on the data presented from the four ACES engines at 0.00042 g/hp

- hr. If the crankcase is vented to the atmosphere
, this value will be added to every NTE emissions value. If the crankcase is closed loop, the blow

- by contribution of 0.00042 g/hp

- hr will not be added to the NTE emission values.

**•** Horiba was allowed to fix a bug in their system in relation to delays between engine and OBS and OBS and TRPM

**•** Horiba was allowed to make modifications on the ion trap voltage of the EAD for the third TRPM unit.

**•** The engine manufacturers agreed to submit the ECM/Lab Torque and Fuel Flow by March 31 st , 2009. The data will no t be linked to a particular manufacturer.

**•** The instrument manufacturers needs to submit the latest post processor by Friday, March 6, 2009

**•** The MASC agreed that the AVL measurement allowance will be performed based on data measured plus thermophoretic corr ection. The MASC agrees to also see data from the AVL MSS total PM prediction on a non

- interference basis from the core of the measurement allowance program.

**•** The MASC was updated on the PM generator via a laboratory tour. The PM generator is currently set up at SwRI Particle Laboratory.

Below is the unedited Matt Spears’ minutes:301 SwRI Project 14936.12 1. January meeting agenda

a. Next meetings? b. EPA participation/management in MASC

c. Review of Meeting minutes d. Regen in NTE discussion

e. Bill Martin’s paired testing proposal f. SwRI data i. Transient ii. 2 nd iii. Lessons learned, problems? set of PEMS

iv. Analysis of MSS with sulfate and HC corrections

v. ECM vs test cell torque and BSFC error surface update 1. SwRI approach g. PM Generator update h. Environmental chamber update i. PEMS mfr readiness ii. SwRI readiness & sc hedule iii. PM Generator readiness i. Next face

- to

- face meeting: April 1 afternoon, to April 3 rd j. May 13

- 15, placeholder

- SwRI afternoon

— SwRI

k. Bypass sizing for model validation work i. DDC engine l. Filter results

m. Friday Schedule i. 9:00am start ii. Review test plan timeline iii. En vironmental chamber update 1. PEMS mfr readiness 2. SwRI readiness & schedule 3. PM Generator readiness iv. AVL data with corrections

v. Horiba general topic vi. Finalize agenda for February 26 th meeting

— finalized, finalize time. 9am central. 4 hrs. 12:00pm CST

– 4:00pm CST (1pm

- 5pm EST) Shirish travelling to NRMM in Ispra Feb 26 th 1. Agenda

a. Make this a LiveMeeting, this is ok w SwRI b. Update on PEMS manufacturer post processors: 1

- hr

c. Timeline updat e

— SwRI: test cell and environmental chamber testing: 30 min d. CE

- CERT update on ability to come to SwRI for correlation testing & progress on bypass: 30 min

e. SwRI data302 SwRI Project 14936.12 i. PEMS set number 2: steady

- state & perhaps transient results

— summary only: anything remarkably different than the 1 st

ii. Any new issues or difficulties: 30 min set of PM PEMS: 1

- hr

f. vii. PM Generator / nanoparticle lab tour viii. Engine manufacturers to submit ECM/test cell torque/fuel rate data by March 31 st ix. Rudy’s for lunch meeting303 SwRI Project 14936.12 PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
April 1

- 3, 2009

PM

- PEMS Meeting: April 1

- 3, 2009.

Wednesday, 2:00

- 6:00 PM

**•** Meeting Minutes from previous meeting were reviewed

**•** Practice measurement allowance on incomplete set of data was presented by Janet, see presentation on website

**•** The viability of using Method 3 was discussed. There was a general agreement that this method needs to be dropped out mainly because the lack of information on time alignment between ECM fuel flow and gas

- based fuel flow, but such decision needs to be reconfirmed during the next meeting.

Thursday, 9:00

- 5:00 PM

**•** The normality test requirement was discussed by Bob Mason, see presentation on website. The decision for now is not to assume normal distribution and use the MAD instead of SD. If the MAD failed to produce a validated measurement allowance, the SD will be revisited.

**•** David Booker from Sensors presented the features of the new PPMD post processor, see presentation on website

**•** Mike Akard from Horiba gav e an update on the status of Horiba’s post processor, see presentation on website

**•** SwRI gave a presentation on the following (see presentation on FTP website):

**•** Status update and project progress

**•** SS data from all the PM

- PEMS

**•** Transient data with and without e ngine drift from all the PM

- PEMS

**•** OC/EC data for SS304 SwRI Project 14936.12 **•** EEPS data for SS

**•** Some MSS corrected data for sulfate and HC

**•** Reference NTE using method 3 gave different values than Method 1 and 2, and decision will need to be made on Method 3 during the next meeting.

Friday, 8:00

- 2:00

**•** Vibration for offroad was discussed. SwRI plans to do a vibration sweep similar to on

highway but while the PEMS sitting at the vibration table at 45 degree angle. SwRI will present a cost estimate for this additional activities when the vibration activities start.

**•** SwRI showed the fuel and torque data submitted by all engine manufacturers that included Caterpillar, Cummins, Detroit Diesel, Navistar, and Volvo Powertrain. SwRI will present all data together during the next meeting.

Some of the decisions/action items made during this meeting were as follows:

**•** A decision on whether or not Method 3 should be used (AVL only) needs to be made

**•** Reference NTEs will be tweaked so the main distribution of events will be between 0.015 g/hp

- hr and 0 .035 g/hp

- hr

**•** SS error surfaces will be presented during the next meeting

**•** Transient data was drift corrected using the CVS, but the MASC desired to look at the integrated AVL data as compared with the CVS to look at the possibility of using real time AVL da ta for drift correction. The integrated AVL for the NTE cycles will be compared with the integrated CVS during the next meeting, before making a final okay on the CVS drift correction method

**•** EPA was to provide some information on temperatures experienced d uring off

- road in

- use activities so it can be incorporated with temperature and humidity profile during environmental testing. EPA was to propose a final temperature and humidity profile for the program, after incorporating the CE

- CERT and off

- road data

**•** Of f

- road vibration tests will be added at 45 degree angle

**•** Fuel and torque errors using the engine manufacturers’ submitted fuel and torque needs to be presented during the next meeting305 SwRI Project 14936.12 PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, San Antonio
May 20

- 22, 2009

PM

- PEMS Meeting, May 20

- 22, Meeting Agenda

1. Review of last meeting minutes 2. Next Meeting Schedule 3. Overall Project Status Update 4. CE

- CERT and SwRI Correlation 5. Update on EMI and RFI Testing 6. Update on Environmental Chamber Testing 7. SS Error Surfaces an d Adjusted Reference NTE events 8. Other agenda items and questions

**•** The last meeting minutes were reviewed

**•** The next meeting schedule was set to July 15

- 17 in Indianapolis

**•** The overall project status we presented. See presentation by SwRI

**•** CE

- CERT and SwRI Correlation was presented. See presentation by SwRI

**•** EMI and RFI testing results were presented. See presentation by SwRI. No error surfaces will be generated. The problems mainly affected instrument functions. i. Horiba will investigate the issue related to Bulk Current Injection effect on the Horiba exhaust flow ii. Sensors will demonstrate a fix to the exhaust flow problem during the next meeting. Sensors will conduct their own testing if necessary.

**•** SS error surface s were presented. Sensors data were not fully analyzed because the new post processor was not provided to SwRI. Sensors promised to provide a new post processor resolving the issues identified by SwRI during the last meeting to increase data yield. Sensors later provided the new post processor, and the SS error surfaces were presented during a conference call that took place on June 29, 2009. See SwRI presentation for the June 29 conference call.306 SwRI Project 14936.12 **•** The transient errors were also discussed, and the approach fo r the transient error surface will be presented during the next meeting in Indianapolis, along with a final recommendation

**•** Sensitivity on fuel flow and CO 2

PM Measurement Allowance Steering Committee Meeting
Meeting at SwRI, Indianapolis
July 14

- 17, 2009 flow for Method 2 and Method 3 will be discussed during the next meeting in Indianapolis

1. Last Minute Review 2. SwRI Presentation

a. Project Update b. SS and Transient Error Surfaces treatment and results

c. Environmental Testing (Atmospheric Pressure, Temper ature and Humidity) Results and Discussion i. Three approaches were presented on how to treat the error surfaces for the environmental testing. ii. Approach 3 was used. The steering committee agreed with using approach 3 iii. No error surfaces were obtained for Horiba and Sensors for environmental testing iv. Error surfaces for AVL, but anything below the lowest MAD, set to a constant equal to the lowest MAD, and anything above the highest MAD set to a constant equals to the highest MAD d. For Method 3, and for the refe rence NTEs, we agreed that the gas based fuel flow will be advanced in order to match the ECM fuel flow. 3. Rules on Measurement Allowance:

a. Pick the positive measurement allowance that is closet to zero based on the Horiba’s and Sensors’ PEMS. b. If both Horiba ’s and Sensors’ PEMS have a negative measurement allowance, pick the one that is closest to zero. 4. Preliminary conference call scheduled for August 20 , 2009. We will confirm it tomorrow. At the August’s conference call a decision on the measurement allowanc e will be made and an instrument will be selected to go to CE

- CERT. After that, SwRI will ship the units to CE

- CERT to arrive at CE

- CERT earlier than September 1 5. Meeting at CE

- CERT on September 22 st nd to observe and check PEMS installation by CE

CERT.307 SwRI Project 14936.12308 SwRI Project 14936.12 PM Measurement Allowance Steering Committee LiveMeeting
August 20, September 11, 2009

1. SwRI presented the Monte Carlo Simulation (See Presented at FTP website) 2. The measurement allowance was determined based on Method 2 using Sensors’ PPMD 3. The measurement allowance was 0.00605 g/hp

- hr 4. Sensors’ PPMD was chosen for CE

- CERT in

- use testing 5. It was agreed by the MASC that the AVL MSS will also participate in in

- use testing along with the PPMD, just like it was done in the laboratory 6. The MASC requested the followi ng from SwRI

a. plot the Sensors and Horiba 95 th b. show the results of the simulation based on reference data available within the concentration range obtained in the laboratory delta on the same plot

c. plot the 95 th , 50 th , and 5 th d. Refreshment on the regression rules for the validation deltas , for the validation

7. Address Item 6 above in a Livemeeting on September 11 8. The requests in Item 6 above were addressed in a livemeeting on September 11 (see presentation on FTP website)

a. The MASC requested that the validation d eltas for the 95 th , 50 th , and 5 th

b. The LOESS fit should be done on Sensors’ validation deltas based on Method 1 and Method 2. be regressed using the LOESS fitting rule since the criteria set for linear regression is not met.

c. SwRI should present the regressi on the CE

- CERT September 22 meeting.309 SwRI Project 14936.12 PM Measurement Allowance Steering Committee Meeting
February 17

- 18, 2010, US EPA, Ann Arbor

Sensors gave a presentation, proposing a correction factor for the exhaust flow.

**•** Sensors calibrated the flow meter in house with the same wrong pressure conf iguration in the field

**•** Sensors determined a correction factor of 1.52 for exhaust flow correction for Unit 3. This correction factor was accepted by the MASC

**•** A similar calibration will be done on Unit 2 to determine the exhaust flow correction factor to be used for this unit

Sensors indicated that additional corrections related to bypass flow that was not corrected for during the in

- use testing is needed:

**•** The sample flow during NTE is determined as the difference bet ween bypass flow before sampling and bypass flow during sampling.

**•** If the bypass flow is not corrected for barometric pressure during sampling, and the barometric changes during the long NTEs, then there would be an error introduced to the sample flow.

AV L discussed the approach they took to calculate the brake

- specific PM emissions using Method 2. The ECM broadcast fuel term in the equation was essentially frozen if it went below 5% of the max fuel flow encountered during the test. However, to move forwar d with Method 2 calculation, the MASC agreed that to the following:

**•** The MAX fuel flow provided by the engine manufacturer should be used not the max fuel determined during a test. Thus, the engine manufacturer of the CE

- CERT vehicle should provide the inf ormation to CE

- CERT to do proper Method 2 calculation

**•** The fuel flow along with the gas concentration terms in the equation will be frozen if the ECM fuel flow dipped below 10% of the MAX fuel flow.

CE

- CERT gave a presentation on the different exhaust flow correction attempts they made. See CE

- CERT presentation at the FTP website.310 SwRI Project 14936.12 SwRI presented validation for PPMD Unit 2 and 3 using Method 1 with an exhaust flow correction of 1.52. The PPMD based on Method 1 failed the validation criteria. SwRI also showe d different scenarios for the AVL, including Method 1, 2, and 3. The presentation is listed on the FTP website.

Below is a summary of the action items for CE

- CERT and SwRI as a results of this meeting. These action items were compiled by Chris Laroo:

Fo llow

- up Work to Finish PM MA Test Program

1) Agreed to exhaust flow correction factor of 1.52 for unit #3.

2) Will use exhaust flow correction factor for unit #2 that comes out of Sensors check of the unit #2 flow meter on their flow stand (value anticipated to be at or near 1.52 but needs to be determined). Value is anticipated to be known the week of March 1 st

.

3) Correct PPMD data for incorrect setting in QCM bypass flow. Software setting change from 0 to 1.

4) Use Semtech DS barometric pressure readings to a ccount for altitude change effects on data. If it is determined that the PPMD barometric pressure reading is faulty, we will request that Sensors upgrade the sensor to the quality used in the DS. This could be deemed a special source of error and account ed for via a future hardware improvement.

5) PPMD Unit #1 data will be reported for single crystal use only to reflect that fact that the Mass Sensitivity was incorrectly entered into the software during testing. This single crystal use data will then be po oled with the multi

- crystal use results from units #2 and #3 to determine the final validation % results.

6) Report PPMD Unit #2 and #3 deltas for single crystal use only as a probing exercise to see if multiple crystal use has an effect on mass loss. The i ntent of this exercise is to gauge the effectiveness of proposed fixes by Sensors to eliminate the PPMD low bias for future pilot and compliance program testing. This data will be pooled with the unit #1311 SwRI Project 14936.12 single crystal use results for plotting in the vali dation window for experimental purposes only. These results will not be used to determine validation.

7) Method #2, if the fuel flow rate drops below 10% of manufacturer declared maximum fuel rate value, then the ratio of the emission concentration terms an d the ECM broadcast fuel rate will be frozen at that value.

8) CE

- CERT will reprocess all PPMD and MSS data with correct factors.

9) CE

- CERT will write a final report to be reviewed by the steering committee before finalization.

10) SwRI will place revised CE

- CER T data in validation windows and calculate new validation percentages. CE

- CERT should have the data to SwRI by the end of March.

11) SwRI will also place revised CE

- CERT single crystal usage data in validation windows and calculate new validation percentages (for experimental purposes only). This again will help gauge whether or not there is an effect on mass loss from multiple crystal usage.

12) SwRI will write a final report to be reviewed by the steering committee before finalization.