# PM

- PEMS Measurement Allowance Determination

Final ReportEPA-420-R-10-902 August 2010 Assessment and Standards Division Office of Transportation and Air Quality U.S. Environmental Protection Agency

and

California Air Resources Board

and

Engine Manufacturers Association

Prepared by Southwest Research Institute SwRI Project 03.14936.12 

# PM

- PEMS Measurement Allowance Determination

Final Report

# PM - PEMS MEASUREMENT ALLOWANCE DETERMINATION

FINAL REPORT

SwRI ® 

### Project 03.14936.12

Prepared for

U.S. Environmental Protection Agency California Air Resources Board Engine Manufacturers Association

June 2010

Prepared by: Approved by:

__________________________________ ________________________________ Imad A. Khalek, Progr am Manager Jeff J. White, Director

DEPARTMENT OF EMISSIONS RESEARCH AND DEVELOPMENT ENGINE, EMISSIONS AND VEHICLE RESEARCH DIVISION

This report shall not be reproduced, except in full, without the written approval of Southwest Research Institute ® Results and discussion given in this report relate only to the test items described in this report. .REPORT 03 .14936. 12 ii FOR EWO RD

Th e P M

- PEMS measurement al lowance pr ogram was performed by t he Department of Emissions R esearch a nd D evelopment unde r Mr. Jeff W hite, Director. Dr. Imad Khalek, Program M anager, was t he P rincipal Investigator and Project M anager

, Mr. T hom as Bougher, Research Engineer, was the Project Leader, and Mr. Daniel Preece, Research Assistant, was the laboratory te chnical a ssistant. D r. Robert Mason, I nstitute Analyst , was t he P rincipal Co

Investigator responsible for statistical analysi s, and Ms. Janet Buckingham, Staff Analyst, was the Project Leader/Statistics. Other SwRI Emissions R&D staff with contribution to the project

were Mr. Michael F eist, Senior R esearch Engineer, Mr. Richard Mechler, Senior R esearch Technologist, Mr. Don ald Parker

, Senior Technician, Mr. Jose Sosa, Principal Technician, Mr. Keith Echtle , Laboratory A ssistant Manager , a nd M r. E rne st Krueger, L aboratory M anag er. Additional SwRI assistance during Environmental Testing was provided by Rick Pit man, Senior Engineering T echnologist, Mr. Mike Negrete, S enior T echnician, Mr. David S mith, Staff Technician, M r. Herbert W alker, S enior E ngineering T echnologist, and M r. E ric D ornes, Principal Engineer.

This w ork w as pe rformed unde r E nvironmental P rotection A gency ( EPA) Work Assignments 2- 7, 3

- 7, 4

- 7, a nd 2

- 12, und er SwRI P roject Numbers 12859.07, 13749.07, 14658.07, a nd 14936.12, r espectively. T he or iginal E PA W ork Assignment Ma nager w as Mr . Matthew Spears. A fter March of 2009, t he E PA W ork A ssignment M anager b ecame M r. Chris topher Laroo. This work started in J une of 2007 a nd will end b y J une 26, 2010. T esting started in June of 2008 and ended in September of 2009.

Fundi ng for t his w ork was pr ovided by U.S E PA

, Engine M anufacturers A ssociation (EMA), a nd the C alifornia A ir R esources B oard ( CARB). Funding b y EMA a nd C ARB w as provided directly to EPA in support of this work.

In -kind engine and technical support were provided b y Volvo Powertrain . In

- kind PM

PEMS and technical support were provided by AVL, Horiba, and Sensors. SwRI acknowledges the following individuals for their laboratory technical and logistical support:

Mr. Jeffrey Saxon and Mr. Steven Trevitz , from Volvo Powertrain

Mr. Scott Porter, Dr. Mi chael Akard and Dr. Qiang Wei, from Horiba

Mr. William Silvis, Mr. Siegfried Roeck, Dr. Wolfgang Schindler, Dr. Roland Wanker, Dr. Michael Arndt, Mr. P.J. Pankratz, and Ms. Sarah Kingham, from AVL

D r. Andrew Reading, Dr. David Booker, Dr. Atul Shah, Mr. Carl Ensfield, Mr. Tim othy Bottomley, and Mr. Kevin Bouma, from SensorsREPORT 03 .14936. 12 iii A m easurement allowance s teering committee ( SC) com posed of E MA, E PA, CARB, and P M

- PEMS m anufacturer m embers m et on a r egular basis t hroughout t he e ntire pr oject t o discuss the progress made and make recommendations. The SC has contributed significantly to this project. SwRI acknowledges the following SC members for their active participation :

Mr. Rey Agama, Caterpillar Dr. Mi chael Akard, Horiba Dr. Dipak Bishnu, CARB Dr. David Booker, Sensors Dr. Bruce Cantrell, EPA Consultant Mr. Tim othy French, EMA Dr. Robert Giannelli, EPA Mr. Denny Hao, PACCAR Dr. Kent Johnson, CE- CERT Mr. Craig Kazmierczak, Detroit Diesel Mr. John Kegebein, John Deere Mr. T hom as Kramer, Navistar Mr. Chris topher Laroo, EPA Mr. Hector Maldonado, CARB Mr. William Martin, Cummins Dr. Shirish Shimpi, Cummins Mr. William Silvis, AVL Ms. Carol Smith, Isuzu Mr. Matt hew Spears, EPA Mr. Steven Trevitz, Volvo Powertrain Dr. Qiang Wei, HoribaREPORT 03 .14936. 12 iv 

### TABLE OF CONTENTS

Page

FOREWORD

.................................................................................................................................. ii

LIST OF FIGURES .................................................................................................................... vii

LIST OF TABLES ....................................................................................................................... xvi

ACRONYMS

............................................................................................................................... xix

EXECUTIVE SUMMARY ......................................................................................................... xxi

1.0 INTRODUCTION ..............................................................................................................1

2.0 MODELING APPROACH ..................................................................................................3 2.1 Purpose of Model .................................................................................................................3 2.2 Model Improvement

................................ ................................ .............................................3 2.3 Monte Carlo Simulation Approach ......................................................................................4 2.4 Calculation Methods ............................................................................................................4 2.4.1 Calculation Method 1 – "Exhaust Flow-Torque-Speed" Method ........................................5 2.4.2 Calculation Method 2 – "Exhaust and Fuel Flow-Torque-Speed" Method .........................6 2.4.3 Calculation Method 3 – "Fuel Flow-Torque-Speed" Method..............................................7 2.5 Reference NTE Events .........................................................................................................8 2.6 Error Surface Generation ...................................................................................................12 2.6.1 PEMS vs. Laboratory Nominal Results .............................................................................12 2.6.2 (PEMS -Laboratory) Deltas vs. Lab ...................................................................................14 2.6.3 Variability Index vs. (PEMS-Laboratory) Deltas and Lab Nominal .................................15 2.7 Error Surface Sampling and Interpolation .........................................................................18 2.8 Brake

- Specific Emissions Calculations .............................................................................20 2.9 Convergence and Number of Trials ...................................................................................22 2.10 Simulation Output ..............................................................................................................23 2.11 Step

- by

- Step Simulation Example .....................................................................................24 2.12 Measurement Allowance Generation .................................................................................27 2.12.1 Regression Method

................................ ................................ .............................................27 2.12.2 Median Method ..................................................................................................................27 2.13 Model Validation ...............................................................................................................28

3.0 PART 1065 PEMS AND LABORATORY AUDIT .........................................................32 3.1 1065 Lab Audit ..................................................................................................................32 3.1 1065 PEMS Audit ..............................................................................................................34 3.2.1 Horiba Flow Audits

................................ ................................ ............................................34 3.2.2 Exhaust Flow .....................................................................................................................37REPORT 03 .14936. 12 v 

### TABLE OF CONTENTS (CONT'D)

Page

4.0 E NGINE DYNAMOMETER TESTING AND RESULTS ..............................................40 4.1 Testing Objective ...............................................................................................................40 4.2 Experimental Setup ............................................................................................................40 4.2.1 Engine and Sampling System .............................................................................................40 4.2.2 Sensors PPMD ...................................................................................................................43 4.2.3 AVL MSS ............................................................................................................................45 4.2.4 Horiba TRPM

................................ ................................ .....................................................46 4.3 Bypass Mixing Verification ...............................................................................................47 4.4 PEMS Loss Corrections .....................................................................................................50 4.4.1 Sensors PPMD Loss Correction ........................................................................................50 4.4.2 AVL MSS Loss Correction .................................................................................................50 4.4.3 AVL MSS Total PM Correction .........................................................................................51 4.5 Steady

- State Testing Procedure .........................................................................................51 4.6 Data Yield During Steady

- State Testing ............................................................................58 4.6.1 Data Yield During Steady- State Testing ............................................................................59 4.7 Accounting for CVS Variability During Steady

- State Testing .........................................62 4.8 Steady

- State Testing Results ..............................................................................................62 4.8.1 Comparison Between PEMS and Lab Delta PM ...............................................................63 4.8.2 Correlation Between PEMS and Lab PM ..........................................................................70 4.8.3 Steady-State PM Error Surfaces ........................................................................................73 4.9 Transient Engine Results ...................................................................................................76 4.10 CE -CERT Mobile Lab Correlation ....................................................................................86 4.11 Invest igation of DPF Regeneration ....................................................................................89 4.12 Investigation of Storage and Release .................................................................................91 4.13 Engine Manufacturers Torque and Fuel Error Surfaces ....................................................93

5.0 ENVIRONMENTAL TESTING AND RESULTS ...........................................................96 5.1 Reference Measurement Testing

................................ ........................................................96 5.2 Pressure Chamber Testing ...............................................................................................102 5.3 Temp erature and Humidity Chamber Testing .................................................................109 5.4 Electromagnetic and Radio Frequency Interference Screening .......................................116 5.4.1 Bulk Current Injection .....................................................................................................117 5.4.2 Radiated Immunity ...........................................................................................................122 5.4.3 Electrostatic Discharge ...................................................................................................125 5.4.4 Conducted Transients ......................................................................................................126 5.5 Vibration Testing

................................ ................................ .............................................128

6.0 MODELNG RESULTS ...................................................................................................137 6.1 Convergence Results from MC Runs ..............................................................................137 6.2 Sensitivity Based on Bias and Variance ..........................................................................142 6.3 Validation Results ............................................................................................................150 6.4 Measurement Error Allowance Results ...........................................................................160REPORT 03 .14936. 12 vi 

### TABLE OF CONTENTS (CONT'D)

Page

7.0 SUMMARY .....................................................................................................................173

8.0 REFERENCES ................................................................................................................174

APPENDICES Page Count

A TEST PLAN TO DETERMINE PEMS MEASUREMENT ALLOWANCE FOR THE PM EMISSIONS REGULATED UDNER THE MANUFACFTURER

- RUN HEAVY

- DUTY DIESEL ENGINE IN

- USE TESTING PROGRAM ..............................46

B STEERING COMMITTEE MEETING MINUTES ..........................................................33

C CRYSTAL BALL OUTPUT FILE DESCRIPTIONS .......................................................6

D MONTE CARLO SPREADSHEET COMPUTATIONS .................................................15

E PEMS OPERATION LOG ................................................................................................11REPORT 03 .14936. 12 vii 

### LIST OF FIGURES

Figure Page

1 Method 1 I deal BSPM Values for Reference NTE Events ..................................................9

2 Error Surface Construction: PEMS vs. Laboratory Results..............................................14

3 Error Surface Construction: (PEMS - Lab) vs. Laboratory Results .................................15

4 Error Surface Construction: Error at Variability Index for 5th and 95th percentiles vs. Laboratory Results .......................................................................................................16

5 Truncated Normal Distribution Percent iles .......................................................................17

6 Error Surface Construction: Error a t Variability Index for 1st and 99th Percentiles vs. Laboratory Results .......................................................................................................18

7 Truncated Standard Normal at 1st and 99th Percentiles and Uniform Probability Density Functions ..............................................................................................................19

8 Steady

- State P M Error Surface for A VL With Example Sampling f or a Reference N TE Event .........................................................................................................................20

9 Overview of Monte Carlo Simulation for B SPM ..............................................................24

10 Error Surfaces Included in Monte Carlo Simulation .........................................................26

11 Linear Regression Fit To 5th and 95th Percentile Deltas ..................................................30

12 Loess Regression Fit To 5th and 95th Percentile Deltas ...................................................31

13 Linearity Check on P EMS -1 Exhaust Flow During Steady- State Engine Testing ............38

14 Linearity Check on P EMS -2 Exhaust Flow During Steady- State Engine Testing ............38

15 Linearity Check on P EMS -3 Exhaust Flow During Steady- State Engine Testing ............39

16 Volvo MP 7 Installed in a CVS Test Cell ...........................................................................40

17 D PF Bypass With D OC .....................................................................................................41

18 Schematic of Engine Dynamometer Experimental Setup ..................................................42

19 P PMD Installed in the Test Cell ........................................................................................44REPORT 03 .14936. 12 viii 

### LIST OF FIGURES (CONT'D)

Figure Page

20 The AVL M SS ...................................................................................................................45

21 The Probe and T PA f or the Horiba O BS -T RPM ...............................................................46

22 The Horiba O BS -TRPM and O BS -2200 ...........................................................................47

23 Experimental Setup for Mixing Verification .....................................................................48

24 Mixing Verification Sample Locations ..............................................................................48

25 Mixing Verification Results ...............................................................................................49

26 Speed and Load for the 80 Points Cycle ............................................................................52

27 Final Six Steady

- State Modes ............................................................................................53

28 Example of Steady

- State Cycle .........................................................................................55

29 Steady

- State Sample Times ...............................................................................................56

30 C VS Filter Weight Gain for Steady

- State Testing .............................................................56

31 Steady

- State Exhaust PM Concentration (μg/Mol) ...........................................................57

32 Steady

- State Brake

- Specific P M, CVS Filter (Mg/Hp

- Hr) ...............................................58

33 Number of Valid Data Points for Steady

- State Testing .....................................................59

34 Horiba Sample Flow to the Filter and Dilution Flow While Compressor Stops ...............61

35 Sensors E FM Zero During Steady

- State Cycle .................................................................61

36 Horiba- 1 Steady

- State PM Concentration Deltas ..............................................................64

37 Sensors- 1 Steady

- State PM Concentration Deltas .............................................................64

38 A VL PM Concentration Deltas for Steady

- State Testing on PEMS 1 ..............................65

39 Horiba- 2 Steady

- State PM Concentration Deltas ..............................................................65

40 Sensors- 2 Steady

- State PM Concentration Deltas .............................................................66REPORT 03 .14936. 12 ix 

### LIST OF FIGURES (CONT'D)

Figure Page

41 Avl

- 2 Steady

- State PM Concentration Deltas ...................................................................66

42 Horiba- 3 Steady

- State PM Concentration Deltas ..............................................................67

43 Sensors- 3 Steady

- State PM Concentration Deltas .............................................................67

44 A VL

- 3 Steady

- State PM Concentration Deltas .................................................................68

45 Steady

- State Concentration Deltas for Horiba ...................................................................69

46 Steady

- State Concentration Deltas for Sensors .................................................................69

47 Steady

- State Concentration Deltas for Avl ........................................................................70

48 Steady

- State Horiba PEMS PM Concentration Versus the Laboratory Reference ...........71

49 Steady

- State Sensors PEMS PM Concentration Versus the Laboratory Reference ..........71

50 Steady

- State AVL PEMS PM Concentration Versus the Laboratory Reference ..............72

51 Linear Regression Correlation Between PEMS and Lab ...................................................72

52 Steady

- State Concentration Deltas for Horiba ...................................................................73

53 Steady

- State Concentration Deltas for Sensors .................................................................74

54 Steady

- State Concentration Deltas for A VL

................................ ......................................74

55 Final Steady

- State PM Error Surface – Horiba..................................................................75

56 Final Steady

- State PM Error Surface – Sensors ................................................................75

57 Final Steady

- State PM Error Surface – A VL ....................................................................76

58 Repeat Engine Speed Traces for N TE Transient Cycle .....................................................77

59 Repeat Engine Torque Traces for N TE Transient Cycle ...................................................78

60 R epeat AVL Soot Concentration Traces for N TE Transient Cycle ...................................79

61 A VL Soot Concentration During N TE Transient Cycles, Events 20-23 ...........................79REPORT 03 .14936. 12 x 

### LIST OF FIGURES (CONT'D)

Figure Page

62 Comparison of AVL and Horiba Real Time Signals During Transient Cycle ..................80

63 C VS BSPM and Correction Factor for Transient Cycle ....................................................81

64 Horiba Concentration Deltas for Transient Engine Testing ..............................................82

65 Sensors Concentration Deltas for Transient Engine Testing .............................................82

66 A VL Concentration Deltas for Transient Engine Testing .................................................83

67 Final Horiba Transient PM Error Surface ..........................................................................84

68 Final Sensors Transient PM Error Surface ........................................................................85

69 Final AVL Transient PM Error Surface .............................................................................85

70 Exhaust Configuration for CE

- CERT Correlation .............................................................86

71 Brake

- Specific PM Results From CE

- CERT Correlation .................................................88

72 C VS Filter Weight Gain During Tunnel Blanks ................................................................88

73 Brake Specific PM Emissions During Active Regeneration .............................................90

74 Total Exhaust Number Concentration During Storage and Release ..................................92

75 Brake

- Specific PM Emissions During Storage and Release Investigation ........................92

76 O EM Supplied Torque Errors ............................................................................................94

77 O EM Supplied Fuel Flow Errors .......................................................................................94

78 Target Dilution Ratio and PM Level Profile for Environmental Testing ..........................97

79 Experimental Setup for Environmental Testing ................................................................98

80 Horiba Environmental Baseline Measurements

................................ .................................99

81 Sensors Environmental Baseline Measureme nts ...............................................................99

82 A VL Environmental Baseline Measurements .................................................................100REPORT 03 .14936. 12 xi 

### LIST OF FIGURES (CONT'D)

Figure Page

83 Reference M SS Environmental Baseline Measurements ................................................100

84 Reference AVL Versus PEMS AVL for Environmental Baseline ..................................101

85 Original and Revised Profile for Altitude Testing ...........................................................102

86 Altitude Testing Chamber ................................................................................................103

87 P EMS Installed in the Altitude Chamber.........................................................................104

88 Horiba Environmental Pressure Measurements ...............................................................104

89 Horiba Total Flow and Dilution Ratio During Pressure Testing .....................................105

90 Sensors Environmental Pressure Measurements .............................................................106

91 A VL Environmental Pressure Measurements ..................................................................106

92 A VL Pressure Median Versus M AD ...............................................................................107

93 Horiba Pressure Median Versus M AD ............................................................................107

94 Sensors Pressure Median Versus M AD ...........................................................................108

95 Final Error Surface for Environmental Pressure AVL PM Concentration ......................109

96 Temperature and Humidity Profile for Environmental Testing .......................................110

97 P EMS Installed in the Temperature and Humidity Chamber ..........................................110

98 The PM Generator Installed Outside the Temperature and Humidity Chamber .............111

99 Horiba Envir onmental Temperature Measurements ........................................................112

100 Horiba Temperature during Environmental Temperature Testing ..................................112

101 Sensors Environmental Temperature Measurements .......................................................113

102 A VL Environmental Temperature Measurements ...........................................................113

103 Horiba Temperature Median Versus M AD .....................................................................114REPORT 03 .14936. 12 xii 

### LIST OF FIGURES (CONT'D)

Figure Page

104 Sensors Temperature Median Versus M AD ....................................................................115

105 Horiba Temperature and Humidity Median Versus M AD ..............................................115

106 Final Error Surface Environmental Temperature and Humidity AVL PM .....................116

107 AVL PEMS in the Radiation Chamber for E MI and R FI Testing ...................................117

108 Bulk Current Injection Probe ...........................................................................................118

109 Horiba PEMS Setup for Bulk Current Injection ..............................................................118

110 Sensors Setup During Bulk Current Injection Testing ....................................................119

111 A VL Setup During Bulk Current Injection Testing .........................................................119

112 Horiba Flow Disturbance From Bulk Current Injection ..................................................120

113 Horiba Exhaust Flow Noise on Analog Cable During Bulk Current Injection ...............121

114 B CI Noise on AVL Analog Output Cable .......................................................................122

115 Horiba PEMS Setup During Radiated Immunity Testing................................................123

116 Sensors PEMS Setup During Radiated Immunity Testing ..............................................124

117 A VL PEMS Setup During Radiated Immunity Testing ..................................................124

118 Horiba Dilution Ratio Fluctuations During Radiated Immunity .....................................125

119 Electrostatic Discharge Simulator

................................ ....................................................126

120 Sensors Setup During Conducted Transients Testing ......................................................127

121 A VL Setup During Conducted Transients Testing ..........................................................127

122 Sensors PEMS Mounted for Transverse Horizontal Vibration........................................128

123 Sensors PEMS Mounted for Longitudinal Horizontal Vibration ....................................129

124 Sensors PEMS Mounted for Transverse 45° Vibration ...................................................129REPORT 03 .14936. 12 xiii 

### LIST OF FIGURES (CONT'D)

Figure Page

125 Sensors PEMS Mounted for Longitudinal 45° Vibration ................................................130

126 Sensors PEMS Mounted for Vertical Vibration ..............................................................130

127 Sensors PEMS Mounted for 45° Vertical Vibration ........................................................131

128 Power Spectral Density for Vertical Vibration Testing ...................................................132

129 Power Spectral Density for Horizontal Vibration Testing ...............................................132

130 Horiba PEMS Vibration Positions ...................................................................................133

131 Sensors Total Flow During Vibration Testing .................................................................134

132 A VL PEMS Vibration Positions ......................................................................................135

133 A VL Soot Measurement Noise During Vibration Testing ..............................................135

134 Convergence for AVL Method 1 As a Percent of BSPM Threshold ...............................138

135 Convergence for AVL Method 2 As a Percent of BSPM Threshold ...............................138

136 Convergence for AVL Method 3 As a Percent of BSPM Threshold ...............................139

137 Convergence for Horiba Method 1 As a Percent of BSPM Threshold ............................139

138 Convergence for Horiba Method 2 As a Percent of BSPM Threshold ............................140

139 Convergence for Sensors Method 1 As a Percent of BSPM Threshold ..........................141

140 Convergence for Sensors Method 2 As a Percent of BSPM Threshold ..........................141

141 Box Plot of Error Surface Sensitivity Based on Bias and Variance for A VL B SPM Method 1 ..............................................................................................................146

142 Box Plot of Error Surface Sensitivity Based on Bias and Variance for AVL Method 2 .................................................................................................................146

143 Box Plot of Error Surface Sensitivity Based on Bias and Variance for AVL Method 3 .................................................................................................................147REPORT 03 .14936. 12 xiv 

### LIST OF FIGURES (CONT'D)

Figure Page

144 Box Plot of Error Surface Sensitivity Based on Bias and Variance for Horiba B SPM Method 1 ..............................................................................................................147

145 Box Plot of Error Surface Sensitivity Based on Bias and Variance for Horiba B SPM Method 2 ..............................................................................................................148

146 Box Plot of Error Surface Sensitivity Based on Bias and Variance for Sensors BSPM Method 1 ..............................................................................................................148

147 Box Plot of Error Surface Sensitivity Based on Bias and Variance for Sensors B SPM Method 2 ..............................................................................................................149

148 Validation Percentiles for the 141 Reference NTE Events for AVL Method 1 ..............151

149 Validation Percentiles for the 141 Reference NTE Events for AVL Method 2 ..............152

150 Validation Percentiles for 141 Reference NTE Events for AVL Method 3 ....................152

151 Validation Percentiles for 141 Reference NTE Events for Horiba Method 1 .................153

152 Validation Percentiles for 141 Reference NTE Events for Horiba Method 2 .................153

153 Validation Percentiles for 141 Reference NTE Events f or Sensors Method 1 ................154

154 Validation Percentiles for 141 Reference NTE Events for Sensors Method 2 ................154

155 Validation 95th Percentile BSPM Deltas Loess Fit for Sensors Method 1 .....................155

156 Validation 5th Percentile BSPM Deltas Loess Fit for Sensors Method 1 .......................156

157 Validation 95th Percentile BSPM Deltas Loess Fit for Sensors Method 2 .....................156

158 Validation 5th Percentile BSPM Deltas Loess Fit for Sensors Method 2 .......................157

159 Validation 95th Percentile BSPM Deltas Loess Fit for AVL Method 1 ..........................157

160 V alidation 5th Percentile BSPM Deltas Loess Fit for AVL Method 1............................158

161 Validation 95th Percentile BSPM Deltas Loess Fit for AVL Method 2 ..........................158

162 Validation 5th Percentile BSPM Deltas Loess Fit for AVL Method 2............................159REPORT 03 .14936. 12 xv 

### LIST OF FIGURES

Figure Page

163 Validation 95th Percentile BSPM Deltas Loess Fit for AVL Method 3 ..........................159

164 Validation 5th Percentile BSPM Deltas Loess Fit for AVL Method 3............................160

165 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for AVL Method 1 .................................................................................................................162

166 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for AVL Method 2 ..........................................................................................................................162

167 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for A VL Method 3 ..........................................................................................................................163

168 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for Horiba Meth od 1 ..........................................................................................................................164

169 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for Horiba Method 2 ..........................................................................................................................165

170 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for Sensors Method 1 ..........................................................................................................................166

171 Regression Plot of 95th Percentile Delta BSPM Versus Ideal BSPM for Sensors Method 2 ..........................................................................................................................167

172 Validation On

- Road and Regression Functions Based on the Simulation Model for B SPM Sensors Method 1 With No Regen .......................................................................170

173 Validation On

- Road and Regression Functions Based on the Simulation Model for B SPM Sensors Method 2 With No Regen .......................................................................170

174 Validation On

- Road and Regression Functions Based on the Simulation Model for B SPM AVL Method 1 With No Regen ...........................................................................171

175 Validation On

- Road and Regression Functions Based on the Simulation Model for B SPM AV L Method 2 With No Regen ...........................................................................171

176 V alidation On

- Road and Regression Functions Based on the Simulation Model for B SPM AVL Method 3 With No Regen .......................................................................... 172REPORT 03 .14936. 12 xvi 

### LIST OF TABLES

Table Page

1 PM

- PEMS Used along w ith Serial Number ........................................................................2

2 Reference NTE Events and I deal BSPM Emissions ..........................................................10

3 Descriptive Statistics for BSPM Emissions for 141 Reference NTE Events ....................10

4 Input Parameters For Reference N TE Events ....................................................................11

5 Error Surfaces for Monte Carlo Simulation .......................................................................13

6 Error Surfaces Used for Computing Brake- Specific PM Emissions by Three Calculation Methods ..........................................................................................................22

7 Example of Selection of Measurement Allowance at 0.02 g/hp-hr NTE Threshold for the AVL PEMS ..................................................................................................................28

8 Linearity Verification Results for Intake Air Flwo and Fuel Flow ...................................32

9 CVS Propane Recovery Check Summar y ..........................................................................33

10 Linearity Verification for PM Balance ..............................................................................33

11 Summary of Part 1065 Audits............................................................................................34

12 Linearity Verifications for Horiba PEMS ..........................................................................35

13 Linearity Verifications for Sensors PEMS ........................................................................36

14 Linearity Verific Ations for AVL PEMS ...........................................................................37

15 List of DPF Bypass Configurations ...................................................................................42

16 Sample Order for Steady

- State Cycle Testing ...................................................................54

17 Data Yield by Each PM

- PEMS

................................ ..........................................................60

18 Test Procedure for CE

- CERT Correlation .........................................................................87REPORT 03 .14936. 12 xvii 

### LIST OF TABLES (CONT'D)

Table Page

19 PM Emissions Results from Active DPF Regeneration ....................................................89

20 Average Brake

- Specific Emissions During Storage and Release Cycle ...........................93

21 OEM Error Surface Deltas for Torque and Fuel Flow ......................................................95

22 Summary of Number of Reference NTEs Meeting 2% Convergence .............................141

23 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for AVL BSPM Method 1 .....................................................................................................143

24 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for
AVL BSPM Method 2 .....................................................................................................143

25 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for
AVL BSPM Me thod 3 .....................................................................................................143

26 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for HORIBA BSPM Method 1 ..............................................................................................144

27 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for HORIBA BSPM Method 2 ..............................................................................................144

28 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for SENSORS BSPM Method 1 ............................................................................................145

29 Error Surface Sensitivity to Bias and Variance for 141 Reference NTE Events for SENSORS BSPM Method 2 ............................................................................................145

30 Summary of Error Surface Sensitivities to B ias and Variance for BSPM Method 1 ......149

31 Summary of Error Surface Sensitive to Bias and Variance for BSPM Method 2 ..........150

32 Summary of Error Surface Sensitive to Bias and Variance for BSPM Method 3 ...........150

33 Loess Smoothing P arameters for V alidation P ercentiles .................................................155REPORT 03 .14936. 12 xviii 

### LIST OF TABLES (CONT'D)

Table Page

34 Measurement Error at Threshold for BSPM Using Regression and Median Methods for AVL Method 1 ...........................................................................................................161

35 Measurement Error at Threshold for BSPM Using Regression and Median Methods for AVL Method 2 ...........................................................................................................162

36 Measurement Error at Threshold for BSPM Using Regression and Median Methods for AVL Method 3 ...........................................................................................................163

37 Measurement Error at Threshold for BSPM Using Regression and Median Methods for HORIBA Method 1 ..........................................................................................................164

38 Measurement Error at Threshold for BSPM Using Regression and Median Methods for HORIBA Method 2 ....................................................................................................165

39 Measurement Error at Threshold for BSPM Using Regression and Median Methods for SENSORS Method 1........................................................................................................166

40 Measurement Error at Threshold for BSPM Using Regression and Median Methods for SENSORS Method 2........................................................................................................167

41 BSPM Measurement Error in Percent of NTE Threshold by PEMS and Calculation Method .............................................................................................................................168

42 Measurement Allowance at NTE Threshold by Emissions for Method 2 .......................169

43 Summary of BSPM Model V alidation R esults ................................................................172REPORT 03 .14936. 12 xix ACRONYMS

ACES Advanced Collaborative Emissions Study BCI Bulk Current Injection BS Brake

- Specific BSFC Brake -Specific Fuel Consumption BS PM Brake

- specific Particulate Matter CARB California Air Resources Board CE

- CERT Bourns College of Engineering Center for Environmental Research & Technology CF Correction Factor CFR Code of Federal Regulations COV Coefficient of Variation CVS Constant Volume Sampling DCS Diffusion Charge Sensor DOC Diesel Oxidation Catalyst DPF Diesel Particulate Filter DR Dilution Ratio EAD Electrical Aerosol Detector (TSI, Inc.) EATS Exhaust after

- treatment system ECM Engine Control Module EE Electrical Enclosure EEPS Engine Exhaust Particle Sizer (TSI, Inc.) EFM Electronic Flow Meter EGR Exhaust Gas Recircu lation EMA Engine Manufacturers Association EMI Electromagnetic Interference EPA Environmental Protection Agency ESD Electrostatic Discharge HDIUT Heavy

- Duty In -U se Testing HE Heated Enclosure HEPA High Efficiency Particulate Air LFE Laminar Flow Element MA M easurement A llowance MAD Median Absolute Deviation ME Mechanical Enclosure MEL Mobile Emissions Laboratory MSS Micro Soot Sensor (AVL) NMHC Non-M ethane Hydrocarbons NO X The Oxides of Nitrogen ( NO + NO 2 NTE Not

- to

- exceed )\

OBS On -board systems OC/EC Organic Carbon/Elemental Carbon PEMS Portable E mission M easurement System PM Particulate Matter PPMD Portable Particulate Measurement Device (Sensors, Inc.)REPORT 03 .14936. 12 xx ACRONYMS (CONT'D)

PSD Power Spectral Density QCM Quartz Crystal Microbalance RFI Radio Frequency Interference RMS Root Mean Square SAE Society of Automotive Engineers SS Steady State SwRI Southwest Research Institute TPA Tail P ipe Adaptor TRPM Transient Response Particulate Matter ULSD Ultra -Low Sulfur Diesel VGT Variable Geometry TurbochargerREPORT 03 .14936. 12 xxi EXECUTIVE SUMMARY

The U nited S tates E nvironmental P rotection A gency ( EPA), E ngine M anufacturers Association ( EMA) a nd C alifornia A ir R esources B oard ( CARB) agreed t o pur sue an experimenta l da ta dr iven program to establish a me asurement a llowance ( MA) f or in -use particulate matter (PM) testing using PM portable emissions measurement systems (PM

- PEMS). The M A i s a br ake

- specific P M e missions e rror a ssociated with using in

- use P M

- PEMS equipme nt compared to the laboratory reference filter method. If the MA error is a positive value above zero, after the EPA rounding method, as described in the Code of Federal Title 40, Part 1065 [1], it will increase the EPA in

- use not

- to

- exceed (NTE) standard by the rounded value. If the error is negative or zero, it will not contribute to any changes to the in-use standard.

The measurement allowance steering committee (SC) accepted the following PM

- PEMS to be part of the MA program:

• Sensors Portable Particulate Measuring Device (PPMD). This is a PM

- PEMS that uses proportional dilution and a series of 8 quartz crystal microbalances to measure total (solid plus volatile) PM. The total PM is measured as a single flow

- weighted value for a n NTE
event.

• Ho riba Transient Particulate Matter (TRPM). This is a PM

- PEMS that uses proportional dilution, a real time electrical aerosol detector, and an integrated filter sample to report total PM. The total PM is measured as a single flow

- weighted value for a NTE eve nt. This instrument can report real time total PM, but was not used as such on this program.

• AVL mic ro

- soot s ensor (MSS). This i s a n i nstrument t hat us es c onstant di lution a nd a photo acoustic detector to measure soot or the elemental carbon portion of P M. Soot is measured in real time during a n NTE event.

The SC agreed that only the PPMD by Sensors and TRPM by Horiba w ould be used for the official determination and validation of the measurement allowance generated because both are designated as comp lete PEMS and both measure total (solid plus volatile) PM, as required by US EPA to be valid PM

- PEMS. The SC also agreed that only the PM

- PEMS that produces the lowest positive 95th percentile measurement allowance, based on Sensors or Horiba only, w ould b e chosen for in

- use validation by CE

- CERT due to funding limitation. The third instrument, the MSS b y A VL, was not a com plete P M

- PEMS a nd w as a lways us ed i n c onjunction w ith t he PPMD on t his pr ogram, ba sed on a n a greement r eached be tween S ensors a nd A VL. T he S C agreed that the MSS w ould not be considered as an option for official measurement allowance determination, unless both the PPMD and the TRPM failed validation.

The PM

- PEMS -MA project included four main elements:

• Laboratory steady

- state (SS) and transient engine NTE testing using PM

- PEMS and CVS filter m easurement dur ing S S t esting onl y. T he S S testing was us ed t o c apture bi as, compared to the CVS, and the transient was used to capture precision only since there is no reference method to measure PM during a short NTE event. A 2007 heavy

- duty diesel Mack M P7, from Volvo P owertrain, w as us ed t o c onduct t he e ngine e xperiments. T heREPORT 03 .14936. 12 xxii engine w as c onfigured w ith a b ypass a round t he di esel pa rticle f ilter ( DPF) t o pr ovide PM concentration levels similar to those expected at the NTE threshold limits between 0.02 g/hp- hr and 0.03 g/hp -hr.

• Environmental t esting s uch a s t he e ffect of s hock a nd vi bration, pr essure, t emperature and relative humidity, and electric noise on PM-PEMS precision.

• Monte C arlo simulat ion t o de termine a br ake

- specific m easurement al lowance va lue using error surfaces generated from the PM

- PEMS laboratory and environmental testing, and from error surfaces generated during the gaseous PEMS program

○ A total of 141 Reference NTEs were pro vided by EMA as an input to the model for calculating ideal brake specific emissions, prior to the Monte Carlo simulation

• Model validation using data generated from in-use PM- PEMS testing by CE

- CERT.

Three methods are used to determine in- use NTE brake- specific PM emissions:

• Method 1 f( PM � � � � � , torque i , speed i , exhaust- flow i

• Method 2 f( PM � � � � � , torque )

i , speed i , exhaust- flow i , G- flow i , fuel ECM i

• Method 3 f(PM )

i , torque i , speed i , and fuel ECM i , G- flow i )

Where PM � � � � � is a f low

- weighted PM m easurement, i is i nstantaneous, E CM i s engine

con trol module, and G

- flow is gas

- based fuel flow. All methods require ECM broadcasted torque and speed. In addit ion, Method 1 r equires measured exhaust flow but not fuel flow; Method 2 requires measured exhaust flow, ECM broadcasted fuel flow, and G

- flow ; Method 3 is similar to Method 2, but it does not require measured exhaust flow. Besides real time PM measurement, all methods can use a single flow

- weighted PM measurement for a n NTE event , except Method 3, where real tim e PM measurement is required. Thus, Method 1 and Method 2 were applied to all three PM -PEMS, but Method 3 was only applied to the AVL MSS.

Compared to the Horiba TRPM, the Sensors PPMD, as shown in Table ES -1 , produced the low est pos itive 95t h pe rcentile measurement a llowance of 0.006 05 g/hp

- hr for a n NTE threshold l evel of 0.02 g/hp

- hr, us ing c alculation M ethod 2. T he H oriba T RPM pr oduced a measurement allowance value of 0.0100 g/hp

- hr. Thus, the PPMD was selected by the SC for in

use te sting b y C E

- CERT for M onte C arlo m odel va lidation. A lthough not a ccepted a s a P M

PEMS, the AVL MSS produced the lowest measurement allowance of essentially zero. The SC agreed to include the AVL MSS during in

- use validation because it was used in conjunction with the PPMD during the laboratory portion of the testing.REPORT 03 .14936. 12 xxiii TABLE ES -1. MEASRUEMENT ALLOWANCE BASED ON 0.02 G/HP

- HR THRESHOLD

Measurement Errors at NTE Threshold, g/hp

- hr
PEMS Method 1 Method 2 Method 3

AVL 0.0001 -0.0005 -0.0005
Horiba 0.0109 0.0100 n/a
Sensors 0.0069 0.0061 n/a

Figures ES

- 1 a nd E S

- 2 show e xamples of va lidation pl ots f or S ensors P PMD us ing Method 1 and AVL MSS using Method 3. The dots on each of the two figures represent the in

use delta BSPM between PEMS and CE

- CERT 47 mm filter measurements on the y

- axis versus CE

- CERT BSPM on the x

- axis (Ideal BSPM determined by the filter measurement). Similarly, the lines represents the 5th and 95th percentile errors produced by Monte

- Carlo simulation based on laboratory testing versus a r eference NTE BSPM value on the x

- axis (Ideal BSPM using 47
mm measurements ). To pass m odel va lidation, ≤ 10% of t he i n

- use N TE delta br ake

- specific PM (BSPM) between the PEMS and the CE

- CERT (dots on F igures ES 1 and ES 2) must reside outside the 95th percentile and 5th percentile lines.

The Sensors PPMD failed validation using Method 1 a nd Method 2, a s shown in Table ES -2 . For Method 1, 32 percent of the data were below the 5th percentile, and for Method 2, 34
perc ent of the data were below the 5th percentile. The failure was mainly due to negative bias. The e xact c ause f or ne gative bi as i s unknow n, and f unding l imitation di d not pe rmit f urther investigations to resolve this issue under the scope of the MA program.

The A VL M SS pa ssed va lidation us ing M ethod 3, a s s hown i n T able ES -3 , w here 9.89 percent of the data were outside the 5th and 95th percentiles, with the majority of the failed points being above the 95th percentile. Method 2 failed validation by one percentage point, and Method 1 failed validation by having 18 percent of the data outside the 5th and 95th percentiles, with the majority of failed points being above the 95th percentile.REPORT 03 .14936. 12 xxiv FIGURE ES -1. IN

- USE VALIDATION PLOT FOR SENSORS PPMD USING METHOD 1. (Y AXIS IS THE DIFFERENCE BETWEEN PEMS AND IDEAL BSPM. IDEAL BSPM

IS THE LABORATORY CVS BSPM FOR THE 5TH AND 95TH AND THE CE

- CERT BSPM FOR THE DOTS)

FIGURE ES -2. IN

- USE VALIDATION PLOT FOR AVL MSS USING METHOD 3. (Y AXIS IS THE DIFFERENCE BETWEEN PEMS AND IDEAL BSPM. IDEAL BSPM IS

THE LABORATORY CVS BSPM FOR THE 5TH AND 95TH AND THE CE

- CERT BSPM FOR THE DOTS)REPORT 03 .14936. 12 xxv TABLE ES -2 . SENSORS PPMD VALIDATION RESULTS

Method 1 Method 2
Total No. CE

- CERT Points 217 217

No. CE -CERT Points within Ideal BSPM Range 210 210

No. CE -CERT Points above 95th Percentile 0 1

No. CE -CERT Points below 5th Percentile 68 70

No. CE -CERT Points between 5th and 95th Percentiles 142 139
% CE -CERT points that did not validate 32.38 33.81

TABLE ES -3 . AVL MSS VALIDATION RESULTS

Method 1 Method 2 Method 3
Total No. CE

- CERT Points 271 271 271

No. CE -CERT Points within Ideal BSPM Range 263 263 263

No. CE -CERT Points above 95th Percentile 47 28 23

No. CE -CERT Points below 5th Percentile 2 2 3

No. CE

- CERT P oints be tween 5t h a nd 9 5th Percentiles 222 233 237

% CE -CERT points that did not validate 18.08 11.41 9.89

Due t o t he l ack of additional f unding, t he measurement allowance pr ogram w as concluded by the SC without being able to solve the lack of validations issue with the Sensors PPMD , or pe rform a dditional C E

- CERT te sting w ith the H oriba T RPM to determine if i t validates the model

. T he AVL MSS passed the validation criteria using Method 3. H owever, the MS S as used in this program is not accepted as a PM

- PEMS by EPA, and the measurement allowance generated b ased on t he pe rformance o f this instrument is not an official p art of the measurement allowance.

The final official measurement allowance accepted by the SC was based on t he Sensors PPMD. Further investigation of why the PPMD did not validate the model is being investigated outside the MA program, and the details are expected to be part of the CE-CERT Final Report on PM

- PEMS In-U se V alidation T esting

.REPORT 03 .14936. 12 1 of 174 1.0 INTRODUCTION

The U.S. EPA, EMA, and CARB agreed to pursue an experimental data driven program to determine PEMS bias and precision measurement errors expected in in

- use NTE testing and compliance b efore en forcement. The i dea i s t o combine t hese e rrors i nto a m easurement allowance which will be used with the EPA in

- use regulatory standard. The combination, in

- use standard plus m easurement al lowance, will al low f or a l arger t h reshold due t o i nstrument a nd measurement uncertainties that the engine manufacturers must comply with. The gaseous PEMS measurement a llowance pr ogram w as c ompleted b y S wRI i n A pril of 2007, a nd t his w ork focuses on the PM- PEMS measurement allowance, as se t forth in Test Plan and meeting minutes, shown in Appendices A and B, respectively.

To determine all bias and precision errors associated with in -use PM

- PEMS measurement would take a very extensive set of experimental data and engines to cover all engine steady

- state and transient NTE operation s, as well as

, different environmental conditions and configurations that may influence the measurement. Instead of focusing on a wide matrix of experimental data, the SC appro ach was to :

a) Perform repeats on a seri es of six laboratory NTE steady

- state laboratory tests using the CVS PM filter method and the PM

- PEMS, and establish a bias error surface for each of the PEMS tested

. b) Perform repeats on a series of laboratory N TE t ransient c ycles c ontaining thirty 32

secon ds N TE e vents, and use t he P M

- PEMS m easurement t o pr oduce p recision e rror surface for each of the PEMS tested

. c) Perform a series of e nvironmental te sts tha t inc ludes e lectromagnetic a nd radio frequency i nterferences, shock a nd vi bration, pr essure, a nd t emperature a nd hum idity, and produce an error surface, if any, for each PEMS associated with each parameter

. d) Use torque and fuel flow error surfaces provided by the engine manufacturers. e) Use ot her error s urfaces es tablished during t he gaseous m easurement allowance
program. f) Use M onte

- Carlo s imulation ba sed on a s et of 141 Ideal br ake

- specific P M ( BSPM) reference NTE events to predict the error distribution at each reference NTE

. g) Use t he 95 th percentile a nd 5 th

h) Perform act ual i n

- use testing w ith the PM

- PEMS using t he C E

- CERT tr ailer to determine whether or not the data generated in -use validates the mode l. percentile of t he error di stribution a t each Id eal B SPM reference N TE l evel as the uppe r a nd l ow bounda ry o f t he de ltas be tween P M

- PEMS BSPM and Ideal BSPM.

.

A total of nine PM-PEMS, three from each manufacturer, were used on this progr am. The PM

- PEMS, shown in Table 1 , included the Sensors PPMD, Horiba TRPM, and AVL MSS. Each set of t hree P M

- PEMS, one PM

- PEMS f rom each manufacturer, was t ested simultaneou sly i n parallel us ing a s eries of s teady

- state and transient N TE engine experiments. After t he completion of steady

- state testing and transient testing, one PM

- PEMS from each manufacturer was us ed on a s eries of e nvironmental t est c onditions t hat i ncluded electromagnetic and radio frequency i nterferences

, shock a nd vi bration, atmospheric pr essure, a nd t emperature a nd humidity.REPORT 03 .14936. 12 2 of 174 TABLE 1. PM

- PEMS USED ALONG WITH SERIAL NUMBER

PEMS Name PEMS Serial Number
Horiba 1 10107-01 Horiba 2 10107

- 02 Horiba 3 10107

- 03
Sensors 1 E08

- PD03
Sensors 2 G08

- PD02
Sensors 3 A08

- PD03

AVL 1 346
AVL 2 472
AVL 3 273

To validate the model, a series of in

- use tests with the PM

- PEMS w as performed by CE

CERT us ing t he C E

- CERT e missions tr ailer. Details a bout thi s te sting will be pr ovided in a separate report by CE

- CERT on PM

- PEMS In

- Use Validation Testing.

This report describes:

**•** Model approach used to perform the modeling portion of this work.

**•** Engine ex periment s with the P M

- PEMS including s teady

- state and transient N TE P M results

**•** Environmental setup and PM results

**•** Measurement allowance produced by the model for each of the three PM

- PEMS provided by each manufacturer

**•** Model validation using the PM- PEMS selected for in

- use testingREPORT 03 .14936. 12 3 of 174 2.0 MODELING APPROACH

2.1 Purpose of Model

Th is program w as designed t o generate BSPM m easurement al lowances b ased on rigorous statistical methods applied to a large body of data. At the same time, it was desirable to exclude out lier da ta ca used by ex treme m easurement er rors w hich were not cons idered representative of nor mal i n

- use ope rations. A di rect a pproach c ould ha ve be en t o t est P EMS against s ome ki nd of mobile l aboratory r eference ( such a s t he C E

- CERT M obile E mission Laboratory) on a l arge number o f ve hicles, a nd qua ntify e rrors di rectly. H owever, s uch a n approach would have been expensive in terms of both time and funding.

Given these f actors, the S teering C ommittee ultimately e lected to us e a s imulation approach in order to generate the BSPM measurement allowances, similar to what was done in the g aseous i n

- use e missions t esting pr ogram [2] . I n t his a pproach, t he S teering C ommittee defined all of the expected sources of PEMS measurement errors based on existing in

- use testing expertise and un derstanding of how the PEMS functioned. Priority w as given to the Horiba's and Sensors' PEMS in the design of experiments. Each of these errors was quantified using a series of cont rolled laboratory experiments, each de signed to isolate er rors r elated to a s ingle error source. The results of each experiment would essentially be an empirical model of a given source of measurement error. In this report, these error models are referred to as error surfaces. It is important to note that each of these error surfaces represents an incremental error of PEMS measurement, as compared to an associated laboratory reference measurement.

2.2 Model Improvement

Several i mprovements t o t he e xecution of t he M onte C arlo s imulation m odel w ere implemented to improve the efficiency and post

- processing of the simulation runs. Eight macros were written to perform various tasks and are summarized in the section below.

Macro 1 : Controls batch processing and allows the ability to run several simulations back

to

- back (batch m ode). Reads each reference NTE event data, number of trials, and the number of reference NTE events in the batch run. Calls other macros.

Macro 2: Clears and deletes ex tra r ows i n error m odel ( see A ppendix C f or a d etailed description). C alculates ideal PM emissions for each calculation method.

Macro 3: Checks e rror s urfaces t urned ‘ off’ i n s imulation r un a nd c lears unus ed e rror surface cha rts and Excel t abs for c alculation s peedup ( see A ppendix D ) f or a de tailed description).

Macro 4: Controls Crystal Ba ll run preferences

, runs Crystal Ball simulation for the given NTE Event, and controls Crystal Ball creation of Report and Extract data files . No longer store s each trial i c value (40,000 – 65 ,000 values). Reduced the number of sensitivity charts created. Only stores BS emissions in g/ hp- hr units. T his r educed EXTRACT and REPORT files from 133 MB to 17 MB per NTE event (≈ 87% reduction).REPORT 03 .14936. 12 4 of 174 Macro 5: Reads EXTRACT file, controls calculation of 5 th , 50 th and 95 th percentiles from full model and calls Macro 6.

Macro 6: Compute s 5 th , 50 th and 95 th percentile delta BS emissions and confidence limits from order statistics from the Full model. These values are used in the measurement allowance plots and to check for convergence.

Macro 7: Reads Extract file Val idation data, computes 5 th , 50 th and 95 th percentiles for the Validation model (calls Macro 6).

Macro 8: Reads RE PORT file, selects a nd formats s ensitivity da ta f or A ssumption

sens itivities greater than 5 percent or less than

- 5 percent contribution to variance and stores in sensitivity file. Create s sensitivity file with “important” error surfaces

. The de velopment of t he m acros r esulted i n s ubstantial i mprovements i n t he s imulation

run

- time and post

- processing of the extract files. The estimated r edu ction in model simulation

run

- time and post processing was approximately 80 to 85 percent of the time used in the gaseous PEMS program. The developed macros eliminated the need to manual ly post- process the files as had been done in the gaseous PEMS progra m . The batch processing allowed up to 20 reference NTE events to be simulated in a single Excel run. The size of the EXTRACT and REPORT files was greatly reduced by only sto ring needed variables and delta PM emissions.

2.3 Monte Carlo Simulation Approac h

The e rror s urfaces r epresenting incremental er ror s of P EMS m easurement w ere programmed into a computer model which employed Monte Carlo random sampling methods to simulate t he c ombined effects of a ll of t hese s ources of e rror on t he f inal m easured br ak especific value. An ideal reference NTE data set (see Section 2.5) for a given test event was run through the Model, and all the various errors were applied to that data set in a randomly chosen manner. Brake

- specific PM emission values were then calcul ated for both the ideal and error

applied data sets, which were compared to yield a final measurement error (see Appendix C and D) . T he p rocess w as repeated t housands of t imes, w ith m any different i deal da ta s ets, t o generate a l arge, robust da ta s et w hich was ev aluated to determine a final s et of com bined measurement errors. These final errors, referred to in this report as deltas, were generated for the PM pollutant for each calculation method and three PEMS model units from each of the three manufacturers, for a final set of seven deltas; Methods 1

- 3 for the AVL PEMS unit, Methods 1

- 2 for the Horiba PEMS unit and Methods 1

- 2 for the Sensors PEMS unit. A complete description of the Monte Carlo methodology and of the model is given in Section 2.5 of this report.

2.4 Calculation Methods

Calculations m ust be pe rformed on t he r ecorded da ta t o de termine br ake

- specific P M emission values in accordance with methods outlined in 40 C FR Part 10 65 Subparts G and J . The symbolic notation given in the formulas shown later in this section is fully described in 40 CFR Part 1065 Subpart K [ 1].REPORT 03 .14936. 12 5 of 174 40 C FR P art 1065 a llows f or t he us e of a ny of t hree di fferent c alculation m ethods i n order to determine brake

- specific emission values from in

- use test data. The basic cal culation of brake- specific emissions requires three main inputs as follows:

BS Emissions = Mass
Work = Concentration x Flowrate
Power

The three calculation methods vary somewhat in the means used to determine either the Flow c omponent or t he W ork c omponent of t his c alculation. E ach of t he t hree m ethods i s summarized be low. B ecause e ach m ethod r elies on di fferent i nputs, i t i s pos sible that e ach method of c alculation w ill r eact di fferently t o va rious m easurement errors. T herefore, measurement allowances m ust be e xamined i ndependently f or e ach m ethod. H owever, according t o the T est P lan, see Appendix A and B

, methodology, o nly on e of t he t hree calculation methods w ould be s elected to generate t he f inal m easureme nt a llowances. T he selection methodology i s outlined later in this introduction under the M easurement Allowance Generation section.

2.4.1 Calculation Method 1 – “ Exhaust Flow -Torque- Speed ” Method

Calculation M ethod 1 i s a nalogous t o t he m ethod us ed b y most d ynamometer laboratories, and relies on direct input of both exhaust flow and torque. In the case of exhaust flow, this is the flow rate measured by the same form of exhaust flow meter. T he Sensors and AVL PEMS relied on the Sensors exhaust flow met er (EFM) while the Horiba PEMS had a tail pipe adapter (TPA) it employed for exhaust flow measurement. Work is not measured directly, but is instead calculated using ECM broadcast engine speed and ECM broadcast engine torque. While eng ine s peed is directly m easured by t he en gine E CM, ECM b roadcast t orque i s an estimate based on a variety of other parameters ; torque cannot be directly verified during in -use testing. A simplified formula for this method is:

Met h od 1 = ∑ mass ∑ workREPORT 03 .14936. 12 6 of 174 The more complete formula used for Method 1 is as follows:

For all PM PEMS :

PM m is a flow weighted particulate matter exhaust concentration in g/mol



## ( ) **( ) ( )** 

# ∑ **∑**

**= =**  

###  ** **  ** **  ** ∆** 

**⋅ **  ** **  ** **  ** ∆**  ** **  ** **  ** **  ** **  ** **  ** ** 

**=⋅** N

i i i N

i i PM

PM 

### t mNT rpmSpeed t
s mol
n mol

g m hr kW ge

1 1



### 3600*1000*60 *14159.3*2** * *
/ 

Where f or AVL:

PM m

1

1 

### *

* N

i i i
PM N

i i 

### g mol mPM n t mol s g m mol mol n t
s **=**

**=** 

###   **   ∗ ∆**  **    **  **    **  ** **

**=  **  **   **  **  ∆**  **   **  **   **  

# ∑

**∑** 

 is computed numerically as follows,

It should be noted that calculation Method 1 is directly dependent on the accuracy of both the e xhaust f low m eter a nd t he t orque e stimation, a s w ell a s on t he m easurement of PM concentration.

2.4.2 Calculation Method 2 – “Exha ust and Fuel Flow -Torque- Speed” Method

This calculation is designated solely for in

- use testing, and is designed to minimize the effect of errors r elated t o the a ccuracy o f t he ex haust f low m easurement. The M ethod 2 calculation adjusts the exhaus t flow measurement by a ratio of the CO 2

Met h od 2 = ∑ ma ss ∑ � CO 2 fuel ECM fuel × Work � - based fuel flow to the ECM r eported fuel f low. T his m eans t hat although t he f low m eter m ust be l inear, i t doe s not necessarily have to be accurate. In addition, Method 2 depends on t he ECM broadcast torque and s peed, a nd on t he r atio of f uel f low c alculated f rom t he c arbon ba lance us ing ga seous measurement over the fuel flow broadcast by the ECM. A simplified version of this method can be expressed as:REPORT 03 .14936. 12 7 of 174 The more complete formula for Method 2 using PM as an example is:

For all PM PEMS :

PM m is a flow weighted particulate matter exhaust concentration in g/mol



## ( )

**( ) ( ) ( )**  

# ( **)**

**( ) ( )** 1

6 2 2 

### * *
/

* *10 % % *10 *

*

* *2*3.14159
60*1000*3600 i

N

PM i i
PM

ii i C

fuel fueli

i i 

### g mol m nt mol s e g kW hr

mol n xTHC ppm xCO xCO t M s

w g m s Speed rpm T N m **=**

**− −**



###   **  ∆**  **   **  **   ** 

**⋅ =  **  **   **  **  ++ ∆**  **   **  **   **  **   **  **   ** 

**⋅  **  ** ** 

# ∑




 1 N

i **=** 

# ∑

Where f or AVL:

PM m

1

1 

### *

* N

i i i
PM N

i i 

### g mol mPM n t mol s g m mol mol n t
s **=**

**=** 

###   **   ∗ ∆**  **    **  **    **  ** **

**=  **  **   **  **  ∆**  **   **  **   **  

# ∑

**∑** 

 is computed numerically as follows,

It should be noted that, as mentioned earlier, Method 2 i s not subject to accuracy errors for the exhaust flow measurement, although that measurement must st ill be linear for the method to function properly.

2.4.3 Calculation Method 3 – “Fuel Flow-Torque- Speed ” Method

Method 3 doe s not us e di rect m easurement of e xhaust f low, but r elies on a carbon balance and ECM br oadcast f uel rate t o determine m ass. T he work term f or M ethod 3 i s determined identically t o t he w ork t erm f or M ethod 1; us ing t he E CM br oadcast va lues f or engine speed and torque to calculate work. Method 3 entirely circumvents the use of an exhaust flow meter, but for the HDIUT program, EPA must approve the use of Method 1 for a given test and manufacturer. A simplified version of Method 1 may be expressed as:

Met h od 3 = ∑ � mass × ECM fuel CO 2 fuel �

∑ WorkREPORT 03 .14936. 12 8 of 174 The more complete formula for Method 3 is:

For AVL Only: 

## ( )

**( ) ( ) ( )**  

# ( **)**

**( ) ( )** 6 2 1 2

1 

### /

* * * *10 % % *10

* *2*3.14159*
60*1000*3600 i PM

ifue l N fuel
PM i C i i

N i i

i 

### e g kW hr

g m w g s m t mol M xTHC ppm xCO xCO

Speed rpm T N m t **− −**

**=**

**=** 

### ⋅ = **   **  **   **  **   **  **∆  ** 
**+ +  **  ** **
**⋅ ∆  **  **   **  

# ∑

**∑** 

Where:



## ( ) 

# ( **) ( )**  **( ) ( )** 

# ( **)**

**( ) ( ) ( )**  

# ( **)** 6 2 1 2

6 2 1 2 

### / *

* * *10 % % *10

* * *10 % % *10 i

i i ifue l N fuel

i

C i i

PM

ifue l N fuel

i

C i i 

### g mPM g mol m w s t M xTHC ppm xCO xCO

g m mol g m w s t M xTHC ppm xCO xCO **− −**

**=**

**− −**

**=** 

###   **   **  **   **  **∆**
**+ +  **  ** **

**=  **  **   **  **   **  **   **  **∆**
**+ +  **  

# ∑

**∑** 



It should be noted that Method 3 i s not subject to exhaust flow measurement accuracy errors, but also that this method is wholly dependent on ECM broadcast values for both mass and work determination.

2.5 Reference N TE Events

The reference data set to which all the simulated er rors were applied represented engine operations ove r a w ide r ange of laboratory NTE eve nts. P arameters i n the r eference da ta s et were scaled in order to exercise the model th rough a more appropriate range of parameters (i.e. concentrations, f lows, ambient c onditions, e tc.). In t his s caling pr ocess, c are w as t aken t o maintain the dynamic characteristics of the reference data set.

The Monte Carlo simulation model was run on a set of 141 reference NTE events that were used during the gaseous MA program [2]. O nly the events that have different speed and torque c ombinations w ere us ed; five e ngine m anufacturers pr ovided a t otal of 97 e vents; 10 reference NTE events came from each of the three engines tested in the lab during the gaseous MA transient testing; and 14 e vents came from the pre

- pilot CE

- CERT data. Because no PM data exist for these reference events, the PM concentration used to calculate the Reference NTE PM emissions was developed by SwRI based on the Volvo engine used in this study. A simple model w as de veloped t o pr edict t he P M c oncentration ba sed on s peed a nd t orque us ing 80 different s teady

- state P M conc entrations t hat w ere m easured with the M SS us ing t he de siredREPORT 03 .14936. 12 9 of 174 exhaust b ypass c onfiguration. A dditional a djustment t o t he m odel us ing t he r ate of c hange i n torque was added to better predict short NTE transient events. The model was further adjusted to produce a br ake

- specific e missions d istribution f or the r eference NTE e vents c entered a round 0.02 g/hp-hr, as shown in Figure 1.

FIGURE 1 . METHOD 1 IDEAL BS PM VALUES FOR REFERE NCE NTE EVENTS

NTE br ake

- specific em issions r esults w ere calculated for PM us ing e ach of t he t hree agreed -upon NTE calculation methods. T h e t hree di fferent BS emission s calculation methods referred to in this test plan are :

1. Method No. 1: Exhaust Flow-Torque-Speed Method 2. Method No. 2: Exhaust and Fuel Flow-Torque- Speed Method 3. Method No. 3: Fuel Flow -Torque- Spee d Method

Table 2 lists the number of NTE events obtained from each data source and the BSPM emissions calculated using Method 1. These emissions have been computed with no error values added to the input parameters. For this report, emissions with no errors added will be labeled the “ideal” emissions. In contrast, t he e missions w ith e rrors added t hrough t he M onte C arlo simulation will be labeled emissions “with errors .”REPORT 03 .14936. 12 10 of 174 TABLE 2. REFERENCE NTE EVENTS AND IDEAL BSPM EMISSIONS

Source Number of NTE Events BSPM Min max International 19 0.01146 0.03069 DDC 18 0.01303 0.03463 Caterpillar 20 0.01097 0.01789 Cummins 20 0.01561 0.03253 Volvo 20 0.01541 0.02701 Engine No. 1 10 0.01215 0.04066 Engine No. 2 10 0.01675 0.04872 Engine No. 3 10 0.01099 0.03669

When the ideal brake

- specific emission values were calculated for the various reference NTE e vents, i t w as not ed t hat t hese i deal e mission va lues w ere f requently di fferent f rom one calculation method to another. While it was recognized that this was a realistic outcome, th e Steering C ommittee f elt tha t the se di screpancies mig ht int roduce a n unintended bias int o the results of the M onte C arlo simulation. T herefore, the S teering C ommittee di rected SwRI to adjust the NTE reference event data in order to alig n the brake

- specific emission levels from all the calculation methods.

The M ethod 1 r esult w as not c hanged, t herefore t orque, s peed, a nd e xhaust f low remained unchanged as well. The CO 2 concentration was adjusted to make the Method 2 r esult equal to the one from Method 1. This was done by using a single multiplier on all CO 2 values for the NTE event in qu estion. Lastly the fuel r ate values and alignments were adjusted to bring Method 3 in line with Method 1 and 2.

The distribution of the ideal BSPM emissions data for the 141 reference NTE events was presented to the Steering Committee. It was noted that very few reference NTE events were at or below t he B SPM t hreshold ( 0.02 g /hp

- hr). T hus, t he r eference N TE events were adjusted to produce more events with ideal BSPM values below 0.02 g/hp

- hr. The original and revised ideal BSPM di stributions a re de picted i n Figure 1 . N ote tha t the B SPM e mission data has va lues spread above and below the corresponding NTE threshold.

Table 3 provides a summary of some descriptive statistics for the reference NTE data set for BSPM emissions.

TABLE 3. D ESCRIPTIVE STATISTIC S FOR BSPM EMISSIONS FOR 141 REFERENCE NTE EVENTS

Descriptive Statistic BSPM g/hp

- hr Minimum 0.010974 Maximum 0.048717 Mean 0.020753 Median 0.019101 Standard Deviation 0.006932REPORT 03 .14936. 12 11 of 174 The pa rameter d ata pr ovided in each reference NTE eve nt w as on a s econd

- by -second basis w ith a m inimum o f 30 s econds a nd a m aximum of 300 s econds. The i nput pa rameters required for the BS PM emissions calculation methods and the Monte Carlo simulation are listed in Table 4

. An E xcel f ile w ith a s pecific input f ormat s tructure w as us ed to standardize the format of the input files. Since the total hydrocarbons (THC) was selected as an input parameter, NMHC was computed as THC*0.98.

TABLE 4 . INPUT PARAMETERS FOR REFERENCE NTE EVENTS

Variable Number Input Variable Units Description

1 NTE Event Number integer All reference NTE events must be identified by an NTE number (e.g., 001). 2 NTE Source alphanumeric The source of the NTE event is the company, organization and/or lab that created the event data. 3 Engine Make alphanumeric Engine Make 4 Engine Model alphanumeric Engine Model 5 Engine Displacement L Engine Displacement (L) 6 Date mm/dd/yyyy The day the NTE event data was created (mm/dd/yyyy).
7 Time Stamp hh:mm:ss.s Time in seconds. Each reference NTE must contain second-bysecond data only. 8 Wet CO2 % CO2 (%) 9 Wet CO % CO (%) 10 Wet kNO ppm NO (ppm) with intake air-humidity correction
11 Wet kNO2 ppm NO2 (ppm) with intake air-humidity correction
12 Wet THC ppm THC (ppm) 13 Exhaust Flow Rate scfm Exhaust flow rate (scfm) To compute the % of PEMS flowmeter maximum flowrate we will need to know what size flowmeter was used for each NTE event. Enter either 3, 4, or 5 to represent the following flowmeters and maximum flow rates: 3 = 3 inch EFM with maximum flow rate = 600 scfm 4 = 4 inch EFM with maximum flow rate = 1100 scfm 5 = 5 inch EFM with maximum flow rate = 1700 scfm 15 Speed rpm Engine speed (rpm) 16 Low Speed, nlo rpm To compute the % of normalized speed we will need nlo and nhi for the engine computed as follows: nlo (rpm) = lowest speed below max power at which 50% max power

occurs

18 Fuel Rate L/sec Fuel rate (L/sec) To compute the % of maximum fuel rate we will need the max fuel rate of the engine for each NTE event. Max fuel rate (L/sec) 20 Derived Torque N

· m Torque (N

· m) 21 Peak Torque N

· m To compute the % of maximum torque we will need the peak torque of the engine for each NTE event Peak torque (N

· m) 22 Flow-weighted Average PM Concentration for Methods 1 & 2 μg/mol Flow-weighted average PM concentration, flow-weighted by the exhaust flow. Values were calculated based on a predictive model developed using transient and steady-state experimental data. 23 Flow-weighted Average PM Concentration for Method 3 μg/mol Flow-weighted average PM concentration, flow-weighted by the exhaust flow. Values were calculated based on a predictive model developed using transient and steady-state experimental data. 19 Max Fuel Rate L/sec nhi (rpm) = highest speed above max power at which 70% max
power occurs 14 Flowmeter Diameter 3, 4, or 5 (inches)

17 High Speed, nhi rpmREPORT 03 .14936. 12 12 of 174 2.6 Error Surface Generation

During t he di scussions he ld at s everal S teering Committee m eetings, 33 error s urfaces were identified and considered for inclusion in the Monte Carlo simulation model. 25 of these error surfaces were the same surfaces used in the gaseous emissions in

- use testing program. [2] Of t he r emaining ei ght er ror s urfaces, two were di scarded (Delta P M E MI/RFI and Delta P M Vibration). because the PM generator was not used during these tests (see Section 5) T he six new er ror s urfaces f or t he P M pr ogram w ere P M S teady

- State ( SS), PM T ransient, PM Atmospheric Pressure, PM Ambient Temperature, Torque Engine Manufacturers and Fuel Rate Engine Manufacturers. This resulted in a final total of 31 error surfaces that were incorporated into the Model. These individual error surfaces encompassed a wide variety of error sources. In addition, all error surfaces distributions used in this program included a range of sampled data between the 1 st and 99 th percentile t o expand the r ange of s ample d ata i n the M onte C arlo simulation. T his w as done a t t he request o f t he S C s ince s ampling for t he gaseous p rogram covered only the range between the 5 th and 95 th

.

Table 5 lists th e error surfaces examined during the study with the surfaces excluded by the S teering C ommittee de signated in italics. A ll r emaining on es w ere impl emented in the simulation model . Each error surface was assigned a number for easy identification.

For eac h of the measurement errors defined in Table 5

, an error surface was created and used i n t he M onte C arlo s imulation. E ach e rror s urface represented an a dditive e rror

— or a subtractive error if the sign wa s negative

— relative to the reference parameter value to which it was applied. Figures 2 through 4 show an example of how these error surfaces were created for every measurement error. Details on t he c onstruction of e ach e rror s urface us ed i n t he simulation are provided in Section 0

. The example illustrated in Figure 2 through 4 represent s the e rror s urface for s teady

- state bi as and precision PM c oncentration e rrors f or an i ndividual PEMS unit

2.6.1 PEMS vs. Laboratory Nominal Results

Figure 2 was constructed from raw data a cquired from steady

- state engine lab tests with the PEMS conducting repeat testing at various concentration levels (PM μg/mol ). The plot pools all bias and precision errors for the PEMS tested f or all steady

- state modes. A nominal target of 10 repeat m easurements of P M

Figure 2 was t aken on each PEMS unit f or each value of the corresponding a verage l ab P M values ( i.e., l ab nominal va lue). The 10 PEMS measurements were pl otted a gainst t he c orresponding measurements using l ab oratory equipment. S hown i n are t he 5 th , 50 th , a nd 95 th percentiles c orresponding t o t he di stribution of t hese 18 0 observations (30 observations at each of the six concentration levels) u sing t he P EMS at each average P M concent ration l evel ( note t hat t he distribution of da ta a t e ach P M level ma y not represent a normal distribution). S ince the 50 th

Figure 2 percentile s do not lie on the dashed (diagonal) line of perfect agreement, the data suggest that there is a bi as error between the P EMS and lab results. In essence this graph summarizes the statistical distribution measured by the PEMS at each concentration level sampled. The example plot in shows only 6 discrete average PM concen tration levels (ranging from 10 -60 μg/mol ). H owever, the actual number of discrete concentration levels was determined using the total number of operating conditions actually run for al l the tests on t he engine . I n the s ection on Steady- State Repeat Engine Testing and Error Surfaces it is reported that 6 operating modes condition s from an initial number of 80 operatingREPORT 03 .14936. 12 13 of 174 conditions w ere s elected f or con struction of t he steady

- state P M error s ur face. From t hese 6 operating modes several discrete PM concentration levels were defined which were used in the error surface plots for the Monte Carlo simulation.

TABLE 5 . ERROR SURFACES FOR MONTE CARLO SIMULA TION

Component # Test Source Error Surface Description 1. Delta PM 1 Engine Dyno Delta PM SS AVL, Horiba and Sensors
2 Engine Dyno Delta PM Transient AVL, Horiba and Sensors 3 Environ Delta PM EMI/RFI Deleted by Steering Committee
4 Environ Delta PM Atmospheric Pressure AVL 5 Environ Delta PM Ambient Temperature AVL 6 Environ Delta PM Vibration Deleted by Steering Committee 2. Delta CO 7 Engine Dyno Delta CO SS Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 10 Environ Delta CO Atmospheric Pressure Same as Gaseous Study 11 Environ Delta CO Ambient Temperature Same as Gaseous Study 3. Delta NMHC 13 Engine Dyno Delta NMHC SS Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% NMHC = 0.98*THC 14 Engine Dyno Delta NMHC Transient Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 16 Environ Delta NMHC Atmospheric Pressure Same as Gaseous Study 17 Environ Delta NMHC Ambient Temperature Same as Gaseous Study 19 Environ Delta Ambient NMHC Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 4. Delta Exhaust Flow 20 Engine Dyno Delta Exhaust Flow SS Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 21 Engine Dyno Delta Exhaust Flow Transient Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 22 Engine Dyno Delta Exhaust Flow Pulsation Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 23 Engine Dyno Delta Exhaust Flow Swirl Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 25 Environ Delta Exhaust EMI/RFI Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 27 Environ Delta Exhaust Temperature Same as Gaseous Study 28 Environ Delta Exhaust Pressure Same as Gaseous Study 5. Delta Torque 29 Engine Dyno Delta Dynamic Torque Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 30 Engine Dyno Delta Torque DOE Testing (Interacting Parameters Test) Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 31 Engine Dyno Delta Torque Warm-up (Interacting Parameters Test) Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 32 Engine Dyno Delta Torque Humidity/Fuel (Independent Parameters Test) Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 34 Engine Dyno Delta Torque Interpolation Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 35 Engine Manuf Delta Torque Engine Manuf New 6. Delta Fuel Rate 42 Engine Manuf Delta Fuel Engine Manuf New 7. Delta Speed 43 Engine Dyno Delta Dynamic Speed Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 8. Delta Fuel Rate 44 Engine Dyno Delta Dynamic Fuel Rate Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th%

9. Delta CO 2 45 Engine Dyno Delta CO 2 SS Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th%

46 Engine Dyno Delta CO 2 Transient Same as Gaseous Study but moved 5th% to 1st% and 95th% to 99th% 49 Environ Delta CO 2 Ambient Temperature Same as Gaseous Study Measurement Error Surfaces and Deltas Used in BSPM CalculationsREPORT 03 .14936. 12 14 of 174 FI GURE 2 . ERROR SURFACE CONSTRUCTION: PEMS VS. LABORATORY RESULTS

2.6.2 (PEMS – Laboratory) Deltas vs. Lab

Figure 3 illustrates the “error band” measured during testing. This plot was created by first subtracting the individual “lab nominal” PM value from the corresponding individual PEMS PM measurement f or each test run. The sampling s ystem used to obtain the "lab nominal" or "lab reference" PM values is described in Section 4.2.1. Th e difference between the PEMS PM and the lab reference was defined as the “delta” error. Second, these “PEMS - Laborator y” delta errors w ere pool e d at each average l ab nom inal P M value t o obt ain t he 5 th , 50 th , a nd 95 th Figure 3 percentile v alues di splayed i n

. Therefore, the pl ot r epresents t he ave rage P M l ab nominal at 6 discrete co ncentration levels ve rsus t he pe rcentiles of t he d elta er rors computed from the PEMS and laboratory individual test results. Error Surface for SS PM Concentration

18 26 38 45 56 69

13 24 36 42 53 64

8 19 33 39 51 61

0 10 20 30 40 50 60 70 80

0 10 20 30 40 50 60 70 80
PM μg/mole (lab,mean)PM μg/mole (PEMS)

diagonal 95th percentile 50th percentile (median) 5th percentileREPORT 03 .14936. 12 15 of 174 FIGURE 3 . ERROR SURFACE CONSTRUCTION: (PEMS - LAB) VS. LABORATORY RESULTS

2.6.3 Variability Index vs. (PEMS – Laboratory) Deltas and Lab Nominal

This step normalized the plot in Figure 3 using what is called a “variability index (i c )”. This i ndex r epresented t he va lue r andomly dr awn b y t he M onte C arlo s imulation i n or der t o select a given error level. For the 5 th and 95 th percentile of the truncated normal it was allowed to vary from

– 1 to +1 , respectively . The likelihood of “i c ” being any value between

– 1 through +1 w as s pecified b y a “ probability d ensity f unction ( PDF)” a ssigned t o i c. In the cas e of t his example, i c. was assumed to vary according to a standard normal (i.e, bell

- shaped) distribution during the Monte Carlo simulations. T his was because it was believed that the distribution of PM errors due to steady

- state bias and precision would be centered about the 50 th

Figure 3 percentile of the full range of conditions mea sured. Each set of data for each lab “setpoint” average (i.e., lab nominal va lue) i n was nor malized b y aligning t he c orresponding 5 th Figure 3 percentile er ror from with i c =

- 1, the 50 th percentile error with i c = 0, and the 95 th percentile error with i c Figure 4 = +1. These values were the n plotted in , where the y

- axis is the variability index, the

x- ax is is the average lab nominal PM value, and the z- axis is the delta PM value. Notice that, when us ing t his nor malization a pproach, t he 5 th , 50 th , a nd 95 th Error Surface for SS PM Concentration

8

6 8

5 6 9

3 4 6

2 3 4

-2 -1 3

-1 1 1

-4 -2 0 2 4 6 8 10

0 10 20 30 40 50 60 70

PM μg/mole (lab,mean)PM μg/mole (PEMS) - PM μg/mole (lab)

95th percentile 50th percentile (median) 5th percentile

percentile va lues r emainREPORT 03 .14936. 12 16 of 174 equivalent between Figure 3 and Figure 4

. This development of the error surfaces from the lab data was the procedure used in the gaseous emissions in -use testing program.

FIGURE 4 . ERROR SURFACE CONSTRUCTION: ERROR AT VARIABILITY INDEX FOR 5 TH AND 95 TH PERCENTILES VS. LABO RATORY RESULTS

For the PM measurement allowance program it was decided by the Steering Committee to expand the tails of the truncated normal distribution in the error surface formation in order to allow larger PM deltas to be sampled during the Monte Carlo simulation. Instead of truncating the lower tail at the 5 th percentile, it was moved to the 1 st percentile. Likewise on the upper t ail, the 95 th percentile w as m oved t o t he 99 th percentile. S ince t he or iginal t runcated normal distribution was defined by a mean = 0 and a standard deviation = 0.60795, the resulting indices corresponding to the 1 st and 99 th Figure 5 percentiles are

- 1.4143 and +1.1413, respectively, as illustrated in

. 8 6 8 6 9

3 4 6 2 3 4

- 2

- 1 3

- 1 1 1

- 1.5

- 1 - 0.5 0 0.5

1 1.5

0 10 20 30 40 50 60 70ic_SS_PM

PM μ g/mole (lab,mean) Error Surface for SS PM Concentration

95th percentile 50th percentile (median) 5th percentile Error Surface: z

- axis = ΔSS_PM_μg/mole

5REPORT 03 .14936. 12 17 of 174 FIGURE 5 . TRUNCATED NORMAL DISTRIBUTION PERCENT ILES

Finally, the 5 th and 95 th percentile (PEMS – LAB) delta values at each lab nominal value were us ed t o c ompute t he c orresponding va lues of t he t runcated nor mal f or t he 1 st and 99 th percentiles. To redefine the error surface delta at the 1 st percentile, the standard deviati on for the normal distribution below the 50 th

Standard Deviation 1 st = Delta Value 50 th − Delta Value 5 th 1. 6449 percentile is defined as follows:

Using the mean = 50 th percentile delta and the standard deviation computed above, the 1 st percentile can be found using the Excel NORMINV function: NORMINV(0.01,mean,standard deviation 1st ). Similarly, to redefine the error surface delta at the 99 th percentile, the standard deviation for the normal distribution above the 50 th

Standard Deviation 99 th = Delta Value 95 th − Delta Value 50 th 1. 6449 percentile is defined as follows:

Using the mean = 50 th percentile delta and the standard deviati on computed above, the 99 th percentile c an be f ound us ing t he E xcel N ORMINV f unction: NORMINV(0.99,mean,standard deviation 99th Figure 4 ). Taking the data for the error surface in and redefining it to include the 1 st and 99 th Figure 6 percentile truncated normal values results in the error surface displayed in . All the error surfaces carried over from the gaseous program that were sampled using a truncated normal were redefined at the 1 st and 99 th percentiles. Those error surfaces that were sampl ed using the uniform distribution remained unchanged. Error surfaces such as the one presented in Figure 6 a re the error deltas the Monte Carlo simulation program used during calculation o f the BSPM emissions “with errors”

. - 3

- 2

- 1 0 1 2 3 

### Monte Carlo Truncated Normal Sampling Distribution

N(0,0.60795) - 1.4143

Area= 0.05 Area = 0.01 1.0

- 1.0

5th 95th 1.4143

1st 99th Percentiles **→**REPORT 03 .14936. 12 18 of 174 FIGURE 6 . ERROR SURFACE CONSTRUCTION: ERROR AT VARIABILITY INDEX FOR 1ST AND 99TH PER CENTILES VS. LABORAT ORY RESULTS

2.7 Error Surface Sampling and Interpolation

The e rror m odel us ed t wo di fferent PDF to sample t he er ror s urfaces, depending upon which e xpe rimental pa rameter the s urface r epresented. T o sample er ror s urfaces t hat w ere generated from the l ab test results (Section on Engine Dynamometer Laboratory Testing ), and the applicable environmental t est r esults, t he m odel us ed a t runcated s tandard nor ma l PD F because these tests were designed to evenly cover the full, but finite, range of engine operation and a mbient c onditions. T o s ample error s urfaces t hat w ere generated from t he pr essure and temperature envir onmental test results (Section on Environme ntal Chamber Testing ), the model used a uni form P DF b ecause t hese t ests were already d esigned to cove r t he t ypical range an d frequency of t he respective c onditions. Both of t hese s ampling di stributions a re depicted i n Figure 7

. 10.1 6.8 8.8 7.2 11.1

3 4 6 2 3 4

- 4.1

- 3.1 1.8

- 2.2 0.2

- 0.2

- 2 - 1.5

- 1 - 0.5 0 0.5

1 1.5

2

0 10 20 30 40 50 60 70ic_SS_PM

PM μ g/mole (lab,mean) Error Surface for SS PM Concentration

99th percentile 50th percentile (median) 1st percentile Error Surface: z

- axis = ΔSS_PM_μg/mole

6.2REPORT 03 .14936. 12 19 of 174 FIGURE 7 . TRUNCATED STANDARD NORMAL AT 1 ST AND 99 TH PERCENTILES AND UNIFORM PROBABILITY DENSITY FUNCTION S

Wh en using the truncated standard normal PDF (see Figure 7 ), the Monte Carlo model sampled normal deviates that ranged between -1.4143 and +1 .4143

. The se were used as the i c values de fined i n t he s ection on Error Surfaces

. Similarly, t he pr essure a nd t emperature environmental tests used a uniform PDF to sample test time, from which calculated errors were used. All temperature error surfaces related to the four emissions were sampled uniformly from 1 to 1080 minutes while the error surfaces related to the pressure were sampled uniformly from 1 to 720 m inutes. E xhaust flow error surface for temperature was sampled uniformly from 1 t o 478 minutes while the exhaust flow for pressure was sampled uniformly from 1 to 360 minutes. The errors from all the other tests were aligned with the truncated standard normal PDF such that each of the 50 th percentile error values at each of the tested signal magnitudes was centered at the median (i.e. , 0 value) of the PDF, and the 1st and 99 th percentile error values at each of the tested signal ma gnitudes w ere a ligned with the e xtreme ne gative ( i c = -1.4143 ) a nd pos itive (i c = +1 .4143 ) edges of the PDF, respectively.

Each error s urface was sampled along i ts i c axis ( y

- axis) onc e p er trial for a reference NTE event simulation. Hence, every error surface had a s eparate randomly selected i c for each trial. S ince e ach reference NTE event contained s econd

- by

- second parameter data, ex cept for PM for the Sensors PPMD and Horiba TRP M PEMS, the error surface was sampled at a given i c on the y

- axis a nd at the several selected parameter value s on the x

- axi s that corresponded to each second of t he r eference N TE eve nt. The s ampled error value was determined f or t he given second and parameter al ong t he error ax is ( z

- axis) a t the int ersection of the i c Pro bability Density Functions for Sampling Error Surfaces Once Per NTE Event

- 1.41 0.00 1.41

0 1 2 3 4 5 Relative Probability i

c Lab Tests, Normal, SD=0.60795, truncate @

- 1.4143 & 1.4143

Environmental Tests, Uniform

Note: A non

- t runcated normal distribution with SD=0.60795 has P values of 0.01 and 0.99 at i c = - 1.4143 and
i c =+1,4143 respectively.

value a nd t he
p arameter value from the reference NTE event. This was accomplished by taking e ach second in the r eference N TE event and finding t he t wo adjacent x

- axis va lues f rom t he er ror s urfaceREPORT 03 .14936. 12 20 of 174 between which t o l inearly i nterpolate t o obt ain the e rror s urface x

- value. Each s econd i n the reference NTE event was linearly interpolated with the same i c value for a pa rticular trial at the error surface x

- value. If any of the sampled lab nominal values (PM, NMHC, CO, Speed, Fuel Rate, etc.) ex ceeded the uppe r or l ower l imits o f t he p arameter er ror s urface, t he value of t he closest endpoint of the err or surface was assigned to them.

Figure 8 depicts an example of the error surface sampling using a steady

- state PM error surface c ontaining 10 l ab nom inal P M x

- axis v alues. F or thi s pa rticular tr ial, the r andomly selected i c is

- 0.5. The example reference NTE event is noted by the symbol ‘*’ an d it plotted at i c = -0.5 for each second in the NTE event.

FIGURE 8 . STEADY

- STATE PM ERROR SURFACE FOR AVL WITH EXAM PLE SAMPLING FOR A REFER ENCE NTE EVENT

2.8 Brake

- Specific Emissions Calculations

Errors from Sections 4, 5, and 6 were combined by adding all of the sampled error s once per trial for each reference NTE event simulation . For example, in order to assess the errors in PM concentration by calculation Method #1, several error surfaces were sampled and added to the corresponding parameter in the Method #1 calculation and the resulting BSPM “with errors” - 2 - 1.5

- 1 - 0.5

0 0.5

1 1.5

2

0 50 100 150 200 250 300 350 400 450ic_SS_AVL

PM μg/mol (lab,nom) SS Error Surface for AVL PM Concentration

99th percentile 50th percentile (median) 1st percentile

ic =

- 0.5 NTE Event Error Surface: z

- axis = ΔSS_PM_AVL_μgmolREPORT 03 .14936. 12 21 of 174 was computed. The errors used in this calculation for the Horiba and Sensors are the following (note that the corresponding error surface n umbers are provided in the subscripts ):

PM μg/mol ‘with errors’ = PM μg/mol_ reference + **∆** PM

 μg/mol _1 **∆** PM +

μg/mol
 _2

Exhaust Flow % ‘with errors’ = Exhaust Flow % reference **∆** Exhaust Flow +

% 20 **∆** Exhaust Flow _% 21 **∆** Exhaust Flow +

% 22 + **∆** Exhaust Flow % 23 **∆** Exhaust Flow +

% 25 + **∆** Exhaust Flow % 27 **∆** Exhaust Flow +

% 28
Torque % ‘with errors’ = Torque % reference **∆** Torque +

% 29 + **∆** Torque % 30 **∆** Torque +

% 31 + **∆** Torque % 32 **∆** Torque +

% 34 + **∆** Torque % 35
Speed % ‘with errors’ = Speed % reference + **∆** Speed % 43

w here,

**∆** 1 ,2 **∆** = PM concentration errors due to steady- state and transient errors,

20,21 **∆** = exhaust flow errors due to steady

- state and transient errors,

22,23 **∆** = exhaust flow errors due to pulsation and swirl,

25 **∆** = exhaust flow errors due to ambient temperature

, 27,28 **∆** = exhaust flow errors due to temperature and pressure,

29 **∆** = torque errors due to dynamic torque,

30,31 **∆** = torque errors due to DOE and warm-up,

32 **∆** = torque errors due to interacting parameters humidity and fuel,

34,35 **∆** = torque errors due to interpolation and engine manufacturers,

43 = speed errors due to dynamic speed

Using the formulas for the calculation methods, the BSPM for Method #1 was computed witho ut errors (“ideal”) and then with all the errors applied as outlined above. Table 6 lists all error surfaces used by each calculation method for the PM emissions.REPORT 03 .14936. 12 22 of 174 TABLE 6 . ERROR SURFACES USED FOR COMPUTING BRA KE

- SPECIFIC PM EMISSIONS BY THREE C ALCULATION METHODS

2.9 Convergence and Number of Trials

Since t he T est P lan did not i nclude a p rovision for conv ergence criteria, the S teering Committee w as t asked t o de velop a c onvergence m ethod. T he m ain g oal w as t o de fine how many s imulation trials at a g iven reference N TE eve nt w ere r equired t o estimate the 95 th percentile BSPM e mission differences w ith a given precision. A lthough the C rystal Ball software contained precision control options, the method used to compute a conf idence interval on pe rcentiles w as b ased on a n a nalytical boot strapping m ethod w hich w as not a dequately documented. T hus, a n independent c onvergence m ethod w as pr oposed a nd a ccepted b y t he Steering Committee. Component # Error Surface Method 1 Method 2 Method 3 1. Delta PM 1 Delta PM SS   
2 Delta PM Transient   
4 Delta PM Atmospheric Pressure   
5 Delta PM Ambient Temperature    2. Delta CO 7 Delta CO SS  
10 Delta CO Atmospheric Pressure  
11 Delta CO Ambient Temperature   3. Delta NMHC 13 Delta NMHC SS   NMHC = 0.98*THC 14 Delta NMHC Transient  
16 Delta NMHC Atmospheric Pressure  
17 Delta NMHC Ambient Temperature  
19 Delta Ambient NMHC   4. Delta Exhaust Flow 20 Delta Exhaust Flow SS  
21 Delta Exhaust Flow Transient  
22 Delta Exhaust Flow Pulsation  
23 Delta Exhaust Flow Swirl  
25 Delta Exhaust EMI/RFI  
27 Delta Exhaust Temperature  
28 Delta Exhaust Pressure   5. Delta Torque 29 Delta Dynamic Torque  
30 Delta Torque DOE Testing (Interacting Parameters Test)  
31 Delta Torque Warm-up (Interacting Parameters Test)  
32 Delta Torque Humidity/Fuel (Independent Parameters Test)  
34 Delta Torque Interpolation  
35 Delta Torque Engine Manuf   6. Delta Fuel Rate 42 Delta Fuel Engine Manuf  7. Delta Speed 43 Delta Dynamic Speed   8. Delta Fuel Rate 44 Delta Dynamic Fuel Rate  9. Delta CO 2 45 Delta CO 2 SS  
46 Delta CO 2 Transient  
49 Delta CO 2 Ambient Temperature  REPORT 03 .14936. 12 23 of 174 A nonparametric statistical technique [3] wa s proposed which defined a 90% confidence interval f or t he 95 th percentile of t he BSPM em issions di fferences f or a n individual r eference NTE simulation. I f the width of the 90% confidence interval was less than 1% of the BSPM emissions t hreshold, t hen c on vergence w as m et. T he f ollowing s teps d efine t he conv ergence method:

1. Run the Monte Carlo simulation for N trials. 2. Order the BS emissions differences from smallest to largest. 3. Identify the trial number at the lower end of the 90% confidence interval

n lowe r 

### N N *05.0*95.0645.1*95.0

**−** = 4. Identify the trial number at the upper end of the 90% confidence interval

n upper 

### N N *05.0*95.0645.1*95.0 **+** = 5. Compute (BSPM difference value at n upper ) – (BSPM difference value at n lower 6. If the result in (5) < 1% of the BSPM emissions NTE threshold then convergence is met. ).

7. The BSPM threshold was defined as 0.02 g/hp

- hr. Thus, 1% of the threshold was 0.0002 g/hp

- hr.

The St ee r ing Committee a greed to the proposed c onvergence c riteria outlined a bove. During the initial simulation runs for 20 reference NTE events

, convergence was not met at the 1 percent criteria level until 60,000 t rials were run. T his only applied to the AVL PEMS unit at each of the three calculation methods. The Horiba and Sensors units only reached convergence at t he 1 percent cr iteria f or appr oximately h alf of t he 20 reference N TE eve nts s imulations. Upon e xamination of t he di stributions of t he de lta BSPM e missions ge nerated f rom t he simulations, some of the distributions were positively skewed whi ch would make convergence very difficult at the 1 percent level. T his information was presented to the Steering Committee wherein a decision was made to relax the convergence level to 2 percent or higher, depending on the outcomes of the simulations.

I n s ummary, t he 141 r eference N TE e vents w ere r un a t 40,000 t rials a nd c onvergence was checked. If the width of the confidence interval on the 95 th percentile delta BSPM emission was approaching 2 p ercent of the threshold, then the simulation was continued for up to 65,000 trials.

2.10 Simulation Output

During t he s imulation of a r eference NTE e vent
, differences b etween the B SPM emissions “with errors” and the ideal BSPM emissions were obtained by each o f the three PEMS model uni ts a nd e ach of t he t hree applicable c alculation m ethods. T hese differences w ere computed thousands of times (once per trial) until the model converged

. Then the 95 th percentile difference value was determined for each reference NTE event’s distributions of BS differences for the PM emission s for all three PEMS units and applicable calculation method s.

The output from the Crystal Ball simulation for each reference NTE event was saved in two separate E xcel f iles: an EXTRACT and a REPORT f ile. T he E XTRACT f ile cont ained descriptive s tatistics on all di fferences c omputed f or B SPM e missions b y all thr ee c alculation methods, pe rcentiles ( 0%, 5% , 10% ,...95%, 100% ) of t he di fferences i n B SPM e missions,REPORT 03 .14936. 12 24 of 174 sensitivity data for all error surfaces, and differences in BSPM emissions computed at each trial in the simulation.

The REPORT file contained a summary of the differences in the BSPM emissions for all three PEMS units and applicable calculation methods including descriptive statistics, the number of trials, a frequency histogram of the differences in BSPM emissions, and percentiles (0%, 5%, 10%,...95%, 100% ) of the di fferences i n B SPM e missions. A lso i ncluded w ere de scriptive statistics on each i c distribution sampled for each error surface. Lastly, sensitivity charts for the dif ferences in BSPM emissions for the three PEMS unit and applicable calculation methods were stored. T hese c harts pr ovided i nformation on how m uch e ach e rror s urface i nfluenced t he differences computed between the BSPM em issions “w ith errors” a nd the i deal BSPM emissions. A m ore de tailed de scription of t he C rystal Ball out put f iles c an be f ound i n Appendix C.

2.11 Step

- by

- Step Simulation Example

I n o rder t o c larify t he s imulation pr ocess

, the f ollowing s tep

- by

- step summary is provided. This example assu mes that a single reference NTE event was simulated for the BSPM difference computations. Figure 9 provides an overview of the simulation process.

FIGURE 9 . OVERVIEW OF MONTE CARLO SIMULATION FOR BSPMREPORT 03 .14936. 12 25 of 174 S tep 1 - Enter the reference NTE input parameters into the Monte Carlo (MC) simulation model. T hese i nclude t he e missions c oncentrations, e xhaust f low, t orque, s peed and f uel r ate data used in all three calculation methods.

S tep 2 - Compute the “ideal” B SPM b y al l three P EMS m odel uni t and a pplicable calculation methods from the reference NTE event.

S tep 3 - Set

- up the M onte C arlo simulation parameters in Crystal B all. An Excel spreadsheet m odel was de veloped for us e w ith Oracle ® Crystal Ball M C s oftware for e rro r analysis of brake- specific emissions. Crystal B all is graphically

- oriented f orecasting a nd simulation software that runs on M icrosoft **** Windows and Excel. The simulations run in this program us ed C rystal Ball V ersion 11.1.1 a nd were run on P Cs c onfigured w ith a P entium 4 CPU, 3.39 G Hz, 3.50 G B R AM, 232 GB ha rd dr ive a nd W indows X P ope rating s ystem. Microsoft **** Excel 2003 SP was the spreadsheet software.

The options exercised in running Crystal Ball included the following:

**•** Number of trials = 40,000

o If convergence was not met at the 2% criteria then # trials = 65,000

**•** Monte Carlo sampling method with random initial seeds

**•** Normal speed run mode

**•** Suppress chart windows (fastest run time)

The Excel spreadsheet i s i n a m odular s tructure f ollowing t he s pecified m odel out line, a nd i t makes pr ovisions f or t he t hree i dentified c alculation m odules. Input c ells t o the m odel a re clearly id entified to f acilitate an y revisions t hat may become ne cessary f or us ers who w ant to exercise the model with other Monte Carlo software such as @Risk or newer versions of Crystal Ball . The spreadsheet was tested with controlled test cases of simplified input distributi ons with the C rystal Ball a dd

- on to confirm c orrect model impl ementation in accordance with this te st plan. A t least one typical analysis was run as an additional confirmation, and two independent checks were made on the ideal emissions by other SwRI staf f. A complete description of the spreadsheet computations is contained in Appendix D.

S tep 4 - Execute a single MC trial by randomly generating a separate i c for each error surface used in the three calculations.

S tep 5 - For each second in the reference NTE event, interpolate the **∆** error for all error surfaces at the input parameter values and the randomly generated i c Figure 10

. illustrates all the error s urfaces a vailable a nd w here t he corresponding **∆** errors are a dded. T he num bers i n parentheses represent the error surface number in the Monte Carlo simulation.REPORT 03 .14936. 12 26 of 174 FIGURE 10 . ERROR SURFACES INCLUDED IN MONTE CARLO SIMULATI ON

Step 6 - Compute one BSPM “with errors” for the given MC trial b y adding all the **∆** error values to the reference NTE data and then calculating the BSPM by all three PEMS units and applicable calculation methods.

Step 7 - Compute BSPM difference for the current trial:

BSPM emission “with errors” – “Ideal” BSPM emission

Step 8 - R epeat Steps 4

- 7 until the number of trials is met.

Step 9 - Check the di fferences i n BSPM f or all t hree P EMS uni ts a nd applicable calculation method s to be certain that the convergence criteria are met. If convergence is met for all three calculation methods, continue to Step 10. Otherwise, return to Step 4 and run the Monte Carlo simulation for an additional 25,000 trials until the total number of trials is 65,000.

Step 10 - Select the 95 th percentile from the distribution of BSPM differences for each of the three PEMS units and applicable calculation methods. S tore the ideal BSPM and the 95 th percentile BSPM differences for computing the measurement allowance.

Step 11 - Repeat Steps 1

- 10 for each reference NTE event. Environ

chambers

Engine
dyno

Environ

chambers Engine
dyno

PM **∆** PM SS_ppm (1)
**∆** PM TR_ppm (2)

**∆** PM press_ppm (4)

**∆** PM temp_ppm (5)

**Ε** xhflow **∆** Exhflow SS (20)
**∆** Exhflow TR (21)
**∆** Exhflow pulsation (22)
**∆** Exhflow swirl (23)

**∆** Exhflow EMI/RFI (25)

**∆** Exhflow temp (27)

**∆** Exhflow pressure (28) Speed **∆** Speed Dynamic (43)

**∆** Fuel Rate Dynamic (44) Fuel Rate

Torque **∆** Torque Dynamic (29)

**∆** Torque warmup (31)

**∆** Torque humidity (32)

**∆** Torque engine

- manuf (35)Engine dyno

**∆** F uel Rate engine

- manuf (42) Engine
dyno

CO **∆** CO SS_ppm (7)

**∆** CO press_ppm (10)
**∆** CO temp_ppm (11)

Engine
dyno

NMHC **∆** NMHC SS_ppm (13)
**∆** NMHC TR_ppm (14)

**∆** NMHC press_ppm (16)
**∆** NMHC temp_ppm (17)

**∆** NMHC ambient (19) Engine
dyno CO 2 **∆** CO2 SS_ppm (45)

**∆** CO2 TR_ppm (46)

**∆** CO2 temp_ppm (49) Environ

chambers

Environ

chambers **∆** Torque DOE (30)

**∆** Torque interpolation (34)REPORT 03 .14936. 12 27 of 174 2.12 Measurement Allowance Generation

The generation of a set of measurement allowances represented the final outcome of this program. T he Test Plan provided a methodology by which all of the data from the millions of Model simulation runs would be collected and analyzed statistically, in ord er to generate a set of potential m easurement a llowances f or each P EMS m odel uni t, one f or e ach of t he c alculation methods. T he Test Plan then outlined a specific method b y which the final set of allowances would be chosen from among deltas generated for each of the three calculation methods. T he assumption made by the Test Plan, was that the final outcome of all previous efforts would be a set of validated pot ential measurement al lowance va lues f or e ach PEMS uni t. E ach potential allowance w as ex pressed as a pe rcentage of i ts as sociated BSPM N TE t hreshold. T hese measurement allowances w ere computed b y a r egression m ethod or a m edian m ethod a s described below.

2.12.1 Regression Method

This method involved determining the correlation between the 95 th pe rcentile differences versus the ideal emission values for the reference NTE dataset. For each combination of PEMS units and calculation method, a least squares linear regression of the 95 th percentile differences versus the ideal emissions results was com puted. If the R 2 value from the regression model was greater than 0.85 and the SEE ( standard error of the estimate or root

- mean

- squared

- error) was less t han 5 percent of t he m edian ideal BS e missions, t hen t he linear r egression equation was used to determ ine the measure ment allowance for that PEMS unit and calculation method. To determine the measurement allowance , the NTE threshold was used to predict the measurement allowance f rom t he r egression model. T he m easurement al lowance was t hen expressed as a percentage of the NTE BSPM threshold value (0.02 g/hp- hr).

2.12.2 Median Method

If t he l inear r egression di d not pa ss t he a forementioned criteria for t he R 2 and SEE statistics, then the median value of the 95 th percentile differences from the 141 reference NTE
events was us ed as t he s ingle m easurement allowance f or a c ombination of e missions a nd calculation method. The measurement allowance was then expressed as a percentage of the NTE threshold value.

After a ll 95 th percentile dist ributions w ere e valuate d, there w ere s even measurement allow ances c orresponding t o t he combinations of t he t hree PEMS uni ts and t he applicable calculation methods.

Next , the calculation method with the minimum normalized PM value will be chosen and the corresponding normalized PM value will be selected as the best measurement allowance for PM, assuming it validates. This PM measurement allowance would be the very last value added to the act ual br ake

- specific N TE PM threshold f or a g iven e ngine, ba sed o n actual f amily emissions limit, mileage, model year, etc. N ote tha t if a ny m easuremen t al lowance w as determined to have a va lue l ess t han zero, then that m easurement al lowance w as s et equa l t o

zero.REPORT 03 .14936. 12 28 of 174 The BSPM NTE threshold used for this program was 0.02 g/hp

- hr. This NTE threshold was determined by EPA and approved by the Steering Committee during the generation of the Test Plan.

These threshold values are of critical importance to the program, as they provide the basis for the scaling of measurement allowances, the assessment of model convergence, and a variety of other calculations performed during this program. T he general philosophy of the Test Plan was to determine measurement allowances based on errors at these emission levels, especially in the case of any errors that scaled with emission level.

The a nticipated out come f rom t he m odel r uns, a nalysis, a nd va lidation efforts c an b e represented as a table similar to the one shown in Table 7. The table illustrates both the model outcome, a nd t he p rocess f or s electing t he f inal m easurement allowance va lues for t he AVL PEMS.

TABLE 7 . EXAMPLE OF SELECTI ON OF MEASUREMENT AL LOWANCE AT 0.02 G/HP

- HR NTE THRESHOLD FOR THE AVL PEMS

Allowance at Respective NTE threshold (%)

Calc. Method  Method 1 Exhaust Flow Torque

- Speed Method 2 Exhaust and Fuel Flow Torque

- Speed Method 3Fuel Flow TorqueSpeed
BSPM 38 % 18 % 20 %
Selected Method  Exhaust and Fuel Flow Torque

- Speed Method

The i ntent of t he f inal s election pr ocess was t o ch oose the s mallest of the thr ee normalized PM values for the final measurement allowance. At that point, the percentages given for the chosen calculation method would be applied to the BSPM NTE threshold value in order to generate the final additi ve, brake- specific measurement allowances.

An implicit assumption of the process, as described in the Test Plan, was that the values produced by the model for all three calculation methods would be successfully validated. In the event that this did not occur, it would be necessary for the Steering Committee to determine a valid alternate course of action, in order to determine the final measurement allowance values. The final model run and the selection and generation of measurement allowances are descri bed fully in Section 0 of this r eport, i ncluding t he f inal a llowances a pproved b y t he S teering Committee.

2.13 Model Validation

For reasons discussed earlier, the measurement allowances were generated using a Monte Carlo computer model. As with all simulations, it is vital that such a model be validated through comparison with real experimental data. In this case, the Measurement Allowance model needed to be va lidated against a da ta s et generated through actual i n

- use f ield t esting. B ecause t he model generates an incremental error in comparison to a Laboratory Reference, a suitable in -use reference measurement was needed for comp arison to the PEMS measurements. T he SteeringREPORT 03 .14936. 12 29 of 174 Committee de termined that the C E

- CERT M obile E mission L aboratory, op erated b y t he University of C alifornia

- Riverside, w ould be a n a ppropriate r eference f or va lidation of t he model- based in

- field testing.

To ensure that the validation was not disturbed by some inherent bias between the SwRI Reference Laboratory and the CE

- CERT MEL validation reference, a correlation exercise was performed between the two laboratories, prior to the start of on

- road validation e fforts. The CE

CERT MEL was brought to SwRI’s laboratory facilities in San Antonio, Texas, and a side

- by

side correlation test was run. During this test, exhaust from the same test engine was alternately routed t o t he m easurements s ystems of bot h S wRI and C E

- CERT. This w as don e r epeatedly over the course of three days of testing. The data was then supplied to the Steering Committee, in order to allow f or a determination to be ma de tha t c orrelation between the f acilities w as acceptable for the purposes o f validation of the model.

After the correlation exercise was completed, a 2007 test truck wit h a Cummins engine was procured by CE

- CERT for use in this validation exercise. I n addition, two Sensors PEMS used at SwRI during the program were also deliv ered to CE

- CERT. A third PEMS unit of the same t ype w as pr ovided b y S ensors. The s teering c ommittee allowed Sensors t o pr ovide a similar model with some small hardware upgrades for testing. CE

- CERT then conducted a series of on

- road test runs over various driving routes in California, which were designed to take the test truck through a wide range o f environmental and ambient conditions. D uring these tests, simultaneous m easurements w ere m ade w ith the P EMS and the M EL in order t o generate a validation data set. This formed the primary validation set for the model.

Because t he C E

- CERT M EL doe s not r eadily i ncorporate a m eans of di rect t orque measurement on a vehicle, the on

- road validation data set could not be used to validate model errors associated with broadcast torque.

The di fference be tween t he P EMS r esults and t he CE

- CERT trailer r esults w ill be compared to the measurement allowance limits predicted by the Monte Carlo Model and defined by the LOESS fit

. Validation will be based on the following procedure. For each reference NTE event, the Monte C arlo m od el w ill be us ed to generate t he 5 th and 95 th percentiles of t he s imulated distribution of t he br ake

- specific P M em ission differences. In o rder t o obt ain s imulations representing similar conditions to those obtained on

- road, some error surfaces may ne ed to be suppressed in the simulations since not all of them may be applicable to the on -road conditions. The c hoice of w hich error s urfaces t o s uppress w ould ne ed t o b e made b y t he S teering Committee.

Next, the 5 th and 95 th delta pe rcentiles obt ained f rom t he a bove s imulations w ill be separately fit to a line or curve using two chosen methods: a linear regression procedure and a local regression (loess) technique [4] . Depending on w hich of the resulting two fits is best for each set of data (i.e., either fo r the 5 th percentile deltas or the 95 th percentile deltas )

, the resulting line or curve will be used as one of the lower or upper limits for the on- road data.REPORT 03 .14936. 12 30 of 174 To determine the best fit for a given set of delta percentiles (i.e., 5 th or 95 th ), a simple regression line initially will be fit to the data. If a least squares linear regression of the 5 th or 95 th percentile deltas versus the idea l PM emission has an r 2 greater than 0.85 and an SEE less than 5 percent of the median ideal PM emissions , then the regression line will be used. If this set of criteria is not met, then a loess fit will be used

. Since a loess regression requires the selection of a smoothing parameter [5] to smooth the data, the chosen smoothness parameter should balance the residual sum of squares against the smoothness of the fit.

The on

- road delta e rrors, obtained from t he results of col lecting d ata on s everal N TE events during on

- road operations, will be plotted on a graph containing the 5 th and 95 th percentile delta limits determined from the regression fits chosen above. The graph will consist of a plot of delta PM versus ideal PM. The number of on

- road points outside these limits will be determined and expressed as a percentage of the total number on on

- road data points. If this number does not exceed 10% of the total number o f on

- road d ata, the simulation data will be considered to be valid.

An example of a validation plot is given in Figure 11 and Figure 12 . T he plots shown correspond to gaseous emissions concentration data th at were collected in the prior PEMS study

. Figure 11 contains the 5 th and 95 th validation limits for NO X Figure 12 data determined by fitting a linear regressio n model to the s imulated data for bot h limits. contains t he 5 th and 95 th
validation limits for NO X data determined by fitting a loess model to the simulated data for both limits

.

FIGURE 11 . LINEAR REGRESSION FIT TO 5TH AND 95TH PERCENTILE DELTASREPORT 03 .14936. 12 31 of 174 FIGURE 12 . LOESS REGRESSION FIT TO 5TH AND 95TH PERCENTILE DELTAS

Validation of the model was assessed independently for the PM pollutant for each of the three P EMS m odel uni ts, a nd f or e ach of t he applicable t hree c alculation m ethods. A full description of the validation efforts, including the data analysis methodology and the results of PM validation for each PEMS unit by all three calculation methods is given in Section 2.4, with the exception of the CE-CERT on road validation testing.REPORT 03 .14936. 12 32 of 174 3.0 PART 1065 PEMS AND LABORATORY AUDIT

Prior to the start of official testing both the laboratory system and each PEMS underwent an extensive audit in accordance with CFR Part 1065. Table 8 summarizes the audits that were performed on each type of system along with the CFR reference for each of the verifications.

TABLE 8. LINEARITY VERIFICATION RESULTS FOR INTAKE AIR FLOW AND FUEL FLOW

Fuel Flow Description |x min (a 1 - 1)+a o | a 1 SEE R 2
Measured

- 0.02 1.00 0.14 1.000 Criteria 0.55 0.98

- 1.02 1.10 0.990 Pass / Fail Pass Pass Pass Pass Intake Air Flow Description |x min (a 1 - 1)+a o | a 1 SEE R 2
Measured 0.00 1.00 1.22 1.000 Criteria 15.68 0.98

- 1.02 31.36 0.990 Pass / Fail Pass Pass Pass Pass

3.1 1065 Lab Audit

The most important audits performed on the laboratory system were those directly related to the P M conc entration measurement ac curacy, na mely t he l inearity of f lows and the P M balance. The i ntake ai r flow and fuel flow were verified for l inearity, while t he C VS and PM sampling flows were verified using a p ropane recovery check. The linearity verifications were performed in accordance with 40 C FR Part 1065.307 a lthough they were performed within 180 days r ather t han 370. T he pr opane c hecks w ere pe rformed w eekly dur ing of ficial t esting i n accordance with 40 CFR Part 1065.341. The maximum allowable interval for the propane check is 35 days. Table 8 shows the initial linearity verifications for fuel and intake air flow.

One additional verification was performed for the fuel and intake air flow with similar results. Two different nominal flow rates were used for the PM secondary dilution system, one for steady state testing and another for transient testing. A total nominal flow of 3.6 m3/hr (2.0 scfm) was used to target a filter face velocity of 100 cm/s during steady state testing. Because the filter measurement was unofficial the total flow was reduced during transient testing to prevent overloading the filter. A nominal total flow rate of 2.4 m 3/hr was used during transient testing. The propane recovery check was performed at a total secondary flow of either 3.6 or 2.4 m 3/hr depending on whether t he t esting at t he t ime w as s teady s tate or t ransient. Table 9 shows a

summa ry of t he pr opane r ecovery results. The pe rcent di fference i s b etween the cal culated propane concentration based on a kno wn f low of pr opane a nd t he m easured pr opane concentration from a hydrocarbon analyzer. The pass limit for the CVS system is plus or min us two percent and plus or minus five percent for the secondary sampling system.REPORT 03 .14936. 12 33 of 174 TABLE 9 . CVS PROPANE RECOVERY CHECK SUMMARY

Test Date CVS Blower,

m3/hr Secondary Dilution,

m3/hr Secondary Total, m3/hr Secondary Sample Diff, % CVS Diff, %

10/29/08 3,941 1.75 3.53

- 1.16 0.75 11/05/08 3,855 1.73 3.64 1.02 1.67 11/14/08 3,887 1.72 3.65 1.17 1.66 12/01/08 3,921 1.73 3.81

- 1.1 0.37 12/08/08 3,931 1.72 3.75 0.06 1.33 12/18/08 3,955 1.75 3.62

- 2.15 1.4 01/05/09 3,832 1.31 2.63

- 1.89

- 0.61 01/12/09 3,875 0.97 2.36

- 2.72

- 0.64 01/20/09 3,928 0.97 2.36

- 2.98

- 0.17 02/11/09 3,899 1.00 2.28

- 0.05

- 1.35 02/18/09 3,896 1.00 2.46 1.13 0.74 02/25/09 3,841 1.00 2.31 2.04 0.77 03/06/09 3,905 1.90 3.72 1.86 1.19 03/16/09 3,893 1.92 3.72 0.68

- 0.56 03/24/09 3,804 1.05 2.17 0.6 0.51 04/07/09 3,927 1.87 3.87 1.37 0.28

Occasionally a p ropane check was out side of t he al lowable l imits, but when the ch eck was r epeated i t us ually pa ssed unde r t he s ame c onditions. It w as ne cessary t o pa ss t wo consecutive propane checks if the initial check failed. The final result from each day is shown in the Table 9.

Linearity w as a lso verified on the P M b alance us ed for f ilter w eights. T he line arity verification results are shown in Table 10

.

TABLE 10. LI NEARITY VERIFICATION FOR PM BALANCE

PM Balance Description |x min (a 1 - 1)+a o | a 1 SEE R 2
Measured 0.00 1.00 0.00 1.000 Criteria 20.00 0.99

- 1.01 20.00 0.990 Pass / Fail Pass Pass Pass Pass

Verification of t he P M ba lance i s r equired e very 370 da ys although the che ck was performed after 180 days for quality purposes and passed with similar results.REPORT 03 .14936. 12 34 of 174 3.2 1065 PEMS A udit

Before the start of engine testing, each PEMS was required to pass the verifications set forth in CFR Part 1065. Because the measurement of PM is a non

- standard process there are no audits s pecified on the actual P M me asurement, but the ins truments s till n eeded to pass t he necessary flow, temperature, and pressure audits. Table 11 lists the Part 1065 audits performed on the lab and on the PEMS.

TABLE 11 . SUMMARY OF PART 1065 AUDITS

Description CFR Reference Lab PEMS Linearity 1065.307 x a x b Torque Meter 1065.310 x Pressure, Temperature, Dewpoint 1065.315 x Fuel Flow 1065.320 x Intake Flow 1065.325 x CVS Verification 1065.341 x PM Balance Verification 1065.390 x

a Linearity for lab performed on flow meters, torque meter, pressures, and temperatures
b Linearity for PEMS performed on flow meters

Since no analyzer ve rifications w ere p erformed, the mos t c ritical a udits on the P EMS were the linearity verifications of flow measurements. Because all three types of PEMS use flow measurements to calculate a di lution ratio, the a ccuracy of the f low me asurements is di rectly related to the accuracy of the reported PM emissions. The AVL and Horiba PEMS measure their total and dilution flow to calculate a dilution ratio while the Sensors PEMS measures the dilution and sample flow. The following equation is used to calculate the dilution ratio:

Dilution Ratio = Total Flow Sample Flow = Total Flow
Total Flow

− Dilution Flow

3.2.1 Horiba Flow Audits

The Horiba PEMS has four flow measurements in the system: dilution flow, DCS flow, make

- up air flow, and total flow. During typical operation all flows but the dilution flow are held constant. The dilution flow rate is varied to sample proportionally from the raw exhaust based on changes in exhaust flow. The CFR requires linearity verifications on s ample, dilution, and total flow (whichever two of the three that are measured) [2]. The DCS and makeup flows affect the accuracy of the dilution ratio and the filter flow, however both of these flows are maintained at a nominal value of 2 lpm it was decided not to perform a linearity v erification in this situation. The steering committee elected to perform a s pot check on the DCS flow to ensure its accuracy and verify the filter flow which includes both the total and make

- up air flow. The filter flow is equal to the difference between the total flow and make

- up air flow. B ecause the filter flow was designed to operate at a constant flow of approximately 28 lpm it was not practical or logical to verify the flow measurement over 10 e ven points down to zero flow as recommended in CFR Part 1065.307. After discussing the issue w ith Horiba and the steering committee it was decidedREPORT 03 .14936. 12 35 of 174 to verify the filter flow over a range of plus and minus ten percent of its operating target, using eleven steps. The maximum range that the dilution flow could vary was approximately 21 to 31 lpm; the d ilution flow was verified over this range in eleven even steps. The flow audits were initially performed with a TSI flowmeter, but this was replaced with a bubble flowmeter when it was discovered that the accuracy of the TSI flowmeter degraded as the press ure during the flow measurement deviated from atmospheric. The results from the linearity verifications are shown in Table 12 for the Horiba PEMS.

TABLE 12 . LINEARITY VERIFICATIONS FOR HORIBA PEM S

Verification D escription Intercept Slope SEE R 2
Horiba1 Dilution Flow Measured Linearity Criteria Pass/Fail 0.29 031 Pass 0.98 0.98-1.02 Pass 0.05 0.62 Pass 1.000 0.99 Pass Filter Flow Measured Linearity Criteria Pass/Fail 0.07 0.30 Pass 0.97 0.98-1.02 Fail 0.03 0.60 Pass 1.000 0.99 Pass Horiba2 Dilution Flow Measured Linearity Criteria Pass/Fail 0.08 0.31 Pass 1.00 0.98-1.02 Pass 0.3 0.61 Pass 0.999 0.99 Pass Filter Flow Measured Linearity Criteria Pass/Fail 0.18 0.30 Pass 0.98 0.98-1.02 Pass 0.03 0.60 Pass 1.000 0.99 Pass Horiba3 Dilution Flow Measured Linearity Criteria Pass/Fail 0.05 0.31 Pass 1.02 0.98-1.02 Pass 0.04 0.62 Pass 1.000 0.99 Pass Filter Flow Measured Linearity Criteria Pass/Fail 0.01 0.30 Pass 0.99 0.98-1.02 Pass 0.04 0.60 Pass 0.999 0.99 Pass

Horiba- 1 na rrowly m issed pa ssing l inearity ve rification for tot al f low w ith a s lope of 0.97. This check was repeated several times without passing. However, the error as a percent of point was better than 0.5 percent for all eleven points. This result is a problem with applying the linearity criteria f rom Part 1065 to a flow that is not verified over the range f rom z ero to full scale. Given the excellent agreement on a point

- by

- point basis, the steering committee elected to proceed without taking corrective action.REPORT 03 .14936. 12 36 of 174 The linearity verification results for the Sensors PEMS are shown in Table 1 3.

TABLE 13 . LINEARITY VERIFICATIONS FOR SENSORS PE MS

Verification Description Intercept Slope SEE R 2
Sensors1 Sample Flow Measured Linearity Criteria Pass/Fail - 0.01 0.02 Pass 1.00 0.98-1.02 Pass 0.0 0 0.04 Pass 1.000 0.99 Pass Major Dilution Flow Measured Linearity Criteria Pass/Fail 0.0 5 0.09 Pass 0.9 8 0.98-1.02 Pass 0.0 8 0.17 Pass 0 .999 0.99 Pass Minor Dilution Flow Measured Linearity Criteria Pass/Fail - 0.04 0.05 Pass 1.01 0.98-1.02 Pass 0.01 0.09 Pass 1.000 0.99 Pass Sensors 2 Sample Flow Measured Linearity Criteria Pass/Fail 0.0 1 0.01 Pass 1.00 0.98-1.02 Pass 0. 01 0.03 Pass 0.999 0.99 Pass Major Dilution Flow Measured Linearity Criteria Pass/Fail 0.07 0.07 Pass 1.01 0.98-1.02 Pass 0.0 5 0.13 Pass 0.999 0.99 Pass Minor Silution Flow Measured Linearity Criteria Pass/Fail 0. 02 0.04 Pass 1.01 0.98-1.02 Pass 0.0 6 0.08 Pass 0 .998 0.99 Pass Sensors 3 Sample Flow Measured Linearity Criteria Pass/Fail 0.0 1 0.01 Pass 1.0 0 0.98-1.02 Pass 0.0 1 0.02 Pass 0.999 0.99 Pass Major Dilution Flow Measured Linearity Criteria Pass/Fail 0.0 1 0.07 Pass 1.00 0.98-1.02 Pass 0.0 5 0.14 Pass 1.000 0.99 Pass Minor Dilution Flow Measured Linearity Criteria Pass/Fail 0.01 0.02 Pass 0.9 6 0.98-1.02 Fail 0.0 1 0.04 Pass 1.000 0.99 Pass

The Sensors PEMS had the capability of performing a self

- audit using 1065 criteria. An external T SI f lowmeter was pr ovided a s pa rt of t he S ensors e quipment a nd i ts m easurements were recorded by the Sensors software to linearity verifications on the dilution and sample flows. The total dilution flow is calculated b y the addition of the major and minor dilution flows, so these two measurements are audited independently. The Sensors PEMS was able to pass all butREPORT 03 .14936. 12 37 of 174 the m inor di lution f low a udit f or S ensors uni t 3. T his ve rification w as pe rformed r epeatedly without passing. Since the total dilution flow would still pass a linearity verification in this case, no further action was taken. In the case of both the Horiba and the Sensors, the absolute accuracy of the flows are important in determining the mass of PM emitted. The AVL s ystem is a real time particle sensor rather than a proportional batch sampler. For this type of instrument, only the di lution ratio a nd n ot t he a bsolute a ccuracy of t he t otal a nd di lution f lows a ffect t he measurement a ccuracy. For t his r eason t he di lution r atio w as a udited i nstead of t he t otal and dilution flow rates which are used to calculate the dilution ratio. It was also no t possible to vary the total flow, which is held constant during normal operation. Although the dilution ratio was maintained at a constant of 5 throughout official testing, a six point check was performed ranging from 2 t o 6 in steps of 1. Table 14 shows the results for the linearity verifications for the AVL units.

TABLE 14 . LINEARITY VERIFICATIONS FOR AVL PEMS

Verification Description Intercept Slope SEE R 2
AVL 1 Flow Based Dilution Ratio Measured Linearity Criteria Pass/Fail 0.0 1 0.07 Pass 0.9 6 0.98-1.02 Fail 0.0 1 0.15 Pass 1.000 0.99 Pass AVL 2 Flow Based Dilution Ratio Measured Linearity Criteria Pass/Fail 0. 03 0. 70 Pass 0.9 7 0.98-1.02 Fail 0.0 2 0.14 Pass 1.000 0.99 Pass AVL 3 Flow Based Dilution Ratio Measured Linearity Criteria Pass/Fail 0.0 5 0.07 Pass 1.02 0.98-1.02 Pass 0.0 0 0.14 Pass 1.000 0.99 Pass

The MSS dilution ratio was initially verified using both flow measurements as well as CO 2 measurements. Because the AVL PEMS had an internal CO 2 measurement it would have been pos sible t o us e a C O 2 span bot tle t o ve rify t he di lution r atio, unf ortunately i t w as not possible to ever introduce an undiluted CO 2 sample to CO 2 sensor to provide a span. When the sample is undiluted, the CO 2 cell is bypassed so that it does not measure. The results shown in Table 15 w ere generated us ing T SI flowmeters t o m easure t he t otal and di lution f low a nd calculate t he di lution ratio in the s ame m anner as t he P EMS. AVL

- 1 and A VL

- 2 w ere bot h unable t o pa ss t he s lope c riteria but w ere w ithin t hree pe rcent of poi nt a cross t he s ix po int verification. The s teering c ommittee a greed to accept the di lution r atio accuracy tol erance of three percent.

3.2.2 Exhaust Flow

Official linearity verifications were not c onducted on t he PEMS exhaust flow meters at SwRI. The steering committee decided that a calibration from the manufacturer was sufficient so long as the flow meter was within five percent of the lab during engine testing. The three flow meters tested from H oriba and the three from Sensors were all within found to be within fiveREPORT 03 .14936. 12 38 of 174 percent of the lab measurement. An unofficial linearity verification was performed using the data from the steady state engine testing. The exhaust flow measurement was averaged during each state measurement sample and compared with the laboratory measured value over the same time period (50

- 270 seconds depending on the engine condition). There were between 64 and 84 data points per exhaust flow meter. The linearity plots are shown in Figure 13 for PEMS -1, Figure 14 for PEMS -2, and Figure 15 for PEMS -3.

FIGURE 13 . LINEARITY CHECK ON PEMS

- 1 EXHAUST FLOW DURING STEADY

STATE ENGINE TESTING

FIGURE 14 . LINEARITY CHECK ON PEMS

- 2 EXHAUST FLOW DURING STEADY

STATE ENGINE TESTING y = 0.977x

- 16.188 R2 = 0.998 y = 0.995x

- 15.427 R2 = 0.999

400 500 600 700 800 900 1000 1100 1200

400 500 600 700 800 900 1000 1100 1200PEMS Exhaust Flow (m3/hr, 20°C)

Lab Exhaust Flow (m3/hr, 20 ° C)

Horiba

- 1 Sensors

- 1

y = 0.995x

- 17.619 R2 = 1.000 y = 1.021x

- 37.608 R2 = 0.997

400 500 600 700 800 900 1000 1100 1200

400 500 600 700 800 900 1000 1100 1200PEMS Exhaust Flow (m3/hr, 20°C)

Lab Exhaust Flow (m3/hr, 20 ° C)

Horiba

- 2 Sensors

- 2REPORT 03 .14936. 12 39 of 174 FIGURE 15 . LINEARITY CHECK ON PEMS

- 3 EXHAUST FLOW DURIN G STEADY

- STATE ENGINE TESTING

Each of t he s ix ex haust f low m eters w as able to pa ss t he s tandard e rror, s lope, a nd correlation coefficient c riteria f or a r aw exhaust f low m easurement s pecifi ed in CFR P art 1065.307 however only one was able to pass the intercept criteria (Sensors

- 3). This is likely due in part to the fact tha t the measurements did not extend below 470 m3/hr making it more difficult to pa ss a n i ntercept c riteria t hat a ssumes e venly spaced d ata poi nts e xtended dow n t o z ero. Conducting a linearity verification on the exhaust flow measurement during engine testing was for informational purposes only. 

### y = 0.986x

- 13.106 R2 = 0.999 y = 0.987x + 3.218 R2 = 0.996

400 500 600 700 800 900 1000 1100 1200

400 500 600 700 800 900 1000 1100 1200PEMS Exhaust Flow (m3/hr, 20°C)



### Lab Exhaust Flow (m3/hr, 20 ° C)

Horiba

- 3 Sensors

- 3REPORT 03 .14936. 12 40 of 174 4.0 ENGINE DYNAMOMETER TESTING AND RESULTS

4.1 Testing Objective

The purpose of the engine dynamometer testing was to characterize the bias and precision errors of t he P EMS dur ing s teady

- state and t ransient en g ine ope ration. D uring s teady

- state engine operation

, the PEMS measurements were compared with that of the CVS filter

- based PM measurement to characterize the bias in each of the three PEMS. The transient engine testing was used t o de termine t he p recision o f each PEMS b y quantifying the v ariability of the P EMS measurement over a series of repeated transient NTE events.

4.2 Experimental Setup

4.2.1 Engine and Sampling System

Preliminary t esting was pe rformed using a 6. 4 liter lig ht he avy

- duty di esel en gin e provided by Navistar, however the engine used to generate all official steady

- state and transient data was a 2007 V olvo MP7 provided b y Volvo Powertrain. T he test plan initially c alled for official dynamometer testing to be performed on two different engines, but funding constraints reduced official testing to a single engine. The Volvo MP7 had a displacement of 10.8 liters and

wa s rated at 280 kilowatts (375 horsepower). The engine wa s equipped with a variable geometry turbocharger (VGT) and a water- co oled high pressure exhaust gas recirculation (EGR) loop. The engine intake system was connected to a test

- cell water

- cooled intercooler. The engine is shown in Figure 16

.

FIGURE 16 . VOLVO MP7 INSTALLED IN A CVS TEST CELLREPORT 03 .14936. 12 41 of 174 The engine wa s also equipped with a close

- coupled diesel oxidation catalyst (DOC) and a diesel particulate filter (DPF) combination. For the purpose of producing higher PM emission levels

, a bypass was created around the aftertreatment system to allow an adjustable amount of exhaust f low a round t he D PF. A DOC w as a dded t o t he b ypass s o t hat a ll of t he P M i n t h e exhaust w ould pa ss t hrough an ox idation c atalyst s imulating a s cenario of a cracked D PF. A picture of the bypass is shown in Figure 17

.

FIGURE 17 . DPF BYPASS WITH DO C

The or iginal s tock af tertreatment w as l ocated in t he m ain leg of t he ex haust, while a separate catalyst was procured for the bypass leg. The DOC was 76 millimeters (six inches) in diameter with a length of 1 52 millimeters (12 inches). Three butterfly valves were placed in the exhaust s ystem t o c ontrol t he a mount of e xhaust pa ssing t hrough e ach l eg. T he D PF w a s regenerated via an exhaust fuel injection system. For all testing

, the b ypass was open to some degre e, how ever, the b ypass l eg w as cl osed when active regenerations w ere p erformed on the DPF. The DPF bypass went through multiple iterations until the proper PM level was achieved during steady

- state testing. A PM level of 0.025 g /hp

- hr was easily obtainable during transient cycles, however it w as extremely di fficult to obtain this s ame P M le vel dur ing s teady

- state engine operation. Table 1 5 lists the five different configurations of the bypass that were tested.REPORT 03 .14936. 12 42 of 174 TABLE 15 . LIST OF DPF BYPASS CONFIGURATIONS

Iteration Pipe Diameter Inlet Probe DOC Diameter Outlet Probe
1 3 No 3 1", Upstream
2 3 No 3 1", Downstream
3 3 3 ", Upstream None 1", Downstream
4 3 3 ", Upstream 6 1", Downstream 5 (Final) 4 3 ", Upstream 6 3", Downstream

In the final configuration (iteration 5 ), two butterfly valves in the main leg of the exhaust were completely closed with only a one or two millimeter gap between the valve and the exhaust pipe. This not only forced a majority of the exhaust through the bypass, but significantly raised the exhaust backpressure. Based on measurements upstream and downstream of the bypass with the AVL PEMS

, it was estimated that well over 50 percent of the exhaust was routed through the bypass in the final configuration. A diagram of the bypass system is shown in Figure 18

.

FIGURE 18 . SCHEMATIC OF ENGINE DYNAMOMETER EXPERI MENTAL SETUP

The schematic is not to scale so that some distances may app ear incorrectly. There are more t han 10 pipe diameters be tween t he m ixing poi nt of t he b ypass a nd t he f irst P EMS sampling pos ition f or t he S ensors S emtech P PMD. Additionally, there are approximately 10 pipe diameters b etween each of t he P EMS s ampling l ocations s o that an y f low di sturbancesREPORT 03 .14936. 12 43 of 174 caused by a different pitot tube or sample probe should not affect the other PEMS. The portion of exhaust between the first and the last sampling location is insulated to minimize the cooling of the e xhaust a nd t hermophoretic de position of pa rticles. T he l ast s ampling pr obe i s t hat of t he Horiba OBS

- TRPM
, which is just upstream of the entrance into the CVS tunnel. The gaseous PEMS units (Sensors Semtech DS and Horiba OBS

- 2200) were used only for data acquisition and t rigger s ignals. N o P EMS g aseous e missions w ere r ecorded dur ing t his pr ogram. T he Semtech DS was used with both the Sensors PPMD and the AVL MSS

, while the Horiba OBS

2200 was used in conjunction with the Horiba OBS- TRPM.

The outlet of the aftertreatment was routed to a constant volume sampling (CVS) tunnel for emissions measurement. The CVS consists of a positive displacement pump and an upstre am heat exchanger; it was maintained at a nominal flow rate of 3,740 m 3/hr ( 2,200 scfm ) for t his testing. The di lution a ir i s e xtracted t hrough a f ilter pa ck f rom a t emperature a nd hum idity controlled area. The po rtion of t he C VS t unnel t hat i s e xposed t o t he test c ell e nvironment upstream of the exhaust is insulated to prevent heating of the dilution air from the test cell. The particulate matter samples were extracted from the CVS as shown in Figure 18

. Intake air flow was measured by a laminar flow element (LFE) with a maximum flow of 1,700 m3/hr (1,000 scfm ). The LFE was oriented so that there we re 10 diameters of straight pipe before the inlet a nd after the outlet to minimize flow disturbances. The fuel flow was measured by a M icro

- Motion flow me ter. The addition of i ntake a ir f low a nd f uel f low w as us ed t o determine the exhaust flow using the equations from CFR Part 89. The exhaust flow was u sed to calculate t he exhaust P M conc entration from t he C VS f ilter as w ell as a che ck on t he P EMS exhaust f low m eters. A r aw C O 2 analyzer w as us ed i n c ombination w ith t he e xhaust f low measurement t o calculate a r aw carbon balance f uel f low w hich was com pare d with t he measured fuel flow and the dilute carbon balance fuel flow as a quality check.

The PM sampling system consisted of a 47mm teflon membrane filter (Whatman Teflo), a fine metal screen backing, and a plastic filter cartridge. The total flow is operated at a nominal flow of 60 s tandard l iters pe r m inute ( 2.1 s cfm) w hich r esults i n a filter f ace ve locity of approximately 100 cm/s. The standard temperature and pressure used for all flow rates in this report is 20 °C and 101.325 kPa as specified in CFR Part 1065. The nominal dilution flow is 30 slpm (1.1 scfm) which resulted in a dilution ratio of 2 and a sample flow of 30 slpm (1.1 scfm). The system maintained a constant dilution ratio and achieved proportionality through sampling from the CVS. The s ys tem was heated to 47°C and had an approximate residence time of 0. 8 seconds from the inlet of the sample probe to the filter. A cyclone with a 2.5 micron cutpoint at 17 slpm was positioned just downstream of the sample probe. The aforementioned PM sampling system was the laboratory reference used to generate all reference PM data used in this program.

4.2.2 Sensors PPMD

The PPMD was the PEMS unit installed closest to the outlet of the bypass, approximately 15 diameters downstream. An experiment was conducted to ensure the flow was fully mixed at this location as will be discussed later. The PPMD was installed in the horiz ontal orientation as shown in Figure 19

.REPORT 03 .14936. 12 44 of 174 FIGURE 19 . PPMD INSTALLED IN THE TEST CELL

The P PMD w as e quipped w ith a l ong s tra ight pipe c ontaining its e xhaust f low me ter (EFM) just upstream of the 90 degree elbow from which the sample is extracted. The dilution in the S ensors s ystem t akes pl ace i nside t he i nstrument. T he m ost c ommon i nstallation of t he PPMD is in the vertical pos ition, although it requires only the rotation of the moisture traps to properly operate the PPMD in the horizontal orientation. The Sensors PEMS was equipped with two s tages of di lution know n a s M PS1 a nd MPS2. A lthough s ome pr eliminary t esting w as conducted using both stages of dilution, the steering committee decided to use only a single stage of di lution for all of ficial te sts. The P PMD is a pr oportional s ampling system tha t va ries its dilution ratio inversely with exhaust f low to maintain a mini mum di lut ion r atio of 6 a t t he maximum e xhaust f low r ate o f a n e ngine. T he P PMD me asures P M u sing a Q uartz C rystal Microbalance (QCM), which charges the particles using a corona needle, deposits the particle on a Quartz Crystal, and then measures the change in frequency of the crystal to determine the mass deposited. The PPMD is a batch sampling device meaning it does not report PM concentration in real

- time but instead reports a s ingle mass value per event. Because each crystal requires a pr e and post frequency mea surement to determine mass, a total of eight crystals are included to allow for continuous op eration b y switching crystals. Crystal s ampling b egins as soon as the engine enters the NTE zone and stops as soon as the engine exits the NTE zone. One of the of the eight crystals was used as a r eference crystal to adjust the measurements for changes in temperature and pressure. This left up to seven crystals available for measurement although it was common to have one or two crystals not working on any given test . The timing of the samples during testing was designed around having a minimum of five working crystals available for measurement. The PPMD was included in the measurement allowance program because inertial microbalances had already been approved for PEMS applications in 40 CFR Part 1065.REPORT 03 .14936. 12 45 of 174 4.2.3 AVL MSS

The AVL Micro Soot Sensor (MSS, also known as the Photo Acoustic Soot Sensor or PASS) w as i nstalled dow nstream of t he P PMD i n t he m iddle of t he ve rtical por tion of t he exhaust pipe leading to the CVS tunnel. The MSS is connected to the tunnel via a 2 meter heated sample line which was maintained at 52°C. The mixing of dilution air takes place in the dilution box just upstream of the sample probe so that the dilute sample is transported through the samp le line. The MSS is shown in Figure 20

.

FIGURE 20 . THE AVL MSS

The MSS consists of two boxes shown in the above figure. The top box is the measuring unit w hich c ontains t he r esonance c hamber f or t he s oot m easurement. T he bot tom box is t he conditioning unit which contains the s ample and dilution pumps as w ell flow controllers. The dilution pump is optional as the MSS can also provide dilution air via an external input of 300 kPa of compressed air. The steering committee requested that the MSS operate using its internal dilution pump, since this is the way it would operate during in use testing. The MSS measures soot by heating the elemental carbon using a laser. When the soot is heated it emits a sound wave that is detected by a microphone. The MSS can report soot concentration on a 1Hz basis and uses a constant dilution ratio, which was set at 5 for all official testing. Because the MSS measures only soot and not total PM, it was included in this program as a partial participant. If both the Sensors and Horiba units could not complete the measurement allowance program it was to be considered for in

- use. The AVL system does not have its own gaseous PEMS; or data storage device; i nstead i t de pends on t he g aseous i nformation f rom t he S emtech D S a nd sends i ts concentration signal to the Sensors Semtech DS where the necessary information is recorded. For all testing in this program a s ingle S emtech D S w as us ed to record the s ignals from bot h theREPORT 03 .14936. 12 46 of 174 Sensors S emtech P PMD a s w ell a s t he A VL M SS. T he pr obe f or t he Horiba O BS

- TRPM i s located approximately 1.65 meters (5 feet) downstrea m of the MSS. The Horiba system contains two separate exhaust pieces that are each about 150 mm (6 inches long). The upstream portion is a tail pipe adapter (TPA) which is a pitot tube exhaust flow measurement. The downstream pipe contained the probe for the PM sampling. The TPA and sampling probe can be seen in Figure 21

.

FIGURE 21 . THE PROBE AND TPA FOR THE HORIBA OBS

- TRPM

4.2.4 Horiba TRPM

The Horiba system is a proportional sampling device that varies its dilution ratio in the same manner as the PPMD. The dilution air is introduced just downstream of the probe before the heated sample line. The point of dilution can be seen in Figure 21 where the three stainless steel lines converge into the stainless steel cylinder. The OBS

- TRPM uses a T SI EAD (referred to here as a DCS) real time particle instrument to measure the particle concentration on a second by second basis and collects PM on a gravimetric filter simultaneously. The filter weight gain is used to provide a calibration constant to the real time particle signal and apportion the PM mass appropriately. The DCS instrument measures continuously, but the filter is designed to sample during v alid N TE event ope ration from t he s ame di luted exhaust s tream . The f ilter s ampling begins after five seconds in the NTE zone and will continue for a minimum sample time of 30 seconds e ven i f t he engine i s no l onger i n t he NTE z one. Because t he E PA’s P M s tandard i s based on gr avimetric filter analysis, the Horiba system was included in the program. The OBS

TRPM is comprised of several different boxes including the heated enclosure (HE), the diffusion charge sensors (DCS), the electrical enclosure (EE), and the mechanical enclosure (ME). The HE contains the 47mm filter holder, and the DCS is the real time particle sensor. Dilution air wasREPORT 03 .14936. 12 47 of 174 provided using a com mercially av ailable oi l

- less c ompressor. T he H oriba O BS

- 2200 ga seous PEMS was used to log the ECM J1939 broadcast, measure the exhaust flow, and provide an NTE trigger to the TRPM to start filter sampling. The combined TRPM

- 2200 sys tem contains a total of 6 box es plus an external compressor. T he components of the Horiba system are depicted in Figure 22

.

FIGURE 22 . THE HORIBA OBS

- TRPM AND OBS -2200

The TRPM used the same 47mm Whatman teflo filter, metal screen, and plastic cartridge as the CVS system. All weighing and conditioning of both the CVS and the TRPM filters was conducted i n t he S wRI f ilter w eighing r oom. The f ilter w eighing r oom is ma intained at a temperature of 22 ± 1°C with a dewpoint of 9.5 ± 1°C in accordance with CFR Part 1065.190. Filters were stabilized in the weighing environment for at least 1 hour prior to both the initi al and final weights. Each filter was weight was determined by the average of three weights on a scale with a resolution of 0.1 μg.

4.3 Bypass Mixing Verification

The flow from the DPF bypass was reintroduced into the main exhaust stream using a 76

mm (3 i nch) pr obe f acing dow nstream w ith a n or ifice ne ar t he t ip of the pr obe t o pr omote mixing. T esting w as c onducted t o e nsure t he e xhaust f low w as f ully mixed pr ior t o t he f irst sampling l ocation, w hich w as oc cupied b y t he Sensors P EMS. T wo pr obe or ientations w ere created at the spot where the Sensors sample was extracted. One of the probe orientations was parallel to the upstream exhaust elbow and one was perpendicular to the elbow. The orientation of the AVL sample probe is shown in Figure 23

.REPORT 03 .14936. 12 48 of 174 FIGURE 23 . EXPERIMENTAL SETUP FOR MIXING VERIFICAT ION

At e ach of t he pr obe orientations t he M SS w as us ed t o m easure t he e xhaust s oot concentration using a variable length probe that could traverse the length of the 127 mm (5 inch) exhaust pipe. There were five sample locations for each orientation as shown in Figure 24

.

FIGURE 24 . MIXING VERIFICATION SAMPLE LOCATIONS

Each of t he pos itions w ere one i nch apart, w ith pos ition 3,8 being a t t he c enter of t he exhaust pipe. For a measurement, the probe was started at location 1, moving down to 5 and then back t o 1. T he pr obe w as t hen m oved t o pos ition 6 m oving dow n t o p osition 10 a nd ba ck t o position 6. T he M SS m easurement w as r ecorded f or 80 s econds a t e ach pos ition, w ith t he average of the last 30 seconds used for comparison. The steady

- state modes with the highest andREPORT 03 .14936. 12 49 of 174 lowest e xhaust f low r ates w ere c hosen t o p erform t he m ixing ve rification, t o e nsure pr oper mixing over the entire range of test conditions.

The mixing verification was performed several times with different bypass config urations with similar results in all cases. Only the results from the final mixing verification are presented since the other results from different configurations are not relevant to the data in this report. The data from the final verification is present ed in Figure 25

.

FIGURE 25 . MIXING VERIFICATION RESULTS

The da ta i s pr esented as t he r atio of t he average conc entration at ea ch location to the centerline concentration. The data was normalized in this way not only to remove some of the engine va riability but also s o t hat t he t wo exhaust f low c onditions w ith di fferent s oot concentrations could be compared on t he same plot. Each data point represents the average of two samples taken at each location. The highest deviation from the centerline concentration was four percent at location 10, low flow which was considered acceptable, especially given that no trends were observed i n the data. A paired t

- test using equal variance was also performed on the data relative to the centerline position for the parallel orientation using the high and low flow. Locations 6 t h r ough 10 pa ssed t he t

- test, e xcept f or l ocation 10 a t low f low, a lthough t he difference in the mean value between locations 10 a nd 7 was less than 4 percent. The steering committee was s atisfied w ith the mix ing r esults and directed SwRI to move f orward with the program without any further modifications for the bypass setup. 0.0 0.2 0.4 0.6 0.8 1.0 1.2

1 2 3 4 5 6 7 8 9 10Concentration/Centerline Concentration

High Flow Low FlowREPORT 03 .14936. 12 50 of 174 4.4 PEMS Loss Corrections

Each P EMS m anufacturer was given t he o pportunity t o c orrect t heir f inal P M measurement t o account f or v arious pa rticle l osses e ncountered du ring the s ampling p rocess. Any l oss c orrection ha d t o be pr esented t o t he s teering committee f or a pproval be fore i t w as implemented in the program. Sensors and AVL both chose to implement loss corrections, while Horiba declined to apply a loss correction to their data.

4.4.1 Sensors PPMD Loss Correction

Sensors c onducted w ork unde r a separate pr oject at S wRI t o experimentally as sess t he losses in the PPMD. David Booker presented the results of this work along with the proposed Sensors loss corrections at the meeting on D ecember 10 th , 2008 a t SwRI. The final PPMD loss correction included thermophoretic, electrostatic, and CVS loss factors. The CVS loss correction factor was meant to estimate the particle losses in the CVS system, since this is the standard to which t he P PMD i s c ompared. A lthough t ypical l oss c orrection factors w ill i ncr ease t he estimated PM concentration, the CVS correction factor actually decreased the PPMD estimated concentration. The S ensors s trategy was to use the th ermophoretic and electrostatic los s corrections to determine what the true PM concentration is and then reduce that number by the amount of PM mass they believe will be lost in the CVS system. They did not wish to merely adjust to the correct concentration since the CVS system to which their instrument was compared did not correct for losses. A total CVS loss of 15 pe rcent was estimated by Sensors based on general experience rather than specific data. The total loss correction was estimated to increase the PM concentration by 5 to 10 percent when including the electrostatic and thermophoretic loss factors. The proposed loss corrections were accepted by the steering committee and implemented in t he S ensors P PMD post pr ocessor. A ll S ensors da ta i n t his r eport i ncludes t hese c orrection factors unless otherwise stated.

4.4.2 AVL MSS Loss Correction

The proposed AVL loss correction was presented at the meeting on November 12 th , 2008 at S wRI. The los s c orrection implemented by A VL w as int ended to correct f or the thermophoretic losses in the system and is based off a paper by Stratmann et al [6] . The equation fo r the loss correction is shown below:


## ( ) ( )( )( ) dttTLtqtt mss M

ex ex soot **+ ∆+=** 



## ∫ 1)(


## ( ) ** **  ** **  ** ** 

**+ <** 

**=** otherwise 300150 150 T if 0 )/b(T-a TL

The m agnitude of t he correction i s t emperature de pendent a nd w as e stimated t o be approximately 10 percent in most cases. This correction was accepted by the steering committee and implemented in the AVL Concerto post processor.

At the meeting at SwRI on December 10 th , 2008, AVL stated that their loss correction was currently capped at a maximum loss of 25 percent regardless of the calculated value. AVLREPORT 03 .14936. 12 51 of 174 requested approval to remove this limitation and allow the equations output to be the correction,
regar dless of its magnitude. This change was accepted by the steering committee.

4.4.3 AVL MSS Total PM Correction

The AVL Concerto post processor includes a function that estimates the total PM based on t he m easured s oot, e xhaust t emperature, a nd total h ydrocarbon c oncentration, a long w ith a number of a djustable i nput pa rameters i ncluding t he vol ume of t he catalyst, t he l ight o ff temperature of the catalyst and the sulfur content of the fuel. Although this calculation was not approved for use with the offic ial measurement allowance data, a portion of the AVL data was processed to examine the results from the total PM model. All AVL data presented in this report refers to the soot concentration corrected for losses unless otherwise stated.

4.5 Steady

- State T esting Procedure

Originally the test plan called for two different engines at three different emission levels of P M: D PF out , 0.02 g/hp

- hr l evel, a nd a 0.03 g /hp

- hr le vel. The D PF out le vel is s imply whatever the emissions happen to be with no bypass which was well below the 2007 standard of 0.01 g /hp

- hr. T he 0.02 and 0.03 g /hp

- hr l evels would be s et b y adjusting t he D PF b ypass t o produce t he corresponding b rake

- specific P M n umber f rom t he C VS filter. D ue t o f unding limitations the testing was reduced to a single engine at a single emission level. Because 0.03 g/hp

- hr will be used for the first year of compliance testing and 0.02 g/hp

- hr is used with the following years, i t w as i mportant t o i nvestigate t he pe rformance of t he P EMS c overing t hese levels . For this reason an average of the two threshold PM levels, 0.025 g/hp

- hr, was used as the target.

The obj ective of t he s teady

- state t esting w as t o evaluate t he bi as and precision of t he PEMS using 180 data points for each PEMS manufacturer. The 30 points co nsists of six steady

state modes of engine operation (6), 10 repeats (10), one emission level (1), one engine (1), and three different PEMS units (3), 6*10*1*1*3=180. A PM steady

- state error surface, 

# ( ) g SS PM mol Δ m , was de veloped for each P EMS m anufacturer s o that t here a re t hree steady

- state PM e rror surfaces for use with calculation methods 1 and 2. For calculation method 3, the AVL MSS will have a unique SS PM 

### Δ m based o n the c alculations for m ethod 3. A s m entioned previously, the Sensors and Horiba PEMS will only use methods 1 a nd 2. T o determine the most suitable six steady engine modes for steady

- state testing screening tests were performed using the 80 points Cummins cycle and measuring the PM levels with the AVL MSS and the TSI Engine Exhaust Particle S izer ( EEPS). The E EPS pr imarily pr ovides s ize di stribution information, but ma ss concentration can be inferred using an assumed density. Since the AVL is the only PEMS that can report a real t ime mass concentration without further efforts such as filter weighing and post processing it was chosen to perform the screening work. The Cummins cycle steps through 80 steady

- state engine modes as a transient cycle. The engine is stepped through 10 diff erent speeds and eight en gine l oads a t each of t he s elected speeds. The m inimum s peed of t he c ycle i s t he minimum NTE speed, and the minimum torque is 30 pe rcent of the torque at the given speed meaning that the cycle effectively maps the NTE zone. Figure 26 shows the speed and torque points of the cycle.REPORT 03 .14936. 12 52 of 174 FIGURE 26 . SPEED AND LOAD FOR THE 80 POINTS CYCLE

The engine remained at each of the points for 88 seconds with a one second transition between modes. Because the speed and load were only incrementally changed in between each mode, it was assumed that each point would stabilize relatively quickly. The fac t that this cycle was us ed onl y as a m ethod of s creening a lso contributed t o t he de cision t o c hange m odes quickly. Even with such short modes, the total cycle length was 2 hour s. To allow for purging, and calibrating the gas analyzers, the cycle was divided into two 40 poi nts cycles each lasting one hour. The gas analyzers were necessary because this cycle was also used to tabulate ECM fuel rate errors for this error surface.

The PM emissions were estimated by measuring each of the 80 modes with the EEPS and MSS, then choosing two of the modes to perform a filter measurement and compare the ratio of the filter measurement to the EEPS and MSS as an estimate of the CVS filter BSPM at each of the 80 m odes. Although this method is not highly accurate, it provided a way to quickly obtain rough estimates of the engine PM levels over a wide range of speed and torque conditions. Of the initial 80 points, only 12 were estimated to produce brake

- specific PM of greater than 0.02 g/hp

hr. The final six modes were chosen with the intent of covering as much of the NTE z one as possible while still maintaining high BSPM levels, and a range of PM exhaust concentrations. Several iterations of adjusting the bypass, taking filter measurements, and narrowing down the number of points occurred before the steering committee approved the final six points for steady

state testing. Originally the DPF bypass was adjusted to produce the 0.025 g/hp

- hr of PM based on a filter measurement during a short version of the NTE transient cycle. The PM emissions at steady

- state were much lower at the same bypass setting. Since it was desirable to conduct the steady

- state testing at the same PM levels as the transient testing, it was necessary to adjust the system to allow a much greater amount of the flow through the bypass. Six points were chosen 

### 0 20 40 60 80 100 120

0 20 40 60 80 100 120Engine Torque (%)



### Engine Speed (%)REPORT 03 .14936. 12 53 of 174 out of t he t welve t o p rovide a range of P M concentration, exhaust f low r ates, a nd engine operating conditions. The six points that were chosen are shown in Figure 27 along with the NTE

zone.

FIGURE 27 . FINAL SIX STEADY

- STATE MODES

Although t he M SS w as used t o s creen t he 80 p oints, t he a ctual c oncentration a t e ach
p osition was verified with CVS filter measurements before selecting the points. The exhaust PM concentration w as calculated b y m ultiplying t he CVS P M c oncentration by t he C VS di lution ratio. T he C VS di lution r atio w as c alculated b y di viding t he a verage C VS flow r ate b y t he average exhaust flow rate. The CVS dilution ratio ranged from 3 t o 7.5 r esulting in an overall dilution ratio of 6 to 15 when including the secondary PM filter dilution.

The steady

- state testing was conduc ted as a modal t ransient c ycle with e ach m ode repeated twice for a total of 12 modes per cycle. 6 different cycles were created with the order of the m odes r andomized in each cycle. Table 16 lists t he s ample or der of t he m odes i n t he 6 steady

- state cycles. 

### 0 500 1000 1500 2000 2500

500 1000 1500 2000 2500Engine Torque (Nm)



### Engine Speed (rpm)

Lug Curve NTE S Line NTE T

- S Line NTE P

- S Line SS Points Mode 1 Mode 2

Mode 3

Mode 4 Mode 5 Mode 6REPORT 03 .14936. 12 54 of 174 TABLE 16. SAMPLE ORDER FOR STEADY

- STATE CYCLE TESTING

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Sample 1 3 1 3 3 4 6 Sample 2 6 4 1 4 2 1 Sample 3 5 3 6 1 1 4 Sample 4 2 6 2 2 5 3 Sample 5 4 5 5 6 3 5 Sample 6 1 2 4 5 6 2 Sample 7 5 2 4 6 5 4 Sample 8 2 1 2 4 3 1 Sample 9 1 4 3 1 4 6 Sample 10 3 5 1 2 6 3 Sample 11 6 6 5 5 2 5 Sample 12 4 3 6 3 1 2

The steady

- state testing was conduc ted as a ramped modal c ycle w ith the eng ine remaining at each operating condition for three minutes before the start of sampling. An external trigger f rom t he l ab w as pr ovided t o e ach of t he P EMS a nd t he C VS f ilter s ystem s o t hat samplin g w ould be gin simultaneously for a ll i nstruments. T he e ngine t hen r emained a t t he operating condition for 5 seconds after the end of sampling to ensure that no delay in the end of sampling by any of the PEMS caused part of the transition period to be captu red as a s ample. The engine remained at the condition for five seconds after sampling had finished to ensure all systems had finished sampling before the operating condition changed. The order of the modes was randomized and each mode was repeated twice wi thin a single cycle for a total of 12 da ta points for each cycle. Six different c ycles were created, which would create a total of 72 data points. Although the target was only 60 valid points for each set of PEMS, in practice several cycles h ad to be r epea ted to collect enough v alid data. An example o f o ne of the steady

- state cycles is shown in Figure 28

.REPORT 03 .14936. 12 55 of 174 FIGURE 28 . EXAMPLE OF STEADY

- STATE CYCLE

To ensure a comparably accurate filter weight, a filter weight gain of 100 μg was targeted for each mode. By collecting this amount of material on each filter, the weighing variability and tunnel background contribution could be mini mized. Collecting more material than this for each steady state sample would have caused problems with the Horiba and Sensors PEMS by limiting the amount of time they could operate before switching filters or cleaning crystals. To produce a similar f ilter weight ga in at s ix di fferent s teady

- state m odes w ith different m ass r ates, the sampling time was adjusted for each mode to meet this target. The sample time ranged from 50 seconds to 245 seconds. Since the sampling time for each mode was different, the total length of time spent at each mode was different as well. The sample time for each mode for each round of the PEMS is shown in Figure 29

. 

### 0 500 1000 1500 2000 2500

0 500 1000 1500 2000 2500 3000 3500 4000Torque (Nm), Engine Speed (rpm)



### Time (sec)

Speed TorqueREPORT 03 .14936. 12 56 of 174 FIGURE 29 . STEADY

- STATE SAMPLE TIMES

Figure 30 shows t he ave rage f ilter w eight gain for al l s ix modes f or each of t he t hree PEMS.

FIGURE 30 . CVS FILTER WEIGHT GAIN FOR STEADY

- STATE TESTING 

### Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6

PEMS 1 245 70 125 190 160 100

PEMS 2 240 50 100 175 90 60

PEMS 3 240 50 100 175 90 60 0 50 100 150 200 250Sample Time (sec)



### 0 20 40 60 80 100 120 140

Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6CVS Filter Weight Gain (μg)



### PEMS 1 PEMS 2 PEMS 3REPORT 03 .14936. 12 57 of 174 Before presenting any of the PM concentration data it should be mentioned the choice of units i n t his w ork. T he t est pl an c alled f or t he PM c oncentration t o be c alculated i n t erms of μg/mol. Although it is an uncommon unit for describing mass concentration it was considered for f undamental t o us e m ol f or vol ume r ather t han m 3 i n w hich case a s tandard r eference condition must be defined. For reference, 1 m g/m3 of air at 2 0° C and 101.325 kPa is equal to 24.0 55 μg/mol. In several cases the values in mg/m3 are provided in parenthesis for reference, but all official data was calculated and plotted using μg/mol.

The preliminary steady

- state results from PEMS 1 were presented at the December 11 th

Figure 31 , 2008 meeting at SwRI. After reviewing the first set of steady

- state data, the steering committee felt t hat t he c oncentrations f rom t he s ix steady

- state points w ere not e ffectively c overing t he desired range. Five of the modes are clustered between 115 and 161 μg/mol (4.8 and 6.7 mg/m3 ) with the remaining mode at 325 μg/mol ( 13.5 mg/m3 ) . At the recommendation of the steering committee the bypass setting was slightly adjusted for PEMS 2 and PEMS 3 in an attempt to fill in s ome of t he r eg ion be tween 161 and 325 μ g/mol . D ue t o a s hift in t he e ngine out P M emissions, i t w as pos sible t o i ncrease t he c oncentration f or m odes 2, 3, 5, a nd 6 w hile maintaining the same levels for modes 1 a nd 4. I n fact, the dampers were actually adjusted to flow l ess exhaust through the b ypass indicating t hat the engine out PM h ad not onl y changed relatively between operating conditions but increased overall. shows the median CVS filter PM concentration for each of the three sets of PEMS.

FIGURE 31. STEADY

- STATE EXHAUST PM CONCENTRATION (μG/MOL) 

### 0 50 100 150 200 250 300 350 400 450 500

Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6PM Concentration, CVS Filter (μg/mol)



### PEMS1 PEMS2 PEMS3REPORT 03 .14936. 12 58 of 174 The m ode w ith t he hi ghest c oncentration, m ode 2, i ncreased up t o ne arly 423 μg/mol while the lowest concentration increased from 115 μg/mol on mode 5 to 132 μg/mol on mode 4. It i s i nteresting t o not e that m ode 5 and 6 i ncreased i n c oncentration b y 87 a nd 64 pe rcent, respectively while mode 4 increased by less than 2 percent. This increase in concentration came while opening the valve in the DPF leg of the exhaust, thereby increasing the flow of exhaust through the DPF. Modes 2 and 3 both shifted downwards between PEMS 2 and PEMS 3 without any c hange i n t he e xhaust va lve pos itions

. Figure 32 shows t he br ake

- specific P M va lues as measured by the lab reference for all three sets of PEMS.

FIGURE 32 . STEA DY

- STATE BRAKE

- SPECIFIC PM, CVS FIL TER (MG/HP

- HR)

The brake -specific PM ranged from 15.7 mg/hp-hr to 43.5 mg/hp- hr.

4.6 Data Yield During Steady

- State Testing

The test plan called for a minimum of 10 valid data points for each mode for each PEMS, allowing for a minimum data set of 30 for each 5 th and 95 th 

### 0 5 10 15 20 25 30 35 40 45 50

Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6Brake-Specific PM, CVS Filter (mg/hp-hr)



### PEMS1 PEMS2 PEMS3

percentile delta that was generated. In reality the target of 10 data points per PEMS per mode was not met in all cases due to additional points that were invalidated by post processing software that had been updated after the testing had been completed. While very few data points were removed during post processing for the Horiba a nd AVL P EMS, S ensors s upplied S wRI w ith s everal n ew po st pr ocessors after t he completion of te sting tha t inva lidated a significant por tion of t he S ensors da ta. S ince t he information on what criteria would invalidate the data was not available at the time of testing, it was not possible to know how many additional tests would be required to achieve the necessary number of data points. The Sensor’s post processor was revised to include some points that were deemed va lid d ata but h ad be en excluded b y t he pos t pr ocessor. T he f inal s et of da ta f or t he Sensors PEMS had between 28 and 34 poi nts per mode. Figure 33 s hows the numb er of valid data points for steady- stat testing.REPORT 03 .14936. 12 59 of 174 FIGURE 33. NUMBER OF VALID DATA POINTS FOR STEADY

- STATE TESTING

The t otal num ber of pos sible da ta poi nts w as e ither 40 or 41 de pending on t he m ode. Samples were taken 41 times at each mode, but modes 2, 3, 4, and 6 each had one point excluded due to mishandling of the CVS filter.

4.6.1 Data Yield During Steady- State Testing

Table 17 shows t he s teady

- state d ata yield by each of t he P M

- PEMS r elative to the possible data yield obtained by the CVS. A total of 29 steady

- state cycles were conducted for all three PEMS with each of the six steady

- state modes repeated twice in each cycle for a total of 12 data poi nts pe r cycle. Twenty -one of t hese cycles w ere cons idered valid tests f rom t he perspective of the function of the cycle command, NTE external trigger, filter sampling, and at least one or m ore of t he P EMS c apturing v alid da ta. D uring t hese va lid t ests f our da ta poi nts were m issed due t o a m ishandling of t he C VS f ilter, but t he r est w ere considered v alid da ta points from the perspective of the lab measurements. Data from the PEMS was removed for a variety of reasons i n cluding pr oblems w ith t he da talogging, s ampling, e xhaust f low measurement, mechanical failures, and filter handling. Appendix E contains a complete list of the reasons data was excluded, but several notable problems will be discussed here. 

### 0 5 10 15 20 25 30 35 40 45

Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6Number of Valid Data Points



### Horiba Sensors AVLREPORT 03 .14936. 12 60 of 174 TABLE 17. D ATA YIELD BY EACH PM

- PEMS

Possible Horiba Sensors - NewPP Sensors – Revised PP AVL Mode 1 41 37 22 34 40 Mode 2 41 36 25 29 33 Mode 3 41 36 25 32 33 Mode 4 41 35 24 28 35 Mode 5 41 37 17 31 37 Mode 6 41 37 23 30 34 Total 246 218 136 184 212

The Horiba s ystem requires an external s ource o f com pressed air capable of s upplying approximately 30 l pm at 400 kP a or higher. A commercially

- available oil- less compressor was provided b y Horiba for use with the s ystem. Unfortunately this compressor had a tendency t o stop working on qui te a few occasions during testing. Three different compressors of the same model w ere pr ovided a nd e ach e xperienced t his problem. It w as be lieved t hat t he compressor was overheating and shutting off to protect itself, althou gh changes in the test cell temperature did not seem to influence its performance. The compressor would begin to work again after 10 to 15 m inutes presumably once i t ha d c ooled of f. I f t he c ompressor s topped w orking w hile t he Horiba system was in operation , the system lost all of its dilution air once the small air tank had been depleted. This resulted in an undiluted exhaust stream being sampled onto the filter which would quickly overload the filter at the PM concentration levels used in this work not to mention fail pr oportionality r equired of t he H oriba s ystem. If t he c ompressor s topped w orking at a ny point dur ing an of ficial t est, t he H oriba da ta f or t hat t est w as voi ded. Figure 34 shows a n example of a steady -state cycle where the compressor stopped working.

A problem occurred with Sensors 1 i nvolving the auto zero function of its exhaust flow meter. Every hour the Sensors system would attempt to z ero the exhaust flow meter by switching the pressure transducers to ambient for a period of time less than a minute. On the Sensors 1 the solenoid switching the pressure transducers from exhaust measurement to zero was not working properly causing t he z ero function t o oc cur w hile pr essures w ere be ing m easured f rom t he exhaust. Because t he steady

- state cycle l asted longer t han on e hour , this w ould c ause a n erroneous exhaust flow measurement on the last two modes of each steady

- state cycle. Figure 35 shows an example of the EFM zero problems during a test.

The EFM auto zero function was disabled for the remaining tests on S ensors 1, a nd for Sensors 2 and 3.REPORT 03 .14936. 12 61 of 174 FIGURE 34 . HORIBA SAMPLE FLOW TO THE FILTER AND DILUTION FLOW WHILE COMPRESSOR STO PS

FIGURE 35 . SENSORS EFM ZERO D URING STEADY

- STATE CYCLE 

### 0 200 400 600 800 1000 1200 1400 1600

0 1000 2000 3000 4000Exhaust Flow (kg/hr)



### Time (sec)

Sensors Lab EFM ZeroREPORT 03 .14936. 12 62 of 174 4.7 Accounting for CVS Variability During Steady

- State Testing

The s teady

- state da ta h as va riability due to the P EMS, CVS, a nd test a rticle. By computing a paired difference for each data point, the test article variability is removed from the data. This concentration delta still contains variability associa ted with the PEMS and the CVS. The data collected in this program does not allow for independent assessment of the PEMS and CVS variability, but by assuming a CVS variability it becomes possible to assign the remaining variability to the PEMS.

The following procedure for estimating the PEMS and CVS variability was proposed by Bill M artin at th e D ecember 12 th 2008 meeting at S wRI and w as a ccepted by t he s teering committee.

1. Since test article variation is the same for each individual observation by th e CVS ( x PEMS,i ) and PEMS ( x CVS ,i 

# ) (

, , i CVS i PEMS i 

# xx **−=∆** ), compute the paired differences,

. These paired differences (i.e., delta values or concentration deltas) contain random variation from the CVS, random variation from the PEMS, and a mean offset between CVS and PEMS (bias error).

2. Divide the entire set of delta values into j = 1 to M su bsets based on the values of χ CVSi . The data sets are not subdivided by engine operating mode or PEMS serial number, but only the level of the reference concentration.

3. For e ach s ubset, t here a re i = 1 t o N j values. T he m edian and the M AD ar e us ed as the descriptive statistics.

4. Calculate t he m edian delta va lue, j ,50 **∆** , t he m edian a bsolute de viation of t he i 

### ∆ values,

j 

### MAD , and the estimate of the standard deviation of the CVS random error, j CVSrandom 

### SD

, .

j Ni i j 

### median ,1 ,50 

### )(

**=** 

### ∆=∆

j Ni j ij j median MAD ,1 ,50 )(

**= ∆−∆=** 


# ∑ **∑ =** 

**= ⋅**  **=** j

j N

i i CVS j N

i i filter CVS j j CVSrandom x N L N g SD

1 ,

1 ,, , 1
1 5 **μ**

**∑**

**⋅ ∑**  **=**

**=**

**=** j

j N

i i CVS N

i i filter CVS j CVSrandom x L g SD

1 ,

1 ,, , 5 **μ**

j CVS j filter CVS j CVSrandom x L g SD

,

,, , 5

**⋅**

**= μ** 

where i filter CVS L

,, is the PM s ample f ilter loa ding. 5 μg is the a ssumed CVS va riability a s proposed by the steering committee based on a nominal filter loading of 100 μg.REPORT 03 .14936. 12 63 of 174 For each subset j, calculate a corrected delta for each i 

### ∆ value, in subset j, as follows:

j ij j j j CVSrandom j j ijij j CVSrandom j

MAD SD MAD SD MAD

50 ,50 2 2

, 2

,50 2

, 2

else ) 45495.0( )( then

45495.0 if

**∆=∆**

**′ ∆+**  **⋅− ⋅∆−∆=∆** 

**′ ⋅>** 

Note that this approach correctly passes through any significant offsets observed in the data. These offsets should be passed through even if they persist only for a subset of data, such as a given mode.

5. The entire set of corrected delta values is then to be used to establish the error surface for the steady

- state data. The 5 th , 50 th , and 95 th delta values are used to establish the 1 st , 50 th , and 99 th percentile values which are the inputs to the Monte Carlo model.

4.8 Steady

- State Testing Results

4.8.1 Comparison between PEMS and Lab Delta PM

All steady

- state data presented has already been corrected for the steady

- state variability as m entioned above. The s teady

- state c oncentration de ltas f rom P EMS 1 f or H oriba, S ensors, and AVL, are shown in Figure 36

, Figure 37 , and Figure 38 , respectively

. Each point on the x

- axis represents the median exhaust PM concentration from the CVS filter f or a s ingle m ode. T he y

- axis r epresents t he 5 th , 50 th , a nd 95 th

Figure 39 percentile of t he de ltas, PEMS – Lab. As mentioned previously there was a clear gap in the data between 162 μg/mol and 325 μg/mol which represents a large portion of the target concentration range for a 0.025 g/hp

- hr level.

, Figure 40 , and Figure 41 show the deltas for PEMS 2 where three of the modes were between 208 and 267 μg/mol.

Horiba- 1 and Horiba

- 2 performed similarly with mode 2 showing a significant negative bias, mode 4 showing a positive bias and the other four modes closer to zero. Sensors

- 2 showed a much greater negative bias than Sensors

- 1. Sensors

- 1 had a 50 th percentile of between 0 and

- 26 μg/mol for five of the six modes while Sensors

- 2 was between

- 32 and

- 104 μg/mol for the 50 th percentile for the same five modes. In addition, mode 2 had a 5 th percentile of

- 340 μg/mol at a reference concentration of 442 μ g/mol. For the same mode on Sensors

- 1 the 5 th percentile

was

- 157 μg/mol at a reference concentration of 326 μg/mol. AVL-2 was lower than AVL- 1 with 50 th percentiles be tween

- 34 μ g/mol a nd

- 70 μ g/mol. T he 50 th

Figure 42 percentiles f or A VL

- 1 were between 13 μ g/mol a nd

- 32 μ g/mol. No i ndication of t he changes i n performance for t hese PEMS was discovered through the recommended checks and audits. The PM deltas for Horiba -3, Sensors-3, and AVL-3 are shown in

, Figur e 43 , and Figure 44 , respectivelyREPORT 03 .14936. 12 64 of 174 FIGURE 36 . HORIBA

- 1 STEADY

- STATE PM CONCENTRATION DELTAS

FIGURE 37 . SENSORS

- 1 STEADY

- STATE PM CONCENTRATI ON DELTAS



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Horiba 50th % Horiba 95th % Horiba 5th %



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Sensors 50th % Sensors 95th % Sensors 5th %REPORT 03 .14936. 12 65 of 174 FIGURE 38 . AVL PM CONCENTRATI ON DELTAS FOR STEADY

- STATE TESTING ON PEMS 1

FIGURE 39 . HORIBA

- 2 STEADY

- STATE PM CONCENTRATION DELTAS



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

AVL 50th % AVL 95th % AVL 5th %

- 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Horiba 50th Horiba 95th Horiba 5thREPORT 03 .14936. 12 66 of 174 FIGURE 40 . SENSORS

- 2 STEADY

- STATE PM CONCENTRATI ON DELTAS

FIGURE 41 . AVL

- 2 STEADY

- STATE PM CONCENTRATION DELTAS 

### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 100 200 300 400PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Sensors 50th % Sensors 95th % Sensors 5th %



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

AVL 50th % AVL 95th % AVL 5th %REPORT 03 .14936. 12 67 of 174 FIGURE 42. HORIBA

- 3 STEADY

- STATE PM CONCENTRATION DELTAS

FIGURE 43 . SENSORS

- 3 STEADY

- STATE PM CONCENTRATI ON DELTAS



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Horiba 50th Horiba 95th Horiba 5th



### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

Sensors 50th % Sensors 95th % Sensors 5th %REPORT 03 .14936. 12 68 of 174 FIGURE 44 . AVL

- 3 STEADY

- STATE PM CONCENTRATION DELTAS

Horiba- 3 and Sensors

- 3 are within the s ame ranges of their first two instruments, while AVL

- 3 p roduced a n e xtremely high bi as a t t he hi ghest concentration ( mode 2 ). T he 50 th percentile o f m ode 2 for A VL

- 3 w as 84 μ g/mol, w hile i t w as 13 μ g/mol a nd

- 62 μ g/mol f or AVL -1 and AVL- 2, respectively.

When t he da ta f rom all three P EMS i s c onsidered, t he range of concentrations f rom 5 mg/m3 to 18 mg/m3 is covered, although there is still a majority of the data located between 5 and 7 mg/m3 as a result of the data from PEMS 1. The individual deltas for the Horiba, Sensors, and AVL are shown in Figure 45

, Figure 46 , and Figure 47 , respectively. Please note that each point represents a single measurement, not pooled data. 

### - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150

0 50 100 150 200 250 300 350 400 450PM Concentration Delta (μg/mol)



### Lab Reference Concentration (μg/mol)

AVL 50th % AVL 95th % AVL 5th %REPORT 03 .14936. 12 69 of 174 FIGURE 45 . STEADY

- STATE CONCENTRATION DELTAS FOR HORIBA

FIGURE 46 . STEADY

- STATE CONCENTRATION DELTAS FOR SENSORS



### - 400 - 300 - 200 - 100 0 100 200

0 100 200 300 400 500PM Concentration Delta (μg/mol)



### Lab PM Concentration Reference (μg/mol)

Horiba

- 1 Horiba

- 2 Horiba

- 3



### - 400 - 300 - 200 - 100 0 100 200

0 100 200 300 400 500PM Concentration Delta (μg/mol)



### Lab PM Concentration Reference (μg/mol)

Sensors

- 1 Sensors

- 2 Sensors

- 3REPORT 03 .14936. 12 70 of 174 FIGURE 47 . STEADY

- STATE CONCENTRATION DELTAS FOR AVL

Because t he r eference concentration of s everal of t he m odes w as similar, the s teering committee decided to group the data by reference concentration rather than by operating mode. The da ta w as s plit up i nto g roups of a pproximately 20 da ta poi nts ba sed on r eference concentration for de veloping t he steady

- state PM er ror s urface. The H oriba da ta w as grouped into 11 sets, the Sensors data into 9 sets, and the AVL data into 10 sets.

Figure 48

, Figure 49 , and Figure 50 show PEMS PM concentration plotted against the PM conc entration determined by t he r eference l aboratory f ilter m ethod for s teady

- state N TEs. These plots mainly show the qualitative PEMS to PEMS scatter relative t o the filter method.

4.8.2 Correlation between PEMS and Lab PM

Figure 5 1 shows a linear regression between each set of PM

- PEMS and the CVS filter method. G ood c orrelation w as obs erved be tween t he M SS a nd l ab , w ith as coe fficient of determination ( R 2 ) o f 0.84 and a s lope of 0 .89. The s lope suggests t hat t he M SS P M concentration is 11 pe rcent lower than that determined by the lab. This trend is expected since the MSS is measuring soot and the lab reports total PM. The slope seemed to be high because the MSS PM concentration in the range between 15 and 20 mg/m3 was higher than that of the lab.

The line a r regression between the Horiba TRPM and the lab resulted in R 2 - 400 - 300 - 200 - 100 0 100 200

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Lab PM Concentration Reference (μg/mol) AVL

- 1 AVL

- 2 AVL

- 3

of 0.55 and a slope of 0.86, indicating some correlation. After further investigation, it was recognized that the weak correlation was due to Mode 4 (high speed, light load) of the SS testing, with concentration leve ls of about 5 mg/m3. By removing this mode from the data, a R2 of 0.85 was obtained and the slope w as moved from 0.86 t o 0.88. It i s l ikely t hat M ode 4, could ha ve r esulted i n overestimation of PM due to nanoparticle formation with the Horiba dilution system, although CVS t esting a t t his condition di d not s how a n anoparticle m ode. Based on t he s lope of t he correlation, the Horiba PM concentration was 14 percent lower than that reported by the lab.REPORT 03 .14936. 12 71 of 174 The linear regression between the Sensors PPMD and the lab results in R2 of 0.34 and a slope of 0.64. The weak correlation wa s due to data scatter. Except in the narrow range between 5 and 7 mg/m3, the Sensors PPMD showed underestimated PM. Based on the slope, the Sensors PPMD PM concentration was 36 percent lower than that reported by the lab.

FIGURE 48 . STEADY

- STATE HORIBA PEMS PM CONCENTRATION VERSUS THE LABORATORY REFERENCE

FIGURE 49 . STEADY

- STATE SENSORS PEMS P M CONCENTRATION VERS US THE LABORATORY REFERENCE 

### 0 50 100 150 200 250 300 350 400 450 500

0 100 200 300 400 500PEMS PM Concentration (μg/mol)



### Lab PM Concentration Reference (μg/mol)

Horiba

- 1 Horiba

- 2 Horiba

- 3 Lab Reference

0 50 100 150 200 250 300 350 400 450 500

0 100 200 300 400 500PEMS PM Concentration (μg/mol)

Lab PM Concentration Reference (μg/mol)

Sensors

- 1 Sensors

- 2 Sensors

- 3 Lab ReferenceREPORT 03 .14936. 12 72 of 174 FIGURE 50 . STEADY

- STATE AVL PEMS PM CO NCENTRATION VERSUS T HE LABORATORY REFERENCE

FIGURE 51 . LINEAR REGRESSION CORRELATION BETWEEN PEMS AND LAB 

### 0 50 100 150 200 250 300 350 400 450 500

0 100 200 300 400 500PEMS PM Concentration (μg/mol)



### Lab PM Concentration Reference (μg/mol) AVL

- 1 AVL

- 2 AVL

- 3 Lab ReferenceREPORT 03 .14936. 12 73 of 174 4.8.3 Steady-S tate PM Error Surfaces

Figure 52

, Figure 53 , a nd Figure 54 , s how t he or iginal c oncentration deltas t hat w ere presented to the steering committee on April 2 nd , 2009 at SwRI. The Sensors data includes some additional poi nts t hat were a dded l ater due t o c hanging t he e rror tolerances of t heir pos t processor. Each marker on the plot represents one of the j = 1 to M subsets of data divided up by concentration value. Each plot shows both the 5 th and 95 th percentiles which are b ased on t he actual da ta. T he e rror s urfaces f or t he Monte Carlo m odel w ere ba sed on t he 1 st and 99 th percentiles w hich were ex trapolated from t he 5 th and 95 th percentiles as suming a nor mal distribution of the data.

The st eering committee elected to smooth the error surface by not including some of the data points that were within the envelope of surrounding points. A similar decision was made on some of the error surfaces included in the gaseous measurement allowance program.

Figure 55 , Figure 56 , and Figure 57 show the final steady

- state error surfa ce of Horiba, Sensors, and AVL, respectively. The data presented in these plots has already been processed to remove th e C VS va riability a nd adjusted to the 1 st and 99 th percentile f orm the 5 th and 95 th percentile. T he lines on t he plot represent the fin al error surfaces as app roved by the steering committee. The Horiba and AVL error surfaces were accepted by the steering committee during the conference call on June 29 th , 2009. Some of the sensors data was reprocessed with different tolerances to increase the data yield, so the sensors error surface was not accepted by the steering committee until the July 14 th , 2009 meeting in Indianapolis.

FIGURE 52 . STEADY

- STATE CONCENTRATION DELTAS FOR HORIBA



### - 200 - 150 - 100

- 50 0 50 100

0 100 200 300 400 500 600PM Concentration Delta (μg/mol)



### Lab Reference Median PM Concentration (μg/mol)

5th % 95th % 50th %REPORT 03 .14936. 12 74 of 174 FIGURE 53 . STEADY

- STATE CONCENTRATION DELTAS FOR SENSORS

FIGURE 54 . STEADY

- STATE CONCENTRATION DELTAS FOR AVL 

### - 400 - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100

0 100 200 300 400 500 600PM Concentration Delta (μg/mol)



### Lab Reference Median PM Concentration (μg/mol)

5th % 95th % 50th %

- 150 - 100

- 50 0 50 100 150

0 100 200 300 400 500 600PM Concentration Delta (μg/mol)



### Lab Reference Median PM Concentration (μg/mol)

5th % 95th % 50th %REPORT 03 .14936. 12 75 of 174 FIGURE 55 . FINAL STEADY

- STATE PM ERROR SURFACE – HORIBA

FIGURE 56 . FINAL STEADY

- STATE PM ERROR SURFACE – SENSORS 

### - 400 - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150 200

0 100 200 300 400 500Corrected PM Delta (μg/mol)



### CVS Filter PM Concentration (μg/mol)

Delta 1st % Delta 50th % Delta 99th %



### - 400 - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150 200

0 100 200 300 400 500Corrected PM Delta (μg/mol)



### CVS Filter PM Concentration (μg/mol)

Delta 1st % Delta 50th % Delta 99th %REPORT 03 .14936. 12 76 of 174 FIGURE 57 . FINAL STEADY

- STATE PM ERROR SURFACE – AVL

With a few exceptions the PEMS generally exhibited a low bias, with the majority of the 50 th percentile deltas falling below zero. The Horiba and Sensors both showed a degree of level dependence with the negative bias generally increasing for higher PM concentrations. At a lab reference concentration of 413 μg/mol the Horiba 50 th percentile was -92 μg/mol and the Sensors 50 th percentile was

- 177 μg/mol at a lab reference concentration of 403 μ g/mol. The AVL 50 th percentile w as ge nerally l evel i ndependent and remained between

- 8 a nd

- 58 μ g/mol f or t he entire r ange o f con centrations t ested. While t he 1 st percentile f or t he A VL also remained relatively constant t hroughout t he c oncentration r ange, t he 99 th percentile j umped to a m uch higher va lue at hi gh co ncentration. The A VL 9 9 th percentile j umped from 25 μ g/mol at 291 μg/mol to 165 μg/mol at 402 μg/ mol. The cluster of positive deltas that caused this large increase in the 99 th percentile data is due solely to mode 2 with PEMS 2. The majority of the positive deltas observed for the Horiba were due to mode 4 which was a high speed light load condition. Since it is understood that the DCS (EAD) real time particle sensor in the Horiba system is more sensitive t o smaller pa rticles i t w as as sumed that m ay have a l arger n umber of s ub

- 50 nm particles than the other 5 modes tested, causing more of the filter mass to be attributed to mode 4.

4.9 Transient Engine Results

The t ransient eng ine t esting w as de signed to characterize t he pr ecision error of t he PEMS. A transient cycle consisting of 30 N TE events, 32 s econds each, was repeated multiple times and any differences in the PEMS measurements w ere attributed to PEMS variability under the assumption that the engine operation and PM emissions remained constant. No lab reference value was captured on an individual NTE event basis, because it i s not possible to collect PM on a CVS filter for each NTE. Instead filter measurements were taken for each integrated transient 

### - 400 - 350 - 300 - 250 - 200 - 150 - 100

- 50 0 50 100 150 200

0 100 200 300 400 500Corrected PM Delta (μg/mol)



### CVS Filter PM Concentration (μg/mol)

Delta 1st % Delta 50th % Delta 99th %REPORT 03 .14936. 12 77 of 174 NTE cycle as a general indicator of the PM emissions of the engine during the entire cycle

. The cycle used in this testing was developed using t he cycle generator from the gaseous measurement allowance program. In the gaseous program the cycle generator was used to develop 20 uni que cycles with different orders of NTE events and different transitions between events. The steering committee elected to generate one single cycle for the current work that would be used for all transient testing. Additional time was added in between NTE events to ensure that the PPMD could sample throughout the cycle without missing any NTE events under the assumption of 7 working c rystals ( 6 f or s ample a nd one r eference c rystal). In a ddition s everal s hort non

- valid NTE events were added to the cycle to challenge the PEMS for measurements of non

- valid NTE
events.

The D PF b ypass w as a djusted t o pr oduce a n i ntegrated cycl e BSPM of appr oximately 0.03 g/hp

- hr based on the CVS filter. Instead of readjusting the bypass for each set of PEMS, the exhaust valves were set to the same position each time. The NTE events were all approximately 34 s econds i n dur ation. T he or iginal e ve nts f rom t he c ycle generator were each 32 seconds, however the Sensors PEMS would occasionally see these events as shorter than 30 seconds and exclude t hem. Many of t he N TE eve nts cont ained extreme acc elerations and decelerations stopping just short of the lower boundaries of the NTE window. Considerable time was spent to ensure that the J 1939 signal remained in the N TE window during events, although the engine performance shifted slightly and occasionally an event was invalidated. If any NTE event did
not remain in the NTE windows for at least 30 s econds it was not included in the data for the transient er ror s urface. A t otal of 16 c ycles were r un f or P EMS, 17 f or P EMS2, a nd 18 f or PEMS3. Figure 58 shows repeat engine speed traces for the transient cycle with the first and last official cycle conduc ted w ith each PEMS. Figure 59 shows t he r epeat t orque t races f or t he transient cycle. The COV on cycle work was 0.7 percent over 51 cycles.

FIGURE 58 . REPEAT ENGINE SPEE D TRACES FOR NTE TRA NSIE NT CYCLE 0 5 10 15 20 25 30 35

600 800 1000 1200 1400 1600 1800 2000 2200

0 500 1000 1500 2000NTE Event NumberEngine Speed (rpm)

Time (sec)

PEMS1

- 1 PEMS1

- 16 PEMS2

- 1 PEMS2

- 17 PEMS3

- 1 PEMS3

- 18 Event NumberREPORT 03 .14936. 12 78 of 174 FIGURE 59 . REPEAT ENGINE TORQUE TRACES FOR NTE TRANSIENT CYCLE

The exhaust concentration of soot, as measured by the AVL PEMS is shown in Figure 60

. The m easured exhaust c oncentrations of s oot r anged from 5 t o 380 0 μ g/mol ( 0.2 t o 160 mg/m3) during NTE operations, with concentrations of 5 to 14 μg/mol (0.2 to 0.6 mg/m3) outside of the NTE window. The widest dynamic range observed during an NTE event was for event 9 where the initial spike in soot concentration was measured as high as 3800 μg/mol before falling down to 55 μg/mol near the end of the event. Figure 61 gives a closer look at events 20 through 23 to show the variability of the real

- time signal over the three sets of PEMS. It should be noted that the AVL PEMS is shown here only because it reports the measured soot concentration on a second by second basis without additional processing.

Although the Horiba PEMS was used in this program as a batch sampler rather than a second by second instrument for official results, Figure 62 shows the second

- by

- second data of the H oriba P EMS a long w ith t he A VL. T here were m ajor di fferences on s ome of t he pe ak concentrations most notably NTE event 9, w here the measured AVL concentration was nearly four times higher than the measured Horiba concentration. With no reference, the purpose of this figure i s onl y t o s how t hat t here a re di fferences w ithout t rying t o qua ntify t he a ccuracy. T he Sensors P EMS pe rforms i ts m easurements as a ba tch sampler s o that t here w as no real

- time exhaust concentration reported. 0 5 10 15 20 25 30 35

- 200 200 600 1000 1400 1800 2200

0 500 1000 1500 2000NTE Event NumberEngine Torue (Nm)

Time (sec)

PEMS1

- 1 PEMS1

- 16 PEMS2

- 1 PEMS2

- 17 PEMS3

- 1 PEMS3

- 18 Event NumberREPORT 03 .14936. 12 79 of 174 FIGURE 60 . REPEAT AVL SOOT CONCENTRATION TRACES FOR NTE TRANSIENT CYCLE

FIGURE 61 . AVL SOOT CONCENTRATION DURING NTE TRANSIENT CYCLES, EVENTS 20 -23 0 5 10 15 20 25 30 35

0 500 1000 1500 2000 2500 3000 3500 4000

0 500 1000 1500 2000NTE Event NumberExhaust Soot Concentration (μg/mol)

Time (sec) PEMS3

- 18 PEMS3

- 1 PEMS2

- 17 PEMS2

- 1



### 18 20 22 24 26 28 30

0 100 200 300 400 500 600 700 800 900 1000

1440 1490 1540 1590 1640NTE Event NumberExhaust Soot Concentration (μg/mol)



### Time (sec) PEMS3

- 18 PEMS3

- 1 PEMS2

- 17 PEMS2

- 1REPORT 03 .14936. 12 80 of 174 FIGURE 62. COMPARISON OF AVL AND HORIBA REAL TIME SIGNALS DURING TRANSIENT CYCLE

To properly quantify the precision of the PEMS it is important that the reference value remain constant otherwise changes in the source PM emissions will be attributed to the PEMS as measurement va riability. In examining the C VS filter r esults, it w as c lear tha t the e ngine P M emissions va ried somewhat dur ing te sting. After r eviewing the tr ansient P EMS da ta a t the January 28 th , 2009 meeting a t S wRI, the s teering c ommittee r equested that the i dea of a correction t o t he P EMS data be a pplied t o a ccount f or c hanges i n t he e ngine pe rformance. A correction factor ba sed on the c ycle i ntegrated CVS B SPM w as pr esented to the com mittee during the April 2 nd

CF i = ∑ CVS i N i=1 N ∙ CVS i = CVS avg CVS i , 2009 meeting at SwRI and was accepted f or use on all of the transient data. The correction factor was calculated as follows:

The correction factor is multiplicative and applied to the PEMS data in the following manner:

PEMS corre cted

, i

, j = CF i ∙ PEMS measured

, i

, j
The CVS BSPM along with the correction factor is shown in Figure 63

. 0 20 40 60 80 100 120 140 160 180

0 500 1000 1500 2000Exhaust Concentration (mg/m3)

Time (sec)

AVL

- Soot Horiba

- PMREPORT 03 .14936. 12 81 of 174 FIGURE 63 . CVS BSPM AND CORRECTION FACTOR FOR TRANSIENT CYCLE

The correction factor ranged from 0.78 to 1.24. There were two cycles in which the CVS filter measurement was void; in these cases the correction factor was set to one. The CVS brake specific PM ranged from 34.4 mg/hp

- hr to 21.7 mg/hp

- hr with an average of 27.2 mg/hp

- hr. The COV was 11.4 percent when cycles for all three PEMS are included. The COV was 7.2, 6.0, and 4.8 percent for cycles from PEMS1, 2, and 3, respectively.

While processing some of the AVL data and comparing to the lab , it was discovered that the incorrect CVS BSPM value had been used for one of the cycles. The value of 31.9 mg/hp -hr from cycle 1956 had also been used for cycle 1965 instead of the correct value of 28.4 mg/hp

- hr. The entire set of transient CVS data was scrutinized again and no a dditional errors were found. This erroneous correction factor was included in the final transient error surface included in the model. The transient error surface was found to shrink for all PEMS by between 1 and 2 percent with t he c orrection a pplied. G iven t he s mall c hange i n t he out come no a ction w as t aken t o correct this mistake. The transi ent error s urfaces pr esented i n this report are t hose us ed i n the Monte Carlo model.

Figure 64

, Figure 65 , and Figure 66 show the concentration deltas for Horiba, Sensors, and AVL, respectively. In the transient testing there was no lab reference to create deltas. Rather the delta was calculated from the 50 th percentile of the lab data as follows:

Delta i,j = CF i * mPM i,j – 50 th i

Where mPM i,j is the average flow

- weighted PM concentration for the j th repeat of the i th NTE event. 50 th i is the 50 th percentile of mPM for the i th NTE event. CF i 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4

15 20 25 30 35 40 45 50 55

0 10 20 30 40 50 60Correction FactorCVS Brake-Specific PM (mg/hp-hr)

Transient Cycle Number

CVS Filter BSPM Correction Factor

is the previously described correction factor to adjust for engine variability.REPORT 03 .14936. 12 82 of 174 FIGURE 64 . HORIBA CONCENTRATION DELTAS FOR TRANS IENT ENGINE TESTING

FIGURE 65 . SENSORS CONCENTRATION DELTAS FOR TRANS IENT ENGINE TESTING 

### - 150 - 100

- 50 0 50 100 150

0 100 200 300 400 500PM Concentration Delta (μg/mol)



### Median Flow

- Weighted PM Concentration (μg/mol)

5th

- 50th Percentile 95th

- 50th Percentile



### - 150 - 100

- 50 0 50 100 150

0 100 200 300 400 500PM Concentration Delta (μg/mol)



### Median Flow

- Weighted PM Concentration (μg/mol)

5th

- 50th Percentile 95th

- 50th PercentileREPORT 03 .14936. 12 83 of 174 FIGURE 66 . AVL CONCENTRATION DELTAS FOR TRANSIENT ENGINE TEST ING

The 5 th and 95 th percentiles of t he H oriba a nd AVL P EMS w ere boun ded b y pl us a nd minus 50 μg/mol. T he Sensors P EMS ha d s lightly l arger de ltas e xtending j ust be yond 100 μg/mol for the 95 th percentile and

- 150 μg/mol for the 5 th percentile. The bias from the data was removed by subtracting the 50 th from the 5 th and 95 th percentiles.

The steady

- state error surface was designed to quantify the accuracy of the PEMS while the t ransient e rror s urface w as de signed t o qua ntify t he pr ecision of t he P EMS. H owever, a portion of the PEMS precision error is inherently captured in the steady

- state error surface. At the meeting on J uly 15 th , 2009 i n Indianapolis four different approaches were presented to the steering c ommittee f or r emoving the steady

- state contribution t o t he pr ecision e rror f rom t he transient er ror s urface. T he s teering com mittee el ected to proc eed w ith a pproach 3; f or simplicity the other three approaches will not be described.

In this approach, the data from all three PEMS from the same manufacturer were pooled together (ie Sensors1, Sensors2, and Sensors3). This resulted in a total of 18 ste ady

- state data points and 90 transient data points. A median and a MAD value were calculated for each of the 108 data points and used to calculate a MAD relative error, transient effect, root mean squared or MAD re,tr,rms . The MAD re,tr,rms is defined as fol lows: 

### - 150 - 100

- 50 0 50 100 150

0 100 200 300 400 500PM Concentration Delta (μg/mol)



### Median Flow

- Weighted PM Concentration (μg/mol)

5th

- 50th Percentile 95th

- 50th PercentileREPORT 03 .14936. 12 84 of 174 MAD re , tr
, rms = � 1
N � � MAD tr
, i Median tr
, i � 2 N

i=1 − 1
N � � MAD ss , i Median ss , i � 2 N

i=1

Where tr denotes transient and ss denotes steady

- state . Using an rms value eliminates the need for estimating the steady

- state variabilit y at each of 90 t ransient data points. The error surface was defined as the 90 percent confidence interval around zero (or no bias) as:

5 th Percentile = − 1. 65 ∙ PM ∙ MAD re , tr
, rms 95 th Percentile = 1. 65 ∙ PM ∙ MAD re , tr
, rms
Effectively the error surface is a straight line with a slope of +/ - 1.65·MAD re,tr,rms Figure 67 and an intercept of z ero. T he t ransient e rror s urface i s shown i n for H oriba, Figure 68 for Sensors, and Figure 69 for AVL.

FIGURE 67 . FINAL HORIBA TRANSIENT PM ERROR SURFAC E - 120

- 80 - 40

0 40 80 120

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Median Flow Weighted PM Concentration ( μ g/mol)

1st Percentile 99th PercentileREPORT 03 .14936. 12 85 of 174 FIGURE 68 . FINAL SENSORS TRANSIENT PM ERROR SURFA CE

FIGURE 69 . FINAL AVL TRANSIENT PM ERROR SURFACE - 120

- 80 - 40

0 40 80 120

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Median Flow Weighted PM Concentration ( μ g/mol)

1st Percentile 99th Percentile

- 120

- 80 - 40

0 40 80 120

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Median Flow Weighted PM Concentration ( μ g/mol)

1st Percentile 99th PercentileREPORT 03 .14936. 12 86 of 174 The median flow weighted PM concentration on t he x

- axis of each graph is extended to the highest measured concentration observed by any of the PEMS during transient testing

. The slope of the transient PM error surface is 0.083 (1.41) for Horiba, 0.228 (3.27) for Sensors, and 0.099 ( 1.66) f or A VL. The 1 st percentile f or t he P EMS ar e -34, - 92, a nd

- 40 μg/mol a t th e maximum PM concentrations observed by each of the PEMS during steady

- state testing (around 400 μg/mol) for the Horiba, Sensors, and AVL systems. The 1 st percentiles during steady

- state testing at t he s ame P M level w ere -177, - 366, a nd

- 110 μ g/mol f or t he Horiba, S ensors, and AVL, respectively. Because the steady

- state error surface was much larger than the transient it tended to dominate the model results as discussed later.

4.10 CE

- CERT Mobile Lab Correlation

The t est pl an c alled for t he v alidation of t he model t o be pe rformed b y the m obile emissions l aboratory ( MEL) ope rated b y t he U niversity o f C alifornia a t R iverside Bourns College of E ngineering C enter f or E nvironmental R esearch and T echnology ( CE

- CERT) . T he mobile laboratory consisted of a full

- flow CVS system inside the trailer of a Class A truck. The mobile lab was capable of measuring gaseous emissions and filter based PM. The m obile lab was arrived at SwRI on A pril 9 th

Figure 70 , 2009 t o compare brake

- specific PM emissions and ensure that the reference during in

- use validation is similar to the reference used during laboratory testing. The CE

- CERT MEL was parked behind the SwRI test cell and an exhaust transfer line was fabricated to allow the M EL to measure the full e ngine e xhaust in the s ame w ay a s the S wRI te st cell. Because the CVS measurement technique requires the full flow of engine exhaust for emissions measurement, the exhaust system was designed in such a way that the exhaust pipe could easily be switched between the SwRI CVS and the MEL CVS using the same length and geometry of exhaust pipe. shows the exhaust system used for the CE- CERT correlation.

FIGURE 70 . EXHAUST CONFIGURATION FOR CE

- CERT CORRELATION 

### To CE

- CERT CVS Tunnel To SwRI CVS TunnelREPORT 03 .14936. 12 87 of 174 Prior t o t he s tart of t esting, t he M EL C VS a nd SwRI C VS bot h unde rwent a pr opane

c heck to ensure the sampling systems were operating at the correct flows. Further details of the audits performed by CE

- CERT on t he MEL can be found in the CE

- CERT report (reference?). The MEL was set to the same CVS flow, secondary dilution ratio, and filt er face velocity as the SwRI C VS t o m ake t he m easurements as c lose as pos sible. O ne ke y factor t hat c ould not b e controlled was the dilution air for the MEL CVS. The SwRI CVS was able to draw flow that is conditioned and maintained between 20 and 30oC. The MEL CVS drew its dilution air from the ambient w hich pr evented t he c ontrol of t emperature a nd r elative hum idity of t he di lution a ir. Each sampling system was conditioned by sampling for 10 hours during steady

- state DPF engine operation a t a hi gh e xhaust t emperature. A ctive D PF r egeneration oc curred dur ing t his conditioning period. A total of 16 s hort NTE transient cycles were conducted using each CVS system. The short NTE transient cycle was a m odified version of the NTE transient cycle used for official t ransient PEMS testing. The short cycle included 16 of the original 30 N TE events and lasted 755 s econds compared to 2130 s econds for the full NTE cycle. Table 18 shows the order of testing.

TABLE 18. TEST PROCEDURE FOR C E- CERT CORRELATION

Active Regen Cycle 1 Cycle 2 Cycle 3 Cycle 4 Active Regen Cycle 1 Cycle 2 Cycle 3 Cycle 4 Day 1 1 SwRI SwRI SwRI SwRI 1 CE-CERT CE-CERT CE-CERT CE-CERT Day 2 1 CE-CERT CE-CERT CE-CERT CE-CERT 1 SwRI SwRI SwRI SwRI Day 3 1 SwRI SwRI SwRI SwRI 1 CE-CERT CE-CERT CE-CERT CE-CERT Day 4 1 CE-CERT CE-CERT CE-CERT CE-CERT 1 SwRI SwRI SwRI SwRI

A manually triggered active DPF regeneration was performed before the start of each set of four test cycles to maintain a similar PM loading level on the DPF. Each day the test order was switched so that the SwRI tests were first on days 1 and 3 and the CE

- CERT tests were first on days 2 and 4. Each test was conducted as a hot

- start with a 20 minute hot soak in between test cycles or in between the DPF regeneration and the first test cycle. Figure 71 shows the brake

specific PM results from the 16 cycles.

The data from test 9 for CE- CERT was removed due to a filter weight that was deemed to be an outlier. The average SwRI BSPM was 0.0287 g/hp

- hr with a COV of 5.2 percent based on 16 repeats. The average CE

- CERT BSPM was 0.0265 g/hp

- hr with a COV of 3.5 percent based on 15 r epeats. T he r eported C E

- CERT e missions w ere on average 7.7 p ercent l ower t han t he SwRI reported em issions. The average reported B SCO 2 by C E

- CERT was 2.6 p ercent l ower than average SwRI reported value. One possible source of discrepancy between the two systems was the heat loss in the exhaust pipe prior to its entrance into the CVS. The SwRI system was completely sheltered within the t est c ell, while the C E

- CERT e xhaust pi pe w as pr otruding outside in such a way that it was exposed to wind and ambient temperature effects. This could have resulted in a higher thermophoretic deposition of particles inside the exhaust pipe resulting in lower emissions for the CE

- CERT system. However, the test plan stated that agreement within ten percent was considered sufficient, so the correlation was considered complete and the issue was not investigated further.

A series of tunnel blanks were measured from both systems over sample periods of 15, 30, and 60 minutes. Figure 72 shows the filter weight gains as a function of sample time.REPORT 03 .14936. 12 88 of 174 FIGURE 71 . BRAKE

- SPECIFIC PM RESULTS FROM CE

- CERT CORRELATION

FIGURE 72 . CVS FILTER WEIGHT GAIN DURING TUNNEL BLANKS



### 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31Brake-Specific PM (g/hp-hr)



### SwRI CE

- CERT Day 1 Day 2 Day 3 Day 4



### 0 2 4 6 8 10 12 14

0 15 30 45 60 75Filter Weight Gain (μg)



### Sample Time (min)

SwRI CE

- CERTREPORT 03 .14936. 12 89 of 174 The filter weight gains during tunnel blank operation were similar for the SwRI and CE

CERT tunnels. T he w eight ga ins r anged f rom 1 .9 t o 12.6 μg a nd g enerally de creased a s t he sampling time increased. The filter weight gains during the correlation testing were greater than 300 μg making any differences in the tunnel blanks insignificant.

4.11 In vestigation of DPF Regeneration

The steering committee requested that screening tests be performed on measurements of active DPF regeneration. Although the majority of the time active DPF regeneration is excluded from i n

- use m easurement as t he r egulations ar e cur rently written there ar e cer tain situations where regeneration could be included in a v alid NTE event. In addition, if the Horiba s ystem were t o t rigger f ilter s ampling dur ing D PF r egeneration t his w ould s till be i ncluded i n t heir calibration fact or. So even if all NTE events containing active DPF regeneration were considered invalid, the measurement accuracy of the Horiba system during valid NTE events could still be affected through the filter calibration. For this testing the engine was operated at a medium speed medium l oad c ondition a t steady

- state and the DPF w as a llowed to accumulate P M unt il th e ECM aut omatically t riggered a r egeneration. The va lves i n the b ypass w ere cl osed forcing a large majority of the exhaust through the DPF although th e bypass was not completely sealed. At the poi nt w hen t he E CM i ndicated i t w as pr eparing f or a n a ctive r egeneration t he P EMS a nd CVS f ilter w ere t riggered t o s ample. T he P EMS s ampled 40 s econds on, f ive s econds of f throughout t he r egeneration, w hile t he C VS f ilter s ampled c ontinuously. Table 19 lists the brake- specific PM results of the PEMS and the CVS.

TABLE 19 . PM EMISSIONS RESUL TS FROM ACTIVE DPF REGENERATION

CVS Filter Horiba Sensors AVL No. of Samples 1 36 31 35 Avg. BSPM, mg/hp

- hr 6.8 4.6 7.1 0.2

It i s l ikely t hat onl y a s mall por tion of t he e missions dur ing t he r egeneration e ven included elemental carbon as shown by the fact that the AVL PEMS measured only three percent of t he e missions m easured b y t he C VS f ilter. T he H oriba a nd S ensors PEMS bot h measured BSPM values on t he same order as the CVS filter although the event

- by

- event emissions were much different. The EEPS also sampled continuously f rom the CVS through a long residence time secondary dilution tunnel with a nominal secondary dilution ratio of 2. T he EEPS number concentration was converted to a mass concentration assuming spherical particles with a density of 1 g/cm3. A comparison of the event

- by

- event emissions of the three PEMS and the EEPS are shown in Figure 73

.REPORT 03 .14936. 12 90 of 174 FIGURE 73 . BRAKE SPECIFIC PM EMISSIONS DURING ACT IVE REGENERATION

Although the average brake

- specific PM value from the Sensors system was very close to the lab value, it is unlikely that the Sensors instrument was properly capturing the behavior of the regeneration. T his i s be cause i ts be havior w as i nsensitive t o t he a ctive regeneration r egion a s shown i n F igure 73. F urthermore, t he i ndividual br ake

- specific e missions s eemed t o be hi gh, particularly during the first three events and the last two events, where no regeneration occurred. The event

- by

- event emissions from the Horiba system had similar trend to the mass determined using EEPS number

- weighted size distribution measurement, assuming spherical particles with a density o f 1 g/cm3. Comparing t he Horiba m easurements t o t he C VS f ilter a nd t he E EPS indicate that it wa s more accurate than the other PEMS at measuring the regeneration emissions. However, one aspect of the Horiba measurement that was not captured in this experiment, and that is the change in sensitivity of the D CS particle ins trument to different s ize part icles. The filter calibration constant for the regeneration event indicated that the DCS was approximately 15 times more sensitive to the particles emitted during the active regeneration compared to the particles emitted during steady

- state engine testing. This means that if a steady

- state engine cycle was s ampled onto the s ame f ilter as t he D PF regeneration the s ystem would be m uch less accurate. The Horiba system would over predict the emissions during the regeneration and under predict t he e missions d uring nor mal e ngine op eration. T hese f indings w ere pr esented t o t he steering committee at the December 11 th 

### 0 100 200 300 400 500 600

0 2 4 6 8 10 12 14 16 18

0 20 40 60 80 100DPF Outlet Temperature (°C)Brake-Specific PM (mg/hp-hr)



### Regeneration Time (%)

Horiba Sensors AVL EEPS DPF Out Temp

, 2008 meeting in San Antonio. Although this could be a major issue in the accuracy of the measurement, due to budget limitation, the steering co mmittee did not add D PF r egeneration t esting t o t he pr ogram given t h at t he r egulated aspect of regeneration during in

- use testing was vague. No data from the DPF regeneration investigation was included in the model. The AVL MSS was insensitive to regeneration e vents as shown in Figure 73 . T his s uggests that the majority of ma ss emitted during r egeneration is likely to be volatile materials that will not be detected by the MSS.REPORT 03 .14936. 12 91 of 174 4.12 Investigation of Storage and Release

At the s ame time as the D PF regeneration investigation, the s teering committee a lso requested that screening tests be performed in the area of storage and release of nanoparticles in the aftertreatment system. It has commonly been observed that volatile material emitted by the engine at low tempe rature will deposit on the DPF only to be released quickly when the engine exhaust temperature climbs. Three different DPF loading conditions were tested:

**•** Low idle: 650 rpm, 65 Nm

**•** Medium idle: 1200 rpm, 135 Nm

**•** High idle: 1800 rpm, 135 Nm.

Idling times of 20, 60, and 90 minutes were tested at each condition. Following the period of low temperature loading, the engine was immediately brought to a high temperature condition to promote the release of the stored particles. Each of the three idling conditions was tested with two high temperature conditions:

**•** Peak torque: 1200 rpm, 2170 Nm

**•** Near rated power: 1800 rpm, 1425 Nm

These t ests , s hown i n F igure 74 were con ducted as a s creening exercise w ith measurements b y t he EEPS t o determine w hich combination produced t he l argest r elease of nanoparticles to use for the PEMS testing. However none of the tested conditions resulted in any significant p article e miss ions on a num ber or mass ba sis. T o e nsure t hat t he E EPS w as not missing something that might have been captured by one of the PEMS, two tests were run with the PEMS: low idle to peak torque and high idle to peak torque. Only the low idle test is shown here because t he r esults a re ve ry s imilar. T he e ngine w as a llowed t o i dle f or one hour be fore going to peak torque for ten minutes; this process was repeated three times consecutively for a cycle length of 3.5 hour s. The PEMS only sampled during the peak torque portion of the cycle with the same 40 seconds sample, five seconds off cycling used in the DPF regeneration study.

A spike of just over 2.0E6 particles/cm3 was observed during the transition from idle to peak torque although this appears to be due to acceleration and a possible slight misalignment between the di lution ratio and EEPS m easurement r ather t han a release of pa rticles f rom t he aftertreatment. The spike was less than 1.0E6 for the second and third transitions to peak torque. Figure 75 shows the brake

- specific PM emissions measured by the three PEMS during the peak torque portion of the storage and release cycle. All samples are taken durin g the three repeats of the t en m inutes a t pe ak t orque a nd no e missions f rom t he i dle por tion of t he c ycle a re represented.REPORT 03 .14936. 12 92 of 174 FIGURE 74 . TOTAL EXHAUST NUMBER CONCENTRATION DUR ING STORAGE AND RELEASE

FIGURE 75 . BRAKE

- SPECIFIC PM EMISSION S DURING STORAGE AND RELEASE INVESTIGATION 

### 0.0E+00 5.0E+05 1.0E+06 1.5E+06 2.0E+06 2.5E+06

0 500 1000 1500 2000 2500

0 2000 4000 6000 8000 10000 12000Exhaust Number Concentration (part/cm3)Engine Speed (rpm), Torque (Nm)



### Time (sec)

Speed RPM Torque N

- m EEPS

- 4.0 - 2.0

0.0 2.0 4.0 6.0 8.0 10.0 12.0

0 10 20 30 40Brake-Specific PM (mg/hp-hr)



### Sample Number

Horiba Sensors AVL 1st Repeat 2nd Repeat 3rd RepeatREPORT 03 .14936. 12 93 of 174 It is unclear why the Sensors were reporting emissions much higher than the other two PEMS. A single CVS filter was sampled for the entire ten minutes at peak torque for each rep eat for a total of three filter measurements. The BSPM from the CVS filter was only between 0.3 and 0.4 m g/hp

- hr for each of the three r epeats. This was lower than any of the PEMS and an order of magnitude lower than the Sensors PEMS. The three negative em issions were attributed to t he s ame c rystal which w as unus ually noi sy during t he t est. A com parison of t he av erage BSPM values for the cycle is shown in Table 20.

TABLE 20 . AVERAGE BRAKE

- SPECIFIC EMISSIONS DURING STORAGE AND RELEASE CYCLE

CVS Filter Horiba Sensors AVL No. of Samples 3 39 36 39 Avg. BSPM, mg/hp

- hr 0.38 0.54 2.97 0.41

The effect of storage and release on the PEMS emissions was not cl ear because of the inability to generate a s ignificant release of nanoparticles, but it was evident that investigating this phenomenon using this experimental configuration was not worthwhile. These findings were presented at t he D ecember 11 th , 2008 m eeting i n S an A ntonio a nd t he s teering committee declined t o pur sue a ny f urther w ork i n t his area. N o da ta f rom t he s torage and release investigation was included in the model.

4.13 Engine Manufacturers Torque a nd Fuel Error Surfaces

The OEM supplied torque error surface was up dated from the gaseous PEMS program and t he O EM s upplied BSFC e rror s urface was replaced with a f uel f low er ror s urface. Five different engine manufacturers supplied data from 61 different engines. In addition data was used from t he f our engines t ested i n t he A CES pr ogram f or a t otal of 2,099 da ta poi nts f rom 65 engines. T he E CM t orque de ltas w ere nor malized b y t he m aximum E CM t orque. T he t orque deltas are shown in Figure 76

. The e rrors as a p ercentage of t he m aximum E CM t orque a re r elatively constant throughout t he e ntire m easured r ange i ndicating t hat t he e rror a s a pe rcentage of poi nt w ould increase as t he abs olute t orque de creases. The much smaller da ta s et us ed for t he ga seous measurement allowance program showed constant err ors as a percentage of point rather than a percentage of maximum. The plot of ECM fuel flow errors as a percentage of maximum ECM fuel flow is shown in Figure 77

.REPORT 03 .14936. 12 94 of 174 FIGURE 76 . OEM SUPPLIED TORQU E ERRORS

FIGURE 77 . OEM SUPPLIED FUEL FLOW ERRORS 

### - 20% - 15% - 10%

- 5%

0% 5% 10% 15% 20%

0.0 0.2 0.4 0.6 0.8 1.0 1.2(ECM Torque-Lab Torque)/Max ECMTorque



### ECM Torque/Max ECM Torque

- 30% - 20% - 10% 0% 10% 20% 30%

0.0 0.2 0.4 0.6 0.8 1.0 1.2(ECM Fuel-Lab Fuel)/Max ECM Fuel



### ECM Fuel/Max ECM FuelREPORT 03 .14936. 12 95 of 174 The fuel flow errors were also constant in relation to max fuel flow rate above 20 percent fuel flow. Since the fuel flow is expected to remain well above 20 percent during NTE event operation, the smaller errors at low fuel flows were not considered important. Although the OEM supplied torque and fuel flow error surfaces were created as percentages of point fo r the gaseous measurement al lowance, t hese s urfaces w ere generated as pe rcentages of m aximum f or t he current program. The 1 st , 50 th , and 99 th percentiles were calculated for torque and fuel flow and sampled in the model using a normal distribution. The err or surface deltas are shown in Table 2 1.

TABLE 21 . OEM ERROR SURFACE DELTAS FOR TORQUE AN D FUEL FLOW

Parameter Percentiles 1 st
% Point , 50 th
% Point , 99 th
% Point ,
Torque

- 7.6

- 1.7 4.1 Fuel Flow

- 4.5

- 0.1 4.9REPORT 03 .14936. 12 96 of 174 5.0 ENVIRONMENTAL TESTIN G AND RESULTS

A s eries of t ests w ere conduc ted to characterize t he P EMS r esponse unde r i n

- use conditions such as changes in pressure, temperature, and humidity, as well as their response to electromagnetic and radio frequency interference and shock and v ibration.

Initially the te st pl an called for the E PA P M G enerator s ystem to be u sed as the P M source for all e nvironmental t esting. T he P M generator i s c apable o f p roducing s oot, vol atile hydrocarbons, and sulfuric acid to simulate the PM emitted from a diesel engine. However, after the system was operated at SwRI it was discovered that the PM generator was too tall to fit into the a ltitude c hamber. It was like ly th at it w ould not be pos sible to operate the P M generator under atmospheric conditions while the output was subject to a constantly varying pressure. The diffusion rate of the hydrocarbon vials is pressure dependant so it would not have been possible to m aintain a c onstant PM s ource w hile v arying t he pr essure i nside of t he chamber. S wRI proposed u sing a J ing mini

- CAST s oot g enerator i n pl ace of t he P M g enerator. T he s oot generator is only a fraction of the size and much easier to operate compared to the PM generator. The steering committee agreed to allow the soot generator to be used for the altit ude testing but requested that the PM generator be used for the temperature and humidity testing.

It w as de cided to operate t he E lectromagnetic Interference / R adio Frequency Interference ( EMI/RFI) and s hock a nd vi bration t esting a s s creening. In t his c ase s creening testing me ant tha t the PEMS w ere ope rated while s ampling z ero air to look for pot ential problems. T he r esults f rom t he s creening t esting w ould t hen be pr esented t o t he s teering committee to decide whether to proceed with official testing to g enerate an error surface. The main motivation to conduct the EMI/RFI and vibration testing as screening was the result of the finding of the gaseous measurement allowance program that in most cases a failure mode of the PEMS was observed only as a malfunction of the system in which it could no longer operated. It was not commonly observed that the accuracy of the PEMS was affected while it continued to measure without detected problems.

SwRI’s M echanical a nd M aterial E ngineering Division ( Division 18) p erformed the environmental testing. T he altitude, temperature and humidity testing was performed b y Rick Pitman and Mike N egrete. The E MI/RFI testing was p erformed by David Smith and Herbert Walker. T he s hock a nd vi bration t esting w as pe rformed b y D avid S mith, M ike N egrete, a nd Mark Orlowski.

5.1 Reference Measurement Testing

An e ight hour b aseline measurement was p erformed f or comparison t o the e ight hour environmental tests. Unlike the gaseous measurement allowance program it was not possible to compare the accuracy of the PM

- PEMS over the measurement period, only the variability. The PM and soot generators provide a particle source, but the correct concentration of the source was unknown.REPORT 03 .14936. 12 97 of 174 During the eight hours of testing the PM concentration and dilution ratio was cycled to allow a more accurate assessment of the PEMS pe rformance over a range of operating conditions. Three PM levels and four dilution ratios were sampled for a total of 12 test conditions. Figure 78 sho ws the schedule of target PM concentration and dilution ratio for one hour of environmental testing.

FIGURE 78 . TARGET DILUTION RATIO AND PM LEVEL PRO FILE FOR ENVIRONMENTAL TESTIN G

Each level was sampled at dilution ratios of 6, 12, 20, and 30 for the Horiba and Sensors PEMS. T he A VL P EMS w as m aintained a t i ts c onstant di lution r atio of 5. T he P EMS w ere maintained at a ta rget d ilution ratio for f ive mi nutes. With 35 seconds remaining, the s ample trigger w as ena bled for 30 seco nds. T his a llowed f or f our m inutes a nd 25 s econds f or stabilization, 30 seconds for sampling, and five seconds after sampling to ensure sampling on all PEMS had stopped before the target dilution ratio was changed. The PEMS were cycled through the four dil ution ratios at a single PM concentration level before the process was repeated at the next concentration level. It took one hour to cycle through each combination of dilution ratio and PM concentration. This profile was repeated eight times for a total test time of eight hours. The schematic showing the experimental setup can be seen in Figure 79. 

### 0 1 2 3 4

0 5 10 15 20 25 30 35

0 10 20 30 40 50 60Target PM LevelTarget Dilution Ratio



### Time (min)

Target DR Target PMREPORT 03 .14936. 12 98 of 174 FIGURE 79 . EXPERIMENTAL SETUP FOR ENVIRONMENTAL TESTING

The particles from the soot generator passed through a Krypton

- 85 neutralizer to bring the cha rge o f t he p articles t o a minimum B oltzmann distribution of c harge [ 7] to minimize particle losses due to electric forces. A large volume was placed downstream o f the neutralizer to minimize the pressure fluctuations observed by the soot generator and also to smooth out any changes in concentration from the generator. One of the AVL PEMS units was placed outside of the environmental chamber upstream of the orifice and overflow so that it was isolated from the chamber conditions. This PEMS served as a reference to verify that the soot concentration from the ge nerator w as s table. T he A VL uni t w as chosen be cause i t c ould pr ovide a r eal t ime measurement of the soot co ncentration. This was not a guarantee that the total PM concentration from the generator was steady, but typically the volatile emissions from the generator would not fluctuate significantly without some change in the soot concentration.

The mini

- CAST s oot generator was used as the particle source for the baseline testing. The num ber m ean di ameter w as approximately 70 nm ba sed on m easurement w ith t he E EPS. The three concentration levels were nominal concentration levels of 25, 75, and 125 μg/mol with
app roximately 30 percent organic carbon based on the OC/EC measurement. Figures 80, 81, and 82 show the concentration measurements during the baseline testing for the Horiba, Sensors, and AVL PEMS, respectively. Figure 83 s hows a comparison between the refere nce M SS and the PEMS MSS which were measuring simultaneously during the baseline testing.REPORT 03 .14936. 12 99 of 174 FIGURE 80 . HORIBA ENVIRONMENTAL BASELINE MEASUREM ENTS

FIGURE 81 . SENSORS ENVIRONMEN TAL BASELINE MEASUREMENTS



### - 20 0 20 40 60 80 100 120 140 160 180 200

0 20 40 60 80 100Measured PM Concentration (μg/mol)



### Observation Number

- 20 0 20 40 60 80 100 120 140 160 180 200

0 20 40 60 80 100Measured PM Concentration (μg/mol)



### Observation NumberREPORT 03 .14936. 12 100 of 174 FIGURE 82 . AVL ENVIRONMENTAL BASELINE MEASUREMENT S

FIGURE 83 . REFERENCE MSS ENVI RONMENTAL BASELINE M EASUREMENTS 

### - 20 0 20 40 60 80 100 120 140 160 180 200

0 20 40 60 80 100Measured PM Concentration (μg/mol)



### Observation Number

0 20 40 60 80 100 120 140

0 20 40 60 80 100Measured PM Concentration (μg/mol)



### Observation Number

Ref MSS PEMS MSSREPORT 03 .14936. 12 101 of 174 Although t he s oot c oncentration f rom t he generator w as l ikely s table based on t he

measu rements from both AVL PEMS it is not clear whether the total PM concentration remained constant throughout the test. In Figure 80, t he Horiba PEMS exhibited a clear downwards trend in measured concentration as t he t est pr ogressed. The r eported PM conc entra tion f rom t he Sensors P EMS in F igure 81 appeared to be s omewhat i ndependent of t he act ual P M concentration sampled. The Sensors data was too scattered to either confirm or dis prove the PM trend obs erved i n t he Horiba da ta. In a ddition t o t he general dow nwards t rend of t he concentration, t he H oriba da ta a ppeared t o s uggest t hat t he a ccuracy of the di lution r atio w as playing a r ole i n t he m easurement. In obs ervations 25 t hrough 96, t he r eported c oncentration decreased each time the dilution ratio target incre ased. This suggests that the Horiba PEMS is has either a positive error on lower dilution ratios or a negative error on higher dilution ratios.

In Figure 83, it is clear that t he PEMS and reference AVL units were both able to resolve the di fferences be tween t he t hree P M l evels cl early even showing s imilar r esponses t o small changes i n concentration. Figure 84 depicts t he r elationship between t he A VL r eference measurement and the AVL PEMS measurement.

FIGURE 84 . REFERENCE AVL VERSUS PEMS AVL FOR ENVIRONMENTAL BASELINE

The correlation coefficient between the two measurements is excellent, better than 0.99. The slope of 0.78 is likely to be a combination of a difference in response of the two instruments and line losses between the two points of measurement. 

### y = 0.7762x R2 = 0.9991

0 20 40 60 80 100 120

0 20 40 60 80 100 120 140AVL PEMS Concentration (μg/mol)



### AVL Reference Concentration (μg/mol)REPORT 03 .14936. 12 102 of 174 5.2 Pressure Chamber Testing

The SwRI altitude chamber is capable of simulating altitudes up to 19.8 km. The chamber is 1.5 m in diameter and 2.1 m tall. Typically the simulated altitude chamber at SwRI is operated only unde r v acuum t o s imulate a ltitudes g reater t han t hat of S an A ntonio. T he g aseous measurement allowance t est plan called for pressures up t o 101.87 kP a or 45 m eters below sea level. T he e levation of S an A ntonio i s a pproximately 240 m eters a bove s ea l evel w ith a barometric pressure near 99 (98.4 based on altitude) kPa. In the gaseous measurement allowance
prog ram, the altitude chamber underwent significant alterations to achieve positive pressures and still m any p roblems w ere en countered. The S wRI engineer i n cha rge o f t he altitude ch amber requested that only negative pressures be tested to preserve the integri ty of their test equipment. Simply changing the positive pressures to ambient would have resulted in a large portion of the testing being conducted at nor mal a tmospheric pr essure. Instead a s light ne gative pr essure of 94.3 kPa (610 m, 2,000 ft) was repeated twice, once at 1.8 hours and once at 7.2 hours. Figure 85 shows the original pr essure profile from the test plan as w ell as the modified profile that was used during testing.

FIGURE 85 . ORIGINAL AND REVIS ED PROFILE FOR ALTITUDE TESTING

Significant efforts were devoted to ensuring that a stable PM source could be generated that was insensitive to pressure. Because the soot generator contains an open flame operating at atmospheric pr essure, the pr operties o f t he flame

, and hence t he p article generation

, tended to change with the pressure of the outlet. By placing an orifice in the transfer line between the soot generator and the pr essure cha mber i t w as pos sible t o operate t he s oot g enerator at a hi gher pressure and maintain a constant pressure through adjustment of an overflow valve upstream of the or ifice. A djustments w ere onl y n ecessary du ring t he pr essure r amps to 82.7 kP a a nd 90.0 kPa. The valve was adjusted to maintain a constant pressure upstream of the orifice. Figure 86 shows the setup outside of the altitude chamber. - 800

- 300

200

700

1200

1700 80 85 90 95 100 105 110

0 1 2 3 4 5 6 7 8Altitude (m)Pressure (kPa)

Time (hr)

Original RevisedREPORT 03 .14936. 12 103 of 174 FIGURE 86 . ALTITUDE TESTING CHAMBER

During the first practice run

, the AVL 3 blew a fuse in its measurement unit. The test was stopped and the fuse was replaced, but i mmediately bl ew a gain indicating an electrical failure within the unit. The unit was replaced with AVL 2 and testing continued with the problem not observed again. Du e to space constraints, the Semtech DS and Horiba OBS

- 2200 gaseous PEMS were located outside of the environmental chamber. Since the purpose of these devices was only communications, it was not considered necessary to test them inside the chamber. The exte rnal compressor us ed for t he H oriba di lution a ir w as a lso i nstalled out side the c hamber. A l arge compressor was supplied by SwRI to provide oil

- less dilution air. The steering committee had requested t hat t he H oriba s upplied c ompressor s hould be us ed but then a greed t o allow t he replacement compressor after it was determined that the original compressor could not operate for the entire eight hours without shutting off. A picture of the PEMS installed in the altitude chamber is shown in Figure 87

.REPORT 03 .14936. 12 104 of 174 FIGURE 87 . PEMS INSTALLED IN THE ALTITUDE CHAMBER

As mentioned previously, analysis of the environmental data was more difficult for the PM

- PEMS pr ogram c ompared t o t he gaseous PEMS pr ogram du e t o t he l ack of a know n reference c oncentration. F or t his r eason, onl y the va riability o f t he P EMS m easurement i n comparison to its average was compared. The individual data points were plott ed as well as the average levels to show how far each measurement deviated from the average. Figure 88 shows the concentration measurements by the Horiba.

FIGURE 88 . HORIBA ENVIRONMENTAL PRESSURE MEASUREM ENTS 80 82 84 86 88 90 92 94 96 98 100

0 20 40 60 80 100 120 140

0 20 40 60 80 100Chamber Pressure (kPa)PM Concentration (μg/mol)

Observation Number

Avg Horiba Horiba PressureREPORT 03 .14936. 12 105 of 174 The Horiba concentration measurements were not consistent for different dilution ratios. In the baseline test it was observed that increasing the dilution ratio in the range of 6, 12, 20, and 30 t ended t o de crease t he r eported c oncentration. In t he pr essur e t est, it appe ars t hat t he measured concentration increases as the dilution ratio increases. The largest deviations from the average occurred during observations 53

- 60 while the pressure was at its lowest indicating that the a mbient pr essure doe s ha ve s o me ef fect on the accur acy of the H oriba r eported concentration. D uring t he pe riod b etween obs ervations 53 a nd 60, t he s ystem w as una ble t o maintain its target total flow rate of 30 slpm, as shown in Figure 89.

FIGURE 89 . HORIBA TOTAL FLOW AND DILUTION RATIO DURING PRESSURE TESTING

The total flow dropped to approximately 28.7 slpm during this period which is below the acceptable t olerance a ccording t o t he m anufacturer, causing t he da ta t o be i nvalidated. T he system was however able to maintain proportionality during this part of the test. The short spikes in the total flow is from each switch from sample to bypass mode.

The S ensors da ta, a s s hown i n F igure 90, e xhibited a l arge a mount of v ariability w ith some da ta m ore t han a f actor of t wo hi gher a nd lower t han t he a verage. With t hat a mount of scatter it was difficult to visually discern an effect of pressure on the measurement.

The AVL data, as shown in Figure 91, was grouped tightly around the average except during t he l ow pr essure e xcursion a round observances 53

- 60. Because of t he ex cellent repeatability of t he m easurement, the e ffect of pr essure on the m easurement w as r eadily apparent. As the ambient pressure decreased, the measurement decreased as well. 

### 80 82 84 86 88 90 92 94 96 98 100

0 5 10 15 20 25 30 35 40

0 20 40 60 80 100Pressure (kPa)Dilution Ratio, Total Flow (slpm)



### Observation Number

Total Flow Dilution Ratio Chamber PressureREPORT 03 .14936. 12 106 of 174 FIGURE 90 . SENSORS ENVIRONMEN TAL PRESSURE MEASUREMENT S

FIGURE 91 . AVL ENVIRONMENTAL PRESSURE MEASUREMENT S 

### 80 82 84 86 88 90 92 94 96 98 100

0 50 100 150 200 250

0 20 40 60 80 100Chamber Pressure (kPa)PM Concentration (μg/mol)



### Observation Number

Avg Sensors Sensors Pressure

80 82 84 86 88 90 92 94 96 98 100

0 10 20 30 40 50 60 70 80

0 20 40 60 80 100Chamber Pressure (kPa)PM Concentration (μg/mol)



### Observation Number

Avg AVL AVL PressureREPORT 03 .14936. 12 107 of 174 To determine w hether a n error s urface w ould be g enerated for ea ch PEMS, the M AD from the baseline was compared to the MAD of the pressure test. If the MAD of the pressure test

wa s greater, an error s urface was generated to present t o the s teering committee. The pl ots of median versus MAD shown in Figures 92, 93, a nd 94 were presented to the steering committee at the meeting in Indianapolis on July 15 th , 2009.

FIGURE 92 . AVL PRESSURE MEDIAN VERSUS MAD

FIGURE 93 . HORIBA PRESSURE MEDIAN VERSUS MAD y = 0.022x

- 0.124 R2 = 0.999 y = 0.047x + 0.539 R2 = 0.413

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0

0 20 40 60 80 100 120MAD PM (μg/mol)

Median PM Concentration (μg/mol)

AVL

- Baseline AVL

- Pressure

y = 0.073x + 5.936 R2 = 0.427

y = 0.121x

- 4.006 R2 = 0.991

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0

0 20 40 60 80 100 120MAD PM (μg/mol)

Median PM Concentration (μg/mol)

Horiba

- Baseline Horiba

- PressureREPORT 03 .14936. 12 108 of 174 FIGURE 94 . SENSORS PRESSURE MEDIAN VERSUS MAD

The M AD va lues were higher f or t he b aseline compared t o the p ressure t est f or bot h Horiba a nd S ensors i ndicating t hat no a ddition al va riability in the me asurement c ould be attributed t o c hanges i n pr essure us ing t his t echnique. N o environmental e rror s urface w as calculated for either Horiba or Sensors. The AVL MAD values were higher for the pressure test compared t o t he ba seline, s o an e rror s urface w as de veloped. It s hould be not ed t hat t he variability of the AVL is much lower for the AVL compared to the Horiba and Sensors, however because of the high precision of the AVL measurements during the baseline it was still possible to discern the added variability due to changes in ambient pressure. The environmental pressure error surface was calculated using the same pooled rms technique that was used for the transient error surface. Figure 95 shows the error surface that was generated for environmental pressure on the AVL PM. The error surface was accepted for use by the steering committee during the July 15 th 

### y = 0.155x + 15.834 R2 = 0.756

y = 0.111x + 15.755 R2 = 0.609

0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0

0 20 40 60 80 100 120 140MAD PM (μg/mol)



### Median PM Concentration (μg/mol)

Sensors

- Baseline Sensors

- Pressure

, 2009 meeting in Indianapolis.REPORT 03 .14936. 12 109 of 174 FIGURE 95. FINAL ERROR SURFACE FOR ENVIRONMENTAL PRESSURE AVL PM CONCENTRATION

This e rror s urface w as similar in magnitude to the A VL tr ansient P M e rror s urface, however it was much smaller than the AVL steady

- state PM error surface.

5.3 Temperature and Humidity Chamber Testing

The temperature and humidity chamber testing was designed to characterize the effects of ambient temperature and humidity on the accuracy of PM measurement. The testing chamber has the capability of independently controlling the temperature and moisture content of the air in the range o f t ypically obs erved l evels from ambient c onditions. T he t emperature a nd hum idity profile from the gaseous program was modified based on data acquired during CE

- CERT testing. At the June 12 th - 12

- 8 - 4 0 4 8 12

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Median Flow Weighted PM Concentration ( μ g/mol)

1st Percentile 99th Percentile

, 2008 m eeting in Madison data was presented by Kent Johnson indicating that temperatur es near the PEMS instruments could be above 60°C during in

- use testing. With this in mind the steering committee elected to add temperatures as high as 50°C to the profile, where the original p rofile h ad a maximum t emperature of j ust ove r 30°C . T he or iginal a nd m odified temperature and humidity profiles are shown in Figure 96.REPORT 03 .14936. 12 110 of 174 FIGURE 96 . TEMPERATURE AND HUMIDITY PROFILE FOR ENVIRONMENTAL TESTING

The temperature chamber was unable to control the humidity under a temperature of 5°C so t he hum idity was un controlled dur ing t he po rtion of t he cycle w here t he t emperature w as between 2°C and 5°C. The moisture content was quite small during this portion o f the cycle, so this was not considered to be a significant issue. Figure 97 shows the PEMS in the temperature and humidity chamber.

FIGURE 97 . PEMS INSTALLED IN THE TEMPERATURE AND HUMIDITY CHAMBER 0 10 20 30 40 50 60 70 80 90 100

0 10 20 30 40 50 60

0 1 2 3 4 5 6 7 8Relative Humidity (%)Temperature (°C)

Time (hr)

Original Temp Modified Temp Original RH Modified RHREPORT 03 .14936. 12 111 of 174 The c ompressor pi ctured w as pr ovided b y S wRI f or us e dur ing environmental t esting. The or iginal c ompressor s upplied b y H oriba was m uch s maller, but a lso ha d di fficulty completing eight hours of operation without shutting off. The compressor was included in the temperature chamber so that the dilution air f or the Horiba system would be affected by the same changes in temperature and humidity that the dilution air of the Sensors and AVL systems would experience. The P M s ample w as t ransported into the cha mber us ing a he ated sample l ine maintained a t 60°C . T he temperature w as s et s lightly above the ma ximum te mperature of the chamber so that a constant temperature in the sample could be maintained throughout the test. There w as a s mall por tion of t he e nd of t he t ransfer l ine t hat w as n ot he ated, but i t w as
extens ively i nsulated to minimize t emperature effects. The E PA P M ge nerator w as us ed in conjunction with the soot generator in this work to provide the particle source. The experimental setup was shown previously in Figure 79. The PM generator is shown in Figure 98.

FIGURE 98 . THE PM GENERATOR I NSTALLED OUTSIDE THE TEMPERATURE AND HUMIDITY CHAMBER

The Horiba and Sensors gaseous PEMS were installed outside of the chamber along with the PM generator. The PM generator is designed to add volatile hydrocarbons, s ulfuric acid, and water vapor to an elemental carbon source. An oxidation catalyst removed any volatile from the soot generator before volatile was added from the diffusion vial ovens. Unfortunately both the sulfur oven and the syringes injecting water malfunctioned during testing and neither was able to be quickly repaired. For the official temperature testing the particle source consisted of elemental carbon from the mini-CAST and volatile hydrocarbon from the PM generator.

Figure 99 s hows t he i ndividual H oriba c oncentration m easurements along with a comparison of t he average concentration measurement f or e ach P M level. T he ch amber temperature is included for reference as well.REPORT 03 .14936. 12 112 of 174 FIGURE 99 . HORIBA ENVIRONMENT AL TEMPERATURE MEASUREMENTS

The Horiba dat a plotted in Figure 99 showed a significant amount of variability although it is unc lear f rom thi s g raph whether the v ariability is di rectly r elated to the c hanging temperature. For ex ample, the m easurements be tween obs ervation 25 and 35 when the temperatu re is below 10 °C tend to be below the c ycle average. However, the second time the temperature dr ops be low 10°C a round obs ervation 70, t he m easurements a re a bove t he c ycle
average. Figure 100 s hows t he c hanges i n f ilter di lution a ir a nd c hamber t emperature. T he behavior of these variables cannot clearly explain why the PM concentration behaved the way it did in Figure 99.

FIGURE 100. HORIBA TEMPERATURES DURING ENVIRONMENTAL TEMPERATURE TESTING 0 10 20 30 40 50 60

0 20 40 60 80 100 120 140 160 180 200

0 20 40 60 80 100Chamber Temperature (°C)PM Concentration (μg/mol)

Observation Number

Avg Horiba Horiba Temperature

0 10 20 30 40 50 60

0 20 40 60 80 100Temperature (°C)

Observation

Filter Temp Dilution Air Temp Chamber TempREPORT 03 .14936. 12 113 of 174 Figure 101 shows t he i ndividual S ensor c oncentration m easurements as w ell a s t he average concentration for each PM level. The temperature measurements for the Sensors PEMS are shown in Figure 101. The Sensors data displayed a high degree of variation from the mean

w ith no c lear t rend r elated t o t emperature. N o valid da ta w as captured f or t he l ast hour o f operation because t he S ensors P EMS w as una ble t o maintain the cr ystals at a t emperature of 50°C when the chamber temperature was above 47°C.

FIGURE 101 . SENSORS ENVIRONMENTAL TEMPER ATURE MEASUREMENTS

The AVL data, shown in Figure 102, e xhibited excellent repeatability in comparison to the other two PEMS. Any temperature dependence by the AVL PEMS was extremely small.

FIGURE 102 . AVL ENVIRONMENTAL TEMPERATURE MEASUREMENTS 0 10 20 30 40 50 60

0 50 100 150 200 250

0 20 40 60 80 100Chamber Temperature (°C)PM Concentration (μg/mol)

Observation Number

Avg Sensors Sensors Temperature

0 10 20 30 40 50 60

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100Chamber Temperature (°C)PM Concentration (μg/mol)

Observation Number

Avg AVL AVL TemperatureREPORT 03 .14936. 12 114 of 174 The t emperature da ta w as com pared in the s ame m anner as t he pr essure da ta w ith the MAD values plotted in relation to the median, as shown in Figures 103, 104, and 105, for the Horiba, Sensors, and the AVL PEMS, respectively. The MAD of the Horiba baseline is slightly higher than during the temperature test meaning that an error surface was not generated in this case. The Sensors and AVL MAD was higher for the baseline at lower concentrations, but higher for t he t emperature data for hi gher concentrations. An attempt was m ade t o calculate an error surface f or S ensors, but t he pool ed r ms t echnique r esulted i n a s lightly hi gher va lue f or t he baseline indicating that an error surface was not necessary. The error surface calculated for AVL using the s ame technique is shown in Figure 10 6. The PM median concentrations tested w ere between 25 a nd 135 μ g/mol. I t w as ne cessary t o e xtend the e rror s urface out t o ne arly 500 μg/mol to encompass the range of concentrations encountered during engine testing. T he steering committee elected to cap the error surface at plus and minus 5.3 μg/mol because it was unclear whether t he errors w ould c ontinue t o i ncrease outside of t he c oncentrations obs erved i n t he temperature and humidity testing. Extending a straight li ne out to the median concentration of 481 μg/mol would have resulted in a 5 th percentile of 18.9 μg/mol.

FIGURE 103 . HORIBA TEMPERATURE MEDIAN VERSUS MAD 

### y = 0.073x + 5.936 R2 = 0.427

y = 0.084x + 3.106 R2 = 0.767

0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0

0 20 40 60 80 100 120 140 160MAD PM (μg/mol)



### Median PM Concentration (μg/mol)

Horiba

- Baseline Horiba

- TemperatureREPORT 03 .14936. 12 115 of 174 FIGURE 104 . SENSORS TEMPERATURE MEDIAN VERSUS MAD

FIGURE 105 . HORIBA TEMPERATURE AND HUMIDITY MEDIAN VERSUS MAD 

### y = 0.155x + 15.834 R2 = 0.756

y = 0.040x + 22.144 R2 = 0.446

0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0

0 20 40 60 80 100 120 140 160MAD PM (μg/mol)



### Median PM Concentration (μg/mol)

Sensors

- Baseline Sensors

- Temperature



### y = 0.022x

- 0.124 R2 = 0.999 y = 0.006x + 1.072 R2 = 0.962

0.0 0.5 1.0 1.5 2.0 2.5

0 20 40 60 80 100 120 140 160MAD PM (μg/mol)



### Median PM Concentration (μg/mol)

AVL

- Baseline AVL

- TemperatureREPORT 03 .14936. 12 116 of 174 The AVL temperature data is shown in Figure 106 including the individual and average measurements as well as the chamber temperature.

FIGURE 106. FINAL ERROR SURFACE ENVIRONMENTAL TEMPER ATURE AND HUMIDITY AVL PM

5.4 Elec tromagnetic and Radio Frequency Interference Screening

The electromagnetic inference (EMI) and radio frequency interference (RFI) testing was conducted a s a s creening e xercise w ithout t he us e of P M s ource. D uring t he gaseous measurement allowance program the majority of the problems encountered caused a malfunction of t he P EMS t o t he p oint w here i t w ould no l onger op erate. T he main pur pose of t he environmental testing was not to test the durability of the PEMS, but to quantify any errors that might occur during the operation of a PEMS that would cause a measurement error. The same series of t ests f rom t he g aseous E MI/RFI t esting w ere c onducted a s a s eries of i ndividual screening t ests us ing o nly HEPA f iltered air. Based on the s creening results i t r emained a possibility to conduct a full test cycle using a particle source to generate an error surface.

By providing filtered room air to the PEMS it w ould still has been possible to detect a wide range of possible measurement accuracy issues, although some problems may only present themselves when a particle sample is present. To shorten the test time, the PEMS were triggered continuously s o tha t ti me in between te sts c ould be mini mized. This me ant th at the Horiba system was continuously sampling on a filter, and the Sensors system was continuously sampling on a crystal. The Horiba system was able to sample continuously for eight hours or more on the filter s ince no particle s ource w as loa ding on the f ilter. However, t he Sensors s ystem ha d a maximum sample tim e that was adjusted so that it would cycle through the crystals every 120 seconds. This was desirable since it exercised the operation of all eight crystals however some issues surfaced due to this testing technique. The problems included: both high voltage power supplies turning on at the same time, two crystals sampling at the same time, no crystal sampling - 12

- 8 - 4 0 4 8 12

0 100 200 300 400 500PM Concentration Delta (μg/mol)

Median Flow Weighted PM Concentration ( μ g/mol)

1st Percentile 99th PercentileREPORT 03 .14936. 12 117 of 174 even t hough on e w as a vailable. E ach of t hese pr oblems oc curred i nfrequently, a nd i t w as believed that the problems were a result of leaving the trigger on continuously for long periods of time s ince none of t hese i ssues were obs erved during en gine t esting. These were functionality issues that did not appear to have any impact on the accuracy of the measurement.

The EMI and RFI testing , shown in Figure 107, was conducted in a radiation chamber , as shown in Figure 105, with walls covered with large cones of carbon impregnated foam designed to a bsorb r adiation a nd minimize r eflections. F our s tandard S ociety of Automotive E ngineers (SAE) te sts w ere c onducted: B ulk C urrent Injection, Radiated Immunity, E lectrostatic Discha rge, and Conducted Transients.

FIGURE 107 . AVL PEMS IN THE RADIATION CHAMBER FOR EMI AND RFI TESTING

The S ensors a nd AVL PEMS w ere bot h de signed t o r un of f t he ve hicle’s 12 V olt electrical s ystem s o bot h s ystems w ere pow ered us ing a 12 V a utomotive b attery t hat w as continually charged using a 120 VAC charger supplied by Sensors. The Sensors PEMS was a 12V system so it connected directly to the battery. The AVL system is powered by 120 Volts 60 Hz AC power so a commercially available inverter was provi ded by AVL to convert the 12 volts DC into 120 volts AC. The Horiba system was designed to operate using a generator, therefore it was still powered using the 120 VAC wall outlets.

5.4.1 Bulk Current Injection

SAE test J1113/4 Immunity to Radiated Electromagnetic Fields
– Bulk Current Injection was performed to determine the effect of electromagnetic radiation on the electrical cables of the PEMS. The specifications used are detailed in Region 2, Class B of the J1113/4 test protocol. A calibrated current probe was place around the electrical cable and used to inject RF current into the cable. Figure 108 shows a cable running through the bulk current injection probe.REPORT 03 .14936. 12 118 of 174 FIGURE 108 . BULK CURRENT INJECTION PROBE

The probe was positioned at 120 mm, 450 mm, and 750 mm from the cable connector to test the cable three times. For each test, the frequency of the current was stepped from 1 MHz to 400 MHz using the following step sizes.

1 MHz to 10 MHz – 1 MHz step size 10 MHz to 200 MHz – 10 MHz step size 200 MHz to 400 MHz – 20 MHz step size

These were the maximum allowed step sizes according to the SAE protocol. As with the gaseous pr ogram a 5 s econd dwell t ime w as used to ensure t he electromagnetic f ield had stabilized before s witching to the ne xt f r equency. T he pr obe w as c alibrated t o de liver 60 milliamps of c urrent a s s pecified in the te st p rocedure. Figure 109 s hows t he bul k c urrent injection probe being used to test a cable on the Horiba PEMS. Figure 110 and Figure 1 11 show the sensors and AVL bulk current injection setup, respectively.

FIGURE 109 . HORIBA PEMS SETUP FOR BULK CURRENT INJ ECTION 

### BCI Probe Test CableREPORT 03 .14936. 12 119 of 174 FIGURE 110 . SENSORS SETUP DURING BULK CURRENT INJE CTION TESTING

FIGURE 111 . AVL SETUP DURING BULK CURRENT INJECTIO N TESTINGREPORT 03 .14936. 12 120 of 174 The c ables tested on the H oriba s ystem w ere th e s ample line temperature, sample line heater power, he ated enclosure h eater power, sample trigger and exhaust flow signal, ethernet connection f rom t he D CS t o t he e lectrical e nclosure, e thernet c onnection f rom the T RPM t o 2200, DCS 12 volt power, and two AC power cords. There were two main problems discovered when the bulk current injection was applied to the cables of the Horiba PEMS. When the line supplying power to the heater in the heated enclosure was probed, the reported total flow of the system and started t o r apidly f luctuate a round t he f requency o f 10 M Hz. T his f requency w as manually repeated after t he s weep and t he s ame be havior oc curred. Figure 112 shows t he response of several of the Horiba signals to the current injection.

FIGURE 112 . HORIBA FLOW DISTURBANCE FROM BULK CURR ENT INJECTION

The noise in the system appeared to have originated with several of the temperature and pressure m easurements i ncluding t he f ilter f ace t emperature and total f low temperature s hown here. The total flow temperature and pressures began to fluctuate causing fluctuations in the total flow measurement, which in turn caused the dilution flow to fluctuate in an effort to maintain proportionality. T his pr oblem w as onl y obs erved on t he c able s upplying he ater pow er t o t he heated enclosure box.

The communications between the laptop and the PEMS was disrupted when the Ethernet cables w ere pr obed. S everal t imes t he c ommunications dr opped out c ompletely. T he p roblem was intermit tent around 40 to 50 MHz and 140 to 160 MHz.

The f inal m ajor pr oblem ex perienced by t he Horiba s ystem dur ing B CI w as a l arge amount of noise in the exhaust flow signal when the analog signal cable between the OBS -2200 and OBS

- TRPM was probed. The exhaust flow is measured by the Horiba gaseous PEMS OBS

2200. The OBS

- 2200 outputs the exhaust flow measurement as an analog voltage which is then 

### 0 10 20 30 40 50 60 70 80

0 5 10 15 20 25 30 35

0 500 1000 1500 2000Temperature (°C)Flow (slpm)



### Time (sec)

OB

- PFSS_Total_1 [L/min] OB

- PFSS_Dil_1 [L/min]

OB

- PFSS_Filter_T [degC] OB

- PFSS_Tot_T [degC] 10 MHz During Sweep 10 MHz RepeatedREPORT 03 .14936. 12 121 of 174 read by the OBS -TRPM Over frequencies between 1 and 40 MHz the exhaust flow signal read as high as 3,000 kg/hr on th e OBS

- TRPM while the measured value on the OBS

- 2200 was less than 10 kg/hr. The exhaust flow signal during the BCI sweep is shown in Figure 113.

FIGURE 113. HORIBA EXHAUST FLOW NOISE ON ANALOG CABLE DURING BULK CURRENT INJECTI ON

It was clear that the problem involved only the transmission of the exhaust flow signal as an analog voltage between the two systems, not the actual measurement of the exhaust flow. For in use testing this problem would be easily detectable when the two exhaust flow data from t he two different files are compared; however, the TRPM relies on the analog exhaust flow signal for adjusting i ts di lution r atio f or pr oportional s ampling. E rrors i n t he e xhaust f low s ignal w ould cause the TRPM to lose its proportionality, and therefore an y data collected with the same filter sampling would be voided.

The cables tested on the Sensors system were the communication cable from the PPMD to DS, and the DC power. As mentioned previously, many of the problems that were encountered with t he S ensors P EMS ha ppened i ntermittently a nd c ould not be reproduced a t any s pecific frequencies. It w as assumed t hat pr oblems i nvolving c rystal s ampling s witching, hi gh vol tage power s upplies, a nd f luctuations i n t he c orona current were not i nduced b y t he bul k c ur rent injection. H owever, t he S ensors s ystem di d e xperience s ignificant c ommunication pr oblems when t he s erial c able be tween t he P PMD a nd DS uni ts w as pr obed. T he c ommunication w as repeatedly disrupted over the entire frequency range causing a loss of data. In nearly every case the c ommunication w as r ecovered onc e t he r adiation s ubsided a llowing nor mal ope ration t o continue. 

### - 500 0 500 1000 1500 2000 2500 3000 3500

0 200 400 600 800 1000Exhaust Flow (kg/hr)



### Time (sec)

OBS

- TRPM OBS

- 2200REPORT 03 .14936. 12 122 of 174 The cabl es t ested on the A VL s ystem w ere t he com munications cabl e be tween the measuring and conditioning unit, the analog output to the Semtech DS, and two AC power cords. The BCI induced a significant amount of noise on the analog signal cable between the AVL MSS and the Sensors Semtech DS. The AVL system can log its own data, but the data used officially in this program and during in use testing is the data logged by the Semtech DS. Two signals are carried on t he analog output wire: the measured concentration and the dilution ratio. These two numbers are multiplied together for the final reported concentration. The measured concentrat ion did not see significant noise, but the dilution ratio signal was strong ly influenced by radiation in the range of 1

- 3 MHz and 40 M Hz. A comparison of the MSS measured dilution ratio and the dilution ratio recorded by the Semtech DS is shown in Figure 1 14.

FIGURE 114 . BCI NOISE ON AVL ANALOG OUTPUT CABLE

The dilution ratio doubled from 5 to 10 during current injection at 1

- 3 MHz. The data in the pl ot w as recorded d uring t he f requency s weep, a nd t he pr oblem w as r eplicated m anually afterwards. S ince t he t rue di lution r atio w as s till r ecorded i n t he M SS l og f ile, t he c orrect reported concentration could be recovered after testing.

5.4.2 Radiated Immunity

The r esponse of t he P EMS t o c ontinuous na rrowband e lectromagnetic f ields w as measured using SAE test J1113/21 Electromagnetic Compatibility Measurement Procedure for Vehicle Components
– Part 21: Immunity to Electromagnetic Fields, 10 kHz to 18 GHz, Absorber- Lined Chamber. The ex act s pecifications of t he t ests pe rformed were dr awn from Region 2, C lass B of t he J 1113/21 pr otocol. Several di fferent a ntennae were us ed t o generate electromagnetic r adiation over t he f requency r ange of 10 kHz t o 1 GHz. T he e lectromagnetic 

### 0 2 4 6 8 10 12

0 100 200 300 400 500Dilution Ratio



### Time (sec)

Semtech DS Analog DR AVL MSS DR 1

- 3 MHz

40 MHzREPORT 03 .14936. 12 123 of 174 susceptibility experts at SwRI recommended ending the test at a frequency of 1 GHz rather than 18 GHz due to the very small probability of detecting any susceptibility above 1 GHz. This same approach was used in the gaseo us PEMS program. The carbon impregnated foam walls of the radiation chamber was designed to absorb any radiation so that the PEMS would only see the direct r adiation ge nerated b y t he a ntenna. T he f ollowing s tep s izes were us ed dur ing t he frequency sweeps:

**•** 10 kHz to 100 kHz – 10 kHz step size

**•** 100 kHz to 1 MHz – 100 kHz step size

**•** 1 MHz to 10 MHz – 1 MHz step size

**•** 10 MHz to 200 MHz – 2 MHz step size

**•** 200 MHz to 1 GHz – 20 MHz step size

The S AE s tandard f ield i ntensity o f 50 vol ts/meter was us ed w ith bot h ve rtical a nd horizontal electromagnetic radiation orientations. Figure 115, Figure 116, and Figure 117 show the Horiba, Sensors, and AVL radiated immunity testing setup, respectively.

FIGURE 115 . HORIBA PEMS SETUP DURING RADIATED IMMU NITY TESTINGREPORT 03 .14936. 12 124 of 174 FIGURE 116 . SENSORS PEMS SETUP DURING RADIATED IMMU NITY TESTING

FIGURE 117 . AVL PEMS SETUP DURING RADIATED IMMUNIT Y TESTINGREPORT 03 .14936. 12 125 of 174 The H oriba P EMS e xperienced several pr oblems r elated to the r adiated immunity t est including a l oss of c ommunications between t he l aptop a nd t he i nstrument. T his pr oblem occurred at several different frequencies for both horizontal and vertical polarizations. Because the data is logged on t he laptop any loss of communication is also a loss of data. This would result in a voided test since the filter calibration relies on collecting data the entire time the filter is s ampling. A p roblem was a lso obs erved with the c ontrol of t he di lution r atio, a s s hown i n Figure 118. During EMI and RFI testing the Horiba dilution ratio was maintained constant at 6.

FIGURE 118 . HORIBA DILUTION RATIO FLUCTUATIONS DURING RADIATED IMMUNITY

The dilution ratio varied between 338 and

- 938 but only a smaller portion of the graph is shown for more detail. The external flowmeter on the end of the sample flow indicated that these rapid changes in flow were real and not just reported. The problem with the exhaust flow analog output signal first encountered during the bulk current injection testing was also observed during radiated immunity.

The A VL system reported a supply voltage error in the range of 200 M Hz to 1 G Hz of horizontal r adiation. T he i nverter pow ered do wn a t 300 M Hz s hutting dow n t he s ystem completely.

5.4.3 Electrostatic Discharge

SAE t est J 1113/13 Electromagnetic Compatibility M easurement Procedure for Vehicle Components— Part 13: Immunity to Electrostatic Discharge was pe rformed to test t he P EMS response t o Electrostatic D ischarges ( ESDs) on surfaces and connectors. T he ex act pr ocedure used was taken from Region 2, Class B of the J1113/13 standard. Approximately 40 ESDs were 

### - 100

- 80 - 60 - 40 - 20 0 20 40 60 80 100

0 500 1000 1500 2000 2500 3000 3500Dilution Ratio



### Time (sec)

Measured Dilution Ratio Setpoint Dilution Ratio 50

- 70 MHz Vertical 120

- 150 MHz VerticalREPORT 03 .14936. 12 126 of 174 supplied to each PEMS at various connector por ts, exposed screws, and general surfaces. The test included both direct contact discharges as well as indirect discharges with the electrostatic discharge gun placed near the surface of interest. The ESD gun is shown in Figure 119.

FIGURE 119 . ELECTROSTATIC DISC HARGE SIMULATOR

The E SD g un w as c alibrated us ing a n e lectrostatic vol tmeter t o de liver 4000 vol ts pe r discharge. Neither the Horiba nor the AVL PEMS exh ibited any susceptibility to the ESD tests. The Sensors unit shut off completely with an ESD near the auxiliary connector port to which an external f low m eter c onnects f or a udit pur poses. T his pr oblem w as c onfirmed w ith a s econd ESD. No issues possibly rel ated to measurement accuracy were found with any of the PEMS.

5.4.4 Conducted Transients

The response of the PEMS to voltage disturbances in the 12 volt power supply cable was checked using SAE test J1113/11 Immunity to Conducted Transients on Power Le ads. The tests were c onducted us ing s pecifications f ound i n R egion 2, C lass B of t he J1113/11 pr otocol. A Schaffner NSG 5000 Automotive Electronics Test System was installed in between the 12 vol t power s upply a nd t he P EMS. T he S chaffner E lectronics T est S ystem de livered vol tage perturbations to the PEMS of varying magnitudes and durations. The voltage spikes range d from

- 200 t o 100 vol ts a nd l asted a nywhere be tween 250 ns a nd 200 m s. The t ests i ncluded qui ck voltage recovery, slow voltage recovery, repeated voltage bursts, and load dump. The conducted transients t ests w ere no t pe rformed on the H oriba P EMS, because t he H oriba s ystem was intended for use with an external generator and does not use 12 volt power. The Schaffner Test System can be seen in Figure 120 with the Sensors PEMS and Figure 121 for the AVL PEMS.REPORT 03 .14936. 12 127 of 174 FIGURE 120 . SENSORS SETUP DURING CONDUCTED TRANSIE NTS TESTING

FIGURE 121 . AVL SETUP DURING CONDUCTED TRANSIENTS TESTING

The S ensors P EMS po wered dow n dur ing a

- 100 vol t s pike w ith qu ick r ecovery a nd again pow ered do wn w hen t he m agnitude of t he s pike w as reduced t o

- 50 vol ts a nd t hen -25 volts. The slow recovery voltage spikes also caused the PEMS to power down with the shortest dwell time of 40 ms causing the unit to shut down. Longer dwell times were not tested.

The quick recovery voltage spike caused the A VL inverter to power down, although it appeared t o be w orking pr operly w hen i t w as restarted. H owever, dur i ng t he s low r ecovery voltage spikes, the inverter shut down again, started smoking and stopped working. The test was not repeated with another inverter to prevent further damage.REPORT 03 .14936. 12 128 of 174 The f indings from t he EMI and R FI t esting were pr esented t o t he s teering c o mmittee during t he M ay 20 th , 2009 i n S an A ntonio. T he s teering c ommittee de clined t o pur sue f urther testing to develop any EMI and RFI error surfaces. The steering committee requested that Horiba investigate t he i ssue of bul k c urrent i njection noi se i n t he analog ex haust f low cabl e and that Sensors investigate the issue of exhaust flow errors during radiated immunity testing.

5.5 Vibration Testing

Figures 122, 123, 124, 1 25, 126, and 127, show the setup for Sensors vibrational testing using di fferent c onfigurations. The vi bration testing w as condu cted as a s creening ex ercise similar to the m ethod us ed f or t he E MI a nd R FI t esting. T he P EMS w ere ope rated on H EPA filtered room air while being subjected to vibration. The response of the PEMS to the screening exercise was used to determine whether further test ing was needed to generate an error surface for vibration. Each PEMS was mounted on an Unholtze

- Dickie Shaker Table which was capable of vibrating the PEMS on all three axes separately by adjusting the or ientation of the PEMS. The system used a large table to vibrate horizontally. By rotating the orientation of the PEMS, this table w as a ble t o s imulate l ongitudinal a nd t ransverse hor izontal vi bration. T he s haker w as rotated into a vertical position and a s maller platform was attached to provide vertical vibration. Due t o c onsiderations f or non

- road i n

- use te sting, the s teering c ommittee r equested that th e PEMS a lso be r otated a t a 45 de gree i ncline o n e ach axis a s w ell. T he hor izontal vi bration platform was large enough to include all of the pieces of any one type of PEMS at a t ime. The vertical platform and 45 degree stand were each capable of installing only a single piece of the instrumentation at a time

.

FIGURE 122. SENSORS PEMS MOUNTED FOR TRANSVERSE HORIZONTAL VIBRATIONREPORT 03 .14936. 12 129 of 174 FIGURE 123. SENSORS PEMS MOUNTED FOR LON GITUDINAL HORIZONTAL VIBRATION

FIGURE 124 . SENSORS PEMS MOUNTED FOR TRANSVERSE 45° VIBRATION 

# Direction of vibration



# Direction of vibrationREPORT 03 .14936. 12 130 of 174 FIGURE 125. SENSORS PEMS MOUNTED FOR LONGITUDINAL 45° VIBRAT ION

FIGURE 126. SENSORS PEMS MOUNTED FOR VERTICAL VIBRATION 

# Direction of vibration



# Direction of vibrationREPORT 03 .14936. 12 131 of 174 FIGURE 12 7 . SENSORS PEMS MOUNTED FOR 45° VERTICAL VIBRATION

The pow er s pectral de nsity ( PSD) f rom t he M il S tandard 810, U S H ighway T ruck Vibration Exposure was used in this testing. To prevent damage to the PEMS, the PSD was only operated at 25%, 50%, and 75% of the energy specified in the Mil Standard 810. A fter a few tests at the 75% level, the energy levels were reduced to 10%, 25%, and 50% to maintain the integrity of t he i nstruments. In a ddition e ach e nergy l evel w as onl y t ested f or 5 m inutes t o minimize the chances of damaging the instruments. Figure 128 show s the PSD used for vertical vibration testing and Figure 130 shows the PSD used for horizontal testing.

Figure 13 0 shows t he Horiba P EMS vi bration pos itions f or di fferent configurations. Because the OBS

- TRPM included three separate boxes, some of the vibration testing had to be performed on i ndividual boxes. The DCS, HE, and MEEE were all able to fit on t he horizontal vibration table however only a single box could fit at a time on the 45 d egree angle stand. The DCS and MEEE were tested on t his stand; the HE box was not tested at 45 degrees, because it was felt that no errors would be detected in a box housing only a filter holder and a cyclone. A particle source would need to be present to detect problems in this portion of the system. The vertical vibra tion platform was able to hold both the DCS and the MEEE, so these two boxes were tested simultaneously while the HE was again not tested. Any piece of the system that was not being tested on t he vibration stand was positioned directly next to the stand so that all flow and electrical cables could still connect to the system normally. 

# Direction of vibrationREPORT 03 .14936. 12 132 of 174 FIGURE 128 . POWER SPECTRAL DEN SITY FOR VERTICAL VIBRATION TESTING

FIGURE 129 . POWER SPECTRAL DEN SITY FOR HORIZONTAL VIBRATION TESTINGREPORT 03 .14936. 12 133 of 174 FIGURE 130 . HORIBA PEMS VIBRAT ION POSITIONS

The H oriba P EMS ex perienced a m echanical f ailure dur ing t esting at t he 75% energy level of t he t ransverse horizontal vi bration. A bracket hol ding a f ilter inside t he M echanical Enclosure (ME) box broke causing a flow line to become blocked. The test was stopped and the bracket w as r eplaced with one f rom a nother uni t. T he t esting was r esumed unt il i t w as discovered t hat one o f the pr essure t ransducers us ed t o m easure t he f low w as not working properly c ausing a n error i n t he r eported f low. This pr oblem w as b elieved t o ha ve originated from t he br oken br acket be cause t he l ine t hat w as c losed of f w as a ttached t o t his pr essure transducer. The pressure transducer w as replaced with one from the other unit and the s ystem functioned properly. All future tests were conducted at 10%, 25%, and 50% energy for all PEMS to prevent further damage. The only other damage incurred by the Horiba was a rubber foot on the bottom of one of the boxes was sheared off. The measured dilution flow exhibited a higher degre e of f luctuations d uring t he vi bration t esting t han w as nor mally observed although t he differences were not significant.

The S ensors P EMS w as ope rated w ithout t he exhaust f low m eter and sample e lbow attached to the unit. As can be seen in Figure 127, the two pieces were removed to allow a HEPA filter t o be pl aced on t he i nlet. A lthough i t w ould ha ve be en de sirable t o l eave t hese pi eces attached it was c onsidered more i mportant t o provide cl ean air t o ensure a z ero particle l evel. The Sensors PEMS experienced no mechanical failures during vibration testing although several functionality i ssues di d oc cur. T he exhaust f low m easurement be came n oisier w ith vi bration although the magnitude of the noise was relatively small. In the EMI and RFI testing values a s high as 1000 kg/hr were observed, but in vibration testing the exhaust flow never went above 50 kg/hr.REPORT 03 .14936. 12 134 of 174 The Sensors PEMS experienced problems maintaining its total flow rate during vibration in the vertical direction. This problem was related to the automatic drain valves on the moisture traps i n t he di lution f low l ine. T hese t raps a re designed t o ope n a utomatically w hen e nough downwards f orce i s a pplied f rom accumulated moisture i n t he reservoir; however t he ve rtical vibration w as c ausing t he va lves t o repeatedly o pen a llowing a por tion of t he f low t o e scape. Figure 131 shows an example of the total flow dropping while experiencing vertical vibration.

FIGURE 131 . SENSORS TOTAL FLOW DURING VIBRATION TESTING

It w as pos sible t o t est both of t he A VL boxes s imultaneously for t he horizontal a nd vertical vibration, however they were tested separately for the 45 degree angle vibration. Figure 132 shows the AVL PEMS vibration testing using different configurations.

No m echanical pr oblems w ere encountered w ith t he A VL uni t dur ing vi bration t esting although the measurement did exhibit vibration induced noise. The measured soot concentration fluctuated with increasing a mplitude a s the vi bration increased for a ll or ie ntations t ested. T he peak m easurement r ecorded w as about pl us a nd m inus 5 μ g/mol w ith m ore t ypical va lues swinging between plus and minus 2.5 μg/mol. A typical measurement is shown in Figure 133. 

### 0 2 4 6 8 10

200 400 600 800 1000 1200 1400 1600Total Flow (slpm)



### Time (sec) None 10% 25% 50% NoneREPORT 03 .14936. 12 135 of 174 FIGURE 132 . AVL PEMS VIBRATION POSITIONS

FIGURE 133. AVL SOOT MEASUREMENT NOISE DURING VIBRATI ON TESTING 

### - 0.20 - 0.15 - 0.10 - 0.05 0.00 0.05 0.10 0.15 0.20

0 200 400 600 800 1000 1200Reported Soot Concentration (mg/m3)



### Time (sec)

Concentration dil. corrected [mg/m3] 30 Sec Avg 75% 25% 50% NoneREPORT 03 .14936. 12 136 of 174 This particular data was from the 45 degree angle longitudinal vibration. The noise in the measurement has no bi as indicating that it will average out over larger periods of time. The 30 second average is s hown s ince t his i s l ikely t he m easurement error t hat w ould be obs erved during an NTE event. The maximum error of the 30 second average was

- 0.009 mg/m3 which is relatively i nsignificant b ased on P M c oncentrations e xpected a t t he t hreshold. H owever, i t is unclear from this testing whether the noise could be greater if a particle source was present. The results from the vibration testing were presented to the steering committee at the September 22 nd

, 2009 meeting in Riverside. The steering committee declined to perform additional tests to create an error surface for vibration since the AVL s ystem was excluded from being an official PM

PEMS.REPORT 03 .14936. 12 137 of 174 6.0 MODEL ING RESULTS

The main objective of this portion of the project was to use Monte Carlo techniques (e.g. random s ampling) in an e rror mode l to simulate the c ombined effects of a ll the a greed -upon sources of PEMS error incremental to lab error on the components of the brake

- specific (BS) PM emissions . T his was accomplished by creating “error surfaces” for the Monte Carlo simulation to sample, based upon t he results of a variety of lab experiments. T he constructed model was simulated for thousands of trials (i.e., iterations) using data taken from a reference data set of 141 unique N TE e vents. T he m odel r esults w ere u sed t o de termine t he br ake

- specific a dditive
measurement a llowances for PM by three different calculation methods for three different PEMS model units.

The error surfaces were generated from the results of each of the engine dynamometer and e nvironmental chamber lab oratory test s . T he engine

- lab

- test error s urfaces cove red the domain of error versus the magnitude of the signal to which the error was to be applied (i.e., 1 st to 99 th percentile er ror vs. concentra tion, f low, t orque, e tc.). T he environmental

- test e rror surfaces f or s hock and vi bration, a nd electromagnetic a nd radio f requenc y i nterference (EMI/RFI) was not included because no e rror surfaces were generated. The environmental test error surfaces for pressure and temperature were characteristically different because they covered the domain of the environmental

- test cycle time versus the magnitude of the signal to which the error was to be applied (i.e., error at a selected time vs. concentration).

6.1 Convergence Results from MC Runs

This section contains a summary of the checks to determine if the convergence criteria were met for the simulation runs. Section 2.9 on Convergence and Number of Trials contains a detailed description of the convergence methodology and the procedures followed to check for convergence for the reference NTE event trials. This procedure was app lied to the simulation data obtained for each of the three PEMS units and all applicable calculation methods.

Figure 134 through Figure 140 contain plots of the 90% confidence interval widths at the 95 th percentile delta differences (expressed as a p ercent of the BSPM emissions NTE threshold) versus the ideal BSPM emissions for the 141 individual reference NTE events. This is d one for each of the three PEMS units and the applicable calculation methods. A summary of the results is given in Table 22 . Of interest was whether or not the simulations converged within 1% of the threshold va lue. A s c an be s een i n t he pl ots, a majority of t he r eference N TE e vents di d not converge within 1% of the BSPM threshold. However, a majority did converge within 2% of the threshold. F or t he t hree P EMS uni ts a nd c alculation m ethods, t he m aximum pe rcent of t he confidence i nterval w idths r anged f rom 1.82% f or t he A VL M ethod 1 t o 2.76% f or S ensors Method 1. U pon e xamination of t he de lta BSPM di stributions f or t he va rious r eference N TE events, t hose t hat ha d a hi gh pe rcentage a bove t he B SPM t hreshold a t t he 95 th percentile generally had low input P M concentration levels.REPORT 03 .14936. 12 138 of 174 FIGURE 134 . CONVERGENCE FOR AVL METHOD 1 AS A PER CENT OF BSPM THRESHOLD

FIGURE 135 . CONVERGENCE FOR AVL METHOD 2 AS A PER CENT OF BSPM THRESHOLD 

### 0.00 0.40 0.80 1.20 1.60 2.00

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

## AVL Method 1

0.00 0.40 0.80 1.20 1.60 2.00 2.40

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

### AVL Method 2REPORT 03 .14936. 12 139 of 174 FIGURE 13 6 . CONVERGENCE FOR AVL METHOD 3 AS A PER CENT OF BSPM THRESHOLD

FIGURE 13 7 . CONVERGENCE FOR HORIBA METHOD 1 AS A PERCENT OF BSPM THRESHOLD 

### 0.00 0.40 0.80 1.20 1.60 2.00 2.40

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

### AVL Method 3

0.00 0.40 0.80 1.20 1.60 2.00 2.40 2.80

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

### Horiba Method 1REPORT 03 .14936. 12 140 of 174 FIGURE 1 38 . CONVERGENCE FOR HORIBA METHOD 2 AS A PERCENT OF BSPM THRESHOLD

FIGURE 1 39 . CONVERGENCE FOR SENSORS METHOD 1 AS A PERCENT OF BSPM THRESHOLD 

### 0.00 0.40 0.80 1.20 1.60 2.00 2.40 2.80

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

### Horiba Method 2

0.00 0.40 0.80 1.20 1.60 2.00 2.40 2.80 3.20

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

### Sensors Method 1REPORT 03 .14936. 12 141 of 174 FIGURE 140

. CONVERGENCE FOR SENS ORS METHOD 2 A S A PERCENT OF BSPM THRESHOLD

TABLE 22 . SUMMARY OF NUMBER OF REFERENCE NTES MEETING 2% CONVERGENCE

PEMS Unit Method Min Max No. N TE s with in 2% Convergence % NTEs within 2% Co n vergence

AVL 1 0.4467 1.8211 141 100% 2 0.4446 1.9766 141 100% 3 0.4240 1.9215 141 100%

Horiba 1 0.4158 2.5663 129 91% 2 0.4414 2.5447 131 93%

Sensors 1 0.7883 2.7664 125 87% 2 0.8613 2.6544 131 93% 

### 0.00 0.40 0.80 1.20 1.60 2.00 2.40 2.80

0 0.01 0.02 0.03 0.04 0.05 0.06% BSPM Threshold



### Ideal BSPM, g/hp

- hr 

### Sensors Method 2REPORT 03 .14936. 12 142 of 174 6.2 Sensitivity Based on Bias and Variance

This section contains a summary of the error surfaces that contributed the most to the bias of the generated BS emissions. The sensitivity charts developed in Crystal Ball help identify the error surfaces (assumptions) that are sensitive to changes in variation with respect to their effect

on the t hree de lta BS emissions. A nother t ype of s ensitivity e xamined i n t his s tudy w as concerned with the effects of potential “bias” in error surfaces and their effects on the forecast values. In or der t o s tudy these e ffects a ne w e rror s urface a ssumption was a dded t o t he M C Monte Carlo simulation model for each of the original 31 error surfaces.

This assumption was sampled as a discrete binary distribution (i.e., on or off) during the simulation. F or e ach tr ial of the s imulation, 31 or iginal e rror s urf aces and 31 ‘ on/off’ error surfaces w ere s ampled according t o t heir de fined s ample di stribution. If t he ‘on/off’ e rror surface produced an ‘off’ condition, the delta emissions from that particular error surface were not added to the BS emissions computati ons for the BS emissions ‘with errors’. Similarly, if the ‘on/off’ error surface produced an ‘on’ condition, the delta emissions from that particular error surface were added to the BS emissions calculations.

During every trial of the simulation, the exclusions due to the ‘off’ conditions resulted in various combinations of the error surface delta emissions being added to the BS emissions ‘with errors’ c omputations. Over t he c ourse of a MC s imulation w ith t housands of t rials, t he sensitivity of a part icular error either ‘on’ or ‘off’ was assessed by examining the change in the forecast de lta em ission. T herefore, in a s ingle M C s imulation of a reference N TE ev ent sensitivities due to variance and/or bias were explored.

Simulation results from the ref erence NTE events produced sensitivity values for all 95th percentile delta emissions by all three PEMS units and applicable calculation methods

. Table 2 3 through Table 2 9 summarize t he e rror s urfaces i n w hich e ither t he c ontribution

- to

- variance normalize d sensitivity value or the ‘on/off’ bias check for the error surface was at least 5% in magnitude compared to all the other error surfaces. If the label in the error surface contains the words ‘Delta’ then it represents a che ck for bias; otherwise, the error surface indicates a ch eck for variance

. Table 2 3 through Table 25 list s the sensitivity and bias descriptive statistics for the delta BSPM emissions for the AVL PEMS for Methods 1, 2 a nd 3, respectively. For all three methods, the largest mean norma lized variance was from the bias effect due to error surface #1, SSPM.REPORT 03 .14936. 12 143 of 174 TABLE 23 . ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR AVL BSPM METHOD 1

Error Surface

No. Error Surface No. Ref NTE Events Avg

. Contribution to Normalized Variance, % Min Contribution, % Max Contribution, % 1 SS PM 141 11.34 7.59 35.25 2 TR PM 4 7.00 6.42 7.34 20 SS Exhaust Flow 4 5.57 5.32 6.05 31 Torque Warm

- up 10

- 6.64

- 11.22

- 5.32 35 Torque Engine Manufacturer 1 -6.27 -6.27 -6.27

Delta Exhaust Flow Pulsation 3 5.78 5.56 5.99

Delta SS PM 141

- 74.37

- 83.67

- 21.02

TABLE 24 . ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR AVL BSPM METHOD 2

Error Surface

No. Error Surface No. Ref NTE Events Avg. Contribution to Normalized Variance, % Min Contribution, % Max Contribution, % 1 SS PM 141 21.48 7.44 60.56 2 TR PM 38 9.20 5.53 12.31 31 Torque Warm

up 8 -6.82 -10.77 -5.24

35 Torque Engine Manuf 1 -6.12 -6.12 -6.12

45 SS CO2 4

- 6.89

- 7.82

- 5.32 Delta SS PM 141

- 63.27

- 83.77

- 20.95

TABLE 25 . ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR AVL BSPM METHOD 3

Error Surface

No. Error Surface No. Ref NTE Events Avg. Contribution to Normalized Variance, % Min Contribution, % Max Contribution, % 1 SS PM 141 20.91 7.71 60.50 2 TR PM 36 9.27 5.17 12.52 31 Torque Warm

up 9 -6.62 -10.77 -5.14

35 Torque Engine Manuf 1 -6.12 -6.12 -6.12

45 SS CO2 4

- 6.96

- 7.89

- 5.40 Delta SS PM 141 -64.04 -83.76 -20.93REPORT 03 .14936. 12 144 of 174 Table 26 and Table 27 list the s ensitivity a nd bias de scriptive s tatistics f or the de lta BSPM emissions for the Horiba PEMS for Methods 1 and 2, respectively. For both methods, the highest mean normalized variances were from the bias and variance due to e rror surface #1, SS PM.

TABLE 26 . ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR HORIBA BSPM METH OD 1

Error Surface

No. Error Surface No. Ref NTE Events Avg. Contribution to Normalized Variance, % Min Contribution, % Max Contribution, % 1 SS PM 138 41.86 6.65 80.39 2 TR PM 10 5.84 5.04 7.18 20 SS Exhaust Flow 26 8.94 5.00 7.80

31 Torque Warm

up 100 -8.34 -15.96 -5.08

35 Torque Engine Manuf 29 -5.72 -8.07 -5.00

Delta Exhaust Flow Pulsation 31 6.67 5.07 10.01

Delta SS PM 83 -32.05 -83.42 73.96

TABLE 27 . ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR HORIBA BSPM METH OD 2

Error Surface

No. Error Surface No. Ref NTE Events Avg. Contribution to Normalized Variance, % Min Contribution, % Max Contribution, % 1 SS PM 138 39.55 6.99 80.30 2 TR PM 10 5.89 5.00 7.42 31 Torque Warm

up 95 -8.19 -15.04 -5.03

35 Torque Engine Manuf 24 -5.72 -7.65 -5.08

42 Fuel Rate Engine Manuf 7 5.52 5.07 6.34

45 SS CO2 54

- 6.60

- 9.83

- 5.09 Delta SS PM 89 -42.60 -83.42 75.48REPORT 03 .14936. 12 145 of 174 Table 28 and Table 29 list the s ensitivity a nd bias de scriptive s tatistics f or the de lta BSPM emissions for the Sensors PEMS for Methods 1 a nd 2, respectively. F or both methods, the highest mean normalized variance was from the bias and variance due to error surface #1, SSPM.

TAB LE 28 . ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR SENSORS BSPM MET HOD 1

Error Surface

No. Error Surface No. Ref NTE Events Avg. Contribution to Normalized Variance, % Min Contribution, % Max Contribution, % 1 SS PM 138 46.92 6.57 78.61 2 TR PM 100 8.48 5.00 12.31 20 SS Exhaust Flow 8 6.11 8.06 7.56 31 Torque Warm

- up 30

- 8.28

- 11.57

- 5.02 35 Torque Engine Manuf 10 -5.60 -6.06 -5.14

Delta Exhaust Flow Pulsation 9 7.01 5.56 8.79

Delta SS PM 120

- 31.19

- 86.58 75.35

TABLE 29 . ERROR SURFACE SENSITIVITY TO BIAS AND VARIANCE FOR 141 REFERENCE NTE EVENTS FOR SENSORS BSPM MET HOD 2

Error Surface

No. Error Surface No. Ref NTE Events Avg. Contribution to Normalized Variance, % Min Contribution, % Max Contribution, % 1 SS PM 138 45.92 6.68 77.78 2 TR PM 101 8.48 5.03 12.52 31 Torque Warm

- up 24

- 7.84

- 12.01

- 5.08 35 Torque Engine Manuf 5 -5.61 -6.06 -5.01

42 Fuel Rate Engine Manuf 4 6.18 5.72 6.53

45 SS CO2 10

- 6.84

- 9.77

- 5.15 Delta SS PM 127 -28.96 -86.67 75.75

The c ontribution t o normalized variance a nd bias s ensitivities f rom Table 23 through Table 29 are illustrated pictorially as box plots in Figure 141 to Figure 147 for BSPM by PEMS unit for Methods 1, 2 and 3. Only the error surfaces with at least 35 of the 141 r eference NTE events (1/4 of the events) are included as box plots. The mean normalized variance for each of the pl otted e rror s urfaces i s not ed b y a “ +” s ymbol i n t he box es. T he e rror s urface with t he largest mean normalized variance is plotted at the left of the chart. The error surface with the second largest mean normalized variance is plotted second from the left, and so on. Figure 142 and Figure 143 demonstrate the high sensitivity to the negative bias for error surface #1, PM SS. Figure Figure 146 and Figure 147 show a l arge variance ef fect due t o PM S S. Table 3 0 and Table 3 1 show a summary of the error surface sensitivity to bi as and variance for the different PEMS using Method 1 a nd Method 2. T able 3 2 shows a similar summary using Method 3 for the AVL PEMS only.REPORT 03 .14936. 12 146 of 174 FIGURE 141 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR AVL BSPM METHOD 1

FIGURE 142 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR AVL METHOD 2REPORT 03 .14936. 12 147 of 174 FIGURE 143 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR AVL METHOD 3

FIGURE 144 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR HORIBA BSPM METHOD 1REPORT 03 .14936. 12 148 of 174 FIGURE 145 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR HOR IBA BSPM METHOD 2

FIGURE 146 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR SENSORS BSPM MET HOD 1REPORT 03 .14936. 12 149 of 174 FI GURE 147 . BOX PLOT OF ERROR SURFACE SENSITIVITY BASED ON BIAS AND VARIANCE FOR SENSORS BSPM MET HOD 2

TABLE 30 SUMMARY OF ERROR SUR FACE SENSITIVITIES T O BIAS AND VARIANCE FOR BSPM ME THOD 1

Method 1

Error Surface

No. Error Surface AVL Horiba Sensors

No. NTE Events Avg Contribution to Normalized Variance, % No. NTE Events Avg Contribution to

Normalized Variance, % No. NTE Events Avg Contribution to

Normalized Variance, %

1 SS PM 141 11.34 138 41.86 138 46.92
2 TR PM 4 7.00 10 5.84 100 8.48

20 SS Exhaust Flow 4 5.57 26 8.94 8 6.11

31 Torque Warm

up 10

- 6.64 100

- 8.34 30

- 8.28

35 Torque Engine Manuf 1

- 6.27 29

- 5.72 10

- 5.60

Delta Exhaust Flow Pulsation 3 5.78 31 6.67 9 7.01
Delta SS PM 141

- 74.37 83

- 32.05 120

- 31.19REPORT 03 .14936. 12 150 of 174 TABLE 31 . SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND VARIANCE FOR BSPM ME THOD 2

Method 2

Error Surface

No. Error Surface AVL Horiba Sensors

No. NTE Events Avg Contribution to Normalized Variance, % No. NTE Events Avg Contribution to

Normalized Variance, % No. NTE Events Avg Contribution to

Normalized Variance, %

1 SS PM 141 21.48 138 39.55 138 45.92
2 TR PM 38 9.20 10 5.89 101 8.48

31 Torque Warm

up 8

- 6.82 95

- 8.19 24

- 7.84

35 Torque Engine Manuf 1

- 6.12 24

- 5.72 5

- 5.61

42 Fuel Rate Engine Manuf 7 5.52 4 6.18
45 SS CO2 4

- 6.89 54

- 6.60 10

- 6.84
Delta SS PM 141

- 63.27 89

- 42.60 127

- 28.96

TABLE 32 . SUMMARY OF ERROR SURFACE SENSITIVE TO BIAS AND VARIANCE FOR BSPM ME THOD 3

Method 3

Error Surface

No. Error Surface AVL

No. NTE

Events Avg Contribution to Normalized Variance, % 1 SS PM 141 20.91 2 TR PM 36 9.27 31 Torque Warm

- up 9

- 6.62

35 Torque Engine Manuf 1 -6.12 45 SS CO2 4

- 6.96 Delta SS PM 141

- 64.04

6.3 Validation Results

This section contains a summary of the model validation results; Section 0 on Validation contains a m ore d etailed de scription of t he validation m ethodology ut ilized bot h i n t he simulation and in the on- road data collection efforts.

During the Monte Carlo simulation of the 141 r eference NTE events some of the error surfaces w ere e xcluded i n t he c omputation of t he B S e missions ‘ with e rrors’ s o t hat t h e simulation r epresented c onditions us ed i n c ollecting t he on

- road data. T he er ror s urfaces excluded were torque errors (Nos. 29

- 32, 34, 35), fuel rate engine manufacturers (#42), dynamicREPORT 03 .14936. 12 151 of 174 speed (#43) and dynamic fuel rate (#44). For each reference NTE ev ent, the difference in BSPM emissions was computed as

delta BSPM = BSPM with “Validation error” – “Ideal” BSPM.

These delta BSPM em issions were com puted for each of t he t hree P EMS uni ts and all applicable calculation methods. T he 5 th , 50 th and 95 th percentiles w ere id entified from the distributions of t he de lta B SPM e missions dur ing t he M onte C arlo simulation us ing t he validation error surfaces only. Figure 148 t hrough Figure 150 depict the validation percentiles for the AVL PEMS unit for methods 1, 2 and 3, respectively. Similar validation plots for the Horiba PEMS unit are illustrated in Figure 151 and Figure 152 for methods 1 and 2, respectively. Sensors PEMS validation plots for methods 1 and 2 are shown in Figure 153 through Figure 154.

FIGURE 148 . VALIDATION PERCEN TILES FOR THE 141 RE FERENCE NTE EVENTS FOR AVL METHO D 1 - 0.020 - 0.015 - 0.010 - 0.005 0.000 0.005

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation AVL 1 5th Validation AVL 1 50th Validation AVL 1 95thREPORT 03 .14936. 12 152 of 174 FIGURE 149 . VALIDATION PERCEN TILES FOR THE 141 RE FERENCE NTE EVENTS FOR AVL METHO D 2

FIGURE 150. VALIDATION PERCENTILES FOR 141 REFERENCE NTE EVENTS FOR AVL MET HOD 3 - 0.020 - 0.015 - 0.010 - 0.005 0.000 0.005

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation AVL 2 5th Validation AVL 2 50th Validation AVL 2 95th

- 0.020 - 0.015 - 0.010 - 0.005 0.000 0.005

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation AVL 3 5th Validation AVL 3 50th Validation AVL 3 95thREPORT 03 .14936. 12 153 of 174 FIGURE 151 . VALIDATION PERCEN TILES FOR 141 REFERENCE NTE EVENTS FOR HORIBA METHOD 1

FIGURE 152 . VALIDATION PERCEN TILES FOR 141 REFERENCE NTE EVENTS FOR HORIBA METHOD 2 - 0.015 - 0.010 - 0.005 0.000 0.005 0.010 0.015 0.020 0.025

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation Horiba 1 5th Validation Horiba 1 50th Validation Horiba 1 95th

- 0.015 - 0.010 - 0.005 0.000 0.005 0.010 0.015 0.020

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation Horiba 2 5th Validation Horiba 2 50th Validation Horiba 2 95thREPORT 03 .14936. 12 154 of 174 FIGURE 153 . VALIDATION PERCEN TILES FOR 141 REFERENCE NTE EVENTS FOR SENSORS METHOD 1

FIGURE 154 . VALIDATION PERCEN TILES FOR 141 REFERENCE NTE EVENTS FOR SENSORS METHOD 2 - 0.035 - 0.030 - 0.025 - 0.020 - 0.015 - 0.010 - 0.005 0.000 0.005 0.010 0.015

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation Sensors 1 5th Validation Sensors 1 50th Validation Sensors 1 95th

- 0.035 - 0.030 - 0.025 - 0.020 - 0.015 - 0.010 - 0.005 0.000 0.005 0.010 0.015

0.00 0.01 0.02 0.03 0.04 0.05 0.06Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr

Validation Sensors 2 5th Validation Sensors 2 50th Validation Sensors 2 95thREPORT 03 .14936. 12 155 of 174 The 5 th and 95 th percentiles of the validation delta BSPM were separately fit for the AVL and S ensors P EMS uni ts us ing a s imple l inear regression m odel. H owever, t he c riteria for accepting the linear fits were not met for any of the calculations methods for these two PEMS units. T hus, loess regression fits were used to determine the best functional representation for the 5 th and 95 th delta BSP M based on the validation simulation modeling. These loess fits for the 95 th and 5 th percentiles for the Sensors unit methods 1 and 2 can be found in Figure 155 through Figure 158

, respectively. T he l oess f its f or t he 95 th and 5 th percentiles f or t he A VL uni ts methods 1, 2 a nd 3 c an be f ound i n Figure 159 t hrough F igure 164 , respectively. T he l oess smoothing parameters for the regression fits are listed in Table 33

.

TABLE 33 . LOESS SMOOTHING P ARAMETERS FOR VALIDA TION PERCENTILES

PEMS Method 5th Percentile 95th Percentile

Sensors 1 0.290 0.570 2 0.290 0.570

AVL 1 0.294 0.755 2 0.294 0.777 3 0.294 0.777

FIGURE 155 . VALIDATION 95TH P ERCENTILE BSPM DELTA S LOESS FIT FOR SENSORS METHOD 1REPORT 03 .14936. 12 156 of 174 FIGURE 156 . VALIDATION 5TH PERCENTILE BSPM DELTAS LOESS FIT FOR SENSORS METHOD 1

FIGURE 157. VALIDATION 95TH PERC ENTILE BSPM DELTAS L OESS FIT FOR SENSORS METHOD 2REPORT 03 .14936. 12 157 of 174 FIGURE 158 . VALIDATION 5TH PE RCENTILE BSPM DELTAS LOESS FIT FOR SENSORS METHOD 2

FIGURE 159 . VALIDATION 95TH PERCENTILE BSPM DELTAS LOESS FIT FOR AVL METHOD 1REPORT 03 .14936. 12 158 of 174 FIGURE 160 . VALIDATION 5TH PE RCENTILE BSPM DELTAS LOESS FIT FOR AVL METHOD 1

FIGURE 161 . VALIDATION 95TH P ERCENTILE BSPM DELTA S LOESS FIT FOR AVL METHOD 2REPORT 03 .14936. 12 159 of 174 FIGURE 162 . VALIDATION 5TH PERCENTILE BSPM DELTAS LOESS FIT FOR AVL METHOD 2

FIGURE 163 . VALIDATION 95TH P ERCENTILE BSPM DELTA S LOESS FIT FOR AVL METHOD 3REPORT 03 .14936. 12 160 of 174 FIGURE 164. VALIDATION 5TH PERCE NTILE BSPM DELTAS LO ESS FIT FOR AVL METHOD 3

6.4 Measurement Error Allowance Result s

This section contains a summary of the measurement error allowance results using both a regression method and a median method to determine the measurement allowance

. Section 2.12

on Measurement Allowance contains a de tailed de scription of t he m ethodology f ollowed i n determining these values. This procedure was applied to the simulation data for all 141 reference NTE events obtained for all three calculation methods for the AVL PEMS and for calculations methods 1 and 2 for the Horiba and Sensors PEMS.

Figure 65 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for AVL Method 1. Included in the plot is the equation for the fitted regression line, and the R

- square (R 2 ) value and root mean square error (RMSE) va lue f or t he r egression f it. T he t wo symbols i n t he pl ot r epres ent r eference N TE events where there was a dominant bias effect due to the SSPM error surface (diamond symbol) or there was a dominant variance effect due to the SSPM error surface (square symbol). The R

square va lue i ndicates t hat 47.95% of t he va riation in t he 95 th percentile B SPM va lues i s explained by the ideal BSPM values for the AVL Method 1 da ta. T he RMSE value of 0.0008 displays t he s ize of t he e stimated s tandard de viation of t he pr edicted 95 th percentile B SPM values.REPORT 03 .14936. 12 161 of 174 Table 34 includes a comparison of the results of the regression method based on Figure 165 and the median method as described in the Section 0 on Measuremen t Allowance . U nder the heading of “Regression Method” in the table, it is shown that only the R

- square criterion was not m et b y t he da ta. T hus, t he M edian M ethod m ust be us ed. Under t he he ading “ Median Method” in the table, the measurement error at the B SPM threshold, based on using the median of t he 141 95 th percentile de lta BSPM va lues, i s 0.661% w hen e xpressed a s a p ercent of t he threshold of 0.02 g/hp-hr.

FIGURE 165 . REGRESSION PLOT O F 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR AVL M ETHOD 1

TABLE 34. MEASUREMENT ERROR AT THRESHOLD FOR BSPM U SING REGRESSION AND MEDIAN METHODS FOR AVL ME THOD 1

Regression Method Median Method

R2 0.4795 Did Not Meet Criteria RMSE (SEE) 0.0008 Met Criteria 5% Median Ideal 0.0191007 Predicted 95th % Delta at Threshold 0.0003399 Median 95th % Delta 0.0001322 Measurement Error @ Threshold = 0.02 1.6993% Measurement Error @ Threshold = 0.02 0.661% 

### y = 0.116x

- 0.002 R2 = 0.4795 RMSE = 0.0008

- 0.0020 - 0.0010 0.0000 0.0010 0.0020 0.0030 0.0040 0.0050

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM g/hp-hr



### Ideal BSPM, g/hp

- hr 

## AVL Method 1

Delta SSPM ic SSPMREPORT 03 .14936. 12 162 of 174 Figure 166 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 r eference NTE events for AVL Method 2. T he R

- square value indicates t hat 46.02% of t he va riation i n t he 95 th percentile B SPM va lues i s e xplained by t he Ideal BSPM values for the AVL Me thod 2 data. The RMSE value is 0.0008.

Table 35 includes a comparison of the results of the regression method based on Figure 166 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- square c riterion for using t his m ethod i s not m et b y t he da ta. T hus, t he M edian Method must be used. Under the heading “Median Method” in the table, the measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSPM values, is -2.375% when expressed as a percent of the threshold value of 0.02.

FIGURE 166 . REGRESSION PLOT O F 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR AVL M ETHOD 2

TABLE 35 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIAN METHODS FOR AVL ME THOD 2

Regression Method Median Method

R 0.4602

2 Did Not Meet Criteria RMSE (SEE) 0.0008 Met Criteria 5% Median Ideal 0.0191007 Predicted 95 th -0.0002124 % Delta at Threshold Median 95 th -0.0004751 % Delta Measurement E rror @ Threshold = 0.02 -1.0618% Measurement E rror @ Threshold = 0.02 -2.375%



### y = 0.0996x

- 0.0022 R2 = 0.4602 RMSE = 0.0008

- 0.0020 - 0.0010 0.0000 0.0010 0.0020 0.0030 0.0040

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr



### Ideal BSPM, g/hp

- hr 

### AVL Method 2

Delta SSPM ic SSPMREPORT 03 .14936. 12 163 of 174 Figure 167 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 r eference NTE events for AVL Method 3. T he R

- square value indicates t hat 46.20% of t he va riation i n t he 95 th percentile B SPM va lues i s e xplained by t he Ideal BSPM values for the AVL Me thod 3 data. The RMSE value is 0.0008.

Table 36 i ncludes a comparison of the results of the regression method based on Figure 167 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- square c riterion for u sing t his m ethod i s not m et b y t he da ta. T hus, t he M edian Method must be used. Under the heading “Median Method” in the table, the measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSPM values, is -2.383% when expressed as a percent of the threshold value of 0.02.

FIGURE 167 . REGRESSION PLOT O F 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR AVL M ETHOD 3

TABLE 36 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIAN METHODS FOR AVL METHOD 3

Regression Method Median Method

R 0.4620

2 Did Not Meet Criteria RMSE (SEE) 0.0008 Met Criteria 5% Median Ideal 0.0191007 Predicted 95 th -0.0002118 % Delta at Threshold Median 95 th -0.0004766 % Delta Measurement E rror @ Threshold = 0.02 -1.0592% Measurement E rror @ Threshold = 0.02 -2.383% 

### y = 0.1001x

- 0.0022 R2 = 0.462 RMSE = 0.0008

- 0.0020 - 0.0010 0.0000 0.0010 0.0020 0.0030 0.0040

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr



### Ideal BSPM, g/hp

- hr 

### AVL Method 3

Delta SSPM ic SSPMREPORT 03 .14936. 12 164 of 174 Figure 168 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for Horiba Method 1. The R

- square value indicates t hat 26.57% of t he va riation i n t he 95 th percentile B SPM va lues i s e xplained by t he Ideal BSPM values for the Hor iba Method 1 data. The RMSE value is 0.0041.

Table 37 incl udes a comparison of the results of the regression method based on Figure 168 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- square and the RMSE criteria for using this method were not met by the data. Thus, the M edian M ethod m ust be us ed. U nder t he heading “ Median M ethod” i n t he t able, t he measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSPM values, is 54.379 % when expressed as a percent of the threshold value of 0.02.

FIGURE 168 . REGRESSION PLOT O F 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR HORIBA METHOD 1

TABLE 37 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIAN METHODS FOR H ORIBA METHOD 1

Regression Method Median Method

R 0.2657

2 Did Not Meet Criteria RMSE (SEE) 0.0041 Did Not Meet Criteria 5% Median Ideal 0.0191007 Predicted 95 th 0.0102566 % Delta at Threshold Median 95 th 0.0108759 % Delta Measurement E rror @ Threshold = 0.02 51.2831% Measurement E rror @ Threshold = 0.02 54.379%

y =

- 0.3555x + 0.0174 R2 = 0.2657 RMSE = 0.0041

- 0.0050 0.0000 0.0050 0.0100 0.0150 0.0200 0.0250

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr 

### Horiba Method 1

Delta SSPM ic SSPMREPORT 03 .14936. 12 165 of 174 Figure 169 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for Horiba Method 2. The R

- square value indicates that 25.97 % of the variation in the 95 th percentile BSPM values is explained by the Ideal BSPM values for the Ho riba Method 2 data. The RMSE value is 0.0041.

Table 38 includes a comparison of the results of the regression method based on Figure 169 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- sq uare and the RMSE criteria for using this method were not met by the data. Thus, the M edian M ethod m ust be us ed. U nder t he heading “ Median M ethod” i n t he t able, t he measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta BSP M values, is 50.079 % when expressed as a percent of the threshold value of 0.02.

FIGURE 169 . REGRESSION PLOT O F 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR HORIBA METHOD 2

TABLE 38 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIAN METHODS FOR HOR IBA METHOD 2

Regression Method Median Method

R 0.2597

2 Did Not Meet Criteria RMSE (SEE) 0.0041 Did Not Meet Criteria 5% Median Ideal 0.0191007 Predicted 95 th 0.0094282 % Delta at Threshold Median 95 th 0.0100158 % Delta Measurement E rror @ Threshold = 0.02 47.1408% Measurement E rror @ Threshold = 0.02 50.079%

y =

- 0.3515x + 0.0165 R2 = 0.2597 RMSE = 0.0041

- 0.0050 0.0000 0.0050 0.0100 0.0150 0.0200

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr 

### Horiba Method 2

Delta SSPM ic SSPMREPORT 03 .14936. 12 166 of 174 Figure 170 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for Sensors Method 1. The R

- square value indicates that 45.49 % of the variation in the 95 th percentile BSPM values is explained by the Ideal BSPM values for the S ensors Method 1 data. The RMSE value is 0.0029.

Table 39 i ncludes a comparison of the results of the regression method based on Figure 170 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- square an d the RMSE criteria for using this method were not met by the data. Thus, the M edian M ethod m ust be us ed. U nder t he heading “ Median M ethod” i n t he t able, t he measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta B SPM values, is 34.361 % when expressed as a percent of the threshold value of 0.02.

FIGURE 170 . REGRESSION PLOT O F 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR SENSORS METHOD 1

TABLE 39 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIAN METHODS FOR SENSORS METHOD 1

Regression Method Median Method

R2 0.4549 Did Not Meet Criteria RMSE (SEE) 0.0029 Did Not Meet Criteria 5% Median Ideal 0.0191007 Predicted 95th % Delta at Threshold 0.0066785 Median 95th % Delta 0.00687227
Measurement Error @ Threshold = 0.02 33.3924% Measurement Error @ Threshold = 0.02 34.361% y =

- 0.3866x + 0.0144 R2 = 0.4549 RMSE = 0.0029

- 0.0150 - 0.0100 - 0.0050 0.0000 0.0050 0.0100 0.0150

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr

Ideal BSPM, g/hp

- hr 

### Sensors Method 1

Delta SSPM ic SSPMREPORT 03 .14936. 12 167 of 174 Figure 171 contains a regression plot of the 95 th percentile delta BSPM values versus the Ideal BSPM values for the 141 reference NTE events for Sensors Method 2. The R

- square value indicates that 45.03 % of the variation in the 95 th percentile BSPM values is explained by the Ideal BSPM values for the S ensors Method 1 data. The RMSE value is 0.0030.

Table 40 includes a comparison of the results of the regression method based on Figure 171 and the median method. Under the heading of “Regression Method” in the table, it is shown that the R

- square and the RMSE criteria for using this method were not met by the data. Thus, the M edian M ethod m ust be us ed. U nder t he heading “ Median M ethod” i n t he t able, t he measurement error at the BSPM threshold, based on using the median of the 141 95 th percentile delta B SPM values, is 30.285 % when expressed as a percent of the threshold value of 0.02.

FIGURE 171 . REGRESSION PLOT O F 95 TH PERCENTILE DELTA BSP M VERSUS IDEAL BSPM FOR SENSORS METHOD 2

TABLE 40 . MEASUREMENT ERROR AT THRESHOLD FOR BSP M USING REGRESSION AND MEDIAN METHODS FOR SENSORS METHOD 2

Regression Method Median Method

R 0.4503

2 Did Not Meet Criteria RMSE (SEE) 0.0030 Did Not Meet Criteria 5% Median Ideal 0.0191007 Predicted 95 th 0.0059333 % Delta at Threshold Median 95 th 0.0060569 % Delta Measurement E rror @ Threshold = 0.02 29.6663% Measurement E rror @ Threshold = 0.02 30.285% 

### y =

- 0.3875x + 0.0137 R2 = 0.4503 RMSE = 0.0030

- 0.0150 - 0.0100 - 0.0050 0.0000 0.0050 0.0100 0.0150

0 0.01 0.02 0.03 0.04 0.05 0.0695th % Delta BSPM, g/hp-hr



### Ideal BSPM, g/hp

- hr 

### Sensors Method 2

Delta SSPM ic SSPMREPORT 03 .14936. 12 168 of 174 Table 41 contains a s ummary of t he m easurement e rror va lues c ontained i n Table 3 4 through Table 40. The values are categorized by PEMS unit and by calculation method.

TABLE 41 . BSPM MEASUREMENT ERROR IN PERCENT OF NTE THRESHOLD BY PEMS AND CALCULATION METHOD

Measurement Errors (%) at Respective NTE Threshold

PEMS Method 1 Exhaust Flow Torque

- Speed Method 2 Exhaust and Fuel Flow Torque- Speed Method 3 Fuel Flow Torque

- Speed AVL 0.661

- 2.375

- 2.383 Horiba 54.379 50.079 n/a Sensors 34.361 30.285 n/a

Table 42 includes i n the m easurement al lowance s elected based on the m inimum normalized PM. The AVL w as not used in the measurement allowance determination because the AVL at the start of the program was not accepted as an official PEMS, and the measurement Steerin g Committee had decided that the measurement allowance would only be based on t he Sensors or the Horiba PEMS

.

TABLE 42 . MEASUREMENT ALLOW ANCE AT NTE THRESHOLD BY EMISSIONS FOR METHOD 2

PEMS Method 2 Measurement Error % NTE Threshold

g/hp

- hr Measurement Allowance,

g/hp

- hr Sensors 30.285 0.02 0.00605

On

- road P M e missions were gathered f rom s elected routes dr iven to collect e missions data with a CE

- CERT trailer and a PEMS installed on the tractor pulling the trailing. For each on-road NTE event, a delta BSPM emissions value was computed as follows:

Delta BSPM = PEMS BSPM – CE

- CERT BSPM.

These differences were computed for the BSPM emissions for each PEMS unit tested in

use. The in

- use BSPM was computed using Method 1 and 2 for the Sensors PPMD and Method 1, 2, a nd 3 f or A VL M SS. The i n

- use de lta B SPM em issions cal culated for t he AVL PE MS using methods 1, 2, a nd 3. CE

- CERT validation data were produced without any diesel particle filter ( DPF) active regeneration (referred to as “no regen”). (refer red to a s ” )informational purposes active AVL methods 1, 2 and 3 including three individual units (#2, #3 and #4) and 271 NTE e vents. T his da ta s et w as computed a s “no r egen”. T he s econd P EMS uni t t ested f or validation w as t he S ensors. T he on

- road delta BSPM em issions w ere calculated f or S ensors methods 1 and 2 also using three individual units in the “no regen” scenario and resulted in 217 NTE events.REPORT 03 .14936. 12 169 of 174 The va lidation pl ots f or t he S ensors a nd t he A VL s ystems a re s hown in Figures 172 through 176. The y

- axis scale on each figure was intended to make the best representation of the data relative to the validation lines shown on each plot. Because Method 1 depends strongly on exhaust flow, and some concerns were raised about the accuracy of exhaust flow measurement during CE

- CERT testing, results reported using Method 1 m ight not be accurate. Details about exhaust flow measurement are expected to be part of CE

- CERT Final Report on PM

- PEMS In

Use Validation Testing.

The loess regression fits in Figures 176 t hrough 176 and the Sensors CE

- CERT BSPM differences col lected on

- road were pl otted i n order t o determine i f t he s imulation m ethod validated. If t he num ber of C E

- CERT de lta BSPM va lues doe s not e xceed 10% o f t he t otal number of on

- road NTE events collected, then the simulation method would be considered valid. Figure 172 represents the validation plot for the BSPM method 1 analysis for Sensors. Note that 7 of the 217 on

- road NTE events were either below or above the range of the ideal BSPM a nd were excluded from the validation percentage calculation. T herefore, 68 of the 210 C E

- CERT NTE e vents (32.38%) fell be low t he s imulation m odel 5 th percentile ba sed on the l oess regression. Thus, the model was not considered valid for the BSPM Sensors Method 1.

Figure 173 represents the validation plot for the BSPM method 2 a nalysis for Sensors. Note that again 7 of the 217 on

- road NTE events were either below or above the range of the ideal BSPM and were excluded from the validation percentage calc ulation. Therefore, 71 of the 210 CE

- CERT NTE events (33.81%) fell above the simulation model 95 th percentile or below the s imulation model 5 th percentile ba sed on t he l oess r egressions. T hus, t he m odel w as not considered valid for the BSPM Sensors Method 2.

Figure 174 represents the validation plot for the BSPM method 1 analysis for AVL with ‘no regen’. In this case, 8 of the 271 on

- road NTE events were either below or above the range of the ideal BSPM and were excluded from the validation percentage calculation. Therefore, 49 of the 263 C E

- CERT NTE events (18.63%) fell above the simulation model 95 th percentile or below the simulation model 5 th percentile based on t he loess regressions. T hus, the model was not considered valid for the BSPM AVL Meth od 1.

Figure 175 represents the validation plot for the BSPM method 2 analysis for AVL with ‘no regen’. Again in this case 8 of the 271 on

- road NTE events were either below or above the range o f t he i deal B SPM and were ex cluded f rom t he v alidation perc entage calculation. Therefore, 30 of the 263 CE

- CERT NTE events (11.41%) fell above the simulation model 95 th percentile or below the simulation model 5 th percentile based on the loess regressions. Thus, the model was not considered valid for the BSPM AVL Method 2.

Figure 176 represents the validation plot for the BSPM method 3 analysis for AVL with ‘no regen’. Again in this case 8 of the 271 on

- road NTE events were either below or above the range o f t he i deal B SPM and were ex cluded f rom t he v alidatio n percentage calculation. Therefore, 26 of the 263 CE

- CERT NTE events (9.89%) fell above the simulation model 95 th percentile or below the simulation model 5 th percentile based on the loess regressions. Thus, the model was considered valid for the BSPM AVL Method 3 since the number of CE

- CERT NTE events outside the 5 th and 95 th percentile loess regression was less than 10%.REPORT 03 .14936. 12 170 of 174 FIGURE 172 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON

THE SIMULATION MODEL FOR BSPM SENSORS MET HOD 1 WITH NO REGEN

FIGURE 173 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON

THE SIMULATION MODEL FOR BSPM SENSORS MET HOD 2 WITH NO REGENREPORT 03 .14936. 12 171 of 174 FIGURE 174 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON THE SIMULATION MODEL FOR BSPM AVL METHOD 1 WITH NO REGEN

FIGURE 175 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON THE SIMULATION MODEL FOR BSPM AVL METHOD 2 WITH NO REGENREPORT 03 .14936. 12 172 of 174 FIGURE 176 . VALIDATION ON

- ROAD AND REGRESSION FUNCTIONS BASED ON THE SIMULATION MODEL FOR BSPM AVL METHOD 3 WITH NO REGEN

Table 43 summarizes the model validation results. Only the AVL Method 3 pa ssed the model validation.

TABLE 43 . SUMMARY OF BSPM MODEL VALIDATION RESU LTS

PEMS Unit Method 1 Exhaust Flow Torque

- Speed Method 2 Exhaust and Fuel Flow Torque

- Speed Method 3 Fuel Flow Torque

- Speed Sensors “no regen” No No No AVL “no regen” No No YesREPORT 03 .14936. 12 173 of 174 7.0 SUMMARY

A s eries of en gine ex periments and environmental t ests w ere p erformed on three P M

PEMS that included the Sensors PPMD, Horiba TRPM, and AVL MSS. The Sensors PPMD and Horiba TRPM were treated as official PEMS, and only those were used for official results and determination of next steps. The AVL MSS were used in conjunction with the Sensors PPMD based on an agreement reached between Sensors and AVL. Error surfa ces were developed based on e xperimental w ork t o de termine P M

- PEMS bi as a nd pr ecision e rrors us ing M onte C arlo simulation model. The o utput of the mod el w as to determine th e e rror d istribution at a s et of reference N TE ev ents, compared t o their i deal va lue. The P M

- PEMS w ith t he l owest 95 th percentile error that is greater than zero was selected for in

- use validation testing of the model.

The P M

- PEMS t hat w as s elected for i n

- use va lidation w as t he S ensors P PMD. T he PPMD produced a 95 th percentile measureme nt allowance error of 0.006 g/hp

- hr at a threshold NTE limit of 0.02 g /hp

- hr using Method 2, c ompared to the 0.01 g /hp

- hr that was produced by the Horiba TRPM. As for the AVL MSS, the instrument produced zero measurement allowance, but its measurement all owance value was not officially used because the MSS only measures the carbon fraction of PM, compared to the required total (solid plus volatile) PM measured by the other two PM

- PEMS. However, because the AVL MSS was used in conjunction with the PPMD, the SC a greed t o i nclude i t dur ing i n

- use va lidation testing. D ue to funding limita tion, the Horiba TRPM was not included in in- use validation testing.

Based on i n

- use v alidation t esting, t he S ensors P PMD f ailed va lidation be cause 3 2 percent and 34 percent of the data produced in

- use were below the 5 th percentile of the validation window, using Method 1 and Method 2, respectively. The SC agreed during the development of the Test Plan that less or equal 10 percent ( ≤ 10 %) of the data are allowed to be outside the validation window to pass validation.

As for the AVL MSS, it passed validation using Method 3 by having 9.89 percent of the data outside the v alidation window, with the majority of these data being hi gher than the 9 5 th percentile. Method 2 failed by two percentage points and Method 1 failed by 8 percentage points.

Because the MSS using Method 3 passed validation and funding run out to do any further work with the Sensors PPMD and/or to perform in

- use validation testing with the Horiba TRPM

, the SC concluded the measurement allowance program. The SC also accepted the measurement allowance ba sed on t he S ensors P PMD a s t he f inal P M

- PEMS m easurement al lowance for Methods 1, 2, and 3REPORT 03 .14936. 12 174 of 174 8.0 REFERENCES

1. “Electronic C ode of Federal R egulations, T itle 4 0 P rotection of t he E nvironment, V ol. 15 Part 1065,” http://www.ecfr.gpoaccess.gov , 2009.

2. “Determination of PEMS Measurement Allowance for Gaseous Emissions Regula ted Under the H eavy

- Duty I n

- Use T esting P rogram: R evised Final R eport,” EPA420 -R-08-005, February 2008.

3. Conover, W.J., Practical Nonparametric Statistics , John Wiley & Sons, 1971

4. Cleveland, W .S., “ Robust Locally

- Weighted Regression a nd S moothing S catterp lots

, ” Journal of the American Statistical Association, 1979, 74, pp. 829-836.

5. Hurviuch, C .M., S imonoff, J .S., a nd T sai, C .L., “Smoothing P arameters S election i n Nonparametric R egression U sing a nd Improved A kaike Information C riterion,” J ournal of the Roy al Statistical Society B, 1998, 60, pp.271-293.

6. Stratmann, F., Otto, E., and Fissan, H., “Thermophoretical and Diffusional Particle Transport in a Cooled Laminar Tube Flow.” Journal of Aerosol Science, March 28th, 1994, vol. 25; pp. 1359-1365.

7. Liu, B .Y.H and D .Y.H. P ui, “ Equilibrium B ipolar C harge D istribution of A erosols,” J . Colloid Interface Science, Vol. 49, 1974.RE PORT 03 .14936. 12 

### APPENDIX A

TEST PLAN TO DETERMINE PEMS MEASUREMENT ALLOWANCE FOR THE PM EMISSIONS REGULATED UNDER THE MANUFACTURER

- RUN HEAVY

- DUTY DIESEL ENGINE IN

- USE TESTING PROGRAM

DEVELOPED BY:

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY, CALIFORNIA AIR RESOURCES BOARD, AND

ENGINE MANUFACTURERS ASSOCIATION

NOVEMBER 11, 2008REPORT 03 .14936. 12 A-1 EXECUTIVE SUMMARY

This test plan sets forth the agreed upon pr ocesses and methodologies to be utilized to develop additive, br ake

- specific, da ta

- driven measurement allowance for PM emissions me as ured b y PEMS as required under the HDIUT regu latory program.

As detailed in this test plan, there is a clear consensus on what components of measurement error are i ntended to be covered by t he m easurement allowance . N amely, t he allowance is to b e calculated in a manner that subt racts lab error from PEMS error. Specifically, utilizing Part 1065 compliant emissions measurement systems and procedures for both the lab and PEMS, the lab error associated with measuring heavy- duty engine emissions at stabilized steady

- state test points within the NTE zone, will be subtracted from the PEMS error associated with measuring heavy

duty e ngin e e missions utilizing P EMS ove r events unde r a br oad r ange of environmental conditions. This subtraction will yield “PEMS minus laboratory” measurement allowance . The experimental methods and procedures specified in this test plan for determining, modeling, and comparing e ach of t he various c omponents of measurement error a re designed t o generate statistically robust data

- driven measurement allowance for the PM emi ssions.

Successful completion of this test plan is part of the resolution of a 2001 suit filed against EPA by E MA and a num ber of i ndividual e ngine m anufacturers. T he s uit c hallenged, a mong ot her things, c ertain s upplemental e mission r equirements r eferred t o as “ not

- to

- exceed” ( NTE) standards. On June 3, 2 003, the parties finalized a settlement of their disputes pertaining to the NTE s tandards. T he pa rties a greed upon a d etailed out line f or a f uture regulation t hat w ould require a manufacture r- run heavy

- duty in

- use NTE testing (“HDIUT”) program for diesel

- fueled engines and vehicles. One section of the outline stated:

“The N TE T hreshold w ill be t he N TE s tandard, i ncluding t he m argins bui lt i nto t he e xisting regulations, plus additional margin to account for in

- use measurement accuracy. This additional margin shall be determined by the measurement processes and methodologies to be developed and a pproved b y E PA/CARB/EMA. T his m argin w ill be s tructured t o encourage i nstrument manufacturers to d evelop more and more accurate instruments in the future.”

Given the foregoing, the work to be completed under this test plan is a vital component to the fulfillment of the settlement agreement, and it is vital to the successful implementation of a fully

e nforceable H DIUT p rogram. Because of thi s significance, it is c ritically impor tant tha t the work de tailed i n t his t est pl an be c arried out i n a s t horough, c areful a nd t imely a m anner a s possible.REPORT 03 .14936. 12 A-2 TABLE OF CONTENTS

Executive Summary ........................................................................................................ A-1 Table of Contents ............................................................................................................ A-2 List of Figures ................................................................................................................. A-4 List of Tables .................................................................................................................. A-4 1 Introduction ............................................................................................................. A-5 2 Monte Carlo Error Model and Measurement Allowance ....................................... A-7 2.1 Objective ......................................................................................................... A-7 2.2 Background ..................................................................................................... A-7 2.3 Methods and Materials .................................................................................. A-13 2.4 Simulation Procedure .................................................................................... A-13 2.4.1 Construction of the Error Surface ......................................................... A-15 2.5 Model Considerations ................................................................................... A-20 2.5.1 Convergence ......................................................................................... A-20 2.6 Simulati on Output ......................................................................................... A-20 2.6.1 Sensitivity Variation Effect

................................ ................................... A-20 2.6.2 Sensitivity Bias Effect

................................ ........................................... A-21 3 Engine Dynamometer Laboratory Tests ............................................................... A-21 3.1 Preliminary Audits ........................................................................................ A-21 3.1.1 Objective

............................................................................................... A-21 3.1.2 Background ........................................................................................... A-21 3.1.3 On

- site meeting to establish 1065 compliance requirements ............... A-22 3.1.4 Methods and Materials .......................................................................... A-22 3.1.5 Data Analysis ........................................................................................ A-22 3.1.6 PEMS Manufacturer PM PEMS Commissioning ................................. A-22 3.2 Bias and Precision Errors under steady state engine operation .................... A-23 3.2.1 Objective ............................................................................................... A-23 3.2.2 Background ........................................................................................... A-24 3.2.3 Methods and Materials

.......................................................................... A-24 3.2.4 Data Analysis ........................................................................................ A-25 3.3 Precision Errors under transient engine operation (dynamic response) ........ A-26 3.3.1 Objective ............................................................................................... A-26 3.3.2 Background ........................................................................................... A-26 3.3.3 Methods and Materials .......................................................................... A-26 3.3.4 Data Analysis ........................................................................................ A-27 3.4 ECM To rque and BSFC ................................................................................ A-30 3.4.1 Objective ............................................................................................... A-30 3.4.2 Data Analysis

........................................................................................ A-30 4 Environmental Chamber ....................................................................................... A-30 4.1 Da ta Analysis for Environmental Tests ........................................................ A-31 4.2 PM Generator Commissioning ..................................................................... A-32 4.3 Baseline ......................................................................................................... A-32 4.3.1 Objective ............................................................................................... A-32 4.3.2 Background ........................................................................................... A-32 4.3.3 Methods and Materials .......................................................................... A-32 4.3.4 Data Analysis ........................................................................................ A-33 4.4 Electromagnetic Radiation ............................................................................ A-33REPORT 03 .14936. 12 A-3 4.4.1 Objective ............................................................................................... A-33 4.4.2 Methods and Materials .......................................................................... A-33 4.5 Atmospheric Pressure ................................................................................... A-33 4.5.1 Objective ............................................................................................... A-33 4.5.2 Background ........................................................................................... A-33 4.5.3 Methods and Materials .......................................................................... A-34 4.5.4 Data Analysis ........................................................................................ A-36 4.6 Ambient Temperature and Humidity ............................................................ A-36 4.6.1 Objective ............................................................................................... A-36 4.6.2 Backgrou nd ........................................................................................... A-36 4.6.3 Methods and Materials .......................................................................... A-37 4.6.4 Data Analysis ........................................................................................ A-39 4.7 Orient ation and Vibration ............................................................................. A-39 4.7.1 Objective ............................................................................................... A-39 5 SwRI CVS and CE

- CERT Trailer Correlation ..................................................... A-39 5.1 Method and Materials ................................................................................... A-39 6 Model Validation and Measurement Allowance Determination .......................... A-40 6.1 Model validation

........................................................................................... A-41 6.1.1 Objective ............................................................................................... A-41 6.2 Measuremen t Allowance Determination ...................................................... A-43 6.2.1 Objective ............................................................................................... A-43 6.2.2 Background ........................................................................................... A-44 6.2.3 Methods and Materials .......................................................................... A-44 6.2.4 Data Analysis ........................................................................................ A-44 7 Time and Cost ....................................................................................................... A-44 7.1 Timeline ........................................................................................................ A-44 7.2 Cos t ............................................................................................................... A-44 8 Abbreviations used in Brake Specific Equations .................................................. A-45REPORT 03 .14936. 12 A-4 LIST O F F IGURES

FIGURE 1. PROBABILI TY DENSITY FUNCTIONS FOR SAMPLING ERROR S URFACES

..................................... A-8
FIGURE 2. BRAKE

- SPECIFIC PM EMISSION S CALCULATION FOR ME THOD 1

............................................... A-9
FIGURE 3. BRAKE

- SPECIFIC PM EMISSION S CALCULATION FOR ME THOD 2

............................................. A- 10
FIGURE 4. BRAKE

- SPECIFIC PM EMISSION S CALCULATION FOR ME THOD 3

............................................. A- 11
FIGURE 5. ERROR SUR FACE: PEMS VS. LAB

.............................................................................................. A- 14
FIGURE 6. ERROR SUR FACE: (PEMS

- LAB) VS. LAB

.................................................................................... A- 14
FIGURE 7. ERROR SUR FACE: FINAL VERSION

........................................................................................... A- 15
FIGURE 8. OVERVIEW O F MONTE CARLO SIMULA TION

............................................................................ A- 18
FIGURE 9. EXAMPLE OF A NTE CYCLE

........................................................................................................ A- 29
FIGURE 10. PRESSURE HISTOGRAM

........................................................................................................... A- 34
FIGURE 11. PRESSURE

- TIME ENVIRONMENTAL T EST CYCLE

..................................................................... A- 35
FIGURE 12. TEMPERATURE HISTOGRAM

................................................................................................... A- 37
FIGURE 13. TIME SERI ES CHART OF AMBIENT TEMPERATURE TEST

......................................................... A- 38

LIST OF TABLES

TABLE 1. ALLOWED MOD IFICA TIONS

.......................................................................................................... A-5
TABLE 2. EXAMPLE OF SELECTION OF MEASURE MENT ALLOWANCE AT 0. 02 G/HP

- HR NTE THRESHOLD A- 12
TABLE 3. ERROR SURFA CES FOR THE BSPM SIM ULATION

......................................................................... A- 19 TABLE 4. ENGINE, EXH AUST CONFIGURATION, AND S TEADY

- STATE MODES

........................................... A- 23
TABLE 5. EXAMPLE OF SS ERROR SURFACE

............................................................................................... A- 25
TABLE 6. NTE TRANSIE NT CYCLE 8

.............................................................................................................. A- 27

T ABLE 7. DYNAMIC RESP ONSE INTER

- NTE EVENTS

................................................................................... A- 29
TABLE 8. CONCENTRATI ON A ND DILUTION RATI O SCHEDULE WITH PM G ENERATOR

............................. A- 31
TABLE 9. ATMOSPHERIC PRESSURE TEST SEQUEN CE

................................................................................ A- 35
TABLE 10. AMBIENT TE MPERATURE TEST SEQUE NCE

............................................................................... A- 38
TABLE 11. PROJECTED PM

- PEMS TIMELINE

............................................................................................... A- 44
TABLE 12. PROJECTED COST ESTIMATE

..................................................................................................... A- 45REPORT 03 .14936. 12 A-5 1 INTRODUCTION

This te st pl an will e stablish a P EMS me asurement a llowances for P M, as r egulated b y the manufacturer

- run on

- highway heavy

- duty diesel engine in

- use test program. T he measurement allowance will be established using various laboratory facilities and PEMS. The measurement allowance will be established in units of brake

- specific emissions (g/hp

- hr), and it will be added to t he f inal N TE P M s tandard, after a ll t he ot her a dditive and m ultiplicative a llowances ha ve been applied. This test plan will establish the PM measurement allowance.

The PEMS used in this test plan must be standard in

- production makes and models that are for sale as commercially available PEMS. In addition, PEMS a nd any support equipment must pass a “red

- face” test with respect to being consistent with acceptable practices for in

- use testing. For example, t he e quipment m ust m eet all s afety and t ransportation regulations f or us e on

- board heavy -duty vehicles.

Even though the PEMS cannot be “prototypes” nor their software “beta” versions, the steering committee has already agreed that after delivery of PEMS to the contractor, there may be a few circumstances i n w hich PEMS m odifications m ight be a llowed, but t hese m od ifications mus t meet certain deadlines, plus they are subject to approval by the steering committee. A lso, any implementation of such approved modifications will not be allowed to delay the test plan, unless the steering committee specifically approves such a delay. Table 1 summarizes these allowable modifications and their respective deadlines:

TABLE 1 . ALLOWED MODIFICATI ONS

Allowed Modifications Before start of... Steering committee approved hardware and software modifications that af fect em issions r es ults; inc luding but not limited to fittings, components, calibrations, compensation algorithms, sampling rates, recording rates, etc. Steady

- State Testing

Steering c ommittee a pproved ha rdware m odifications f or D OT approval or any other safety requirement approval Environmental Chamber Testing
Delivery of any environmental / weather enclosure to contractor Environmental Chamber Testing Post

- processing software to determine NTE results Model Validation DOT approval and documentation Model Validation Steering committee appr oved hardware or s oftware t hat i mproves the contractor’s efficiency to conduct testing and data reduction Always Allowed

The s teering com mittee appr oved three di fferent P EMS t hat i ncludes t he A VL M icro -Soot Sensor ( MSS), t he H oriba Transient P articulate Matter ( TRPM), and t he S ensors P roportional Particulate M atter D iluter ( PPMD). H owever, because of t he different m easurement technologies employed by each of these systems, the three different PEMS hold slightly different status w it h respect t o de termining t he P M m easurement allowance. Because i nertial microbalances a re already approved for PEMS a pplications in 40 C FR Part 1065, t he Sensors PPMD will be one of the PEMS used to determine the measurement allowance. A nd because EPA’s PM standard is based upon a gravimetric filter analysis, the Horiba TRPM will also be used to determine t he measurement al lowance. T he l owest m easurement al lowance v alueREPORT 03 .14936. 12 A-6 between the two will be selected as the final measurement allowance for PM. If that value does not validate, then the lowest validated value will be chosen. If the lowest validated value chosen is w ithin 0.0075 g /hp

- hr f rom t he l owest non

- validated va lue, t hen t he l owest va lidated va lue will be the measurement allowance. Otherwise, the MAS C will spend up to a $100,000 to figure out a resolution to the problem by generating more data or changing the way the validation was performed. If that does not lead to a resolution, then Executive Management of EMA and EPA will have to settle the issue.

Note tha t at the c onclusion of s uccessful te sting o f the H oriba s ystem in this me asurement allowance program, EPA intends to approve the Horiba system as an alternative for use, or EPA may elect to amend 40 CFR Parts 86 and/or 1065 to allow the use of the Horiba TRPM or other PEMS t hat ope rate upon similar m easurement pr inciples. B ecause t he A VL s ystem m easures only the soot component of PM, the measurement allowance will not be determined using the AVL r esults, unl ess bot h t he S ensors and H oriba s yste ms f ail to complete the me asurement allowance program. Note that the steering committee may determine at the conclusion of the program that the AVL MSS is a viable alternative for demonstrating compliance. Under such a circumstance EPA may amend the Heav y- Duty In

- Use regulation to allow for its use.

This test plan describes a computer model, a series of experiments that are used to calibrate the model, and another series of experiments that are used to validate the calibrated model.

The te st pl an first describes t he c omputer m odel. T he c omputer m odel s tatistically combines many sources of PEMS and lab error, which are nearly impossible to capture simultaneously in a single te st. T he mode l w ill us e s tatistics to apply the e rrors in a way that s imulates actual running o f a P EMS i n

- use. T he m odel w ill a lso c onsider onl y t he po rtion of e rror t hat i s attributable to PEMS, and it will subtract the error that is already tolerated in an emissions lab today. The model will also calculate and validate results a ccording to 40 CFR Part 1065.

The test plan then describes the series of experiments. These tests will characterize the m any sources of PEMS and lab error so that the specific nature of the errors can be programmed into the computer model. The nature of the error has to do with the way PEMS and the lab react to certain conditions. For example, under varying environmental conditions such as temperature or vibration, a PEMS might exhibit signal drift, or it may record noise that is not a part of the true emissions.

Next, t he e xperimental r esults w ill be e ntered i nto t he c omputer m odel, and t he m easurement allowances are calculated by the model. The m odel uses a “reference" PEMS data set, which will ha ve m any “reference N TE ev ents.” T he model s tatistica lly applies a ll the e rrors to the reference data set, calculates results, and saves the results. Then the model will be run with all errors set to zero to calculate the ideal results of the reference data set. Each difference between a reference NTE even t’s result with errors and its respective ideal result will be a brake

- specific difference that is recorded for later use. Then the process repeats using the same reference data set, t o w hich ne w, s tatistically s elected e rrors are a pplied, and t hus another unique s et of differences is calculated. As the model continues to iterate and generate more and more results, patterns are expected to appear in the output data. These patterns should be the distributions of differences, ba sed upon the er ror t hat w as s tatistically and repeatedly ap plied to the r eference data set. M any difference distributions will be determined: for each reference NTE event, for each of the two brake

- specific calculation methods (three in case of the AVL system only), andREPORT 03 .14936. 12 A-7 for each PEMS . It has been agreed that the 95 th percentile values of these distributions will be taken as reasonable “worst case” results for each reference NTE event. Details on how all these distributions will be reduced to determine the PM measurement allowance is given in the “Error Model” section of this test plan.

Because the calculation based on Method 2 and Method 3 require gas- based fuel flow calculation based on t he measurement of CO 2 , CO, and NMHC, a decision w as made to use the gaseous PEMS data for this purpose, without the need to perform gaseous measurement during the PM

PEMS program.

Finally, the te st pl an describes how the c omputer mode l w ill be v alidated against r eal

- world over-the- road i n

- use P EMS ope ration a s w ell a s a dditional l ab t esting. For t he ove r

- the

- road testing, PEMS emissions measurements will be conducted, while at the same time a reference laboratory will be towed along to measure the same emissions. For the lab testing, an attempt will be made to simulate real

- world engine operation to “replay” an over -the- road test in the lab. Data f rom t hese f inal experiments w ill be us ed t o va lidate t he m odel, w hich m ust be done i n order to gain sufficient confidence that the model did not establish unreasonable measurement allowances.

The following sections of this test plan are written as instructions to the contractor or contractors who will complete the test plan.

2 MONTE CARLO ERROR MODEL AND MEASUREMENT ALLOWANCE

2.1 OBJECTIVE

Use Monte Carlo (e.g. random sampling) techniques in an error model to simulate the combined effects of al l t he a greed

- upon s ources of P EMS e rror i ncremental t o l ab e rror. C reate e rror “surfaces” for the Monte Carlo simulation to sample, based upon results from the experiments described in Sections 3 and 4. Exercise the model over a wide range of NTE events, based on a single, reference data set of at least 150 but no more than 200 unique NTE events. Determine the pollutant- specific brake

- specific additive measurement allowance for PM.

2.2 BACKGROUND

The error model uses Monte Carlo techniques to sample error values from “error surfaces” that are generated from t he results of each of t he e xperiments de scribed in Section 3 on engine dynamometer laboratory tests and Section 4 on e nvironmental chamber tests. The lab test err or surfaces cover the domain of error versus the magnitude of the signal to which the error is to be applied (i.e. 1 st to 99 th percentile error vs. concentration, flow, torque, etc.). T his is illustrated later in this section. The e nvironmental te st e rror surfaces f or s hock & vi bration a nd electromagnetic & r adio f requency i nterference (EMI/RFI) cover t he s ame dom ain as t he l ab tests. The environmental test error surfaces for pressure and temperature are characteristically different because they cover the domain of environmental test cycle time versus the magnitude of the s ignal t o which the error i s t o be applied (i.e. error at a s elected t ime vs . concentration). Details on how e ach s urface i s generated are given in each of t he r espective s ections. T hes e surfaces w ill al ready b e adj usted to represent P EMS er ror i ncremental t o lab error; t herefore, these surfaces are sampled directly by the model.REPORT 03 .14936. 12 A-8 The error model will use two different probability density functions (PDFs) as shown in Figure 1 to sample t he e rror s urfaces, de pending upon which e xperiment t he s urface r epresents. T o sample error surfaces that are generated from all the laboratory test results (Section 3), and the environmental t est r esults f or s hock & vi bration ( Section 4) , t he m odel w ill u se a t runcated normal PDF because these tests are designed to evenly cover the full, but finite, range of engine operation and ambient conditions. To sample error surfaces that are generated from the pressure and temperature e nvironmental te st r esults ( Se ction 4) , t he m odel w ill us e a uni form P DF because t hese t ests are al ready d esigned to cover t he t ypical range and frequency of t he respective conditions.

FIGURE 1 . PROBABILITY DENSITY FUNCTIONS FOR SAM PLING ERROR SURFACES

The random values that are obtained from both distributions are labeled i c in Figure 1 and range from

- 1 to 1. Note that for the pressure and temperature environmental tests, a uniform PDF will be used to sample test time, from which the nearest (in time) calculated errors are used. The errors from the other tests will be ali gned with the truncated normal PDF such that each of the 50 th percentile va lues at each of t he t ested signal m agnitudes i s cent ered at t he m edian of t he PDF (i c = 0), and the 1 st and 99 th percentile error values at each of the tested signal magnitudes will be a ligned with the e xtreme ne gative ( i c =

- 1) a nd pos itive ( i c = +1) ed ges o f t he P DF, respectively.

Each error surface will be sampled along its i c axis (y

- axis) once per reference NTE event trial, and it will be sampled along its parameter value axis (x

- axis, e.g., concentration (only for AVL MSS), flow, torque, etc...) once per second, within a given reference NTE event trial. An error will be de termined for a given second and parameter al ong t he error ax is ( z

- axis) a t the intersection of an i c value and a parameter value. Probability Density Functions for Sampling Error Surfaces Once Per NTE Event

- 1.00 - 0.75 - 0.50 - 0.25 0.00 0.25 0.50 0.75 1.00

0 1 2 3 4 5 Relative Probability i

c Lab Tests, Normal, SD=0.60795, truncate @

- 1 & 1

Environmental Tests, Uniform

Note: A non

- truncated normal distribution with SD=0.60795 has P values of 0.01 and 0.99 at i c = - 1 and
i c =+1, respectively.REPORT 03 .14936. 12 A-9 To e nsure t hat t he m agnitudes of t he error s urfaces are a ppropriate, each da ta poi nt us ed t o generate the surfaces will be a mean or a weighted mean of 30 seconds of sampling.

Interpolation will be p erformed b y first line arly int erp olating e rror values at ea ch tested magnitude al ong t he s elected line pe rpendicular t o the i c axis. T hen f rom t hat l ine of e rrors, individual error values will be linearly interpolated at each second

- by

- second signal magnitude of the given NTE event in th e reference data set.

The r eference d ata s et t o which all er rors w ill be appl ied will be a l arge d ata s et of engine operation over a wide range of NTE events. This reference data set will be initially generated from col lections of real

- world PEMS data sets. The reference data s et s hould contain at l east 150 but no m ore t han 2 00 uni que N TE e vents. P arameters i n t he r eference da ta s et m ay be scaled in order t o exercise t he m odel t hrough a m ore appr opriate r ange of p arameters ( i.e. concentrations, f lows, ambient c onditions, e tc.). If t he pa rameters a re scaled, c are s hould be taken to maintain the dynamic characteristics of the reference data set.

After the errors are applied, NTE brake

- specific PM emissions results are calculated, using each of the thre e agreed

- upon NTE calculation methods. The three different brake

- specific emission calculation methods for PM referred to in this test plan are i) Torque

- Speed method, ii) BSFC method, a nd i ii) E CM

- Fuel S pecific m ethod, and these ar e i llustrated in Figure 2, 3, a nd 4

, respectively.

For all PM PEMS:

PM m is a flow weighted particulate matter exhaust concentration in g/mol



## ( ) **( ) ( )** 

# ∑ **∑**

**= =**  

###  ** **  ** **  ** ∆** 

**⋅ **  ** **  ** **  ** ∆**  ** **  ** **  ** **  ** **  ** **  ** ** 

**=⋅** N

i i i N

i i PM

PM 

### t mNT rpmSpeed t
s mol
n mol

g m hr kW ge

1 1



### 3600*1000*60 *14159.3*2** * *
/ 

Where for AVL:

PM m

1

1 

### *

* N

i i i
PM N

i i 

### g mol mPM n t mol s g m mol mol n t
s **=**

**=** 

###   **   ∗ ∆**  **    **  **    **  ** **

**=  **  **   **  **  ∆**  **   **  **   **  

# ∑

**∑** 

 is computed numerically as follows,

FIGURE 2. BRAKE- SPECIFIC PM EMISSION S CALCULATION FOR ME THOD 1REPORT 03 .14936. 12 A-10 For all PM PEMS:

PM m is a flow weighted particulate matter exhaust concentration in g/mol



## ( )

**( ) ( ) ( )**  

# ( **)**

**( ) ( )** 1

6 2 2 

### * *
/

* *10 % % *10 *

*

* *2*3.14159
60*1000*3600 i

N

PM i i
PM

ii i C

fuel fueli

i i 

### g mol m nt mol s e g kW hr

mol n xTHC ppm xCO xCO t M s

w g m s Speed rpm T N m **=**

**− −**



###   **  ∆**  **   **  **   ** 

**⋅ =  **  **   **  **  ++ ∆**  **   **  **   **  **   **  **   ** 

**⋅  **  **   **  

# ∑




 1 N

i **=** 

# ∑

Where for AVL:

PM m

1

1 

### *

* N

i i i
PM N

i i 

### g mol mPM n t mol s g m mol mol n t
s **=**

**=** 

###   **   ∗ ∆**  **    **  **    **  ** **

**=  **  **   **  **  ∆**  **   **  **   **  

# ∑

**∑** 

 is computed numerically as follows,

FIGURE 3. BRAKE- SPECIFIC PM EMISSION S CALCULATION FOR ME THOD 2REPORT 03 .14936. 12 A-11 For AVL Only: 

## ( )

**( ) ( ) ( )**  

# ( **)**

**( ) ( )** 6 2 1 2

1 

### /

* * * *10 % % *10

* *2*3.14159*
60*1000*3600 i PM

ifue l N fuel
PM i C i i

N i i

i 

### e g kW hr

g m w g s m t mol M xTHC ppm xCO xCO

Speed rpm T N m t **− −**

**=**

**=** 

### ⋅ = **   **  **   **  **   **  **∆  ** 
**+ +  **  ** **
**⋅ ∆  **  **   **  

# ∑

**∑** 

Where:



## ( ) 

# ( **) ( )**  **( ) ( )** 

# ( **)**

**( ) ( ) ( )**  

# ( **)** 6 2 1 2

6 2 1 2 

### / *

* * *10 % % *10

* * *10 % % *10 i

i i ifue l N fuel

i

C i i

PM

ifue l N fuel

i

C i i 

### g mPM g mol m w s t M xTHC ppm xCO xCO

g m mol g m w s t M xTHC ppm xCO xCO **− −**

**=**

**− −**

**=** 

###   **   **  **   **  **∆**
**+ +  **  ** **

**=  **  **   **  **   **  **   **  **∆**
**+ +  **  

# ∑

**∑** 



FIGURE 4. BRAKE- SPECIFIC PM EMISSION S CALCULATION FOR METHOD 3

Next, the NTE events are calculated by each of the three calculation methods, but with no error sampled or applied to the reference data set. These results are considered the “ideal” results of the reference NTE events. These ideal results are subtracted from each respective NTE event result ‘with errors’, and the difference is recorded. T hen a new set of errors are sampled and applied to the reference NTE event, and the NTE results ‘with errors’ are calculated again. The ideal results are aga in subtracted, and the difference is recorded. T his is repeated thousands of times so that the model converges upon distributions of brake

- specific differences for each of the original NTE events in the reference data set.

Then the 95 th percentile diffe rence value is determined for each NTE event distribution of brake

specific differences for PM for each calculation method. At this point there is one distribution of 95 th percentile differences for PM, where all the NTE events are pooled by the PM emissions for each of t he t hree di fferent c alculation m ethods. E ach of t he 95 th percentile di stributions represents a range of possible measurement allowance values.

From e ach of t hese t hree di stributions of p ossible m easurement a llowance v alues, one

measureme nt allowance per distribution must be determined. First the correlation between 95 th percentile differences versus the ideal PM emission is tested. For each calculation method, if a least squares linear regression of 95 th percentile differences versus id eal PM emissions has an r 2 (squared correlation coefficient) > 0.85 and an SEE (standard error of the estimate or root

- mean

-REPORT 03 .14936. 12 A-12 squared

- error) < 5 % of the median ideal PM emission, then that linear regression equation will be us ed to determine t he m easurement allowance f or that calculation method at the f ollowing NTE threshold:

PM = 0.02 g/hp-hr and 0.03 g/hp-hr

In cases where ex trapolation is r equired to determine t he m easurement allowance at t he N TE threshold, t he m easurement a llowance w ill be de termined u sing t he l inear r egression, but evaluated at the ideal PM emission that is closest to the NTE threshold, not extrapolated to the NTE t hreshold i tself. If t he l inear r egression does not pa ss t he aforementioned r 2 and SEE criteria, then the m edian value of the 95 th percentile di fferences i s us ed as t he s ingle measurement allowance for that calculation method.

Next, the calculation method is selected. T he above procedure will provide three measurement allowances, where applicable, one for each of the three different calculation methods. To make them com parable, the t hree m easurement al lowance va lues w ill be n ormalized by t he P M threshold and expressed as a percent. Also, if any measurement allowance is determined to have a va lue l ess t han zero, t hen that m easurement al lowance w ill be s et equa l t o zero. T he calculation m ethod w ith t he m inimum nor malized P M va lue w ill be c hosen a nd t he corresponding normalized PM value will be selected as the best measurement allowance for PM, assuming it va lidates. If it doe s not va lidate, then the mini mum va lue that va lidates w ill be chosen as long as it is within 0.0075 g/hp

- hr from the minimum value that did not validate. If the difference b etween the minimum va lue tha t va lidates a nd the mini mum va lue tha t di d not

v alidate i s greater t han 0.0075 g/hp

- hr, additional i nvestigation w ith up t o a $100,000 w ill be spent in order to understand why the minimum value chosen did not validate. If the problem is not r esolved a fter s pending t he $100,000, t hen t he m atter w ill be r eferred to executive management of EPA and EMA to decide on the PM measurement allowance.

Table 2 below i llustrates t he s election of t he c alculation m ethod. T he example i s ba sed on a hypothetical s et of nor malized PM me asurements f or the th ree calculation m ethods. The minimum of these normalized allowances is used to select the best method (highlighted in blue). In this hypothetical case, the BSFC method would be selected.

TABLE 2 . EXAMPLE OF SELECTI ON OF MEASUREMENT AL LOWANCE AT 0.02 G/HP

- HR NTE THRESHOLD

Allowance at Respective NTE Threshold (%)

Calc. Method ==> Torque- Speed BSFC ECM fuel specific

BSPM 38 % 18 % N/A
Selected Method==> BSFC Method

Therefore, 18% w ould be s elected as t he be st m easurement al lowance f or P M, assuming i t validates. Otherwise, the 38 % will be chosen if it validates. T hus, the additive brake

- specific measurement allowance would be:

PM = 18 % * 0.02 g/hp-hr = 0.0036 g/hp-hr, if it validates, and if not, then:REPORT 03 .14936. 12 A-13 PM = 38 % *0.02 g/hp-hr = 0.0076 g/hp- hr, if it validates, and if not, then:

spend up to a $100,000 to figure out why it did not validate in the first place, and then apply the above strategy again, assuming the value now validates. If not, then EPA and EMA executive management will decide on the PM measurement allowance value.

This PM value would be the value added to the actual brake

- specific NTE threshold for a given engine, based on actual family emissions limit, mileage, model year, etc.

2.3 METHODS AND MATERIAL S

Exercise the model using three different calculation methods: a) Torque

- Speed method, b) BSFC method, and c) ECM

- Fuel Specific method (only for AVL MSS). Determine which calculation method is the most accurate, and use it to estimate the measurement allowance. Each calculation method i s described in Figured 2, 3, and 4.

Prepare an Excel spreadsheet model for use with the Crystal Ball Monte Carlo software for error analysis o f br ake s pecific e missions, B SE, a s ou tlined i n s ection 2.4. C hanges t o t he m odel specifications m ay b e requested as a greed up on by t he S teering C ommittee. Prepare t he spreadsheet in a modular structure following the specified model outline, and make provisions for the identified calculation modules. Additionally, clearly identify and easily locate input cells
to t he model to facilitate any revisions that may become necessary for users who want to exercise the model with other Monte Carlo add

- ins such as @Risk or the newest versions of Crystal Ball. Test the spreadsheet with controlled test cases of simplified input distributions with the Crystal Ball add

- in to confirm correct model implementation in accordance with this test plan. Run at least one typical analysis as an additional confirmation.

Deliver t he electronic s preadsheet and a br ief r eport de scribing t he m odel, pr esenting t he t est cases, and describing pertinent information including the Crystal Ball version number, the Excel version number, the operating system and the computer. U se standard spreadsheet calculations so that no serious di fficulties w ill be anticipated r egarding a pplication i n ot her s preadsheet versions. Use Crystal Ball Version 7 or higher, and confirm test cases using Excel 2003.

Control revisions of the spreadsheet model using descriptive file names. Extensive revisions or testing with other software v ersions be yond that initially proposed m ay be re

- proposed b y the Steering Committee if and when a need for such additional work is identified.

2.4 SIMULATION PROCEDURE

For each of the measurement errors in Section 3, create an error surface and sample it according to the aforementioned PDFs. Each error surface represents an additive error

— or a subtractive error if the sign is negative

— relative to the reference value to which it is applied. F igure 5, Figure 6, and Figure 7 serve as a hypot hetical PM example of how these error surfaces should be created f or e very e rror. T he pl ots s hown c orrespond t o PM emissions c oncentration da ta representing 1 P EMS, t wo e ngines, and t hree exhaust c onfigurations e ach, w ith a ll 6 s ets of PEMS data pooled to gether. Note that separate error surfaces will be constructed for each of the three P EMS uni ts ( AVL, H oriba a nd S ensors). The e xample a pplies t o the e rror m odule f orREPORT 03 .14936. 12 A-14 steady

- state (SS) bias and precision PM concentration errors (Section 3.2). These figu res will be referenced by each “Data Analysis” section for the various errors discussed in this test plan.

FIGURE 5. ERROR SUR FACE: PEMS VS. LAB

FIGURE 6. ERROR SUR FACE: (PEMS

- LAB) VS. LAB Error Surface for SS PM Concentration

18 26 38 45 56 69

13 24 36 42 53 64

8 19 33 39 51 61

0 10 20 30 40 50 60 70 80

0 10 20 30 40 50 60 70 80
PM μg/mole (lab,mean)PM μg/mole (PEMS)

diagonal 95th percentile 50th percentile (median) 5th percentile

Error Surface for SS PM Concentration

8

6 8

5 6 9

3 4 6

2 3 4

-2 -1 3

-1 1 1

-4 -2 0 2 4 6 8 10

0 10 20 30 40 50 60 70

PM μg/mole (lab,mean)PM μg/mole (PEMS) - PM μg/mole (lab)

95th percentile 50th percentile (median) 5th percentileREPORT 03 .14936. 12 A-15 FIGURE 7. ERROR SURFACE: FINAL VERSION

Errors f rom Section 3 ( Engine D ynamometer Laboratory t ests) and S ection 4 ( Environmental Chamber Tests) are combined by adding all of the sampled errors once per NTE event trial. For example, in order to assess the errors in PM concentration for each NTE event, several modules will be created such that:

PM_with errors = PM _ideal + **∆(** μg/mole) 1 + **∆(** μg/mole) 2 + **∆(** μg/mole) 3 + ...

where, **∆(** μg/mole) 1 **∆(** μg/mole) = PM concentration errors due to steady state bias and precision errors,

2 **∆(** μg/mole) = PM concentration errors due to ambient temperature,

3 etc.... = PM concentration errors due to ambient pressure,

2.4.1 Construction of the Error Surface

2.4.1.1 PEMS vs. Lab

Acquire raw data with the PEMS at various average concentration levels as per Section 3.2. Plot the “P EMS” s ignal s ve rsus t he c orresponding “lab” s ignals t hat w ere m easured us ing l ab equipment. This plot pools all bias and precision errors for one PEMS and for all data from all engines for all steady

- state modes. Shown in Figure 5 are the 5 th , 50 th and 95 th percent iles at the

mean PM concentration level from the lab (note that the distribution of data at each level is not necessarily Gaussian). If t he 50 th Error Surface for SS PM Concentration

10.1 6.8 8.8 7.2 11.1

3 4 6 2 3 4

-4.1 -3.1 1.8 -2.2 0.2 -0.2

-1.5

-1 -0.5

0 0.5

1 1.5

0 10 20 30 40 50 60 70

PM μg/mole (lab,mean)ic_SS_PM

99th percentile 50th percentile (median) 1st percentile Error Surface: z-axis = ΔSS_PM_μg/mole

6.

percentile is di fferent tha n the line of p erfect a greement (diagonal), the data suggests that there is a bias error between PEMS and Lab. In essence this graph s hows t he s tatistical di stribution m easured b y the P EMS a t each a verage c oncentrationREPORT 03 .14936. 12 A-16 level sampled. The example shows only 6 discrete PM concentration levels (ranging from 10 -60 μg/mole). However, the a ctual number of discrete levels will be determined by the total number of operating conditions actually run for all the tests of all the engines. For example, the SS PM testing will select 6 modes representing typical operating conditions. Thus, the actu al plot for SS PM will likely have 36 discrete concentration levels (6 modes x 1 PEMS x 2 engines x 3 exhaust configurations).

2.4.1.2 (PEMS – Lab) vs. Lab

The plot in Figure 6 basically shows the “additive error band” measured during testing. The plot is creat ed by first subtracting the “lab” PM value from the corresponding individual PEMS PM measurement for each test run. This difference is defined as the ‘delta’ error. Next, the “PEMS

– Laboratory” delta errors are pooled at each average lab PM value to obt ain the 95 th , 50 th , and 5 th percentile values, respectively, displayed in Figure 5. N otice that if lab error exceeds PEMS error at a given percentile, crossover o f va lues can occur. T his i s ac ceptable b ecause t he crossover effectively reduces PEMS error whenever lab error exceeds PEMS error.

In order to obtain estimates of the 1 st and 99 th percentiles for the delta errors for a given “lab” PM value, each side of the corresponding error distribution will be assumed to independently fit a normal di stributi on. Because o f t he a symmetry of t he da ta, t his m ethodology w ill yield t wo halves of a nor mal di stribution. The m edian of e ach nor mal di stribution w ill be the m edian based on t he delta errors given in Figure 6. The 95 th percentile delta error will form th e upper boundary of one half of the normal distribution, and the 5 th percentile delta error will form the lower bounda ry of t he ot her ha lf of t he nor mal di stribution. W hen e ach s ide of t he da ta distribution is fitted to a normal distribution using the above boundary conditions, one can then expand each half of the distribution from the error surface to obtain the 1 st and 99 th percentiles of the data for the given “lab” PM value.

2.4.1.3 Error Surface

This step normalizes the data in Figure 7 using what is called a “variability index (i c )”, which represents the random sampling by the Monte Carlo technique, in order to select a given error level. This variability index is allowed to vary from

– 1 to +1. T he likelihood of i c being any value between

– 1 through +1 is specified by the PDF assigned to i c . In the given example, i c is assumed t o va ry according t o a nor mal di stribution dur ing M onte C arlo calculations. T his i s because it is believed that the distribution of errors due to steady

- state bias and precision wil l be centered about the 50 th percentile of the full range of conditions measured according to Section 3.2. The pressure and temperature environmental error modules use uniform probability density functions for their respective variability index. Each set of data for each lab set

- point mean (i.e., lab reference value) in Figure 6 is normalized by aligning the 1 st percentile error from the fitted normal distributions with i c =

- 1, the 50 th percentile error with i c = 0, and the 99 th percentile error from the fitted normal distribution with i c = +1.

Error surfaces such as the one presented in Figure 7 are the input modules that the Monte Carlo simulation program will use during calculations of brake

- specific PM emissions. For exam ple, for a given NTE calculation a random i c value is chosen once per NTE event trial. Let us assume that the first random sample produced an i c = 0.5. Let us also assume that during this NTE event trial, the reference PM concentration is 10 μg/mole. In this case,REPORT 03 .14936. 12 A-17 **∆(** μg/mole) 1 = (3 + 10.1) / 2 = 6.55 μg/mole.

Also, from Figure 7, for i c = 0.5, the reference PM = 10 μg/mole.

For that step in the calculation, the Monte Carlo approach will add this “delta” to the reference concentration value of 10 μg/m ole (10 μg/mole + 6.55 μg/mole = 16.55 μg/mole) to represent errors in steady

- state bias and precision for i c = 0.5, and reference NTE PM = 10 μg/mole. If during the same NTE event in the reference data set, a reference concentration of 35 μg/mole is read , then,

**∆(** μg/mole) 1 = ((6 + 8.8) / 2 + (2 + 6.2) / 2) / 2 = 5.75 μg/mole (from Figure 7)

Note that first the error along the i c line perpendicular to the i c axis (in this case the line along 0.5) is linearly interpolated at each discrete concentratio n level. T hen those interpolated values are t hemselves l inearly interpolated to determine t he e rror corresponding t o each reference concentration in the NTE event. Note that the random selection is once per reference NTE event trial, but the error along that i c line is applied to every second

- by

- second value within the given reference NTE event, except for PM concentration in the case of Horiba and Sensors, where no second

- by

- second i nformation a re a vailable, b ut di fferent P M c oncentration l evels m ay be

a vailable for a specific NTE event.

Now let us assume that the error in PM concentration is composed of only 3 deltas: **∆(** μg/mole) 1 , **∆(** μg/mole) 2 , and **∆(** μg/mole) 3 . And let us assume that for a given reference NTE event trial we have the following values:

**•** Reference PM at one second= 30 μg/mole

**• ∆(**  μg/mole) 1

**• ∆(**  μg/mole) = 6 μg/mole

2

**• ∆(**  μg/mole) = -2 μg/mole

3 = -3 μg/mole.

When the model calculates brake

- specific emissions by each of the three calculation methods, it will use the following PM value, which ha s all of its error applied:

PM = 30 + 6 –2 – 3 = 31 μg/mole.

The appl ication of er ror at t he f irst s elected i c continues dur ing t he e ntire N TE e vent w ithout having to randomly sample again. In other words, i c

PM m will not change during that random trial. For all of the variables except for , the errors may continue to change during an NTE event
on a s econd

- by

- second ba sis i f t heir e rror s urface ha ppens t o be a f unction of l evel. F or t he second randomly selected i c this entire process of determining the **∆** μg/mole errors is repeated. The s imulation will c ontinue to randomly s elected i c values f or t housands of t rials unt il convergence is met.

For the Horiba and Sensors generated reference NTE events, there is only one flow- weighted PM value for the entire NTE event. D uring the simulation for these t ypes of reference NTEs, the single PM value will be used in the interpolation of the corresponding PM error surfaces (i.e.,REPORT 03 .14936. 12 A-18 steady

- state PM, tra nsient PM) at all seconds of the reference NTE event. Since the PM value will not vary from second- to -second, the only interpolation will occur according to the i c value at each of the simulation trials.

The same second

- by

- second sampling and interpolation approach would be used for other deltas such a s a mbient t emp, a mbient pr essure, s hock a nd vi bration, B SFC i nterpolation, t orque, exhaust f low r ate, e tc. A n ove rview o f t he Monte C arlo s imulation f or P M i s de tailed i n Figure 8.

FIGURE 8. OVERVIEW OF MONTE CARLO SIMULATIONREPORT 03 .14936. 12 A-19 Table 3 lists the e rror s urfaces tha t w ill be c reated for us e in simulating the B SPM e rror differences.

TABLE 3. ERROR SURFACES FOR T HE BSPM SIMULATION

Calculation Component Test Source Error Surface Delta PM Engine Dyno Delta PM SS Delta PM Engine Dyno Delta PM Transient Delta PM Environ Delta PM Ambient Temperature Delta PM Environ Delta PM EMI/RFI Delta PM Environ Delta PM Atmospheric Pressure Delta PM Environ Delta PM Vibration Delta CO Engine Dyno Delta CO SS Delta CO Environ Delta CO Atmospheric Pressure Delta CO Environ Delta CO Ambient Temperature Delta CO Engine Dyno Delta CO Time Alignment Delta CO2 Engine Dyno Delta CO2 SS Delta CO2 Engine Dyno Delta CO2 Transient Delta CO2 Environ Delta CO2 Ambient Temperature Delta NMHC Engine Dyno Delta NMHC SS Delta NMHC Engine Dyno Delta NMHC Transient Delta NMHC Environ Delta NMHC Atmospheric Pressure Delta NMHC Environ Delta NMHC Ambient Temperature Delta NMHC Environ Delta Ambient NMHC Delta Exhaust Flow Engine Dyno Delta Exhaust Flow SS Delta Exhaust Flow Engine Dyno Delta Exhaust Flow Transient Delta Exhaust Flow Engine Dyno Delta Exhaust Flow Pulsation Delta Exhaust Flow Engine Dyno Delta Exhaust Flow Swirl Delta Exhaust Flow Envi ron Delta Exhaust EMI/RFI Delta Exhaust Flow Environ Delta Exhaust Temperature Delta Exhaust Flow Environ Delta Exhaust Pressure Delta Torque Engine Dyno Delta Dynamic Torque Delta Torque Engine Dyno Delta Torque DOE Testing Delta Torque Engine Dyno Delta Torque Warm

- up Delta Torque Engine Dyno Delta Torque Humidity/Fuel Delta Torque Engine Dyno Delta Torque Interpolation Delta Torque Engine Manuf Delta Torque Engine Manuf Delta Speed Engine Dyno Delta Dynamic Speed Delta Fuel Rate Engine Dyno Delta Dynamic Fuel RateREPORT 03 .14936. 12 A-20 2.5 MODEL CONSIDERATIONS

2.5.1 Convergence

The ma in goal of the c onvergence criteria is to define how ma ny s imulation trials a t a given reference NTE event are required to estimate the 95 th percentile BSPM emission differences with a given precision. T he convergence method to be used is based on a nonparametric statistical technique 3 which de fines a 90% confidence i nterval f or t he 95 th percentile of t he BSPM emissions di fferences f or an ind ividual r eference N TE s imulation. I f t he w idth of t he 90% confidence interval is less than 1% of the BSPM emissions threshold, then convergence is met. The following steps define the convergence method:

1. Run the Monte Carlo simulation for N trials for a single reference NTE event. 2. Order the BSPM emissions differences from smallest to largest. 3. Identify the trial number at the lower end of the 90% confidence interval

n lower N N *05.0*95.0645.1*95.0

**−** = 4. Identify the trial number at the upper end of the 90% confidence interval

n upper N N *05.0*95.0645.1*95.0 **+** = 5. Compute (BSPM difference value at n upper ) – (BSPM difference value at n lower 6. If the result in (5) < 1% of the BSPM emissions NTE threshold (0.02 g/hp-hr) then convergence is met. ).

2.6 SIMULATION OUTPUT

It is i mportant t o unde rstand a nd i dentify w hat e rror s urfaces ha ve t he m ost influence ( i.e., sensitivity) on t he B SPM e missions ‘ with e rrors’ a nd, t hus, t he resulting B S e missions differences. Contributions to sensitivity can be attributable to changes in variance and/or bias.

2.6.1 Sensitivity Variation Effect

During the Monte Carlo simulation for each reference NTE event, sensitivity charts produced by Crystal B all w ill be g enerated and stored in output R EPORT f iles. Crystal B all c alculates sensitivity b y co mputing t he r ank c orrelation coefficient be tween every assumption ( error surface) and forecast value (delta BS emissions) while the simulation is running. Positive rank correlations i ndicate t hat an increase i n the assumption is as sociated with an increas e i n t he forecast. The larger the absolute value of the rank correlation the stronger the relationship.

Sensitivity charts in Crystal Ball provide a means to determine how the variances of the error surfaces affect the variance in the forecast values. H ence, the sensitivity charts developed during a simulation are displayed as “Contribution to Variance” charts which are calculated by squaring the r ank c orrelation coefficients for all a ssumptions us ed i n a pa rticular f orecast a nd t hen normalizing the m to 100%. T he assumption ( error s urface) w ith t he hi ghest c ontribution t o variance (in absolute value of the percent) is listed first in the sensitivity chart.

Simulation results f rom a ll r eference NTE e vents w ill pr oduce s ensitivity values f or the 95 th

perc entile delta PM emissions by all three calculation methods.REPORT 03 .14936. 12 A-21 2.6.2 Sensitivity Bias Effect

Another type of sensitivity to be examined in this study is concerned with the effects of potential “bias” in error surfaces and their effects on the forecast values. In order to study these effects a new error surface assumption will be added to the simulation model for each of the original error surfaces.

This a ssumption w ill be s ampled a s a di screte binary distribution ( i.e., on or of f) during t he simulation. For each trial of the simulation, the original error surfaces and ‘on/off’ error surfaces will be s ampled a ccording t o t heir de fined s ample di stribution. I f t he ‘ on/off’ e rror s urface produces a n ‘off’ c ondition, t he de lta e missions f rom t hat pa rticular error s u rface w ill not be added to the BSPM emissions computations for the BSPM emissions ‘with errors’. Similarly, if the ‘ on/off’ error s urface pr oduces a n ‘ on’ c ondition, t he de lta e missions f rom t hat pa rticular error surface will be added to the BSPM emission s calculations.

During e very t rial of t he s imulation, t he exclusions due t o t he ‘ off’ c onditions w ill r esult i n various combinations of the error surface delta emissions being added to the BSPM emissions ‘with e rrors’ computations. O ver t he c ourse of a s imulation with thousands of tr ials, the sensitivity of a pa rticular error either ‘on’ or ‘off’ will be assessed by examining the change in the forecast delta emission. Therefore, in a single Monte Carlo simulation of a r eference NTE event sensitivities du e to variance and/or bias will be explored.

3 ENGINE DYNAMOMETER LABORATORY TESTS

Utilize e ngine d ynamometer la boratory te sting t o establish the di fference be tween PM P EMS and PM based on laboratory measurement in accordance with Part 1065. Also establish how well ECM parameters can be used to estimate torque and BSFC.

First, however, audit all the PEMS and lab equipment to ensure that they are operating properly, according t o 40 C FR P art 1065, S ubpart D . N ext, c onduct s teady

- state engine d ynamometer
tes ts to establish PEMS s teady

- state bi as a nd pr ecision r elative t o t he l ab. T hen, c onduct transient e ngine d ynamometer te sting to determine P EMS tr ansient pr ecision by r epeating transient NTE events. Finally, compare ECM derived torque and BSFC to laborato ry measured torque and BSFC.

3.1 PRELIMINARY AUDITS

3.1.1 Objective

Conduct 40 CFR Part 1065, Subpart D audits of all engine dynamometer laboratory systems and all PEMS.

3.1.2 Background

Because the overall purpose of this entire test plan is to establish measurement allowance that account for the incremental difference in the performance of PEMS versus engine dynamometer laboratory s ystems, the f irst task is to audit all of the measurement s ystems to ensure that theREPORT 03 .14936. 12 A-22 specific systems used for testing meet EPA’s minim um performance requirements. T he audits also help to minimize bias errors between PEMS and lab s ystems measurements. However, in case a specific PM

- PEMS does not meet the specifics of Part 1065 requirement, the MASC will decide on how to move forward by perhaps allowing some flexibility in passing Part 1065 audit, in situations where it mi ght be needed, especially if the performance of a system is within the expectation of the manufacturer.

3.1.3 On

- site meeting to establish 1065 compliance requirements

In ord er to clarify what are all the requirements expected from the lab

- grade instrumentation and PEMS equipment, with respect to 1065 compliance, a meeting will be held between the test plan steering c ommittee a nd the c ontractor at the c ontractor s ite to provid e t he c ontractor w ith guidance regarding which specific sections of Part 1065 S ubpart D are required and which are optional. In case Part 10 65 requirement is demonstrated to be too stringent or impractical, the contractor may seek approval from the MASC to lessen the stringency of Part 1065 in relation to the PEMS.

3.1.4 Methods and Materials

Use the methods and materials described in 40 CFR Part 1065, Subpart D to conduct audits of all lab a nd P EMS m easurement s ystems. E ven i f l ab s ystems a nd P EMS pa ss initial S ubpart D audits, allow lab operators and PEMS manufacturers to make on

- site adjustments to improve the performance o f t heir s ystems pr ior t o e ngine t esting. A llow a djustments t o be ba sed on recalibrations w ith reference s ignals t hat a re a llowed in 40 C FR P art 1065. T he s teering committee may direct the contractor to calibrate or adjust the laboratory sampling system based on audit results. The steering committee may also suggest that a PEMS manufacturer calibrate or adjust one or more PEMS based on lab audits.

3.1.5 Data Analysis

Use t he da ta a nalyses d escribed i n C FR P art 1 065 S ubparts D , J a nd G. F or a ll s ubsequent testing, use onl y thos e measurement s ystems tha t pa ss the mini mum p erformance c riteria in Subpart D, unless a deficiency is deemed accep table in writing b y all parties including PEMS manufacturers. P rovide a l ist a nd br ief de scription of a ll t he a udits c onducted f or each P EMS manufacturer type. EPA would likely use this list as a template for the data requirements in the PM portion of the HDIU testing program.

3.1.6 PEMS Manufacturer PM PEMS Commissioning

Notify PEMS manufacturers when the 1065 audits are complete and the first set of PM PEMS are completely ins talled in the e ngine d ynamometer te st c ell

— in pr eparation f or emissions testing. Schedule dates and times that are prior to the start of emissions testing for each PEMS manufacturer to conduct a final commissioning of all their PEMS that are on site, including those PEMS that are not installed in the test cell. PEMS manufacturers may in spect their PEMS and make a ny f inal a djustments to their r espective PEMS in order f or the P EMS to meet the ir specifications. Allow PEMS manufacturers to inspect the installation of their PEMS in the test cell. If P EMS m anufacturers t ake exception to any portion of t he i nstallation or on

- site configuration, a ttempt t o r esolve any s uch i nstallation i ssues. If s uch i ssues a re not e asilyREPORT 03 .14936. 12 A-23 resolvable, notify the steering committee, who will determine a course of action. Once PEMS manufacturers ha ve com pleted their commissioning, notify the s teering c ommittee. From thi s point any further modifications to the PEMS may only be made according to Table 1 of this test plan.

3.2 BIAS AND PRECISION E RRORS UNDE R S TEADY S TATE EN GINE OPERATION

3.2.1 Objective

Evaluate the bias and precision using one engine and one exhaust configuration, shown in Table 4, a nd 10 r epeats of s teady

- state m odes, a nd t hree s ets of P EMS uni ts, each s et i ncluding t he MSS, TRPM, and PPMD. Thus, the total number of NTE steady

- state points required to c onduct the s teady

- state experiments is 30. This constitutes six steady

- state modes of engine operation (6), 10 r epeats (10), one exhaust configuration, one engine (1), and three different PEMS units (3), 6x10x1x1x3= 180.

TABLE 4. ENGINE, EXHAUST CONF IGURA TION, AND STEADY

- STATE MODES

No. of Steady

- State Modes for Bypass Setting 1 (BSPM and PM Concentration, representative of PM threshold of 0.025 g/hp- hr under NTE Transient Operation) PM

- PEMS Units Number of Repeats 07 Engine 1 SS1, SS2, SS3, SS4, SS5, SS6 Three Sets of (MSS, TRPM, and PPMD) 10 per Mode per PM

PEMS Set

Determine t he 

# ( ) g SS PM mol Δ m surface plots for the error model based upon a ll data pooled. Note

that e ach br and of P EMS w ill ha ve i ts ow n SS PM Δ m error s urface generated for us e i n bot h calculation methods 1 and 2. For calculation method 3, t he AVL brand PM PEMS will have a unique SS PM Δ m calculated according to Figure 4 of this test plan.

Recommend six steady

- state points based on the PM measurement, using the AVL MSS, of 80 SS points of the Cummins cycle that is typically used to generate ECM torque and BSFC errors versus laboratory. The MASC will accept the six steady

- state points or choose alternative points for each exhaust configuration. The objective for the MASC will be to select steady

- state points within a given exhaust configuration that provides a nominal spread of concentrations within that configuration’s target brake

- specific levels. Note that to achieve the brake

- specific targets under steady

- state conditions, the bypass might have to be opened further, relative to the transient NTE bypass settings.REPORT 03 .14936. 12 A-24 3.2.2 Background

Testing will be conducted to capture bias and precision errors in PEMS’ emissions instruments versus the laboratory filter-based method. The tests will be steady- state only.

Note: S ection 3.3 ( next s ection) w ill e valuate pr ecision e rrors ( not bi as) due to the d ynamic response of the PEMS instrumentation. The precision error captured during steady state testing (section 3.2) will have to be subtracted from the overall precision error captured in section 3.3 in order not t o doubl e

- count t he s teady s tate precision e rrors of P EMS i nstrumentation. T his process is detailed in Section 3.3.

3.2.3 Methods and Materials

a) One model year 2007 heavy duty diesel engines, equipped with a DPF in the exhaust (Mack MP9) Use the following systems:

b) Nine PM PEMS (3 Sensors PPM D, 3 AVL MSS, 3 Horiba TRPM) c) One PEMS exhaust flow-meter from Sensors, Inc., and one and from Horiba, applicable to the engine to be tested d) DPF with Bypass Setting 1 for SS testing, representing a threshold level of about 0.025 g/hphr under NTE transient testing

e) Measure PM via the CVS, Part 1065 Lab Method (most recent publication) Use the following overall guidelines:

f) Measure engine inlet airflow through use of LFE or equivalent g) Use a series of six steady

- state modes, and set each mode time to collect a CVS filter mass of at least 75 microgram per mode, simultaneously with other PM

- PEMS h) Regenerate DPF system prior to each series of steady

- state tests i) Capture ECM broadcast channels and other common diagnostic channels, as recommended by engine manufacture r(s), to ensure proper engine operation j) Do not measure gaseous species by the PEMS k) Stabilization time = 180 seconds, with a different running time per mode to achieve a 75 microgram or higher of PM on the CVS filter l) Always power off PEMS equipment at end of each day, according to PEMS manufacturer instructions. Re- start start -up process every day according to PEMS manufacturer instructions and Part 1065, Subpart J. m) Whenever PEMS are exchanged, swap the order of the Horiba and Sensors flowmeters, if the
ste up allows for it.

6 point steady- state repeat

- testing, evaluate bias and precision errors:

a) The MASC will select 6 SS operating conditions for repeat testing from a matrix of 80 SS points containing information on PM emissions using the AVL MSS b) Randomize the order of the six modes c) Repeat each six steady

- state cycle two or three times, prior to DPF regeneration d) Each test will use three PEMS (Sensors, AVL, and Horiba) at a time, to measure PM emissions concentration and exhaust flow rate. e) Expected test duration is 5 days per PEMS set, with a total of 15 days for all three sets.REPORT 03 .14936. 12 A-25 Bypass Setting:

a) Run NTE transient cycle using the CVS filter -based method b) Set bypass to produce CVS filter

- based average brake -specific of about 0.025 g/hp- hr c) Determine the average PM mass concentration d) Run t he 80 S S C ummins c ycle t o c apture P M c oncentration a t e ach m ode us ing t he A VL MSS e) Check the PM concentration levels and select the six

- steady state modes from the 80 poi nt matrix. As a first order, check the concent ration at the pre

- selected steady

- state modes to see if they spread within reason around the concentration produced for the NTE transient cycle. If not, adjust the bypass as needed to establish the right spread in brake

- specific emissions and concentration for the six steady

- state modes f) Make s ure t hat t he poi nts s elected spread around a br ake s pecific l evel and concentration level of a threshold of 0.025 g/hp

- hr, and concentration range of 4 to 15 milligram per cubic
meter.

3.2.4 Data Analysis

Use the acquired data to create the “error surfaces” to be used by the Monte Carlo simulation. An example of the steady -state error surface determination is shown in Table 5 for PM.

TABLE 5. EXAMPLE OF SS ERROR SURFACE

Error Surface for SS PM Concentration Figure 5

x

- axis PM μg/mole (lab mean at setpoint)
y

- axis PM μg/mole (PEMS) Figure 6

x

- axis PM μg/mole (lab mean at setpoint)
y

- axis 5th percentile 5th [PM μg/mole (PEMS)

- PM μg/mole (lab)]
y

- axis 50th percentile 50th [PM μg/mole (PEMS)

- PM μg/mole (lab)]
y

- axis 95th percentile 95th [PM μg/mole (PEMS)

- PM μg/mole (lab)] The 5th, 50th and 95th percentiles from the (PEMS

- lab) delta data will be used to estimate the 1st and 99th percentiles from assumed Gaussian distributions. Figure 7

x

- axis PM μg /mole (lab mean at setpoint)
y

- axis ic_SS_PM

z

- axis = ΔSS_PM_μg/mole 1st Percentile from Gaussian distribution based on 5th and 50th [PM μg/mole (PEMS) - PM μg/mole (lab)] deltas. 99th Percentile from Gaussian distribution based on 50th and 95th [PM μg /mole (PEMS) - PM μg/mole (lab)] deltas. 50th Percentile based on [PM μg/mole (PEMS) - PM μg/mole (lab)] deltas. ic sample frequency once per NTE event trial ic sample distribution Gaussian (normal distribution)REPORT 03 .14936. 12 A-26 3.3 PRECISION ERRORS UND ER T RANSIENT E NGINE OPERATION (DYNAMIC RESPONSE)

3.3.1 Objective

The objective of this portion of the work is to determine the precision error, TR PM Δ m **,** with each
PM

- PEMS under NTE transient engine operation. This will be achieved by creating a 20 to 25

- minute transient NTE cycle where the PEMS measure in each NTE.

3.3.2 Background

PEMS are expected to operate in a repeatable manner over NTE events as short as 30 s econds. Two sources of PEMS precision error are hypothesized: 1) dynamic response to rapidly changing signals, and 2) susceptibility to “history” effects. Dynamic response error includes error due to measurement signal time alignment, and the dissimilarity of the dynamic response and aliasing of s ignals; i ncluding t hose s ignals us ed t o d etermine e ntry i nto a nd exit f rom t he NTE z one. History effects i nclude the e ffects of p reviously measured qu antities on c urrently m easured quantities. F or e xampl e, this m ay be c aused b y i neffective s ample ex change i n the P M emissions sampling volumes, or it may be caused by one or more sensors’ characteristic rise time or fall time. To account for any dynamic response precision error, the increase in precision er ror incremental t o the s teady

- state e missions me asurement pr ecision will be inc orporated into the overall error model.

Selection of short NTE cycles (each 32 seconds) maximizes the sensitivity of this test to effects of dynamic response. Thirty

- two secon ds was chosen as the minimum instead of thirty seconds, which is the shortest NTE event time, to ensure that 1 H z ECM updating of torque and speed values w ould be unl ikely to i nterfere with c apturing N TE e vents. For each r epeat of t he t est cycle, the order of the 30 different NTE events will be the same. In addition the 29 di fferent intervals separating each NTE event from the next will have a r ange of durations and these will be randomly arranged in each test cycle as well. Fixed arrangement of the NTE e vents and the inter

- NTE events w ill maximize the s ensitivity of thi s te st to dynamic r esponse a nd history effects, and make the DPF and bypass operation very consistent.

The total length of the NTE transient cycle will assume that only 5 quartz crystal of the Sensors PPMD are working, and it takes five minutes of stabilization time for reusing a crystal after PM collection. Thus, the same NTE transient cycle used in the gaseous PEMS program will be used here, except for changes in the inter

- NTE times to ac commodate the Sensors PPMD.

3.3.3 Methods and Materials

a. Use a transient engine dynamometer emissions laboratory. b. Use a l aboratory t hat can accommodate at l east three P EMS, their pow er s upplies, the PEMS flow meters, cables and lines.

c. Use s ame over all guideline s de scribed i n s ection 3.2, but applied t o t ransient e ngine testing. d. Record the EEPS’ total mass signal during transient testing.REPORT 03 .14936. 12 A-27 Challenge PEMS to 30 different 32

- second NTE events, shown in Table 5, over about 23 minute test c ycle, or w hatever needed t o a ccommodate t he ne ed f or f ive c rystals of t he P PMD t o be operational. Randomize the NTE events shown in Table 6 once, scale up every fifth inter

- NTE time, shown in Table 7, to accommodate the PPMD, and use the same order for repeat testing. Repeat the test cycle 10 times for each set of three PEMS. N ote that for any torque command that is less than zero, command closed throttle (i.e. zero or minimum fuel command), and motor the engine at the commanded speed for that data point. An example of an NTE tr ansient cycle is shown in Figure 9.

Based on 10 repeats with each set of P EMS, the t otal num ber o f r epeats w ill be 30 cycles, assuming 1 NTE cycle x 10 repeats x one exhaust configuration x 3 sets of PEMS x one engine (1x10x1x3x1 = 30). Assuming a 25 minutes of NTE with 30 minutes of forced regeneration and preparation for the second repeat, the total number of days for NTE transient testing is 10 days (8 hours per day). This time includes PEMS and engine setup, PEMS warm up, and daily checks.

Prior t o ex ecuting t he first repeat, setup each PEMS and stabilize engine ope ration at t he first inter

- NTE ope rating poi nt. S etup t he P EMS a ccording t o 40 C FR P art 1065 a nd P EMS manufacturer instructions, including any warm- up time, zero -spans of the analyzers and t he setup of all accessories including flow meters, ECM interpreters, etc. Then, when the test cycle starts, switch the PEMS’ to sample emissions from the engine. When the text cycle ends, switch the PEMS back to ambient sampling. Complete all post

- test lab and PEMS validations according to 40 CFR Part 1065 and according to PEMS manufacturer instructions.

3.3.4 Data Analysis

Discard from further data analysis an y NTE events invalidated by an y criteria in 40 CFR Part 1065 Subpart J. For each NTE i event (i =1 to 30), which was repeated 30 times per engine with a specific e xhaust c onfiguration ( j = 1 t o 30) , calculate t he t ransient m edian a bsolute de viation, MAD TRi PM m , for , where for each NTE i event, MAD TRi = median[| NTE ij – median (N TE ij ) |]. Next cal culate t he di fference of M AD b y s ubtracting a cor responding s teady

- state M AD, MAD SSi PM m for . MAD TRi

- SSi = MAD TRi – MAD SSi . To determine a corresponding MAD SSi , calculate t he P EMS M AD SS at each steady

- state me dian lab va lue, a nd t hen us e t he m edian PEMS N TE i value a long t he m edian l ab va lue’s a xis t o f ind M AD SSi for t he c orresponding MAD TRi . D o not e xtrapolate a ny M AD SSi beyond the mini mum or ma ximum me dian lab values. Note that some MAD SSi values might be zero because the lab data for that median failed the F -test in the previous section.

For any MAD TRi

- SSi less than zero, set that MAD TRi

- SSi equal to zero.

Create a transient error surfaces using all of the MAD TRi

- SSi . Be sure to include any MAD TRi

- SSi data points that are equal to zero because they will affect the 1 st and 99 th percentile values.REPORT 03 .14936. 12 A-28 TABLE 6 . NTE TRANSIENT CYCL E

NTE Event 1 Speed % Range 2 Description Torque % Range NTE1 17% 3 Steady speed and torque; lower left of NTE 32% NTE2 59% 3 Steady speed and torque; lower center of NTE 32% NTE3 Governor line 3 Steady speed and torque; lower right of NTE 32% NTE4 17% 66% Steady speed and torque; middle left of NTE NTE5 59% 66% Steady speed and torque; middle center of NTE NTE6 Governor line 66% Steady speed and torque; middle right of NTE NTE7 17% 100% Steady speed and torque; upper left of NTE NTE8 59% 100% Steady speed and torque; upper center of NTE NTE9 100% 100% Steady speed and torque; upper right of NTE NTE10 Lower third 3 Highly transient torque; moderate transient speed 32%

- 100% NTE11 Upper third 3 Highly transient torque; moderate transient speed 32%

- 100% NTE12 Middle third 3 Highly transient torque; moderate transient speed 32%

- 100% NTE13 17%

- governed Lower third Highly transient speed; moderate transient torque NTE14 17%

- governed Upper third Highly transient speed; moderate transient torque NTE15 17%

- governed Middle third Highly transient speed; moderate transient torque NTE16 Lower right diagonal Transient; speed increases as torque increases NTE17 Upper left diagonal Transient; speed increases as torque increases NTE18 Full diagonal; lower left to upper right Transient; speed increases as torque increases NTE19 Lower left diagonal Transient; speed decreases as torque increases NTE20 Upper right diagonal Transient; speed decreases as torque increases NTE21 Full diagonal; lower right to upper left Transient; speed decreases as torque increases NTE22 Third light

— heavy

- duty NTE event from International, Inc. data set Sample from LHDE

NTE23 Cruise; ~ 50 mph Sample from HDDE NTE24 Cruise; ~ 75 mph Sample from HDDE NTE25 Small bulldozer Sample from NRDE NTE26 Large bulldozer Sample from NRDE NTE27 Second of three NTE events in FTP Seconds used from FTP: 714

- 725, 729

- 743, 751

- 755 NTE28 Third light

— heavy

- duty NTE event from International, Inc. data set Sample from LHDE

NTE29 First of two NTE events in NRTC Seconds used from NRTC: 423

- 430, 444, 448

- 450, 462

- 481, increased 464 speed from 40% to 42% NTE30 First of two NTE events in NRTC Seconds used from NRTC: 627

- 629, 657

- 664, 685-696, 714-722 1 Speed (rpm) = Curb Idle + (Speed % * (MTS

- Curb Idle) 2 Torque (lbf-ft) = Torque % * Maximum Torque At Speed (i.e. lug curve torque at spe ed) 3 Torque (lbf-ft) = Maximum of (32 % * peak torque) and the torque at speed that produces (32 % * peak power)REPORT 03 .14936. 12 A-29 TABLE 7 . DYNAMIC RESPONSE I NTER

- NTE EVENTS

INT Event Duration (s) 1 Frequency Description INT1 10 1 Initiation of cycle; INT1 is always first INT2

- 6 2 5 Shortest and most frequent inter

- NTE events INT7

- 10 3 4 Short and frequent inter

- NTE events INT11

- 14 4 4 Short and frequent inter

- NTE events INT15

- 18 5 4 Short and frequent inter

- NTE events INT19

- 21 6 3 Short and frequent inter

- NTE events INT22 7 1 Medium inter

- NTE event INT23 8 1 Medium inter

- NTE event INT24 9 1 Medium inter

- NTE event INT25 11 1 Medium inter

- NTE event INT26 13 1 Long inter

- NTE event INT27 17 1 Long inter

- NTE event INT28 22 1 Long inter

- NTE event INT29 27 1 Long inter

- NTE event INT30 35 1 Longest inter

- NTE event INT31 5 1 Termination of cycle; INT31* is always last Interval speeds and torques are not identical, but they are clustered around zero torque and the speed at which 15% of peak power and 15% of pe ak torque are output.

FIGURE 9 . EXAMPLE OF A NTE CYCLE Torque-Speed Domain

-100 0 100 200 300 400 500 600 700

0 500 1000 1500 2000 2500 3000 3500 4000 4500

Speed (rpm)Torque (lbf-ft) Lug Curve NTE S line NTE T-S line 1 SS Lower Left 2 SS Lower Mid 3 SS Lower Right 4 SS Mid Left 5 SS Mid Mid 6 SS Mid Right 7 SS Upper Left 8 SS Upper Mid 9 SS Upper Right NTE Cycle 22 LD NTE3 23 OH 50mph Cruise 24 OH 70mph Cruise 25 NR Small Dozer 26 NR Large Dozer 27 FTP NTE2 28 LD NTE6 29 NRTC NTE1 30 NRTC NTE2REPORT 03 .14936. 12 A-30 3.4 ECM TORQUE AND BSFC

3.4.1 Objective

Compare t he E CM

- based t orque a nd f uel r ate w ith t hat of t he l aboratory

- based measurement using the Cummins 80 SS mode cycles. For the laboratory purposes, use the gas

- based fuel flow values instead of the measured fuel flow. Repeat the Cummins 80 SS cycle three times, and use the average values produced.

Use at least six engines for these experiments that include the one engine to be used in the PM PEMS program and Engine B, C, and D of the ACES program.

3.4.2 Data Analysis

Use the acquired data pooled and normalized to % of max torque and % of maximum fuel rate to replace t he m anufacturer s ubmitted e rror s urfaces t hat w ere pr eviously us ed i n t he g aseous portion of the Monte Carlo simulation. Refer to section 2.4 f or description and example of an error surface. Include any bias error, unless there is an assignable cause that would not occur in

use and the steering committee approves to eliminate such bias error.

4 ENVIRONMENTAL CHAMBE R

The envi ronmental cha mber t ests cha llen ge P EMS t o a va riety o f e nvironmental di sturbances, namely electromagnetic interference, atmospheric pressure, ambient temperature and humidity, and shock and vibration.

During e ach of t he t ests, pl us a ba seline t est, the P EMS w ill c ycle t hrough s ampling f our different di lution pr eparations of a erosol pa rticles t hat c ontain vol atile h ydrocarbon a nd elemental c arbon using a pa rticle generator tha t mimic s the f ormation of di esel pa rticles. The OC/EC w ill be us ed to determine t he conc entration levels ne eded for t he P M g enerator. Essentially, after de termining the s teady

- state p oints t o r un on t he e ngine, t he O C/EC s emi

continuous ins trument w ill be us ed along w ith the f ilter

- based m ethod. T hen, f or t he concentration levels to be used with the PM generator, t he OC/EC instrument will be used to set the PM generator to produce the desired composition and concentration levels, similar to those encountered under steady

- state. T hree particle concentration levels of 5, 10, a nd 15 m g/m 3 , as shown in Table 8, will be generated by the particle generator. Each concentration will be fed to the PEMS after applying dilution ratios of 6, 12, 20, and 30. For each concentration and dilution ratio combination, the PM generator will be stabilized for 4.5 minutes, and data will be collected by the PEMS for 30 seconds. The test will continue for a period of 8 hours. The first six cycles of every test will serve to be the baseline before any environmental change is made.

The t emperature/humidity and pr essure t ests a re de signed t o mimic r eal

- world e nvironmental disturbances w ith t he m agnitude a nd f requency of t he di sturbances a djusted t o r eal

- world conditions. Randomly sample a uniform distribution of probability for their i c. , from any minute of the test. By randomly sampling from the minutes of these tests the magnitude and frequency of the real

- world error will be built into the error model, which is described in Section 2. T he other e nvironmental t ests r epresent t he f ull r ange o f pos sible c onditions. F or t hese t ests, randomly sample the normal distribution in Figure 1 for their i c .REPORT 03 .14936. 12 A-31 TABLE 8 . CONCENTRATION AND DILUTION RATIO SCHED ULE WITH PM GENERATOR

Raw PM Concentration,
μ g/m3 Dilution Ratio
DR1 DR2 DR3 DR4

6 12 20 30 Concentration at Above Dilution Ratio 5000  833.3 416.7 250.0 166.7 10000 1666.7 833.3 500.0 333.3 15000 2500.0 1250.0 750.0 500.0

For E MI/RFI and vi bration, t he i nstruments w ill be s ubjected t o s creening t ests w ith H EPA filtered air t o detect i f t here an y cha nges i n the r esponse of t he i nstruments. Based on these results, the MASC will decide if the particle generator will need to be used with these tests.

For the vibration screening test, in order to avoid damage to the instruments, a frequency sweep will be us ed at low a mplitude. The ide a he re i s to detect th e f requency that ma y tr igger a response by the instrument, without doing any damage due to high amplitude.

4.1 DATA ANALYSIS FOR EN VIRONMENTAL TESTS

Reduce d ata b y first c alculating means for each 30

- second period of stabilized measurements. Subtract f rom each mean the respective ba seline con centration. T he r esults ar e errors or “deltas”. C orrect e ach of t hese er ror di stributions b y r emoving their r espective b aseline variances, which were determined b y quantifying PM Generator output with no e nvironmental perturbations. C alculate t he va riance of each of t he di stributions. S ub tract t he respective baseline variance from each calculated variance. Use the resulting difference in variance as the target va riance f or a djusting t he e rror di stributions. If t he t arget va riance i s z ero or ne gative, leave all error values of the distrib ution as is and do not proceed to the next step. If the target variance is positive, iteratively solve to find a single numerical value that can be used to divide each error in a given distribution such that the resulting distribution has a variance equal to the target variance. Now each of the errors is corrected for baseline variance.

Then, calculate the NTE result with all errors, including torque and flow errors set to zero. This is the true value. Then subtract the true NTE value from the result with all errors and record this difference i n one of t he 7 m easurement a llowance di stributions: PM m times t hree cal culation methods (torque

- speed, fuel

- specific * BSFC, ECM fuel flow) times three PEMS manufacturers, except Sensors and Horiba can not use the ECM fuel flow calculation method. Then proceed to the next NTE event in the nominal data set. R epeat the entire nominal data set over and over until all 7 measurement allowance distributions converge. Follow the data reduction steps set out in Section 2 to select the final measurement allowance.REPORT 03 .14936. 12 A-32 4.2 PM GENERATOR COMMISS IONING

The P M g enerator i s de veloped by E PA. The P M g enerator can create various h ydrocarbon mixtures along with solid particle generation using carbon rods arcing. T he PM generator is also equipped with a micro

- proportional diluter, and is intended to simulate diesel exhaust particle phase compounds.

EPA will ship the PM generator to SwRI. EPA (Matt Spears) will train SwRI staff on using it. In addition, SwRI togeth er with EPA may incorporate to it a soot particle generation mechanism that i s di fferent t han t he c arbon r od a rcing, using i nstead a pr opane f lame m ini

- CAST technology.

The PM generator will be used during atmospheric chamber testing, temperature and hum idity testing, and may be used during EMI/RFI and vibration experiments.

4.3 BASELINE

4.3.1 Objective

The ba seline variance will be e stablished us ing a n 8 hour ba seline t est i n w hich t he PM generator cycles through the same compositions and concentrations of PM used during the actual environmental tests. M ean values will be determined from the first five cycles through the PM concentrations. D eviations ( deltas) from t hese mean va lues dur ing s ubsequent c ycles t hrough the c oncentrations w ill be us ed t o d etermine the ba seline va riance. This va riance w ill be subtracted from the environmental test results.

4.3.2 Background

All of the ot her environmental te sts inhe rently inc orporate the ba seline bi as v ariance o f the PEMS. B ecause t he M onte C arlo s imulation m odel a dds all the e rrors de termined from the various environmental tests, it would add the baseline variance of PEMS to the model too many times. In order to compensate for this in the model, the baseline variance of PEMS is determined and subtracted from each of the environmental tests’ results.

Note that the baseline variance of PEMS is measured and modeled (i.e. added) once as part of the steady

- state engine dynamometer laboratory experiment.

4.3.3 Methods and Materials

For thi s e xperiment us e a w ell ve ntilated EMI/RFI s hielded r oom c apable of m aintaining reasonably constant temperature and pressure. Use a room that can house one of each PEMS, their power supplies, the PEMS flow meters, cables and lines.

Prior t o executing t he b aseline t est, s etup each PEMS and stabilize the P EMS in the r oom. Perform P EMS s etup a ccording t o 40 C FR P art 1065 S ubpart J a nd P EMS m anufacturer instructions, including any warm

- up time, and audit. Then supply the PEMS’ sample ports with the sequence of PM from the PM generator as des cribed at the beginning of Section 4.REPORT 03 .14936. 12 A-33 At each PM concentration, flow PM long enough so that stable readings of the PEMS can be recorded. W hen t he O C/EC a nalyzer i s us ed t o s pot

- check t he out put of t he P M g enerator, ensure that enough time has elapsed to achieve an accurate OC/EC analysis.

Position PEMS and configure PM transport tubing to minimize transport delays and PM losses.

Test at least one PEMS from each PEMS manufacturer.

4.3.4 Data Analysis

Reduce t he b aseline d ata f or each P M P EMS, using artif icial N TE s ampling e vent time s. Subtract from each PM m the mean PM m from the initial (short) baseline test of six cycles through the PM concentrations, which were conducted at the beginning of the test. The re sults are errors or “deltas”. Calculate the variance of these values, and use them for baseline variance correction in the data reduction of the remaining environmental tests.

4.4 ELECTROMAGNETIC RADI ATION

4.4.1 Objective

Evaluate t he ef fect of Electromagnetic Interference (EMI) and Radio frequency Interference (RFI) on t he performance of the PEMS and determine error factors for the PEMS due to these effects. First, a screening test on each instrument will be performed with HEPA filtered air to determine if the EMI/RFI affects the instrument response. If it doe s, the MASC will decide on the test matrix required for this evaluation.

4.4.2 Methods and Materials

Use an EMI test facility capable of running the SAE tests listed abo ve. This would include: Signal generators, Power amplifiers, Transmit antennas, Electric Field Sensors, Measurement Receiver, Data recording device, LISNs (Line Impedance Stabilization Networks) and shielded enclosure.

4.5 ATMOSPHERIC PRESSURE

4.5.1 Objective

Ev aluate the effects of ambient pressure on PEMS PM concentration outputs.

4.5.2 Background

PEMS are expected to operate over ranges of ambient pressures. It is hypothesized that some of the errors of the PEMS concentration outputs may be a function of ambient pressure. Therefore, this experiment will change the ambient pressure surrounding PEMS to evaluate its effects on PEMS m easured c oncentrations a nd f low m eter t ransducer out puts. A s w ith a ll of t he environmental tests, the test c ycle f or thi s test is ba sed on t he be st

- known di stribution of r eal world conditions. For this test, the test cycle pressure distribution was matched to the county- by

county annual average atmospheric pr essure di stribution i n E PA’s 2002 N ational E missionsREPORT 03 .14936. 12 A-34 Inventory (NEI) model. Figure 10 depicts the NEI data distribution (based on 3149 data points) and the test cycle pressure distribution.

FIGURE 10. PRESSURE HISTOGRAM

4.5.3 Methods and Materials

Use a barometric chamber that can be well ventilated and capable of controlling a wide range of pressure changes (82.74 to 101.87 kPa). Use a chamber that can house at least three PEMS at a time, one of each PEMS manufacturer, their power supplies, the PE MS flow meters, cables and lines, plus the PM generator.

Follow a pattern of first soaking the PEMS at a constant pressure, then ramp the pressure to a new pressure, soak the PEMS at that new pressure, and then ramp to another pressure. Use the
sequence of pressures and times, as shown in Table 9 along with Figure 11 , to simulate a typical distribution of real

- world pressures and changes in pressure, which are believed to be dominated by changes in altitude during driving in the United States. Pressure Histograms

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

84.1 86.8 89.6 92.3 95.0 97.8 100.5 Pressure (kPa)Frequency (% of total time) 2002 NEI Pressure Distribution <5500 ft

Test Cycle PressureREPORT 03 .14936. 12 A-35 TABLE 9. ATMOSPHERIC PRESSURE TEST SEQUENCE

Atmospheric Pressure Test Sequence

Phase Pressure Time Rate Comments kPa Alt. ft. min ft/min 1 Soak 101 89 10 0 Flat near sea

- level 2 Ramp 101

- 97 89

- 1203 20 56 Moderate hill climb from sea level 3 Soak 97 1203 20 0 Flat at moderate elevation
4 Ramp 97

- 101.87 1203

- - 148 60

- 23 Moderate descent to below sea level 5 Soak 101.87

- 148 20 0 Flat at extreme low elevation
6 Ramp 101.87

- 101

- 148

- 89 20 12 Moderate hill climb to near sea level 7 Soak 101 89 20 0 Flat near sea level 8 Ramp 101

- 97 89

- 1203 20 56 Moderate hill climb from sea level 9 Soak 97 1203 25 0 Flat at moderate elevation
10 Ramp 97

- 96.6 1203

- 1316 20 6 Slow climb from moderate elevation 11 Soak 96.6 1316 20 0 Flat at moderate elevation 12 Ramp 96.6

- 82.74 1316

- 5501 20 209 Rapid climb to NTE limit 13 Soak 82.74 5501 20 0 Flat at NTE limit 14 Ramp 82.74

- 96.8 5501

- 1259 30

- 141 Rapid descent from NTE limit 15 Soak 96.8 1259 20 0 Flat at moderate elevation 16 Ramp 96.8

- 90 1259

- 3244 15 132 Rapid hill climb to mid elevation 17 Soak 90 3244 10 0 Flat at mid elevation
18 Ramp 90

- 96.8 3244

- 1259 20

- 99 Rapid descent within middle of NTE 19 Soak 96.8 1259 20 0 Flat at moderate elevation
20 Ramp 96.8

- 99.2 1259

- 586 20

- 34 Moderate descent to lower elevation 21 Soak 99.2 586 20 0 Flat at lower elevation 22 Ramp 99.2

- 101 586

- 89 10

- 50 Moderate decent to near sea

- level 23 Soak 101 89 20 0 Flat near sea

- level

FIGURE 11 . PRESSURE

- TIME ENVIRONMENTAL TEST CYCLE Pressure-Time Environmental Test Cycle

-9000 -7000 -5000 -3000 -1000 1000 3000 5000 7000 9000

0 1 2 3 4 5 6 7 8 Time (hr)Altitude (ft) & dA/dt (ft/hr) 71
75
79
83
87
91
95
99
103
107
111
115
119
123
127
131Approximate Pressure (kPa)

______ Altitude (ft)

_ _ _ _ 1/2-hr moving avg dA/dt (ft/hr)

______ Vertical gridlines = hours

______ Vertical gridlines = 7-min gas cylinder cycle timesREPORT 03 .14936. 12 A-36 Prior t o e xecuting t his pressure s equence, s etup e ach P EMS a nd s tabilize t he P EMS i n t he chamber’s first pressure. P erform PEMS setup according to 40 C FR Part 1065 S ubpart J and PEMS manufacturer instructions, including any warm

- up time, zero

- span -audits of the analyzers and the setup of all accessories including flow meters, ECM interpreters, etc. T hen supply the PM P EMS’ s ample por t w ith the s equence o f P M f rom t he P M g enerator as de scribed at t he beginning of Section 4.

Flow each generated PM sample long enough so that at least 30 s econds of stable readings are recorded for the slowest responding gas concentration output of all the PEMS. P osition PEMS and c onfigure gas t ransport t ubing t o m inimize t ransport d elays. T arget t o s ample about 30 seconds . Repeat this cycle over the 8

- hr test cycle, by cycling through the concentration shown in T able 8, w hich r epresents one hour of t esting, us ing a 4.5 m inutes of s tabilization a nd 30 seconds of sampling at each condition.

Perform this test once for one s et of PEMS with as many PEMS tested at once.

4.5.4 Data Analysis

Perform data analysis according to Section 4.1.

4.6 AMBIENT TEMPERATURE AND HUMIDITY

4.6.1 Objective

Evaluate the effects of ambient temperature and humidity on P EMS PM concentration outputs. The histogram i n F igure 12, a long w ith T able 10 a nd F igure 13, w ill be upda ted b y a ne w temperature profile that takes into consideration the data generated by CE

- CERT.

4.6.2 Background

PEMS ar e ex pected t o operate ove r a wide r ange of changing ambient t emperature s. It i s hypothesized t hat s ome of t he e rrors of t he P EMS out puts may be a f unction of c hanges i n ambient te mperature. T herefore, this experiment w ill c hange the a mbient te mperature surrounding P EMS t o e valuate i ts e ffects on P EMS m easured c oncentrations and flow m eter transducer outputs. As with all of the environmental tests, the test cycle for this test is based on the be st

- known di stribution of r eal w orld c onditions. F or t his t est, t he t est c ycle t emperature distribution was matched to the hour

- by h our county

- by

- county average atmospheric temperature distribution, weighted by vehicle mile s tr aveled a ccording to EPA’s 20 02 National E missions Inventory ( NEI) m odel. F igure 12 d epicts t he NEI da ta di stribution ( based on ov er 900,000 temperatures and ove r 270 trillion vehicle miles) and the test cycle temperature distribution.REPORT 03 .14936. 12 A-37 FIGURE 12 . TEMPERATURE HISTOG RAM

4.6.3 Methods and Materials

Use a w ell ventilated room capable of controlling a w ide range of temperature changes (

- 23 to 100°F). U se a room t hat c an hous e at l east s ix P EMS, t heir pow er s upplies, t he P EMS f low meters, c ables and l ines, pl us s even di fferent z ero, a udit, and s pan gas c ylinders, and a gas switching system.

Follow a pa ttern of f irst s oaking t he P EMS a t a c onstant r oom t emperature, t hen ramping th e room t emperature t o a n ew t emperature, soaking the P EMS at t hat new t emperature, and then ramping to another temperature. Use the following sequence of temperatures, shown in Table 10 and F igure 13 , and times t o simulate t he r ange of r eal

- worl d temperatures and changes i n temperature: Temperature Histograms

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

28.5 39.5 50.5 61.5 72.5 83.5 94.5 Temperature (F)Frequency (% of total time) Real World Temperature 23 -100 F

Test Cycle TemperatureREPORT 03 .14936. 12 A-38 TABLE 10 . AMBIENT TEMPERATUR E TEST SEQUENCE

Ambient Temperature Test Sequence

Phase Temperature Time Rate Comments °C °F min °C/min 1 Soak 13.89 57 10 0.00 Cool in

- garage pre

- test PEMS operations 2 Ramp 13.89

- 5.00 57

- 23 5

- 3.78 Leaving cool garage into cold ambient

3 Soak -5.00 23 5 0.00 Operating at cold temperature outside of vehicle 4 Ramp

- 5.00

- 12.78 23

- 55 145 0.12 Diurnal warming during cool day 5 Soak 12.78 55 40 0.00 Steady cool temperature during testing 6 Ramp 12.78

- 28.33 55

- 83 5 3.11 Return to hot garage on a cool day

7 Soak 28.33 83 52 0.00 Hot in

- garage pre

- post

- test PEMS operations 8 Ramp 28.33

- 37.78 83

- 100 5 1.89 Leaving ho garage into hot ambient

9 Soak 37.78 100 8 0.00 Operating at hot temperature outside of vehicle 10 Ramp 37.78

- 22.22 100

- 72 100

- 0.16 Diurnal cooling during hot day

11 Soak 22.22 72 60 0.00 Steady moderate temperature during testing 12 Ramp 22.22

- 13.89 72

- 57 5

- 1.67 Return to cool garage on a moderate day 13 Soak 13.89 57 40 0.00 Cool in

- garage post

- test PEMS operations

FIGURE 13 . TIME SERIES CHART OF AMBIENT TEMPERATURE TEST Temperature-Time Environmental Test Cycle

-60 -50 -40 -30 -20 -10

0 10 20 30 40 50

0 1 2 3 4 5 6 7 8 Time (hr)Temperature (C)

-76 -64 -52 -40 -28 -16 -4 8 20 32 44 56 68 80 92 104 116Temperature (F)

_____ Temperature

_ _ _ _ 1/2-hr moving average dT/dt (C/hr)

_____ Vertical gridlines = hours

_____ Vertical gridlines = 7-min gas cylinder cycle timesREPORT 03 .14936. 12 A-39 Prior to executing this temperature sequence, setup each PEMS and stabilize the PEMS in the chamber’s first temperature. Perform PEMS setup according to 40 CFR Part 1065 Subpart J and PEMS manufacturer instructions, including any warm

- up time, zero

- span -a udits of the analyzers and the setup of all accessories including flow meters, ECM interpreters, etc.

Run t he 8

- hour c ycle t est b y s tepping t hrough t he c oncentration a nd di lution r atio s hown i n Table 8.

4.6.4 Data Analysis

Perform data analysis according t o Section 4.1.

4.7 ORIENTATION AND VIBRATION

4.7.1 Objective

Evaluate the effect of vehicle vibration on the performance of the PEMS and determine error factors for the PEMS due to these effects. Prior to doing extensive vibration work, perform a screening using HEPA filtered air sampling at a sweep of different frequencies with low amplitude. If any of the PEMS shows a response to a particular frequency, propose a frequency test and submit it for the MASC for approval.

5 SWRI CVS AND CE

- CERT TRAILER CORRELA TION

Prior to performing the in

- use work with the PM

- PEMS, it is important to establish the degree of correlation between SwRI C VS

- based PM m easurement and CE

- CERT CV S

- based PM measurement. For t his pur pose, t he C E

- CERT tr ailer w ill move to SwRI f acilities a nd PM measurement will be conducted on the engine used for the PM-PEMS program.

Prior t o t he c orrelation w ith S wRI, t he C E

- CERT M EL w ill c onduct a 1065 a udit of t he P M measurement s ystem an d associated weighing c hambers and stations an d associated electron ic sensors a nd m onitors. T his a udit w ill i nclude ve rification of t he s econdary di lution f low a nd temperature c ontrollers. T he s ampling s ystem w ill be c hecked t o m ake s ure i t hol ds t he appropriate temperatures and within the appropriate limits. The filter h olders will be checked for compliance a nd t he l og books w ill be e xamined t o e nsure t hey a ppropriately m onitor a ll t he parameters ne eded f or 1 065 c ompliance. C E

- CERT w ill i dentify ar eas where t he cu rrent C E

CERT pr ocedures or equipment doe s not m eet the 106 5 r egulations a nd w ill upg rade t hese systems or procedures so that they are compliant with the 1065 regulations.

The C E

- CERT’s ME L w ill be c ross

- correlated with an engine cel l at S wRI us ing an engine selected by the SC. Testing will be conducted under the NTE transient cycle with the 0.025 g/hp

hr bypass setting as determined by the SC.

5.1 METHOD AND MATERIALS

Below is a list of a step by step approach for the correlation between SwRI and CE- CERTREPORT 03 .14936. 12 A-40 1. Perform a propane check on SwRI CVS and 47 mm filter and CE-CERT CVS and 47 mm filter. Both s ystems should pass Part 1065 on propane. However, even if they pass, note any difference between the two. 2. Set the CVS flow rate to be the same on both systems 3. Set the filter face temperature and velocity to be the same on both systems 4. Set the secondary dilution ratio to be the same on both systems. 5. Use W hatman P TFE m embrane f ilters ( 7592

- 104), a nd f ilter s creens t hat m eet t he latest Part 1065. 6. Modify the exhaust path to SwRI CVS to be comparable with that for the CE

- CERT Tr ailer 7. Since SwRI is using a test cell that may have had various PM levels, both SwRI and CE

- CERT s hould pr econdition on t he s ame e ngine. T hus, i t i s r ecommended t hat CE

- CERT c lean t heir t unnel pr ior t o t raveling t o S wRI s o t hat bot h c an be conditioned on a similar emissions level. 8. Pre

- condition t he S wRI C VS t unnel a nd t he C E

- CERT tr ailer C VS tu nnel f or a period of 10 hour s a t e ngine r ated pow er us ing e xhaust c onfiguration w ith D PF without bypass. The conditioning time may include active DPF regenerations. 9. R un a total of 12 repeats of the NTE transient cycle using DPF with Bypass Level at 0.025 g/hp

- hr emission level, over a period of three days. Four repeats per day with the CE

- CERT followed by four repeats with SwRI CVS and then alternate. Prior to each set of four repeats manually regenerate the DPF. 10. Use S wRI DM M

- 230 a nd C E

- CERT D MM

- 230 t o m ake s ure t hat t he e ngine P M source is not shifting and being consistent. 11. SwRI s hould h andle a nd w eigh a ll t he f ilters for bot h S wRI and C E

- CERT i n accordance with their protocol. 12. The CE

- CERT trailer is needed at SwRI for at least two weeks per engine. One week for setup and two weeks of testing assuming the above schedule. 13. In a s eparate t ask, EPA w ill equi librate and pre

- weigh 20 f ilters us ing E PA’s weighing protocol. EPA will then ship them to SwRI for repeat preweighing using their protocol. S wRI w ill then ship the s ame f ilters to EPA f or r eweighing. After reweighing at EPA, EPA will ship the filters to CE

- CERT for weighing using CE

CERT’s w eighing pr otocol. F inally, CE

- CERT w ill s hip the f ilters to E PA f or reweighing. R esults will be reported by EPA for MASC discussion. N o threshold for acceptance has been established at the time of this testplan writing. 14. The t arget f or c orrelation a t t he 0.025 g /hp

- hr le vel is C E

- CERT’s m ean of 12 repeats being within +/ -10% of the mean value reported by SwRI.

6 MODEL VALIDATION AND MEASUREMENT ALLOWANC E DETERMINATION

The pr e

- validated measurement allowance va lue f or bot h or at l east one P EMS w ill b e determined pr ior t o t he i n

- use model va lidation at C E

- CERT. If bot h P EMS s ystems ha ve determined reasonable measurment allowances, then the validation testing will be performed on the PEMS that shows the lowest measurement allowance.

The MA SC decided to validate only one of the comple te PM PEMS systems Horiba or Sensors where AVL will “piggy back” on either PEMS as part of the model validation. Thus the testing is a full set of tests where three model PEMS from one manufactures over three routes with one bypass setting and one vehicle will be tested. If the selected model PEMS does not validate theREPORT 03 .14936. 12 A-41 MA S C ha s t he opt ion t o t ry va lidating t he second m anufactures P EMS. T his a dditional validation i s not c overed i n t he C E

- CERT s cope of w ork a nd w ould r equire a budg et modification.

6.1 MODEL VALIDATION

6.1.1 Objective

The objective of the validation testing is to validate the Monte Carlo model by

1. Testing the PEMS in parallel with the CE

- CERT trailer 2. Checking the data to see if it fits the model predicted based on the SwRI laboratory efforts

For the mode l va lidation testing effort, CE

- CERT w ill c onduct pr eliminary pl anning for t he PEMS installation and commissioning. For each PEMS model tested, a total of 5 t est days are allocated for commissioning. Subsequent PEMS commissioning of like models should take less time and thus only 3 t est days are allocated. The PEMS commissioning will be performed with the as sistance of t he P EMS m anufacturer on s ite. CE

- CERT w ill procure W hatman filters for both CE- CERT & Horiba filter weighing process.

CE

- CERT will design, construction, and install a bypass. CE

- CERT will purchase the parts for a bypass. The bypass will be “tuned” to the BSPM level requested by the committee. Initially, it is planned to tune to 0.025 g/hp

- h at clean DPF condition, which could giv e a range of values from 0.01 to 0.04 g /hp

- h depending on i n

- use conditions and DPF regeneration status. The use of a PM PEMS may be incorporated into this part of tuning to provide instantaneous feedback on the PM level in addition PM filters will be used to determine actual level. This tuning data will be made av ailable t o the M A S C as addi tional P EMs

- MEL d eltas, but be denoted as p reliminary tuned data since values could exceed the desires of the MA program.

Long line lengths will be employed to ensure good mixing. CE

- CERT will use good engineering judgment to determine if good mixing is established. CE

- CERT will evaluate good mixing b y measuring the real time PM with the AVL PEMS while attempting to traverse the exhaust stack. Given the limitation to w ork around a vehicle during in

- use testing, traversing the exhaust may require s ome t ype of al ternative t est pr ocedure once t he b ypass i s f abricated. CE

- CERT w ill work with the MA SC to determine when a suitable well mixed bypass has been achieved.

The t est matrix and test costs depend on t he actual number of PEMS tested, number of bypass configurations, and the number of routes. For this scope of work three model PEMS from one manufactures over three routes with one bypass setting and one vehicle will be tested. This test matrix i s ba sed on t he recommendation of t he SC. I f a s econd m anufacture P EMS r equires testing then a new scope of work will be needed and a budget change.

The pr imary te sting w ill be f ocused on true N TE e vents if pr actically p ossible a nd/or f orced triggered events. The target level of ≥ 50 ug will be set for the filter measurements by the MEL. If H oriba P EMS i s c hosen t hen t he H oriba f ilter w ill be r eplaced one t ime f or e very 8 M EL filters to simulate an 8 hour operation for the Horiba filter. All filter weighing for both the CE

-REPORT 03 .14936. 12 A-42 CERT a nd Horiba filters ( if s elected) w ill be pe rformed by C E

- CERT us ing CE

- CERT’s weighing procedures.

For t he t est m atrix c hosen t hree m odels of one m anufacture P EMS w ill be t ested ove r t hree routes with one additional test day allocated, see Table below . One additional day is allocated for repeating a test route or for operating the PEMS in a “true” NTE mode or combination of both. The PEMS will be tested over each route/test

- bypass configuration for a total of 4 t est days. A total of 10 prep days are allocated for the preparation and installation of the first PEMS for each manufacturer, 5 test days are allocated for commissioning each PEMS, and 3 days are allocated for changing between PEMS of a single manufacturer. Subsequent PEMS commissioning of like models should also take less time and thus only 3 test days are allocated.

Table - Three Models of One PEMS Manufacture Test Matrix

Mfg Unit # Route Test Conditions Total test days PEMS plus AVL 1 Palm Springs San Diego Baker 1 bypass 0.025 g/hp-h 4

PEMS plus AVL 2 Palm Springs San Diego Baker 1 bypass 0.025 g/hp-h 4

PEMS plus AVL 3 Palm Springs San Diego Baker 1 bypass 0.025 g/hp-h 4

Truck rental for extended period of time for setup and PEMS installation is included under this task. This could include a Volvo because of parts availability or a different model for ease of bypass installation.

Data analysis with engines outside of the NTE requires additional data processing for Method 2. During Method 2 calculation there is a s ummation of the inverse of fuel rate. The fuel rate on some conditions outside the NTE can go to zero causing the calculation to go to infinity. In these situations it was decided by the MASC to freeze the bsFC to a consta nt value during out -of- NTE operation using the last valid BS FC NTE value. CE

- CERT will perform this bsFC freezing in the Method 2 summation during data post processing for both the PEMS and MEL. The logic to start and stop freezing will be determined by th e MASC a nd provided to CE

- CERT before processing Method 2 results.



# ∑ **∑**

** **  ** **  ** **  ** ** 

**× =** 
Wor k E CM Carbon g Method

fuel fuel 2REPORT 03 .14936. 12 A-43 6.1.1.1 CE

- CERT Validation

The difference between the PEMS results and the CE

- CERT trailer results will be compared to the error predicted by the Monte Carlo model. To validate the Monte Carlo model, data must be run t hrough t he m odel a nd t he m odel r esults must pr edict t he a ctual t est r esults w ithin a reasonable level of accuracy.

Validation will be based on the following procedure. For each reference NTE event, the Mo nte Carlo model will be used to generate the 5 th and 95 th percentiles of the simulated distribution of the brake

- specific PM emission differences. In order to obtain simulations representing similar conditions t o those obt ained on

- road, some e rror s urfaces may ne ed t o be s uppressed i n t he simulations since not all of them ma y be applicable to the on

- road conditions. T he choice of which error surfaces to suppress would need to be made by the Steering Committee.

Next, the 5 th and 95 th delta percentiles obt ained from the above simulations will be separately fit to a line or curve using two chosen methods: a linear regression procedure and a local regression (loess) technique 1 . Depending on which of the resulting two fits is best for each set of data (i.e., either for the 5 th percentile deltas or the 95 th percentile deltas), the resulting line or curve will be used as one of the lower or upper limits for the on- road data.

To determine the best fit for a given set of delta percentiles (i.e., 5 th or 95 th ), a s imple regression line initially will be fit to the data. If a least squares linear regression of the 5 th or 95 th percentile deltas versus the ideal PM emission has an r 2 >0.85 and an SEE < 5 % of the median ideal PM emissions, then the regression line will be used. If this set of criteria is not met, then a loess fit will be used. Since a loess regression requires the selection of a smoothing parameter 2 to smooth the data, the chosen smoothness parameter should balance the residual sum of squares against the smoothness of the fit.

The on

- road de lta e rrors, obt ained f rom t he r esults of c ollecting da ta on s everal N TE e vents during on

- road operations, will be plotted on a graph containing the 5 th and 95 th percentile delta limits determined from the regression fits chosen above. The graph will consist of a plot of delta PM versus ideal PM. The number of on

- road points outside these limits will be determined and expressed as a percentage of the total number on on

- road data points. If this number does not exceeds 10% of the total number of on

- road data, the simulation data will be considered to be valid.

6.2 MEASUREMENT ALLOWANC E DETERMINATION

6.2.1 Objective

Use t he M onte C arlo s imulation pr ogram de veloped w ith data from s ections 2, 3 a nd 5, and validated with section 5.1 to determine the measurement allowance for all regulated emissions, at 2007 emissions standards.REPORT 03 .14936. 12 A-44 6.2.2 Background

After the Monte Carlo model has been validated and confidence in its ability to p redict errors from PEMS instrumentation, the last step in this program will be to actually calculate a single set of measurement allowance for PM.

6.2.3 Methods and Materials

Using the criteria explained in section 2.2 c alculate the various levels of measureme nt accuracy corresponding t o t he t hree P EMS m anufacturers a nd t he b rake s pecific P M e missions calculations. U se a ll t he va rious e rror s urfaces developed dur ing t his t est pr ogram, i ncluding those provided by engine manufacturers to the EPA and ARB.

6.2.4 Data Analysis

Use the methodology explained in section 2.2, and Table 2.2 to arrive at the final measurement allowance.

7 TIME AND COST

7.1 TIMELINE

Table 11 is a te ntative time line pr ojecting th e ma jor ta sks to be a ccomplished during thi s program. The additional work if needed option is the work that may need to be done if the model did not validate. Otherwise, the final report will be submitted by September 30, 2009.

TABLE 11 . PROJECTED PM

- PEMS TIMELINE

7.2 COST

The rough estimated cost is shown in Table 12. Based on the current estimate, a $125,000 of the $200,000 is needed to complete the project.REPORT 03 .14936. 12 A-45 TABLE 12 . PROJECTED COST EST IMATE

PEMS Training, Setup , Audit, and Debug $660,000 Steady

- State and Transient Experiments $190,000.00 SwRI and CE

- CERT Correlation (1 engine) $75,000.00 PM Generator and Environmental Testing Activities $200,000.00 Modeling Activities (Including CO 2 $225,000.00 )
Data and analysis, reporting, and final report $150,000.00 Contingency if validation fails $100,000.00 General Contingency $200,000.00 Grand Total $1,800,000.00 Grand Total Without General Contingency $1,600,000.00 Grand Total without General Contingency and Contingency if Validation Fails $1,500,000.00

8 ABBREVIATIONS USED IN BRAKE SPECIFIC EQUATIONS

Method 1:

ePM = brake -specific emission, PM (g/hp- hr) N = total number (of time intervals) in series x = amount of substance fraction (mol PM/mol exhaust; note that 1 **μ** mol (emission constituent)/mol (exhaust) = 1ppm (part per million)

. n = amount of substance rate (mol/sec, in this case, mol (exhaust)/sec
t = time interval (sec) fn = rotational frequency (shaft), rev/min T = torque (N- m)

NOTE: The units of the numerator work out to gemission as is. However, using the units given for the denominator (RPM * N

- m * s ), you would still need to divide by 1.978 to get to hp -hr (using RPM * N-m = kW * 9550, 1 hour = 3600 sec, and kW = hp*0.7457)

Method 2:

ePM = brake -specific emission, PM (g/hp- hr) MNO2 = Molecular weight, NO2 (~46 g/mol) N = total number (of time intervals) in series x = amount of substance fraction (mol PM/mol exhaust; note that 1 **μ** mol (emission constituent)/mol (exhaust) = 1ppm (part per million)

~

. n = amount of substance rate (mol/sec, in this case, mol (exhaust)/sec) that is linearly
proportional to . n (Note: this is a proportional sample, which means that you may use a flow meter that has a span error, as long as its calibration is linear)
t = time interval (sec)REPORT 03 .14936. 12 A-46 MC = Atomic weight of carbon (~12 g/mol) wfuel = g (carbon)/g (fuel); Note fuel is roughly 86% carbon by mass xCproddry = amount of carbon products on a C1 basis per dry mol of measured flow (exhaust), mol/mol, solved iteratively per 1065.655 xH2O = amount of water in measured flow, mol/mol (see 1065.645 for calculations) efuel = brake -specific fuel consumption (g (fuel)/hp- hr)

Method 3:

ePM = brake

- specifi c emission, PM (g/hp- hr) MNO2 = Molecular weight, NO2 (~46 g/mol) wfuel = g (carbon)/g (fuel); Note fuel is roughly 86% carbon by mass MC = Atomic weight of carbon (~12 g/mol) N = total number (of time intervals) in series

x = amount of substance fraction (mol PM/mol exhaust; note that 1 **μ** mol (emission constituent)/mol (exhaust) = 1ppm (part per million)

fuel m . = mass rate of fuel (g/sec) xH2O = amount of water in measured flow, mol/mol (see 1065.645 for calculations) xCproddry = amount of carbon products on a C1 basis per dry mol of measured flow (exhaust), mol/mol
t = time interval (sec) fn = rotational frequency (shaft), rev/min T = torque (N- m)
t = time interval (sec)

NOTE: The units of the numerator work out to g emissio n as is. However, using the units given for the denominator (RPM * N

- m * s ), you would still need to divide by 1.978 to get to hp -hr (using RPM * N-m = kW * 9550, 1 hour = 3600 sec, and kW = hp*0.7457)REPORT 03 .14936. 12 

### APPENDIX B

STEERING COMMITTEE MEETING MINUTESREPORT 03 .14936. 12 B-1 PM Measurement Allowance Steering Committee Meeting Meeting at CE

- CERT April 23, 24, and 25, 2008

Future Meetings:

1. May 15 -16 meeting, SwRI 2. June 12 and 13 at for PM -PEMS and 10 and 11 for EMTC, ACS. 3. July 29 and 30, San Antonio, SwRI 4. August 28, 29, San Antonio, SwRI

On

- Road PM PEMS Discussion and Action Items

CO 2 Activities

1. Use CO 2 data pr ovided by C E

- CERT dur ing t he ga seous pr ogram f or CO 2 validation,

an d share the information during the May 15- 16 meeting at SwRI.

Test Plan Activities

a. Explain in test plan why Horiba can’t use Method 3 b. Replace BSFC i n t he e quation of M ethod 2 with ( fuel flow/torque *speed). Essentially, remove BSFC error surface

c. Show how (mp bar) for the AVL is calculated d. For AVL Method 3, use a different error surface for PM.

e. For reference NTEs, use the existing reference NTEs, and PM concentrations to produce different concentration from DPF out to threshold levels. f. Check section 2.3 with Bob Mason to make sure it is correct
g. Change t he example t o PM i n the t est pl an and give an appropriate P M concentration range per mole basis. h. Reexamine the text to explain the new equations better. No second by second for AVL i. Change Figure 8 i n t he test pl an t o r eflect t he f uel f low i nclusion, a nd r emove BSFC error. j. Change Figure 8 to update all errors that are required k. State t he t ime and d ate b y when t he m odel could be available t o t he group, assuming no last minute changes are required.

General Notes:

1. For AVL, use a 3 t o 1 dilution ratio and change the selectable range if needed but not the dilution ratio. 2. For the NTE windowing, use the EEPS to determine the windowing sensitivity, assuming a maximum of 6 seconds delay at the beginning and end. 3. M easure the CVS dilution air temperature as close as close as possible to the exhaust and dilution air mixing point. This may require insulation of SwRI CVS system. 4. Use Whatman Teflon membrane, 2 micrometer for the entire programREPORT 03 .14936. 12 B-2 5. Use the fine ambient backup screen instead of the coarse or diesel backup screen 6. List all the error surfaces in the test plan 7. Plan Training and Comissioning of the PM Generator 8. If 32 seconds NTE is not sufficient for the PM-PEMS, extend the length 9. Use 10 % difference as an acceptable difference between SwRI and CE

- CERT

1065 Audit (For reference see SwRI presentation)

TSI Flow Meters

**•** Using reference flow meters like the TSI flow meter as a transfer standards is okay, if independently checked via a master flow meter

Sensors PPMD

a. Verify that the external and bypass flow TSI flow meter certificates are valid for duration of testing b. Verify tha t the da ta publ ished on their indi vidual c ertificates me et the 1065 linearity thresholds

Horiba TRPM

**•** Use total flow and dilution air flow for Horiba and make sure they meet 1065 linearity verifications.

AVL MSS

**•** Meeting +/ - 3 % per point on dilution ratio is acceptable.

Vibration

Perform a f requency s weep w ith m oderate a mplitude a nd l og t he r eal t ime s ignals f rom t he PEMS using a fixed PM generator level. If changes occur, design a frequency cycle around that frequency range to test for delta changes in PM with and without vibration. Obtain approval on the f requency cycle s elected from t he M ASC be fore pr oceeding. If no changes are obs erved during a frequency sweep, there is not a need to test for vibration.

Use t wo o rientations ve rtical a nd ho rizontal f or t he P PMD. If t here i s enough s pace on t he vibration table at SwRI, test all PEMS units at once. If not, use them one by one.

EMI

Use E MI on and of f, and s creen t he r eal t ime r esponse of t he i nstrument a t a s pecific concentration l evel. If a r esponse i s obs erved d uring E MI on/ off s witching, t he M ASC w ill decide on what the next steps are.REPORT 03 .14936. 12 B-3 Environmental Chamber:

1. Temperature and Humidity cycle will be made available by Matt Spears 2. Use a high velocity fan to blow over the 4 inch inlet section of the PPMD 3. PM Generator Setup and Verification

4. How to do the environmental pressure test?

a) Use three different levels concentration at the 0.02 and 0.03 g/hp- hr b) Run a long baseline of 8 hour s using three different toggeled PM concentration levels. For baseline, toggle the process five times, and use that as a baseline, followed b y the continuation of the baseline. c) Run the temperature and humidity test using the same toggeling profile. For temperature, humidity, and p ressure, start w ith a b aseline with f ive r epeats, and t hen ki ck of f t he environmental cycle. d) A toggeling of zero, mid, and high is on the order of 15 minutes.

There i s a pos sibility of e liminating t he z ero a nd a dd one c oncentration l evel, a nd r andomly sample from all the deltas generated. We need to talk to Bob Mason about this.

Use just one of each of the PEMS for environmental activities.

Finally, changes w ere m ade t o the t est p lan du ring di scussions. A lso, S wRI pr esented t hat attached document on the test plan. More test plan discussions will take place in the next meeting on May 15 and 16, at SwRI.REPORT 03 .14936. 12 B-4 PM Measurement Allowance Steering Committee Meeting Meeting at SwRI May 15-16, 2008

Day 1

1. Tim French indicates that there are several issues came up with the PM

- PEMS onboard testing performed at CE

- CERT before the CE

- CERT meeting was presented.

2. Kent’s presentation, posted on FTP website:

Some highlights, but see presentation for more details:

a. Problems with valve timing opening and closing consistency and long delays over 3 seconds. b. There was one crystal that was very noisy but the software did not reject it, and reported data using it.

c. A drop in frequency but an increase in mass deposit. (This is a non- issue) d. Semtech DS 10[1].09 SP2 b5. T his beta J1939 includes filtering and was able to include NTEs.

e. Post te st J1939 filtering ve rsus r eal time J 1939 filtering. Post te st c aptured different start up NTE than the real time filtering one. E ssentially, post processing software needs to be scrutinized. f. NTE yield produced by PPMD was low. Some of the valid NTE that was captured in PPMD post processing was rejected, after carefully reviewing the data.
g. Others. See presentation.

3. SwRI update:

a. Horiba linearity check was resolved by using the Gilibrator directlty b. Engine A, B, C, and D 80 steady- state testing was complete

c. Engine A was tested for 40 SS points in sub- NTE runs d. International engine was also tested

e. All above will be posted on the websi te after careful review

4. Janet and Bob Presentation, please see website.

Bill Martin questioned the idea of excluding the 95 th to 100 th data twice. One in the error surfaces, and the other in the measurement allowance. He was concerned whether such practi ce will lead to truncation of the distribution. Janet will show in the next meeting that such practice will not lead to any truncation of data or it will effect the shape of the error distribution.

5. General Discussion and Action Items:

a. Find ways to load u p the can bus during the actual testing to simulate real world operation by making the can bus communication busy.REPORT 03 .14936. 12 B-5 b. Keep us ing t he gaseous error s urfaces f or M ethod 2 a nd M ethod 3 on t he P M

PEMS.

c. Environmental Chamber:

1. Use a 9 m inute c ycle for t he environ mental chamber a nd go i nto t hree concentration levels, 3 m inutes per level. Sample for a period of 35 s econds from each level. 2. Use five levels of dilution ratio ranging from 6 to 50. 3. Check with Sensors to see if they accept an exhaust flow meter analog in put.

For M odel out put, ke ep t he 95 th as t he m easurement a llowance. H owever, i f i t di d not validate c onsider t he p otential of us ing ot her t han t he 95 th , if t he MASC r eaches an agreement on that.

Day 2:

1. Move forward with the International Engine 2. Communicate via J 1939 to load the engine can bus during testing. Use only the J 1939 communication with the heavy- heavy duty diesel engine 3. Accept the idea that Horiba will use test cell engine speed and load analog output signals 4. The right to remove outliers using good engineering judgement. 5. Starting on June 2 nd
6. Spend one week of commissioning before we start. with commissioning

Plot:

x-axis percent of max torque or fuel rate
y- axis absolute difference over max torque or fuel rate

Matt’s Discussion:

Mod el Validation Testing at CE

- CERT

a. Number of engines/vehicles b. Number of bypass conditions, at least two, maybe 3

c. Number of PEMS: at least two of each, highly desirable three d. Number of NTE events: total 100-200 per PEMS

e. Number of route repeats: 1 to 3 f. Types of NTE events i. CE

- CERT limits 1. Minimum filter loading: 50 microgramsREPORT 03 .14936. 12 B-6 Gaseous plus PM. But PM set priorities. Sensors goes first.

NTE Events Route Bypass Repeat Horiba plus AVL 30

- 50 Palm Springs 1 One

- Run Sensors plus AVL 30

- 50 Palm Springs 1, review One

- Run

Horiba plus AVL 30

- 50 San Diego 1 or change One

- Run

Sensors plus AVL 30

- 50 San Diego 1 One

- Run Horiba plus AVL 30

- 50 Baker 1 One

- Run Sensors plus AVL 30

- 50 Baker 1 One

- RunREPORT 03 .14936. 12 B-7 PM Measurement Allowance Steering Committee Meeting Meeting in Madison, Wisconsin June 12-13, 2008

The meeting started by reviewing and approving the last meeting minutes.

PPMD Commissioning

a. Following the meeting minutes, Imad Khalek gave a presentation on the status of the PPMD commissioning at SwRI. The presentation was sent to the MASC, but was not posted on the website.

b. David Booker also gave a presentation on the SwRI commissioning activities. The presentation w as s ent t o t he M ASC, but w as n ot pos ted on t he w ebsite a t t he request of Sensors.

c. As a r esult of t hese t wo pr esentations, t he M ASC de cided t o g ive S ensors a chance t o f ix s ome of t he pr oblems e ncountered a nd c ome ba ck t o S wRI f or additional commissioning during the week of June 16. Also, the MASC requested that a conference call should take place on Friday, June 20, and SwRI should give a status update on w hether or not the Semtech

- DS/PPMD issues were fixed to a satisfactory level so the program can proceed.

d. The MASC also made the following points on the PPMD:

i. It is up to Sensors to decide on the quartz crystal equilibration time, after a crystal goes into a invalid NTE window. ii. It i s r equested that w hen all c rystals ar e l ocked o ut, a nd one o f t hem becomes available during an NTE, the crystal should wait until the current NTE terminates, and a new NTE starts before it samples from an NTE.

PM

- PEMS Engine Selection

Imad Khalek pointed out to the MASC the fact that the PM

- PEMS p rogram is moving forward with some deficiencies related to the Horiba system inability to communicate with the Navistar engine ECM ISO protocol. It was recommended to the MASC that a heavy

- heavy duty diesel engine be installed in the test cell first so the Horiba s ystem can communicate with the engine ECM using the J1939 protocol. In addition, this will give a chance for Horiba to upgrade their system in preparation for the Navistar engine after a heavy heavy

- duty diesel engine.

EMA ag reed to take a l ook at t he pos sibility of pr oviding a he avy h eavy

- duty di esel engine in a timely manner. EMA agreed to make a final decision on t his issue by the June 20 conference call.REPORT 03 .14936. 12 B-8 Horiba Concern

Horiba was concerned about the fact that Sensors brought a new model of the PPMD to be used on the PM- PEMS

- MA program. They wanted to have a chance to update their system as well.

The MASC decided to give Horiba a chance to work and upgrade their systems . It was decided that SwRI should ship back to Horiba one of their system present at SwRI, and give them until July 14 to ship back the system.

Horiba was also given until September 15 t o upgrade their system so it can communicate with the Navistar ECM pr otocol using ISO-15765.

Bill M artin made c omments a bout the di fficulty he s ees in the H oriba system me eting 1065 requirements. Matt indicated that EPA would enter an allowance for Horiba specifically, at the time of Direct Final Rule (DFR), through the a lternate procedure approval.

AVL Presentation

Bill Silvis presented results on the MSS, where a compensation algorithm is added to account for organic carbon and sulfate. No copy of the presentation was given to SwRI for distribution. Matt Spears was not convinced that such compensation will be acceptable for EPA approval due to the significant correction required and due to the MSS principle itself. SwRI also had some technical reservation a bout t he pr ocess due t o i ts t echnical c omplexity. E ssentially, the pr oblem i s not trivial and more thorough work and understanding is still needed.

AVL was encouraged to continue working on t his issue and refine it. It is understood that they will submit the compensating algorithms prior to the start of steady

- state testing. They may also submit their compensating algorithms at any time for the MASC to have analyzed by SwRI or CE

- CERT.

Rey A gama s uggested t hat t ime w ould be t he be st w indow of oppor tunity f or A VL t o be included via DFR through alternate procedure approval.

SwRI Test Cell

The MASC requested the following:

**•** SwRI should install all PM

- PEMS as close as practically possible to the entry of t he C VS. This w ill r educe an y p article l osses be tween t he poi nt of measurement among t he P M

- PEMS and also relative t o the C VS. Furthermore, this will minimize the backpressure experienced by the PPMD, by shortening the length of exhaust piping present downstream of the PPMD, prior to entry to the CVS. The target backpressure is (-1 to +4 kpa)

**•** SwRI should try to maintain temperature of 35°C ± 5°C in the vicinity of the PM

- PEMS inside the test cell.REPORT 03 .14936. 12 B-9 CE

- CERT Presentation

Kent Johnson gave a presentation on the temperature distribution on various location of the MEL such a s be hind t he C AB, unde r M EL t rail, a nd unde r pa ssenger door . K ent’s pr esentation i s posted on t he w ebsite. One of i ntriguing hi ghlights of t he pr esentatio n is that the temperature profile can reach as high as 60 °C to 90 °C during in

- use in the vicinity of the PM

- PEMS. This triggers t he i dea of c hanging t he e nvironmental t emperature pr ofile t hat w as us ed dur ing t he gaseous PEMS program that was based on amb ient temperature. Matt spears will be modifying the existing environmental chamber temperature profile, taking into account Kent’s finding.

Budget

CE

- CERT presented their budget with some options. The overall impression was that the total budget f or bot h SwRI a nd C E

- CERT i s be yond t he f unding l evel a vailable f or t his pr ogram. Below are some of the options entertained that will be discussed during the next meeting at SwRI starting on July 21 st

.

1. Reduce the scope of work by cutting the number of routes and the number of PEMS used by CE

- CERT 2. Reduce the scope of work at SwRI by reducing the number of engines to be tested from two engines to one engine 3. Increase the overall budget by eliminating this year EMA pilot program requirement, and add funding to the MA program instead, assuming that the funding will be cost shared among EMA, EPA, and CARB

Additional Comments and Action Items:

1. If PPMD and OBS200, both, resulted in negative allowance, e.g.

- 0.01 to

- 0.02 g/hp

- hr, the MA will be zero, essentially one MA for the entire program and for all the PM

- PEMS used. Under such circumstances, both instruments will be allowed to be used with a zero MA. 2. Keep aware of alignment issues. One may want to investigate the brake

- specific emission value reported by PM

- PEMS by shifting the numerator and denominators relative to one another and relative to absolute time. This could be done with initial transient test results with bypass. 3. Sensors w ill pr ovide a VI t o simulate t he ex haust f low r ate s o we c an exercise t he multiple dilution ratios in the environmental chamber. 4. SwRI will ne ed t o r esubmit s ome budge t opt ions of doi ng t he w ork w ith one e ngine versus two engines.REPORT 03 .14936. 12 B-10 PM Measurement Allowance Steering Committee Meeting Meeting at SwRI, San Antonio July 21-23, 2008

July 21

The meeting started at 1:00 PM. It started by reviewing and approving the previous minutes, and also by discussing future meeting schedules, which were already sent by EMA as future meeting notices. T he next meeting is scheduled for August 27 ( starts at 2:00 PM), 28, a nd 29 (ends at 2:00 PM).

Imad Khalek started a presentation, posted on FTP website , on project status update. He showed a comparison between laboratory

- based exhaust flow and the 4

- inch Sensors exhaust flow that was used on the Navistar engine. The slope of the correlation was too high at 1.27. As a result of the discussions, the MASC decided on the following with the current 5

- inch exhaust flow meters used w ith t he M ack engine, which t he f irst engine t o be us ed as a pa rt of t he of f icial measurement allowance work.

**•** Check laboratory

- based exhaust flow rate with Horiba and Sensors 5

- inch exhaust flow meters. If a problem is obvious or the slope of the correlation differs by more than +/ - 0.05 from a slope of 1, then send the flow meters back to the manufacturers to check on the calibration.

After finishing a part of the presentation, Janet Buckingham and Bob M ason showed up f or a scheduled presentation at 3:00 PM. The presentation is posted on the FTP website and addressed the double truncation issue raised by Bill Martin at the 5 th and 95 th percentile. The presentation is posted on the FTP website. Based on the presented work, the following was agreed upon:

a. The 95 th
b. The 5 percentile is still desired for the measurement allowance
th an d 95 th

c. There was still a remaining unresolved issue about where you assign the -1, 0,1 on the error surfaces for the delta change between lab and PEMS. It was decided that this i ssue s hould be a ddressed d uring the la st da y of the me eting, but th at w as never brought up a gain. There was a proposal by Bill Martin to fit both sides of the error distribution independently using a normal distribution fit. The median of such distribution will be the median based on previous practice along with 95 are still acceptable to bound the validation range.

th for one s ide and 5 th for t he ot her s ide. W hen e ach side of t he di stribution i s f itted using the above boundary conditions, one can then expand the data picked from the error surfaces to cover 1 % to 99 % of the data or even 0.1 % to 99.9 %. No decision has been made on this issue yet. This topic will require more discussion during the next meeting. July 22

A significant part of the day was allocated for budget discussion. The different budget scenarios are posted in th e FTP website . Below are some of the highlights:REPORT 03 .14936. 12 B-11 For budget cutting purposes, one engine will be used for the measurement allowance at SwRI. In addition, work will be performed with one bypass setting that gives brake specific PM emissions levels between the two thresholds of 0.02 a nd 0.03 g/hp

- hr. For CE

- CERT, choose the PEMS with the l owest pos itive m easurement al lowance. If one P EMS cl early s hows a l ower measurement a llowance t han t he r est, a nd bot h a re pos itive, pi ck t he one w ith t he l ow
measurement a llowance for the rest of the validation testing. If one allowance is positive by one PEMS, and if the other one is negative, then, in principle, choose the positive allowance if it slightly positive. (no clear cut agreement yet).

After the budget discussion, Tim French from EMA walked the group (via phone) through the EMA proposal, posted on F TP website , to substitute year 1 pi lot program and to also provide some supplemental funding on the order of $200 K to inject more funding in to the measurement allowance program. The overall EMA budget funding was projected to be on the order of $500 K to $700 K . Most or all the proposed activities will be performed in

- use by hiring a third party that is not part of EMA and does not belong to one of the engine manufacturer to conduct the testing. As a result of the EMA proposal the following will take place:

**•** Matt Spears will speak with EPA upper

- management to consider the EMA proposal in the context of the one year pilot program.

**•** EPA will also make a final determination on whether to use the shortened version of the measurement allowance program. E.g. one versus two engines, by-pass, no- bypass, etc...

After the budget discussions, Imad Khalek continued his presentation that was started on the fi rst day. T he w ork s howed the di fferent t orque a nd f uel errors s urfaces de termined be tween t he laboratory and the engine ECM public broadcast. The data covered a total of five engines (four heavy

- heavy a nd one l ight

- heavy) t hat i ncluded a D DC S eries 60, C AT C13, C ummins ISM, Mack MP7, and Navistar 6.4 liter engine.

Also the work covered sub

- NTE fuel flow errors. Based the sub

- NTE results, it was decided that for f orced N TEs, i f t he engine ope ration f alls be low N TE, us e t he l ast BSFC va lue obs erved within t he NTE. Use that only for Method 2 calculations.

After this presentation, Imad Khalek refreshed the memory of the group by giving a status update presentation, posted on t he FTP website , based on the last commissioning work done at SwRI. The pr esentation w as p osted on t he F TP w ebsite. A fter t hat, K ent J ohnson f rom C E

- CERT shared s ome obs ervations a bout t he S ensors da ta pr oduced b y S wRI d uring c ommissioning. There was no conf lict between SwRI and C E

- CERT r eporting on t he r esults. K ent’s W ord document t hat w as s hared w ith t he group w ill be pos ted a t t he FTP w ebsite w hen i t be comes available to SwRI.

July 23, 2008

The focus of this day was on t he Test Plan, particularly the environmental testing. However, at the beginning of this day, the idea brought by SwRI earlier of adding short NTE windows for the laboratory t ransient t esting, along w ith s ome l ow or m edium i dle ope ration pr ior t o t he NTE transient cycle was discussed.REPORT 03 .14936. 12 B-12 **•** It was decided that Short NTE windows should be added to the thirty 32

- seconds NTE cycle, in the inter time. A total of 10 short NTEs will be added as follows:

1. 4 five seconds NTEs, with two back to back within an inter- time 2. 3 10 seconds NTEs 3. 3 18 seconds NTEs

Position two f ive s econds N TEs ba ck t o ba ck i n t he i nter

- NTEs. F or H oriba, t hey need about 2 t o 3 seconds to exit an NTE. Thus, place the short NTEs at least five seconds after the end of a valid NTE to avoid continuous sampling with Horiba.

**•** On the conditioning pr ior to the start of a transient NTE, start with five to 10 minutes at medium idle time before the NTE transient cycle to observe if there is an effect on the laboratory P M e missions r esults. If t here i s a nd e ffect, t hen pr opose an i dle t ime o r something si milar prior to starting the NTE transient cycle. (Note that before each NTE transient cycle, the plan is to force

- regenerate the DPF first, medium speed idle will then be added after the forced regeneration).

EMI/RFI/Shock &Vibration

1. For EMI/RFI, expose the PM PEMS to EMI/RFI using HEPA filtered air, similar to the gaseous pr ogram. T hen decide w ith t he M ASC after reviewing t he r esults, w hat i s t he next step. As an option, we could use the PM generator to perform this work. One PEMS from each manufacturer will be used for these activities. 2. Vibration and Orientation (non-road only)

a. On t he or ientation, a sk PEMS m anufacturers on t he w orst or ientation s cenario postion. U se such orientation at 45 o

b. Perform a frequency s weep, with a v ery m oderate am plitude, then share w ith MASC to decide on how to go forward. Eric should propose the sweep frequency, amplitude a nd dur ation. Do t wo r epeats. O ne of e ach i nstrument w ill be us ed. This is only for nonroad. Outside the scope of the program. with the appropriate orientation, and survey the worst case sce nario for all gaseous and for PPMD, and TRPM.

3. As for shock, ask Eric about a recommended on-highway profile for shock.

Temperature/Humidity chamber and Pressure Chamber

For bot h t emperature a nd hum idity a nd pr essure c hamber w ork, us e t hree P M c oncentration levels around the two threshold levels such as 4000, 5000, a nd 6000 μ g/m3. Use a total of four dilution r atios of 6, 12, 20 a nd 30. A t e ach di lution r atio, s tabilize f or 4.5 m inutes a t e ach concentration level and measure for 30 seconds before you go to the next concentration level to do the same. E.g.

1. Set the dilution ratio to 6, stabilize at 4000 μg/m3 for a 4.5 minutes 2. Measure PM using PEMS for 30 seconds 3. Move to next concentration level of 5,000 μg/m3 and stabilize for 4.5 minutes 4. Measure PM using PEMS for 30 seconds 5. Move to the next concentration level of 6,000 μg/m3REPORT 03 .14936. 12 B-13 6. Measure PM using PEMS for 30 seconds 7. Repeat 1 through 6 at different dilution ratio

A spreadsheet on PM concentration and loading was origin ally made by Matt Spears during the meeting and was very slightly modified by Imad is posted to the FTP website

As a r esult of t he pr oposed s cenario a bove, S ensors P PMD w ill r equire a s econdary di lution using M PS2 t o pe rform a n 8 hour s of a ctivities us ing the a bove s cenarios. In c ase of H oriba, either we need to allow higher dilution ratio than those adheres to 1065 or we need to allow more loading on the filter past a 0.4 mg. More discussions on t his will need to take place during the next meeting.

The PM cycle to run by the PM generator contains five 15 minutes cycle to be used as a baseline. E.g. R un t he f irst f ive 15 m inutes a t nor mal t emperature, us e t hose a s a ba seline l ine, t hen proceed for the rest of the day to capture an eight hours of similar repeats. Do the same thing prior to starting the Temperature/Humidity profile and well as the Pressure Chamber profile . One of the remaining issues that has not been resolved around this topic is how do w e capture the baseline i nformation w ith t he H oriba s ys tem without the ne ed to change the f ilter. More discussion on this subject is needed during the next meeting.

New Temperature Profile

Take the mean of CE

- CERT Cab minus ambient and add it to original temperature profile, and solve for new temperature an d humidity profile that maintains derived from the real world data.

Perform a F ourier t ransform on t he C E

- CERT Cab data, eliminate f requency content t hat ar e similar to the base ambient profile, and use a magic synthesizer to superimpose the frequency on top of the new temperature profile. Matt Spears is assigned to do this task.

Pressure Chamber

We resolve the logistical issue of the pressure chamber work by placing the PM generator inside the chamber. Matt will need to send me some dimension on the PM generator to see if it fits in the Chamber. Other gases as well as 30 amp circuits should all be accommodated.

For Horiba, heat trace the segment of the transition to 250 o C, similar to the PM generator outlet temperature for both the Horiba and AVL.

U se each instrument separately for these tests.REPORT 03 .14936. 12 B-14 PM Measurement Allowance Steering Committee Meeting Meeting at SwRI, San Antonio August 27,28, & 29, 2008

Meetings

October: 9,10- at EPA Nov: 12,13, and 14- at SwRI Dec.: 10,11,12- at SwRI

Discussion Points and Decisions Made

**•** For t he e rror di stribution c hoose t he 1 st and 99 th at

- 1 a nd 1, ba sed on a nor mal distribution fitting to extrapolate down to the 1 st and 99 th

**•** Recheck the 4 inch flowmeter calibration that was on the International engine . For the height of the median, if there is a discontinuity, choose the average or randomly pick one or the other.

**•** For Horiba, filter loading is allowed to go beyond 400 m icrogram up t o 700 microgram or beyond as long as the flow is controlled.

**•** For Horiba, do the five cycle baseline at the beginning for humidity and temperature and do it at the end for the pressure. This will require changing the filter. For the pressure, if it can be done within an hour than do it at the beginning, otherwise, do it at the end.

**•** Which of the three units are acceptable to be used in the pilot program:

 PPMD is approved

 Horiba might be approved

 AVL will have to make the case at EPA to see if approved

 EMA would like to know what instrument would be acceptable for pilot
one program

 Can EMA combine pilot efforts within companies

 EMA will fund the additional funding required that will be required to do the intermediate testing that involves:

**•** One bypass setting, one engine at SwRI

**•** Three PEMS, one manufacturer with CE- CERT

**•** We s hould m ove f orward on R MI, R FI, and vi bration s weep, as s oon a s w e s tart t he official testing

**•** Due to spikes, we may revisit the post processing of the Horiba results, especially during the N TE tr ansient te st. As of r ight now , Horiba f irst c orrelates the e ntire E AD s ignal ( including spike) with the filter weight (not including spikes), then apply the relationship for the NTE window portion of the cycle.

SwRI Presentation and Action Items

During t he m eeting, SwRI gave a s tatus upda te vi a pr esentation, see enc losed presenta tion (Update 6), on the PPMD, MSS, and TrPM. SwRI, also discussed the bypass tuning for using 10 NTE transient c ycle, 80 poi nts steady

- state us ing M SS, and pr ojected filter m ass concentration, us ing a r elationship be tween t he f ilter a nd t he M SS. In a ddition, t he b ypass mixing w as s hown qualitatively. Furthermore, th e tr ansient c oncentration tr ace w ith the M SS was presented for the 10 NTE transient cycle.REPORT 03 .14936. 12 B-15 As a result of the work reported several action items were born:

**•** Make sure that the filter

- based stea dy

- state projected concentration is consistent with the filter

- based measured for t he s ix s teady

- state poi nts s elected. T hus, pe rform an experiment to determine the f ilter

- based m easured c oncentration f or the s ix poi nts selected and share the results with MASC

**•** Instead of s howing a qualitative r esults on t he m ixing, pr ovide s ome qua ntitative assessment such as a T

- test.

**•** Based on the high spikes observed with the MSS, check if the PPMD results in a low bias due to the f act that it m ay be mis sing the early spi ke. For t hese experiments, pi ck t he transient N TE w indow with t he hi ghest s pike and c reate a c ycle t hat consists of 10 repeats of that same window. During this exercise, use only three working crystals and vary the PPMD trigger into the NTE with time advancement of 1,2 and 3 seconds, time delay of 1 second, no time delay, and ECM trigger

The above work and the problems encountered were presented to the MASC via two conference calls t hat w ere don e on S eptember 12 a nd S eptember 30, 2008. T he p resentations a r e al so enclosed as Update 7, and Update 8.REPORT 03 .14936. 12 B-16 PM Measurement Allowance Steering Committee Meeting Meeting at EPA, Ann Arbor October 9 and 10, 2008

SwRI gave a status update on the progress of testing. See enclosed presentation.

There w as a con cern about the pr edicted lasting time of the P PMD in the f ield, which was predicted to be on t he order of 1 hour, using an average NTE threshold of 0.02 g/hp

- hr. There was a desire to extend the running time of the PPMD to at least 2 hour s. This is indeed met b y the ins trument if on e ta kes int o account over all collection efficiency o f the ins trument. In the example given, it was assumed that the collection efficiency was a 100 %, where in reality it was on the order of 50 %. Thus the one hour of lasting time reported is in reality two hours.

A decision was made to move forward with testing using only the MPS1, as shown below, after final c ommissioning of the P PMD b y S ensors. Actually, D avid Booker f rom S ensors f lew t o SwRI late on October 9 to be at SwRI on Octo ber 10.

1. Test on MPS1 only, as its currently configured at one microgram. 2. Okay to use external trigger for steady- state testing 3. For steady

- state testing:

a. Target 100 microgram on the filter b. 50 microgram for TRPM

c. 0.66 microgram for PPMD

Do not clean crysta ls until it is apparent that the next run will likely overload the filter.

The second day of the meeting was spent at Sensors. Matt Spears gave a presentation on the PM generator. Later, he s howed t he P M g enerator s etup, a nd e xplained t o t he g roup t he v ar ious elements of the PM generator and the equipment used. A copy of Matt’s presentation is enclosed. Also, a copy of Matt’s note is shown below, particularly to 1065 PEMS changes:

Matt’s Note:

1. Review minutes from last meeting 2. Discuss 1065 changes required for PM PEMS 3. Update from SwRI on recent activities 4. Update from PEMS manufacturers regarding recent phone conference 5. Friday afternoon @ Sensors

a. SUN conference presentation b. PM Generator 6. Discuss 1065 changes required for **

a. Di lution Air PEMS field testing only**

i. Temperature control 1. 1065 = 25±5 ° CREPORT 03 .14936. 12 B-17 2. PM P EMS= no di rect f eedback c ontrol f rom a di lution a ir temperature m easurement r equired

-- use good e ngineering judgment; if directly and actively controlled then target 25 ° C . ii. You may us e a f ixed molar m ass o f th e di luted e xhaust mixt ure f or a ll PEMS field testing, as determined by engineering analysis. b. “Filter” i. Media 1. PTFE membrane or TX -40 ii. Face velocity 1. 1065= target near 100 cm/s actual, unless overloading 2. PM PEMS

a. Flow

- through media: (10 to 100) cm/s actual, which can be verified by engineering analysis b. Non -flow through media: no specification iii. Temperature 1. 1065 is 47±5 ° C 2. PM PEMS target (42 to 52) at all times, with a minimum tolerance of 32 ° C and a maximum tolerance of 62 ° C, where the tolerances apply only during filter sampling. iv. Conditions during mass determination 1. 1065=see subpart B 2. PM PEMS

a. If mass is not determined in

- situ

— i.e. within the PEMS

— then t he s ample c ollection m edia m ust be pr e a nd pos t analyzed according to 1065.190x. b. If mass is determined in -situ, follow .195.

c. In s ubpart J , ha ve no r equirement t o hol d t o de wpoint specs for in

- situ analyzers.

c. Absolute reference for inertial balance i. Current status: QCM OEM stated specs are assumed. ii. For 1065 m easurement allowance audit we had Sensors verify frequ ency measurement circuit. iii. No immediate solution available d. Cleanup 915 table for inertial batch PM analyzers: no freq, or rise/fall time specs. Recommend a process for determining noise, accuracy, and repeatability

e. 1065 Subpart J needs to state that field testing applies at any ambient temperature, pressure and humidity, unless otherwise specified in the standard setting part. f. State that EPA approves of electrostatic deposition technique for PM collection. Must meet 95% collection efficiency, as stated by the manufacturer.
g. Overall P EMS t est r equirement s hould be r eread a nd e dited t o be applicable t o batch analyzers. For example describe how to use a combination of steady

- state and transient test modes to determine accuracy and repeatability separately; like what we’re doing in the measurement allowance. h. 1065.308- 09: a lso r equired f or c ontinuous PM a nalyzers

— read a nd edit accordingly i. 1065 clarify that options after 400 ug loading are optional j. Clarify whether or not ambient air may be used for zero air for PEMS, including for hangup check.REPORT 03 .14936. 12 B-18 k. Drift: allow any drift that doesn’t affect your ability to demonstrate compliance with the applicable standard. 7. Timetable

a. Next meetings 8. Test Plan

- documentation reflecting the latest agreements

a. Validation

— get from CE

- CERT b. Modeling 9. For PEMS testing set dilution ratio based on manufacturers literature regarding maximum exhaust f low. Y ou m ay also us e ot her m anufacturer i nformation t o pe rform a n engineering analysis to estimate the maximum. 10. There will be no dilution ratio veri fication.REPORT 03 .14936. 12 B-19 PM Measurement Allowance Steering Committee Meeting Meeting at SwRI, San Antonio November 12 and 14, 2008

Flow Alignment in PEMS

In case of continuous sampling, use a step function to measure the delay from the probe to the instrument.

In case of batch sampling, use the geometry to account for time delay from the probe to the batch sampler

The above is done to align the flow or to account for time delay.

Leave alone any time alignment between exhaust flow and ECM torque and speed. Use the ECM torque and speed to determine the integral over the NTE.

Loss Correction

a. The principle of PM loss corrections for PM PEMS is agreed upon by the steering committee b. EMA desires a legal construct in 1065 for allowing the use of PM loss corrections i. Open up 1065.295 to allow more types of compensating algorithms, based upon other variables ii. Utilize S ubpart J ove rall a pproval te st to validate e ntire P M P EMS, including its loss corrections.

The PEMS manufacturers will be allowed to use compensation alg orithms.

**•** Horiba decided to use no particle loss algorithm.

**•** AVL p resented a l oss a lgorithm t o c orrect f or t hermophoresis. T he l oss c orrection i s already defined and will be implemented via a post processor provided by AVL

**•** Sensors plans to correct for particle loss and will share the process with MASC during the next meeting

As of today, no filter data can be shared with Sensors unless the loss correction is shared with the MASC.

Test Matrix, DPF out

Full day no QCM cleaning or filter changing

Storage, high speed, light load Release at peak torque, run for 20 minutes Store again at high speed, light load Release at rated powerREPORT 03 .14936. 12 B-20 Do another storage at low idle, equivalent

Cycle the PEMS every one minute to sample into NTE. Cycle the PEMS every 32 seconds 5 second dwell time in between the NTEs

PEMS Daily Checks Tolerance

Slope of 0.96 is acceptable by Sensors For Horiba, 3% on filter flow, and 3 p ercent at a dilution ratio of 5, and 5 percent at a dilution ratio of 15, and Mike will reconfirm. We will get a feedback from AVL on the accepted tolerance for a dilution ratio of 5.f

Milestones

Milestone for the Model dry run PM generator milestone for commissioning, week of January 5 Milestone for the fuel flow error surface delivery by the engine manufac turers (EMA will target the end of February for these data to be available. The eight or nine engines that are available now will be used for the dry run).

Fixed date for the delivery of environmental chamber kit by the PEMS manufacturer. Ship it on the 19 th of January by Sensors and Horiba.

We will schedule EMI, RFI, on t he 26 th of January, and shock and vibration on t he following week.

Matt Spears’ Note:

Wednesday 1. Upcoming meetings, December, January, and March all at SwRI

a. December 10 -12 at SwRI (10 th
b. January 28 2pm start, 12 2pm close)
th – 30 th (28 th 2pm start, 30 th

c. March 18 2pm close)
th – 20 th (18 th 2pm start, 20 th
2. Review October meeting minutes 2pm close)

3. EPA / SwRI / CE

- CERT PM filter round -robin

a. Initial results 4. Temperature

- Humidity test cycle

a. EPA cycle b. SwRI addition of CE-CERT frequency content 5. Status and progress at SwRI since last meeting

a. Update on mission time projections i. Relook at projections to see if it is possible to collect 20 failed NTE events ( 0.02 g/bhp-hr) in one mission b. Test plan review

c. Decisions for MASC 6. Budget update

a. Arrangements for EPA/EMA/ARBREPORT 03 .14936. 12 B-21 Thursday 7. PEMS PM loss algorithms

a. AVL b. Sensors

c. Horiba 8. Test plan development

a. Inclination discussion 9. Continue if necessary: Status and progress at SwRI since last meeting

a. Decisions for MASC

Friday 10. Continue PM PEMS related Part 1065 changes 11. Other meetings during our November meetings

a. Wednesday i. Wrap -up by 5:30pm due to another room reservation b. Thursday i. 12:00pm EMA, Rey Agama – break for lunch at 12:00pm ii. 1:00pm – 2:00pm, Shirish Shimpi, continue meeting

c. Friday i. Matt Spears 10:00am to 10:45am, continue meeting 12. Loss corrections resolutions

a. The principle of PM loss corrections for PM PEMS is agreed upon by the steering committee, including EMA, EPA, ARB. b. EMA desires a legal construct in 1065 for allowing the use of PM loss corrections i. Open up 1065.295 to allow more types of compensating algorithms, based upon other variables. ii. EPA a grees tha t P M los s c orrections w ill not be a pplied to certification testing. If i n t he f uture E PA de sires t o a pply P M l oss c orrections t o certification t esting, s uch a pr ovision w ould b e pr oposed as pa rt of a notice of pr oposed r ulemaking be cause E PA a cknowledges t hat s uch a change would cause a cha nge i n the s tringency of t he ce rtification standard. iii. EPA w ill be t he a pproving bod y w ith r espect t o P M P EMS P M l oss correction. 1. May make case

- by

- case approvals, based upon specific PM PEMS manufacturer circumstances, such as, but not limited to submitted, models, t heory, v alidation da ta, or e ven s imply t he m agnitude of the correcti on. 2. May utilize S ubpart J overall approval t est to validate entire P M PEMS, including its loss corrections. 3. May develop (with c onsultation w ith E MA) ot her pr ocedures f or codification within Part 1065. 13. Sensors w ill pr ovide a n a dvance c opy of S ensors’ D ecember pr esentation, r equesting EMA question consolidation ahead of meeting.REPORT 03 .14936. 12 B-22 PM Measurement Allowance Steering Committee Meeting Meeting at SwRI, San Antonio December 10 -12, 2008

SwRI p resented t he s teady

- state t esting r esults, and the s torage and release and r egeneration results. The presentations were posted on the FTP site.

Sensors pr esented t heir approach t o pa rticle l oss c orrection. A pr esentation i s a vailable at t he FTP site.

Craig Kazmierczak presented some in

- use PM

- PEMS work that was done on one of DDC trucks using the PM-PEMS equipment. The work was done by Sensors and presented by Craig.

The MASC requested that the reference crystal be used during SwRI laboratory activities, if that is to be used on the road.

The t esting done b y S wRI s o f ar i s acceptable. F or t he pa rticular P PMD us ed, di sable t he reference crystal, and use a working c rystal t o be a r eference. Essentially, operate w ith six working crystals, and use one for reference.

Sensors intends to use the reference crystal for correction, but they are going to use a logic to decide whether or not it will be used in the post processor.

The pos t pr ocessor s hould be a vailable be fore January 23, 2009. T he post pr ocessor s hould include any loss correction intended or any reference crystal corre ction model, remove all Part 1065 excursions, and all other miscellaneous items that will make a measurement invalid.

In t he current A VL pos t pr ocessor, t hermophoretic l oss w as c apped at 25 % . A VL w ants t o change that to include the full range of the mode l, and remove the 25 % cap. Both corrected and uncorrected data will be provided in the output of the post processor.

We agreed to use the paired analysis for steady -state and for model validation. We agreed to use 5 % COV at a 1 sigma standard deviation for the CVS at 100 microgram filter loading.

The a bove s ubject w as t abled f or f urther di scussion on how t o s ubtract t he C VS e rror contribution. Bill Martin will send his proposal to Bob and Imad.

How to account for a regeneration in the field:

NTE event > 30 seconds to be valid

1- Discrete 2-Triggered by ECM 3-The regen is defined between two regen flags 4- If the regen occurs during an NTE event, and the length between two consecutive NTE flag x 2 is shorter than the NTE event, then the regeneration is countedREPORT 03 .14936. 12 B-23 5-If the regen occurs before an NTE event, and end outside the NTE event, the NTE will count.

Tim F rench men tioned that t he s ubject of r egeneration a nd i ts i nclusion i n N TE ne ed t o be discussed in a different forum.

Blow

- By and how it will be computed for the measurement allowance. Next meeting

As of now, the results reported for storage and release and for r egeneration will not impact the MA program.

Matt will share the filter results during the next meeting.REPORT 03 .14936. 12 B-24 PM Measurement Allowance Steering Committee Meeting Meeting at SwRI, San Antonio January 28-30, 2009

The l ast m inutes w as reviewed and a pproved. T he s cheduling for f uture m eetings and conference calls was as follows:

Conference Call, Thursday, Feb. 26, 2009, 9:00 am Central, length will be decided on Friday.

Next MA -SC, April 1 st (2:00 to 6:00), 2 nd (9:00 to 5:00) and 3 rd (8:00 to 2:00), San Antonio.

Week of the 18 th of May for EMTC and MASC (May 20, 21, 22, same as the April meeting)

**•** Some of the in

- use testing performed by Ce

- Cert will include regeneration events. No action will be taken on modifying the test plan at SwRI. The test plan will remain the same and it will not include active regeneration.

**•** Bill Martin explained his proposal on the CVS variability addition to SS delta data. Bill Martin presented the analysis on how to assign the CVS variability into the delta between PM

- PEMS and CVS in order to shrink the 95 th and 5 th A copy of his write

up is on the FTP website

**•** SwRI presented work on the progress made. A copy of the presentations is posted on the FTP website

**•** Rey Agama presented an argument about using standard deviation instead o f MAD for the data analysis. Matt Spears suggested that the MAD should be used, and if no validation w as obs erved a t t he e nd of t he p rogram, ot her pos sibilities c an be considered. The group agreed to move forward with this approach. Rey asked that we apply a normality test on the data, and he will consult with Bob Mason on that. This is in relation to applying a MAD or SD for the model. Bob Mason will present some material on the normality criteria during the next meeting.

**•** The contribution of blow

- by will be a constant based on t he data presented from the four ACES engines at 0.00042 g/hp

- hr. If the crankcase is vented to the atmosphere, this va lue w ill be added t o e very N TE e missions va lue. If t he c rankcase i s c losed loop, t he bl ow

- by contribution of 0.000 42 g /hp

- hr w ill not be a dded t o t he N TE emission values.

**•** Horiba was allowed to fix a bug in their system in relation to delays between engine and OBS and OBS and TRPM

**•** Horiba was allowed to make modifications on the ion trap voltage of the EAD for the third TRPM unit.REPORT 03 .14936. 12 B-25 **•** The engine manufacturers agreed to submit the ECM/Lab Torque and Fuel Flow by March 31 st , 2009. The data will not be linked to a particular manufacturer.

**•** The i nstrument m anufacturers ne eds t o s ubmit t he l atest pos t pr ocessor b y F riday, Mar ch 6, 2009

**•** The MASC agreed that the AVL measurement allowance will be performed based on data m easured plus t hermophoretic cor rection. The M ASC a grees t o also see d ata from the AVL MSS total PM prediction on a non

- interference basis from the core of the measurement allowance program.

**•** The MASC was updated on the PM generator via a laboratory tour. The PM generator is currently set up at SwRI Particle Laboratory.

Below is the unedited Matt Spears’ minutes:

1. January meeting agenda

a. Next meetings? b. EPA participation/management in MASC

c. Review of Meeting minutes d. Regen in NTE discussion

e. Bill Martin’s paired testing proposal f. SwRI data i. Transient ii. 2 nd
iii. Lessons learned, problems? set of PEMS

iv. Analysis of MSS with sulfate and HC corrections

v. ECM vs test cell torque and BSFC error surface update 1. SwRI approach
g. PM Generator update h. Environmental chamber update i. PEMS mfr readiness ii. SwRI readiness & schedule iii. PM Generator readiness i. Next face

- to

- face meeting: April 1 afternoon, to April 3 rd j. May 13 -15, placeholder- SwRI afternoon— SwRI

k. Bypass sizing for model validation work i. DDC engine l. Filter results

m. Friday Schedule i. 9:00am start ii. Review test plan timeline iii. Environmental chamber update 1. PEMS mfr readiness 2. SwRI readiness & schedule 3. PM Generator readinessREPORT 03 .14936. 12 B-26 iv. AVL data with correc tions

v. Horiba general topic vi. Finalize agenda for February 26 th meeting

— finalized, finalize time. 9am central. 4 h rs. 12: 00pm C ST – 4:00pm C ST ( 1pm

- 5pm E ST) S hirish travelling to NRMM in Ispra Feb 26 th
1. Agenda

a. Make this a LiveMeeting, this is ok w SwRI b. Update on PEMS manufacturer post processors:

c. Timeline upda te

— SwRI: t est cel l and environmental chamber testing: 1-hr

d. CE

- CERT upda te on a bility t o c ome t o S wRI f or correlation testing & progress on bypass: 30 min

e. SwRI data 30 min

i. PEMS s et nu mber 2: s teady

- state & pe rhaps transient r esults

— summary onl y: a nything remarkably different than the 1 st set of PM PEMS:

ii. Any new issues or difficulties: 1-hr

f. 30 min

vii. PM Generator / nanoparticle lab tour viii. Engine manufacturers to submit ECM/test cell torque/fuel rate data by March 31 st ix. Rudy’s for lunch meetingREPORT 03 .14936. 12 B-27 PM Measurement Allowance Steering Committee Meeting Meeting at SwRI, San Antonio April 1 -3, 2009

PM

- PEMS Meeting: April 1 -3, 2009.

Wednesday, 2:00- 6:00 PM

**•** Meeting Minutes from previous meeting were reviewed

**•** Practi ce measurement allowance on incomplete set of data was presented by J anet, see presentation on website

**•** The viability of using Method 3 w as discussed. There was a general agreement that this method ne eds t o be dr opped out m ainly b ecause t he l ack o f i nformation on t ime alignment between ECM fuel flow and gas

- based fuel flow, but such decision needs to be reconfirmed during the next meeting.

Thursday, 9:00-5:00 PM

**•** The nor mality t est r equirement w as di scussed b y B ob M ason, s ee p resentation on website. The d ec ision f or now i s not t o a ssume normal di stribution a nd us e t he M AD instead of SD. If the MAD failed to produce a validated measurement allowance, the SD will be revisited.

**•** David Booker from Sensors presented the features of the new PPMD post processor, see presentation on website

**•** Mike A kard f rom H oriba ga ve a n upd ate on t he s tatus of H oriba’s pos t processor, s ee presentation on website

**•** SwRI gave a presentation on the following (see pres entation on FTP website):

**•** Status update and project progress

**•** SS data from all the PM

- PEMS

**•** Transient data with and without engine drift from all the PM- PEMS

**•** OC/EC data for SS

**•** EEPS data for SS

**•** Some MSS corrected data for sulfate and HC

**•** Reference N TE using m e thod 3 g ave di fferent va lues t han M ethod 1 and 2, a nd decision will need to be made on Method 3 during the next meeting.

Friday, 8:00-2:00

**•** Vibration for offroad was discussed. SwRI plans to do a vibration sweep similar to on

highway but while the PEMS si tting at the vibration table at 45 degree angle. SwRI will present a cost estimate for this additional activities when the vibration activities start.

**•** SwRI s howed t he f uel a nd t orque da ta s ubmitted b y all e ngine m anufacturers t hat included C aterpillar, C ummins, D etroit D iesel, Navistar, a nd V olvo P owertrain. S wRI will present all data together during the next meeting.REPORT 03 .14936. 12 B-28 Some of the decisions/action items made during this meeting were as follows:

**•** A decision on whether or not Method 3 should be used (AVL on ly) needs to be made

**•** Reference NTEs will be tweaked so the main distribution of events will be between 0.015 g/hp- hr and 0.035 g/hp -hr

**•** SS error surfaces will be presented during the next meeting

**•** Transient data was drift corrected using the CVS, but the MASC desired to look at the integrated AVL data as compared with the CVS to look at the possibility of using real time AVL data for drift correction. The integrated AVL for the NTE cycles will be compared w ith t he i ntegrated C VS dur ing t he ne xt m eeting, be fore m aking a f inal okay on the CVS drift correction method

**•** EPA was to provide some information on t emperatures experienced during off

- road in

- use a ctivities s o i t c an be i ncorporated w ith t emperature a nd hum idity pr ofile during environmental testing. EPA was to propose a final temperature and humidity profile for the program, after incorporating the CE -CERT and off-road data

**•** Off -road vibration tests will be added at 45 degree angle

**•** Fuel a nd t orque e rrors us ing t he e ngine m anufacturers’ s ubmitted f uel a nd t orque needs to be presented during the next meetingREPORT 03 .14936. 12 B-29 PM Measurement Allowance Steering Committee Meeting Meeting at SwRI, San Antonio May 20-22, 2009

PM -PEMS Meeting, May 20-22, Meeting Agenda

1. Review of last meeting minutes 2. Next Meeting Schedule 3. Over all Project Status Update 4. CE -CERT and SwRI Correlation 5. Update on EMI and RFI Testing 6. Update on Environmental Chamber Testing 7. SS Error Surfaces and Adjusted Reference NTE events 8. Other agenda items and questions

**•** The last meeting minutes were reviewed

**•** The next meeting schedule was set to July 15 -17 in Indianapolis

**•** The overall project status we presented. See presentation by SwRI

**•** CE -CERT and SwRI Correlation was presented. See presentation by SwRI

**•** EMI and RFI testing results were presented. See presentat ion by SwRI. No error surfaces will be generated. The problems mainly affected instrument functions. i. Horiba will investigate the issue related to Bulk Current Injection effect on the Horiba exhaust flow ii. Sensors will demonstrate a fix to the exhaust flow p roblem during the next meeting. Sensors will conduct their own testing if necessary.

**•** SS error s urfaces were pr esented. Sensors data were not fully analyzed because the new post processor was not provided to SwRI. Sensors promised to provide a new pos t pr ocessor r esolving t he i ssues i dentified b y S wRI dur ing t he l ast meeting to increase data yield. Sensors later provided the new post processor, and the SS error surfaces were presented during a co nference call that took place on June 29, 2009. See SwRI presentation for the June 29 conference call.

**•** The transient errors were also discussed, and the approach for the transient error surface w ill be p resented dur ing t he ne xt m eeting i n Indianapolis, a long w ith a final recommendation

**•** Sensitivity on f uel flow a nd CO 2 flow f or M ethod 2 a nd M ethod 3 will be discussed during the next meeting in IndianapolisREPORT 03 .14936. 12 B-30 PM Measurement Allowance Steering Committee Meeting Meeting at SwRI, Indianapolis July 14-17, 2009

1. Last Minute Review 2. SwRI Presentation

a. Project Update b. SS and Transient Error Surfaces treatment and results

c. Environmental T esting ( Atmospheric P ressure, T emperature and H umidity) Results and Discussion i. Three approaches were presented on how to treat the error surfaces for the environmental testing. ii. Approach 3 was u sed. The steering committee agreed with using approach 3 iii. No error surfaces were obtained for Horiba and Sensors for environmental testing iv. Error s urfaces f or A VL, but a nything b elow t he l owest M AD, s et t o a constant equal to the lowest MAD, and anything abo ve the highest MAD set to a constant equals to the highest MAD d. For Method 3, and for the reference NTEs, we agreed that the gas based fuel flow will be advanced in order to match the ECM fuel flow. 3. Rules on Measurement Allowance:

a. Pick t he pos itive m easurement a llowance t hat i s c loset t o zero ba sed on t he Horiba’s and Sensors’ PEMS. b. If bot h H oriba’s and S ensors’ P EMS ha ve a ne gative m easurement a llowance, pick the one that is closest to zero. 4. Preliminary conference call scheduled for August 20 , 2009. We wil l confirm it tomorrow. At the August’s conference call a de cision on the measurement allowance will be made and an instrument will be selected to go to CE

- CERT. After that, SwRI will ship the units to CE- CERT to arrive at CE

- CERT earlier than September 1 5. Meeting at CE

- CERT on September 22 st

nd to observe and check PEMS installation by CE

CERT.REPORT 03 .14936. 12 B-31 PM Measurement Allowance Steering Committee LiveMeeting August 20, September 11, 2009

1. SwRI presented the Monte Carlo Simulation (See Presented at FTP website) 2. The m easurement a llowance w as de termined b ased on M ethod 2 us ing S ensors’ PPMD 3. The measurement allowance was 0.00605 g/hp- hr 4. Sensors’ PPMD was chosen for CE- CERT in

- use testing 5. It was agreed by the MASC that the AVL MSS will also participate in in

- use test ing along with the PPMD, just like it was done in the laboratory 6. The MASC requested the following from SwRI

a. plot the Sensors and Horiba 95 th
b. show the results of the simulation based on reference data available within the concentratio n range obtained in the laboratory delta on the same plot

c. plot the 95 th , 50 th , and 5 th
d. Refreshment on the regression rules for the validation deltas , for the validation

7. Address Item 6 above in a Livemeeting on September 11 8. The requests in Item 6 above were addressed in a livemee ting on S eptember 11 ( see presentation on FTP website)

a. The M ASC r equested t hat t he va lidation de ltas f or t he 95 th , 50 th , a nd 5 th

b. The LOESS fit should be done on Sensors’ validation deltas based on Method 1 and Method 2. be regressed using the LOESS f itting r ule s ince the c riteria s et f or l inear regression is not met.

c. SwRI should present the regression the CE- CERT September 22 meeting.REPORT 03 .14936. 12 B-32 PM Measurement Allowance Steering Committee Meeting February 17-18, 2010, US EPA, Ann Arbor

Sensors gave a presentation, proposing a correction factor for the exhaust flow.

**•** Sensors calibrated the flow meter in house with the same wrong pressure configuration in the field

**•** Sensors de termined a c orrection f actor of 1.52 for e xhaust f low c orrection for U nit 3. This correction factor was accepted by the MASC

**•** A s imilar c alibration will be done on Unit 2 to de termine the e xhaust f low c orrection factor to be used for this unit

Sensors i ndicated t hat a dditional c orrections r elated t o b ypass f low t hat w as not c orrected f or during the in

- use testing is needed:

**•** The sample flow during NTE is determined as the difference between bypass flow before sampling and bypass flow during sampling.

**•** If t he b ypass f low i s n ot c orrected f or ba rometric pr essure dur ing s ampling, a nd t he barometric ch anges during the long NTEs, then there would be an error introduced to the sample flow.

AVL di scussed t he a pproach t hey t ook t o c alculate t he br ake

- specific P M e missions us ing Method 2. The ECM broadcast fuel term in the equation was essentially frozen i f it went below 5% of the max fuel flow encountered during the test. However, to move forward with Method 2 calculation, the MASC agreed that to the following:

**•** The MAX fuel flow provided by the engine manufacturer should be used not the max fuel determine d during a test. Thus, the engine manufacturer of the CE

- CERT vehicle should provide the information to CE-CERT to do proper Method 2 calculation

**•** The fuel flow along with the gas concentration terms in the equation will be frozen if the ECM fuel flow dipped below 10% of the MAX fuel flow.

CE

- CERT gave a pr esentation on the different exhaust flow correction attempts they made. See CE

- CERT presentation at the FTP website.

SwRI pr esented va lidation f or P PMD U nit 2 and 3 us ing M ethod 1 with a n e xhaust f low

co rrection of 1.52. T he P PMD ba sed on M ethod 1 f ailed t he va lidation c riteria. S wRI a lso showed different scenarios for the AVL, including Method 1, 2, and 3. The presentation is listed on the FTP website.

Below is a summary of the action items for CE

- CERT and SwRI as a results of this meeting. These action items were compiled by Chris Laroo:

Follow- up Work to Finish PM MA Test Program

1) Agreed to exhaust flow correction factor of 1.52 for unit #3.REPORT 03 .14936. 12 B-33 2) Will use exhaust flow correction factor for unit #2 that comes out of Sensors check of the unit #2 flow meter on their flow stand (value anticipated to be at or near 1.52 but needs to be determined). Value is anticipated to be known the week of March 1 st

. 3) Correct PPMD data for incorrect setting in QCM bypass f low. S oftware setting change from 0 to 1.

4) Use Semtech DS barometric pressure readings to account for altitude change effects on data. If it is determined that the PPMD barometric pressure reading is f aulty, we will request t hat S ensors up grade t he s ensor t o t he q uality us ed i n t he D S. This c ould be deemed a special source of error and accounted for via a future hardware improvement.

5) PPMD Unit #1 data will be reported for single crystal use only to reflect that fact that the Mass S ensitivity w as i ncorre ctly e ntered into the s oftware dur ing te sting. T his s ingle crystal use data will then be pooled with the multi -crystal use results from units #2 and #3 to determine the final validation % results.

6) Report PPMD Unit #2 a nd #3 de ltas for single crystal use only as a pr obing exercise to see if multiple crystal use has an effect on mass loss. The intent of this exercise is to gauge the effectiveness of proposed fixes by Sensors to eliminate the PPMD low bias for future pilot and compliance program testing. T his data will be pooled with the unit #1 single crystal use results for plotting in the validation window for experimental purposes only. These results will not be used to determine validation.

7) Method #2, i f the fuel flow rate d rops below 10 % of manufacturer de clared maximum fuel rate value, then the ratio of the emission concentration terms and the ECM broadcast fuel rate will be frozen at that value.

8) CE

- CERT will reprocess all PPMD and MSS data with correct factors.

9) CE

- CERT w ill w rite a final r eport t o be reviewed by t he s teering committee be fore finalization.

10) SwRI w ill pl ace r evised CE

- CERT da ta i n v alidation w indows a nd c alculate ne w validation percentages. CE-CERT should have the data to SwRI by the end of March.

11) SwRI will also place revised CE -C ERT single crystal usage data in validation windows and calculate n ew validation percentages ( for experimental purposes only). T his a gain will help gauge whether or not there is an effect on mass loss from multiple crystal usage.

12) SwRI w ill w rite a f inal report t o be r eviewed by t he s teering committee be fore finalization.REPORT 03 .14936. 12 

### APPENDIX C

CRYSTAL BALL OUTPUT FILE DESCRIPTIONSREPORT 03 .14936. 12 C-1 EXTRACT DATA FILES

1.0 Simulation Variables

The simulation variables listed in Table B

- 1 we re extracted at the completion of the Monte Carlo simulation run for each reference N TE ev ent. C rystal Ball cl assifies va riables i nto two categories: assumptions and forecasts. Assumpt ions are the estimated inputs into the simulation model such as the variability indices used to sample each error surface. Assumption variables in this study are identified by an “i c ” at the beginning of the variable name, or by “Delta” at the beginning o f the variable name. The “i c ” variables are the simulation error model inputs such as “01_ic_SS_PM ”. The “Delta” variables serve as switches that turn a given error surface on or off in the model, e.g. “ Delta PM SS”. The Delta switch variables when turn ed on and off during a s imulation a re applied i n pos t

- simulation analysis to determine s ensitivity of results to the particular error surfaces. Forecasts are values calculated by a forecast formula in the spreadsheet cells. Examples of forecast variables used in this study are “ 001AVL_D ePM (g/hp

- hr), Method 1 ” and “ 005AVL_Valid D ePM (g/hp -hr), Method 2 ”.

TABLE B-1 . SIMULATION VARIAB LES

Variable Name Description 001AVL_DePM (g/hp

- hr), Method 1 MC Delta PM Method 1 for AVL PEMS 001Horiba_DePM (g/hp

- hr), Method 1 MC Delta PM Method 1 for Horiba PEMS 001Sensors_DePM (g/hp

- hr), Method 1 MC Delta PM Method 1 for Sensors PEMS 002AVL_DePM (g/hp

- hr), Method 2 MC Delta PM Method 2 for AVL PEMS 002Horiba_DePM (g/hp

- hr), Method 2 MC Delta PM Method 2 for Horiba PEMS 002Sensors_DePM (g/hp

- hr), Method 2 MC Delta PM Method 2 for Sensors PEMS 003AVL_DePM (g/hp

- hr), Method 3 MC Delta PM Method 3 for AVL PEMS
004AVL_Valid DePM (g/hp -hr), Method 1 Validation M C D elta P M M ethod 1 f or AVL PEMS 004Horiba_Valid DePM (g/hp

- hr), Method 1 Validation M C D elta P M M ethod 1 f or Horiba PEMS
004Sensors_Valid DePM (g/hp-hr), Method 1 Validation M C D elta P M M ethod 1 f or Sensors PEMS 005AVL_Valid DePM (g/hp -hr), Method 2 Validation M C D elta P M M ethod 2 f or AVL PEMS 005Horiba_118_Valid DePM (g/hp-hr), Method 2 Validation M C D elta P M M ethod 2 f or Horiba PEMS
005Sensors_Valid DePM (g/hp-hr), Method 2 Validation M C D elta P M M ethod 2 f or Sensors PEMS 006AVL_Valid DePM (g/hp -hr), Method 3 Validation MC D elta P M Method 3 f or AVL PEMS

01_ic_SS_PM Random Sampling Variability Index for SS PM Error Surface, applied to AVL, Horiba and Sensors PEMS

02_ic_TR_PM Random S ampling V ariability I ndex f or Transient P M E rror S urface, applied to AVL, Horiba and Sensors PEMS

04_ic_Atm.Pres_PM_AVL Random S ampling V ariability I ndex f or PM A tmospheric P ressure E rror S urface, applied to AVL PEMSREPORT 03 .14936. 12 C-2 05_ic_Amb.Temp_PM_AVL Random S ampling V ariability I ndex f or PM Temperature Error Surface, applied to AVL PEMS 07_ic_SS_CO Random Sampli ng Variability Index for SS CO 10_ic_Pressure_CO Random S ampling V ariability I ndex f or CO Pressure 11_ic_Temperature_CO Random S ampling V ariability I ndex f or CO Temperature 13_ic_SS_NMHC Random Sampling Variability Index for SS NMHC 14_ic_TR_NMHC Random S ampling V ariability I ndex f or Transient NMHC
16_ic_Pressure_NMHC Random S ampling V ariability I ndex f or NMHC Pressure 17_ic_Temperature_NMHC Random S ampling V ariability I ndex f or NMHC Temperature 19_ic_NMHC_Ambient Random S ampling V ariability Index f or Ambient NMHC
20_ic_SS_flow Random Sampling Variability Index for SS Exhaust Flow
21_ic_TR_Flowrate Random S ampling V ariability I ndex f or Transient Exhaust Flow
22_ic_Pulsation_flow Random S ampling V ariability I ndex f or Exhaust Flow Pulsation
23_ic_Swirl_flow Random S ampling V ariability I ndex f or Exhaust Flow Swirl
25_ic_Radiation_Exhaust Flow Random S ampling V ariability I ndex f or Exhaust Flow EMI/RFI Radiation
27_ic_Temperature_Exhaust Flow Random S ampling V ariability I ndex f or Exhaust Flow Temperature 28_ic_Pressure_Exhaust Flow Random S ampling V ariability I ndex f or Exhaust Flow Pressure
29_ic_TR_Torque Random S ampling V ariability I ndex f or Dynamic Torque
30_ic_Torque_DOE Random S ampling V ariability I ndex f or Torque Design of Experiments Testing
31_ic_Torque_Warm Random S ampling V ariability I ndex f or Torque Warm

- up

32_ic_Torque_IP Random S ampling V ariability I ndex f or Torque I ndependent P arameters H umidity and Fuel
34_ic_Torque_Interpolation Random S ampling V ariability I ndex f or Torque Interpolation
35_ic_Torque_Engine Manufacturers Random S ampling V ariability I ndex f or Torque Engine Manufacturers
42_ic_Fuel_Engine Manufacturers Random S ampling V ariability I ndex f or Fuel Engine Manufacturers
43_ic_TR_Speed Random S ampling V ariability Index f or Dynamic Speed 44_ic_TR_Fuel Rate Random S ampling V ariability I ndex f orREPORT 03 .14936. 12 C-3 Dynamic Fuel Rate
45_ic_SS_CO2 Random Sampling Variability Index for SS CO2 46_ic_TR_CO2 Random S ampling V ariability I ndex f or Transient CO2
49_ic_Temperature_CO2 Random Sampling V ariability I ndex f or CO2 Temperature

Delta PM SS Model s witch c ontrolling P M S S e rror surface application

Delta PM Transient Model s witch c ontrolling P M Transient error surface application
Delta PM Atmospheric Pressure Model switch controlling PM Atmospheric Pressure error surface application
Delta PM Ambient Temperature Model s witch c ontrolling P M A mbient Temperature error surface application

Delta CO SS Model s witch c ontrolling C O S S e rror surface application
Delta CO Atmospheric Pressure Model switch controlling CO Atmospheric Pressure error surface application
Delta CO Ambient Temperature Model s witch c ontrolling C O A mbient Temperature error surface application

Delta NMHC SS Model switch controlling NMHC SS error surface application

De lta NMHC Transient Model switch controlling NMHC Transient error surface application

Delta NMHC Atmospheric Pressure Model s witch c ontrolling N MHC Atmospheric P ressure er ror su rface application
Delta NMHC Ambient Temperature Model switch controlling NMHC Ambient Temperature error surface application

Delta Ambient NMHC Model switch controlling Ambient NMHC error surface application

Delta Exhaust Flow SS Model switch controlling Exhaust Flow SS error surface application

Delta Exhaust Flow Transient Model switch c ontrolling E xhaust F low Transient error surface application

Delta Exhaust Flow Pulsation Model s witch c ontrolling E xhaust F low Pulsation error surface application

Delta Exhaust Flow Swirl Model s witch c ontrolling E xhaust F low Swirl error surface application

Delta Exhaust EMI/RFI Model switch controlling Exhaust EMI/RFI error surface application
Delta Exhaust Temperature Model s witch c ontrolling E xhaust Temperature error surface application

Delta Exhaust Pressure Model switch controlling Exhaus t Pressure error surface application
Delta Dynamic Torque Model switch controlling Dynamic Torque error surface application
Delta Torque DOE Testing Model s witch c ontrolling T orque D OE Testing error surface application Delta Torque Warm

- up Model switch controlling Torque Warm

- upREPORT 03 .14936. 12 C-4 error surface application
Delta Torque Humidity Model switch controlling Torque Humidity error surface application
Delta Torque Interpolation Model s witch c ontrolling T orque Interpolation error surface application
Delta Torqu e Engine Manuf Model s witch c ontrolling Torque ( Engine Manufacturer) error surface application
Delta Fuel Engine Manuf Model s witch c ontrolling F uel ( Engine Manufacturer) error surface application
Delta Dynamic Speed Model s witch c ontrolling D ynamic Speed error surface application
Delta Dynamic Fuel Rate Model s witch c ontrolling Dynamic F uel Rate error surface application

Delta CO2 SS Model s witch controlling C O2 S S error surface application

Delta CO2 Transient Model s witch c ontrolling C O2 Transien t error surface application
Delta CO2 Ambient Temperature Model s witch controlling C O2 A mbient Temperature error surface application

2.0 Statistics

Descriptive statistics summarizing the values obtained during a single reference NTE event simulation are provided in Table B -2.

TABLE B-2 . DESCRIPTIVE STATISTICS FOR SIMULATION VARIABLES

Statistic Definition Trials Number of times the simulation was repeated Mean Arithmetic average Median The value midway b etween the s mallest value a nd the largest value Mode Value that occurs most often Standard Deviation Measurement of variability of a distribution. The square root of the variance Variance The a verage of t he s quares of t he de viations of a num ber of values from their mean Skewness A measure of the degree of deviation of a distribution from the norm of a symmetric distribution Kurtosis A measure of the degree of peakedness of a distribution Coefficient of Variability Standard deviation/Mean Minimum Smallest value Maximum Largest value Range Width Largest value

– smallest value Mean Standard Error Standard deviation of the distribution of possible sample meansREPORT 03 .14936. 12 C-5 3.0 Percentiles

Percentiles a re t he probability of achieving va lues be low a pa rticular percentage i n the following increments: 0, 5, 10, 15, 20, 25, 30, 35 , 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, and 100%. Percentiles are computed for each of the simulation var iables described in Table

.

4.0 Sensitivity Data

Sensitivity da ta a re pr ovided b y c omputing t he r ank c orrelation c oefficient f or a ll e rror surfaces an d all simulation variables. T he EXTRACT data file contains the absolute value of the rank correlation. In post

- simulation processing, values of control variable Delta PM SS in the simulation results were applied to dichotomize the data. 5.0 Trial Values

The value for all simulation variables is provided at each trial of the simulation.

REPORT FILES

1.0 Report Summary

This s ection includes th e s imulation start d ate and time, stop date a nd time , number o f trials run, sampling type (Monte Carlo), random seed used, and run statistics.

2.0 Forecasts

Descriptive s tatistics, percentiles, and a frequency histogram a re p rovided for forecast variables 001AVL_D ePM (g/hp -hr), Method 1 through 006AVL_Valid DePM (g/hp-hr), Method 3 (see Table ).

3.0 Assumptions

Descriptive s tatistics, percentiles, distribution parameters, and a di stribution chart a re provided for assumption variables 01_ic_SS_PM through 49_ic_Temperature_CO2 (see Table ).

4.0 Sensitivity Charts

Sensitivity charts are provided for forecast variables 001AVL_D ePM (g/ hp

- hr), Method 1 through 006AVL_Valid D ePM ( g/hp

- hr), M ethod 3 (see Table ). C rystal Ball cal culates sensitivity by computing rank correlation coefficients between every assumption (error surface) and forecast (delta BSPM emissions) while the simulation is running. Positive rank correlations indicate that an increase in the assumption is associated with an increase in the forecast. The larger t he absolute value of the rank correlation the stronger the relationship.

The sensitivity charts developed during the MC simulation are displayed as ‘Contribution to Variance” ch arts w hich are cal culated by s quaring the r ank correlation coefficients f or a ll assumptions used in a particular forecast and then normalizing them to 100%. Figure displays a sensitivity c hart f or t he A VL delta P M M ethod #1. T he a ssumptions w ith t he hi ghest contribution t o va riance ( in a bsolute va lue) are plotted a t t he t op of t he c hart. T his e xampleREPORT 03 .14936. 12 C-6 shows a typical dominant effect of the PM SS error surface represented by the 79.5% negative effect of Delta PM SS. As seen in the example in Figure B -1 , as you increase the SS PM there is an increase i n t he de lta P M M ethod #1 values r epresented by t he 7.6 % pos itive ef fect o f ic_SS_PM, and as you increase the torque warm

- up there is a decrease in the delta PM Method #1 va lues r epresented b y the 2.8 % ne gative e ffect of i c_Torque_Warm. O nly t he t op e ight assumptions are plotted in this sensitivity chart.

FIGURE B-1 . SENSITIVITY CHART FOR AVL DELTA PM MET HOD 1REPORT 03 .14936. 12 

### A PPENDIX D

MONTE CARLO SPREADSH EET COMPUTATIONSREPORT 03 .14936. 12 D-1 1.0 DESCRIPTION OF ASSUM PTIONS

The following assumptions were made in running the Monte Carlo model:

**•** Only on e r eference N TE e vent can b e r un at a t ime t hrough t he M onte Carlo simulation workbook . However, NTE event cases can be stacked and run in a batch mode.

**•** Uniform (1 second in duration) time steps are used in the reference NTE events.

**•** Standard f ormat and e ngineering uni ts f or r eference N TE da ta established f or t he project are observed, and applied to the reference NTE event before the NTE event is entered in the Error Model workbook for Monte Carlo simulation.

**•** Any we t – dry m atter c onversions, i f not ne gligible, h ave b een p erformed on t he appropriate reference NTE event values before the reference NTE event was entered in the Error Model workbook for Monte Carlo simulation. No wet – dry conversions are performed in the workbook.

**•** Any reference NTE event normalizations to produce similar emission s brake- specific

res ults f rom the thr ee e mission s calculation m ethods ha ve be en a ppropriately performed before the reference NTE event was entered in the Error Model workbook for M onte C arlo s imulation. N o no rmalizations a mong t he t hree methods a re performed in the workbook.

**•** PM e mission s models f or t hree c alculation m ethods a re c omputed f or t he A VL PEMS. O nly calculation methods 1 a nd 2 are computed for the Horiba and Sensors PEMS.

**•** Error surface models and supporting data were approved by the Steering Committee

.

**•** The error model spreadsheet has been correctly implemented, and its interaction with Monte Carlo tools like Crystal Ball is correctly understood.

**•** Random number generation by a Monte Carlo tool like Crystal Ball is correct.

**•** Convergence of the completed MC simulation was processed and checked outside of the simulation workbook. B enchmark checks on the convergence calculations were made using a SAS **** computer program.

2.0 WORKSHEET DESCRIPTIONS

2.1 Macro Description

The M acro c an be vi ewed i n t he E xcel s preadsheet ‘ Batch C ontrol’ with t he m enu selections Tools>Macros>Macro1>Edit. T he purpose of Macro1 is to control NTE event batch processing of stacked cases. For each NTE event case processed, the macro expedites clear ing extra cells below the reference NTE e vent in the spreadsheet ‘Error Model’ Methods worksheetREPORT 03 .14936. 12 D-2 and delete s extra rows i n the D elta error w orksheet s. T he m acro a lso pe rforms M ode 0 calculation s and s tores r esultant ' ideal e missions' va lues for a pplica tion i n s ubsequent M onte Carlo simulation

. The user must begin with the starter version of the ‘Error Model’ Excel file which has 300 rows of equations in columns X - CF and in rows 52 - 351 in the Methods worksheet. The starter spreadsheet a lso has 300 rows of equations below charts in columns B – F, or B – L, in applicable Delta worksheets. The us er (when n ot unde r automatic b atch c ontrol) c op ies the reference NTE e vent into columns A – V, row 52 and down , in the Methods worksheet

. It is can then be c onfirm ed that cell J45 in the Methods worksheet displays the correct number of rows of the reference NTE event.

Macro e xecution c an be a ccomplished t hrough t he m enu s elections Tools>Macros>Macro1>Run. Note that t his macro clears cel ls w ithout de leting rows i n t he Methods worksheet , and deletes rows in the Delta worksheet s. T his macro will not work if the reference NTE e vent has only one row. For a reference NTE e vent with exactly two rows, this macro will c orrupt the second " check" values i n columns B- F t ype Delta w orksheets. C heck values are not used in the simulation, but are provided as a diagnostic aid

. Apply th e macro for reference NTE events with no more than 300 rows.

The reader can follow the description of execution that follows by viewing the macro and observing the comment rows provided throughout the macro text. In execution, the macro first reads the contents of J45 in the Methods worksheet. It uses the number of rows in the reference NTE event defined by J45 to de termine how many rows to clear and delete in the spreadsheet. It checks that the number of rows is between 2 and 299, inclusive. It will also execute correctly for 300 rows.

Next, the macro clears cell contents in columns X – CF below the reference NTE event in the M ethods w orksheet. N ote t he m acro, as w ritten, will not ex ecute properly i f t he s tarter spreadsheet has been revised with row insertion or deletion in certain areas of the spreadsheet. As written, the macro initiates in cell X52, counts down through the NTE Event rows, and clears contents in the range from there in column X through cell CF351.

Next, the macro deletes extra rows below the reference NTE event, where applicable, for example in Delta worksheet 07. For Delta worksheet 07 it initiates in cell B79 and counts down through the rows of the reference NTE event to the first row to be deleted. It selects the range of rows from there down through row 378, deletes the rows, copies some equations and a value to the last row in the ran ge the charts use, and returns the cursor to c ell F68 leaving the display more or less centered on the charts in the worksheet.

Subsequently, the macro performs similar operations in other Delta worksheets; however, the initiating cell and final row diff er among the worksheets. The Delta worksheets processed in this way are 7, 10, 11, 16, 17, 20, 21, 22, 23, 29, 30, 43, 44, 45, 46 and 49.

Following t he r ow d eletion ope rations i n t he D elta w orksheets, or di rectly w hen t he reference N TE e vent ha s 300 r ows, t he m acro pr epares f or t he M ode 0 ( ideal e missions) calculation. F irst, i n t he M ethods w orksheet i t c opies t he e quations i n r ow 52, c olumns X through CF, to the last row in the reference NTE event. This clears any errors introduced in theREPORT 03 .14936. 12 D-3 last row; however, it assumes that row 52 is correct. The last cell in column AC ( **∆** t) is cleared for aesthetics, since the **∆** t values are not applied in the model calculations.

The Mode 0 c alculation is performed by the macro by changing the value in cell A6 of the Summary worksheet t o 0. T hen i n t he M ethods w orksheet, t he va lues f rom c ells C U22 through CU30 are pasted (values only) to cells O22 through O30 where they are referenced by formulas during Monte Carlo simulation. T he macro changes the value of A6 in the Summary worksheet to 2 in preparation for the Monte Carlo simulation, and moves the cursor to cell CT18 of the Methods worksheet.

Additional c omments r egarding t he m acro ope ration a re presen ted in the f ollowing section descriptions of the model spreadsh eet.

2.2 Worksheet 1: ErrorControl

The E rrorControl w orksheet of t he E rror M odel w orkbook i mplements 31 logic s witch functions. T he user enters a numerical “1” in column A D in each row corresponding to error surfaces t o be i ncluded in the cal culation. A n umerical “0 ” i s appl ied to error s urfaces t o be excluded i n t he c alculation. Corresponding r andom va riables f or e rror

- surface on

- off s witch random effects sensitivity modeling are implemented in column W under Crystal Ball control.

Error s urfaces are num bered 1 t hrough 49 . T he num bered e rror s urfaces a re de fined i n columns A – C, and information pertinent to their usage is presented in columns E – V of the worksheet. Column E displays warning messages when an unusual value is monitored in column D.

The c ontrol s witch e lements i n t he w orksheet a re de liberately pl aced on r ows i n t he worksheet c orresponding t o t he e rror s urfaces t o e xpedite e quation c hecking i n t he M ethod s worksheet where the control switch variables are applied in conjunction with error surfaces from the correspondingly numbered “Delta” worksheets

The numbered error surfaces and time alignment controls that have been implemented are defined in the follo wing Table C -1.REPORT 03 .14936. 12 D-4 TABLE C-1 . ERROR SURFACES USED IN SIMULATION

Component No. Error Surface Delta PM 1 Delta PM SS 2 Delta PM Transient 4 Delta PM Atmospheric Pressure 5 Delta PM Ambient Temperature Delta CO 7 Delta CO SS 10 Delta CO Atmospheric Pressure 11 Delta CO Ambient Temperature Delta NMHC NMHC = 0.98*THC 13 Delta NMHC SS 14 Delta NMHC Transient 16 Delta NMHC Atmospheric Pressure 17 Delta NMHC Ambient Temperature 19 Delta Ambient NMHC Delta Exhaust Flow 20 Delta Exhaust Flow SS 21 Delta Exhaust Flow Transient 22 Delta Exhaust Flow Pulsation 23 Delta Exhaust Flow Swirl 25 Delta Exhaust EMI/RFI 27 Delta Exhaust Temperature 28 Delta Exhaust Pressure Delta Torque 29 Delta Dynamic Torque 30 Delta Torque DOE Testing (Interacting Parameters Test) 31 Delta Torque Warm

- up(Interacting Parameters Test) 32 Delta Torque Humidity / Fuel(Independent Parameters Test) 34 Delta Torque Interpolation 35 Delta Torque Engine Manuf acturers Delta Fuel 42 Delta Fuel Engine Manufacturers Delta Speed 43 Delta Dynamic Speed Delta Fuel Rate 44 Delta Dynamic Fuel Rate Delta CO 45 2 Delta CO 2 SS
46 Delta CO 2 Transient
49 Delta CO 2 Ambient Temperature

The thirty

- one (31 ) error surfaces that have been implemented are included or excluded by the controls numbered 1 – 49 identified in Table . When all 31 error controls are on (included in calculation), the sum of column D in the worksheet ErrorControl is 31.

2.3 Worksheet 2: Summary

The Summary worksheet in the Error Model workbook comprises input mode control i n rows 4 – 10 and output summary in rows 8 8 and 119

. Other rows in this worksheet are available for diagnostic purposes.REPORT 03 .14936. 12 D-5 The c alculation m ode c ontrol i s a ccomplished with c ell A 6 w here t he us er nor mally confirms that a numerical value of “2” is designated. M ode 2 de signates emission s calculation with all errors applied. Mode 1 corresponds to a calculation of emissions with all errors appl ied except environmental errors . Mode 0 designates an “ideal” emission s calculation with no errors applied . In Monte Carlo error model si mulation performed in this study Mode 2 w as used.

Mode 0 i s used prior to Monte Carlo simulation to generat e the “ideal” emissions for a given reference NTE e vent. The Mode 0 values are calculated by entering a value of “ 0” in cell A6. T he M ode 0 c alculation a n d subsequent s toring of the “i deal” em ission s resul ts ma y b e accomplished m anually (as de scribed a bove) or by ex ercising a pr ovided macro . T he m acro automatically sets the value in cell A6 to zero, calculates and saves the “ideal” emission s values, and returns the value in A6 to “2” in preparation for the Monte Carlo simulation. The locations where the reference NTE e vent must be entered manually, and the locations where the “ideal” emissions must be saved (done automatically if the macro i s used) are described in the Methods worksheet section.

Mode 1 in cell A6 is not typically used but can be applied for diagnostic purposes.

The out put s ummary s ection of t he S ummary worksheet i n r ows 88 a nd 119 presents numerically and descriptively lab eled outputs of the emissions and emissions error calculations. In the output summary, the cells that are highlighted in turquoise color are designated by Crystal Ball as “Forecast” (or output) random variables.

A total of 14 outputs (“Forecasts”) a re designated in the Summary worksheet rows 88 and 119 covering t he num ber of out put va lues f rom PM e mission , t hree c alculation m ethods (Methods 1, 2 and 3) for the AVL PEMS and two methods (Methods 1 and 2) for the Horiba and Sensors P EMS , for t he ful l e rro r m odel a nd f or t he va lidation m odel (designated Valid in Summary worksheet variable labels). All of these “Forecasts” are provided in units of grams/hp

hr. This variety of calculations was accomplished with the Methods worksheet.

2.4 Worksheet 3: Metho ds

The Methods worksheet of the Error Model workbook comprises the following areas:

**•** Notes and diagnostic guides are located principally in rows 1 – 21 in columns A – CF, continuing on row 5 through column DD.

**•** Reference NTE event d ata a re l ocated i n row s 35 - 351 of columns A – W. A ctual reference NTE e vent d ata m ust be ent ered manually (or aut omatically under ba tch control) starting on r ow 52 i n columns A – V. Cell W52 data must be entered, and is provided f or s pecial c ase s tudy w here t he M ethod 3 f lo w- weighted PM conc entration may di ffer f rom M ethods 1 a nd 2. One t o 300 rows of reference NTE e vent da ta are allowed. Uniform ( one s econd i nterval) time steps ar e as sumed r epresented by t he reference NTE data.

**•** Parameters calculated are located in rows 35 – 351 of columns X – CF. The number of rows of these parameter equations must match the number of rows in the reference NTEREPORT 03 .14936. 12 D-6 e vent. Excess cells in these columns may be cleared manually or automatically during execution of the macro.

**•** Mode 0, Ide al Emissions for this reference NTE event are stored in column O rows 22 – 30 (either manually or automatically by the macro). Related data on the same rows are located in columns CT – DD.

**•** Input i c random variable distributions (Crystal Ball uses the terminology “Assumptions” for these inputs) are located in rows 26 – 32 of columns AG - CC

.

**•** Emission s calculations by three methods are located in rows 6 - 81 of columns CH – CJ (Method 1) , C L – CN (M e thod 2) a nd C P (Method 3) . T his part of t he w orksheet calculates full model and validation model.

2.5 Methods Worksheet: Notes and Diagnostic Guide

In r ows 1 – 22 for columns A – CF, several descriptive labels and references are defined for us e i n n avigating t hrough t he w orksheet. R ow 5 , column s A

- DD
, contain s column identification num bers r eferenced i n r ows 7 t hrough 22 ( depending on t he c olumn). F or example, in column H the values 65 on row 8 indicate s that the values in column H (rows 52 and follow ing rows) are a pplied i n c olumn 65 ( BM) labeled on r ow 5. If t he us er s crolls t o c ell BM52 it is observed that the spreadsheet formula in the cell refer s to values from column H. The information in the notes and diagnostic guide was not applied by the sp readsheet in any of t he emissions c alculations. It w a s i ncluded w ith the intent to simplify di agnostics b y pr oviding informati on on l ocations w here spreadsheet va lues w ere applied elsewhere i n the s preadsheet. Outside the a reas indicated above, some other n otes, comments and di agnostic guides may be found in other areas of the spreadsheet.

2.5.1 Methods Worksheet: Reference NTE Event

The reference NTE e vent used in the simu lation was entered in rows 35

- 351 of columns A – W. A ctual reference N TE e vent data m ust be e ntered m anually s tarting on r ow 52 i n columns A – V. Cell W52 data must be entered, and is provided for special case study where the Method 3 f low

- weighted PM concentration may differ from Methods 1 and 2. A minimum of one and a maximum of 300 rows of reference NTE e vent data are allowe d. Equal time steps ( 1 second intervals) are as sumed in the reference NTE d ata r ows . T he s tandard f ormat a nd engineering uni ts of reference NTE event da ta e stablished f or t his pr oject m ust be obs erved. These are described in the column headings on rows 47 – 51, columns A – V.

2.5.2 Methods Worksheet: Parameters

Parameters applied in the t hree emissions m ethods ar e c alculated in rows 35 – 351 of columns X – CF. The number of rows of these parameter equations must match the number of rows i n t he reference NTE e vent. E xcess cel ls i n these col umns m ay b e cl eared manually o r automatically during execution of the macro.

The formulas applied in rows 52 a nd down in columns X – CF have been produced by normal edit

- copy (typically of row 52 in these columns) and edit

- paste to rows 53 and followingREPORT 03 .14936. 12 D-7 rows in these columns. The **∆** t values displayed in column AC are not used in any calculation, but are displayed so a user can confirm uniform reference NTE e vent time sampling. T he last cell in column AC can be cleared (done automatically by the macro). Note that excess cells in these c olumns m ust be cleared, a nd r ow de letion ope rations s hould not be a pplied s ince t his would affect other areas in the Methods worksheet.

Certain s ums a re pe rformed i n s everal c olumns ove r t he Parameter r ows (range of t he reference NTE event). These are accomp lished in row 46 in columns AI, AW, AX, BO, BQ, BU and CA. Flow

- weighted PM concentration SS and TR errors are consolidated, with and without environmental errors, i n t he area of c ells A N40:AB47. Certain constants appl ied in the calculation are s tored i n c ells A W42, BC42, BI42, BP40 and BP42. O ther c onstants or conversion f actors a re i ncorporated num erically i n s preadsheet f or mulas. T ypical of the se is “0.01” to convert a percentage to a fraction.

Specific parameters or variables are calculated in the various columns for application in all three methods, full model and validation model. Table C -2 lists the parameters used in the Methods w orksheet, t he c olumns w here t hey are c omputed a nd a br ief de scription of t he parameters.

TABLE C-2 . METHODS WORKSHEET PARAMETER COLUMN DESCRIPTIONS

Methods Worksheet Parameters Column Descriptions Subject Column Description Engine operating state percentages X

– AB Convert NTE Event variables to percentages: speed, torque, fuel rate, exhaust flow

**∆** Time AC Displays **∆** t between NTE Event rows NMHC AD Calculate NMHC ppm as 0.98 of THC ppm Fuel Rate AE Calculate fuel rate g/s based on fuel density of 851 g/L Exhaust Flow Calculations AF Convert exhaust flow SCFM to mol/s AG Sum exhaust flow errors from Delta tabs 20, 21, 22, 23, 25 27 and 28 expressed in % of mol/s maximum. Respective ErrorControl tab switches are applied. AH Convert the total exhaust flow error in % of maximum mol/s to mol/s AI Add the mol/s exhaust flow error to the exhaust flow in mol/s. Mode control logic is applied. Speed with error AJ Add engine speed error from Delta tab 43 expressed as % of engine r ange converted to engine speed in rpm. Mode control logic and ErrorControl switch are applied. Fuel rate with error AK Combine Delta tab 42 Fuel (engine manufacturer) with fuel rate from D elta tab 44 expressed a s % o f maximum fuel r ate converted t o g /s t o en gine fuel r ate i n g/s. Mode control logic and ErrorControl switch are applied. Torque AL Sum torque errors from Del ta tabs 29, 30, 31, 32, 34 expressed as % of peak torque, and from Delta tab 35 e xpressed as % of NTE point torque converted to % of peak torque. ErrorControl switches are applied. AM Add the total torque error expressed as % of peak torque converted to N·m to engine torque in N·m. Mode control logic is applied. PM, μg/mol AN42, AN43 Sum P M SS a nd TR e rrors from D elta t abs 1 a nd 2 f or AVL P EMS e xpressed a s μg/mol, AN 42 for Methods 1 and 2, AN43 for Method 3. ErrorControl switches are applied. AO42 Sum PM SS and TR errors from Delta tabs 1 and 2 for Horiba PEMS expressed as μg/mol, for Methods 1 and 2. ErrorControl switches are applied. AP42 Sum PM SS and TR errors from Delta tabs 1 and 2 for Sensors PEMS expressed as μg/mol, for Methods 1 and 2. ErrorControl switches are applied. AQ42, AQ43 Sum AVL f low

- weighted P M concentration to a ll e rrors e xcept environmental P M errors, A Q 42 f or M etho ds 1 a nd 2, A Q43 f or M ethod 3. M ode c ontrol l ogic i s applied. AR42 Sum Horiba flow

- weighted PM concentration to all errors except environmental PMREPORT 03 .14936. 12 D-8 errors. Mode control logic is applied. AS42 Sum Sensors flow

- weighted PM concentration to all errors except environmental PM errors. Mode control logic is applied. AQ46, AQ47 Sum AVL flow

- weighted PM concentration to all errors including environmental PM errors Delta tabs 4 a nd 5, AQ 46 for Methods 1 and 2, AQ47 for Method 3. M ode control logic is app lied. AR46 Same formula as AR42 as no environmental PM error model applied to Horiba flow

weighted PM concentration. AS46 Same formula as AS42 as no environmental PM error model applied to Sensors flow

weighted PM concentration. Speed

· Torque AW Form product of Speed (rpm, all errors case, column AJ) and Torque (N

· m, all errors case, column AM) for application in Methods 1 a nd 3. C onvert rpm to radians/sec with 2 **π** radians/revolution, m inutes t o s econds w ith 60s ec/min, N

· m/sec to w att h r with 3600Joule s/watt hr, and watt to kw with 1000w/kw. AX Form product of Speed (rpm, no errors for validation case, column O) and Torque (N

· m, no e rrors for validation case, column T) for application in Methods 1 a nd 3. Convert r pm to r adians/sec w ith 2 **π** radians/revolution, m inutes t o s econds w ith 60sec/min, N

· m/sec to watt h r w ith 3 600Joules/watt h r, a nd w att to k w with 1000w/kw. CO and **∆** CO, % AY Sum environmental CO errors including errors from Delta tabs 10 and 11. AZ Sum other CO errors. Error from Delta tab 7 is the only one developed. BA Add the total CO errors expressed as % to engine CO in %. Mode control logic is applied. NMHC and **∆** NMHC, ppm BE Sum environmental NMHC errors including errors from Delta tabs 16, 17 and 19.
BF Sum other NMHC errors including errors from Delta tabs 13 and 14. BG Add t he t otal N MHC e rrors e xpressed a s ppm t o e ngine N MHC i n PPM. M ode control logic is applied.
CO 2 and **∆** CO 2 BK , % Sum environmental CO 2 errors. Error from Delta tab 49 is the only one developed.
BL Sum other CO 2 errors including errors from Delta tabs 45 and 46.
BM Add the total CO 2 errors expressed as % to engine CO 2 in %. Mode control logic is applied. Exhaust Flow

· [ NMHC + ( CO +CO 2 BO ) ] / [fuel mass flow rate / Speed

· Torque ] Form product of hydrocarbons fraction plus CO and CO 2 fractions (all errors case, columns BG, BA and BM) and exhaust flow (mol/s, column AI) divided by ratio of fuel r ate ( g/s, al l er rors c ase, co lumn AK) t o s peed

- torque p roduct ( all er rors cas e, column AW) fo r application in PM Method 2. BQ Form product of NMHC fraction plus CO and CO 2 fractions (all errors case, columns BG, BA and BM) and exhaust flow (mol/s, column AI) divided by ratio of fuel rate (g/s, no errors case, column AE) to speed

- torque product (no errors case, column AX) for application in PM Method 2 validation. NMHC + ( CO+CO 2 BS ) Form sum o f NMHC fraction plus CO and CO 2 fractions ( all er rors cas e, co lumns BG, BA and BM) for application in Method 3. Fuel R ate / [ N MHC + ( CO+CO 2 BU ) ] Form quotient, Fuel Rate (g/s, all errors case, column AK) divided by sum of NMHC fraction plus C O a nd C O 2 fractions ( all er rors cas e, co lumn B S) f or ap plication i n Method 3. Fuel R ate / [ N MHC + ( CO+CO 2 CA ) ] Form quotient, Fuel Rate (g/s, no errors case, column AE) divided by sum of NMHC fraction plus C O a nd C O 2 fractions ( all er r ors c ase, c olumn B S) for a pplication i n Method 3 validation.

2.5.3 Methods Worksheet: Mode 0 Ideal Emissions

For the reference NTE e vent in rows 52 and down in columns A – V, an ideal emissions value must be calculated and stored for application in th e emissions difference calculations. The ideal cas e c an be cal culated either m anually

, or a utomatically b y the ma cro. F ollowing the calculation, the i deal v alues ar e s tored by edit

- copy e dit

- paste

- special

- values ope ration t o t he cells i n c olumn O , r ows 2 2 – 30 . T he m anual ope rations de scribed below ar e p erformed automatically by the macro, if executed, after manually entering the reference NTE event.REPORT 03 .14936. 12 D-9 After manually entering the reference NTE e vent to be simulated and checking that the number of rows of equations in the Parameters section matches the rows in the reference NTE

e vent, a num erical “0” can be ent er ed i n c ell A 6 of t he S ummary w orksheet. T he M ethods worksheet should h ave calculated M ode 0 r esults us ing t he reference NTE e vent. If e rror messages like “#VALUE or #DIV/0!” are displayed, there is probably still a m ismatch between the rows of the reference NTE e vent and Parameter equations. When calculated properly (with 0 in Summary A6), the values displayed in the Methods worksheet columns CU, CV and DB will be e qual on e ach of t he r ows 22 – 30 . T he va lues i n c olumn CX a re not yet e qual ( unless previously calculated and stored for this reference NTE e vent) be cause they reflect t he values stored i n M ethods worksheet column O , r ows 2 2 – 30 . T he ne xt m anual s tep i s t o edit

- copy column CU, rows 22 - 30 , and store the values by edit

- paste

- special

- values in column O, rows 22

- 30. Now in rows 22 - 30 the columns CU , CV, CX and DB should be equal. The final step is
to return to Summary w orksheet cell A6 and change the value from 0 t o 2. A t this point the spreadsheet c ould be r un i n M onte C arlo s imulation t o pr oduce pr operly s ampled va lues. Howev er, i f t he us er de sires t o m onitor charts pr ovided i n t he D elta worksheets dur ing t he simulation, f urther r ow

- matching to the reference NTE e vent i s r e quired in most of the D elta worksheets.

The manual operations described in the previous paragraph are intended to explain how the Mode 0 ideal e mission s are calculated and stored for use in the Monte Carlo simulation when **∆** emission s values a re calculated using th e ide al emission s r esults s tored i n O 22 – O30 . T he reference NTE e vent m ust be e ntered w ith an o peration s uch as a m anual edi t

- copy and e dit

paste or edi t

- paste

- special

- values op eration. T he m acro a utomatically performs t he m ode 0 calculation, stores the mode 0 results in O22 – O30 , and changes Summary A6 back to mode 2.

The macro also deletes extraneous rows from all the appropriate Delta worksheet s so the charts t herein di splay pr operly. It i s i mportant to c opy t he reference NTE e vent i nto a f ully ‘loaded’ starter file with equations filled on 300 rows in the Parameters area, and with full 300 row complement of equation

- rows in each of the appropr iate Delta worksheets for the macro to modify the spreadsheet properly.

2.5.4 Methods Worksheet: Input ic Random Variable Distributions

Probability distribution parameters are applied, and simulation trial values of the inputs are generated i n rows 2 6 – 32 o f c olumns A G - CC . R ows 26 a nd 27 a re us ed t o i nput distribution parameters . Rows 28 and 29 contain descriptive labels brought from the appropriate Delta w orksheet. R ow 30 i s a n i nformation

- only num ber

, row 31 c ontains t he na me l abel applied i n M o nte C arlo s imulation to the inpu t i c , and r ow 32 is w here t he M onte C arlo simulation tool places generated randomly

- sampled values during simulation. The values in row 32 are r eferenced by f or mula i n the r espective D elta w orksheets w here t hey are us ed fo r interpolation on the error surfaces.

The M onte C arlo t ool in Crystal B all us es the te rminology “Assumptions” f or the se inputs. Two distribution forms are applied: truncated normal (Gaussian), and discrete uniform. For t he nor mal di stribution, t he ap plied s tandard de viation i s i n row 27 . In C rystal Ball, t he standard deviation cell on row 27 and the label cell on row 31 were referenced by equation in the Crystal Ball a ssumption s etup w indow, t he m ean w as i nput as 0, a nd t he di stribution w as truncated at -1.414319083 and at +1. 414319083. Since all the truncated normal i c distributionsREPORT 03 .14936. 12 D-10 are identical (although the sampled trial v alues from each will be random in the Monte Carlo simulation), the Crystal Ball define

- copy data and define

- paste data operati ons were applied to define t he t runcated no rmal di stributions f or ot her i c variables on ce t he f irst one ha d be en defined.

For the discrete uniform distributions, the minimum d iscrete value (1 in all cas es) was applied i n r ow 26, t he m aximum di screte va lu e w as appl ied i n r ow 2 7 a nd t he ot her r ow descriptions ar e t he s ame as before. A gain, on e of t hese i nputs w as s etup w ith C rystal B all

“ define assumption ” and then applied with Crystal Ball define

- copy data and define

- paste data operations to other i c cell s on row 32 where a discrete uniform distribution wa s applied. When Crystal Ball “Assumptions” were defined, Crystal Ball colored each input cell bright green

During a M onte C arlo s imulation, t he M onte Carlo tool ( e.g. Crystal B all) pl aced a numerical value in each of the i c cells on row 32. Then the spreadsheet wa s exercised to perform int erpolations i n a ll t he D elta w orksheets. T he r esulting e rror s ample va lues f or t he e ntire reference NTE event wer e r eturned t o t he M ethods worksheet Pa ra meters ar ea, and then the Methods w orksheet Emission Calculations s ection computes **∆** emissions us ing t hree m ethods, full model and validation to generate one set of the 14 output values described in the Summary section. T he simulation tool stores the set of random input values from row 32 a s well as the output va lues i n a n Excel data b ase from w hich t he cor responding s ets of va lues can later b e extracted. Once each trial wa s completed, the simulation tool randomly sampled a second set of input values from the respective probability distributions, placed the values i n the cells on r ow 32, exercised the spreadsheet again, stored the input and output values, and went to a third trial, etc. Typically 4 0,000 to 65,000 trials, depending on the reference NTE event, were used in this project with this Error Model workbook.

Note t hat t here ar e t hree w ays t he us er c an control t he ef fect of the i c values in the emission s calculations:

Mode control in Summary A6, Include / exclude switches in ErrorControl column AD, and Specification of i nput r andom va riables ( “Assumptions”) a nd t heir pr obability distributions in the Methods worksheet row 32.

These three ways of controlling the i c values are independent , but the effects are i nterdependent as follows. M ode control determines what categories of errors are added into the calculations. Mode controls categories of errors are classified as :

1. Mode 0 - no errors included 2. Mode 1 - “all ” but ‘environmental’ errors included 3. Mode 2 - “all” erro rs added into the calculations

“All” in this context represent s those error surfaces turned on by the switches in the ErrorControl worksheet . T he i nput r andom va riable di stribution c ontrols t he di stribution of t he s ampled i c values applied du ring Monte Carlo simulation for the several D elta e rror surfaces. M ode and ErrorControl switches must be appropriately turned on for the effects of the sampled i c values to be i ncluded i n t he emissions di fference results. These controls af fect t he calculati ons i n t he Methods worksheet Parameters and Emission Calculations sections.REPORT 03 .14936. 12 D-11 2.5.5 Methods Worksheet: Emission Calculations

In t he ar ea of r ows 6 - 81 of c olumns C H – CP the brake

- specific emissions a nd **∆** emissions cal culations ar e pe rformed using t he va riables and parameters ge nerated in the Parameters s ection. T hree s ets of col umns , structured similarly, calculate the f ull mode l, validation model, time alignment and drift correction for the following methods:

1. Method 1 calculations are applied in columns CH – CJ,

2. Method 2 calculations are applied in columns CL – CN, and

3. Method 3 calculations are in columns CP.

Columns CH - CJ for Method 1 are typical of the methods where the structure is the same, but the formulas are a little different. Column CH performs the PM emission calculations for the AVL PEMS

, column CI performs for the Horiba PEMS and column CJ for the Sensors PEMS

. The structure of the three columns is the same. Formulas implemented in the three columns are the s ame, but t he equations i mplementing t he f ormulas a pply va riables a nd pa rameters appropriate to the respective PEMS

. As an example of the calculation for PM Method 1 we will examine column CH in detail. The full model calculation was accomplished in cells CH48 – CH5 4 . The ideal emissions result

wa s br ought i nto t he a rea b y e quation i n CH51. Full m odel P M emissions (e PM) in g/kw -hr

were calculated in CH54. C ells CH55 – CH59 are information

- only diagnostic aids. T he full model Method 1 result in CH54 is calculated by the formula s in Figure C-1.

PM 

### m is a flow weighted particulate matter exhaust concentration in g/mol



## ( ) **( ) ( )** 

# ∑ **∑**

**= =**  

###  ** **  ** **  ** ∆** 

**⋅ **  ** **  ** **  ** ∆**  ** **  ** **  ** **  ** **  ** **  ** ** 

**=⋅** N

i i i N

i i PM

PM 

### t mNT rpmSpeed t
s mol
n mol

g m hr kW ge

1 1



### 3600*1000*60 *14159.3*2** * *
/ 

FIGURE C -1. BRAKE

- SPECIFIC PM BY METHO D 1

In the formula for the full model mode 2, delta error values sampled from the Delta worksheet s 1 , 2, 4 a nd 5 ha ve be en a dded i n PM 

### m . S imilarly, de lta e rror va lues s ampled from D elta worksheets 20

- 23, 25, 27 and 28 have been added to the exhaust flow, delta errors sampled from worksheets 29

- 32 and 34- 35 were added to torque, and worksheet 43 deltas were added to speed. The **∆** t values are equal (1 second) and therefore cancel out of the equation.

The va lidation m odel c alculation was accomplished in the cel ls C H79 – CH8 1. Validation model PM emiss ions (g/kW

- hr) wa s calculated in cell CH81.REPORT 03 .14936. 12 D-12 Calculations for PM by Method 1 described above for the AVL PEMS in column CH are similar f or the H oriba a nd S ensors P EMS by Method 1 i n c olumns C I and C J, r espectively. Similar calculations for PM by Method 2 are presented in columns CL – CN, and by Method 3 in column CP.

2.6 Worksheet 4: Constants and Equations

The Constants&Eqns tab was strictly a snapshot of equations used in the brake

- specific emissions c alculations. It di splayed the e quations a nd c onstants i mplemented i n s preads heet formulas of the Methods worksheet

2.7 Worksheet 5: SS PM Error Surface

The 7 Delta C O SS worksheet is a t ypical D elta w orksheet . Its f unctional s tructure, formulas, charts and operation are very similar to the following worksheets:

**•** 20 Delta Exhaust Flow SS

**•** 22 Delta Exhaust Flow Pulsation

**•** 23 Delta Exhaust Flow Swirl

**•** 30 Delta Torque DOE Testing

**•** 45 Delta CO2 SS

**•** With minor changes in charts and structure, its function, formulas and ope ration are als o similar to the following worksheets:

**•** 1 AVL Delta PM SS

**•** 1 Horiba Delta PM SS

**•** 1 Sensors Delta PM SS

**•** 2 AVL Delta PM Transient

**•** 2 Horiba Delta PM Transient

**•** 2 Sensors Delta PM Transient

**•** 4 AVL Delta PM AtmosPressure

**•** 5 AVL Delta PM Ambient Temp

**•** 13 Delta NMHC SS

**•** 14 Delta NMHC Transient

**•** 19 Delta Ambient NMHC

**•** 21 Delta Exhaust Flow Transient

**•** 25 Delta Exhaust EMI- RFI

**•** 29 Delta Dynamic Torque

**•** 31 Delta Torque Warm- up

**•** 32 Delta Torque Humidity

**•** 34 Delta Torque Interpolation

**•** 35 Delta Torque Engine Manuf

**•** 42 Delta Fuel Engine Manuf

**•** 43 Delta Dynamic Speed

**•** 44 Delta Dynamic Fuel RateREPORT 03 .14936. 12 D-13 **•** 46 Delta CO2 Transient The following provides a brief summary of the 7 Delta CO SS worksheet:

**•** Rows 1 – 7 contain descriptive information about the error surface implemented in the worksheet.

**•** Rows 8 - 42 present the error surface in columns A – L. Other columns

, M – W, on these rows generate a lookup table used with an interpolation routine.

**•** Figures A, B and C follow.

**•** R ows 76 – 379 calculate t he **∆** C O S S e rror va lues for e ach row o f t he reference NTE

e vent. These values were returned to the Methods tab Parameters section.

The following paragraphs describe in further detail functions in the 7 Delta CO SS worksheet :

Data from the error surface (rows 13 – 42 , columns A – L, in this Delta worksheet ) must be e ntered i n s orted or der ( sorted on Lab N ominal c olumn C i n a scending or der) f or pr oper operation of t he x -lookup- interpolation f unction. T he three f igures chart t he e rror function. Figure A , in similar D elta ta bs, may pl ot several da ta s ets v er sus t he x

- value, Lab N ominal (column C ). F igure A y

- values are C O % Lab N ominal ( column C ), and m ay a lso pl ot 99 th percentile , 50 th percentile (median) and 1 st percentile.

Related error surface data are plotted in Figure B. Figure B plots several data sets versus the s ame x

- value, Lab Nominal ( column C ). F igure B y

- values ar e t he di fference, C O % (PEMS) – CO % (lab, nom). T he differences plotted may not correspond exactly to the values shown i n F igure A be cause of t he s tatistical pr ocedure a pplied i n calculating t he di fferences shown in Figure B. Figure B plots the 99 th percentile (column I), the 50 th percenti le (median) (column H) and the 1 st percentile (column G). In addition to the error surface data, Figure B also shows the interpolation line des ignated i c = xx (column V), and the reference NTE e vent values on the interpolation line (column F rows 80 through end of the reference NTE e vent versus Lab Nominal x

- values in column B rows 80 t hrough end of the reference NTE e vent). W hen i c = +1 .414319083 , the interpolation line plots on the 9 9 th percentile. When i c = 0, the interpolation line plots on the 50 th percentile. When i c = -1.414319083 , the interpolation line plots on the 1 st percentile. The reference NTE e vent always plots on the interpolation line, with points at the x

values in the reference NTE event.

The error surface data were also plotted in the format o f Figure C. Again the x

- axis was the same Lab nominal (column C). T his time the y

- axis data are the i c values. T hus, the 99 th percentile pl ots at +1 .414319083 , the 50 th percentile plots at 0 a nd the 1 st percentile plots at

- 1.414319083 . T he interpolation line plots at the value of i c , and the reference NTE e vent plots on the interpolation line at the x

- values in the reference NTE e vent. If appropriate value labels were di splayed i n F igure C , t he va lues w ould represent t he e rror s urface pl otted on a z

- axis above the two

- dimensional x

- y pl ane. These error s urface v alues a re di splayed graphically i n Figure B.

Now consider inner rows 13 – 41 in the look

- up table in columns T – W. Column T is a repetition of the x

- value from column C. Column U calculates a row

- to

- row **∆** for the x

- values in c olumn T f or us e i n i nterpolation. C olumn V c omputes t he i nterpolation l ine l inearly interpolated according to the value of i c between the median and the 99 th percentile if i c > 0 (on median if i c = 0 and on 99 th percentile if i c = +1 .414319083 ); and between the median and the 1 stREPORT 03 .14936. 12 D-14 percentile if i c < 0 (on median if i c = 0 and on 1 st percentile if i c = -1.414319083 ). Only one i c value (from cell E80) is applied in this calculation of the interpolation line. The Microsoft Excel vertical lookup function VLOOKUP is applied to the table in rows 12 – 42 in columns T – W. This is done in rows 80 and down in column F. B ecause of the way the VLOOKUP function operates, the first row cells T12 and V12, and the last row cell W42 (all three cells distinguished by darker line borders) contain formulas or values different from the formulas of the inner rows. The formula in cell T12 assures that the lookup function can always find an x

- value in its table. The formula in V12 and the value in W12 assure that the interpolation in cells F80 to the end of the reference NTE event data returns the nearest **∆** CO SS value on the interpolation line if t he x

value is outside the range of the error surface lab nominal values.

Before going t o t he i nterpolation a ccomplished i n F 80 a nd dow n, c onsider br iefly t he formulation on rows 12 – 43 in columns O – R. This formulation considers one x

- value from the

r eference NTE e vent, t he f irst one , i n c ell B80 and s elects t he t wo adjacent r ows i n t he e rror surface be tween w hich t o i nterpolate on t he B80 x

- value. T he r esult i s f ormed on r ow 43 i n these columns and then the “check” cell G80 accomplishes the i c contro lled interpolation. This provides an alternative calculation check on one row in the reference NTE event.

Now consider the interpolation for each point in the reference NTE e vent. C olumn B, row 80 a nd down, brings the lab nominal x

- value from the Metho ds worksheet reference NTE

e vent. For this D elta worksheet, that x

- value i s C O % (lab,nom). T he out -of- range flags ar e information

- only indicating points in the reference NTE e vent with x

- value out of the range of the error surface lab nominal. The i c v alue for this De lta worksheet was brought in to cell E80 from the Methods worksheet i c area. E ach point in the reference NTE event was interpolated with the same i c value, but with its own x

- value. Recalling that the interpolation line in column V was com puted with this one i c value, the x

- interpolation between the appropriate two adjacent rows in the error surface can now be accomplished. This requires using the x

- value on each row in c olumn B , B 80 a nd dow n, i n the V LOOKUP f unction, a nd pe rforming t he r e quired calculation using the looked-up values and deltas from the look-up table. The calculation is done with the formulas in cell F80 and down. The values computed in column F, cell F80 and down through the reference NTE e vent, could be considered elements of a column matrix or vector, and are returned to the Methods worksheet Parameters section.

In t he M onte C arlo s imulation, t he M ethods w orksheet combines t his reference NTE

e vent r esult ve ctor f rom t he 7 Delta C O SS worksheet with similar r esults f rom ot her e rror surfaces, calculates **∆** emissions by three methods, full model and validation to produce a set of 14 output values (“Forecasts” in Crystal Ball terminology) described in the Summary worksheet section. This was done having input i c values (including one i c value for this Delta C O SS) all chosen b y random s ample f rom t he a ppropriate t runcated no rmal or u niform di stribution as explained i n t he M ethods w orksheet section. T hen a nother s ample s et of randomly s ampled values w as input (only one i c value coming to this Delta function again). T he reference NTE event CO SS vector wa s recomputed with the one new i c value, returned to Methods worksheet and another set of 14 output values was produced. This process wa s repeated man y times until a statistical conve rgence criterion, d escribed i n S ection 2, w a s s atisfied. Typically, 40,000 t o 65 ,000 s ets of i nput va lues and 14 out put va lues w ere produced t o s atisfy t he c onvergence criterion with this Error Model spreadsheet.REPORT 03 .14936. 12 D-15 The nu mber of r ows i n t he D elta w orksheet r eference N TE e vent a rea ( rows 80 a nd down) s hould m atch t he num ber of r ows i n the reference NTE event applied i n t he M ethods worksheet for proper function of Figures B and C. The starter spreadsheet has been set up wit h the r ange of charted reference NTE e vent s eries e xtending t hrough r ow 379 i n t his D elta worksheet . T he ba lance of t he s preadsheet s ho uld calculate cor rectly w hen a r eference N TE

e vent is properly entered in the Methods tab and Parameters formulas proper ly aligned, although figures like B and C will not display properly until the last row of the reference NTE e vent is coincident with the end of the range of the charted reference NTE e vent series. T his could be done m anually i n each Delta w orksheet wh ere needed, ho wever, t he m acro wa s d esigned to convert t he fully ‘loaded’ s tarter w orkbook after t he reference NTE event wa s e ntered i n t he Methods worksheet . The macro uses the row count in the reference NTE e vent, al igns formulas in the Methods worksheet Pa rameters area, and eliminates extra rows in the reference NTE e vent

ar ea of each appropriate Delta worksheet . Again, the macro will do the operations correctly only on a fully ‘loaded’ starter workbook set up with 300 rows of formulas in the Methods works heet Parameter ar ea, and in each of the Delta worksheets using the reference NTE event.REPORT 03 .14936. 12 

### APPENDIX E

EMS OPERATION LOG1 REPORT 03 .14936.12 E-1 Date PEMS Description Reason Solution

2/25/2008 Horiba Lost communication with TRPM laptop on a regular basis Standard ethernet cable does not fit properly into this laptop A industrial grade ethernet cable was provided by Horiba which solved t his problem

2/28/2008 Horiba 1 Could not control dilution ratio if external flow meter is not connected Unknown Horiba fixed the problem

2/28/2008 Horiba 2 Java software does not display the measured values that are in the Labview software Unknown Software update fixed the problem

3/3/2008 Horiba 2 Dilution air flow not stable Dilution air pressure too high Performed a dilution air flow adjustment per Horiba

3/4/2008 Sensors 1 Lookup table for MPS2 repeatedly failed Unknown Sensors said the criteria is too s tringent and it is fine as long as it visually looks good

3/4/2008 Horiba 1 Dilution Ratio c ontrol is still somewhat erratic Bad PID

constants Adjusted PID constants to new values suggested by Horiba

3/14/2008 Horiba 1 Pressure transducer Pt1 would not respond to calibration Pressure transducer was broken Replaced with new part from Horiba

3/21/2008 Horiba 1,2,&3 Unable to pass Part 1065 sample flow linearity verification Sample flow is an inferred not measured value Linearity verification performed on dilution and total flow at MASC request

3/24/2008 Horiba 1,2,&3 Sample flow failed Horiba check with provided external flow

meter Flow calibrations needed updating Flow coefficients updated for all three units

3/26/2008 Horiba 1,2,&3 Unable to pass Part 1065 dilution flow linearity verification Dilution flow coefficients needed updating Performed "Dilution Ratio Accuracy Adjustment" as instructed by Horiba2 REPORT 03 .14936.12 E-2 Date PEMS Description Reason Solution

4/14/2008 Horiba 3 Java software freezes when loading Wrong parameters set in a config file AVL programmed an offset into their software to account for this, laterit appeared to be a grounding problem in the Semtech DS

4/22/2008 Horiba 3 Java softare is not communicating with the Labview software Improper software configuation Parameters adjusted to fix the problem

6/13/2008 Horiba
2200 Horiba system is unable to log the ISO-1576 ECM broadcast from the International Engine Software did not have this capability International engine wasreplaced with a heavyduty Volvo engine that uses J1939 broadcast

6/19/2008 Horiba 1 TRPM software not reading the same exhaust flow as OBS-2200 software Calibration coefficient is

wrong Manually adjust the calibration coefficient to get the readings to match

6/28/2008 Sensors 1 Sample valve for crystal remains in "transient" state every time it attempts to sample Stepper motor attempting to turn too quickly (not enough torque) Sensors readjusted the stepper motor speed

7/31/2008 Horiba 2 The OBS-2200 software would not trigger the OBS-TRPM to start sampling during an NTE Connector wired incorrectly Reduced the speed of the stepper motor

8/8/2008 Horiba 2 Dilution flow is too l ow Replaced with connector from second unit

8/12/2008 Horiba
2200 OBS-2200 software unable to read the reference torque value from J1939 broadcast Unable to enter the proper data bit location Perform dilution flow adjustment using internal pressure regulator

8/14/2008 Sensors 2 Could not get any of the crystals to os cillate Power supply in CQCM head was likely burnt out Enter the reference torque value manually

8/15/2008 Horiba
2200 OBS-2200 laptop would not boot up Unknown Hard drive was placed in another identical laptop, broken laptop was shipped back to Horiba3 REPORT 03 .14936.12 E-3 Date PEMS Description Reason Solution

8/18/2008 Sensors 1 Bypass flow was not updating

8/21/2008 Sensors 1 Crystal frequencies, corona currents, and voltages were dropping

out High temperature causing communication problems Attempted to reduce test cell temperature, but problem persisted

8/26/2008 AVL 1 Unit switched into Z ero Check in the middle of the test Uknown, possibly a pressure out of limit Unit switched back into sample after approx. 30 seconds

8/27/2008 Horiba 1 OBS-2200 laptop failed, would no longer boot Laptop broken New laptop provided by Horiba

8/27/2008 Horiba 1 OBS-2200 laptop would not read J1939 Unknown Sent back to Horiba for repair

8/27/2008 Horiba 1 TPA would not zero Zero function tied to gas analyzers, looking for gas flow Use the CAL function instead of the ZERO to just zero TPA

8/27/2008 Horiba 1 TRPM Java software unable to log exhaust flow (Labview works) Error in software Installed new version of Java software

8/27/2008 Horiba 1 EAD check w ill fail repeatedly Tolerances are too tight for test cell operation, tolerances relaxed. Ignored tolerances set in software for check

8/27/2008 AVL 1 Analog output signal to MSS would clip at 2 mg/m3 range Soot concentration too high Output switched to 0-10 mg/m3, D R setpoint increased from 3 to 6

8/27/2008 AVL 1 DR would oc casionally stop controlling Too much moisture in the system Unit purged for moisture overnight, and firmware upgraded

8/27/2008 Sensors 1 PPMD internal temperature was out of limit Test cell temperature too hot A new back panel was installed w ith two fans to promote cooling

8/27/2008 Sensors 1 Bypass flow from TSI flowmeter was not reading in software Faulty com cable Replaced TSI cable4 REPORT 03 .14936.12 E-4 Date PEMS Description Reason Solution

8/27/2008 Sensors 1 CQCM communication dropped out during testing CQCM power supply failed Power supply replaced

8/27/2008 Sensors 1 CQCM communication dropped out during testing Unknown CQCM comm chip replaced with a newer model

8/27/2008 Sensors 1 Corona needle high voltage erratic Unknown Fixing the comm issue resolved this problem

8/27/2008 Sensors 1 PPMD communication was dropping out High internal temperature Directed cooling air at
PPMD

9/10/2008 Horiba 1 Compressor tripped breaker, turned off AVL unit Combining AVL and compressor on same 15A circuit was too much current Horiba compressor moved to dedicated 15A circuit

9/10/2008 Horiba 1 Make up air flow was 0.7 lpm instead of 2.3 lpm PID constants incorrect Modified PID constants

9/12/2008 AVL 1 Analog output signal to MSS would s till br iefly clip a t 10 m g/m3 range Soot concentration too high Logarithmic analog output added as a firmware update

9/12/2008 Sensors 1 Negative emissions reported even at high emission levels on NTE High dilution, crystal saturation (no grease), crystal stabilization Do not use MPS2, grease crystals, wait longer for PPMD to warm up

9/18/2008 Sensors 1 The bypass flow would increase when crystal 1 on PPMD1 would sample Crystal may be installed backwards so that it is always sampling Set crystal 1 to be the reference crystal

9/18/2008 Sensors 1 Sample flow dr ifting during steady state engine operation Temperature estimate is based on mixing of two flows, if the estimate is off temperature will change the flow Adjusted the parameters in the temperature estimate5 REPORT 03 .14936.12 E-5 Date PEMS Description Reason Solution

9/18/2008 Sensors 1 No corona current was measured for two crystals Crystals were shorted to ground Crystals were replaced

9/18/2008 Sensors 1 High voltage reading was low on two crystals Corona needles were too close to the crystals Needles repositioned

9/20/2008 Sensors 1 Sample flow

measurement was too low Sample flow TC was in excess flow return, which had low flow due to connection to CVS TC moved into t he main exhaust

9/26/2008 AVL 1 Error: MFD Temperature out of Spec Test cell temperature too hot Repos itioned c hiller air to blow on the AVL unit

9/30/2008 Sensors 1 Lower than expected emissions for the PPMD Inaccurate sample flow temp c ausing low sample flow

meas, corona needles not positioned properly Repos ition the sample flow thermocouple, adjust corona needle

10/9/2008 Sensors 1 PPMD sampling de layed several seconds after trigger Delay in communications Increased residcence time inside PPMD up to 3 seconds

10/9/2008 Sensors 1 Inaccuracies in flow

measurement System uses an assumed inlet pressure for MPS2 Added pressure measurement downstream of MPS1 to account for changes in pressure due to MPS2

10/9/2008 Sensors 1 Crystal sampling as soon as it becomes available, including in an NTE event Software logic Software modified so that an available crystal waits for the start of the next
NTE event

10/9/2008 Horiba 1 Compressor supplying dilution air stops working in middle of test (2 units) Unit shutting off due to overheat protection Cool air provided to the compressor (temporary solution)6 REPORT 03 .14936.12 E-6 Date PEMS Description Reason Solution

10/9/2008 AVL 1 MSS concentration reporting too high in Sensors software Sensors not correcting concentration from 0°C to 20°C (~8%) Semtech DS software updated to include the log range and volume correction

10/10/2008 Sensors 1 Bypass flow too low Flow leaking through the carbon filter connection Tightened the carbon filter

10/15/2008 Sensors 1 PPMD emission results were inconsistent Booker had decreased the crystal flow from 0.4 to 0.2 slpm to increase loading time Final crystal sample flow set at 0.5 slpm

10/15/2008 Sensors 1 PPMD emissions were lower than expected Uknown Crystal sensitivity adjusted from 125 hz/μg to 100 h z/μg (increasing sensitivity by 25% )

10/15/2008 Sensors 1 PPMD wouldn't sample when external trigger activates on Semtech DS Unknown Powered off hardware and laptop and restarted, problem was resolved

10/16/2008 Sensors 1 PPMD unable to communicate with DS Unknown Triggered the PPMD manually through Host software (unofficial testing) 10/28/2008 Horiba 1 Dilution flow inaccurate Unknown Recalibrated VFM

11/3/2008 Sensors 1 PPMD block pressure low Uknown, block

pressures were activated in the software Aborted cycle, powered down the PPMD and restarted, problem was fixed

11/4/2008 Sensors
1 Multiple crystals stopped oscillating, when one was enabled another would disable Grease loading slightly too high? Re cleaned and greased crystals

11/4/2008 Horiba 1 Unable to maintain setpoint for total flow near end of the cycle High filter loading Adjusted the valve on the total flow pump to allow more flow7 REPORT 03 .14936.12 E-7 Date PEMS Description Reason Solution

11/5/2008 Horiba 1 Sample flow accuracy is out of spec Caused by inaccurate dilution flow measurement (cause of this uknown) Recalibrated VFM

11/6/2008 Horiba 1 Dilution flow is still inaccurate Unknown Recalibrated VFM

11/7/2008 Sensors 1 MPS Dilution Flow Major audit could not

pass Overheated None, cycle voided

11/7/2008 Horiba 1 External compressor stopped working Overheated None, cycle voided

11/7/2008 Horiba 1 External compressor stopped working Overheated None, cycle voided, installed 2nd compressor in pa rallel

11/12/2008 Sensors 1 PPMD results not included in the DS results file Descrepancies in the time stamps Never resolved

11/12/2008 Horiba 1 Dilution compressor still stoppi ng sometimes Overheating Connected two compressors in pa rallel, so the 2nd will run if the first stops

11/12/2008 Sensors 1 Slope on flow for daily audits sometimes fails to 0.97 or 1.03 Monthly tolerances too tight for da ily checks Tolerances relaxed slightly for daily checks to save time

11/12/2008 Horiba 1 Sample flow check is sometimes not within tolerance Monthly tolerances too tight for da ily checks Tolerances relaxed slightly for daily checks to save time

11/24/2008 Horiba 1 Sample flow accuracy is out of spec Inaccurate dilution flow

measurement Recalibrated VFM

11/25/2008 AVL 1 MSS failed the external DR audit repeatedly Unknown - all internal checks passed Requirement of performing external audit on da ily ba sis was removed8 REPORT 03 .14936.12 E-8 Date PEMS Description Reason Solution

12/1/2008 Sensors 1 EFM reading inaccurate at the end of each test Solenoid not switching properly during the 1 hour autozero, EFM zeroing while open to flow Solenoid fixed, auto zero disabled?

12/1/2008 Horiba 1 Newest compressor that was shipped still stopping sometimes Overheating Continued to use two compressors in parallel, a

new compressor was shipped from Horiba

12/1/2008 Horiba 1 Post processor file size too big for Excel (>200 MB) Steady state data processed at 1Hz instead of 10Hz Continued to use two compressors in parallel

12/11/2008 Horiba 1 DCS signal flat lined during testing Uknown Problem did not occur again when system was restarted

12/15/2008 Horiba 1 TRPM data filed was not saved by Java software Unknown None, cycle was void

12/16/2008 Horiba 1 External compressor stopped working Overheated None, cycle voided

12/18/2008 Horiba 1 External compressor stopped working Overheated None, cycle voided

1/14/2009 Horiba 1 Sample flow accuracy is out of spec Caused by inaccurate dilution flow measurement (cause of this uknown) Recalibrated VFM

1/14/2009 Horiba 1 Dilution flow inaccurate Unknown Recalibrated VFM

1/26/2009 Sensors 2 Semtech DS lost communication with laptop Unknown Couldn't monitor data, but data was still recorded on compact flash card

1/28/2009 Horiba 2 TPA switched from measure to standby at the start of the test Software glitch? Did not occur again9 REPORT 03 .14936.12 E-9 Date PEMS Description Reason Solution

1/28/2009 Horiba 2 TRPM would switch out of filter sample mode once 30 seconds had elapsed Faulty software logic Software modified so that filter will never switch out of sample mode while in
an NTE event

1/28/2009 Horiba 2 Filter sampling was beginning 10 s ec after start of NTE instead of 5 OBS-2200 is delayed 5 seconds in its

response Horiba chose not to make any changes

1/30/2009 AVL 2 Error: No di lution air available On-board pump was leaking Pump was replaced with new part shipped from AVL

2/2/2009 Sensors 2 Semtech DS lost communication with laptop, could not

reconnect Unknown Powered down unit, recovered data the next day

2/5/2009 Sensors 2 Semtech DS was losing communication with laptop Unknown Couldn't monitor data, but data was still recorded on compact flash card

2/18/2009 Horiba 2 External compressor stopped working Overheated None, cycle voided

3/2/2009 Horiba 3 Software zero pressure transducer function would not work after repeated attempts Unknown Shut dow n equipment attempted it again several hours later and it worked

3/3/2009 Horiba 3 External compressor stopped working Overheated None, cycle voided

3/5/2009 Sensors 3 Semtech DS lost communication with laptop Unknown Couldn't monitor data, but data was still recorded on compact flash card

3/6/2009 Horiba 3 External compressor stopped working Overheated None, cycle voided

3/23/2009 Horiba
2200 OBS-2200 software locked up during test Unknown Software worked when rebooted, but data for the cycle was lost

3/24/2009 Sensors DS Couldn't connect to the Semtech DS repeatedly LAN circuit board was likely damaged Unit sent back to Sensors for repair10 REPORT 03 .14936.12 E-10 Date PEMS Description Reason Solution

3/25/2009 Sensors 3 PPMD wouldn't sample when NTE trigger activates on the Semtech DS Unknown Turning the unit off and back on fixed the problem

4/3/2009 AVL 1 There is a voltage offset between what the AVL unit outputs and the Semtech DS reads Unknown Switched constants

4/13/2009 AVL 2 Power inverter shut dow n during radiated immunity

test EMI/RFI Test Replaced bypass pump

4/15/2009 Sensors 2 Communication with the laptop dropped out repeatedly during bulk current injection EMI/RFI Test Head sent back for repair

4/20/2009 Sensors 2 Communication with the laptop dropped out during radiated immunity EMI/RFI Test Replaced with bracket from other unit

4/21/2009 AVL 2 Power inverter shut dow n during conducted transient test EMI/RFI Test No corrective action taken

4/21/2009 AVL 2 Power inverter shut down, then start smoking during conducted transient EMI/RFI Test Unit was shipped back to AVL, replaced with other unit, problem was not observed again

4/21/2009 Sensors 2 PPMD shut dow n completely several times during conducted transient tests EMI/RFI Test No corrective action taken

4/24/2009 Horiba 3 Flow was erratic during bulk current injection test EMI/RFI Test No corrective action taken

4/27/2009 Horiba 3 Exhaust flow was reading very high during bulk current injection test EMI/RFI Test No corrective action taken

4/28/2009 Horiba 3 Lost communication with laptop during radiated immunity test EMI/RFI Test Inverter replaced with backup, test was not repeated11 REPORT 03 .14936.12 E-11 Date PEMS Description Reason Solution

4/28/2009 Horiba 3 Exhaust flow was reading very high during radiated immunity test EMI/RFI Test No corrective action taken

5/15/2009 AVL 3 AVL PEMS blew a fuse when trying to pow er up after 1st pressure test Uknown Replaced with AVL 2

7/15/2009 Sensors 3 PPMD reported a barometric pressure increase when altitude chamber was at vacuum Constants in the software were backwards for the barometric

pressure No corrective action taken

7/15/2009 Sensors 3 PPMD could not maintain a bypass flow of 4 slpm Bypass pump was dying No corrective action taken

7/21/2009 AVL 2 Soot concentration reading erratic during vibration (all or ientations) Vibration Rebooted hardware to

reconnect

7/22/2009 Horiba 3 L-bracket holding filter in ME box broke during vibration testing Vibration Did not continue with

more extreme tests to prevent damage

7/22/2009 Horiba 3 Total Pi pressure transducer would not read correctly after problem with L-bracket occurred Vibration Test Pressure transducer was replaced

7/24/2009 Sensors 2 PPMD could not maintain total flow due to moisture traps opening Vibration Test No corrective action taken