# Test Plan to Determine PEMS Measurement Allowance for the PM Emissions Regulated under the Manufacturer

- Run Heavy

- Duty Diesel Engine in

- use Testing Program

# Test Plan to Determine PEMS Measurement Allowance for the PM Emissions Regulated under the Manufacturer

- Run Heavy

- Duty Diesel Engine in

- use Testing Program

Assessment and Standards Division Office of Transportation and Air Quality U.S. Environmental Protection Agency

and

California Air Resources Board

and

Engine Manufactures Association

EPA-420-B-10-901 August 2010EXECUTIVE SUMMARY

This test plan sets forth the agreed upon pr ocesses and methodologies to be utilized to develop additive, brake-specific, data-driven measurement allowance for PM emissions measured by PEMS as required under the HDIUT regulatory program.

As detailed in this test plan, there is a clear consensus on what components of measurement error are intended to be covered by the measurement allowance. Namely, the allowance is to be calculated in a manner that subtracts lab error from PEMS error. Specifically, utilizing Part 1065 complia nt emissions measurement systems and procedures for both the lab and PEMS, the la b error associated with measuring heavyduty engine emissions at stabili zed steady-state test points within the NTE zone, will be subtracted from the PEMS error associated with measuring heavy- duty engine emissions utilizing PEMS over events under a broad ra nge of environmental conditions. This subtraction will yield “PEMS minus labor atory” measurement allowance. The experimental methods and procedures specifi ed in this test plan for determining, modeling, and comparing each of the various components of measurement error are designed to generate statisti cally robust data-driven measurement allowance for the PM emissions.

Successful completion of this test plan is part of the resolution of a 2001 suit filed against EPA by EMA and a number of individual engi ne manufacturers. The suit challenged, among other things, certain supplemental em ission requirements referred to as “not-toexceed” (NTE) standards. On June 3, 2003, the parties finalized a settlement of their disputes pertaining to the NTE standards. The parties agreed upon a detailed outline for a future regulation that would require a ma nufacturer-run heavy-duty in-use NTE testing (“HDIUT”) program for diesel-fueled engines and vehicles. One section of the outline stated:

“The NTE Threshold will be the NTE standa rd, including the margins built into the existing regulations, plus additional margin to account for in-use measurement accuracy. This additional margin shall be determ ined by the measurement processes and methodologies to be developed and approved by EPA/CARB/EMA. This margin will be structured to encourage instrument manuf acturers to develop more and more accurate instruments in the future.”

Given the foregoing, the work to be completed un der this test plan is a vital component to the fulfillment of the settlement agreement, and it is vital to the successful implementation of a fully-enforceable HDIUT program. Because of this significance, it is critically important that the work detailed in this test plan be carried out in as thorough, careful and timely a manner as possible.

2TABLE OF CONTENTS

Executive Summary

............................................................................................................ 2 Table of Contents

.............................................................................................................. .. 3 List of Figures

..................................................................................................................... 5 List of Tables

...................................................................................................................... 5 1 Introduction

................................................................................................................. 6 2 Monte Carlo Error Model and Measurement Allowance

........................................... 8 2.1 Objective

............................................................................................................. 8 2.2 Background

......................................................................................................... 9 2.3 Methods and Materials

...................................................................................... 15 2.4 Simulation Procedure

........................................................................................ 16 2.4.1 Construction of the Error Surface

............................................................. 16 2.5 Model Considerations

....................................................................................... 24 2.5.1 Convergence

............................................................................................. 24 2.6 Simulation Output

............................................................................................. 24 2.6.1 Sensitivity Variation Effect

....................................................................... 24 2.6.2 Sensitivity Bias Effect

............................................................................... 25 3 Engine Dynamometer Laboratory Tests

................................................................... 26 3.1 Preliminary Audits

............................................................................................ 26 3.1.1 Objective

................................................................................................... 26 3.1.2 Background

............................................................................................... 26 3.1.3 On-site meeting to establish 1065 compliance requirements

................... 26 3.1.4 Methods and Materials

.............................................................................. 26 3.1.5 Data Analysis

............................................................................................ 27 3.1.6 PEMS Manufacturer PM PEMS Commissioning

..................................... 27 3.2 Bias and Precision Errors under steady state engine operation

........................ 27 3.2.1 Objective

................................................................................................... 27 3.2.2 Background

............................................................................................... 28 3.2.3 Methods and Materials

.............................................................................. 28 3.2.4 Data Analysis

............................................................................................ 30 3.3 Precision Errors under transient e ngine operation (dynamic response)

............ 31 3.3.1 Objective

................................................................................................... 31 3.3.2 Background

............................................................................................... 31 3.3.3 Methods and Materials

.............................................................................. 31 3.3.4 Data Analysis

............................................................................................ 35 3.4 ECM Torque and BSFC

.................................................................................... 37 3.4.1 Objective

................................................................................................... 37 3.4.2 Data Analysis

............................................................................................ 37 4 Environmental Chamber

........................................................................................... 37 4.1 Data Analysis for Environmental Tests

............................................................ 38 4.2 PM Generator Commissioning

......................................................................... 39 4.3 Baseline

............................................................................................................. 39 4.3.1 Objective

................................................................................................... 39 4.3.2 Background

............................................................................................... 39

34.3.3 Methods and Materials

.............................................................................. 40 4.3.4 Data Analysis

............................................................................................ 40 4.4 Electromagnetic Radiation

................................................................................ 40 4.4.1 Objective

................................................................................................... 40 4.4.2 Methods and Materials

.............................................................................. 40 4.5 Atmospheric Pressure

....................................................................................... 41 4.5.1 Objective

................................................................................................... 41 4.5.2 Background

............................................................................................... 41 4.5.3 Methods and Materials

.............................................................................. 41 4.5.4 Data Analysis

............................................................................................ 43 4.6 Ambient Temperature and Humidity

................................................................ 44 4.6.1 Objective

................................................................................................... 44 4.6.2 Background

............................................................................................... 44 4.6.3 Methods and Materials

.............................................................................. 45 4.6.4 Data Analysis

............................................................................................ 46 4.7 Orientation and Vibration

................................................................................. 46 4.7.1 Objective

................................................................................................... 46 5 SwRI CVS and CE-CERT Trailer Correlation

......................................................... 47 5.1 Method and Materials

....................................................................................... 47 6 Model Validation and Measurem ent Allowance Determination

.............................. 48 6.1 Model validation

............................................................................................... 48 6.1.1 Objective

................................................................................................... 48 6.2 Measurement Allowance Determination

.......................................................... 49 6.2.1 Objective

................................................................................................... 49 6.2.2 Background

............................................................................................... 49 6.2.3 Methods and Materials

.............................................................................. 49 6.2.4 Data Analysis

............................................................................................ 49 7 Time and Cost

........................................................................................................... 50 7.1 Timeline

............................................................................................................ 50 7.2 Cost

................................................................................................................... 51 8 Abbreviations used in Brake Specific Equations

...................................................... 51

4LIST OF FIGURES

Figure 1. Probability Density Func tions For Sampling Error Surfaces

............................. 9 Figure 2. Brake-Specific PM Emi ssions Calculation For Method 1

............................... 11 Figure 3. Brake-Specific PM Emi ssions Calculation For Method 2

............................... 12 Figure 4. Brake-Specific PM Emi ssions Calculation For Method 3

............................... 13 Figure 5. Error Surface: PEMS Vs. LAB

....................................................................... 17 Figure 6. Error Surface: (PEMS-LAB) Vs. LAB

........................................................... 18 Figure 7. Error Surface: Final Version

........................................................................... 19 Figure 8. Overview of Monte Carlo Simulation

............................................................... 22 Figure 9. Example of A NTE Cycle

.................................................................................. 35 Figure 10. Pressure Histogram

.......................................................................................... 41 Figure 11. Pressure-Time Environmental Test Cycle

....................................................... 43 Figure 12. temperature histogram

..................................................................................... 44 Figure 13. Time Series Chart of Ambient Temperature Test

........................................... 46

LIST OF TABLES

Table 1. Allowed Modifications

......................................................................................... 6 Table 2. Example Of Selection Of Meas urement Allowance At 0.02 g/hp-hr NTE Threshold

.................................................................................................................. 15 Table 3. Error Surfaces for the BSPM Simulation

........................................................... 23 Table 4. Engine, Exhaust Configur ation, And Steady-State Modes

................................. 28 Table 5. Example Of SS Error Surface

............................................................................. 30 Table 6. NTE Transient Cycle

.......................................................................................... 32 Table 7. Dynamic response inter-NTE Events

................................................................. 34 Table 8. Concentration and Dilution Ra tio Schedule with PM Generator

....................... 38 Table 9. Atmospheric Pressure Test Sequence

................................................................. 42 Table 10. Ambient Temperature test sequence

................................................................. 45 Table 11. Projected PM-PEMS Timeline

......................................................................... 50 Table 12. projected cost estimate

...................................................................................... 51

51 INTRODUCTION

This test plan will establish a PEMS measur ement allowances for PM, as regulated by the manufacturer-run on-highway heavy-duty dies el engine in-use test program. The measurement allowance will be established usi ng various laboratory facilities and PEMS. The measurement allowance will be established in units of brake-specific emissions (g/hp-hr), and it will be added to the final NTE PM standard, after all the other additive and multiplicative allowances have been applie d. This test plan will establish the PM measurement allowance.

The PEMS used in this test plan must be standard in-production ma kes and models that are for sale as commercially available PEMS. In addition, PEMS and any support equipment must pass a “red-face” test with re spect to being consistent with acceptable practices for in-use testing. For example, the equipm ent must meet all safety and transportation regulations for use on-board heavy-duty vehicles.

Even though the PEMS can not be “prototype s” nor their software “beta” versions, the steering committee has already agr eed that after delivery of PEMS to the contractor, there may be a few circumstances in which PEMS modifications might be allowed, but these modifications must meet certain deadlines, plus they are subjec t to approval by the steering committee. Also, any implementation of such approved modifications will not be allowed to delay the test plan, unle ss the steering committee specifically approves such a delay. Table 1 summarizes these a llowable modifications and their respective deadlines: TABLE 1. ALLOWED MODIFICATIONS Allowed Modifications Before start of... Steering committee approved hardware and software modifications that affect emissions results; including but not limited to fittings, components, calibrations, compensation algorithms, sampling rates, recording rates, etc. Steady-State Testing

Steering committee approved hard ware modifications for DOT approval or any other safe ty requirement approval Environmental Chamber Testing Delivery of any environmental / w eather enclosure to contractor Environmental Chamber Testing Post-processing software to determine NTE results Model Validation DOT approval and documentation Model Validation Steering committee approved hardware or software that improves the contractor’s efficiency to conduct testing and data reduction Always Allowed

The steering committee approved three diffe rent PEMS that includes the AVL MicroSoot Sensor (MSS), the Horiba Transient Particulate Matter (TRPM), and the Sensors Proportional Particulate Matter Diluter (PPMD) . However, because of the different measurement technologies employed by each of these systems, the three different PEMS hold slightly different status with respect to determining the PM measurement allowance.

6Because inertial microbalance s are already approved for PE MS applications in 40 CFR Part 1065, the Sensors PPMD will be one of the PEMS used to determine the measurement allowance. And because EPA’s PM standard is based upon a gravimetric filter analysis, the Horiba TRPM will also be used to determine the measurement allowance. The lowest measurement allowan ce value between the two will be selected as the final measurement allowance for PM. If that value does not validate, then the lowest validated value will be chosen. If the lowest validated value chosen is within 0.0075 g/hp-hr from the lowest non-validated value, th en the lowest validated value will be the measurement allowance. Otherwise, the MA SC will spend up to a $100,000 to figure out a resolution to the problem by generating more data or changing th e way the validation was performed. If that does not lead to a resolu tion, then Executive Management of EMA and EPA will have to settle the issue.

Note that at the conclusion of successful testing of the Horiba system in this measurement allowance program, EPA intends to approve the Horiba system as an alternative for use, or EPA may elect to amend 40 CFR Parts 86 an d/or 1065 to allow the use of the Horiba TRPM or other PEMS that operate upon sim ilar measurement princi ples. Because the AVL system measures only the soot component of PM, the measurem ent allowance will not be determined using the AVL results, un less both the Sensors and Horiba systems fail to complete the measurement allowance progr am. Note that the steering committee may determine at the conclusion of the program th at the AVL MSS is a viable alternative for demonstrating compliance. Under such a circumstance EPA may amend the Heavy-Duty In-Use regulation to allow for its use.

This test plan describes a computer model, a series of experiments that are used to calibrate the model, and another series of experiments that are used to validate the calibrated model.

The test plan first describes the computer model. The computer model statistically combines many sources of PEMS and lab erro r, which are nearly impossible to capture simultaneously in a single test. The model will use statistics to apply the errors in a way that simulates actual running of a PEMS in-use. The model will also consider only the portion of error that is attributable to PEMS, and it will subtract the error that is already tolerated in an emissions lab today. The model will also calculate and validate results according to 40 CFR Part 1065.

The test plan then describes the series of experiments. These tests will characterize the many sources of PEMS and lab error so that the specific nature of the errors can be programmed into the computer model. The na ture of the error has to do with the way PEMS and the lab react to certain conditions . For example, under varying environmental conditions such as temperature or vibration, a PEMS might exhibit sign al drift, or it may record noise that is not a pa rt of the true emissions.

Next, the experimental results will be en tered into the computer model, and the measurement allowances are calculated by the model. The model uses a “reference" PEMS data set, which will have many “refer ence NTE events.” The model statistically

7applies all the errors to the reference data set, calculates results, and saves the results. Then the model will be run with all errors set to zero to calculate the ideal results of the reference data set. Each difference between a reference NTE event’s result with errors and its respective ideal result will be a brake-sp ecific difference that is recorded for later use. Then the process repeats using the sa me reference data set, to which new, statistically selected errors are applied, a nd thus another unique se t of differences is calculated. As the model continues to iter ate and generate more and more results, patterns are expected to appear in the out put data. These patterns should be the distributions of differences, based upon the error that was st atistically a nd repeatedly applied to the reference data set. Many diffe rence distributions will be determined: for each reference NTE event, for each of the two brake-specific calculation methods (three in case of the AVL system only), and for each PEMS. It has been agreed that the 95 th percentile values of these distributions will be taken as reasonable “worst case” results for each reference NTE event. Details on how all these distributions will be reduced to determine the PM measurement allowance is given in the “Error Model” section of this test plan.

Because the calculation based on Method 2 a nd Method 3 require gas-based fuel flow calculation based on the measurement of CO 2 , CO, and NMHC, a decision was made to use the gaseous PEMS data for this purpos e, without the need to perform gaseous measurement during the PM-PEMS program.

Finally, the test plan describes how the com puter model will be validated against realworld over-the-road in-use PEMS operation as well as additional lab testing. For the over-the-road testing, PEMS emissions measur ements will be conducted, while at the same time a reference laboratory will be to wed along to measure the same emissions. For the lab testing, an attempt will be made to simulate real-world engine operation to “replay” an over-the-road test in the lab. Da ta from these final experiments will be used to validate the model, which must be done in order to gain sufficien t confidence that the model did not establish unreasonable measurement allowances.

The following sections of this test plan are wr itten as instructions to the contractor or contractors who will complete the test plan.

2 MONTE CARLO ERROR MODEL A ND MEASUREMENT ALLOWANCE

2.1 Objective Use Monte Carlo (e.g. random sampling) techni ques in an error model to simulate the combined effects of all the ag reed-upon sources of PEMS error incremental to lab error. Create error “surfaces” for the Monte Carl o simulation to sample, based upon results from the experiments described in Sections 3 and 4. Exercise the model over a wide range of NTE events, based on a single, refe rence data set of at least 150 but no more than 200 unique NTE events. Determine th e pollutant-specific brake-specific additive measurement allowance for PM.

82.2 Background The error model uses Monte Carlo techniques to sample error values from “error surfaces” that are generated from the result s of each of the experiments described in Section 3 on engine dynamometer laborator y tests and Section 4 on environmental chamber tests. The lab test error surfaces c over the domain of error versus the magnitude of the signal to which the error is to be applied (i.e. 1 st to 99 th percentile error vs. concentration, flow, torque, etc. ). This is illustrated later in this section. The environmental test error surfaces for shoc k & vibration and electromagnetic & radio frequency interference (EMI/RFI) cover th e same domain as the lab tests. The environmental test error surfaces for pressu re and temperature are characteristically different because they cover the domain of environmental test cycle time versus the magnitude of the signal to which the error is to be applied (i.e. error at a selected time vs. concentration). Details on how each surfa ce is generated are given in each of the respective sections. These surfaces will alr eady be adjusted to represent PEMS error incremental to lab error; th erefore, these surfaces are sa mpled directly by the model.

The error model will use two different probability density functions (PDFs) as shown in Figure 1 to sample the error surfaces, de pending upon which experiment the surface represents. To sample error surfaces that are generated from all the laboratory test results (Section 3), and the environmental test resu lts for shock & vibration (Section 4), the model will use a truncated normal PDF because these tests are designed to evenly cover the full, but finite, range of engine operati on and ambient conditions. To sample error surfaces that are generated from the pressure and temperature environmental test results (Section 4), the model will use a uniform PDF because these tests are already designed to cover the typical range and frequenc y of the respective conditions.

Probability Density Functions for Samp ling Error Surfaces Once Per NTE Event

-1.00 -0.75 -0.50 -0.25 0.00 0.25 0.50 0.75 1.00

0 1 2 3 4 5 Relative Probability i
c Lab Tests, Normal, SD=0.60795, truncate @ -1 & 1
Environmental Tests, Uniform

Note: A non-truncated normal distribution with SD=0.60795 has P values of 0.01 and 0.99 at i c =-1 and i c =+1, respectively.

FIGURE 1. PROBABILITY DENSITY FUNCTIONS FOR SAMPLING ERROR SURFACES

9The random values that are obtained fr om both distributions are labeled i c in Figure 1 and range from -1 to 1. Note that for the pressure and temperature environmental tests, a uniform PDF will be used to sample test time, from which the nearest (in time) calculated errors are used. The errors from the other tests will be aligned w ith the truncated normal PDF such that each of the 50 th percentile values at each of the tested signal magnitudes is centered at the median of the PDF (i c = 0), and the 1 st and 99 th percentile error values at each of the tested signal magnitudes will be aligned with the extreme negative (i c = -1) and positive (i c = +1) edges of the PDF, respectively.

Each error surface will be sampled along its i c axis (y-axis) once per reference NTE event trial, and it will be sampled along its parame ter value axis (x-axis, e.g., concentration (only for AVL MSS), flow, torque, etc...) on ce per second, within a given reference NTE event trial. An error will be determined for a given second and parameter along the error axis (z-axis) at the intersection of an i c value and a parameter value.

To ensure that the magnitudes of the error surf aces are appropriate, each data point used to generate the surfaces will be a mean or a weighted mean of 30 seconds of sampling.

Interpolation will be performed by first linearly interpolating error values at each tested magnitude along the selected line perpendicular to the i c axis. Then from that line of errors, individual error values will be lin early interpolated at each second-by-second signal magnitude of the given NTE even t in the reference data set.

The reference data set to which all errors will be applied will be a la rge data set of engine operation over a wide range of NTE events. This reference data set will be initially generated from collections of real-world PEMS data sets. The reference data set should contain at least 150 but no more than 200 un ique NTE events. Parameters in the reference data set may be scaled in order to exercise the model through a more appropriate range of parameters (i.e. concentr ations, flows, ambient conditions, etc.). If the parameters are scaled, care should be taken to maintain the dynamic characteristics of the reference data set.

After the errors are applied, NTE brake-specific PM emissions results are calculated, using each of the three agreed-upon NTE calcu lation methods. The three different brakespecific emission calculation methods for PM refe rred to in this test plan are i) TorqueSpeed method, ii) BSFC method, and iii) EC M-Fuel Specific method, and these are illustrated in Figure 2, Figure 3, and Figure 4, respectively.

10For all PM PEMS: is a flow weighted particulate matte r exhaust concentr ation in g/mol m PM 

###  g  N 

###   mol  

m PM 

###   * 

#   n  i 

###   *  t   mol  i  1 

###   s  
e 

##  g / kW

 hr 

## 

 N

PM 

###  Speed i 

##  rpm 

##  * T i 

##  N

 m 

##  * 2*3.14159*  t  

# 

i  1 

###   60*1000*3600  

Where for AVL: is computed numerically as follows, m PM
N 

###   g   mol   

#   mPM i 

###    n  i 

###   *  t   g  i  1 

###   mol   s  
m 

PM 

###   N 

###  mol    mol  

n  *  t 

#   i 

###   

i  1 

###   s  
FIGURE 2. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 1

11For all PM PEMS: is a flow weighted particulate matte r exhaust concentr ation in g/mol m PM



###  g  N 

###   mol 

m

PM 

###   * 

#   n

 i 

###    *  t   mol  i  1 

###   s  
e

PM 

##  /

  

##  g kW hr         mol 

 6

 2 

### 
n * xTHC 

##   ppm  xCO 

##   %  xCO % *10 *  N 

###   i 

###     i 

### *10 

# 
i 2 

##   

#    t  M C 

###  s 

* 

#   i 

### 

w

fuel 

###   g   i  1 

###  

m   fueli 

###     s         i 

### rpm * i 

###  *2*3.14159 

Speed T N m 

##         60*1000*3600     

Where for AVL: is computed numerically as follows, m PM
N 

###   g   mol   

#  mPM i 

###    n  i 

###   *  t    g  i  1 

###   mol   s  
m 

PM 

###   N 

###  mol    mol   

#   n  i 

###   *  t 

i  1 

###   s  
FIGURE 3. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 2

12For AVL Only:



### e PM 

##  g / kW hr 

## 
   g   

m

 N 

###  ifuel 

###    g w   fuel 

### s m

PM 

###   * * 

#    *  t

 6

 2 

###  mol  M C i 

###  xTHC i 

### ppm *10  

#  xCO i 

### %  xCO 2 

### % 

#  *10 

 1 

###  

##       i 

###     Speed 

##    * T N

 m 

##  

#  N 

###  i 

### rpm i 

### *2*3.14159*  t  

i  1 

### 60*1000*3600  

Where:


###  g   

N 

###  

#  mPM i 

##  g / mol 

##  

#  * m  i

f u e l 

###    w

fuel 

### s * 

#     *  t 

 6

 2 

### M xTHC ppm *10  xCO %  xCO % *10 C i  1 

###  

##   

#   

i 

##   

#   i i 2 

###  g     
m 

PM 

###   mol  g     

m

 w N 

###  ifuel 

###   

s fuel 

###   * 

#   *  t 

 6

 2 

### M C i  1 

###  xTHC 

##   *10  

#  xCO 

##    xCO % 

#   ppm % 

##   *10 i i 2 

###  i 

###   

FIGURE 4. BRAKE-SPECIFIC PM EMISSIONS CALCULATION FOR METHOD 3

Next, the NTE events are calculated by each of the three calculation methods, but with no error sampled or applied to the reference da ta set. These results are considered the “ideal” results of the reference NTE events. These ideal results are subtracted from each respective NTE event result ‘with errors’, and the difference is recorded. Then a new set of errors are sampled and applied to the re ference NTE event, and the NTE results ‘with errors’ are calculated again. The ideal result s are again subtracted, and the difference is recorded. This is repeated thousands of times so th at the model converges upon distributions of brake-specific differences for each of the original NTE events in the reference data set.

Then the 95 th percentile difference value is determ ined for each NTE event distribution of brake-specific differences for PM for each calcul ation method. At this point there is one distribution of 95 th percentile differences for PM, where all the NTE events are pooled by the PM emissions for each of the three diffe rent calculation methods. Each of the 95 th percentile distributions repres ents a range of possible measurement allowance values.

13From each of these three distributions of possible measurement allowance values, one measurement allowance per distribution must be determined. First the correlation between 95 th percentile differences versus the id eal PM emission is tested. For each calculation method, if a least s quares linear regression of 95 th percentile differences versus ideal PM emissions has an r 2 (squared correlation coefficient) > 0.85 and an SEE (standard error of the estimate or root-mean-s quared-error) < 5 % of the median ideal PM emission, then that linear regression equation w ill be used to determine the measurement allowance for that calculation method at the following NTE threshold:

PM = 0.02 g/hp-hr and 0.03 g/hp-hr

In cases where extrapolation is required to determine the measurement allowance at the NTE threshold, the measurement allowance will be determined using the linear regression, but evaluated at th e ideal PM emission that is closest to the NTE threshold, not extrapolated to the NTE threshold itself. If the linear regression does not pass the aforementioned r 2 and SEE criteria, then the median value of the 95 th percentile differences is used as the single measurement allowance for that calculation method.

Next, the calculation method is selected. The above procedure will provide three measurement allowances, where applicable, one for each of the three different calculation methods. To make them comparable, the th ree measurement allowance values will be normalized by the PM threshold and expressed as a percent. Also, if any measurement allowance is determined to have a value less than zero, then that measurement allowance will be set equal to zero. The calculation method with the minimum normalized PM value will be chosen and the corresponding nor malized PM value will be selected as the best measurement allowance for PM, assuming it validates. If it does not validate, then the minimum value that validates will be chosen as long as it is within 0.0075 g/hp-hr from the minimum value that did not valida te. If the difference between the minimum value that validates and the minimum value th at did not validate is greater than 0.0075 g/hp-hr, additional investigation with up to a $100,000 will be spent in order to understand why the minimum value chosen di d not validate. If the problem is not resolved after spending the $100,000, then the matter will be referred to executive management of EPA and EMA to decide on the PM measurement allowance.

Error! Reference source not found. below illustrates the selection of the calculation method. The example is based on a hypothetical set of normalized PM measurements for the three calculation methods. The minimum of these normalized allowances is used to select the best method (highlighted in blue). In this hypothetical case, the BSFC method would be selected.

14TABLE 2. EXAMPLE OF SELECTION OF MEASUREMENT ALLOWANCE AT 0.02 G/HP-HR NTE THRESHOLD

Allowance at Respective NTE Threshold (%)

Calc. Method ==> Torque-Speed BSFC ECM fuel specific

BSPM 38 % 18 % N/A Selected Method==> BSFC Method

Therefore, 18% would be selected as the be st measurement allowance for PM, assuming it validates. Otherwise, the 38 % will be chos en if it validates. Thus, the additive brakespecific measurement allowance would be:

PM = 18 % * 0.02 g/hp-hr = 0.0036 g/hp-hr, if it validates, an d if not, then: PM = 38 % *0.02 g/hp-hr = 0.0076 g/hp-hr, if it validates, and if not, then: spend up to a $100,000 to figure out why it did not validate in the first place, and then apply the above strategy again, assuming the value now validates. If not, then EPA and

EMA executive management will decide on the PM measurement allowance value.

This PM value would be the value added to the actual brake-specifi c NTE threshold for a given engine, based on actual family emissions limit, mileage, model year, etc.

2.3 Methods and Materials Exercise the model using three different cal culation methods: a) Torque-Speed method, b) BSFC method, and c) ECM-Fuel Specifi c method (only for AVL MSS). Determine which calculation method is the most accurate, and use it to estimate the measurement allowance. Each calculati on method is described in Figure 2, 3, and 4.

Prepare an Excel spreadsheet model for use with the Crystal Ball Monte Carlo software for error analysis of brake specific emissions, BSE, as outlined in section 2.4. Changes to the model specifications may be requested as agreed upon by the Steering Committee. Prepare the spreadsheet in a modular struct ure following the specified model outline, and make provisions for the identified calculation modules. Additionally, clearly identify and easily locate input cells to the model to facilitate any revisions that may become necessary for users who want to exercise the model with other Monte Carlo add-ins such as @Risk or the newest versions of Crystal Ba ll. Test the spreadsheet with controlled test cases of simplified input distributions with the Crystal Ball add-in to confirm correct model implementation in accordance with this test plan. Run at least one typical analysis as an additional confirmation.

Deliver the electronic spreadsh eet and a brief report describi ng the model, presenting the test cases, and describing pertinent inform ation including the Crystal Ball version number, the Excel version number, the operating sy stem and the computer. Use standard

15spreadsheet calculations so that no serious difficulties will be anticipated regarding application in other spreadsheet versions. Use Crystal Ball Version 7 or higher, and confirm test cases using Excel 2003.

Control revisions of the spr eadsheet model using descriptive file names. Extensive revisions or testing with othe r software versions beyond that initially proposed may be reproposed by the Steering Committee if and when a need for such additional work is identified.

2.4 Simulation Procedure

For each of the measurement errors in Section 3, create an error surface and sample it according to the aforementioned PDFs. Each error surface represents an additive error— or a subtractive error if the sign is negative—relative to the reference value to which it is applied. Figure 5, Figure 6, and Figure 7 serve as a hypothetical PM example of how these error surfaces should be created for ev ery error. The plots shown correspond to PM emissions concentration data representing 1 PEMS, two engines, and three exhaust configurations each, with all 6 sets of PEMS data pooled together. Note that separate error surfaces will be constructed for each of the three PEMS units (AVL, Horiba and Sensors). The example applies to the e rror module for steady-st ate (SS) bias and precision PM concentration erro rs (Section 3.2). These fi gures will be referenced by each “Data Analysis” section for the various errors discussed in this test plan.

Errors from Section 3 (Engine Dynamo meter Laboratory tests) and Section 4 (Environmental Chamber Tests) are combined by adding all of the sampled errors once per NTE event trial. For example, in order to assess the errors in PM concentration for each NTE event, several modules will be created such that:

PM_with errors = PM _ideal +  μ g/mole) 1 +  μ g/mole) 2 +  μ g/mole) 3 + ...

where,  μ g/mole) 1 = PM concentration errors due to steady state bias and precision
errors,  μ g/mole) 2 = PM concentration errors due to ambient temperature,  μ g/mole) 3 = PM concentration errors due to ambient pressure,
etc....

2.4.1 Construction of the Error Surface

2.4.1.1 PEMS vs. Lab

Acquire raw data with the PEMS at various average concentration levels as per Section 3.2. Plot the “PEMS” signals versus the corresponding “lab” signals that were measured using lab equipment. This plot pools all bi as and precision errors for one PEMS and for all data from all engines for all steady-st ate modes. Shown in Figure 5 are the 5 th , 50 th

16and 95 th percentiles at the mean PM concentration level from the lab (note that the distribution of data at e ach level is not necessarily Gaussian). If the 50 th percentile is different than the line of perfect agreement (diagonal), the data sugg ests that there is a bias error between PEMS and La b. In essence this graph sh ows the statistical distribution measured by the PEMS at each average concentration level sampled. The example shows only 6 discrete PM concentrati on levels (ranging from 10-60 μ g/mole). However, the actual number of discrete leve ls will be determined by the total number of operating conditions actually run for all the tests of all the engines. For exampl e, the SS PM testing will select 6 modes representing typical operating conditions. Thus, the actual plot for SS PM will likely have 36 discrete concentration levels (6 modes x 1 PEMS x 2 engines x 3 exhaust configurations).

0 10 20 30 40 50 60 70 80

0PMμg/mole (PEMS) Error Surface for SS PM Concentration

18 26 38 45 56 69

13 24 36 42 53 64

8 19 33 39 51 61

10 20 30 40 50 60

PM μ g/mole (lab,mean) 70 80

diagonal 95th percentile 50th percentile (median) 5th percentile

FIGURE 5. ERROR SURFACE: PEMS VS. LAB

179 Error Surface for SS PM ConcentrationPMμg/mole (PEMS) - PMμg/mole (lab) 10

8

6

4

2

0

-2

-4 8

6 8

5 6

3 4 6

2 3 4

-2 -1 3

-1 1 1

0 10 20 30 40 50 60 70

PM μ g/mole (lab,mean)

95th percentile 50th percentile (median) 5th percentile

FIGURE 6. ERROR SURFAC E: (PEMS-LAB) VS. LAB

18Error Surface for SS PM Concentration

10.1 6.8 8.8 7.2 11.1

3 4 6 2 3 4

-4.1 -3.1 1.8 -2.2 0.2 -0.2

-1.5

-1 -0.5 0 0.5

1 1.5

0 10 20 30 40 50 60 70

PM μ g/mole (lab,mean)ic_SS_PM

99th percentile 50th percentile (median) 1st percentile Error Surface: z-axis = Δ SS_PM_ μ g/mole

6.

FIGURE 7. ERROR SURF ACE: FINAL VERSION

192.4.1.2 (PEMS – Lab) vs. Lab

The plot in Figure 6 basically shows the “addi tive error band” measur ed during testing. The plot is created by first subtracting the “lab” PM value from the corresponding individual PEMS PM measurement for each test run. This difference is defined as the ‘delta’ error. Next, the “PEMS – Laboratory” delta errors are pooled at each average lab PM value to obtain the 95 th , 50 th , and 5 th percentile values, re spectively, displayed in Figure 5. Notice that if lab error exceeds PEMS error at a given percentile, crossover of values can occur. This is acceptable becau se the crossover effectively reduces PEMS error whenever lab error exceeds PEMS error.

In order to obtain estimates of the 1 st and 99 th percentiles for the delta errors for a given “lab” PM value, each side of the corresponding error distribution will be assumed to independently fit a normal distribution. Becau se of the asymmetry of the data, this methodology will yield two halves of a normal distribution. The median of each normal distribution will be the median based on the delta errors given in Figure 6. The 95 th percentile delta error will form the upper boundary of one half of the normal distribution, and the 5 th percentile delta error will form the lo wer boundary of the other half of the normal distribution. When each side of the da ta distribution is fitted to a normal distribution using the above boundary conditions, one can then expand each half of the distribution from the erro r surface to obtain the 1 st and 99 th percentiles of the data for the given “lab” PM value

.
2.4.1.3 Error Surface

This step normalizes the data in Figure 7 us ing what is called a “variability index (i c )”, which represents the random sampling by the M onte Carlo technique, in order to select a given error level. This variab ility index is allowe d to vary from –1 to +1. The likelihood of i c being any value between –1 through +1 is specified by the PDF assigned to i c . In the given example, i c is assumed to vary according to a normal distribution during Monte Carlo calculations. This is because it is be lieved that the distribution of errors due to steady-state bias and precision will be centered about the 50 th percentile of the full range of conditions measured according to Sec tion 3.2. The pressure and temperature environmental error modules use uniform probability density functions for their respective variability index. Each set of data for each lab set-point mean (i.e., lab reference value) in Figure 6 is normalized by aligning the 1 st percentile error from the fitted normal distributions with i c = -1, the 50 th percentile error with i c = 0, and the 99 th percentile error from the fitted normal distribution with i c = + 1.

Error surfaces such as the one presented in Figure 7 are the input modules that the Monte Carlo simulation program will use during calculations of brake-specific PM emissions. For example, for a given NTE calculation a random i c value is chosen once per NTE event trial. Let us assume that the first random sample produced an i c = 0.5. Let us also

20assume that during this NTE event trial, the reference PM concentration is 10 μ g/mole. In this case,

 μ g/mole) 1 = (3 + 10.1) / 2 = 6.55 μ g/mole.

Also, from Figure 7, for i c = 0.5, the reference PM = 10 μ g/mole.

For that step in the calculation, the Monte Ca rlo approach will add this “delta” to the reference concentration value of 10 μ g/mole (10 μ g/mole + 6.55 μ g/mole = 16.55
μ g/mole) to represent errors in steady-state bias and precision for i c = 0.5, and reference NTE PM = 10 μ g/mole. If during the same NTE event in the reference data set, a reference concentration of 35 μ g/mole is read, then,

 μ g/mole) 1 = ((6 + 8.8) / 2 + (2 + 6.2) / 2) / 2 = 5.75 μ g/mole (from Figure 7)

Note that first the error along the i c line perpendicular to the i c axis (in this case the line along 0.5) is linearly interpolated at each di screte concentration level. Then those interpolated values are themselves linearly interpolated to determine the error corresponding to each reference concentration in the NTE event. Note that the random selection is once per reference NTE ev ent trial, but the error along that i c line is applied to every second-by-second value within the given reference NTE event, except for PM concentration in the case of Horiba and Se nsors, where no second-by-second information are available, but different PM concentrati on levels may be available for a specific NTE
event.

Now let us assume that the error in PM c oncentration is composed of only 3 deltas:  μ g/mole) 1 ,  μ g/mole) 2 , and  μ g/mole) 3 . And let us assume that for a given reference NTE event trial we have the following values:

 Reference PM at one second= 30 μ g/mole

  μ g/mole) 1 = 6 μ g/mole

  μ g/mole) 2 = -2 μ g/mole

  μ g/mole) 3 = -3 μ g/mole.

When the model calculates brake-specific em issions by each of the three calculation methods, it will use the following PM value, which has all of its error applied:

PM = 30 + 6 –2 – 3 = 31 μ g/mole.

The application of error at the first selected i c continues during the entire NTE event without having to randomly sample again. In other words, i c will not change during that random trial. For all of the variables except for m PM , the errors may continue to change during an NTE event on a second-by-second basi s if their error surface happens to be a function of level. For the second randomly selected i c this entire process of determining

21the  μ g/mole errors is repeated. The simulatio n will continue to randomly selected i c values for thousands of tria ls until convergence is met.

For the Horiba and Sensors generated refere nce NTE events, there is only one flowweighted PM value for the entire NTE event. During the simulation for these types of reference NTEs, the single PM value will be used in the interpolation of the corresponding PM error surfaces (i.e., steady-st ate PM, transient PM) at all seconds of the reference NTE event. Since the PM va lue will not vary from second-to-second, the only interpolation will occur according to the i c value at each of the simulation trials.

The same second-by-second sampling and interpolation approach would be used for other deltas such as ambient temp, ambient pressu re, shock and vibrati on, BSFC interpolation, torque, exhaust flow rate, etc. An overvie w of the Monte Carlo simulation for PM is detailed in Figure 8.

FIGURE 8. OVERVIEW OF MONTE CARLO SIMULATION

Table 3 lists the error surfaces that will be created for use in simulating the BSPM error differences.

22TABLE 3. ERROR SURFACES FOR THE BSPM SIMULATION Calculation Component Test Source Error Surface Delta PM Engine Dyno Delta PM SS Delta PM Engine Dyno Delta PM Transient Delta PM Environ Delta PM Ambient Temperature Delta PM Environ Delta PM EMI/RFI Delta PM Environ Delta PM Atmospheric Pressure Delta PM Environ Delta PM Vibration Delta CO Engine Dyno Delta CO SS Delta CO Environ Delta CO Atmospheric Pressure Delta CO Environ Delta CO Ambient Temperature Delta CO Engine Dyno Delta CO Time Alignment Delta CO2 Engine Dyno Delta CO2 SS Delta CO2 Engine Dyno Delta CO2 Transient Delta CO2 Environ Delta CO2 Ambient Temperature Delta NMHC Engine Dyno Delta NMHC SS Delta NMHC Engine Dyno Delta NMHC Transient Delta NMHC Environ Delta NMHC Atmospheric Pressure Delta NMHC Environ Delta NMHC Ambient Temperature Delta NMHC Environ Delta Ambient NMHC Delta Exhaust Flow Engine Dyno Delta Exhaust Flow SS Delta Exhaust Flow Engine Dyno Delta Exhaust Flow Transient Delta Exhaust Flow Engine Dyno Delta Exhaust Flow Pulsation Delta Exhaust Flow Engine Dyno Delta Exhaust Flow Swirl Delta Exhaust Flow Environ Delta Exhaust EMI/RFI Delta Exhaust Flow Environ Delta Exhaust Temperature Delta Exhaust Flow Environ Delta Exhaust Pressure Delta Torque Engine Dyno Delta Dynamic Torque Delta Torque Engine Dyno Delta Torque DOE Testing Delta Torque Engine Dyno Delta Torque Warm-up Delta Torque Engine Dyno Delta Torque Humidity/Fuel Delta Torque Engine Dyno Delta Torque Interpolation Delta Torque Engine Manuf Delta Torque Engine Manuf Delta Speed Engine Dyno Delta Dynamic Speed Delta Fuel Rate Engine Dyno Delta Dynamic Fuel Rate

232.5 Model Considerations

2.5.1 Convergence

The main goal of the convergence criteria is to define how many simulation trials at a given reference NTE event are required to estimate the 95 th percentile BSPM emission differences with a given precision. The convergence method to be used is based on a nonparametric statistical technique 3 which defines a 90% conf idence interval for the 95 th percentile of the BSPM emissions differences for an individual reference NTE simulation. If the width of the 90% confiden ce interval is less th an 1% of the BSPM emissions threshold, then convergence is met. The following steps define the convergence method:

1. Run the Monte Carlo simulation for N trials for a single reference NTE event. 2. Order the BSPM emissions differences from smallest to largest. 3. Identify the trial number at the lower end of the 90% confidence interval

n lower = 0.95 * N

 1.645 0.95 * 0.05 * N

4. Identify the trial number at the upper end of the 90% confidence interval

n upper = 0.95 * N  1.645 0.95 * 0.05 * N

5. Compute (BSPM difference value at n upper ) – (BSPM difference value at n lower ). 6. If the result in (5) < 1% of the BSPM emissions NTE threshold (0.02 g/hp-hr) then convergence is met.

2.6 Simulation Output

It is important to understand and identify what error surfaces have the most influence (i.e., sensitivity) on the BSPM emissions ‘w ith errors’ and, thus , the resulting BS emissions differences. Contributions to sens itivity can be attributable to changes in variance and/or bias.

2.6.1 Sensitivity Variation Effect During the Monte Carlo simulation for each re ference NTE event, sensitivity charts produced by Crystal Ball will be generated a nd stored in output REPORT files. Crystal Ball calculates sensitivity by computing the rank correlation coefficient between every assumption (error surface) and forecast value (delta BS emissions) while the simulation is running. Positive rank correlations indicat e that an increase in the assumption is associated with an increase in the forecast. The larger the absolu te value of the rank correlation the stronger the relationship.

Sensitivity charts in Crystal Ball provide a m eans to determine how the variances of the error surfaces affect the variance in the for ecast values. Hence, the sensitivity charts developed during a simulation ar e displayed as “Contribution to Variance” charts which

24are calculated by squaring the rank correlation co efficients for all assumptions used in a particular forecast and then normalizing them to 100%. The assumption (error surface) with the highest contribution to variance (in absolu te value of the percent) is listed first in the sensitivity chart.

Simulation results from all reference NTE even ts will produce sensitivity values for the 95 th percentile delta PM emissions by all three calculation methods.

2.6.2 Sensitivity Bias Effect Another type of sensitivity to be examined in this study is concerne d with the effects of potential “bias” in error surfaces and their effects on the forecast values. In order to study these effects a new error surface assump tion will be added to the simulation model for each of the original error surfaces.

This assumption will be sampled as a discrete binary distribution (i.e ., on or off) during the simulation. For each trial of the simulation, the original error surfaces and ‘on/off’ error surfaces will be sampled according to their defined sample distribution. If the ‘on/off’ error surface produces an ‘off’ condition, the delta emissions from that particular error surface will not be added to the B SPM emissions computations for the BSPM emissions ‘with errors’. Similarly, if th e ‘on/off’ error surf ace produces an ‘on’ condition, the delta emissions from that partic ular error surface w ill be added to the BSPM emissions calculations.

During every trial of the simulation, the exclusio ns due to the ‘off’ conditions will result in various combinations of the error surface delta emissions being added to the BSPM emissions ‘with errors’ computations. Over th e course of a simulation with thousands of trials, the sensitivity of a particular erro r either ‘on’ or ‘off’ will be assessed by examining the change in the forecast delta emission. Therefore, in a single Monte Carlo simulation of a reference NTE event sensitivit ies due to variance and/or bias will be explored.

253 ENGINE DYNAMOMETER LABORATORY TESTS

Utilize engine dynamometer laboratory testing to establish the difference between PM PEMS and PM based on laboratory measurement in accordance with Part 1065. Also establish how well ECM parameters can be used to estimate torque and BSFC.

First, however, audit all the PEMS and lab equipment to ensure that they are operating properly, according to 40 CFR Part 1065, Subpa rt D. Next, conduct steady-state engine dynamometer tests to establish PEMS steady-st ate bias and precision relative to the lab. Then, conduct transient engine dynamometer testing to determine PEMS transient precision by repeating transient NTE events. Finally, compare ECM derived torque and BSFC to laboratory measured torque and BSFC.

3.1 Preliminary Audits

3.1.1 Objective Conduct 40 CFR Part 1065, Subpart D audits of all engine dynamometer laboratory systems and all PEMS.

3.1.2 Background Because the overall purpose of this entire test plan is to establish measurement allowance that account for the incremental difference in the performance of PEMS versus engine dynamometer laboratory systems, the first task is to audit all of the measurement systems to ensure that the specific systems used for testing meet EPA’s minimum performance requirements. The audits also help to minimize bias errors between PEMS and lab systems measurements. However, in cas e a specific PM-PEMS does not meet the specifics of Part 1065 requirement, the MASC will decide on how to move forward by perhaps allowing some flexibility in passing Part 1065 audit, in situations where it might be needed, especially if the performance of a system is within the expectation of the manufacturer.

3.1.3 On-site meeting to establis h 1065 compliance requirements In order to clarify what are all the requirements expected from the lab-grade instrumentation and PEMS equipment, with respect to 1065 compliance, a meeting will be held between the test plan steering committ ee and the contractor at the contractor site to provide the contractor with guidance re garding which specific sections of Part 1065 Subpart D are required and wh ich are optional. In case Part 1065 requirement is demonstrated to be too stringent or imprac tical, the contractor may seek approval from the MASC to lessen the stringency of Part 1065 in relation to the PEMS.

3.1.4 Methods and Materials
Use the methods and materials described in 40 CFR Part 1065, Subpart D to conduct audits of all lab and PEMS measurement syst ems. Even if lab systems and PEMS pass initial Subpart D audits, allow lab operators and PEMS manufacture rs to make on-site

26adjustments to improve the performance of th eir systems prior to engine testing. Allow adjustments to be based on recalibrations with reference signals that are allowed in 40 CFR Part 1065. The steering committee may direct the contractor to calibrate or adjust the laboratory sampling system based on audi t results. The steering committee may also suggest that a PEMS manufactur er calibrate or adjust one or more PEMS based on lab audits.

3.1.5 Data Analysis Use the data analyses described in CFR Part 1065 Subparts D, J and G. For all subsequent testing, use only those meas urement systems that pass the minimum performance criteria in Subpart D, unless a deficiency is deemed acceptable in writing by all parties including PEMS manuf acturers. Provide a list and brief description of all the audits conducted for each PEMS manufacturer type. EPA would likely use this list as a template for the data requirements in the PM portion of the HDIU testing program.

3.1.6 PEMS Manufacturer PM PEMS Commissioning Notify PEMS manufacturers when the 1065 audits are complete and th e first set of PM PEMS are completely installed in the engine dynamometer test cell—in preparation for emissions testing. Schedule dates and times that are prior to the star t of emissions testing for each PEMS manufacturer to conduct a fina l commissioning of all their PEMS that are on site, including those PEMS that are not insta lled in the test cell. PEMS manufacturers may inspect their PEMS and make any final adjustments to their respective PEMS in order for the PEMS to meet their specificati ons. Allow PEMS manufacturers to inspect the installation of their PEMS in the test cell. If PEMS manufacturers take exception to any portion of the in stallation or on-site configurati on, attempt to re solve any such installation issues. If such issues are not easily resolvable, notify the steering committee, who will determine a course of action. Once PEMS manufacturers have completed their commissioning, notify the steering committee. From this point any further modifications to the PEMS may only be made according to Table 1 of this test plan.

3.2 Bias and Precision Errors under steady state engine operation

3.2.1 Objective Evaluate the bias and precisi on using one engine and one e xhaust configuration, shown in Table 4, and 10 repeats of steady-state modes, and three sets of PEMS units, each set including the MSS, TRPM, and PPMD. Thus, the total number of NTE steady-state points required to conduct the steady-state e xperiments is 30. This constitutes six steadystate modes of engine operation (6), 10 rep eats (10), one exhaust configuration, one engine (1), and three differe nt PEMS units (3), 6x10x1x1x3= 180.

g Determine the Δ SS m PM 

# 
mol 

#  surface plots for the error model based upon all data pooled.

Note that each brand of PEMS will have its own Δ SS m PM error surface generated for use

27in both calculation methods 1 and 2. Fo r calculation method 3, the AVL brand PM PEMS will have a unique Δ m calculated according to Figure 4 of this test plan. SS PM
Recommend six steady-state points based on the PM measurement, using the AVL MSS, of 80 SS points of the Cummins cycle that is typically used to generate ECM torque and BSFC errors versus laboratory. The MASC will accept the six steady-state points or choose alternative points for each exhaust configuration. The objective for the MASC will be to select steady-state points within a given exhaust configuration that provides a nominal spread of concentrations within that configuration’s target brake-specific levels. Note that to achieve the brake-specific ta rgets under steady-state conditions, the bypass might have to be opened further, rela tive to the transient NTE bypass settings.

TABLE 4. ENGINE, EXHAUST CO NFIGURATION, AND STEADY-STATE MODES

No. of Steady-State Modes for Bypass Setting 1 (BSPM and

PM Concentration, representative of PM

threshold of 0.025 g/hp-hr under NTE Transient Operation) PM-PEMS Units Number of Repeats

07 Engine 1 SS1, SS2, SS3, SS4, SS5, SS6 Three Sets of (MSS, TRPM, and PPMD) 10 per Mode per PMPEMS Set

3.2.2 Background Testing will be conducted to capture bias and precision errors in PEMS’ emissions instruments versus the laboratory filter-based me thod. The tests will be steady-state only.

Note: Section 3.3 (next section) will evaluate precision errors (not bias) due to the dynamic response of the PEMS instrumenta tion. The precision error captured during steady state testing (section 3.2) will have to be subtracted from the overall precision error captured in section 3.3 in order not to double-count the steady state precision errors of PEMS instrumentation. This process is detailed in Section 3.3.

3.2.3 Methods and Materials Use the following systems: a) One model year 2007 heavy duty diesel engi nes, equipped with a DPF in the exhaust (Mack MP9) b) Nine PM PEMS (3 Sensors PPMD, 3 AVL MSS, 3 Horiba TRPM) c) One PEMS exhaust flow-meter from Sens ors, Inc., and one and from Horiba, applicable to the engine to be tested

28d) DPF with Bypass Setting 1 for SS testing, representing a threshold level of about 0.025 g/hp-hr under NTE transient testing

Use the following overall guidelines: e) Measure PM via the CVS, Part 1065 Lab Method (most recent publication) f) Measure engine inlet airflow thr ough use of LFE or equivalent g) Use a series of six steady-state modes, and set each mode time to collect a CVS filter mass of at least 75 microgram per mode, simultaneously with other PM-PEMS h) Regenerate DPF system prior to each series of steady-state tests i) Capture ECM broadcast channels and other common diagnostic channels, as recommended by engine manufacturer(s ), to ensure proper engine operation j) Do not measure gaseous species by the PEMS k) Stabilization time = 180 seconds, with a diffe rent running time per mode to achieve a 75 microgram or higher of PM on the CVS filter l) Always power off PEMS equipment at end of each day, according to PEMS manufacturer instructions. Re-start st art-up process every day according to PEMS manufacturer instructions and Part 1065, Subpart J. m) Whenever PEMS are exchanged, swap the order of the Horiba and Sensors flowmeters, if the steup allows for it.

6 point steady-state repeat-testing, ev aluate bias and precision errors:

a) The MASC will select 6 SS operating conditi ons for repeat testing from a matrix of 80 SS points containing information on PM emissions using the AVL MSS b) Randomize the order of the six modes c) Repeat each six steady-state cycle two or three times, prior to DPF regeneration d) Each test will use three PEMS (Sensors, AVL, and Horiba) at a time, to measure PM emissions concentration and exhaust flow rate. e) Expected test duration is 5 days per PEMS set, with a total of 15 days for all three
sets.

Bypass Setting:

a) Run NTE transient cycle usin g the CVS filter-based method b) Set bypass to produce CVS filter-based aver age brake-specific of about 0.025 g/hp-hr c) Determine the average PM mass concentration d) Run the 80 SS Cummins cycle to capture PM concentration at each mode using the AVL MSS e) Check the PM concentration levels and sele ct the six-steady state modes from the 80 point matrix. As a first order, check the c oncentration at the pre-selected steady-state modes to see if they spread within reas on around the concentration produced for the NTE transient cycle. If not, adjust the bypass as needed to establish the right spread in brake-specific emissions and concentration for the six steady-state modes

29f) Make sure that the points selected sp read around a brake specific level and concentration level of a threshold of 0.025 g/ hp-hr, and concentration range of 4 to 15 milligram per cubic meter.

3.2.4 Data Analysis Use the acquired data to create the “error surfaces” to be used by the Monte Carlo simulation. An example of the steady-state error surface determinatio n is shown in Table 5 for PM.

TABLE 5. EXAMPLE OF SS ERROR SURFACE Error Surface for SS PM Concentration

Figure 5 x-axis PM μ g/mole (lab mean at setpoint) y-axis PM μ g/mole (PEMS)

Figure 6 x-axis PM μ g/mole (lab mean at setpoint) y-axis 5th percentile 5th [PM μ g/mole (PEMS) - PM μ g/mole (lab)] y-axis 50th percentile 50th [PM μ g/mole (PEMS) - PM μ g/mole (lab)] y-axis 95th percentile 95th [PM μ g/mole (PEMS) - PM μ g/mole (lab)]

The 5 th , 50 th and 95 th percentiles from the (PEMS - l ab) delta data will be used to estimate the 1 st and 99 th percentiles from assumed Gaussian distributions.

Figure 7 x-axis PM μ g/mole (lab mean at setpoint) y-axis i c_SS_PM z-axis = Δ SS_PM_ μ g/mole 1 st Percentile from Gaussian distribution based on 5th and 50th [PM μ g/mole (PEMS) - PM μ g/mole (lab)] deltas.

99 th Percentile from Gaussian distribution based on 50th and 95th [PM μ g/mole (PEMS) - PM μ g/mole (lab)] deltas.

50 th Percentile based on [PM μ g/mole (PEMS) - PM μ g/mole (lab)] deltas.

i c sample frequency once per NTE event trial

i c sample distribution Gaussian (normal distribution)

303.3 Precision Errors under transient en gine operation (dynamic response)

3.3.1 Objective

The objective of this portion of the work is to determine the precision error, Δ m  TR PM with each PM-PEMS under NTE transient engi ne operation. This will be achieved by creating a 20 to 25-minute transient NTE cycl e where the PEMS measure in each NTE.

3.3.2 Background PEMS are expected to operate in a repeatab le manner over NTE events as short as 30 seconds. Two sources of PEMS precision erro r are hypothesized: 1) dynamic response to rapidly changing signals, and 2) susceptibility to “history” effects. Dynamic response error includes error due to measurement signal time alignment, and the dissimilarity of the dynamic response and aliasing of signals ; including those signal s used to determine entry into and exit from the NTE zone. Histor y effects include the effects of previously measured quantities on currently measured quan tities. For example, this may be caused by ineffective sample exchange in the PM emissions sampling volumes, or it may be caused by one or more sensors’ characteristic rise time or fall time. To account for any dynamic response precision error, the increa se in precision error incremental to the steady-state emissions measurement precision wi ll be incorporated into the overall error model.

Selection of short NTE cycles (each 32 seconds) maximizes the sensitivity of this test to effects of dynamic response. Thirty-two s econds was chosen as th e minimum instead of thirty seconds, which is the shortest NTE ev ent time, to ensure that 1 Hz ECM updating of torque and speed values wo uld be unlikely to interfere with capturing NTE events. For each repeat of the test cycle, the order of the 30 different N TE events will be the same. In addition the 29 different intervals separatin g each NTE event from the next will have a range of durations and these will be randomly a rranged in each test cycle as well. Fixed arrangement of the NTE events and the inter-N TE events will maximize the sensitivity of this test to dynamic response and history e ffects, and make the DPF and bypass operation very consistent.

The total length of the NTE transient cycle w ill assume that only 5 quartz crystal of the Sensors PPMD are working, and it takes five minutes of stabilization time for reusing a crystal after PM collection. Thus, the same NTE transient cycle used in the gaseous PEMS program will be used here, except for changes in the inter-NTE times to accommodate the Sensors PPMD.

3.3.3 Methods and Materials

a. Use a transient engine dynamometer emissions laboratory.

31b. Use a laboratory that can accommodate at least three PEMS, their power supplies, the PEMS flow meters, cables and lines.

c. Use same overall guidelines described in section 3.2, but applied to transient engine testing. d. Record the EEPS’ total mass sign al during transient testing.

Challenge PEMS to 30 different 32-second N TE events, shown in Table 5, over about 23 minute test cycle, or whatever needed to accommodate the need for five crystals of the PPMD to be operational. Randomize the NTE events shown in Table 6 once, scale up every fifth inter-NTE time, shown in Table 7, to accommodate the PPMD, and use the same order for repeat testing. Repeat the test cycle 10 times for each set of three PEMS. Note that for any torque command that is le ss than zero, command closed throttle (i.e. zero or minimum fuel command) , and motor the engine at the commanded speed for that data point. An example of an NTE tr ansient cycle is shown in Figure 9.

Based on 10 repeats with each set of PEMS, the total number of repeats will be 30 cycles, assuming 1 NTE cycle x 10 repeats x one exha ust configuration x 3 sets of PEMS x one engine (1x10x1x3x1 = 30). A ssuming a 25 minutes of NTE with 30 minutes of forced regeneration and preparation for the second repeat, the total num ber of days for NTE transient testing is 10 days (8 hours per day) . This time includes PEMS and engine setup, PEMS warm up, and daily checks.

TABLE 6. NTE TR ANSIENT CYCLE

NTE Event 1 Speed % Range 2 Torque % Range Description

NTE1 17% 3 32% Steady speed and torque; lower left of NTE NTE2 59% 3 32% Steady speed and torque; lower center of NTE NTE3 Governor line 3 32% Steady speed and torque; lower right of NTE NTE4 17% 66% Steady speed and torque; middle left of NTE NTE5 59% 66% Steady speed and torque; middle center of NTE NTE6 Governor line 66% Steady speed and torque; middle right of NTE NTE7 17% 100% Steady speed and torque; upper left of NTE NTE8 59% 100% Steady speed and torque; upper center of NTE NTE9 100% 100% Steady speed and torque; upper right of NTE NTE10 Lower third 3 32% - 100% Highly transient torque; moderate transient

32speed NTE11 Upper third 3 32% - 100% Highly transient torque; moderate transient speed NTE12 Middle third 3 32% - 100% Highly transient torque; moderate transient speed NTE13 17% - governed Lower third Highly transient speed; moderate transient torque NTE14 17% - governed Upper third Highly transient speed; moderate transient torque NTE15 17% - governed Middle third Highly transient speed; moderate transient torque NTE16 Lower right diagonal Transi ent; speed increases as torque increases NTE17 Upper left diagonal Transient; speed increases as torque increases NTE18 Full diagonal; lower left to upper right Transient; speed increases as torque increases NTE19 Lower left diagonal Transi ent; speed decreases as torque increases NTE20 Upper right diagonal Transient; speed decreases as torque increases NTE21 Full diagonal; lower right to upper left Transient; speed decreases as torque increases NTE22 Third light—heavy-duty NTE event from International, Inc. data
set Sample from LHDE

NTE23 Cruise; ~ 50 mph Sample from HDDE NTE24 Cruise; ~ 75 mph Sample from HDDE NTE25 Small bulldozer Sample from NRDE NTE26 Large bulldozer Sample from NRDE NTE27 Second of three NTE events in FTP Seconds used from FTP: 714-725, 729­ 743, 751-755 NTE28 Third light—heavy-duty NTE event from International, Inc. data
set Sample from LHDE

NTE29 First of two NTE events in NRTC Seconds used from NRTC: 423-430, 444, 448-450, 462-481, increased 464 speed from 40% to 42% NTE30 First of two NTE events in NRTC Seconds used from NRTC: 627-629, 657­ 664, 685-696, 714-722 1 Speed (rpm) = Curb Idle + (Speed % * (MTS - Curb Idle) 2 Torque (lbf-ft) = Torque % * Maximum To rque At Speed (i.e. lug curve torque at speed) 3 Torque (lbf-ft) = Maximum of (32 % * pe ak torque) and the torque at speed that produces (32 % * peak power)

33TABLE 7. DYNAMIC RESPON SE INTER-NTE EVENTS

INT Event 1 Duration (s) Frequency Description

INT1 10 1 Initiation of cycle; INT1 is always first INT2-6 2 5 Shortest and most frequent inter-NTE events INT7-10 3 4 Short and frequent inter-NTE events INT11-14 4 4 Short and frequent inter-NTE events INT15-18 5 4 Short and frequent inter-NTE events INT19-21 6 3 Short and frequent inter-NTE events INT22 7 1 Medium inter-NTE event INT23 8 1 Medium inter-NTE event INT24 9 1 Medium inter-NTE event INT25 11 1 Medium inter-NTE event INT26 13 1 Long inter-NTE event INT27 17 1 Long inter-NTE event INT28 22 1 Long inter-NTE event INT29 27 1 Long inter-NTE event INT30 35 1 Longest inter-NTE event INT31 5 1 Termination of cycle; INT31* is always last Interval speeds and torques are not identical , but they are clustered around zero torque and the speed at which 15% of peak power and 15% of peak torque are output.

34Torque (lbf-ft) Torque-Speed Domain
700 Lug Curve NTE S line 600 NTE T-S line 1 SS Lower Left 2 SS Lower Mid 3 SS Lower Right 500 4 SS Mid Left 5 SS Mid Mid 6 SS Mid Right 400 7 SS Upper Left 8 SS Upper Mid 9 SS Upper Right 300 NTE Cycle 22 LD NTE3 23 OH 50mph Cruise 200 24 OH 70mph Cruise 25 NR Small Dozer 26 NR Large Dozer 27 FTP NTE2 100 28 LD NTE6 29 NRTC NTE1 30 NRTC NTE2 0
0 500 1000 1500 2000 2500 3000 3500 4000 4500

-100 Speed (rpm)

FIGURE 9. EXAMPLE OF A NTE CYCLE

Prior to executing the first repeat, setup each PEMS and stabilize engine operation at the first inter-NTE operating point. Setup the PEMS according to 40 CFR Part 1065 and PEMS manufacturer instructions, includ ing any warm-up time, zero-spans of the analyzers and the setup of all accessories incl uding flow meters, ECM interpreters, etc. Then, when the test cycle starts, switch the PEMS’ to sample emissions from the engine. When the text cycle ends, switch the PEMS back to ambient sampling. Complete all post-test lab and PEMS validations accordi ng to 40 CFR Part 1065 and according to PEMS manufacturer instructions.

3.3.4 Data Analysis Discard from further data analysis any NTE events invalidated by any criteria in 40 CFR Part 1065 Subpart J. For each NTE i event (i=1 to 30), which was repeated 30 times per engine with a specific exhaust configuration (j = 1 to 30), calculate the transient median absolute deviation, MAD TRi , for m PM , where for each NTE i event, MAD TRi = median[|
NTE ij – median (NTE ij ) |]. Next calculate the difference of MAD by su btracting a corresponding steady-state MAD, MAD SSi for m PM . MAD TRi-SSi = MAD TRi – MAD SSi . To determine a corresponding
MAD SSi , calculate the PEMS MAD SS at each steady-state median lab value, and then use the median PEMS NTE i value along the median la b value’s axis to find MAD SSi for the corresponding MAD TRi . Do not extrapolate any MAD SSi beyond the minimum or

35maximum median lab values. Note that some MAD SSi values might be zero because the lab data for that median failed the F-test in the previous section. For any MAD TRi-SSi less than zero, set that MAD TRi-SSi equal to zero. Create a transient error surfaces using all of the MAD TRi-SSi
. Be sure to include any MAD TRi-SSi data points that are equal to zer o because they will affect the 1 st and 99 th percentile values.

363.4 ECM Torque and BSFC

3.4.1 Objective Compare the ECM-based torque and fuel ra te with that of the laboratory-based measurement using the Cummins 80 SS mode cycles. For the laboratory purposes, use the gas-based fuel flow values instead of th e measured fuel flow. Repeat the Cummins 80 SS cycle three times, and use the average values produced.

Use at least six engines for these experiments that include the one engine to be used in the PM PEMS program and Engine B, C, and D of the ACES program.

3.4.2 Data Analysis Use the acquired data pooled and normalized to % of max torque and % of maximum fuel rate to replace the manufacturer submitted erro r surfaces that were pr eviously used in the gaseous portion of the Monte Carlo simulation. Refer to section 2.4 for description and example of an error surface. Include any bias error, unless there is an assignable cause that would not occur in-use and the steering committee approves to eliminate such bias

error.

4 ENVIRONMENTAL CHAMBER

The environmental chamber tests challenge PEMS to a variety of environmental disturbances, namely electromagnetic inte rference, atmospheric pressure, ambient temperature and humidity, and shock and vibration.

During each of the tests, plus a baseline te st, the PEMS will cycle through sampling four different dilution preparations of aerosol particles that cont ain volatile hydrocarbon and elemental carbon using a particle generator that mimics the formation of diesel particles. The OC/EC will be used to determine the concentration levels needed for the PM generator. Essentially, after determining the steady-state points to run on the engine, the OC/EC semi-continuous instrument will be used along with the filter-based method. Then, for the concentration levels to be used with the PM generator, the OC/EC instrument will be used to set the PM generator to produ ce the desired composition and concentration levels, similar to those encountered under st eady-state. Three pa rticle concentration levels of 5, 10, and 15 mg/m 3 , as shown in Table 8, will be generated by the particle generator. Each concentration will be fed to the PEMS after applying dilution ratios of 6, 12, 20, and 30. For each concentration and d ilution ratio combination, the PM generator will be stabilized for 4.5 minutes, and data will be collected by the PEMS for 30 seconds. The test will continue for a period of 8 hours. The first six cycles of every test will serve to be the baseline before any environmental change is made.

37The temperature/humidity and pressure tests are designed to mimic real-world environmental disturbances with the magn itude and frequency of the disturbances adjusted to real-world conditions. Randomly sample a uniform distribution of probability for their i c. , from any minute of the test. By randomly sampling from the minutes of these tests the magnitude and frequency of the real-world error will be built into the error model, which is described in Section 2. The other environmental tests represent the full range of possible conditions. For these tests, randomly sample the normal distribution in Figure 1 for their i c .
For EMI/RFI and vibration, the instruments will be subjected to screening tests with HEPA filtered air to detect if there any changes in the response of the instruments. Based on these results, the MASC will decide if the pa rticle generator will need to be used with these tests.

For the vibration screening test, in order to avoid damage to the instruments, a frequency sweep will be used at low amplitude. The idea here is to detect the frequency that may trigger a response by the instrument, withou t doing any damage due to high amplitude.

TABLE 8. CONCENTRATION AND DILU TION RATIO SCHEDULE WITH PM GENERATOR

Raw PM Concentration, Dilution Ratio
DR1 DR2 DR3 DR4
6 12 20 30

 g/m3 Concentration at Above Dilution Ratio
 833.3 416.7 250.0 166.7
10000 1666.7 833.3 500.0 333.3
15000 2500.0 1250.0 750.0 500.0

4.1 Data Analysis for Environmental Tests

Reduce data by first calculating means fo r each 30-second period of stabilized measurements. Subtract from each mean th e respective baseline concentration. The results are errors or “deltas”. Correct each of these error distributions by removing their respective baseline variances, which were determined by quantifying PM Generator output with no environmental perturbations . Calculate the variance of each of the distributions. Subtract the respective base line variance from each calculated variance. Use the resulting difference in variance as the target variance for adjusting the error distributions. If the target variance is zer o or negative, leave al l error values of the distribution as is and do not proceed to the ne xt step. If the target variance is positive, iteratively solve to find a single numerical valu e that can be used to divide each error in a given distribution such that the resulting dist ribution has a variance equal to the target variance. Now each of the errors is corrected for baseline variance.

38Then, calculate the NTE result with all errors, in cluding torque and flow errors set to zero. This is the true value. Then subtract the true NTE value from the result with all errors and record this difference in one of th e 7 measurement allowance distributions: m PM times three calculation methods (torque-spee d, fuel-specific * BSFC, ECM fuel flow) times three PEMS manufacturers, except Sens ors and Horiba can not use the ECM fuel flow calculation method. Then proceed to the next NTE event in the nominal data set. Repeat the entire nominal data set over and over until all 7 measurement allowance distributions converge. Follow the data reductio n steps set out in Section 2 to select the final measurement allowance.

4.2 PM Generator Commissioning

The PM generator is developed by EPA. The PM generator can create various hydrocarbon mixtures along with solid partic le generation using carbon rods arcing. The PM generator is also equipped with a micro- proportional diluter, and is intended to simulate diesel exhaust particle phase compounds.

EPA will ship the PM generator to SwRI. EP A (Matt Spears) will train SwRI staff on using it. In addition, SwRI together with EPA may incorporate to it a soot particle generation mechanism that is different th an the carbon rod arcing, using instead a propane flame mini-CAST technology.

The PM generator will be used during atmospheric chamber testing, temperature and humidity testing, and may be used duri ng EMI/RFI and vibration experiments.

4.3 Baseline

4.3.1 Objective
The baseline variance will be established using an 8 hour baseline test in which the PM generator cycles through the same compositions and concentrations of PM used during the actual environmental tests. Mean values wi ll be determined from the first five cycles through the PM concentrations. Deviations (deltas) from these mean values during subsequent cycles through the concentrati ons will be used to determine the baseline variance. This variance will be subtracted from the environmental test results.

4.3.2 Background All of the other environmental tests inherently incorporate the baselin e bias variance of the PEMS. Because the Monte Carlo simulati on model adds all the errors determined from the various environmental tests, it woul d add the baseline variance of PEMS to the model too many times. In order to compensate for this in the model, the baseline variance of PEMS is determined and subtract ed from each of the environmental tests’ results.

Note that the baseline variance of PEMS is measured and modeled (i.e. added) once as part of the steady-state engine dyn amometer laboratory experiment.

394.3.3 Methods and Materials For this experiment use a well ventilated EM I/RFI shielded room capable of maintaining reasonably constant temperature and pressure . Use a room that can house one of each PEMS, their power supplies, the PEMS flow meters, cables and lines.

Prior to executing the baseline test, setup each PEMS and stabilize the PEMS in the room. Perform PEMS setup according to 40 CFR Pa rt 1065 Subpart J and PEMS manufacturer instructions, including any warm-up time, a nd audit. Then supply the PEMS’ sample ports with the sequence of PM from the PM generator as described at the beginning of Section 4.

At each PM concentration, flow PM long eno ugh so that stable readings of the PEMS can be recorded. When the OC/EC analyzer is used to spot-check the output of the PM generator, ensure that enough time has elapsed to achieve an accurate OC/EC analysis.

Position PEMS and configure PM transport t ubing to minimize transport delays and PM losses.

Test at least one PEMS fro m each PEMS manufacturer.

4.3.4 Data Analysis Reduce the baseline data for each PM PEMS, us ing artificial NTE sampling event times. Subtract from each m PM the mean m PM from the initial (short) baseline test of six cycles through the PM concentrations, which were co nducted at the beginni ng of the test. The results are errors or “deltas”. Calculate th e variance of these values, and use them for baseline variance correction in the data reduction of the remaining environmental tests.

4.4 Electromagnetic Radiation

4.4.1 Objective Evaluate the effect of Electromagnetic Interference (EMI) and Radio frequency Interference (RFI) on the performance of th e PEMS and determine error factors for the PEMS due to these effects. First, a screening test on each instrume nt will be performed with HEPA filtered air to determine if the EM I/RFI affects the instrument response. If it does, the MASC will decide on the test matrix required for this evaluation.

4.4.2 Methods and Materials Use an EMI test facility capable of runni ng the SAE tests listed above. This would include: Signal generators, Power amplifiers , Transmit antennas, Electric Field Sensors, Measurement Receiver, Data recording devi ce, LISNs (Line Impedance Stabilization Networks) and shielded enclosure.

404.5 Atmospheric Pressure

4.5.1 Objective Evaluate the effects of ambient pressure on PEMS PM concentration outputs.

4.5.2 Background PEMS are expected to operate over ranges of ambient pressures. It is hypothesized that some of the errors of the PEMS concentr ation outputs may be a function of ambient pressure. Therefore, this experiment wi ll change the ambient pressure surrounding PEMS to evaluate its effects on PEMS measured concentratio ns and flow meter transducer outputs. As with al l of the environmental tests, the test cycle for this test is based on the best-known distribu tion of real world conditions. For this test, the test cycle pressure distribution was matched to the co unty-by-county annual average atmospheric pressure distribution in EPA’s 2002 National Emissions Inventory (NEI) model. Figure 10 depicts the NEI data distribution (based on 3149 data points) and the test cycle pressure distribution.

Pressure Histograms

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

84.1 86.8 89.6 92.3 95.0 97.8 100.5 Pressure (kPa)Frequency (% of total time) 2002 NEI Pressure Distribution <5500 ft

Test Cycle Pressure

FIGURE 10. PRESSURE HISTOGRAM

4.5.3 Methods and Materials
Use a barometric chamber that can be well ve ntilated and capable of controlling a wide range of pressure changes (82.74 to 101.87 kPa). Use a chamber that can house at least

41three PEMS at a time, one of each PEMS ma nufacturer, their power supplies, the PEMS flow meters, cables and lines, plus the PM generator.

Follow a pattern of first soaking the PEMS at a constant pressure, then ramp the pressure to a new pressure, soak the PEMS at that new pressure, and then ramp to another pressure. Use the sequence of pressures and times, as shown in Table 9, to simulate a typical distribution of real-world pressures and ch anges in pressure, which are believed to be dominated by changes in altitude during driving in the United States.

TABLE 9. ATMOSPHERIC PRE SSURE TEST SEQUENCE

Atmospheric Pressure Test Sequence

Phase Pressure Time Rate Comments kPa Alt. ft. min ft/min 1 Soak 101 89 10 0 Flat near sea-level 2 Ramp 101-97 89-1203 20 56 Moderate hill climb from sea level 3 Soak 97 1203 20 0 Flat at moderate elevation
4 Ramp 97-101.87 1203- -148 60 -23 Moderate descent to below sea level 5 Soak 101.87 -148 20 0 Fl at at extreme low elevation
6 Ramp 101.87-101 -148-89 20 12 Moderate hill climb to near sea level 7 Soak 101 89 20 0 Flat near sea level 8 Ramp 101-97 89-1203 20 56 Moderate hill climb from sea level 9 Soak 97 1203 25 0 Flat at moderate elevation
10 Ramp 97-96.6 1203-1316 20 6 Slow climb from moderate elevation 11 Soak 96.6 1316 20 0 Flat at moderate elevation 12 Ramp 96.6-82.74 1316-5501 20 209 Rapid climb to NTE limit 13 Soak 82.74 5501 20 0 Flat at NTE limit 14 Ramp 82.74-96.8 5501-1259 30 -141 Rapid descent from NTE limit 15 Soak 96.8 1259 20 0 Flat at moderate elevation 16 Ramp 96.8-90 1259-3244 15 132 Rapid hill climb to mid elevation 17 Soak 90 3244 10 0 Flat at mid elevation
18 Ramp 90-96.8 3244-1259 20 -99 Rapid descent within middle of NTE 19 Soak 96.8 1259 20 0 Flat at moderate elevation
20 Ramp 96.8-99.2 1259-586 20 -34 Moderate descent to lower elevation 21 Soak 99.2 586 20 0 Flat at lower elevation 22 Ramp 99.2-101 586-89 10 -50 Moderate decent to near sea-level 23 Soak 101 89 20 0 Flat near sea-level

42_ _____ Pressure-Time Environmental Test Cycle

9000

7000

5000

3000

1000

-1000

_ _____ Altitude (ft)

_ _ _ _ 1/2-hr moving avg dA/dt (ft/hr)

_ _____ Vertical gridlines = hours Vertical gridlines = 7-min gas cylinder cycle times 71
75
79
83
87
91
95
99
103
107
111
115
119
123
127Approximate Pressure (kPa)Altitude (ft) & dA/dt (ft/hr)

-3000

-5000

-7000

-9000 131
0 1 2 3 4 5 6 7 8 Time (hr)

FIGURE 11. PRESSURE-TIME ENVIRONMENTAL TEST CYCLE

Prior to executing this pressure sequence, setup each PEMS and stabilize the PEMS in the chamber’s first pressure. Perform PEMS setup according to 40 CFR Part 1065 Subpart J and PEMS manufacturer instructi ons, including any warm-up time, zero-spanaudits of the analyzers and the setup of all accessories including flow meters, ECM interpreters, etc. Then supply the PM PEMS’ sample port with the sequence of PM from the PM generator as described at the beginning of Section 4.

Flow each generated PM sample long enough so that at least 30 seconds of stable readings are recorded for the slowest res ponding gas concentration output of all the PEMS. Position PEMS and configure gas trans port tubing to minimize transport delays. Target to sample about 30 seconds. Repeat th is cycle over the 8-hr test cycle, by cycling through the concentration shown in Table 8, wh ich represents one hour of testing, using a 4.5 minutes of stabilization and 30 se conds of sampling at each condition.

Perform this test once for one set of PEMS with as many PEMS tested at once.

4.5.4 Data Analysis Perform data analysis ac cording to Section 4.1.

434.6 Ambient Temperature and Humidity

4.6.1 Objective Evaluate the effects of ambient temperatur e and humidity on PEMS PM concentration outputs. The histogram in Figure 12, along with Table 10 and Figur e 13, will be updated by a new temperature profile that takes in to consideration the data generated by CECERT.

4.6.2 Background PEMS are expected to operate over a wide rang e of changing ambient temperatures. It is hypothesized that some of the errors of th e PEMS outputs may be a function of changes in ambient temperature. Therefore, this ex periment will change the ambient temperature surrounding PEMS to evaluate its effects on PEMS measured concentrations and flow meter transducer outputs. As with all of th e environmental tests, the test cycle for this test is based on the best-known distribution of real world cond itions. For this test, the test cycle temperature distribution was matc hed to the hour-by hour county-by-county average atmospheric temperature distributio n, weighted by vehicle miles traveled according to EPA’s 2002 National Emissions In ventory (NEI) model. Figure 12 depicts the NEI data distribution (based on over 900,000 temperatures and over 270 trillion vehicle miles) and the test cy cle temperature distribution.

Temperature Histograms

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

28.5 39.5 50.5 61.5 72.5 83.5 94.5 Temperature (F)Frequency (% of total time) Real World Temperature 23 -100 F

Test Cycle Temperature

FIGURE 12. TEMPERATURE HISTOGRAM

444.6.3 Methods and Materials Use a well ventilated room capable of controlling a wide range of temperature changes (­ 23 to 100 °F). Use a room that can house at least six PEMS, their power supplies, the PEMS flow meters, cables a nd lines, plus seven different zero, audit, and span gas cylinders, and a gas switching system.

Follow a pattern of first soaking the PEMS at a constant room temperature, then ramping the room temperature to a new temperature, soaking the PEMS at that new temperature, and then ramping to another temperature. Use the following sequence of temperatures, shown in Table 10, and times to simulate the range of real-world temperatures and changes in temperature:

TABLE 10. AMBIENT TEMPE RATURE TEST SEQUENCE

Ambient Temperature Test Sequence

Phase Temperature Tim

e Rate Comments °C °F min °C/mi

n
1 Soak 13.89 57 10 0.00 Cool in-garage pre-test PEMS operations 2 Ramp 13.89-5.00 57-23 5 -3.78 Leaving cool garage into cold ambient

3 Soak -5.00 23 5 0.00 Operating at cold temperature outside of vehicle 4 Ramp -5.00­ 12.78 23-55 145 0.12 Diurnal warming during cool day

5 Soak 12.78 55 40 0.00 Steady cool temperature during testing 6 Ramp 12.78­ 28.33 55-83 5 3.11 Return to hot garage on a cool day

7 Soak 28.33 83 52 0.00 Hot in-garage pre- post- test PEMS operations 8 Ramp 28.33­ 37.78 83-100 5 1.89 Leaving ho garage into hot ambient

9 Soak 37.78 100 8 0.00 Operating at hot temperature outside of vehicle 10 Ramp 37.78­ 22.22 100-72 100 -0.16 Diurnal cooling during hot day

11 Soak 22.22 72 60 0.00 Steady moderate temperature during testing 12 Ramp 22.22­ 13.89 72-57 5 -1.67 Return to cool garage on a moderate day

13 Soak 13.89 57 40 0.00 Cool in-garage post-test PEMS operations

45_ ____ Temperature-Time Environmental Test Cycle

-60 -50 -40 -30 -20 -10

0 10 20 30 40 50

0 1 2 3 4 5 6 7 8 Time (hr)Temperature (C)

-76 -64 -52 -40 -28 -16 -4 8 20 32 44 56 68 80 92 104 116Temperature (F)

_ ____ Temperature

_ _ _ _ 1/2-hr moving average dT/dt (C/hr)

_ ____ Vertical gridlines = hours Vertical gridlines = 7-min gas cylinder cycle times

FIGURE 13. TIME SERIES CHART OF AMBIENT TEMPERATURE TEST

Prior to executing this temperature sequence, setup each PEMS and stabilize the PEMS in the chamber’s first temperature. Perform PEMS setup according to 40 CFR Part 1065 Subpart J and PEMS manufacturer instructi ons, including any warm-up time, zero-spanaudits of the analyzers and the setup of all accessories including flow meters, ECM interpreters, etc.

Run the 8-hour cycle test by stepping throu gh the concentration and dilution ratio shown in Table 8.

4.6.4 Data Analysis Perform data analysis ac cording to Section 4.1.

4.7 Orientation and Vibration

4.7.1 Objective
Evaluate the effect of vehicl e vibration on the performance of the PEMS and determine error factors for the PEMS due to these effect s. Prior to doing ex tensive vibration work, perform a screening using HEPA filtered air sampling at a sweep of different frequencies with low amplitude. If any of the PEMS shows a response to a particular frequency, propose a frequency test and submit it for the MASC for approval.

465 SWRI CVS AND CE-CERT TRAILER CORRELATION

Prior to performing the in-use work with the PM-PEMS, it is important to establish the degree of correlation between SwRI CVS-based PM measurement and CE-CERT CVS-based PM measurement. For this pur pose, the CE-CERT trailer will move to SwRI facilities and PM measurement will be conducted on the engine used for the PM-PEMS program.

5.1 Method and Materials

Below is a list of a step by step appro ach for the correlation between SwRI and CECERT

1. Perform a propane check on SwRI CVS and 47 mm filter and CE-CERT CVS and 47 mm filter. Both systems should pass Part 1065 on propane. However, even if they pass, not e any difference between the two. 2. Set the CVS flow rate to be the same on both systems 3. Set the filter face temperature and velo city to be the same on both systems 4. Set the secondary dilution ratio to be the same on both systems. 5. Use Whatman PTFE membrane filters (7592-104), and filter screens that meet the latest Part 1065. 6. Modify the exhaust path to SwRI CVS to be comparable with that for the CECERT Trailer 7. Pre-condition the SwRI CVS tunnel a nd the CE-CERT trailer CVS tunnel for a period of 10 hours at engine rated power using exhaust configuration with DPF without bypass. The conditioni ng time may include active DPF regenerations. 8. Run a total of 12 repeats of the NTE transient cycle using DPF with Bypass Level at 0.025 g/hp-hr emission level, over a period of three days. Four repeats per day with the CE-CERT follo wed by four repeats with SwRI CVS and then alternate. Prior to each set of four repeats manually regenerate the DPF. 9. Use SwRI DMM-230 and CE-CERT DMM-230 to make sure that the engine PM source is not shifting and being consistent. 10. SwRI should handle and weigh all the filters for both SwRI and CE-CERT in accordance with their protocol. 11. The CE-CERT trailer is needed at SwRI for at least two weeks per engine. One week for setup and two weeks of te sting assuming the above schedule. 12. In a separate task, EPA will equilibrat e and pre-weigh 20 filters using EPA’s weighing protocol. EPA will then ship them to SwRI for repeat preweighing using their protocol. SwRI will then ship the same filters to EPA for reweighing. After reweighing at EPA, EPA will ship the filters to CE-CERT for weighing using CE-CERT’s weighing protocol. Finally, CE-CERT will

47ship the filters to EPA for reweighing. Results will be reported by EPA for MASC discussion. No threshold for acceptance has been established at the time of this testplan writing. 13. The target for correlation at the 0.02 5 g/hp-hr level is CE-CERT’s mean of 12 repeats being within +/-10% of the mean value reported by SwRI.

6 MODEL VALIDATION AND MEASUREMENT ALLOWANCE DETERMINATION

6.1 Model validation

6.1.1 Objective Validate the Monte Carlo model by 1. Testing the PEMS in parallel with the CE-CERT trailer 2. Checking the data to see if it fits the model predicted based on the laboratory efforts

6.1.1.1 CE-CERT Validation

The difference between the PEMS results and the CE-CERT trailer results will be compared to the error predicted by the Mont e Carlo model. To validate the Monte Carlo model, data must be run through the model an d the model results must predict the actual test results within a reas onable level of accuracy.

Validation will be based on the following proce dure. For each reference NTE event, the Monte Carlo model will be used to generate the 5 th and 95 th percentiles of the simulated distribution of the brake-speci fic PM emission differences. In order to obtain simulations representing similar conditions to those obta ined on-road, some error surfaces may need to be suppressed in the simulations since not al l of them may be applicable to the on-road conditions. The choice of which error surfac es to suppress would n eed to be made by the Steering Committee.

Next, the 5 th and 95 th delta percentiles obtained from the above simulations will be separately fit to a line or curve using two chosen methods: a linea r regression procedure and a local regression (loess) technique 1 . Depending on which of the resulting two fits is best for each set of data (i.e., either for the 5 th percentile deltas or the 95 th percentile deltas), the resulting line or curv e will be used as one of the lower or upper limits for the on-road data.

To determine the best fit for a given set of delta percentiles (i.e., 5 th or 95 th ), a simple regression line initially will be fit to the data. If a leas t squares linear regression of the 5 th
or 95 th percentile deltas versus the ideal PM emission has an r 2 >0.85 and an SEE < 5 % of the median ideal PM emissions, then the regression line will be used. If this set of criteria is not met, then a loess fit will be used. Since a loess regression requires the

48selection of a smoothing parameter 2 to smooth the data, the chosen smoothness parameter should balance the residual sum of square s against the smoothness of the fit.

The on-road delta errors, obtained from the results of collecting data on several NTE events during on-road operations, will be plotted on a graph containing the 5 th and 95 th percentile delta limits determined from the re gression fits chosen above. The graph will consist of a plot of delta PM versus ideal PM. The number of on-road points outsidethese limits will be determined and expressed as a percentage of the total number on onroad data points. If this number does not ex ceeds 10% of the total nu mber of on-road data, the simulation data will be considered to be valid.

6.2 Measurement Allowance Determination

6.2.1 Objective Use the Monte Carlo simulation program develope d with data from sections 2, 3 and 5, and validated with section 5.1 to determine the measurement allowance for all regulated emissions, at 2007 emissions standards.

6.2.2 Background
After the Monte Carlo model has been validated and confidence in its ability to predict errors from PEMS instrumentation, the last step in this program will be to actually calculate a single set of m easurement allowance for PM.

6.2.3 Methods and Materials Using the criteria explained in section 2.2 calculate the various le vels of measurement accuracy corresponding to the three PEMS manufacturers and the brake specific PM emissions calculations. Use all the various error surfaces developed during this test program, including those provided by engine manufacturers to the EPA and ARB.

6.2.4 Data Analysis Use the methodology explained in section 2.2, and Table 2.2 to arrive at the final measurement allowance.

497TIME AND COST7.1TimelineTable is a tentative timeline projecting themajor tasks to be accomplished during thisprogram. The additional work if neededoptionis the work that may need to be done if the model did not validate. Otherwise, the final report will be submitted by September30,2009.TABLE 11. PROJECTED PM-PEMS TIMELINE7.2 Cost

The rough estimated cost is shown in Tabl e 12. Based on the current estimate, a $125,000 of the $200,000 is needed to complete the project.

TABLE 12. PROJECTED COST ESTIMATE

PEMS Training, Setup, Audit, and Debug $660,000 Steady-State and Transient Experiments $190,000.00 SwRI and CE-CERT Correlation (1 engine) $75,000.00 PM Generator and Environmental Testing Activities $200,000.00 Modeling Activities (Including CO 2 ) $225,000.00 Data and analysis, reporting, and final report $150,000.00 Contingency if validation fails $100,000.00 General Contingency $200,000.00

Grand Total $1,800,000.00 Grand Total Without General Contingency $1,600,000.00 Grand Total without General Contingency and Contingency if Validation Fails $1,500,000.00

8 ABBREVIATIONS USED IN BRAKE SPECIFIC EQUATIONS

Method 1:

ePM = brake-specific emission, PM (g/hp-hr) N = total number (of time intervals) in series x = amount of substance fraction (mol PM/mol exhaust; note that 1  mol (emission constituent)/mol (exhaust) = 1ppm (part per million)

n . = amount of substance rate (mol/sec, in this case, mol (exhaust)/sec
�t = time interval (sec) fn = rotational frequency (shaft), rev/min T = torque (N-m)

NOTE: The units of the numerator work out to gemission as is. However, using the units given for the denominator (RPM * N-m * s), you would still need to divide by 1.978 to get to hp-hr (using RPM * N-m = kW * 9550, 1 hour = 3600 sec, and kW = hp*0.7457)Method 2:

ePM = brake-specific emission, PM (g/hp-hr) MNO2 = Molecular weight, NO2 (~46 g/mol) N = total number (of time intervals) in series x = amount of substance fraction (mol PM/mol exhaust; note that 1  mol (emission constituent)/mol (exhaust) = 1ppm (part per million)

~

n . = amount of substance rate (mol/sec, in this case, mol (exhaust)/se c) that is linearly
proportional to n . (Note: this is a proportional sa mple, which means that you may use a flow meter that has a span error, as long as its calibration is linear)
�t = time interval (sec) MC = Atomic weight of carbon (~12 g/mol) wfuel = g (carbon)/g (fuel); Note fuel is roughly 86% carbon by mass xCproddry = amount of carbon products on a C1 basis per dry mol of measured flow (exhaust), mol/mol, solved iteratively per 1065.655 xH2O = amount of water in measured fl ow, mol/mol (see 1065.645 for calculations) efuel = brake-specific fuel consumption (g (fuel)/hp-hr)

Method 3:

ePM = brake-specific emission, PM (g/hp-hr) MNO2 = Molecular weight, NO2 (~46 g/mol) wfuel = g (carbon)/g (fuel); Note fuel is roughly 86% carbon by mass MC = Atomic weight of carbon (~12 g/mol) N = total number (of time intervals) in series

x = amount of substance fraction (mol PM/mol exhaust; note that 1  mol (emission constituent)/mol (exhaust) = 1ppm (part per million)

m . fuel = mass rate of fuel (g/sec) xH2O = amount of water in measured fl ow, mol/mol (see 1065.645 for calculations) xCproddry = amount of carbon products on a C1 basis per dry mol of measured flow (exhaust), mol/mol
�t = time interval (sec) fn = rotational frequency (shaft), rev/min T = torque (N-m)
�t = time interval (sec) NOTE: The units of the numerator work out to gemission as is. However, using the units given for the denominator (RPM * N-m * s), you would still need to divide by 1.978 to get to hp-hr (using RPM * N-m = kW * 9550, 1 hour = 3600 sec, and kW = hp*0.7457)

52