Document ID: EPA-HQ-OAR-2004-0072-0091
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2008-03-13T04:00Z

Heavy-Duty In-Use Testing (HDUIT) Measurement Allowance Test Plan
Meeting Minutes 2005-2007

HDIUT Meeting Minutes 12-8-05

CSTF Meeting at SwRI starting at 1 pm Monday January 30th through
February 1st.  One day discussion on measurement allowance work. 
Gaseous HDIUT allowance start at noon on 31st.  Probably close at noon
on 1st.

Follow-up meeting in March around the 14th or 15th in Ann Arbor.  15th
will be HDIUT meeting.

Mason presentation on convergence criteria for iterating model.  Around
10,000 iterations are needed for 95% confidence interval.

CE-CERT proposal

A discussion ensued on what will be done with the CE-CERT data.  How
will it be used to validate the SwRI testing?  It was agreed that it
will be a coarse comparison of model validation.  A request was made for
a summary write up to state what the CE-CERT results will be used for. 
The lower bound write-up exists, but the upper does not.  There is some
inherent uncertainty in the model validation step.  CE-CERT funding
issue.  Would like to test an engine on engine dyno, compare SwRI to
CE-CERT MEL.  It is possible that they will test the RR engine.  We
would not be able to fund CE-CERT’s participation on the RR.  It would
have to be on a voluntary basis.  We would like to fully fund Task 1, 2,
3, 4, and 5; but not the RR.  Follow-up conference call between EPA, ARB
and CE-CERT to discuss funding of the options.  Project schedule Task 1
timing was OK, but tasks 2 to 5 need to be rescheduled.  Task 2
(correlation to SwRI) needs to be moved to August/September.  Task 3
(on-road test) in October, Task 4 and 5 will need to be complete in the
first and second weeks in November.  Would need a comprehensive report
in that time frame.  Task reporting could take place as late as
December.  Need to get contract funding straightened out.  It will take
a long time if we need to establish a new contract.  Need to find an
existing contract to establish funding .  This will save time.

DFR must be out by April 2007 or it will skip a whole model year. 
Delays from January to March will push the effective date by 1 month
respectively.  The MOA needs to be adjusted to match the schedule of the
test plan.  EPA and EMP\A will work to revise the MOA.

Discuss differences in SwRI NOx measurement vs. EPA in RR.  SwRI is
reading 9% high.  SwRI found a 3% difference between span port and
overflow port measurements.  This, however, only accounts for 3 of the
9%.  SwRI does not perform overflow zero and span during calibration. 
There also was a difference in how the engine ran.  SwRI had less power,
but more NOx.  This is opposite the NOx/power trade off.

NMHC analyzer availability.  Do we do the testing with only two NMHC
analyzers or just use the 0.98*THC function.  If it doesn’t look like
anyone is going to use the NMHC analyzer, why test?  Some engine
manufacturers would like to see the NMHC results.  Note the current NMHC
Semtech upgrade is a box separate from the main Semtech unit.  For
steady-state testing, one of the NMHC analyzers will be swapped from
Semtech to Semtech so that three can be tested.  Also, NMHC analyzers
can be alternated for randomness.  When three Semtechs are used
simultaneously, use data from one of the NMHC analyzers twice. 
Sensor’s NMHC analyzer is AC powered.  We are not sure how well
developed the analyzer really is.  So, it was decided that the NMHC
analyzer would not be tested and the 0.98*THC would be used to determine
NMHC concentrations.

Appropriateness of EMI/RFI shock and vibration testing.  Current test is
rough as stated in test plan.  Time that instrument is subjected to the
test is too long, making the test quite severe.  Damage is 40 times what
Sensors has seen in the road test.  Test duration is 8 hours.  Sensors
would like the time reduced.  The test is a military specification for
something surviving shipping.  It appears that the severity of the test
should be reduced while maintaining the duration.  Cummins suggested
that we ask SwRI’s vibration experts how the current test should be
scaled down.  Then it would be not so much of an accelerated reliability
test.  It was decided that we would make some sort of a change to the
current 8 hour test.  1 hour equals 1000 miles of testing.

CE and SAE EMI/RFI testing.  SAE requirements are generally used for
under the hood ECM tests.  This is most likely too severe for our
application due to the high frequency required in the test.  Atul stated
that EMI/RFI spec in the CE test should be more than adequate.  SwRI
will follow up on this and explain both tests.

Edits to test plan.  Discussed added language.  No prototypes or betas
are allowed.  Any changes to the hardware must be steering committee
approved.  A discussion of Table 1 was performed.  EPA will used
existing data sets for development of reference data sets to test
SwRI’s model.  Will use existing 20 OH and NR events along with 30 NTE
transient cycles and 1 set from DDC.  Will scale and create about 200
unique NTE events.  Data will fit within corresponding surfaces.  Also
scale torque, speed, bsfc, and fuel flow if necessary to exercise those
error surfaces.

PM HDUIT

Will be testing over road using Dodge truck retrofitted with DPF and DPF
bypass in 2006.  Bypass will have an uncoated flow through substrate.

Gaseous proportionality testing, fast FID and propane.    Will include
QCM and MPS.

Establishing some sort of reference PM generation.  Reference carbon PM,
semi-volatile PM, H2SO4, and humidity.  This will be done using QCM,
CPCs, and EC/OC analyzers.  Idea is to create a reference PM generator. 
Permeation tubes will be used for C18 to C28 volatilization.  Will use
an electric arc furnace to generate black carbon (99.99% pure).

In near term, repeating some of E-66 testing with MPS and filters.  Will
test in HDD site 5.  Will consider use of EC/OC analyzers.

What PM PEMS will be available by the end of 2006?  QCM 8 crystal
instrument is imminent.

Will test 3 trap formulations on 2007 engines.  Skip ECM evaluation. 
Dilution air challenge.  Need end of stack vibration test (Fleetguard). 
Temperature, pressure, and EMI/RFI.

HDIUT Meeting Minutes 1-31-06

Next measurement allowance meeting the afternoon of March 14th.

Tuesday 31st, afternoon

A revised timeline was presented.  Testing really hasn’t started. 
Have just done engine install and PEMs audit.  Timeline has slipped. 
Some contractual issues, some due to RR.  First three weeks of January
was down time due to contract issues.  EV tests in August/September. 
CE-CERT testing would start in October.  Reporting will still be going
on into 2007.  The three measurement allowances should be ready by
December in time for the DFR.  Target FR notice date is January 14,
2007.  Hector was concerned about CE-CERT’s flexibility.  

Horiba may have a unit available in time to participate in April, engine
2 testing.  They should also be able to participate in EV tests and
CE-CERT testing.

Data distribution was discussed.  It was agreed that data would be
shared with Sensors and Horiba.

There was a discussion of the timeline.  Per the MOA, if the final
report and measurement allowance values are not available by February 1,
2007 for inclusion into the FRM, the implementation date of the FEP is
delayed by one year.

Wednesday 1st, morning

On the problem with the torque curve in table 3.5.3.1a,  17%, 100% was
choose to replace the double 100%, 100% point.

There was a discussion on which fueling parameters will be needed for
error surface interpolation.  The third engine will not have BSFC via
J-1939.  In fact it does not have any J-1939 outputs.  We will use
International’s proprietary software (engineering tool) to pull the
Torque and Fuel data and then input it into the test cell data logger.

Environmental test discussion

In the current mil spec for vibration, 1 hour of testing is the
equivalent of 1000 miles of transport.  So this is an accelerated
durability test.  We targeted 8 hours just so that we could see the
contribution of shock and vibration over a typical day of testing.  The
mil spec had 3G peaks.  SwRI will come up with a new test indicative of
a typical 8 hour test.  It will be non-accelerated.

The EMI/RFI test was discussed.  The current test is too severe as it
was designed for an engine ECM.  We need something that is a close to a
CE test.  Something that is at the 95th percentile worst case is
desired.  SAE has some different test levels in SAE HS-3600, 2004
edition.  We will need to pick one of the regions.  We decided to go
with Class B Region II.  Class B definition is and function that
enhances, but is not essential to vehicle operation.  The test signal E
field severity is 0.3 X E (volts/m).  The bulk current injection limit
will be 60 milliamps.  The other limits will be sent out later via
e-mail.

The application of a fixed warm-up delta to about 1% of NTE events was
discussed (3.5.2.2).  At issue is the time to stability for entering the
NTE zone.  Very few NTE events are warm-ups.  So for these warm-ups a
correction needs to be made.  The agreement made was to leave it as
sample the warm-up 2% of the time until thermostat opening, temperature
is stable, and take the mean after the engine is out of warm-up.  The
mean error will be applied to 2% of the Monte Carlo runs.  At least a
250°C exhaust temperature.

The results of the 1065 audit were discussed.  The slope and intercept
were off slightly on the NO analyzer of one of the PEMs.  This will be
sent to Sensors for relinerization.  One 5”, three 4”, and one 3”
flow meters failed.  The 5” was sent back for recalibration and it now
passes.  These will be sent back to Sensors for recalibration.

HDIUT Meeting Minutes @ SwRI, 4-13-06

Program Schedule Update (Sharp)

Final report December 1st.  Schedule slide one month.  EV test start May
1st.  Plan on using same PEMs.  3 week slide is allowable without
hindering CE-CERT schedule.  2 weeks until NO2 fix is needed.  CE-CERT
correlation with engine 2 C-9.  SwRI has received the request for
overtime from Cincinnati.

Engine 1 Steady-State Results

Went over 10 selected SS points.  Points 40, 37, and 27 are not
correlating well.  SwRI not sure why.  About 4% low (dilute
measurement).  Could be due to dilution ratio.  CAT made statement about
need to still use Raw and dilute NOx.  This will affect correction. 
Mass comparison of lab raw vs. lab dilute shows much better agreement. 
Cummins supports.

Improved background bag measurement procedure.  Added more purges and
switched to baked Tedlar to reduce bag out gassing (0.2 ppm).  Lowered
vacuum on bag evacuation to prevent crinkling.  International wondered
if this exotic use would adversely affect measurement allowance.

Data action items.  Reliable charts.  Reprocess data. Will be discussed
at next conference call.

Environmental Testing – Discussion of any shock and vibration testing
open issues.  EFI/RFI spectrum analyzer cost, complexity.  Test plan
calls for 2 three-axis accelerometers.  PEMs location.  Push is for
mounting to frame, so testing should reflect that.  SwRI only has data
on cab, not on frame rail.  SwRI recommends that we gather data from
frame.  They do have data from frame (thee locations).  IM1 is chassis
frame location.  Will test 1 or 2 PEMs based on Horiba availability. 
Action item, SwRI will get chassis data and we will discuss during next
conference call.  Agreed to vibration data for CE-CERT testing.  Based
on EV test results at SwRI.  Also would like to decide on CE-CERT
EMI/RFI based on EV results.  This would sweep for frequency ranges. 
$40,000 for vibration if done at SwRI.  Much more expensive for EMI/RFI.
 Will table until results of EV data is available.

EMI/RFI Class B Region II SAE JSS1-1.  Do not have intensities for all
of the tests put together, -4, -11, -13, -27.

Update on Monte Carlo Model

Action item, discuss getting mol/s from Semtech.  5 engine companies
will provide 100 NTE events for model validation.  These will be
provided by June 1.  Ask EPA about June 1st date on manufacturer Torque
and Speed.  What transformation does EPA need to perform.  What if EMA
submits it in the same format?  Can it be more near July 1st then?

NO2 loss correction for DDC data

Sensors progress to remedy 1065 deficiency at high NO2 conc., (Shah -
Sensors).

Max NO2 is about 180 ppm, 106 ppm average.  During 40 pts, DPF
temperature never got below 300C.  There are two NO2 loss mechanisms.

Open Issues 

CO2 error surface?  Committee agreed on the addition of a CO2 error
surface.  Also agreed that HC = NMHC on Method 3.  Agreement adds 6 more
error surface for CO2.

Emission control systems for the next two engines.  Options?  Insuring
that ECS chosen is representative of a 2007 engine and ECS platform. 
Possible use of non-catalyzed DPF for second engine.  EMA will take
lead.

EMA arrangements for trap-equipped 2.5g/hp NOx HHDV to be ready and at
CE-CERT NLT July 21, 2006.  Will Caterpillar provide this vehicle
because Cat volunteered to champion data collection issues for the
subsequent replay portion of model validation?  CAT will provide
vehicle.

NMHC reference values.  Discuss using lab raw as the reference.

FID fuel.  EPA will small ship bottles to SwRI for use during
environmental and over the road testing only.  Will use SwRI cylinders
for test cell testing.  Need steering committee agreement/approval.

Span gases:  Gas concentration renaming and gas error tolerance. 
Discussion of SwRI naming vs. using concentrations as determined by gas
supplier.  About a 2% error on SwRI naming of span bottles due to CO2
quench. (12% CO2 in bottle).  Agreed, will not have SwRI name span
bottles.

Administrative (next meeting, confirm regular conf calls etc)

HDIUT Meeting Minutes 5-23-06

SwRI fuel flow measurement improvements have been achieved.  Rerun SS
map last week.  DOE should be done tomorrow.  4 units will be upgraded
at SwRI Thursday of this week.  The other 2 Semtechs will arrive Monday
the 29th.  SwRI will rerun NO2 audits early next week.  Also will
recheck analyzer cal curves as it should be time per 1065.

Not sure if testing on engine 1 or 2 when CE-CERT shows up.

Test plan not specific on what is correlation and what happens if SwRI,
PEMs, and CE-CERT do not correlate.

Next meeting in SA June 28 – 29.

Will reshuffle EV testing so that multi PEMs tests will occur first. 
This will enable CE-CERT, EV, and engine 2 testing to occur
simultaneously.

With reshuffle, looking to be 1.5 months short, finishing December 8th.

Talk of modifying EV test.  There is only ambient humidity.  Cummins
wants wet gasses flowing through for EV test.  Only way to check that
there is no water condensation issues with humidity sensor.  EMA
suggested going with DDC DPF to increase NO2 concentrations to further
test Semtech.

Sensors will use RH, ambient T, and NO2 concentration to correct for NO2
loss.  Humidity sensor will be located outside of manifold, to prevent
water impacting RH measurement.  No change in SS coil or chiller. 
Sensors will share additional data in future.  Stated that dissolved NO2
conversion to HNO3 causes the % increase in NO2 loss as NO2
concentration increases.

Still a push on revising the EV tests for humidity.  How do you humidify
the gas?

CAT wants to reapply 3.5.2.2 to all of the time.  Not warm-up errors,
but interactive parameters.  AK and AZ examples.  CAT calls for 100% of
the time to roll the dice.

Wind test data was reviewed.  Will take std. dev. of the points through
0.  If results don’t show anything we will drop for consideration for
error surface.

CAT concerned that there is no SS interpolation for torque.  Not a
legitimate error surface.  EPA, 20 pts are reasonable.  Agreement to go
from single point to multipoint.  See table 3.5.4.4-a.  It turns out it
is described in TP on page 41.  Issue resolved.

Next conference call, discuss warm-up and interpolation of error
surfaces.

Swirl test – Waiting on SS data to compare flows for bias correction. 
Data shows trend with delta.  Note 5 repeats at each set point.  There
was a question on if parametric statistics should be used.  Mason will
be asked.  Was data normalized?

Pulsation results – need bias correction.  Data indicates that
pulsation has an effect.  Pulsation, swirl, and wind errors could be
counted 3 times, therefore we will use the same statistical analysis as
the EV test.  To insure that the errors will not be counted 3 times. 
Note, that error in 3.4.4.1 is not defined adequately.  Will run by
SwRI’s modelers.  Discuss on June 2nd.  Therefore subtracting out bias
and precision error.  Same as EV.

New time line discussed.  There might be some room to shorten engine 2
and 3 test time.  Just depends on how it goes.  CE-CERT on road testing
pushed back to July 21.

For fuel interactive parameters study we will use:

Texas LED fuel, 10% aromatics and high cetane

CA ULSD, greater than 20% aromatic, high cetane

Federal Certification ULSD, 28 to 30% aromatics, 44 cetane.

Quibbled (CAT) over the time for final review.  Final three values
should be ready by late November.  Therefore, we must meet in early
December to discuss error surface results.

There has been little discussion on validation. This will be on the
agenda for next meeting.

Want to delay delivery of torque and BSFC values until August or
September.  We will get two weeks before SwRI’s modelers require them
unless they need more time.

Would like to time align analyzers during audit using bench tests. 
Sharp will run these regardless.

HDIUT Meeting Minutes 6-28-06

Wednesday 28th, afternoon

Schedule Update

SwRI Progress – EV testing will start second week of July.  Engine 1
testing is complete.

CE-CERT Correlation - CE-CERT correlation @ SwRI should finish July 7th.
 The correlation consists of three NTE cycles followed by 2 RMCs.  Each
facility will run the test sequence once per day for three days. 
Primary data set is NTE cycle.  Sharp feels that focus should be on the
overall cycle comparison rather then discrete NTE events.  Will also use
NTE events to verify in/out logic.  J1939 will be used for torque.  RMC
data for diagnostic and secondary comparison.  Analyze NTE cycle data
results using f-test/t-test from 1065.12.  If we pass this, then the
test facilities correlate.  This will not work for HC and CO due to the
low levels.  Agreement was reached on this test plan by the steering
committee.  CE-CERT MEL passed 1065 audit.  SwRI has the data. 
Correlation of mass flow was added as a general check (more for an FYI).

Status of the Truck and DPFs for on road validation – Truck is in
Riverside.  International has come up with the two DPFs.  These will be
installed in the next week by CAT dealer.  On road evaluation will start
in early August.  The projected length of time needed to run the on-road
test is 2 weeks.  Atul voiced concern over CE-CERT’s QA/QC.  Wants to
insure that the data we get is good.  Atul wants data reviewed before
the setup is dismantled.  The concern is CE-CERT’s monitoring of the
data.  We want to avoid special sources of error (analyzer failure,
humidity probe failure, etc.).  Review of the data could be done via the
FTP site.  This would be both Semtech and MEL data.

Data Review

Steady-state Data – The most recent data was reviewed.  Comparison of
light load points are not perfect.  Lab vs. PEMs shows a high bias on
NOx, but it is consistent as NOx concentration increases.  This is
different then before the NO2 loss fix.  The high bias verifies the NO2
penetration audit data where 2 of the analyzers reported NOx
concentrations at 101%.  Outlier in exhaust flow comparison is due to
the engine not running the same way at the mode in questions, mode 10. 
NOx SS error surface was reviewed.

CO data shows high error at low concentrations.  This was to be
expected.

HC data more or less read zero.

Cummins commented that maybe we could turn the HC and CO error surfaces
to zero since we aren’t measuring anything.  This will be discussed at
a later date.

CO2 data appeared to be missing a wet to dry corrections.  There was
about a 4% offset.  Sharp will investigate.

Revised Torque/BSFC DOE – Rerun went well.  Torque running errors were
lower.  SwRI was able to maintain load.  On BSFC, error magnitudes were
cut in half.

Flow Meter Results -

Thursday 29th

Interpolation Error Surface Discussion

SwRI presented interpolation error surfaces for both a 40 and 20 point
interpolation.  9000 pairs of speed and fuel rate data were put into a
map.  Torque value from 40 pt minus 20 pt divided by 40 point.  The
number was negative so the error was positive.  As long as you are in
the blue, the error is 1 to 1.5%.  Green regions are inside of 2% error.
 The biggest errors are along the governor line at points 4 and 7.  For
the interpolation algorithm point 7 causes a lot of trouble.  Choosing
point 4 would have reduced error.  Points 5, 7, 10, and 1 are the points
chooses in the 20 point map.  This causes error around point 7 and this
is the cause of the high error in this area.  5% of the data is beyond
the 3% error of data out to the 95th percentile.  We are centered around
the 50th percentile around zero.  5th would be 2% and the 95th would be
-1.5%.  This would be a normal distribution.

Cummins agreed to use 40 pt error model for all error look ups.  He also
agreed to look at the normal distribution from 40pt minus 20 pt (sampled
normally).  He is not sure that this is what should be used to subtract
the interpolation error from all torque and BSFC surfaces.  He doesn’t
want to double count error from the 40 pt to the 20pt.  He did state
that we could look into a similar map to subtract the error out of the
DOE.

Cummins stated that if reference T is provided, you don’t need a look
up table for base torque.  You still need to perturb value.  How do you
scale delta fuel into delta torque if you are not applying fuel rate
error to torque or BSFC.  This error surface had been zeroed out, but
needs to be added back in.  This issue is that we do not have a 40 pt
map for this.  This could be derived from the transient and normalized
to percent torque.  Then we look only at perturbations.  Use 40 pt map
to get perturbation in units of torque and BSFC.

For NTE events you will get toque and BSFC value from the speed and fuel
rate, each of those points will be integrated.  The average of all 20
events will be the 0 point and how far each individual point is off from
the average will be the delta.  This generates a dynamic error for
speed, commanded fuel, torque, and BSFC.

Dynamic error surface added back to 29.  Number 34 we will add a dynamic
torque error surface.  This will be generated using the 40 point
interpolation maps.  We would add a 40 pt to 20 pt delta torque error
surface.  This will be run on all 3 engines.  It will be sampled
normally and sampled 20 pt minus 40 pt using the 9,000 points (this is
to make the delta negative).  The current plot is in percent of reading.
 So the error needs to be as percent of peak torque.  The will be the
5th, 50th, and 95th percentiles, as similar to other error surfaces for
the y-axis.  There will be no x-axis.  This will be a single line error
surface.  For surface 35, EMA provided torque error surface, there will
be no correction.

We need x-axis as a function of torque to avoid them worst casing it
(along governor line) and smearing across the whole map.  This should
mimic the number of torque horizontal lines on the 40 pt map.

Test Plan for SwRI/CE-CERT Correlation

We need to rectify the work term for the three calculation methods.  For
calculation #1 and #3 we will use speed and J1939 broadcast torque work
for both CE-CERT and PEMS.  The SC agreed to this.  For calculation #2,
work is BFSC and flow based.  For this we could use the same BSFC value.
 CAT states you can get this by dividing broadcast torque by broadcast
fuel rate.  This will require CE-CERT to post process the data using the
calculated BSFC values.  The SC agreed to this.

Model Validation

We went over the methods as described in the test plan and the method
proposed by EPA/SwRI.  Cummins questioned if the model validation is
dependent of concentration, thus not including all of the reference NTE
events.  We could select a subset of 100 events that is within the range
of the NTE concentrations generated during model validation.  If it is
independent of concentration, the model can proceed as planned.  This is
something that we will investigate.  SwRI will provide the range of
brake specific NOx.  SwRI stated that once the decision is made on how
to run the validation, it will only be run once.

It was difficult for the group to reach a consensus on the statistical
analysis of the validation data.  Cummins wanted a data analysis with
error bars around individual points.  He did not like SwRI’s KS method
with a 5th and 95th percentile.  He did not like rank ordering the data.

In the end the confidence will be bounded by percentile deltas.  SwRI
felt that this was OK.  We need a criteria for the data fit.  For order
selection then the marginal increase in SEE is less than 10% you will go
with the previous order.

If we don’t see a trend from the Monte Carlo run, we would go with
SwRI’s KS plot.  SwRI said that in this case we might as well stay
with the KS plots and disregard Cummins’ suggestion.  We would also
insure that the reference NTE data is in the range of the data from the
CE-CERT validation.  CAT stated that the NTE data from the CAT data will
be in the 3 g NOx range.  Once the NTE data is chosen, the SC will
review and approve.  For flyer selection, 90% of the data must be within
the 5th and 95th percentile.  If 90% of the deltas are within 90%, the
model is valid.  Note that if 10% of the data is out all on the high or
low side, it will be investigated as this can bias the model high or
low.  The validation will be done on all three methods.  The group
unanimously agreed to the validation scenario.

On-Road Evaluation Details

CE-CERT may run for more than 6 to 8 hours.  This will maximize NTE
collection.  At the time of FID bottle replacement, CE-CERT will zero
and span the Semtech.  Sensors will provide a new software that will add
keys to the data to allow a post drift corrections using the
certification post processing software.  EPA will talk to Atul to see if
we can get the software upgrade for free for use during the program.

There is an issue with how to correct for multiple zeros, but Cummins
stated this is described in 1065.

HDIUT Meeting Minutes 7-27-06

Schedule Update

A schedule update was given.  Engine 2 installation will begin this
week.  Overall the timeline has slipped by another two weeks with a
finish time in the first week of January 2007 (January 1st). 
Temperature chamber testing has not started yet.  This is due to lack of
power supplies and the frying of the power input boards on two of the
units.  These will be to SwRI by Friday.  Once these arrive the testing
will begin.  Temp chamber test will start the second week of August. 
CE-CERT has had delays with getting the DPFs installed.  They believe
they can get on the road by August 21st.  There was a discussion about
the use of the Horiba.  If a Horiba fails, we will not stop testing to
fix or take one unit from another test to replace.  

Test Results Data Review

a. Transient Results

There is an issue with some of the transient data.  The emissions data,
NOx, was not repeatable in all cases even though speed and torque were
repeatable and controlled (i.e. the engine ran differently).  This was
also evident for CO2.  These are probably due to AECDs.  SwRI will go
over the data with DDC, but this doesn’t matter for the data toss out
election.  A proposal was made to throw the data out where the data was
not repeatable.  This was agreed upon.  SwRI is not sure how much data
will need to be thrown out (looks like 3 or 4%).  Also a proposal was
made to lock engines 2 and 3 in a mode of operation to prevent the
engine from switching modes during the transients.  CAT and
International will look into this, but this is not likely.  The
transient MADs were gone over.  The effect of the outliers was evident.

b. New Warm-up Test Results

New warm up data was reviewed.  Two different torque points were run,
WOT and 1260 lb-ft.  It was proposed that both of these results be
pooled and two data points be taken for each of the other engines (using
similar torque levels).  The largest difference between Lab Torque and
Bias Corrected Interpolated Torque after entering the NTE zone will be
used.

BSFC was also looked at.  The bias corrected interpolated BSFC and Lab
BSFC converge.  The number is small.  As torque error is high, BSFC goes
down.

Audit Data

a. Updated Sensors Audits

The Semtech and Horiba audit data was reviewed.  There were no
outstanding issues with the data.  Cummins pointed out that some of the
Horiba data looks “synthetic”.  SwRI will investigate further. 
Sensors went over time alignment as there was still issue as to how the
data will be time aligned.  He went over a new Semtech application note
that provides a QA/QC check on how the data is time aligned.  The
question is how the precision of the transient is affected by the data
time alignment selection.  So they would like to modify the test plan to
take into account that error.

There was an in depth discussion of the flow meter audits.  The
discussion was lengthy.  Sensors was present and went over the issues. 
SwRI flow bench does not match up with the Sensor’s bench.  Different
configurations and LFEs were tested at SwRI.  Error was reduced, but
4” flow meters did not pass slope and intercept in some cases.  Error
was negative, so it would bias the emissions concentrations low.  A
proposal was made to send flow meters or SwRI and Sensor’s LFEs to CZ
for calibration.  Either SwRI is off or Sensor’s is off.  SwRI was
calibrated by CZ and Sensor’s was calibrated by Miriam, an LFE
manufacturer in Cleveland.  There was agreement that the 5”
calibrations used on engine 1 stand as it.  Overall, it was agreed that
we would calibrate the Sensor’s flow meters using the SwRI stand and
then audit on the SwRI stand.  This was a win/win for all.  Sensors
would in turn send their LFE to CZ for calibration.  It was speculated
that Sensor’s LFE is off based on SwRI’s test cell LFE’s
correlation to their cal stand and carbon balance.  Note the errors for
the Sensor’s flow meter was consistent between SwRI’s cal stand LFE
and their test cell intake air LFE.

b. Horiba Audits

The Horiba audit was discussed.  Cummins pointed out that some of the
Horiba data looks “synthetic”.  SwRI will investigate further.

Interpolation Error Surface

For the error surface, we are centered on the 50th percentile around
zero.  5th would be 2% and the 95th would be -1.5%.  This would be a
normal distribution.  By taking just the 5th and 95th percentile, we are
bounded by -2% and 1.5%.  This will eliminate the green zones on the map
SwRI presented, thus eliminating the higher error areas near the
governor line.  SwRI will look to see if it truly is level dependant and
if it is we will use the 5th and 95th as a function of level.  We have
reached an agreement on this, but is tentative assuming that it is not
dependant on level.  This will be presented during the next conference
call for final resolution.

Additional Stats Questions

Approve method to generate interpolation error surface for torque and
BSFC (i.e. approx 9,000 differences(%) will be computed between the
40-pt and 20-pt mappings for each of the 3 engines.  This will create
3x9,000=27,000 differences (%).  The 5th, 50th and 95th percentiles
will be computed from the 27,000 differences and sampled in the MC
simulation using a normal distribution.) 

- This was approved.

Include Error surfaces #29 (Delta torque dynamic), #34 (delta torque
interpolation), #36 (delta BSFC dynamic) and #41 (delta BSFC
interpolation) back in the MC simulation model.  There are now 49 error
surfaces. 

- We confirmed the addition of error surfaces 29, 34, 36, and 41 back
into MC simulation model.  There are now 49 error surfaces.

Finalize list of error surfaces that will be used to validate the model.
 These error surfaces are included in the Excel spreadsheet provided in
the last meeting.  What error surfaces will be dropped in the MC
simulation to validate the model to the CE-CERT and PEMS on-road
differences? 

- List of final error surfaces to validate the model was sent out by
Cummins.  We need to confirm that SwRI got it and EPA need to review it
and get back to group on it.  This will be discussed at the next
conference call.  Cummins will forward to me.

What criteria will be used to validate the model?  Will we have to
validate all 3 emissions x 3 calculation methods?  What happens if only
some of the emissions or calculation methods validate?  Will we only
have to validate one emission (EPA.e.NOx) or one calculation method
(i.e., method #1)?  This was not discussed at the last steering
committee meeting. 

- We confirmed that we would use the model validation scenario agreed
upon at the last meeting.  It is not entirely clear of all of the
regulated pollutants need to be valid in order for the whole model to be
valid.  Most important is NOx.  Most are in agreement, but this does not
need to be resolved now.  It will be looked at on a later date.

Convergence criteria – in running the MC simulation for each reference
NTE we have proposed to run as many samples as needed to compute a 90%
confidence interval at the 95th percentile differences so that the width
of the confidence interval is less than a percentage of the emissions
threshold.  For example, the NOx threshold is 2.0 g/hp-hr.  If we take
1% of the NOx threshold then we will run the simulation until the width
of the 90% confidence interval at the 95th percentile is less than 0.02
g/hp-hr.  We would like the steering committee to agree on the width of
the confidence interval as a percent of the threshold – is it 1%, 2%
or some other value?

We will start at 1% if there is enough processing power in the computer.
 The thinking is that the tighter it is, the better it is for both
parties.  If 1% is at some inordinately long computing times, we could
go to a higher percent.

HDIUT Meeting Minutes 9-7-06

Schedule Update

EMA asked for a catalog of all of the PEMS repairs that needed to be
made during the HDUIT program.  Target finish date is second week of
January 2007.  EMA asked about interim reports that are to be due after
each chunk of testing.  Sharp stated that we would prefer to deliver all
reports in November and he can deliver the reports until the testing is
done.  EMA stated that he is concerned about the review and comment
period.  Sharp stated that reports would be done in mid November and
give everyone a good month or two to review a chunk of what will be in
the final report before the final report due date.  EMA is fishing for
more delays to get the 1 year delay in the enforceable program.  A
question was asked about the PM program by Cummins.  EMA stated that we
are behind and the PM pilot program start date will roll back at least a
year.

Engine 2 40-Point Steady State Data

One PEMS read NOx high, one low, and one compared very well with the
Horiba raw and dilute.  The exhaust flow meters for 2 of the PEMS start
to diverge high from the lab reference at flows greater than 550 scfm. 
The % error gets larger as flow increases with a 6% error for 2 of the
PEMS at the highest flow rate tested (700 scfm).  As a result the mass
flow rate of NOx was higher for 2 of the PEMS, especially at the high
flow end.  There was some discussion on the NMHC data.  Some of the lab
data showed negative values.  The issue is what to do with that data. 
Should the lab data be collapsed to zero when negative.  There was a
proposal to treat it as a single value error surface and use the value
at all concentrations.  This seems like a good idea, but we will wait
and see.

Time Alignment

The time alignment issue was reviewed.  The 5th, 50th, and 95th
percentiles were determined for all three calculation methods.  They
varied by calculation method (data in file PEMS Time Alignment Shift
V3).  It was brought up that EMA could realign the data before
submitting it to EPA if they were close to failing an enforcement test,
thus allowing them to pass.  So they could manipulate that data outside
of the measurement allowance.  Cummins believes that the course taken
over-predicts the 5th and 95th percentiles.  CAT is concerned that for
method 1, that if it is included, it could shrink the measurement
allowance.  Thus EMA is interested in getting rid of the time alignment
error surface if it pushes their measurement allowance down.  At issue
is that if we go with this allowance, Cummins wants the most accurate
calculation.  Sharp doesn’t want to use this due to the amount of
resources that it will pull and the added complexity.

EPA argued to add 8 more observations to the data set at 0.5 and -0.5
second shifts.  Then there will be a total of 17 points.  From there,
weighting factors will need to be developed.  To summarize, we will
weight the inner 9 points equally and then have reduced weighting for
the other points.  Cummins put up a weighting matrix up on the board
with 0.125 and 0.25 weighting factors.  This was agreed upon.

Overall, agreement was reached on this topic.

Engine 1 Transient Data Outlier Analysis

Sharp went over the NOx outlier determination data.  This is the NOx
data that was thrown out based on the engine not performing the same at
repeat engine operating modes.  These outliers were determined using a
Grubbs-type outlier test.  The group was OK with the results.  Outliers
will be thrown out.

Environmental Testing Data

Baseline

A review of the original baseline data was performed.  The CO2 zero data
that is not stable, due to not all of the CO2 being flushed out of the
cell, will be thrown out.  Cummins was OK on PEMS 5 Audit and Span
baseline errors.  Also there were no issues with CO, CO2, and THC
errors.  The Horiba baseline data was reviewed.  It was noted that there
is a lot of drift on the analyzers as well as their flow meter.

Baseline Issues and Diagnostics

The NO2 loss and NO formation was discussed.  The group understood the
issues and solutions and there are no outstanding issues with this
testing.

Temperature Testing

This data was reviewed.  There was some drift associated with
temperature.  Overall the data was well received.  There was an issue
with the Semtech truncating the negative flow meter values to zero,
making it difficult for statistical analysis in the error surface. 
Sensors will look into backing out the truncated values.

Environmental Test Data Processing

The NOx pressure calculation was reviewed.  There was a long discussion
on how the drift limit of 4% was derived.  CAT stated that he was not
happy with limit.  His argument was that this caps the environmental
effects at 4%.  There was one change to the error surface.  The final
values will not be averaged and the dice will be rolled across the
results for all 3 PEMS.

CE-CERT Plan Discussion

Kent went over the validation test routes.  CAT wanted CE-CERT to run
the route as a test to determine the number of NTE events.  CE-CERT will
zero only on ambient air.  They initially wanted to use bottled zero air
at start.

We will give CE-CERT an extra week before the start of the validation
testing.  This should not adversely affect the schedule.  CE-CERT will
start testing on 9/18/06.  Testing and data review should be complete by
the end of October.  They will be testing for the measurement allowance
on the 20th.  CAT wants CE-CERT MEL audit to occur before the beginning
of the validation testing.  This will occur on the 18th and take the
whole week.  The audit will occur on all routes.  The PEMs work will
start late the week of the 25th with the San Diego route.  Then Mammoth
will start the week of the 2nd.  The meeting in October will be moved to
October 5th and 6th.

OEM Error Surfaces

EPA described the error surface data submitted by the manufacturers. 
Most seemed OK with it.  Manufacturers will get back with any issues by
10/3/06.  This will also be discussed during the conference call on
9/19/06.

PEMS Steering Committee Meeting Minutes 10-2-06

Schedule Update

Timeline has been pushed back to a completion date of the end of January
(last week).  Latest PEMS hardware issues have set us back another week.
 There was a drain pump failure on one of pressure test units.  There
was a functional failure of NDUV bench on PEMS 4.  It was replaced and
exhibited same failure two weeks later.  This was subsequently replaced.
 Then there was a failure on the NO2 humidity correction sensor.  The
issue turned out to be wetting of the humidity sensor due to a leak in
the material that seals the sensor to the Teflon mounting block.  This
leak facilitated flow to the sensor and allowed water to condense and
contact the sensor.

EMA wanted to know if we still thought that the PEMS were commercially
available.  EPA hit back hard and told him that per the test plan the
PEMS did not have to be 1065 compliance.  Also, EPA informed him that
the manufacturer was allowed to make changes per the test plan to become
1065 compliant.  He keyed on the 2 NDUV bench failures on PEMS 4.  EPA
also told him that the PEMS from EPA were dated and we did not know the
run history on them.  He also wanted to know if the changes were
commercially available.  EPA told him they were.  EMA then reiterated a
request for a list of failures on the PEMS.  SwRI is working on this. 
Cummins stated that we recognize that the NO2 penetration fix was a
conscious decision that was made by the committee.  Note that EMA may
take this list once it is put together and try to make a claim that the
Semtech is not commercially available.  CAT made a statement that they
cannot get through a days testing with the Semtech without the unit
failing.  On a side note, Louie told me last week at CE-CERT that CAT
was running 5 Semtechs 24/7, and weren’t having the failures that CAT
seems to claim.  EPA decided not to ask CAT what the test durations were
for the testing that he was running.  Sharp will try to get the reports
out by the end of November, one for engine testing and one for
environmental testing.  The failure log will be out before the reports.

EMA would like to get a look at the model.  SwRI thought that they might
have something to distribute at the next steering committee meeting
(PowerPoint).  This will be a detailed discussion of the model with
error surfaces.  CAT want the Semtech issue log done quickly.  Why? 
What does it contribute to the test program?  EPA think this is EMA
wanting this info to try and claim non-commercial viability of the
Semtech.

Engine 2 will be done mid-October and engine 3 will be done
mid-November.  Sharp will add report dates to timeline.

Sharp stated that any other delays will most likely push us past the end
of January and thus the enforceable program will slip one year.

EPA gave a CE-CERT update.  Just went over the routes and preliminary
results of the MEL audit.  The only analyzer issue is that the FID zero
drifts down as elevation goes up and returns back to zero as elevation
decreases.  CE-CERT will develop a correction for this based on the
data.  NTE update was 60 on San Diego, 100 going out to Mammoth, and 78
coming back from Mammoth.  There are roughly 238 total NTE events.

Engine 2 Steady-State Data

SwRI went over the engine 2 steady state data.  Error of lab raw vs. lab
dilute was up to 2%.  PEMS vs. lab was close to 0% error at the 50th
percentile.  Lab flow rate vs. PEMS showed that as flow increased, error
increased.  Atul thinks this is due to backpressure.  NOx mass flow PEMS
vs. lab 50th percentile is biased high at the high end.

There was discussion about a possible change to 1065 to change the NO2
intercept requirement.  EPA needs to address this as the Semtech will
not currently pass.

Lab reference is indicating that all of the HC is methane.  It was
requested that we pool all data with no x-axis, all observations on all
engines and sample normally for NMHC.  We are in agreement that lab
reference is zero.  We will do this for the first two engines and wait
on results for the third engine.  What is at issue is that we might want
to throw out the data on the low PGM catalysts that are not production
intent, those that had zero NMHC.  This will be discussed at a later
date, pending the results of engine 3.  The data that would be thrown
out is engine 1 and 3.

Exhaust flow rate data showed high error at highest flow rate, 61 scfm
discrepancy (7.7% error).

Group agreed to run the EFM on 10 SS points with CVS disconnected to
check for CVS effect.

Environmental Testing Data	

-Baseline

Env_BL_Results.xls.  This shows the test procedure.  PEMS 2, 3, and 5
were used for the baseline results.  Baseline data was reviewed.  SwRI
noted that if you have a span off during your baseline, you can create
error simply based on the baseline test.  This makes it difficult to
determine the environmental errors if the baseline is not stable.  HC
span error is as high as 10 ppm and this is about one third of the HC
standard.

-Temperature

Env_Temp_Results.xls.  There is large variation on PEMS 2 span data
versus temperature.  This was not evident on audit, however.  There is a
question as to why.  It seems that most of the offsets are due to the
auto zero.  Overall this won’t have any effect on the measurement
allowance.

-Pressure

Env_Pressure_Results.xls.  There didn’t appear to be much of a
response to pressure.  There was a large span response to CO, up to 140
ppm.  This was consistent from PEMS 2 to 3.  This was at equivalent high
elevation (5000 ft).  The flow meter showed 0.4% error minimal).  There
is an ambient effect on the delta P pressure sensor.  There is concern
that this is scalable to % of full scale.  There are a lot of issues
with this.  Since there is more than one delta P sensor and only the low
end was checked, there is concern that this is the best case error. 
This will be discussed during a follow-up conference call.  CAT has
stated that if the sensor range is 0 to 100 cfm, the average error for
that range is 5% based on the data.  Then there is additional data for
the sensors covering flows above 100 cfm which could possibly make the
error higher.  Atul will get back to the SC on this.  The error will be
based on % of dP error corresponding to a flow error.

Error Surfaces

-Environmental

Went over NOx Pressure surface first.  Most of error surface deviations
are between -5 and +5 ppm for zero.  The audit data showed a positive
bias on the data.  For PEMS 3 there is no NOx pressure effect.  The
shifting baselines make it difficult to determine what environmental
error and what is not.

Really there are X options:

1. Go through the data and manually see what is real and what isn’t.

2. Run more baselines.

3. Ignore the baseline bias correction all together.

4. Use only baselines with clear historical bias.

5. Continue with the planned method.

Due to the small sample size we will go with option 1.  SwRI will go
through the data and suggest what is real bias and what isn’t.  We
will then follow-up on this at a later meeting.

There was a slight change to the vibration and EMI/RFI tests.  SwRI will
run a one hour baseline before each of the tests which will act as a
reference to determine the effects of the environmental tests.  Thus
each test will run 9 hours instead of 8.

-Transient / Dynamic (Engine 1)

We went over the error surfaces that were generated from engine 1.  Flow
meter error surface is noisy.  Looked at Dynamic Torque error surface
(speed weighted average torque).  Error surfaces look OK.  Nothing seems
out of the ordinary.  Two points were outliers on NTE events that were
large transients.  The dyno wasn’t able to hold the speed trace.  The
outliers were tossed.  Dynamic fuel rate ES was discussed.

Time Alignment

The errors came down and were more centered around zero for the weighted
results.  The group agreed to use the technique described in PEMS Time
Alignment Shift V4 Weighted Results V2.xls.

CAT was concerned that this surface would be weighted heavier than some
of the other error surfaces.  However, since it is a % of Pt error, the
magnitude will be based on the magnitude of the error itself.  However,
SwRI’s modelers are not sure how it will play out and will monitor it
when they run the model.  CAT is on record that he does not want this
error included.  Presumably because it has a negative effect.  Cummins
stated that it is multiplicative, not additive, so it will be treated
like all of the error surfaces.  An issue arose on whether or not we
should sample normal or random.  Normal will give a higher measurement
allowance versus random in some of the calculation methods.  Based on
the environmental tests and since the data is already weighted, we will
sample random (originally it was supposed to be normally).  Run this by
EPA.

We will sample off of weighted raw data set.  We will sample the entire
data set randomly in a method that is analogous with the Temperature and
Pressure environmental tests.  If this is a big process problem, we will
use normal sampling and sample off of the weighted data set.

The group agreed on this strategy.

NTE Reference Events

Cummins showed concern that there were differences in the 3 method
calculations.  He feels that there should be no favoritism for a
specific method for initial comparison.  There are carbon balance errors
in the data that was sent by the manufacturers.  So we need to make an
adjustment to the data.  Cummins suggests that we adjust the CO2 (derive
it) to better align the methods and derive BSFC.  BSFC is already
derived.  Method 1 is out of line, so we need to adjust it to get it
closer to the other two.

Laboratory Replay Validation

An overview of the replay testing was discussed.  J1939 torque and speed
from CE-CERT will be taken and replayed in the SwRI test cell to try and
determine torque and speed errors.  They might also use commanded
throttle instead of torque.  This would mimic what the driver was doing.
 They decided to use commanded throttle.

Revisit Shock and Vibration Testing Parameters

We need a final decision on which spectra to use for the test.  We will
test the vertical axis last as that showed the worst damage in
Sensor’s test.  We will average the 55 mph and 70 mph spectra to
effectively create a 62.5 mph spectra.  This was agreed upon by the
steering committee.

Future Meetings

November Whole day on the 2nd 8am till 6 pm, 3rd till 2pm.  Note, fly
out on 1st.

December 12th 8 am to 6 pm and 13th till 2 pm.  Note, fly out on 11th.

January 8th afternoon from 1 pm to 6 pm, 9th all day, 10th until 2pm.

PEMS Steering Committee Meeting Minutes 11-2-06

November 2nd

Monte Carlo Model and Model Test Run Results

SwRI gave overview of MC simulation (see presentation).  She then gave
an overview of the model status.  Only 7 of 49 error surfaces are final.
 8 contain engine 1 data only.  There is no NMHC data.  She has not
thrown out data that failed the drift correction.  SwRI is running
extreme speed professional version.  This has cut the model run time
significantly.  CAT asked if any conclusions could be drawn from the
results.  SwRI reminded him that we are missing data and error surfaces.
 Distributions of ic samples were given.  The distributions looked good.
 CO uniform distribution example was given.  Ideal NOx was not the same
for all three calculation methods.  Method 1 was the highest.  There is
still feeling in the group that the values need to be closer.  Last
time, we were comfortable with a 3% error.  This appears to be too high.
 EMA stated that about 0.1% is what is required.  SwRI will attempt to
readjust manufacturer supplied NTE data to better match the calculation
methods to within 0.1%.  There was unanimous agreement from the group on
this.  SwRI went over data removal due to drift.  The Max and Min are
ballpark to what EPA was thinking.

An overview of convergence was given.  SwRI wants to use order
statistics for convergence.  There was an issue with Crystal Ball’s
boot strapping technique.  SwRI also presented MC simulation with
different numbers of samples.  There wasn’t that much improvement to
run 25,000 vs. 10,000.  SwRI asked group to agree on her recommendations
on slide 24.  The group unanimously agreed.

Check on NOx threshold value from EPA for all three emissions.  The test
plan states 2.0 g for NOx.  We need to know the exact number.  EPA
stated that we may have calculated that at 1.9 and rounded up to 2.0. 
EPA stated that most manufacturers will be averaging, banking, and
trading so the average will be about 1.3.  Thus there will be a range of
FELs or emission levels.  Cummins stated that we should just leave it at
2.0.  EPA has spreadsheet that he used to calculate the value.  CAT
finally agreed to leave it at 2.0.  EPA will follow up with some info
for the group on where the 2.0 came from.  This will appear in the final
report.

SwRI reviewed the process to determine the measurement allowance.  The
current r2 value is 0.88 and does not pass the criteria of 0.9.  If this
stays the case when the model is done, we may have to change the
criteria to less than 0.9 if the curve fit looks good.

SwRI reviewed model validation.  There was discussion of what to do with
the allowance in the 2010 time frame and it extrapolation of the curve
fit off of the chart on slide 28.  This will be discussed next year.

Cummins stated that if there is bias in the calculation methods, it
won’t affect the validation event even though the model simulation
reference NTE events were corrected for the bias.  SwRI’s question is
when will the model be considered valid?  What if only one method
validated or CO and HC validate, but NOx doesn’t for a specific
method.  No one has a cut and dry method.  All believe that CO is a
non-issue and the real concern is with NOx and NMHC.  Cummins states
that we should just invalidate an entire calculation method if one or
more of the pollutant calculations does not validate.  This will
probably remain unresolved until the validation takes place.  SwRI just
wanted to make sure that we didn’t need her to save any additional
data during the model run.  We told her we didn’t need anything else
saved.

SwRI went through time alignment for 9 NTEs.  The time alignment
adjustment delta is negative in 8 out of 9 times.  CAT didn’t like the
results because there is a decrease.  SwRI explained that there is a
50/50 chance that a negative will be picked, regardless of the magnitude
of the value.  CAT’s concern is that the error surface has a lot of
power.  SwRI confirmed this.  However, the group agreed that no change
was warranted and we stayed with the original plan for the time
alignment.  

There was discussion on the remaining issues that need to be resolve for
the model.  It was asked it the model could be run with just engine one
data to be debugged.  This is time consuming and there may not be time
for it.  Also it will cost more money.  It is unclear if there is any
advantage to running it this way.

OEM Error Surfaces

Update on revisit of OEM error surfaces on AECDs.  Additional data from
two more manufacturers.  Focus on Cummins submission which was very
comprehensive.  Torque error would be 6.5% and the BSFC error would be
5.9% at 5th and 95th percentiles assuming a 50th percentile error of 0. 
EPA proposed that SC adopt the error values that are the conclusion of
the Cummins analysis.  He would then follow up with an amended memo to
the SC on the agreed upon values.  The SC unanimously agreed to the
values.

SwRI reiterated that this will be sampled normally at every roll of the
dice.  This is a % of Pt adjustment that is performed line-by-line.

Review of Previous Conference Call Minutes/Decisions

SwRI reviewed the conference call minutes from last week.  There was no
link between PEMs bias and NO2 penetration audit results.

Schedule Update

Will have to rerun SS engine 2 tests and this should finish up early
next week.  Engine 3 should end in the first week of December.  There
was a discussion of the replay test.  SwRI will run the Semtech during
the replay.  SwRI stated that they will not run the Horiba in
environmental tests or on engine 3 since they have been without it for
the last 2.5 months and just got it back this week.

Engine 2 Warm-up Test

We looked at the CAT Warm WOT and 700 lb-ft files.  We will take the
largest difference after we have crossed 250C DPF outlet temperature. 
If it’s negative it will be the 5th and the inverse will be the 95th. 
If it’s positive it will be the 95th and the inverse will be the 5th. 
The trigger is entry into the NTE zone so that for Engines 1 and 3 it
the trigger is when DPF Outlet temp, ECT, and IMT are all in the zone
(effectively IMT since that was slowest).  For Engine 2 as a non-EGR
engine, the ECM and IMT are not used, so the only relevant trigger is
DPF Outlet Temperature.  There was unanimous agreement on this.

NMHC Error Surfaces

Lab reference mean for engine 2 is -0.5 ppm.  There was unanimous
agreement to pool the data with no level dependence.  If the Semtech
does not report non-zero error, which it doesn’t, then the 5th and
50th percentiles will be zero.  The 95th will be some sort of positive
value.  The data for engine 2 will be the only data used, thus engine 1
and 3 data will be thrown out.  There was agreed to unanimously.

Engine 2 Steady-State Data Issues

SwRI went over the MAD from engine 2 and showed that many of the
transient points are at zero.  He stated that it would be really
difficult to correct and thus proposed that the SS points be rerun.  CAT
wanted to know the effect on the agenda (two days).

Engine 2 Transient Data

SwRI went over the transient repeats BS NOx repeatability.  The
distribution is symmetrical.  COV is about 1.5%.  SwRI also went over
all of the other transient repeat data.  Discussion was
non-controversial.

November 3rd

Environmental Testing

Ambient HC

We went through the ambient HC environmental data.  There was no
response to hexane, but there was response to methane.  There was up to
60 ppm response through burner air port for 8 ppm methane.  Tests 3, 5,
and 7 were voided because the CH4 concentrations were too high during
span and the Semtech would not allow the FID to span.  At issue is what
CH4 span value do we used for the measurement allowance.  Cummins wants
to use the data to come up with a sensitivity factor for methane and
hexane and then determine an ambient level from the CE-CERT data.  This
is because the current surface would be skewed due to the non-effect of
hexane.  Sharp pointed out that the skew would be realistic.  Another
option was to force the 50th to zero and then get the 5th and 95th based
on the CE-CERT data.  We will use tests 2, 4, and 8 to establish a 5th,
50th, and 95th percentile as a response factor sampled normally.  How we
do this could effect the drift limit in the model.  If we get the
distribution wrong, a huge amount of the data could be thrown out on the
model drift correction.  A motion was made to use a single error line
with a 5th, 50th (at 0), and 95th percentile with the 5th and 95th to be
determined after we received the CE-CERT data.  SwRI will use temporary
values to get the error into the model; using 5th and 95th from tests 2,
4, and 8 (the data will be pooled).  The agreement was approved
unanimously and we agreed to not go back and change the model to sample
randomly.  We will await CE-CERT results to determine the 5th and 95th.

EMI / RFI

SwRI went over the electrostatic discharge results.  80 different points
were tested.  No effect from electrostatic discharge on THC, CO, CO2,
NO, NO2, and NOx for zero, audit, and span.

Bulk current injection results were gone over.  There were functional
failures on the FID temp and pressure readings.  The intensity was
dialed down to prevent FID shutdown.  It doesn’t appear that the
faults affected the data quality.  No effect on zero.  On audit and
span, CO and CO2 showed slight effect.  There was a request to look at
the RH and Temp sensor data just to make sure that there is no effect on
those as they are used to correct analyzer signals.  SwRI will evaluate
any points that are out of the range by looking at the continuous data
file.  If they verify, they will then calculated the 5th, 50th, and 95th
for both the baseline and non-baseline data.  From there they will
subtract the baseline from the non and see if there is anything left for
an error surface.  It is important to note that the disturbances took
place during the Ethernet cable testing.  The group was asked what we
should do with the BCI Ethernet cable.  The group unanimously voted to
discard this data.  How do we resolve any error from the baseline?  We
have two things laid into the data.  We are trying to resolve if the
data points are shifted from their neighbors due to the environmental
disturbances.  For baseline shift we will take the average of our
baseline values that bracket each individual disturbance band, take the
average and correct the disturbance data using the average of the
bracketed baselines.  Then to deal with short term shifts in each
baseline segment, we will do a MAD and then pool them together to have
an assessment that shows the effect of the short term variability during
the tests (not long term drift).  Then we will take the MAD and apply it
to each segment where we do have a disturbance.  This will correct away
bias baseline shifts and short term variability.  So we should be left
with a bunch of zeros and some disturbances.  From there we will verify
the disturbances in the continuous data and in cases where the
disturbances are there, we will determine the 5th, 50th, and 95th
percentiles and rank order them.  This will evaluate both spread and
bias.  Cummins stated that while this will work, there will always be
some bias that is unaccounted for and thus there will still be a small
error surface.  It is unclear if this residual is actual environmental
error or not.  It was proposed that if the 5th and 95th difference is
very small, then we disregard the error surface.

Radiated immunity was discussed.  There were FID shutdown problems up to
25 V/m.  The test was adjusted and run at 10 V/m from 100 to 300 MHz and
then run at 25 V/m at frequencies greater than 300 MHz (300 to 1000
MHz).  The data did show effect on CO and CO2 during zero, audit, and
span.  There were one or two points on NO2 audit, maybe on span (one or
two points).  The EFM showed some susceptibility.

In the end it was proposed that the overall contribution to the error
would be zero.  This is because the lowest and top 50 points will be
thrown away, thus the top and bottom 5% of the data.  Since the
observable deviations during these tests amount to less than 50
observations when pooled, the deviations would be thrown out, thus
making the error zero. The conductive transient test still needs to be
completed.  If there is no error from this test, it tells us that there
is no error surface here.  It was proposed that SwRI will examine the
data.  If less than 5% are deviations, then the error surface would be
zero or near zero and not a major contributor to the measurement
allowance.  If it is greater than 5% for any one pollutant, then we will
use the pooling with MADs technique that was described before (above in
red) for that specific pollutant.  If the pooled MAD of the baseline is
greater than the MAD of the segment, the 50th percentile will be zero. 
This was put to a vote and there was unanimous agreement to this.

Flow Meter Environmental (Pressure)

Atul gave an overview of the function of the EFM using a flow schematic.
 Sensor precision is 0.1% of full scale on all P transducers.  For most
of the barometric test the testing was performed on the 1” of water
sensor.  CAT revoiced his concern that the pressure test only addressed
the 1” of water sensor and not the rest.  Sharp stated that maybe the
error us a function of the ambient pressure sensor (gage).  Thus there
is no correlation with delta P.  Cummins stated that the ambient
pressure sensor should not drive the flow meter zero value.  Atul then
reiterated that the dP sensor is not at zero at zero flow.  In theory it
should, however, in reality it is an extremely low number.  Sharp argued
that a higher range transducer may never react to the error the lower
range sensor does.  Atul stated that for a 5” flow meter, for a flow
of 8 cfm, dP is 0.001 kPa or 0.5% on the dP.  Sharp proposed to send the
xml file from the test with the dPs to Sensors.  Sensors would then pull
the dPs out.  If the dP reading by sensor increases by sensor range,
then the error is scaleable.  If it is not and just the 1” shows a dP,
then the error is just linked to the 1” sensor.  Sharp proposed that
we sample the set randomly and scale the error by % of FS of the flow
meter.  We will subtract the median baseline value and subtract from the
environmental data.  There was unanimous agreement from the group on
this.  EPA pointed out that if it is shown that the upper flow
transducers show no dP error, then the error should only apply to the
1” sensor.

For the other environmental EFM data, we will turn the crank on it with
the baseline correction and MAD.  We will pull all of the EMI/RFI data
together.  It will be sampled normally and get a 5th, 50th, and 95th. 
We will enter the error as percent of max flow and hold on to any other
changes until we get the results of the dP analysis by Sensors.

Engine 2 Independent Parameters Data

SwRI went over the results.  Fuel type changed interpolated torque due
to change in fuel quantity needed to produce same torque.  On the
humidity test there were some results where the error on the baseline
fuel error wasn’t 0.  SwRI wants to adjust the results to take out any
base fuel bias.  Interpolation correction was agreed to.

Updated Error Surfaces - Engine 2 Data Added

Did not get to.

Reporting Date Revisions

The CE-CERT report will come out in mid December.  Data will be to SwRI
by November 10th.

Engine 3 testing will not be done until the end of November.  SwRI will
have an engine 1 and 2 report by the end of November.  The engine 3 data
will be in the final report.

The environmental parameters report will be done in mid December.  SwRI
will try to get it sooner, but can’t commit to it.

PEMS Steering Committee Meeting 12-12-06

PM In-Use Testing Update

MAL Program Schedule Update

Timeline is still on track to finish before February 1st.  The engine 3
data is complete and was sent to SwRI on time.  Engine 3 removal will
occur later this week.  For replay they will use DPFs that were
installed on the truck.  SwRI is working with CAT and CAT to set up the
replay methodology.  Lab validation should be completed by early to mid
January.  Model running should start late next week and conclude mid
January.  Reporting is behind.  A draft report should be available for
engine 1 by the end of this week, with it being complete, with all
engines, by the end of next week.  The environmental report will follow.
 The January meeting topics are uncertain at this time.  It depends on
if there are any issues with the model.  If there are no issues, we can
review and discuss the draft reports.  During replay a PEMS and OBS will
be run along with the test cell instrumentation.

Engine 3 Data

Steady-State – The data reflects outlier removal to get the MAD for
steady-state.  Point 35 shows scatter on all PEMS and the lab equipment.
 This is an engine repeatability issue.  There was a proposal to discard
point 35 from the MAD calculation.  PEMS 1 is about 5% low at the 50th
percentile, PEMS 4 is right on the line, and PEMS 6 is about 10% low
(this is the same unit that had negative bias on the 40 point repeats). 
SwRI overflowed span gas through the sample port of PEMS 6 and it showed
a loss of NO and NO2.  Ran NO2 penetration check and span overflow check
a few days after and the problem went away.  During the modes where
there is loss, the RH sensor is running in the 90% area.  Flow meter
data was also reviewed.  The flow meter results are up to 10% high
versus the lab as some modes.  This error is a span error and was
present for all three 3” flow meters.  We will remove point 35 from
the SS MAD for NOx.  It was agreed that the NOx and flow biases are what
they are and the data will be used as it stands.  CO was reviewed as
well as CO2 and NMHC.

Transient – The 30 transient repeat results was reviewed.  There was
no real negative bias on NOx at the 2 high NOx transient modes. 
However, PEMS 6 does show a little low bias.  The 9 to 10% bias is still
present for the flow meter.

Interacting Parameters (Warm-up) – Reviewed the Warm-up tests for
engine 3.  IMT was the trigger point for both the torque and BSFC tests.
 For WOT torque test, error is small, about 5%.  For the WOT BSFC test
the error is 3.2%.  For fixed torque, IMT was trigger point and torque
error is small, about 0.4%.  For fixed torque, BSFC error is small,
about 2.4%.  These results were OK with the steering committee.

Interacting Parameters (DOE) – Run only on engines 1 and 3.  The
engine 3 results were reviewed.  The parameter settings for each of the
10 runs were reviewed (file INT DOE Matrix Torque and BSFC.xls).  Set
points are intake air restriction, exhaust restriction, CVS pressure,
boots after temperature, inlet air temperature, and spray chamber
temperature (humidity).  Baseline corrected delta torque show errors
ranging from -1 to 4% at 5th and 95th percentiles.  The group was fine
with these results.  The BSFC was then reviewed.  The group was fine
with these results.

Review of Previous Call Minutes

Last week’s call was reviewed; see 11-28-06 call minutes.  There were
no issues with what was agreed to.  Cummins was concerned with the
exhaust flow clipping, but we worked it out.

Ambient HC Error Surface

6.8 ppm of FID reading per ppm of methane shift via the analysis by
SwRI.  Need to decide on range.  CAT had a spread of 1.54 (80% of THC),
Cummins 1.95 ppm (80% of THC), SwRI 0.7 ppm, EPA 0.4 ppm, and CE-CERT
0.9 ppm.  Overall average of the 5 observations is 1.1 ppm.  We
unanimously agreed to 1.1 ppm which is 7.5 ppm at the 95th percentile. 
EPA will change the NMHC drift correction threshold from 4% to 10% via a
DFR.

Time Alignment Engines 2 and 3

This error surface application was reviewed by the SC.  The group had no
issues with the data.

CE-CERT Data

Review of Selected 100 NTE Events – EPA explained all of the selection
criteria for the model validation.  The group was happy with the
selections and there were no suggestions for point replacement.  The NOx
range was 2.08 g/hp-hr to 3.24 g/hp-hr.

Ideal Emissions

Summary sheet of all reference NTE events was discussed.  This just
lists all of the reference events that will be used to generate the
measurement allowance.

Error Surface Review

Steady-State – The error surfaces for NOx were reviewed and agreed to.
 CAT wants SwRI to see if we would benefit from putting a curve fit to
the percentiles.  HC was reviewed and there were no issues.  CO was
reviewed and there were no issues.  Exhaust molar flow rate was reviewed
and there were no issues.  CO2 was reviewed and there were no real
issues.

However there is disagreement on what people think the error surfaces
should look like.  The pooled error surfaces are volatile (defined as
jagged).  The group would like SwRI to address this Wednesday morning to
see if they are concerned with the jaggedness of the surfaces.

SwRI’s modelers were present Wednesday morning to go over the groups
concern.  SwRI stated that regardless of the jaggedness of the pooled
error surfaces, the delta NOx distribution after 10,000 iterations was
well behaved with the pooled data and all error surfaces.

There was a technique proposed to take the highest 95th and lowest 5th
percentile for pooled error surfaces that are jagged.  For the 50th
percentile, we would take the one of the three that fell between the
other two.  This would eliminate the jaggedness and where data was
short, it would be extrapolated.  It will only be applied to surfaces
where the data is jagged.  Since this only applied to 4 surfaces, should
produce more robust data, and would not affect the timeline, EPA
approved of the proposal.  The group unanimously approved.

Surfaces where applied:

Steady State

SS Exhaust Molar Flow Rate – Yes

SS NOx Concentration – Yes

SS CO Concentration – Yes

SS CO2 Concentration – Yes

Transient

Transient NOx – No Change

Transient Exhaust Molar Flow Rate – No Change

SS CO2 Concentration – No Change

SS CO Concentration – No Change

BSFC

Dynamic Transient – No Change

DOE – No Change

Warm-up – No Change

mFuel

Dynamic Fuel Rate – No Change

Speed

Dynamic Speed – No Change

Torque

Dynamic torque – No Change

DOE – No Change

Warm-up – No Change

No changes were needed for the other surfaces as they were environmental
or only run on one engine.

Overall, there will be changes to 4 pooled error surfaces.

Side Note

EPA talked with SwRI’s modelers and they informed me that on the trial
model runs, it appears that the SS NOx concentration and
Manufacturers’ Torque error surfaces are dominating the measurement
allowance determination.

Also the drift correction threw out about 50 NOx results out of 1000
(about 5%) and 140 THC out of 1000 (about 14%).  Note THC was using 10%
drift correction limit.

PEMS Steering Committee Meeting Minutes

January 8th – January 10th, 2007

Schedule Update

SwRI gave a general update of the time line.  We are still on schedule
to finish at the end of January.

Replay Testing – SwRI will start running next week with feedback from
CAT whether we are replaying as well as can be done.  EPA has no idea
what this really means as no specifications or goals were given.  This
should be completed by the end of next week, January 19th.

Model Run Update – All of the event runs were completed last week. 
They are not doing sensitivity analysis, validation, etc.  The total
number of iterations was 40 million.  CAT made a statement that the
model was run for the first time.  SwRI alluded that this would be the
only time the model would be run.  CAT seems to believe that the final
report holds evidence that shows that we will need to make changes and
rerun the model.  EPA is not sure what he is getting at yet as we have
no discussed the final report.

Prior to the 24th of January, SwRI will have all of the draft reports
submitted for review.

EMA discussed the timeline issues and the fact that the draft reports
were not out early enough for adequate comment and review.  He asked if
it is fair to roll out the results at the end of the month and adsorb
them on the fly.  EMA went over the proposal that went out in the letter
that he sent to Glenn on January 5th.  This news was new to the EMA
members as far as EPA can tell.  CAT wanted more than 30 days.  He feels
that it is not appropriate and wants more days.  EMA asked the group if
30 days was better than nothing.  Most agreed, but felt that more time
might be needed.  EMA stated that there was no need to roll over the
enforceable program.  CAT stated that he does not think that the error
surfaces are precise enough.  He stated that some of them do not make
any sense.

Convergence Review

All of the reference NTE events converged within 1%.  3 or 4 were taken
out to 40,000 iterations.  Note the convergence percentile was tied to
the threshold values.  We reviewed the model run of NTE 18, which was at
the NTE threshold value of 2.  CAT noted that he would like to look at
the 5th and 50th percentiles to insure that there is no negative bias
(he specifically cited method 2).  For methods 2 and 3, there are
negative biases (5th percentile) for NTE 18.  CAT again stated that the
removal of the negative biases is necessary and will require reworking
of the error surfaces.  Note that as a percentage of point, each
method’s data seems to fall into a discrete band for 5th and 95th
percentile values (method 1 is high, method 2 is low, and method 3 is
low).  CAT showed some data that he plotted.

CAT showed that the CE-CERT method 2 deltas were up to 0.33 g/hp-hr. 
The 5 reference NTE events sent out by SwRI show that the BS NOx error
95th percentile is about 0.08 to 0.1 g/hp-hr.  So he is speculating that
method 2 NOx will not validate.  He cited that CO2 span error or bias
error high is causing a negative bias on method 2 (at the 5th
percentile).  Sharp stated that method 1 was closest to what CE-CERT was
getting.

CAT also claimed that the design of the test program does not adequately
allow us to assess the PEMS bias.  He claimed that we are blindly
stating how PEMS behave.  He points to the engine 3 SS NOx bias at high
NOx concentration.  He stated that it should be symmetric at the high
NOx end.  This ultimately forces the 5th percentile low on the final SS
NOx error surface.  CAT is fishing for a higher NOx allowance by asking
to make the error surface symmetrical at higher NOx values.  There was a
lot of discussion about the engine 3 SS NOx results.  CAT is really
pushing for a rework of the error surface.  This would require a model
rerun.  CAT wants to increase the 95th percentile to mirror the 5th. 
This is not justifiable.

CAT also stated that he is concerned about the positive CO2 bias because
it is driving method 2’s measurement allowance value low.  His stand
point is that it will preclude anyone from purchasing a PEMs that has a
lower CO2 bias as there is no advantage.  Of course the con is that by
making the CO2 SS error surface symmetrical (CAT’s proposal) this will
increase method 2’s measurement allowance value.

Replay Validation

Cycles – The NTE events that were chosen for the 3 routes are
highlighted in blue in the “EPA 100 Validation NTE Down Select-cas”
excel file.  The first is at elevations approximate to the elevation at
SwRI so it will be run at SwRI’s elevation.  For the second and third
routes SwRI will use the average elevation during the NTEs.  So here is
one point of difference between the actual run and the replay.  They
will not be mimicking actual elevation.  Note this replay brings torque
and fueling into the validation.  The raw and dilute lab analyzers will
be run during the replay testing.  CAT also stated that the testing at
SwRI of SwRI vs. CE-CERT, the 2 to 4% variation is the best you can ever
get.  This was a rare test and shows that 1065 compliant error at its
best is 2 to 4%.  They will use an inline torque meter and tweak the
reference.  CAT stated that if some of the NTE results do not match, we
would just say that we can’t get the conditions to match and drop
those events.  What are the criteria for an NTE match?  This will not be
emissions based, but based on timing, fueling, etc.  SwRI will be using
IMT to try and match after cooler temperatures; however they will only
be able to do so much.  This will make it difficult to match NOx
emissions.

Calculations – How do we derive BSFC and torque for the PEMS?  For the
test program we interpolated.  We could use broadcast values.  CAT wants
SwRI to take the torque and fuel rate direct from the CAN signal.  The
SC elected to let CAT decide what will work best.

Engine-Audit Report Review

We went through, page-by-page, the draft SwRI report.  The report turned
into a why we did what we did for many of the error surfaces. 
Indications seem to be that EMA wants to redo many of the error surfaces
without even seeing the sensitivity analysis from the model run.  There
is a major concern with bias from the EMA members, specifically from CAT
regarding the effects of bias on method 2.  This must stem from the fact
that preliminary indications are that the method 2 NOx allowance value
will be the lowest of the three methods.  EPA feels that there is going
to be a push by EMA to rerun the model if method 2 NOx does not
validate.

CE-CERT Report

The CE-CERT report was reviewed.  There was a discussion of the CE-CERT
vs. SwRI correlation testing and the error associated with the NOx
measurement.  Cummins thought that maybe the error could be due to high
background concentrations.  For NOx there is a bias shift for Method 1,
however the spread is the same for all three methods.  There was a
debate about why the CE-CERT deltas were more or less the same for the 3
methods and why method 1 was reading high.

There was a problem with the Sensor’s temperature measurement because
the Viasala was not shielded from the sun.  This affected the Semtech
humidity measurement by about 2%.

SwRI asked CE-CERT to do a carbon balance check to see if any exhaust
leaks developed during the testing.  They said that they would look into
this.

Engine 3 Horiba Data

We reviewed the Horiba engine 3 data.  Some of the results were compared
to the Semtech.  CO was biased low.  NOx was high at high
concentrations.  Flow was decent, but showed some error.

HDIUT MA Steering Committee Meeting Minutes 1-24-07

Monte Carlo Model Results

SwRI reviewed the reference NTE events and how the model was run.  We
then went into the measurement allowance generation.  CAT commented that
he didn’t like the Minimum of Maximum selection process for selecting
method 2 as the validation error surface.  In the end, method 2 NOx is
still the lowest.  CAT commented that the range of the CE-CERT NOx
values is different than the reference NTE data.  CAT would like to see
the 5th percentile extrapolation for the data.

NOx does not validate on methods 2 and 3.  CO validates on the 95th for
methods 1, 2, and 3, but does not validate on the 5th for method 1. 
Methods 2 and 3 will probably validate on the fact that less than 10%
are out of the 5th.  NMHC validates on all 3 methods.

Sensitivity

Sensitivity was discussed for the full MC simulations.

Variation (controls spread)

For box and whisker plots starting at slide 51, the + is the mean, the
box lines from bottom to top are the 25th, 50th, and 75th percentiles;
the whiskers are the intercortile range (the 10th and 90th percentiles),
and the small boxes are the points out in the tails.

For method 1 dominant errors – Torque warm-up, SS NOx, Eng Manu
Torque, and SS Flow.  These tell you what is driving the spread.  To
reduce spread you would get the error surface closer to zero.

Method 2 NOx – all four dominant are positive.  SSNOx (biggest), Eng.
Manu. BSFC, Warm-up BSFC, and DOE BSFC.  Note that CO2 did not have a
dominant effect.

Method 3 NOx – Warm-up Torque, SS NOx, and Eng. Manu. Torque.

Bias (does not effect distribution, but controls shift)

SwRI’s modelers turned off error surfaces to assess bias.  They rolled
the dice to see if the error surface was turned on or off. 

Method 1 - SS Flow and Pulse Flow showed positive bias effects.

Method 2 - SSCO2 and DOEBSFC showed negative bias effects.  The CO2
surface is causing the method 2 95th percentile push to left, preventing
validation.  It is biasing the NOx MA value low.

Method 3 - SSCO2 has a negative bias effect.

For Bias and Variation:

Method 1 - Torque Warm-up and SS NOx are driving method 1 NOx.

Method 2 - SS NOx and SSCO2OnOff are driving method 2 NOx.

Method 3 – SSNOx, Warm-up torque, and SSCO2OnOff are driving method 3
NOx.

Look at slide 79 in SwRI’s presentation.  It shows that for NTE#1, the
NOx error (emission delta) for method 2 goes from 0.121 to 0.281 when
the SSCO2 error surface is turned off.

The group felt that the model results don’t show any major flaw in the
model.  There maybe some little ones, but no major ones.

Note that 5th percentile BSCO values could be affected by SSCO.

We went over 9 PEMS ideal emissions plots.  See PEMS Ideal Emissions
Plot PowerPoint presentation.  This shows the validation BSNOx deltas
versus the BSNOx validation values at the 5th and 95th percentiles.  CAT
pointed out that there is positive bias on method 1 and negative bias on
method 2.

CE-CERT Data 

For CE-CERT data:

1) kH factor for NOx.  There was much debate about this.  Specifically
if the use of a shield was something that would normally be done.  The
issue is that if the measurement, it would also affect the dry-to-wet
correction.  There was a motion that the humidity PEMS error was a
special source of error and it was proposed that we would take the MEL
ambient data and replace the PEMS ambient data with that of the MEL. 
This would match up the humidity measurements between the two for both
the intake air humidity correction and the wet-to-dry correction.  This
would be a special source of error in the MA program that we don’t
wish to investigate further.  CAT and DDC dissented, however the motion
carried.

Note: SwRI did a quick check of the temp probe in and out of the sun and
there was a difference of 7°C.  This proves that it could be due to sun
exposure in the field.

We need to get Sensors the raw MEL data.  Contact Tom about this.

2) We got agreement on drift correction and check.

3) We got agreement on time alignment.  Each systems analyzer signals
will be aligned to ECM torque and speed.  We did not get agreement on
dispersion.  EPA stated that EPA does not agree with EMA decision not to
use dispersion or not to time align the labs together.  As decided, the
labs will align their analyzer signals to EMC speed and torque in their
own respective files.

4) We did not get agreement on use of BSCF table for method 2.  The
group felt that there would be no difference in results.  Also, there is
a problem with the CAT engine and an accurate BSFC table cannot be
generated.  Any BSFC table will do.  EPA agreed to disagree with EMA
decision no to use dispersion.  EPA asked SwRI to run a 40pt BSFC map on
that engine anyway.

Replay Validation Data

The CAT engine had a boost leak in field.  This made replay a little
difficult.  Note that data replay is only for out of cab measurements. 
CAT stated that intake leak did not allow a perfect match of the replay.
 Also, exhaust pressures were not matched yet.  This will allow us a
comparison of torque and BSFC errors.  Also, intake air humidity was not
matched.  Also they are not running the same fuel.  SwRI is running 2007
ULSD Cert fuel.  PEMS 5 used same 5” exhaust flow meter.

Boost and fuel correlation between SwRI and CE-CERT is not perfect,
especially boost.  IMT was not matched perfectly.  There was also a
timing discrepancy.  PEMS 5 field vs. PEMS 5 SwRI is very close.  PEMS
vs. PEMS exhaust flow matches well.

On altitude simulation there was a problem with the PEMS setup.  The
flow meter and sample probe was the only part of the PEMS seeing
altitude.  The PEMS static pressure sensor goes negative and flat lines
because the setup is artificial.

There was a discussion of the offset of the 3” flow meter due to the
calibration under negative pressure that took place at SwRI.  Sharp
stated that it didn’t go negative enough to clip the sensor negative. 
Also, it seems to clip negative at about 20 in H2O.

For route 2 at 4500 ft. the timing was not a good.  There was a bias on
wet NOx concentration.  This is probably due to not matching the intake
air humidity.  Flow looked good from the PEMS flow meter.  NOx mass flow
rate is low due to humidity.

For route 3 at 3500 ft. speed and fuel rate were right on.  Boost was
not, it was low.  IMT was low.  Timing was OK.  Wet NOx agreement was
good.  Flow meter was better at this altitude.  NOx mass had a low bias
on some points.

CAT made a disclaimer that he did not know how good of a match it was
due to the intake leak.

Lab Raw to PEMS NOx concentration had errors from 0 to 20 ppm.  None of
the deltas were negative.  The percent errors were 0 to 4% with most of
the data under 3%.  CO2 errors were biased high for PEMS up to 0.3% vs.
lab.  Lab carbon balance was within 1%.  Flow meter showed high bias at
4500 ft. altitude test for NOx.  This is due to vacuum effect on flow
meter.

NOx deltas at elevation showed NOx was 25% higher.  This was due to ECM
errors as processing it with just ECM torque; the error went down
significantly (to about 5%).  This basically shows that there is high
ECM torque error due to altitude.  Work is bias low, up to 15% error due
to broadcast torque.  In reality the engine was making more torque then
what the ECM thought it was making.  This really looks suspect.  The
boost leak could have hampered the ECM’s ability to predict torque. 
This calls into question the validity of the replay.  These results were
the same for methods 1, 2, and 3.  When lab torque is used for lab and
PEMS, the error drops to a range of 0 to -0.1 g/hp-hr and almost exactly
mimics the model.  This is very important.  This shows that when you
exclude the ECM torque error, the PEMS matches the model and SwRI for
all three methods.  CAT stated that a boost problem could affect ECM
torque prediction.  Since the validation used the same ECM torque for
MEL and PEMS, what went wrong on the road to cause the large PEMS –
MEL deltas?

***Need to ask Kent how on-road audit was run.  Where was span gas
sampled from?  My guess is the span port.

***During MEL audit, did CE-CERT perform zero and span calibrations?

SwRI will close off boost leak and check to see if the torque error
persists.  If it does, then we missed a huge error in the model.  If the
torque error goes away, then the boost leak is causing the problem and
we cannot use the replay to validate the torque error for the model.  If
the latter occurs, then what do we do to verify the incremental error
due to BSFC and torque by running the replay with the boost seal closed.
 While this may validate the torque component of the model, it is not in
the original spirit of the replay as the replay boost will not be the
same as the boost during the validation testing.

EMA has proposed that if the CE-CERT validation data does not allow
methods 2 and 3 to validate, we reach an impasse and the enforceable
program slips one year.  CAT doesn’t understand why we would consider
an impasse if method 1 validates.  He stated that this is the method
that has been used for years.  CAT also feels that this group should be
empowered to negotiate a measurement allowance.  EPA told him if we
can’t resolve why methods 2 and 3 did not validation, then EPA would
pass the negotiation up to EPA management.

Next Steps

CAT feels that there was nothing in the test plan that states that we
need to find out why methods 2 and 3 validation.  Sharp said that in the
technical spirit of the program, we need to figure out what went wrong
to the extent that time and money will allow.  EMA has shown interest in
rerunning the model.  CAT has now changes his position on rerunning the
model versus the last meeting.  Now he says he thinks if we do not rerun
the model, we could technically defend the result.  This is basically
because he wants the method 1 number.  CAT does feel that the biases
should be removed from the model if money and time allow, we should
rerun the model with the changes.

Model Rerun Wish List

Model Rerun

Cummins thought that if we rerun the model, we might not need to rerun
all 195 NTE events.  We could run the minimum events needed that would
allow the EDF plot.  This would be about 50 reference NTE events.  SwRI
thinks that this would be OK.  CAT doesn’t like just using 5; he feels
that we would be taking a huge short cut.  He is OK with 50.

CO

For CO, to get the model to validate, we will need to take the SSCO
error surface and shift the surface closer to zero.  SSCO was the
dominant factor in the model for CO.  There was then discussion of just
setting the MA value for CO to zero.  EPA has no issue with this.

We did decide to just center the 50th percentile for CO at zero.  This
should allow CO to validate.  EPA is OK with this.  It should make a
smaller measurement allowance.  This will not affect the Maximum Error
by Method or the Minimum of Maximum Errors determination to select the
MA value.

Take the 95th percentile on the original error surface and mirror it for
the 5th.  The 50th will be set to zero.

NMHC

There will be no change.

NOx

The discussion was what error surfaces would need to be adjusted.  It
should be the ones that are heavy hitters in the sensitivity analysis.

CAT thinks that the model is not capturing the spread and would want the
model to be reworked to capture the spread that CE-CERT shows during the
validation.  CAT also stated that if the CE-CERT recalculation allows
method 2 to validate, we still shouldn’t be done because of the bias
effects on the method 2.  A lot of the group reflects this concern about
the bias.

There was concern about the fact that a good portion of the PEMS data
may invalidate from the drift correct.

SSNOx Error Surface

CAT would like to increase the SSNOx error surface values to match the
CE-CERT deltas.  What would justify this?  He states that he has seen
historical data from WVU that shows a higher NOx concentration error. 
However, that wasn’t part of this test program.

We will take the SSNOx error surface and go through SD calculation and
see what effects it has on it.  This will be an experiment to see what
effect it has on the original error surface.  Note that this will remove
any bias, but will increase the NOx error from this error surface.  They
are trying to match the CE-CERT results.

SSCO

It was proposed to do the same here as to the SSNOx error surface.  This
will remove the SSCO positive bias and will increase the magnitude of
the SSCO error surface.

SS Exhaust Molar Flow

Sharp proposed that we should go back and look at the data for the 3”
flow meter to figure out why the error was biased high.  CAT does not
want to figure out why it is high.  It was agreed to have SwRI and
Sensors look at the data and see if there is any reason for the 3”
high bias.  We would them have SwRI reprocess the error surface to see
what the effects will be.

Schedule Update and Reporting Status

HDIUT MA Steering Committee Meeting Minutes 2-14-07

Review of Action Items from Previous Meeting

SwRI went over the action items and status since the last meeting.

Schedule Update

Draft final report should be finished on or before March 1st.  Review
and Comment period means final will be delivered by the end of March.

CE-CERT Data Re-analysis (EPA)

Carl used 4, 4, 1, 1 for PEMS time alignment.  EMA does not like the
provision that would allow a change in the measurement allowance value
before 2010.  EMA might want the program to roll if we insist on
revising before 2010.

Replay Validation Data analysis (SwRI)

Cummins noted that we spent a lot of time on the replay only to find
that we could not mimic what was done in the field.  For method 1 at the
threshold, the torque error is 0.1.  The pink outside of the blue are
low load, high BSFC values.  Highest load points are route 1, which
doesn’t make sense.  The ECM torque and fuel for this route correlates
best to the model.  Torque and BSFC maps are most accurate at high
torque near or on the lug curve.  Note that the negative % of Pt error
was as low as -15%.  This is outside of the bounds of the
Manufacturer’s supplied ECM error surfaces.  CAT made a statement that
the error from this engine is a good example of an engine that has been
out in the field and had the ECM reflashed, turbo changed, etc.  Thus he
is supporting the results.  We will review the data from the CAT and DDC
engines tested during the program and see what those errors look like. 
For BSFC, the error approached that of the manufacturer’s submitted
values for the error surface.  Cummins stated, however, that the numbers
are higher than what was used in the manufacturers supplied data for
BSFC.

SwRI has the BSFC and Torque errors from engines 1 and 2 processed from
the 40-point maps generated during testing.  The DDC engine showed % of
Pt error out to ± 10%.  The CAT engine showed % of Pt error out to +40%
and -10%.  CAT stated that there error was so high due to the HUEY fuel
system on the C-9.  CAT stated that the HUEY are harder to map.

Reprocessing data with lug curve instead of J1939 for CAT validation
engine reduced the errors to +2.8% to -2.9%.  Problem is that lug curve
used was generated with boost leak.  SwRI will get cert lug curve from
CE-CERT and reprocess.  SwRI will also get lug curves for the 3 engines
tested during this program and will determine the % of Pt error using
these lug curves.  This will tell us what the ECM torque error is for
each of these engines.  Note that EMA may want to revisit the
Manufacturer ECM Torque error surface.

Model Error Surface Re-analysis (Cummins)

Cummins did a great reanalysis of the SSNOx error surface.  The analysis
was solid, with the exception of the slope generated by the 3 high
fliers with negative bias (from engine 3).  This drove the slope
creating larger deltas in the 100 to 400 ppm range than what the data
actually implies.  There was a lot of debate on what we should do next. 
CAT feels that the error in the 100 to 300 ppm range should be higher.

Cummins presented a level independent case that assesses both
repeatability and reproducibility with 90 data sets (takes PEMS results
separately).  The overall output would be a standard deviation.  Thus
the error surface would be a line.  Cummins will put this together for
presentation at the next meeting.

Review of Environmental Draft Report Sections

We reviewed the draft environmental report.  There were no major issues
with the discussion.

Discussion of Individual Error Surfaces

How to modify surfaces?

EMA reiterated that if EPA plans to change the MA before 2010, he would
recommend to his members that a deal not be reached and the program
rolls one year.

HDIUT MA Steering Committee Meeting Minutes 2-26-07

Review of Action Items from Previous Meeting

Reprocessing of Replay Validation data with Cert Lug Curve

Reprocessing of 40-pt Torque Errors with Cert Lug Curves

Discussions on Latest EPA Proposal for Measurement Allowances

Status / Approval of EPA Proposal for Measurement Allowances and
Follow-on Work

EMA agreed to the proposal to use the method 1 MA values.  0.45 g/hp-hr
NOx, 0.02 g/hp-hr NMHC, and 0.5 g/hp-hr CO.  EPA agreed to provide
stability on these values through 2009.  EMA understands that new values
could be proposed for 2010.  EMA will provide feedback on error surface
changes to the model that could push methods 2 and 3 toward validation. 
They agreed to let EPA probe the model and then reengage EMA for a phase
2 if it is warranted.

Schedule Update

SwRI is targeting getting the draft report out by March 1st.  Final
report delivery is scheduled for March 31st.

Is there any plan to present anything on measurement allowance at the
CRC meeting March 26th – 28th?  EPA will follow up with this. 
Modeling of emissions measurement is the 27th.  PEMS is session 3 of day
one.  Format is fifteen minutes.  Contact SwRI Tennant to get on
schedule.  Agency 15 minute presentation and SwRI 15 minute
presentation.  CAT sent an e-mail regarding this to SwRI T.

Replay Validation Data Re-analysis

Reprocessed C-15 torque data using certification lug curve.  Results
show CAN – Lab Delta at less than ± 5%.  This is within the bound of
the manufacturer supplied torque error surface.  This did show that for
this C-15, the map torque is not accurate.  The International lug curve
errors showed errors as high as 12% at low torque levels (this is fuel
ratio to lug curve).  There is up to 20% error that is out of the realm
of the NTE or on the boarder.  At low power points you will over predict
torque.  DDC stated that you need the brake curve to get correct values
and friction torque needs to be subtracted out.  The data for the C-9
Huey engine was shown.  The torque errors were as high as 40% of Pt in
the NTE zone using the lug curve method.  Average high errors are on the
order of 20%.  For the DDC engine, the lug curve results were worse than
the ECM error results.  The lug curve data showed error as high as 20 to
25% of Pt.  SwRI is not positive that they have the correct lug curve
for the ECM on this engine.  This could account for the large error.

So overall, based on the data, we cannot make the statement that one
method is better than the other.  Overall the error is greater than the
5.9% error reported by the manufacturers.  Thus for the enforceable
program, the current method to predict torque could have higher error. 
Overall, due to the differing results from engine-to-engine, the results
are inconclusive.  J1587 might be a better method to use.  For torque
error, it might have been better to have created an error surface that
was generated from the lab map vs. predicted torque for the three
engines tested.

If you really wanted to do proper brake torque calculation you will need
to de-convolute friction torque from the torque signal.  Also, it states
that if the lug curve is the best way to do this, manufacturers should
use the lug curve during compliance testing.  The manufacturers might
ask Sensors to add a column for friction torque that would allow this to
be subtracted from the torque prediction.

Cummins gave a breakdown of the calculations that we should have used
(subtracting out friction torque) instead of % load X brake torque.  The
way it was used and presented here by SwRI, it will over predict torque
in the lower right corner of the NTE zone.  This is what we saw in the
SwRI analysis for the three test engines.

CAT has stated that changes to turbochargers, injectors, etc. can cause
error in torque prediction.  This is a compliance problem since changes
to the engine compromises the ECU’s torque prediction.  Then there was
a statement the ECM parameters were never intended for use in
compliance.

Error Surface Discussions for Follow-on Work

CO - We will use Cummins’s process with level independence.  We would
look at the mean values of each PEMS set versus zero using the existing
data.

CO2 - There are some data with large error on the 7 to 8% range.  The
proposal was made to zero the 50% percentile.  We will create a new
error surfaces using Cummins’s technique with level dependence until
the plateau.  Then we will extrapolate the higher error from 7 to 8% and
hold the last value at 8%, carrying it through 9%.

NOx - It was decided that we would add in the additional data set from
engine 2.  This will increase the error slightly.  We will then generate
2 new error surfaces, one with level dependence and one without.

Therefore we would generate 3 surfaces:

One error surface level independent with all points, symmetrical.

One error surface level independent without the 3 high negative points
on the one PEMS.

One error surface level independent up to 350 ppm, with line segments to
encompass the 3 high negative values.  We will use the max of the
absolute value of the 5th and absolute value of the 95th percentiles.

SS molar exhaust flow – SwRI will correct the 3” flow meter span
error (this will reduce the error from 7% to 4% and then run through
Cummins’s process as level dependant.  After the adjustment we will
mirror the 95th to the 5th.

The level independent data work up was described by Cummins and SwRI has
the details.

Report Review

We discussed and made a list of lessons learned.