NOTICE

This technical report does not necessarily represent final EPA decisions or positions. It is intended to present technical analysis of issues using data that are currently available. The purpose in the release of such reports is to facilitate the exchange of technical information and to inform the public of technical developments. Assessment and Standards Division Office of Transportation and Air Quality U.S. Environmental Protection Agency

National Renewable Energy Laboratory U.S. Department of Energy

Coordinating Research Council 

# EPAct/V2/E

- 89: Assessing the Effect of Five Gasoline Properties on Exhaust Emissions from Light

- Duty Vehicles Certified to Tier 2 Standards

Final Report on Program Design and Data Collection

Appendix L LOD/LOQ Method

E PA-420-R-13-004 April 2013L-2 LOD/LOQ Method for EPAct/V2/E -89 Program at SwRI ®

This document is a description of the method that will be used for handling chemistry data in the EPAct/V2/E

- 89 Program , with respect to Limit of Detection (LOD) and Limit of Quantification (LOQ). In addition to describing the final method for processing the data, the methods used to determine and track LOD and LOQ are described, as well as the reasoning behind the method and the background and experiments which led to its development.

Most of this discussion is related specifically to the carbonyl and a lcohol data, where potential interference from the sampling media is significant issue.

Root Issue

The primary problem for which this method was developed is how to properly address media interference in the measurement process when the exhaust samples themselves are at levels similar to that interference. This problem arises because we are using methods and media which were developed for use at much higher measurement levels (and which are perfectly adequate for those higher levels).

For example, base d on our experiments, most of the media interference for carbonyls is on the order of 0.5 mg/mile or less, which is similar to the levels were are trying to quantify in the exhaust samples. It is important to recognize that the traditional process has bee n developed to determine compliance with formaldehyde standards on the order of 4 to 18 mg/mile, and to quantify even higher values on pre

- Tier 2 vehicles, where this level of blank interference represents at most 10% of the standard. We are trying to use the same process to quantify values an order of magnitude lower than that, and this has required some refinement of the process.

Proposed QA Blank Tracking and Data Analysis Process

It should be noted that all of this tracking is done on a compound- by

- compound basis, so that (for instance) an issue might be observed only on acrolein while all other compounds could still be within limits.

1. The laboratory and field blanks are analyzed in a daily basis to determine a daily average blank value ( i blank ).
2. Data from these blanks are tracked over time, and are used to generate a 5-day moving average ( blank ), and an associated standard deviation (σ blank )

. 3. The daily blanks for a given test day ( i blank ) are first evaluated in comparison to the previous 5-day average as follows:

Is blank i 

# blank blank σ 3 ≤− ?L-3 a. If the above answer is Yes, update the 5

- day blank and σ blank , and proceed with data analysis. This indicates that the current blank falls within acceptable variation from the running average, and is described in Step 4 below.

b. If the above answer is No , then the current blank is outside the normal range of variation and is flagged for review. This process is described further below i n Step 5.

4. Data analysis for “normal” blanks (Case 3a, answer to 3 is Yes) . Each analyzed sample (which is either a dilution air background or a dilute exhaust sample) is compared individually to the 5- day blank and σ blank
.

a. Is blank d uncorrecte 

# blank sample σ 3 ≤− ?

b. If the answer to 4a is No , then calculate the blank corrected sample as follows:



# blank sample sample d uncorrecte corrected 

# − =

c. If the answer to 4a is Yes , then report 

# 0 = corrected 

# sample for that given sample. The means that the difference between the sample and the blank does not rise above the level of noise in the blanks (i.e., we cannot tell the difference between the sample and a media blank).

d. Analysis then proceeds normally with background correction (using the dilution factor) and mass calculations. The final analyzed mass is always reported. Negative masses are reported as zero and set to zero on a bag

- by

- bag basis prior to composite calculations.

e. Analysis Complete, Proceed to Reporting.

5. Review of and processing “outlier” blanks (Case 3b, answer to 3 is No ).

a. Manual review of blank data chromatograms to determine if an analytical problem is at fault, and correct if possible. b. Manual review of blank data and association with sample/background samples to determine if the blank itself is an outlier or if there is a shift observed for all samples and blanks on that day.

c. If blank itself is an outlier, discard and process test data (as described in Step 4) using the previous 5 -day average. Do not update blank and σ blank
. d. If the shift is consistent for all samples, backgrounds, and blanks on that day, process test data (as described in Step 4) BUT use the daily average blank value

i blank in place of blank for that day only. Do not update blank and σ blank
.L-4 6. Media Shift.

a. If outlier behavior is consistent for three days running, blanks may have shifted. In other words, if a blank value shifts from low to high, but then stays high for three days, this may indicate a shift in the media, rather just an outlier. If this is the case, data will be reviewed.
b. If it is determined to be appropriate, blank and σ blank will be reset (initially with values from the three days of question). Note this will be done on a compound

by -compound basis.

It should be noted that there are multiple review steps throughout this process in the event of outliers. It is hoped that over the course of these review steps, any proces s issues which may have contributed to either an increase in the frequency of outliers or a shift in blank values can be identified and corrected.

Background on Process Development

To establish an appropriate method for determination of the LOQ for a given measurement, it is necessary to understand the key factors driving measurement variability. These factors will vary depending in the measurement in question.

In the case of gaseous HC speciation measurements, the primary driver is analytical variab ility. Experiments have indicated that the bag media do not contribute in a significant manner to the measurement, which is not unexpected given the requirements on the media with respect to HC off

- gassing, as well as the multiple purge

- evacuation process es designed to eliminate carryover. Therefore, the repeatability of the GC instrument was quantified and an LOQ was established in terms of raw area counts at 200 counts. This analytical LOQ was determined by examining repeat measurements of low

- level st andards. LOD and LOQ are then determined by examining the ratio between the standard peak height and the noise response. The LOD is the lowest concentration where the standard

- to

- noise ratio is 3 to 1, while the LOQ is defined as the lowest concentration where the standard

- to

- noise ratio is 10 to 1. These ratios follow standard good laboratory practice for GC analysis. Any analyzer response below this LOQ in terms of area counts is reported as a zero, because we cannot reliably quantify a number below this threshold. Note that this process is done on every individual measurement, including samples and backgrounds, before the numbers are fed into calculations to determine mass.

For the carbonyls, and to a lesser extent alcohols, the sample media have a much higher potential for interference. Table 1 shows a comparison of analytical instrument LOD versus the average blank levels, with the data given as raw area counts. The values in the table compare 3σ for the instrument (as quantified by multiple anal yses of a standard of 0.0003 ug/ml) against 3σ for the blanks (as quantified by examining multiple blanks). This comparison demonstrates that the variation in blank area counts is at a much higher level than the instrument LOD, which indicates that blank variation is the dominant source of variability in the measurement.L-5 TABLE 1. COMPARISON OF INSTRUMENT AND BLANK VARIABILITY FOR CARBONYLS (AREA COUNTS)

To determine an appropriate method to deal with this variation, it was also necessary to determ ine if this variation is present on a day

- to

- day basis, a batch

- to

- batch basis (i.e., different batches or lots of cartridges), or on the basis of individual blanks. Two data sets were used for this analysis. One data set is similar to the set shown abov e which includes area count values for blanks determined over a period of three months’ time, including about 60 days of data and covering more than one batch. The second data set was generated but taking 10 blanks from a single batch and analyzing all of them in a single analytical run on the same day. The results are summarized below in Table 2, with the data given as raw area counts.

TABLE 2. COMPARISON OF LONG

- TERM AND SHORT

- TERM CARBONYL COMPOUND BLANK VARIABILITY Compound Instrument LOD Blank Average
FORMALDEHYDE 984 2823 ACETALDEHYDE 1285 9217 ACROLEIN 757 1115 ACETONE 661 39183 PROPIONALDEHYDE 850 1070 CROTONALDEHYDE 392 1605 N-BUTYRALDEHYDE + MEK 785 6002 BENZALDEHYDE 437 1707 HEXANALDEHYDE 326 1154 ISOVALERALDEHYDE 371 4155 VALERALDEHYDE 716 976 O-TOLUALDEHYDE 389 1659 M/P-TOLUALDEHYDE 383 1762 DIMETHYLBENZALDEHYDE 496 1291

90-day Single batch 90-day Single batch Formaldehyde 3460 3647 941 1070 Acetaldehyde 7858 6791 3072 2677 Acrolein 253 354 372 389 Acetone 21482 20767 13061 5020 Propionaldehyde 256 389 357 409 Crotonaldehyde 432 430 535 465 N-butyraldehyde & MEK 2172 1722 2001 2303 Benzaldehyde 783 593 569 316 Isovaleraldehyde 262 381 385 486 Valeraldehyde 1787 1733 1385 1283 o-Tolualdehyde 128 43 325 108 m/p-Tolualdehyde 437 255 553 149 Hexanaldehyde 1512 1898 587 746 Dimethylbenzaldehyde 267 8 430 25 Standard Deviation Average CompoundL-6 In most cases the average an d standard deviations for both data sets were very similar. This indicates that the variation exists on a cartridge

- to

- cartridge basis, even within a single batch, and that this short

- term media variation cannot be distinguished from any longer term facto rs. There are a few compounds that do not follow this trend such as acetone, wherein the variations are likely also driven by daily variations in the laboratory environment.

As a result of this determination, the practice of using a daily blank for correction of samples for that given day is problematic, because the cartridge

- to

- cartridge variability means that a daily blank set may not necessarily represent the sample cartridges. Therefore, it is necessary to average a number of blanks over time to repr esent the media correctly. However, any method must also track sudden shifts in media which do occur from time to time, as well as dealing with individual outliers.

Blank data for both formaldehyde and acetaldehyde are given in Figure 1 and Figure 2, respectively, showing different kinds of behavior that need to be addressed. In the case of formaldehyde, the blanks are generally well behaved, with some slow movement over time indicated. However, there are also individual outliers that need to be dealt with. On the other hand, the acetaldehyde blanks show periodic and significant shifts, such as a shift which occurs at about Blank 59 from a low level to a higher level. On review this shift is also present in actual test samples and backgrounds from that day, indicating a real shift in the media or process.

Similar data were examined for the heavier carbonyls, and a similar analysis was also performed for the alcohols (although in the case of alcohols only the methanol data showed evidence of variabi lity). Based on all of these data, it was determined that a 5

- day moving window for would be appropriate to properly characterize media blank variation, while also being short enough to detect movement of the blanks. In addition, the inclusion of a 3

- sig ma outlier test to trigger manual data review accounts for detection of outlier days, which might result from either an individual blank issue or a shift which affects an entire test day. Finally, a provision is included to reset the daily average, in the event that a real long

- term shift is observed over several days, such as observed from time to time in the acetaldehyde data.

Figure 3 shows acetone blanks over the same time window. In this case there are several outlier days, which upon analysis were reflected in the data for that day, but then the blank levels returned to normal. Thus far, we believe that acetone is uniquely affected by laboratory environment given the presence of acetone in many places. In these cases, the blanks for that day shou ld be used to process that data for that day, but the running average should not be disturbed for an outlier day.L-7 FIGURE 1. FORMALDEHYDE BLANK TRACKING

FIGURE 2. ACETALDEHYDE BLANK TRACKING Formaldehyde Blank Tracking (March 2009 - June 2009)

0 1000 2000 3000 4000 5000 6000 7000

0 20 40 60 80 100 120 140 Blank Number (2 per day)Area Counts Formaldehyde Blank Area Counts 5-Day Moving Average

Acetaldehyde Blank Tracking (March 2009 - June 2009)

0 2000 4000 6000 8000 10000 12000 14000 16000

0 20 40 60 80 100 120 140 Blank Number (2 per day)Area Counts Acetaldehyde Blank Area Counts 5-Day Moving AverageL-8 FIGURE 3. ACETONE BLANK TRACKING

Figure 4 illustrat es a running example of the final blank process that was developed using formaldehyde as an example. The figure shows the five

- day running average blank value (note that there are two blanks analyzed per day, as well as the five

- day running 3σ upper and l ower control limits. As noted there are occasional single blanks which rise above those limits, which would trigger review. In these cases, the review indicates that these were outliers that were not represented in the data, and therefore, these measurem ents were not used to modify the running average or control limits. In one case, near Run 45 the control limits are very tight due to a period of time when the variation was small. In this case, although the blanks shown are slightly above the 3σ limit, the movement is small and still well within the historical norm, and in that case the judgment is made to utilize the data. This example also underscores the fact that a set of process rules are not a complete replacement for good engineering judgment. Acetone Blank Tracking (March 2009 - June 2009)

0 10000 20000 30000 40000 50000 60000 70000 80000

0 20 40 60 80 100 120 140 Blank Number (2 per day)Area Counts Acetone Blank Area Counts Series2 5-Day Moving Average

Outlier days - reflected in data for that day but not stable for more than 2-days - blanks used to process that day but NOT in running averageL-9 FIGURE 4. FINAL BLANK PROCESS EXAMPLE WITH FORMALDEHYDE

Figure 5 illustrates a similar example for methanol tracking. Methanol appears to be affected by periodic issues which generate a low level noise in both blanks and samples, which could result in erroneous reporting of methanol and therefore affect NMOG calculation. This is confirmed by the fact that when the blanks are clean there is also no methanol detected in exhaust samples. The developed QA process is able to detect this noise and account for its effect on the data, so that only a significant quantity of real methanol emission would be reported, and the data are not disturbed by this process noise. In areas of media/process variability, the 3

- sigma screening value (yellow triangles) rises , as shown in Figure 5. Therefore, the LOQ screening process outlined earlier of checking the different between sample and blanks against the five

- day 3

- sigma would be an effective screening for this issue. Formaldehyde Blank Tracking (March 2009 - June 2009)

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000

0 10 20 30 40 50 60 70 Day NumberArea Counts 5-DayAverage 5-Day 3Sigma + 5-Day 3Sigma - Individual Blanks

Outlier blanks not used in 5-day
average

Small shift outside current 3sigma but still acceptableL-10 FIGURE 5. FINAL BLANK PROCESS EXAMPLE WITH METHANOL 

### Methanol Blank Tracking (March 2009 - June 2009)

10 210 410 610 810 1010 1210

0 10 20 30 40 50 60 70 80 Test DayArea Counts Individual Blanks 5-Day Average 5-Day 3Sigma +

Outlier day, significant effect on samples Region of blank noise, similar behavior of samples, 3sigma data screening cleans up this process issue in data

Clean blanks, no methanol in samples either