2

Measurement variability was assessed using the 90th percentile upper bound coefficient of 

variation (CV) which is the same statistic used to estimate the PM2.5 variability of collocated 

sites for the primary standard. Precision estimates where made at each 10th percentile 

visibility index concentration at each collocated site as well as providing an overall estimate 

of precision for each site and for all collocated sites. The 90th percentile upper bound CV for 

all sites averaged about 5.5% with a per site range of 3.7% to 9.4%. 

Although neither network has a program to measure bias, there are a number of sites were 

sampling instruments from both networks are collocated. Estimates of relative bias between 

the collocated sites demonstrated that VI estimates were within 0.5 to 6.1% of each other 

with an overall average percent difference of 1.5%. It was concluded that with the current 

quality systems being implemented in both the CSN and IMPROVE programs, EPA can 

expect that measurement uncertainty of the VI measurement can be controlled to the current 

PM2.5 precision data quality objective of 10% . As mentioned in the report, CVs around the 

proposed secondary NAAQS value are in the range of 5% and that both the CSN and 

IMPROVE networks were producing comparable results. 
1

Measurement Uncertainty Estimates of Collocated CSN  and IMPROVE Data for Use in the Secondary

PM2.5 Standard for Visibility

Part of the NAAQS review process includes an assessment of the quality of the monitoring data in order

to determine whether data quality objective (DQO) goals for precision and bias are acceptable. These

DQO goals are then used to set measurement quality objectives that the monitoring organizations can

use to assess and control the quality of its ambient air data.   In 1997, data quality objectives goals were

developed for precision and bias related to making PM2.5  NAAQS decisions and published in the ambient

air quality assurance regulations (40 CFR Part 58, Appendix A).   The DQO goals established for the

primary PM2.5 standard will not change.  However, a secondary PM2.5 visibility standard is being

proposed that will use data from the Chemical Speciation (CSN) and Interagency Monitoring of

Protected Visual Environments (IMPROVE) networks.  Since these two networks are already collecting

data for objectives other than NAAQS decisions, OAQPS needs to assess the measurement uncertainty

relative to use in NAAQS decisions.  

The secondary PM2.5 visibility standard will be measured in units of deciviews.  Daily estimates of PM2.5‐

related reconstructed light extinction, bext (expressed in units of inverse megameters (Mm‐1)), are

derived from the following equation:

Where:

PM2.5 bext: i = PM2.5‐related light extinction in Mm‐1 for day i;  

Sulfate i        = ammonium sulfate for day i;  

Nitrate i        = ammonium nitrate for day;  

OM i          = organic mass for day;  

EC i         = the reported concentration of elemental carbon for day i;  

FS i         = fine soil for day i; and  

f(RH) m,gp = the RH hygroscopic growth factor determined from the EPA  “climatological f(RH)

database” corresponding to month m for day i for the grid point gp closest in distance to

the monitoring site  

Fine soil (FS i) is PM2.5 crustal material.  It is derived from the reported PM2.5 concentrations of

aluminum (AL), silicon (Si), Calcium (Ca), Iron (Fe), and Titanium (TI) using the formula:  FS = 2.20[Al] +

2.49[Si] + 1.63[Ca] + 2.42 [Fe] + 1.94 [Ti].  f(RH) is a hygroscopic factor used to relate a given relative

humidity (RH) to its impact on PM2.5 light‐scattering.

Daily estimates of βext, in units of Mm‐1, are converted into PM2.5 visibility index values, in units of

deciviews,1 according to the following equation:

                                                           

1 Pitchford M (2010). Assessment of the Use of Speciated PM2.5 Mass‐Calculated Light Extinction as a Secondary PM NAAQS

Indicator of Visibility. Memorandum to PM NAAQS review docket. November 17, 2010. Docket ID number EPA‐HQ‐OAR‐2007‐

0492‐0337















   

10

( _ 10) _ _ 10 ln 2.5

2.5 i PM bext

PM visibility indexi

PM = (3 f(RH) ) + (3 f(RH) ) + (4 OM ) (10 ) 2.5 ext: i m ,gp i m ,gp i i b  Sulfate   Nitrate     ECi
2

The reader is referred to the NAAQS proposal 2

  for a more detailed explanation of the visibility index

(VI) calculation.  

Approaches to the Evaluation of Measurement Uncertainty for the Visibility Index

The best information to assess measurement uncertainty can be derived from the collocated data that

are collected at a number of sites in the CSN and IMPROVE networks.  One could evaluate measurement

uncertainty by two possible approaches:

1. Estimate variability of each parameter used in the VI calculation separately and then perform

some additive error (propagation of error) assessment.  

2. Estimate the VI value of the primary sampler and the collocated sampler each day both samplers

provide valid results.  Estimate variability of the VI pairs and provide some statistically relevant

estimate of the average variability or a distribution of variability. 

OAQPS discussed both approaches internally and with QA representatives from the IMPROVE network

and there was agreement to proceed with approach #2 and assess precision between VI values at sites

with collocated CSN samplers and sites with collocated CSN/IMPROVE samplers3

.   The assessment

should inform us of the measurement uncertainty associated with the VI estimate using the current data

quality requirements documented in the IMPROVE and CSN QA Project Plans (QAPPs) and standard

operating procedures (SOPs).  Although we do not plan to use approach # 1, this paper will provide

precision estimates, by parameter and site, for the six collocated CSN sites and for the nineteen

IMPROVE sites that collocate PM2.5 sampler modules (further explained below).  Since the VI values

require input of the parameters listed in Table 1, data quality for these parameters must be evaluated

and controlled to acceptable levels in order to achieve an acceptable level of data uncertainty of the VI

estimate.   

It is worth noting that f(RH), a parameter that can influence measurment uncertainty, was not

evaluated in this assessement. It was not estimated in the field and since the same f(RH)  value was used

in the VI calculations for both collocated samplers,  the parameter would not influnce data uncertainty.

Data Sets and Data Manipulation

The evaluations that follow are derived from different sources of data.  They can be characterized as

follows:

 collocated  CSN data for parameter level evaluations,

 collocated IMPROVE data for parameter level evaluations, and  

 collocated CSN and IMPROVE data for VI evaluations.

   

                                                           

2 40 CFR part 50 Appendix N 3 Comparing two different samplers can provide an estimate of relative bias which is evaluated later in the paper
3

Collocated CSN Data for Parameter Level Evaluations  

Details of the CSN program, including the QAPP and

SOPs can be found on the AMTIC Webpage4

.  Table

1 provides a list of the parameters used for

precision estimates.  Collocated data from May

2007 through December 2010 were downloaded

from AQS for the precision estimates.  The list of

collocated sites, along with the dates of carbon

conversion (described below), is provided in Table

2.    

Table 2 represents the collocated CSN sites that were used in the evaluation.  Two CSN samplers are

collocated at each of the sites and can consist of up to three sampler types:  

1. URG 3000N for the new carbon parameters  

2. Met‐one SASS or super SASS for mass, ions,  elements, and old carbon (prior to conversion)

3. URG MASS 400 for mass, elements and ions prior to replacement with a MetOne

The data from these sites were used to perform evaluations of variability for each parameter for the

days both samplers operated and provided valid data. Data were removed for any pair of data points

where one measurement was missing.   The data were also removed for any pair of data points where

one measurement was below the average parameter method detection limit (MDL).  An MDL is

associated with every sample value.  For each parameter, the MDLs for each sample value were

averaged.  This average was then used to remove data for any measurement value below this MDL

average. The MDLs used for this assessment can be found in Table 5 for each CSN parameter. No other

data manipulation or reduction occurred.   

Table 2. CSN Collocated Sites    

CSN/CSN Collocation

Site Name Location Site ID Date of Carbon

Conversion

Number of

Paired Values

Bakersfield Bakersfield, CA 060290014 May 1, 2007 127

Rubidoux Rubidoux, CA 060658001 May 1, 2007 177

New Brunswick New Brunswick, NJ 340230006 October 1, 2009 141

GT Craig Cleveland, OH 390350060 October 1, 2009 133

Roxbury Boston, MA 250250042 October 1, 2009 162

Deer Park Houston, TX 482011039 April 1, 2009 123

                                                           

4 http://www.epa.gov/ttn/amtic/speciepg.html

Table 1. Parameters Used in Parameter Level and

VI Measurement Uncertainty Evaluations

AQS Parameter Code Parameter

88104 Aluminum

88111 Calcium

88126 Iron

88161 Titanium

88165 Silicon

88306 Nitrate

88403 Sulfate

88305 Organic Carbon*

88307 Elemental Carbon*

88370 Organic Carbon **

88380 Elemental Carbon **

* Parameter codes for carbon before the carbon conversion

** Parameter codes for carbon after the carbon conversion

(described below)  
4

Carbon conversion‐ After review of the CSN and IMPROVE carbon data (around 2005) there was a

realization that there were differences in the OC and EC estimates between the networks.   This

disparity led to a carbon conversion project5 which began conversion of the CSN sites to IMPROVE‐like

carbon sampling and analysis6 from May 2007 through October 2009. In general, it is expected that the

CSN results for OC were higher than IMPROVE prior to the conversion due to differences in sampling

filter face velocity (CSN had lower face velocity, resulting in higher OC). Differences in EC were generally

smaller, not related to filter face velocity, and also site specific.

In addition, one CSN site (Deer Park) replaced their legacy sampler for mass, elements, and ions (URG

MASS 400) with a MetOne sampler on February 3, 2009.   

Collocated IMPROVE Data for Parameter Level Evaluations  

Details of the IMPROVE program including the QAPP and SOPs can be found on the IMPROVE Webpage7

.

Table 3 lists the IMPROVE sites that had sampling modules collocated.  The IMPROVE sampler is made

up of four sampling modules8

.  Three of the modules are for PM2.5 sample collection (A, B, and C) and

the fourth is for PM10.  Unlike CSN, the IMPROVE program does not collocate two complete samplers at

each site but collocates one of the three

PM2.5modules as a fifth sampling module

at select sites.  This results in about six

sites with one of the collocated sampling

modules (A, B, or C).  The IMPROVE

program provided the data for the

collocated modules.  Collocated data for

the fine soil fraction were provided for the

years 2006‐2009 and data for OC, EC,

sulfate and nitrate were provided for years

2007‐2010.   

Similar to the CSN data reduction, data

were removed for any pair of data points

where one measurement was missing or if

the data points were below the average

MDL for that parameter.  MDLs were

averaged in the same manner as described

for the CSN data.   MDLs are identified for

the IMPROVE data in Table 6. No other

data manipulation or reduction occurred.   

                                                           

5 AMTIC Carbon Implementation  Website http://www.epa.gov/ttn/amtic/specurg3000.html 6 PM2.5 Speciation Newsletter Issue #5 http://www.epa.gov/ttn/amtic/spenews.html 7 http://vista.cira.colostate.edu/IMPROVE/

8 http://vista.cira.colostate.edu/IMPROVE/Publications/OtherDocs/IMPROVEDataGuide/IMPROVEdataguide.htm

Table 3. IMPROVE/IMPROVE  Module Collocations

Common Site Name Location Site Code

Big Bend, NP TX BIBE1

Blue Mounds MN BLMO1

Everglades NP FL EVER1

Frostburg MD FRRE1

Gates of the Mountains MT GAMO1

Hercules‐Glades MO HEGL1

Hoover CA HOOV1

Lassen Volcanic NP CA LAVO1

Medicine Lake MT MELA1

Mammoth Cave, NP KY MACA1

Mesa Verde CO MEVE1

Phoenix AZ PHOE1

Proctor Maple Research Facility VT PMRF1

Olympic WA OLYM1

Sac and Fox KS SAFO1

Saint Marks FL SAMA1

Sawtooth NF ID SAWE1

Seney MI SENE1

Trapper Creek AK TRCR1
5

Collocated CSN and IMPROVE Data for VI Evaluations

This section will describe the CSN and IMPROVE data used for the uncertainty evaluation using the

visibility index.

Collocated CSN  VI Data  

Members of EPAs Air Quality Analysis Group (AQAG)  extracted the CSN data from AQS for years 2007 to

2009 and performed the VI calculation for the collocated CSN sites listed in Table 2.   Sampling

frequency for the collocated CSN sampler was every 6 days.

As proposed in 40 CFR part 50 Appendix N, all 5 VI parameters need to be present  for each day the

index is calculated.  The Appendix A quality assurance regulatory proposal for light extinction suggests a

cutoff value of 5 deciviews be used before any collocated pair is included in the coefficient of variation

(CV) calculation (both values would have to be greater than 5 deciviews).  Therefore VI values less than 5

deciviews were removed from the database. There were no more than 4 values removed from the

entire CSN and CSN/IMPROVE  collocated sites based on this restriction.   There was no data

manipulation or reduction beyond the restrictions mentioned above. Unlike the CSN/IMPROVE

collocation, there was no removal of data from sites that used other samplers to collect OC/EC before

conversion to the URG 3000N, since the collocated pair of samplers at the site were the same make and

model.   Therefore,  parameter codes 88305, 88307, 88370 and 88380 were used in the collocated CSN

assessment

Collocated CSN/IMPROVE VI Data   

The AQAG provided a VI data set of the collocated CSN/IMPROVE Sites (January, 2007 through

December, 2009).  Table 4 lists the sites with collocated CSN/IMPROVE samplers along with the dates of

carbon conversion for the CSN samplers.  Sampling frequency at the CSN/IMPROVE sites was every three

days.

Based on the issues

described earlier in relation

to the carbon conversion,

data evaluations were

performed only on data

from CSN samplers after

conversion to the URG

3000N.  This had an effect

on the number of collocated

VI values reported for the Phoenix and Fresno sites.  Other than this data reduction, the same data

manipulation techniques used in the collocated CSN data were used for this data set.

Table 4. Sites with Collocated CSN and IMPROVE   

CSN/IMPROVE Collocation

Site AQS Site ID Date of Carbon

Conversion

Number of

paired values

Birmingham, AL 010730023 May 1, 2007 288

Phoenix, AZ 040139997 October 1, 2009 30

Fresno, CA 060190008 April 1, 2009 80

New York, NY 360050110 May 1, 2007 263

Seattle, WA 530330080 May 1, 2007 254
6

Data Evaluation

Data for each parameter were sorted by site and parameter and the 90th percentile upper bound

coefficient of variation (CV) estimated.  CV is estimated as follows:

Estimate relative percent difference for each collocated pair. The relative percent difference, di, is

calculated by Equation 1.

Equation 1

where Xi is the concentration of the primary sampler and Yi is the concentration value from the

collocated sampler.

The precision upper bound statistic, CVub, is a standard deviation on di with a 90 percent upper

confidence limit (Equation 2)9

.    

Equation 2

  2

0.1, 1

2

1

2

1 1

2 1 _



    





 



  



 

n

n

i

i

n

i

i

n

n n

n d d

CV ub



where, n is the number of valid data pairs being aggregated, and χ2

0.1,n‐1 is the 10th percentile of a

chi‐squared distribution with n‐1 degrees of freedom.  The factor of 2 in the denominator adjusts for the

fact that each di is calculated from two values with error. Since this estimate is currently being

performed for the collocated sites in the PM2.5 network for the primary standard, and EPA is proposing

to use the same statistics in the secondary standard, it made sense to use this statistic to evaluate the

CSN and IMPROVE data.   Tables 5 and 6 provide the variability estimates by site and parameter for CSN

and IMPROVE collocated sites respectively.  

Data evaluations were generated using the Data Assessment Statistical Calculator10 (DASC) software tool

that was developed for monitoring agencies to evaluate data quality using the current CFR statistics.  

The tool can be use to generate the equation 1 and 2 statistics and graphically display the data.  

Appendix A  and B provide the DASC tool results for individual parameters for both CSN and IMPROVE  

respectively.   

In reviewing Tables 5 and 6, one can see there is more imprecision associated with the CSN fine soil

fraction elements compared to fine soil fraction derived from the IMPROVE collocated sampling

modules.  The CSN precision values compare reasonably well to an assessment performed on 2002‐2004

data11.  The measurement quality objectives (MQOs) listed in that assessment were 20% for trace

                                                           

9 Equation 2 is equation 11 in  40 CFR Part 58 Appendix A and is so referenced in the Appendix A graphs 10 http://www.epa.gov/ttn/amtic/qareport.html   11 PM2.5 Speciation Network Newsletter #4 http://www.epa.gov/ttn/amtic/spenews.html

  d

X Y

X Y i

i i

i i

 

  / 2

100
7

elements, 15% for carbon and 10% for sulfate and nitrate. With the exception of the trace elements,  

the overall site averages are reasonably close to the original MQOs.  A portion of the greater imprecision

are a result of using the 90th percentile CV statistic that was used  in this evaluation but not used in the

earlier assessment.  There is some disparity in the  site level precision estimates that may be associated

with sampler or lab errors (no outliers were removed from the evaluations).   

Some of the differences between the CSN and IMPROVE data can also be attributed to smaller sample

volumes and lighter particulate loadings in the CSN (where one or a few larger imprecision values can

have an  affect on  the CV statistic) as well as lower concentration sites, where  a relatively small

difference can cause higher imprecision than sites with higher concentrations. IMPROVE samplers use

higher flow rates along with smaller filters which results in higher particle loadings on the filter (~10

times higher) which can help improve the instrument’s ability to detect the trace elements and resulting

precision estimates. The sulfate, nitrate, organic carbon and elemental carbon precision results are

more comparable between the networks  with sulfate and nitrate having higher, but acceptable,

imprecision at the CSN collocated sites and organic carbon and elemental carbon having a slightly

higher, but acceptable, imprecision at the IMPROVE sites.  

                                                                                                                                                                                               
8

Although this paper is not intended to provide a detailed discussion on the differences in precision

estimates  between the CSN and IMPROVE networks, some assessments related to this data have

already been performed by IMPROVE 12,13 and the EPA CSN contractor laboratory through technical

papers 14 and annual data summaries.  

The parameters evaluated in Tables  5 and 6 are input parameters (along with RH) to the light extinction

equation which produces a visibilty index value in deciviews.  The next step is to determine the

variability in the visibility index  value for collocated CSN/CSN sites (those listed in Table 2) and

collocated CSN /IMPROVE sites.

                                                           

12 Hyslop NP and Warren White; Sources of Uncertainty from the Inter‐species Covariance of Measurement  Errors..

Environmental Science and Technology, 2011  

13  Appendix E of Spatial and Seasonal Patterns and Temporal Variability of Haze and its Constituents in the United States:  

Report IV

November 2006 http://vista.cira.colostate.edu/IMPROVE/Publications/Reports/2006/2006.htm

14 James B. Flanagan, R.K.M. Jayanty, Edward E. Rickman. PM2.5 Speciation Trends Network: Evaluation of Whole‐System

Uncertainties Using Data from Sites with Collocated Samplers, 2006  J. Air & Waste Manage. Assoc. 56:492–499
9

Table 7 provides  site level summary results of the 90th percentile upper bound CV  for the visibilty index  

aggregated at 10th percentile  visibility index  levels. The overall CV results for the collocated CSN sites

and the CSN/IMPROVE  sites are very similar with an average CV of around 5.5%.   

There was an expectation that the CV results between CSN/IMPROVE would be somewhat higher than

the collocated CSN instruments due to difference in samplers,  analytical methods and laboratories, and

the fact that variability in the trace elements were greater in the CSN network.   The CV results of the

Phoenix and Fresno sites are somewhat higher than the other sites but those two sites have the least

amount of data (30 and 80 collocated pairs respectively which result in 3 and 8 pairs respectively at each

10th percentile range) which will have an effect on the 90th percentile upper bound CV precision

estimate.  However, even with the complication of sample size for the Phoenix and Fresno sites, based

on the assessment of precision, it would appear that both networks are within the current 10% precision

goals for the primary standard.  Appendix C  provides the results of the data evaluations for each

collocated CSN site and Appendix D  provides the results of the data evaluations for each CSN/IMPROVE

site.   

Discussion

The secondary PM2.5 NAAQS  is being proposed at a range of 28‐30 deciviews15.  This equates to the data

from the collocated CSN and CSN/IMPROVE sites in the 60th percentile VI range and above (site

dependent).  Within the 60th through 100th percentiles,  the 90th percentile upper bound CVs  range from

a high of  5.52%  to a low of 1.11%.  As the data in Appendix C and D illustrate, when collocated VI

estimates are aggregated at 10th percentiles, the CVs are estimated on around 30 collocated pairs where

a few values can influence the estimate.  As mentioned in the data manipulation section, no outliers

were removed from this data set so individual values may have an influence.   Even with limited data

manipulation (values < 5 deciviews removed) the average CV results from all the collocated sites in the

range of the standard are below 5% CV (see Table 8).   At the proposed NAAQS range a 5% CV would

equate to about  +1.5 deciview variability around the estimated value.

                                                           

15  At the time this paper was written the NAAQS had not been proposed and therefore a range was used.
10

Bias Evaluation

In evaluating the collocated CSN/IMPROVE precision graphs in Appendix D,  a pattern appeared to

emerge.   Most of the relative percent differences in the graphs were above zero indicating a bias

between the two networks VI estimates.  In order to asses this further, EPA ran the current PM2.5 bias

statistics on the collocated VI values.  The bias statistic assumes one value is an audit value (truth) of

known concentration.  In this case,  we do not have a known value but as long as we identify a network

as the “audit value” and we are consistent in applying the statistic, we can determine if there is a

relative bias between the networks.  

The curent bias statistic simply estimates the percent difference between the two collocated values as

follows:  

݀݅ ൌ

ݐ݀݅ݑܽ െ ݁݀ݎݑݏ݉݁ܽ

100 כ ݐ݀݅ݑܽ

where, measured  is the concentration indicated by the CSN light extinction values and audit is the

IMPROVE light extinction values.  We then take the average of the percent differences to estimate

average bias for a site.   Appendix D also calculates confidence limits for these average bias estimates

using equations 13 and 14 in 40 CFR Part 58 Appendix A:

Where, t0.95,dfis the 95th quantile of a t‐distribution with degrees of freedom df = nj−1 and s is an

estimate of the variability of the average bias calculated using equation 15 in 40 CFR Part 58 Appendix A:
11

Table 9 provides the bias estimates for the collocated

CSN/IMPROVE sites.  The CSN VI values are usually higher

compared to the IMPROVE values but as Figure 1

indicates, the overall average of the 915 collocated

CSN/IMPROVE  pairs greater than 5 deciviews is 1.5%.

Although we do not currently use the absolute value bias

statistic that is in use for the gaseous pollutants,16 we ran

the CSN/IMPROVE data through that statistic.  This

statistic assigns a positive or negative bias if both the 25

and 75 percentile values (when the bias values are rank ordered lowest to highest) have the same sign.  

With the exception of the Phoenix site (only 30 pairs), the other four  CSN/IMPROVE sites had the 25th

and 75th percentile bias value as positive,  indicating the CSN VI estimate as biased high compared to

IMPROVE.   

There may be a few technical issues that can

account for some of the bias. The IMPROVE

program reports OC that are blank corrected.  EC

is not adjusted.  CSN is recommending an

approach to blank correction in the proposed

secondary standard but it was not implemented

for this evaluation.  Therefore, CSN values for the

OC fraction will be somewhat higher.  The

magnitude of this affect are unknown at this time

but will be evaluated.

The PM2.5 DQO goals for bias for the primary

standard is currently 10%.  Although we do not

have a performance evaluation program in the

CSN or IMPROVE networks the evaluation seems

to suggest that the networks are producing  VI

estimates within 2% of each other.  

Conclusion

It appears that with the current quality systems being implemented in both the CSN and IMPROVE

programs,  we can expect that measurement uncertainty of the VI measurement can be controlled to

the current PM2.5 precision data quality objective of 10% .   As stated earlier, CVs around the  NAAQS

value are in the range of 5%.  In addition, although we do not have a performance evaluation program in

place, both networks are producing comparable VI results with the CSN network producing slightly

higher VI values.

The CSN network is in the process of reviewing its QAPP and will look to determine if the current  

measurement quality objectives of  the VI parameters  need to be revised.  Although the CSN and

                                                           

16 Guideline on the Meaning and Use of Precision and Bias Data required by 40 CFR Part 58 Appendix A

http://www.epa.gov/ttn/amtic/qareport.html  

Table 9.  Visibility Index Bias Estimates   

    for Collocated CSN/ IMPROVE Sites

Site Average Bias (%)

Birmingham, AL 2.1

Phoenix, AZ 6.1

Fresno, CA 0.54

NY, NY 3.9

Seattle, WA 3.7

Average‐All sites 1.5
12

IMPROVE participate in performance evaluation programs that can determine laboratory comparability,

neither have a performance evaluation program that will allow for an estimate of overall bias from field

sampling and laboratory measurement phases.  Ensuring that collocated CSN/IMPROVE samplers

continue to operate at a reasonable number of sites will help to evaluate data comparability.   

Decision makers need to determine whether this measurement uncertainty is acceptable or not.  If not,

further evaluation would need to occur to determine what error(s) play the largest roles in the VI

estimate and attempt to look at ways of reducing and controlling these uncertainties.  If the

measurement uncertainty is  acceptable, then  the current IMPROVE and CSN quality systems  are

providing adequate data quality for the decisions makers estimates around the proposed NAAQS.

   
Appendix A

Graphs of Relative Percent Difference of Collocated CSN Data by Parameter and Site  

Data set from 2007 – 2010 where values from the routine and collocated samplers were

greater than the average MDL. Data was assessed using the Data Assessment Statistical

Calculator (DASC).  Some statistical summations are used in the derivation of the overall

coefficient of variation and are an artifact of the output.
   
Collocated CSN Precision Estimates for Data Greater than the MDL‐ Parameter Aluminum

  
Collocated CSN Precision Estimates for Data Greater than the MDL  – Calcium
Collocated CSN Precision Estimates for Data Greater than the MDL  – Iron

   
Collocated CSN Precision Estimates for Data Greater than the MDL ‐ Titanium

  
Collocated CSN Precision Estimates for Data Greater than the MDL ‐Silicon

  
Collocated CSN Precision Estimates for Data Greater than the MDL ‐ Nitrate

     

  
Collocated CSN Precision Estimates for Data Greater than the MDL ‐ Sulfate

  
Collocated CSN Precision Estimates for Data Greater than the MDL ‐ Organic Carbon

   

  
Collocated CSN Precision Estimates for Data Greater than the MDL ‐ Elemental Carbon

    

    

  

Appendix B

Graphs of Relative Percent Difference of Collocated IMPROVE Modules by

Parameter and Site  

Data set from 2006 – 2010 where values from the collocated sampler modules

were greater than or equal to the average MDL. Data was assessed using the Data

Assessment Statistical Calculator (DASC).  Some statistical summations found

above the graphs are used in the derivation of the overall coefficient of variation

and are an artifact of the DASC output.
   
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Aluminum
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Calcium

  

   
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Iron
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Titanium

  
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Silicon

  
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Nitrate

  
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Sulfate

  

  
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Organic

Carbon

  
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Organic

Carbon (Continued)
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Elemental

Carbon
Collocated IMPROVE Precision Estimates for Data Greater than the MDL‐ Parameter Elemental

Carbon (continued)

   

Appendix C

Relative Percent Difference and upper 90th Percentile Coefficient of Variation Estimates of

Collocated Visibility Index (in Deciviews) from Sites with Two CSN Samplers

   
   
  

  

  
Measurement Uncertainty Evaluations at 10th Percentile Concentrations

   
  
  
Appendix D

Relative Percent Difference and Upper 90th Percentile Coefficient of Variation Estimates of

Visibility Index (in Deciviews) from Sites Collocated with an IMPROVE Sampler and a CSN

Sampler

and  

Bias Assessment of CSN/IMPROVE Sites Using Current PM2.5 Bias Statistic

CSN visibility index values used in the following assessments include data after conversion to URG samplers.  

With fewer data points, collocated pairs with large imprecision or bias estimates have more effect on the

Phoenix and Fresno statistics especially when segregating data at the 10 percentile VI ranges.

   
   
  
Measurement Uncertainty Evaluations at 10th Percentile VI  values

  
  

Bias Assessment of CSN/IMPROVE Sites Using Current PM2.5 Bias Statistic

Bias Estimate = D (green highlight)