NISTIR 5951 

Guide for Quality Control on the 

Qualitative and Quantitative Analysis 

of Bulk Asbestos Samples: 

Version 1 

Jennifer R. Verkouteren 

David L Duewer* 

U.S. DEPARTMENT OF COMMERCE 

Technology Administration 

National Institute of Standards 

and Technology 

Surface and Microanalysis Science Division 

Chemical Science and Technology Laboratory 

Gaithersburg, MD 20899-0001 

*Analytical Chemistry Division 

March 1997 

U.S. DEPARTMENT OF COMMERCE 

William M. Daley, Secretary 

TECHNOLOGY ADMINISTRATION 

Mary L. Good, Under Secretary for Technology 

NATIONAL INSTITUTE OF STANDARDS 

AND TECHNOLOGY 

Arati Prabhakar, Director 
1 

PURPOSE 

This guide provides a set of approaches for dealing with quality control (QC) data in the bulk 

asbestos analysis laboratory. This guide was developed in response to numerous requests for 

detailed information on quality control :from the laboratories involved in the National Voluntary 

Laboratory Accreditation Program (NVLAP) for bulk asbestos. · 

SCOPE 

The guide outlines procedures for characterizing within analyst and within laboratory accuracy 

for the analysis ofbulk asbestos samples. Methods are given for generating control charts using 

statistics calculated :from control data. Real laboratory QC data provided by an anonymous 

NVLAP laboratory is presented and analyzed. 

STATISTICAL APPROACH 

A common approach to collecting QC data in the analytical laboratory is to have a small set of 

samples appropriate for the type ofmeasurements being performed that are analyzed repeatedly. 

Control charts are used to track analytical response with time for this non-changing set of 

samples. A real hazard with applying this approach to a measurement in which humans are the 

detectors is the potential for bias due to familiarity. Any QC system for bulk asbestos analysis 

must be sensitive to the needs for blind analysis ofQC samples, and for limiting the number of 

times a particular sample is used. To avoid problems with sample familiarity, we recommend 

using the samples analyzed by the laboratory in the normal course ofbusiness as the QC samples. 

These are ''unknowns" in that the amount and type ofasbestos are not known by some 

independent means. 

The unknowns are used to determine within analyst accuracy and within laboratory accuracy. 

Within analyst accuracy is a measure ofan individual analyst's ability to repeat an analysis, and 

within laboratory accuracy is a measure ofthe laboratory's ability to repeat an analysis, given 

different analysts using different methodologies and equipment. The terms are italicized because 

we mean them as very specific terms which will be defined for this document but not meant to be 

used in any general sense. We use the word accuracy in both cases since it combines errors that 

are both random and systematic in nature, and is therefore a more general term than precision, 

bias, repeatability, reproducibility, or any number ofterms that are used for QC measurements. 

We would have preferred to use the term precision, rather than accuracy, for the two QC 

activities described as there is no knowledge ofthe "true" value ofthe asbestos concentration for 

eaqh sample. However, precision is a well-defined term that describes random measurement 

errors. Even in the case ofthe within analyst accuracy, which would seem to be a measurement 

ofprecision, we saw evidence ofa systematic error, or bias, and therefore could not use the tenn 

precision for this activity. We want to emphasize, however, that the terms within analyst 
2 

accuracy and within laboratory accuracy, as defined in this document, describe QC that is 

internal to the laboratory, and do not say anything about the laboratories performance 

with respect to other laboratories or with respect to the "true" value. There is a discussion 

ofthe analysis of reference samples which will give the laboratory some measure ofits 

ability to obtain the "true" value. 

The QC system is designed to emphasize control near the critical regulatory value of I% asbestos1 

by analyzing the data with respect to three concentration ranges: 1) trace to less than 1 % 

asbestos, 2) 1% to 10% asbestos, and 3) greater than 10% asbestos. Medians, interquartile 

ranges (IQRs) encompassing the central 50% ofthe data, and control limits that approximate 2 

times the standard deviation (SD) are calculated for each concentration range. Medians and IQRs 

are used because they are less sensitive to outliers than means and standard deviations, and 

therefore are more robust estimators. 

DATA MANAGEMENT 

Although data management can be done manually, laboratories with computers can choose to 

enter and manipulate data using commercially available or internally generated spreadsheets or 

databases. The calculations performed on the data are common to commercially available 

spreadsheets and can also be performed on hand calculators. Graphical representation ofthe data 

is highly recommended, and can be done easily with common spreadsheets or by hand. All the 

data entry and statistical analysis shown in this guide were performed in Microsoft Excel* and 

specific information regarding this spreadsheet program is given where applicable. 

DATA COLLECTION AND QUALITATIVE ANALYSIS 

The NVLAP Bulk Asbestos Handbook.2 stipulates that a minimum of 10% ofthe analyses 

performed by the laboratory be QC analyses. The breakdown ofthe 10% into the various types of 

analyses and the timing ofthe QC data collection is left to the discretion ofthe laboratory. The 

types ofanalyses described in this guide are: 

1) within analyst accuracy : an analyst receives a sample twice and performs each analysis 

independently. This tests the ability ofan analyst to repeat a measurement. 

2) within laboratory accuracy: two or more analysts receive the same sample and conduct the 

analyses independently. This tests differences among analysts. 

•certain commercial equipment, instruments, or materials are identified to specify 

adequately the procedure. Such identification does not imply recommendation or endorsement by 

the National Institute of Standards and Technology, nor does it imply that the materials or 

equipment identified are necessarily the best available for the purpose. 
3 

3) reference sample analyses: a material with a known concentration ofasbestos is submitted for 

analysis. This tests the ability ofthe analyst/laboratory to obtain the "true" value. 

A laboratory may choose to have all their analysts do I 0% QC for the number ofsamples they 

analyze although beginning analysts should perform I 00% QC until their proficiency is determined 

to be within acceptable limits. The relative amounts ofthe three types ofQC analyses are up to 

the laboratory, and may change with time as needed. 

A completely random selection ofQC samples is probably not the best approach, given that the 

concentration range near I% asbestos is the most critical for QC, and those samples may be 

relatively uncommon. We recommend selecting all ofthe samples whose first analysis is between 

trace and 5% asbestos for inclusion in the QC samples. Samples that are particularly difficult, 

such as floor tiles, should also be preferentially included in the QC samples. The remaining 

samples can be selected randomly. The reference materials should cover the concentration range 

ofthe samples received for analysis, and should also emphasize the concenJ;ation range near 1% 

asbestos. The proficiency testing materials received by labs in the bulk NVLAP and New York 

State ELAP programs are appropriate as reference materials and cover a useful range of 

concentrations. Reference samples containing close to I% asbestos are relatively uncommon and 

may need to be synthesized3

• 

The data entered into the QC records from the laboratory analysis record includes: I) sample 

identification number, 2) date, 3) analyst, 4) type(s) of asbestos, and 5) concentration(s) of 

asbestos. Additional data not included in the following examples could be matrix type and other 

sample information to characterize errors with respect to sample. This information should be in 

the laboratory records to be referred to when needed. Ifentering data in a database, fixed names 

should be adopted for the asbestos types and analysts in order to aid search and sort functions. 

An example ofdata entry is given below: 

Sample Date 1 Analyst 1 Type 1 Conc1 Date2 Analyst 2 Type 2 Cone 2 ACM 1 ACM 2 Flag Type Flag ACM 

24 115194 Jill CHRY 7 1/7/94 Harry CHRY 15 Y Y O 0 

36 1/11/94 David NONE 0 1/12/94 David CHRY 3 N Y F F 

40 1/22/94 Harry AMOS 13 1/23194 David AMOS 5 Y Y O 0 

Combining the two analyses for each sample can be done at the time ofdata entry, or afterwards 

by sorting and merging according to sample number. To handle samples with multiple asbestos 

types a field ( column) can be created that indicates the number of asbestos types in the sample. 

The information for each asbestos type can be entered as a separate row with the sample number 

as the common factor. When the data are sorted by sample number, the analyses will be grouped 

together, and if a 2 has been entered into the field for multiple asbestos types, there should be 2 

rows ofinformation per sample per analysis. [In the example above, samples 24 and 40 are tests 

ofwithin laboratory accuracy and sample 36 is a test ofwithin analyst accuracy.] 

A column is added for each analysis (ACM I and ACM 2) to characterize the sample as asbestoscontaining based on the concentration reported. A logical statement can be used to determine 

whether the concentration is I% or more, in which case a Y for yes is placed in the column, 
4 

otherwise N for no is entered. [In Excel, IF column 5 >=I,Y,N.] Two error flags are detennined 

for the QC analyses at this point: 1) whether both analyses have the same asbestos type (s), and 2) 

whether both analyses agree on asbestos-containing or not. The columns are identified above as 

Flag Type, and Flag ACM, with an F indicating an error. [In Excel, IF column 4=column 8, 0, F.] 

The entries can then be sorted by error flags, to pull out all entries that are flagged. Errors in 

detennining whether the material is asbestos-containing, or the type ofasbestos, are qualitative 

errors, and should be considered as very serious. Every analysis that gets a qualitative flag 

should be reviewed to determine the source of the problem. Real data from the volunteer 

laboratory was analyzed in this manner, and some ofthe qualitative errors are shown in Table 1. 

Unfortunately, for these samples, the type ofasbestos was not recorded and therefore only the 

ACM flag was determined. 

i····~·~·~·P..~~.... ········~~!~.......J~.~~.!¥.~.!.....l.~.~.~.~¥..~.!..~.J.~.=·~·~·~!...t.i.~~.~.~!!..~..L~.9..~...t.........l~.9..~...?...........! 

l..-~~.t?.1.~.~... ....~!.J..~!.~1~....i............t .......... ............. .............1.........~~..........!........... ...........!.............'!............·.L......... .............J 

i··::~~~~~·+··!~;~~~:··..·I···········~........... ·············:···········l ····:···1 

2

5·········l·····....~o .........1............ ..:........+•-•m•..·~····..···..··l 

I

6430217 6/6/94 ! 5 4 1.5 j O 1 Y i N ! ...6446608 · .... 1211194....r········· 4...........1············ 3············· ········a. 1·········r··· o. 1 .... v ............r.......... N............1 .....................................................................................~·--············..•···• ....................................·----····--····...........·.............................. 6447423 12/21/94 l 7 i 3 0 i 8.5 i N i Y 1 ............................. ••••••••••••..•••••••........❖ .........................t ..................................................... ❖••····· ................ ' ·····························=···········..················< 

6447424 12/21/94 i 7 1 3 0 i 6.2 j N i Y l 

6447425 12/21/94 l 7 I: 3 o~·--r 12.2 l N i Y l r6447561··· .. 12/22/94···t--·······7........... ·······...... 3............. ·········..o···········t--··2 5. g·····--(.......... N............f...........y ··········..·1 ...........- ................•--••••••••<• tt•4•H••••••u .....................,,._.,.,.,.,.._... ,..., ..u--.~•nuoo ••Ht••••..h .. u+••..••..•••............,.uuou~nu~u•••uouu...u ................................Huou~nH<'n•••9 • 

.. :::~ :;~.....~ ~~;~~::·+..········;...........,··········..;....................~: ~ .......--l-·······!:!--··..+··········· ~ ............t·······.... ~ .............: 

6448683 12/28/94 l 4 i 3 ! o i 47.1 I N i Y i 

:::~:~~~t _2;~~:9~ ··-!:::::::~:::::::::!:::::::::::~:::=±::::~: L. o~1:::±=:: =:::::::!=::::::~:: J 

Table 1. Qualitative errors from laboratory QC data. 

It will generally be impossible to assign an error to an individual analyst for any ofthe samples 

until some way ofdetermining the correct answer is found. Some ofthe differences in the results 

may be due to measurement uncertainty (e.g., small particle numbers in the point counts) for 

samples containing close to 1% asbestos, which may be the case for samples 6417848, 6430217, 

and 6447572. The samples analyzed between 12/21 and 12/28 all have the same analyst pair (7 

and 3) and it is possible that the same error occurred for all those samples. All the samples that 

have qualitative errors such as those shown above must be reviewed by having additional analysts 

look at them or by using different analytical techniques such as TEM, XRD, or gravimetry, 

depending on the nature ofthe discrepancy. Once the nature ofthe problem is determined, 

corrective action must be taken. 
5 

WITHIN ANALYST ACCURACY 

Within analyst accuracy is determined by having an analyst perform 2 independent measurements 

on a number ofsamples. The first step in managing the data is to form separate sets ofdata for 

each analyst. The data are then analyzed for qualitative errors as described earlier, and any 

flagged samples are segregated as are samples that do not contain asbestos. [ Although it is 

important to include blank samples in the QC analyses, they are not included in the determination 

ofaccuracy.] The data for each analyst can be graphed at this point by plotting the second result 

versus the first result, as shown in Figure 1 for the real data from the volunteer laboratory. This 

type ofgraph, called a Youden plot4

, is very useful because it is easy to interpret and provides 

qualitative information on the overall scatter in the data and any consistent repetition errors. The 

line drawn on each graph is the y=x line, which is the line that represents exact repetition. 

One observation from the graphs in Figure 1 is that analyst 4 is an anomaly i.n having virtually no 

repetition error. This data set should be rejected as an indicator ofwithin qnalyst accuracy 

because it is unrealistic. Analyst 4 is probably not receiving these QC samples in a blind fashion. 

A second observation from Figure 1 is the consistent repetition error expressed by analyst S's and 

2's data where the data do not scatter uniformly about the y=x line. For both analysts, the second 

result is systematically higher than the first result, particularly at higher asbestos concentrations. 

This type ofbias is expected for measurements involving two different analysts, but it is peculiar 

for data from a single analyst. We can speculate that it may be due to fatigue ( all the second 

analyses are conducted in the afternoon) or perhaps to bias during data entry. This should be 

investigated to discover the cause and eliminate the problem. 

The next step in analyzing the data is to calculate the difference between the two results for each 

sample by subtracting result 2 from result 1 (rl•r2). The average ofthe two results (rl +r2)/2 is 

also calculated, and both calculations are shown in Table 2. The data in Table 2 have been sorted 

to separate the analyses with qualitative errors, as determined by an F in the "Flag ACM" column. 

Samples with qualitative errors are not included because they represent a large deviation from 

analytical control. An exception to this would be a sample in which the true amount ofasbestos is 

near 1 % and one result is slightly less than 1 % and the other result is slightly more than 1 %. An 

analysis ofthis type may be indicative oftrue repetition error, whereas a result of26% versus a 

result of0 reflects some gross error. 

An additional column shown in Table 2 is labeled "category" and represents the separation ofthe 

data into the three concentration categories referred to earlier. A value of 1 bas been entered for 

all analyses in which the average ofthe two results ((rl+r2)/2) is less than 1%, a value of2 has 

been entered for (rl+r2)/2 between 1% and 100/o, and a value of3 has been entered for (rl+r2)/2 

greater than 100/o. [In Excel, IF(column6<1,l,IF(column6>10,3,2)).] All results that contain zeros 

in qoth the first result and second result columns have been segregated as blanks. It is not 

absolutely necessary to create the "category'' column, but it helps in sorting and analyzing the 

data. 
• •• • • • • 

• • • • • • 

• • 

•• 

Analyst 6 5 Analyst4 

100 

80 

N 60 •

:t::! 

• :II 

Ill 40 .. :• ! • 

: •• 

20 

• 

0 

100 

80 

N 60 

~ Ill 

! 40 

20 

0 

0 20 40 60 80 100 0 20 40 60 80 100 

result 1 

Analyst 6 

100 

•

80 

• 

N 60 

:I::! 

• ::, 

Ill 

! 40 

20 

0 

result 1 

~alyst2 

100 

•

80 

N 60 • 

:t::! 

:II 

! 

Ill 40 .. • 

20 

0 

0 20 40 60 80 100 

0 20 40 60 80 100 

result 1 

result 1 

Figure I. Within analyst accuracy data for 4 analysts. The results are in units of% concentration 

ofasbestos. 

NOTE: Samples can contain a trace amount ofasbestos, defined as asbestos present in the sample 

but not at a concentration high enough to be counted in a 400-point point count. A numeric value 

must be assigned for trace concentrations for entry in a spreadsheet, and O. I is commonly used for 

this purpose. 
7 

anal}'.st samele date result 1 result 2 {r1+r2}g ACM 1 ACM 2 fla9 ACM r1-r2 cateQO!)'. rel. diff. 

5 6445257 11121/94 5 0 y N F 

5 6504790 3/1/95 14 0 y N F 

5 6508212 4127/95 28 0 y N F 

5 6512992 7/18/95 2 0 y N F 

5 6512232 7/17/95 0.1 0.1 0.1 N N 0 0 1 0 

5 6506131 3/31/95 0.1 0.1 0.1 N N 0 0 1 0 

5 6512588 6/21/95 0.1 0.1 0.1 N N 0 0 1 0 

5 6512127 6/16/95 0.3 0.1 0.2 N N 0 0.2 1 100 

5 65131TT 7/5/95 0.3 0.1 0.2 N N 0 0.2 1 100 

5 6513288 7124/95 0.3 0.1 0.2 N N 0 0.2 1 100 

5 6511434 6/28/95 0.5 0.1 0.3 N N 0 0.4 1 133.33 

5 6443432 11/9/94 0.6 0.1 0.35 N N 0 0.5 1 142.86 

5 6503589 2/14/95 5 10 7.5 y y 0 -5 2 -66.667 

5 6503601 2/14/95 2 5 3.5 y y 0 -3 2 -85.714 

5 6447942 12/19/94 4 5 4.5 y y 0 . -1 2 -22.222

5 6519696 8/29/95 2.1 3 2.55 y y 0 -0.9 2 -35.294

y y 5 6500493 1/9/95 2 2 2 0 0 2 0 

y y 5 6503611 2/14/95 4 2 3 0 2 2 66.667 

§ §50~§Q 2l:1§li§ jQ 5 7.5 y 'i.. Q 5 2 66.§§7 

5 6441798 9/26/94 5 40 22.5 y y 0 -35 3 -155.56 

5 6449321 1/3/95 40 60 50 y y 0 -20 3 -40 

5 6508021 4125/95 46 65 55.5 y y 0 -19 3 -34.234 

5 6503678 2/14/95 24 40 32 y y 0 -16 3 -50 

y 5 6507625 4/17/95 39 50 44.5 y 0 -11 3 -24.719 

y 5 6504028 2123/95 31 40 35.5 y 0 -9 3 -25.352 

5 6505830 3/20/95 17 25 21 y y 0 -8 3 -38.095 

y y 5 6503855 2/15/95 28 35 31.5 0 -7 3 -22.222 

5 6431034 6/10/94 38 45 41.5 y y 0 -7 3 -16.867 

5 6448615 12127/94 43 50 46.5 y y 0 -7 3 -15.054 

5 6447864 12/19/94 29 35 32 y y 0 -6 3 -18.75 

5 6443740 11/9/94 15 20 17.5 y y 0 -5 3 -28.571 

5 6503608 2/14/95 15 20 17.5 y y 0 -5 3 -28.571 

26 30 28 y y 0 -4 3 -14.286 5 6506336 3124/95 

33 35 34 y y 0 -2 3 -5.8824 5 6507121 4/17/95 

5 6506422 3127/95 31 31 31 y y 0 0 3 0 

5 6512484 6/19/95 60 60 60 y y 0 0 3 0 

5 6506383 3/24/95 11 10 10.5 y y 0 1 3 9.5238 

y 5 6444178 11/4/94 21 15 18 y 0 6 3 33.333 

5 6429212 612/94 36 30 33 y y 0 6 3 18.182 

Statisti£§ for r 1-r2 

Concentration category 2•: Median= -0.9%, IQR = -2% to 1% or 3%, Control limits= ±4.4% 

Concentration category 3: Median= -6.5%, IQR = -9.5% to -1.5% or 8%, Control limits ±11.9% •Only 7 values are available in this concentration category, which is below the recommended minimum of 8 as discussed later in this 

document 

Table 2. Within analyst accuracy data for analyst 5. 
• • 

•• 

8 

Analyst 5 

10 

5 ____________,_ 

.........................•........................... 

0.1 • 1u ······•····•·····················roo

-5 ---------- .- . ---.__.,............--- -10 ························•··· _________...······-···-········· _____ -15 • 

-20 

-25 

-30 

-35 • 

(r1+r2)/2 

Figure 2. Within analyst accuracy data for analyst 5 showing calculated medians (solid lines), 

IQRs (dotted lines) and control limits (dashed lines) for concentration categories 2 and 3. 

The data in Table 2 are shown graphically in Figure 2 by plotting rl-r2 versus (rl+r2)/2. The xaxis is plotted using a logarithmic scale to emphasize the concentration categories. The fact that 

most ofthe points plot below the zero-difference line reflects the bias discussed earlier in which 

the second result is consistently higher than the first. The data should scatter uniformly above and 

below zero. Also shown on the graph are the calculated medians for concentration categories 2 

and 3, interquartile ranges (IQRs) that bound the central 50% ofthe data, and control limits. The 

calculation ofeach statistic is discussed below. These statistics were not calculated for 

concentration category 1, as this is taken to be simply a "trace concentration" category. 

Median 

The median is the number in the middle ofa set ofnumbers; halfofthe numbers in the set have a 

value greater than the median and halfofthe numbers have a value less than the median. The 

median ofthe differences between the first and second results is used to estimate the bias. There 

is abias ifthe median is large in absolute value. 

We recommend using the median rather than the mean because it is less sensitive to outliers and 
9 

therefore eliminates the necessity ofidentifying and discarding outliers prior to calculating 

statistics from the data. (The median is equivalent to the mean for symmetrically distributed 

data.) For example: there appears to be an obvious outlier in rl-r2 of-35% for concentration 

category 3 (see Table 2). Most people looking at Figure 2, or the same data displayed differently 

in Figure 1, would identify that particular point as an outlier. The median. ofrl-r2 in 

concentration category 3 is -6.5% but the mean (or average) is -7.4%, due to the presence ofthe 

outlier. Although some outlier points are easily discriminated by eye, others are not, and to avoid 

rigorous tests for outliers we recommend using the median. The median is a standard calculation 

in spreadsheets [in Excel, =MEDIAN(value 1..value n)] and it can be determined by sorting the 

data and finding the value at the midpoint. The data in Table 2 were first sorted by concentration 

category, and then by difference within each category. There are 20 difference values in 

concentration category 3, and the median can be found at the midpoint between the 10th and 11th 

value. [See Appendix 1 for manual procedures for calculating medians.] 

Interquartile range (IQR) . 

The IQR encompasses the central 50% ofthe data and gives an estimate ofthe spread or scatter 

ofthe data. We recommend the use ofthe IQR as opposed to the standard deviation (SD) 

because, like the median, it is relatively insensitive to outliers. The calculation ofquartiles is a 

standard feature ofspreadsheets and can also be approximated by sorting the data and dividing it 

into 4 segments. For example, there are 20 entries in concentration category 3 as given in Table 

2 that are sorted according to rl-r2. The data separate into 4 segments of 5 entries each, and to 

get the central 50%, the top five values and the bottom five values are discarded. The first 

quartile falls between -11% and -9%, and is calculated to be -9.5%. The third quartile falls 

between -2% and 0%, and is calculated to be -1.5%. [See Appendix 1 for manual calculation 

procedures.] The IQR is the absolute difference ofthe first quartile and the third quartile. The 

IQR ofrl-r2 for concentration category 3 is therefore -9.5% to -1.5% or 8%. [In Excel: 

=QUARTILE(value 1..value n, 1) and =QUARTILE(value 1..value n, 3).] 

Control limits 

Control limits are calculated from the IQR to approximate a 2xSD limit and serve to identify 

points as in or out of control. The IQR is related to the SD through the relation 

0.741xIQR = SD for normally distributed data5

• Again, we recommend using the IQR rather than 

the SD because ofthe relative insensitivity ofthe IQR to outliers. To calculate the control limit 

for concentration range 3, the IQR (absolute value of-9.5% minus -1.5% = 8%) is multiplied by 

0.741 to get the SD, which is 5.93%, and then by 2 to get a 2xSD limit of:±:11.9%. By 

comparison, calculating the SD directly for this concentration range yields a 2xSD limit of 

:±:19.1%, which is larger due to the outlier. (Removing the outlier point and calculating the SD 

directly yields a 2xSD limit of:±:14.4%.) The control limits of :±:11.9% are placed about zero, not 

about a possibly biased median. 

Discussion 

A problem with using rl-r2 as an indicator ofmeasurement performance is that the absolute 

difference should not be independent of concentration, i.e. low concentration samples should have 
10 

much smaller values ofrl-r2 than high concentration samples. For example, an rl-r2 of 5% for a 

sample containing 50% asbestos is not as significant as for a sample containing 5% asbestos. 

There are at least 2 ways to address this problem: !)calculate measurement performance (i.e. 

medians, IQRs, control limits) as a function ofconcentration, or 2)reference the difference to the 

average value ofthe concentration to determine a relative difference. We have chosen to use the 

first approach and have selected the concentration categories to treat Iow concentration samples 

separately from the high concentration samples. We saw no significance in further segregating the 

samples containing greater than I0% asbestos, since the primary goal is to insure that samples 

containing close to 1 % asbestos are closely controlled. 

Analysts 

200 

•• 150 

• 100 

• 50 8

I

I 0.1 

i -50 

-100 

-150 

-200 

----- • ---- • ---------------

• ······················---··························································· 

·········-·····-·· • ·················10.......... ........ •• ....100 

• -----------·· --------------- • 

• 

(r1+r2)12 

Figure 3. Within analyst accuracy for analyst 5 using relative differences (the difference divided 

by the average and multiplied by 100) rather than differences. Medians (solid lines), IQRs (dotted 

lines) and control limits (dashed lines) are calculated for the entire data set, with the exception of 

values from concentration category I. 

We also tested the usefulness ofusing relative differences, but found that with the data set 

available to us, calculating relative differences alone was insufficient to achieve the goal of 

appropriately characterizing measurement performance. (Using relative differences in conjunction 

with concentration categories is perfectly acceptable.) Relative differences were calculated by 

dividing rl-r2 by (rl+r2)/2 and multiplying by 100, as shown in Table 2. The data were then 

graphed as shown in Figure 3 and medians, IQRs and control limits were calculated for the entire 
11 

set ofdata, without respect to concentration category ( excluding data from concentration 

category 1 ). As expected, the relative differences decrease with concentration, just as the 

differences (r 1-r2) increase with concentration. Therefore. control limits calculated for the entire 

set ofdata are probably too restrictive for the low concentration samples (samples with results of 

2% and 4%, and 2% and 5% are out ofcontrol) and too generous for the high concentration 

samples (anything within :!::60% relative is in control, which is :!::300/o asbestos for a sample 

containing 50% asbestos). We simply do not have enough data at this point to say which is the 

best approach, or even what are acceptable performance levels. The laboratories using this 

document will need to decide which approach best demonstrates acceptable results for low 

concentration samples for their data. 

W/111/NLABORATORY ACCURACY 

. Within laboratory accuracy is determined by performing multiple measure(Ilents ofthe same 

sample using different analysts. The scatter in the data for within laboratory accuracy should be 

larger than the scatter in the data for within analyst accuracy, as there may be systematic 

differences among analysts. Within laboratory accuracy data can be collected in two ways: 1) by 

randomly forming pairs ofanalysts to produce 2 analyses per sample, or 2) by having all analysts 

measure a set ofsamples in random order. The first approach allows for a larger number ofQC 

samples to be analyzed and provides an estimate ofthe scatter in the data expected for the entire 

laboratory. whereas the second approach is a systematic way oftesting for relative differences 

among analysts. Ifit is a two-person laboratory, the two approaches are, ofcourse, the same. 

The volunteer laboratory submitted data using the first approach with 2 analyses per sample by 2 

different analysts. Their data contains 28 different analyst pairs. The difference (rl-r2) between 

the two results for each sample was determined, and plotted with respect to (rl+r2)/2 as shown in 

Figure 4. The medians, IQRs, and control limits were calculated for rl-r2 for the two 

concentration categories, and are: 

1%-10%: Median= -1%, IQR =-3% to 1.3% or4.3%, Control limit =:!::6.4% 

>10%: Median= -1%, IQR = -100/o·to 7.5% or 17.5%, Control limit= :!::25.9% 

The within laboratory accuracy data show more scatter than the within analyst accuracy data for 

both concentration regions with a 400/o increase for the 1%-100/o IQR and more than a doubling of 

the > 10% IQR. The medians for both concentration categories are close to zero, implying either 

random variation among the analysts, or that there are just as many analysts that report high 

values as report low values. [It is important when collecting data in this fashion that the analyst 

pairs are randomly selected and that the position ofeach analyst in the pair is also random to 

prevent an apparent bias due to weighting ofone analyst's data.] To determine whether the 

variation among the analysts is random or systematic, the second approach where 
••• 

• • 

12 

60 

50 

40 

30 

20 

\'.j! 10 

....

.. 0 

-10 

-20 

-30 

-40 

-50 

• 

----------+---....-----

•• ~ ~·=·=·=·=·~=·--~.=·=·=·~·=·!·=~····.···················· 1 ·· .,• ••............... 

........... . ··········~·•··· ... --------------- ~~• ........• #"·••• 1.·•·················100 

______ ..,,, ---l -- • -­ • .. 

(r1+r2)/2 

Figure 4. Within laboratory accuracy data in which each point represents the difference in results 

between 2 analysts for a given sample. Medians (solid lines), IQRs (dotted lines) and control 

limits (dashed lines) are shown for concentration categories 2 and 3. 

all analysts measure the same set of samples should be used. The volunteer laboratory did not 

submit data ofthis type, and so an example set ofdata was created, containing results for 4 

samples from 4 analysts. The results for each analyst were plotted against the average for each 

sample, as shown in Figure S. It is easy to see from this graph that analyst 2 is consistently lower 

than the other analysts. There are more rigorous ways to investigate data ofthis type, but we 

suggest using this approach in a qualitative sense to investigate gross differences among analysts 

and try to correct large systematic differences. This type ofapproach does not indicate which 

analyst is correct, only the relative differences among analysts. 
• • 

13 

- 30 ~ 

X 

25 II 

.e• 

J 20 

X

•

• 15 

d 

C X

u 

0 10 •

';fl. A •

5 • 

. 4 

0 

111 Analyst 1 

1J,,Analyst2 

• Analyst3 

xAnalyst4 

0 5 10 15 20 25 

Average, %cone. asbestos 

Figure 5. An example ofwithin laboratory accuracy data to test for systematic differences 

among analysts. The results for each of4 analysts for 4 samples are plotted with respect to the 

average for each sample. 

REFERENCE ANALYSES 

Reference materials are samples for which the concentration ofasbestos is known, either through 

an independent, accurate, technique such as XRD or gravimetry, or because the materials are 

synthesized to contain known concentrations ofasbestos. Analysis ofreference materials is the 

only way to determine the ability ofan analyst to obtain the "true" value. The difference between 

the analyst's result and the reference value is determined, but it is called an error, not a difference, 

because the true value is known. The data are handled in exactly the same way as described for 

within analyst and within laboratory accuracy by separating the blanks and the samples with 

qualitative errors, and then determining medians, IQRs, and control limits on the errors according 

to concentration category.· 

The reference data for one ofthe analysts from the volunteer laboratory are shown in Table 3 and 

in Figure 6. The "error'' column in Table 3 was determined by subtracting the reference value 

from the analyst's result. The reference sample data for an individual analyst should show scatter, 

as indicated by the IQR, comparable to the within analyst accuracy for that analyst. The medians 

determined for reference data indicate a bias, or systematic error, on the part of the analyst. The 
• • 

• • • 

14 

reference data should be looked at quite seriously, since these are the data that will demonstrate 

the laboratory's accuracy in detennining whether a sample is asbestos-containing or not. 

20 

15 

10 

5-i-----------­• ·························•··· 0 +---.....--...--.----.-...,.....,,-,-......___..,.. _ 

.. 

g -5 

CIJ 

-10 

-15 

-20 

-25 

-30 

Reference samples 

Analyst4 

• 

··························~::... 

10 

.•.•••••• •••.• • ............. 

• ... ......................... .................,. .......... T".......... 

• • 

• 

%cone. asbestos 

__,_....,___,..__,......,......-,-, 

• 100 

Figure 6. Reference data for analyst showing errors, calculated as result minus reference value, 

versus concentration. Medians (solid lines), IQR.s (dotted lines) and control limits (dashed lines) 

are shown for concentration categories 2 and 3. 
,.................................................~·················--···· ........................................................................... ,........................................................ . 15 

j Analyst i Sample j Date Reference!. Result j ACM Ref ACM ResjACM flagj error.Tcategoiy·l 

l 4 ! 7217 j 34584 0.1 l 20 i N Y l F I i i 

i I 1 I ; I i t····················1 

1 4 i 4118 i 34414 10 10 Y Y i O ! 0 i 2 i ········:.······.....i.......sfitr...T....34a1f.............1·0·........ ..... 10 v v i o i o 1 2 ·····1 

4 .·······... 4 l 7628 ··········r·----7628 i 34456 7.5 10 i Y Y i 0 i 2.5 i 2 ......( ...34en····· .........7.5....... .........10··-·....1····....... v··········· .........y.........t"........o' ......-r··..·2.5 ....t.........:i........ i

i 

..........44 ..........l,......98711189......i.... 3443445686... ........ 1100.........j..........1155 ........l...........Y..........t........ y .......,.!......... 0.......) ....... 5·······]·........ ........_j l i i t i Y Yi 0 i 5 i 2 i 

l.·.·.·.·.·.·.·.·.:·.·.·.~·.·.·.~~·l·.·.·_-_-...~::......l.... :::::·.·.·.·.·......... ....... 9\ .•.._.......-/·.-.-.-.-.·.-.~·~~~~·.~~~·... l.·.·...·.·........... ............._....... r~~.. ......... ~ ................. f...-............ ..............~·f·.~·.·1 :·~····r:.:.~~~~~].:.:·:·:·:·~:.:_:_i 

1 4 i 1929 l 34368 9.7 i 20 i Y l Y l 0 l 10.3 i 2 i 

;:::.=.}: t=~~:_:f.::Effi:: =-:H:=:+::::~:::+:::~~=:::r::::~:::::¥::}=+~§~r-.~_:r~1 

r -··• ..-.L.. a131......r._34612····r· ff7 r so r.......... v-···....f·····-··v·····-·L··-·o. ·.L-11 .. L.-a .....! 

l··········t·······+····{~; r······..4 .......... .......1024 .....+....~~·····t···-····l~ ; ~~·········l·-········~·······..t········~·..····+·····... .........f..···=~t····l······•-;••······· ......r...34359.... i.........ss·........r···.... so........r······... v..........f...... v.........i......... o..·······r·.._15.... t....... 3........ t

i 

L........4..........L....636.......L..34397.....L···es·.......L......so........ f.......... v...........i·--v:--.L......o.........L..-15 ....L......a....--1 

l 4 l 535 l 34346 74 l 60 l Y I Y ! 0 i -14 ! 3 l 

t.......... .........J.......~~.....l.....~?.~...........:...~Q.........~ ..........~~·········L·······...'!.............i..........'!...........1.......... .........J.....:~.~......i.......... ..........i 

1 4 ! 1223 i 34351 65 1 55 1 v ! v ! o ! -10 ! 3 ! 

l 4 l 2720 i 34612 70 l 60 ~---r---- Y .... . Y l o l -10 i 3 i •ou••••n•uuu•u••-- ••••u••unu•uu•o••' ....................uuunn uuuuun• nou..oo uu..u••••••••••n•••• ••u•••••••........•••••U•U•• •••• u ••uw ................. ...........................4-........................ .............................,ci 4 l 2138 I 34379 70 i 60 i Y Y l O l -10 l 3 l 

i 4 l no i 34547 95 l 85 i Y Y ! 0 l -10 l 3 >...................................___,O.uuuunuu•••u·HH• ,u............................ •.......................................,O, .......o ............. uouuo ................................................,, • uuuou'"uu•u•..n•nn•H••• 

l 4 I 8625 i 34470 70.5 1 65 l Y Y I o i -5.5 i 3 i 

l 4 ! 114 l 34612 ! 15 i 10 i Y , Y l o i -5 i 3 i ~·········· 4..........t...... 1130......~ .....34486.............. 20·····..·r···-..·15.........r········· v...........t.........v.........r........o...... r......-5······r.........3 .......1 

i 4 l 6436 i 34605 i 20 15 Y Y . . o i -5 i 3 i 

f 4 ! 5a1s i 34470 ' 35 30 v v i o i -5 i 3 ; 

l 4 l 4116 l 34536 ! 95 l 90 l Y i Y l 0 l -5 l 3 l 

f ::: : :::: Li:::::LEE:::.l ~·E: ·: l-::E:::A·::::::::.~::.:--1'_::::.~ ::: L:· tJ::L:L.. l 

l 4 a 8625 l 34605 1 70.5 l 75 l Y Y i 0 l 4.5 l 3 i 

L........ 4 .........t......5346...__j_... 346os..J ........ao........1... 35 ........L.........v ................... v ........1........ o........1......s ......1........ 3........1 

L........ ...........L.....~!~?. i 34~.....f.... 30 i 40 i X...........L.......Y.........J.........~.... j 10......i,......... ..........j

L......... ......... 1......~~~......J.....~~~-·-· L.......~.~....... L.......?.~.......J...........Y..........J ........Y..........J .......~......L ....~.~..l ...... ........J 

Statistics for rl-r2 

Concentration category 2: Median= 5%, IQR = 2.5% to 10.3%, Control limits= ±11.6% 

Co!lcentration category 3: Median= -7.75%, IQR = -14.75% to -5%, Control limits= ±14.4% 

Table 3. Reference data for analyst 4. 
16 

The first entry in Table 3 represents a serious problem in that a result of20% was returned for a 

reference sample containing 0.1% asbestos, generating an ACM flag. Unfortunately, there are no 

other reference samples close in concentration to this one, and so the single data point that would 

demonstrate proficiency near 1 % shows the absence ofproficiency. All the samples in 

concentration category 2 (1%-10% asbestos) are in the 7.5%-10% rang~, which is not satisfactory 

for demonstrating proficiency near I% or for generating statistics that are appropriate for the full 

concentration range. The median, IQR, and control limit calculated for this range are given at the 

bottom ofTable 3, and indicate a large positive bias (+5%) and a larger scatter than was seen 

even in the within laboratory accuracy data. The analyst has a negative bias at higher 

concentrations (concentration category 3), so much so that the lower limit ofthe IQR falls outside 

ofthe control limit, which is positioned about zero. 

The reference sample data should be used initially to demonstrate the ability ofthe analyst to 

perform an adequate analysis; in particular, to discriminate between sample~ that contain asbestos 

and those that do not. Only after general proficiency has been demonstrated do "control limits" 

have any meaning. After a level ofcompetence has been demonstrated, the reference sample data 

can be used to characterize the accuracy ofthe analyst by demonstration ofminimum bias and 

acceptable scatter. 

IMPLEMENTATION 

A sufficient amount ofQC data must be collected prior to determining any statistics to have a 

meaningful result. We recommend a minimum of 8 points per concentration category because of 

the symmetry needed for a useful interquartile range (an equal number ofhigh points to low 

points). Once the minimum number ofpoints has been collected for each concentration category 

for each type ofmeasurement (within analyst accuracy, within laboratory accuracy, and 

reference) the graphs can be generated and the statistics calculated. 

Once ~e minimum amount ofdata has been collected, additional data are appended to the file as 

collected. The results ofthe QC data should be analyzed and summarized monthly. The 

tabulation should include any qualitative errors and their resolution, the within analyst accuracy 

and reference sample statistics ofeach analyst including graphs, and the within laboratory 

accuracy statistics, including graphs. The number ofpoints that fall outside the control limits, and 

the actions taken to remedy the problems should be identified. Any improvements seen in the 

data that imply a reduction ofthe bias or decrease in the scatter should be discussed. At some 

point, older data can be segregated by simply excluding some data from the analysis according to 

date (but without deleting the data). Ifthe data are improving with time, the statistics will better 

reflect this ifthe older data are gradually removed. The implications ofthe QC data on the ability 

ofthe laboratory to perform quality analyses should be stated, in particular the data that support 

the laboratory's ability to discriminate asbestos-containing samples from samples containing <1% 

asbestos. As the data for the low concentration samples increases, it may be advisable to further 

segregate concentration category 2 (1-10% asbestos) into smaller categories such as 1-5% and 5-
17 

10%. This would provide more meaningful limits for the critical 1 % samples. 

REFERENCES 

1. AHERA- 40 CFR Part 763, Vol. 52, No. 210: "Asbestos Containing Materials in Schools; 

Final Rule and Notice. 

2. Steel, E.B., Verkouteren, J.R., Alderman, D.F., "Bulk Asbestos Analysis", NIST Handbook 

150-3, U.S. Dept. OfCommerce, Technology Administration, National Institute ofStandards and 

Technology. 

3. Perkins, R.L. and Harvey, B.W., Test Method, Method for the Determination ofAsbestos in 

Bulk Building Materials, EPA/600/R-93/116, July 1993. 

4. Youden, W.J. "Graphical Diagnosis oflnterlaboratory Test Results", Industrial Quality Control 

1959, 15. 

5. Stuart, A., and Ord, J.K., Kendall's Advanced Theoty of Statistics. Vol. 1, 5th Ed., Oxford 

University Press, New York, 1987, pp. 332-334. 
APPENDIX 1 

METHOD FOR CALCULATING QUARTILES AND MEDIANS 

Preliminary steps 

la) Sort data into ascending order 

I b) Define the first datum as "D1", the second as "D2

", and so on. 

2a) Count the number ofdata. 

2b) Define "N" to be the number ofdata: N =COUNT(data) 

2c) Define "R" to be the remainder ofthe integer division ofN/4: R =MOD(N,4). R will be 

equal to: 1, 2, 3, or 0. (R is the numerator ofthe fractional remainder ofthe division, e.g. 

9/4 =2 1/4, R=I.) 

2d) Define 11N1" to be the integer part of(3+N)/4: N1 = FLOOR((3+N)/4,1). This is not a 

rounding procedure, e.g. the integer part of3.75 is 3. 

2e) Define "N2" to be the integer part ofN/2: N2 =FLOOR(N/2,1). • 

2f) Define "N3" to be the integer part of(1+3*N)/4: N3 =FLOOR((l+3*N)/4,1). 

To determine the first Quartile (Q1

) 

3a) IfR is 1, the first quartile is the N1th data value: Q1 =DN1 

(DN1 is the value found at the position in the sorted data given by Ni, not the value 

calculated for N1) 

3b) IfR is 2, the first quartile is I/4th the sum of3*(the N1th) and the (N1+1)th data values: 

QI = (3*DN1 + DN1+1)/4 

3c) IfR is 3, the first quartile is I/2th the sum ofthe N1th and (N1+l)th data values: 

Q. =(DNl + DN1+1)/2 

3d) IfR is 0, the first quartile is I/4th the sum ofthe N1th and 3*(the (N1+ l)th) data values: 

Ql = (DNl + 3*DN1+1)/4 

To determine the Median (Q2) 

4a) IfR is either I or 3 (N is odd), the Median is the (N2+1)th data value: Median= DNl+t 

4b) IfR is either Oor 2 (N is even), the Median is I/2th the sum ofthe N2th and (N2+ 1 )th 

data values: Median= (Dm + DN2+1)/2 

To determine the third Quartile (Q3) 

Sa) IfR is 1, the third quartile is the N3th data value: Q3 =Om 

Sb) IfR is 2, the third quartile is I/4th the sum ofthe N3th and 3 *(the (N3+ 1 )th) data values: 

~ =(DN3 + 3*DN3+1)/4 

Sc) . IfR is 3, the third quartile is I/2th the sum ofthe N3th and the (N3+ 1 )th data values: 

Q3 =(DN3 + DN3+1)/2 

Sd) IfR is 0, the third quartile is I/4th the sum of3*(the N3th) and the (N3+l)th data values: 

Q3 =(3*DN3 + DN3+1)/4 
To determine the InterQuartile Range (IQR) 

6) The InterQuartile Range is the difference between Q3 and Q1: IQR == Q3 - Q1 

Example Calculations 

Data 

Datum 

D, 

Set 1 

1.0 

Set2 

1.0 

Set3 

1.0 

Set4 

1.0 

D2 2.0 2.0 2.0 2.0 

D3 3.0 3.0 3.0 3.0 

D• 4.0 4.0 4.0 4.0 

D, 5.0 5.0 s.o '5.0 

D, 6.0 6.0 6.0 6.0 

D, 7.0 7.0 7.0 7.0 

D, 8.0 8.0 8.0 8.0 

D9 9.0 9.0 9.0 9.0 

. D,o 10.0 10.0 10.0 

Du 11.0 11.0 

D12 12.0 

Intermediate tenns 

N 9 10 11 12 

R 1 (9/4) 2 (10/4) 3 (11/4) 0 (12/4) 

N1 3 (12/4) 3 (13/4) 3 (1414) 3 (15/4) 

N2 4 (912) 5 (10/2) 5 (11/2) 6 (1212) 

N, 7 (28/4) 7 (31/4) 8 (3414) 9 (37/4) 

Statistics 

Q, 3.00 (D3) 3.25 (3*0,+D.>14 3.50 (D,+DJ/2 3.75 (D,+3*D~)/4 

Median(QJ 5.00 (D~ 5.50 (D5+DJ/2 6.00 (DJ 6.50 (D6+0.,)12 

Q, 7.00 <D-7) 7.75 (D7+3*Ds)/4 8.50 (Dg+D9)/2 9.25 (3*D9+D1o)/4 

. IQR 4.0 4.5 s.o 5.5