identified by symbols. The hollow symbols show the
results for the first pair of samples. ‘The corresponding
solid symbols show the work of these same laboratories

the test procedure and about the laboratories. Clearly
this procedure is one that is vulnerable to individual
bias. Two of the eight laboratories appear in the same
region for both pairs. The circle laboratory is very con-

Summory of Advantages of Graphical Diagnosis
The double-sample, graphic analysis scheme described
in this article offers a number of advantages.

(2) An unusually light burden is imposed on each lab-
oratory

(2) The graphical procedure greatly facilitates presen-
tation of the results in a convincing manner

(3) No statistical background is required to follow the
Teasoning and no computations are required to dem-
onstrate the general presence of constant errors
and the gross deviations of individual laboratories

(4) A minimum of computation is imposed upon the in-
dividual collating the results

(5) The use of a circle of 25 or 3.0 o radius shows the
individual laboratories whether or not their method
of carrying out the test has in some way become
saddled with a substantial constant error

(6) Most important the direction for improvement is
clearly indicated

a, A long, narrow ellipse directs attention to a more
careful description of the procedure or even to the

need for modification

b. Wild points far out near either axis indicate er-
ratic work

¢. Wild points far out along the 45 degree line are
strong i ions from the
specified

d. General of errors is indicat-

eo OCD CCC; CCC OCCCCET

137-5


TABLE !—Data and Calculations Percent Insoluble Residue
in Cement Roported by 29 Loboraterinn’

A—B (AB) ~— 0.095

|

E EGGSE GEESE ERGES SsceRl

i

pexty
0.005

ck

Peo ae ek
bi id HE GGGRE AEG BE

BERN DENN BEES USES Sense veunn i
p

KI FGER SES ReeRG REGGE Seeee Recech

EICRSR ace

|

‘The data on percent insoluble residues reported
laboratories are given in Table I and plotted in

ag
wR

This estimate of the standard deviation for precision
leads to the construction of circles (centered on the in-
tersection of the median lines) within which any given
Percentage of the points can be expected to fall should
the laboratories be able to eliminate all bias or constant
errors. The multiples of the standard deviation that
Rilude various percents of the points are given in

Thus a circle whose radius is about 25 to 3.0 times
the standard deviation gives a fair idea of the smallest
circle that could be expected to contain nearly all points
after the elimination of the constant errors that are

st the points to congregate in the upper left and
lower right quadrants. Generally a fair number of points
will lie outside such a circle. The laboratories respon-

TABLE

g | Skssaagessnys

z

136-4

Figure 4—Ca0 in Cement (interval equals one percent)



(Reprinted from INDUSTRIAL QUALITY CONTROL, Vol. XV, No. 11, May 1959)

Graphical Diagnosis
of Interlaboratory

Test Results

W. J. YOUDEN
National Bureau of Standards, Washington, D. C.

Introduction
Interlaboratory or round robin programs to evaluate
the performance of test procedures will always be with
us, New materials require new tests. New, and hope-

uct.

sults made on a given sample of the product. The im-
pact of the errors of measurement associated with the
test procedure is obvious because half the tests made on
a product that just meets specification will rate the
product below specification.

Test Procedures and Production Costs

equal to or better than the specification. The manu-
facturer already has to contend with variation in the

Process. Considerable saving in manufacturing costs

can be affected by reducing the margin between the

quality level set for production and that called for in the
‘ficati i

Missed Opportunities in Interlaboratory
Test Programs

Strangely enotgh modern statistical tests such as the
analysis of multifactor studies and the iselation of com-
ponents of variance have not made the contribution ex-
pected of them. Part of this no doubt comes about be-
cause these more sophisticated statistical techniques are
not too well understood by some of those in the labora-
tories that run the tests. It is all very well for someone
with statistical skill to set up an intricate interlaboratory
test program and analyse the data but this still leaves
the problem of interpreting the statistical jargon to those
directly concerned. Even when this interpretation is
undertaken the report is apt to read somewhat along
these lines. “Duplicates run by the same operator in the

133-1


same laboratory show excellent agreement. Agreement
between different operators in the same laboratories is
not quite so good, and very poor between results from
i t laboratories. Results on different days do not
agree as well as those obtained on the same day.” This
is a brief summary of the interpretation that is made

cemed were already aware of the state of affairs just
described and want to know what can be done to im-
Prove matters. It is just here that statisticians have not
risen to the opportunities Presented by interlaboratory
test programs.

When all is said and done, what we want is rather
simple. We want to know whether the test procedure as
set forth is capable of yielding acceptable agreement
among results from different laboratories. If the results
are not acceptable, we would like some specific indica-
tion of what is wrong with the procedure. If the pro-
cedure appears to be reasonably good but there are
some disturbing di ies, we would like to know
which laboratories are having trouble and if possible
why they are having trouble. And most important. we
should be able to get this information back to the labo.
ratories concerned in such a form that the diagnosis is
believed. For only so will these laboratories take any
action. to correct the difficulties.

Graphical Representation of Results

‘The graphical procedure is based upon a very simple
interlaboratory program. Samples of two different ma-
terials, A and B, are sent to a number of laboratories
which are asked to make one test on each material. The
two materials should be similar and be reasonably close
in the magnitude of the property evaluated. This will
avoid complications that may arise from differential be-

reported
for material B. ‘The pair of results reported by a labo-
ratory are then used to plot a point. ‘There will be as
many i

the x-axis so that there are as many points
line as there are below it. A second medi:

line is drawn parallel to the y-axis and so placed that
there are as many points on the left as there are on the
right of this line. Figure 1 shows the seven-day tensile

Diagnosis of the Contiguration of Points

The two median lines divide the graph paper into four
quadrants. In the ideal situation where only random
isi to

approximately bisecti1 upper right and lower left
qi is reason to expect the line to make a
45 degree angle with the axes when the same scale is

7

134-2

6 7 8
Figure 2—Percent Phthalic Anhydride


are following rather carefully their own versions of the
test procedure.
Checking on Sample Variation

i 48 BS Bho

iy
i
i
i
a
i

egy
Hil
i
rel

He
Hl
id
F
FE

F
Hy
i
EE

ag
ef
Hf
i
Fi

i
i
ett

or more points are so far out of the picture that it is
better not to compress the scale in order to show them.
Such points should be ignored in locating the median
lines. (See Fig. L) The more distant points tend to fall

Points in the upper right or lower left quadrants tha
are far removed from the intersection of the median

?
HW

fully the prescribed procedure for performing the test
and endeavor to locate the cause of the large bias.
All of the above interpretation can be made while
keeping anonymous the identity of the plotted points.
When circulating a report of the interlaboratory test it
might be helpful i
the in the copy going to that laboratory.

7
i
|

Estimating the Precision of the Test Procedure

‘The above discussion does not exhaust the informa-
tion to be gleaned from

:
|

ences d,, dz, ... d,. Calculate d, the algebraic average
difference. Subtract d from each difference and obtain
a set of corrected differences d,’, dy’, ... d,’. The aver-
age of the absolute values of these differences when
multiplied by \/x/2 or 0.886 gives an estimate of the
standard deviation.

135-3