August 22, 1966 M20-2

CHAPTER 3. SAMPLING VARIATION IN QUALITY CONTROL

3.01 RECOGNITION OF SAMPLING VARIATION

a\. Some form of sampling is used in almost all quality control
procedures to estimate quality levels, to evaluate operations in quality
terms, and frequently as the basis for management decisions affecting
the quality of operations. Any form of sampling involves chance sampling
variation which is unavoidable and cannot be eliminated. Sampling
vari­ation or sampling \"error\" refers to the chance differences that
occur among the statistics derived from different samples drawn from the
same population.

b\. Therefore, the limitations arising from sampling variability must be
recog­nized when quality control methods or other sampling procedures are
used in estimating the underlying \"true\" error rate or other
characteristic of the sampled population. These limitations apply to all
sampling, whether from finite populations, such as claims folders in the
files, or the theoretically infinite population that is generated by a
work process, such as the processing of applications for compensation
and pension benefits.

3.02 ERROR DISTRIBUTION\--SMALL FINITE POPULATION

a\. To illustrate the chance variation which cannot be eliminated in
sample results, consider a very small finite population of only five end
products. The end products are classified as being either \"correct\" or
\"in error,\" and the population is as follows:\
\
[End Product]{.underline} [Classification]{.underline}\
\
A Correct\
B Correct\
C In Error\
D Correct\
E In Error

b\. Therefore, three (60%) of these five end products are \"correct\"
and two (40%) are \"in error.\" If a random sample of two end products
is to be drawn from this population, it follows that every combination
of two end products out of the population of five then has an equal
chance of being selected as the sample. The different sample
combinations pos­sible are:

Possible Combinations

SampleNo. of End Products\
\
1 A & B\
2 A & C\
3 A & D\
4 A & E\
5 B & C\
6 B & D\
7 B & E\
8 C & D\
9 C & E\
10 D & E

3.03 CHARACTERISTICS OF SAMPLE COMBINATIONS

The tabulation in the preceding paragraph shows 10 different sample
combinations which could be selected if a random sample of two end
products is to be drawn from the population of five end products in the
above example. Each combination is as likely to be selected as any other
if the sample of two end products is selected purely at random. How­ever,
these 10 possible sample combinations have varying characteristics.
Sample I gives

3-1

M20-2 August 22, 1966

two \"correct\" end products, sample 2 contains one \"correct\" and one
\"in error,\" etc. The 10 possible sample results may be listed as
follows:

Sample No. of \"Correct\" [End Products]{.underline} \"In
[Error\"]{.underline}

No. End Products No. Percent

1 2 0 0

2 1 1 50

3 2 0 0

4 1 1 50

5 1 1 50

6 2 0 0

7 1 1 50

8 1 1 50

9 0 1 100

10 1 1 50

3.04 FREQUENCY OF OCCURRENCE OF POSSIBLE SAMPLE COMBINATIONS

a\. Note carefully the last column of the tabulation shown in the
preceding para­graph. It illustrates that the percent of end products
\"in error\" in the sample may be either 0%, 50%, or 100%, depending
upon which of the 10 sample combinations happens to be selected. But,
these varying results do not occur with equal frequency. The following
data summarizes the previous information as to frequency of occurrence:

Percent of Sample Number of Proportion of\
End Products Sample All Sample\
\"In Error\" Combinations Combinations\
\
0% 3 .30\
50% 6 .60\
100% 1 .10\
10 1.00

b\. This summary indicates what is likely to happen if the selection of
a random sample of two end products from this same population of five
end products were repeated many, many times. Since all sample
combinations are equally likely to be selected, the probabilities are
that about 60% of the samples drawn at random will contain one end
prod­uct \"in error,\" 30% will contain no end products \"in error,\" and
10% will contain two end products \"in error.\"

3.05 VARIABILITY IN A RANDOM SAMPLE RESULT

a\. The illustration described in the preceding paragraphs is an extreme
one, of course, because the population size (five) and the sample size
(two) are both extremely small. The probability that the sample result
assumes either one extreme value, 0% \"in error,\" or the other, 100%
\"in error,\" is very large. This example does show, however, that some
uncertainty must be attached to any sample result, since it is only one
of a num­ber of different possible sample results.

b\. The degree or amount of variability among sample proportions of end
prod­ucts \"in error\" is directly related to the proportion of end
products \"in error\" in the population from which the sample is
selected. It also is inversely related to the sample size. Sampling
variability is reduced as the sample size is made larger. Sampling
vari­ation cannot be eliminated\--except by abandoning sampling.

c\. In sampling many finite populations, it would be possible\--but not
necessarily wise\--to examine the entire population. But in sampling to
determine the underlying quality level of the output from a work
\"process,\" sampling cannot be replaced by inspection of the total
population, since a \"process\" is regarded as an infinite population,
and even a recognized in quality control procedures to avoid erroneous
conclusions and decisions. The mathematical principles involved will
remain true and will affect the reliability of sample

3 -2

August 22, 1966 M20-2

3.06 APPLYING PRINCIPLES OF STATISTICAL SAMPLING TO QUALITY CONTROL

a\. Statistical Control. The word control has a special technical
meaning in the language of SQC. process is described as in statistical
control when a stable system of chance causes is operating. We must be
satisfied with the variation due to chance causes,

or else change the work process fundamentally. This principle is
illustrated below.

b\. Example: A container of 2,000 beads was used to represent a work
process. Each bead represented a work unit. Forty (2.O%) of the beads
were red. These were re­garded as representing work units with one or
more errors. The remaining 1,960 beads were white, and were regarded as
work units without errors. A paddle with 100 holes was used to scoop up
random samples of 100 beads. Since the beads were returned to the
con­tainer after each scoop, each random sample was drawn from a
continuing \"operation\" in which the \"error rate\" remained constant
at a known 2.0%. In effect, this experiment simulated the infinite
population associated with a work process, since under the proce­dure
followed the population could not be exhausted by sampling. The results
of 100 ran­dom scoops shown as 25 samples of 400 each (4 successive
scoops per sample) are shown in the following tabulation of sample
results;

Sample Sample Number of Work Units Percent of Work Units\
Number Size \"In Error\" I/ \"in Error\" I/\
\
1 400 5 1.25\
2 400 8 2.00\
3 400 6 1.50\
4 400 5 1.25\
5 400 13 3.25\
6 400 10 2.50\
7 400 7 1.75\
8 400 14 3.50\
9 400 5 1.25\
10 400 9 2.25\
11 400 3 0.75\
12 400 2 0.50\
13 400 16 4.00\
14 400 5 1.25\
15 400 11 2.75\
16 400 4 1. 00\
17 400 5 1.25\
18 400 7 1.75\
19 400 10 2.50\
20 400 10 2.50\
21 400 10 2.50\
22 400 11 2.75\
23 400 10 2.50\
24 400 8 2.00\
25 7 1.75\
10,000 201 2.01

[1/]{.underline} The number of red beads in each sample is taken to
represent the \"number of work units in error.\"

c\. In the sample results tabulated above, note carefully the variation
in the num­ber of work units in error (number of red beads) that occurred
in the successive random samples of 400 work units (total number of
beads in sample) drawn from the population hav­ing a known constant error
rate of 2.O%. They ranged from a low of 2 errors (0.50%) in sam­ple
number 12 to a high of 16 errors (4.00%) in sample number 13. Only two
of the samples, numbers 2 and 24, contained exactly 2.O% \"in error,\"
the known constant \"error rate\" of the population. The differences
between the error rates of the different samples and the true error rate
of the population represents \" sampling variability\" or \" sampling
error\" (not to be

3-3

M20-2 August 22, 1966

confused with a sample \"error rate\"). In sampling from a work process,
which represents an infinite population, sampling error cannot be
eliminated, but it may be reduced by taking larger samples.

d\. The \"process\" illustrated above has to be \"in control\" because
each sample was drawn from the same population, in which the \"error
rate\" was a known 2.0%. For this error rate and a sample size of 400,
the table of \"Three Standard Deviation Control Limits\" in appendix E
shows control limits of 0.00% and 4. 1O%. Note that all of the sample
error rates tabulated above fell within these control limits.

e\. Any similar experiment of drawing samples from a population will
demon­strate the same point; i.e., that successive sample estimates will
vary, within limits, due to chance causes alone. This same kind of
variation occurs with respect to any random sample of work units drawn
from a work process to estimate and evaluate the quality level.
Therefore, [reliable estimates and]{.underline} evaluations [of quality
levels cannot be made based on]{.underline} [sample data]{.underline}
unless allowance is made for [the possible]{.underline} variation which
could be due to chance [causes alone]{.underline}.

f\. This principle of statistical control is fundamental and must be
recognized in the interpretation of sample results. Assume, for example,
that the population in the illus­tration cited above represented
completed compensation and pension rating actions, with a standard
percent in error of 2.0%. [If adjective evaluations were assigned to
individual]{.underline} [sample]{.underline} results, a higher
evaluation [could not be assigned reliably to sample number]{.underline}
12 (containing 0.50% \"in error\") [than could be assigned to sample 13
(containing 4.00% \"in]{.underline} [error\").]{.underline} Both samples
yielded an error rate which could come from the same population, with an
underlying \"error rate\" of 2.0% and, in the illustration, both error
rates in fact did come from the same population. Unless it can be shown
with a high degree of confid­ence that the different sample results
reflect different quality levels, it would be a serious measurement
error to label the quality level as better on the basis of any one
sample re­sult compared to another.

3 -4