78
Size of Experiments for Testing Sera or Vaccines
arising through pure chance is (¾) 12 or only 0.0317 per cent, nearly five times as
small as in the previous case and sufficiently small for all practical purposes to
differentiate between chance and real deviations. Or again if we suppose that
in an experiment with 24 animals only two caught infection we shall have three
events to consider (i) the event observed giving a departure of four from the
expected number of six (ii) the event in which one caught infection giving a
departure in each class ' infected' and ' non-infected ' of five and (iii) the event
in which none caught infection giving a departure in each class of six. The
probability of the first event is obviously 24×23/2 (1/4) 2 (¾) 22 =0.03077, that
of the second is 22(¼) (¾) 23 =0.00803 and that of the third is (¾) 24 =0.00100
the sum of the three being 0.03980 or 4.0 per cent, again sufficiently small to
differentiate with a fair degree of confidence between chance and real devia-
tions. As to what is a sufficiently small probability in order that the departure
may be regarded as really due to the treatment effect is left to the discretion
of the research worker. Generally a probability of 5 per cent or less is con-
sidered sufficiently small by the statistician and the departure is considered
to be significant.
The foregoing approach illustrating the principle and the procedure
involved in the test of significance for the departure of the observed result
from the expectation can be readily used to determine the number of animals
necessary to conclude whether the observed reduction in the rate of normal
infection by the experimental treatment is significant. For instance, in the
experiment with 12 animals none catching infection, considered above, we
might say that the result like the one observed will occur with a sufficiently
low probability and that therefore 12 animals will be sufficient to reach a
conclusive verdict on the efficacy of the experimental treatment. Or again
referring to the experiment with 24 animals, two catching infection, giving a
rate of infection of less than 10 per cent we might say that in experiments
of this size a result such as the one observed or better lowering the normal
rate of infection by 15 per cent or more will occur with a sufficiently low
probability and that, therefore, 24 animals will be sufficient to conclude
that the experimental treatment significantly lowers the rate of infection by
10 per cent or more than the normal. Actually in the first example, working
at the 5 per cent level of significance, even 11 animals will suffice since the
probability that none out of 11 is infected is 0.04224, but with 10 animals
and none-catching infection the probability is larger than 0.05, and we,
therefore, conclude that it will not be advisable to experiment with less than
11 animals. Similarly, in the second example if the margin of reduction desired
to result from the experimental treatment is 10 per cent it will be found
that a minimum number of 23 animals will suffice to secure a conclusive verdict.
There is, however, one aspect of this problem arising from the discontinuity
of the binomial distribution which must be examined before finally accepting
this conclusion about the minimum number of animals. For, just as a sample
of 23 appears to be adequate to test the point at issue, a sample of 18 is equally
so, since the probability of observing the rate of infection of 10 per cent or less
in a sample of this size is also less than 0.05. The same is true, when the size
is 19 but with 20 or higher numbers upto 23 the case is different since with