# On the Importance of Systematic Biases in Science

From pg812-1020 of [Chapter 8 “Sufficiency, Ancillarity, And All That”](http://omega.albany.edu:8008/ETJ-PS/cc8k.ps) of [_Probability Theory: The Logic of Science_](http://omega.albany.edu:8008/JaynesBook.html) by E.T. Jaynes:

> The classical example showing the error of this kind of reasoning is the fable about the height of the Emperor of China. Supposing that each person in China surely knows the height of the Emperor to an accuracy of at least ±1 meter, if there are _N_=1,000,000,000 inhabitants, then it seems that we could determine his height to an accuracy at least as good as
> 
> ![](http://www.codecogs.com/png.latex?\frac{1}{\sqrt{1,000,000,000}}m = 0.003cm "\frac{1}{\sqrt{1,000,000,000}}m = 0.003cm") (8-49)
> 
> merely by asking each person’s opinion and averaging the results.
> 
> The absurdity of the conclusion tells us rather forcefully that the ![](http://www.codecogs.com/png.latex?\sqrt{N} "\sqrt{N}") rule is not always valid, even when the separate data values are causally independent; it requires them to be _logically_ independent. In this case, we know that the vast majority of the inhabitants of China have never seen the Emperor; yet they have been discussing the Emperor among themselves and some kind of mental image of him has evolved as folklore. Then knowledge of the answer given by one does tell us something about the answer likely to be given by another, so they are not logically independent. Indeed, folklore has almost surely generated a systematic error, which survives the averaging; thus the above estimate would tell us something about the folklore, but almost nothing about the Emperor.
> 
> We could put it roughly as follows:
> 
> error in estimate = ![](http://www.codecogs.com/png.latex?S \pm \frac{R}{\sqrt{N}} "S \pm \frac{R}{\sqrt{N}}") (8-50)
> 
> where _S_ is the common systematic error in each datum, _R_ is the RMS ‘random’ error in the individual data values. Uninformed opinions, even though they may agree well among themselves, are nearly worthless as evidence. Therefore sound scientific inference demands that, when this is a possibility, we use a form of probability theory (i.e. a probabilistic model) which is sophisticated enough to detect this situation and make allowances for it.
> 
> As a start on this, equation (8-50) gives us a crude but useful rule of thumb; it shows that, unless we _know_ that the systematic error is less than about ![](http://www.codecogs.com/png.latex?\frac{1}{3} "\frac{1}{3}") of the random error, we cannot be sure that the average of a million data values is any more accurate or reliable than the average of ten^[1](#fn1)^. As Henri Poincare put it: “The physicist is persuaded that one good measurement is worth many bad ones.” This has been well recognized by experimental physicists for generations; but warnings about it are conspicuously missing in the “soft” sciences whose practitioners are educated from those textbooks.

Or pg1019-1020 [Chapter 10 “Physics of ‘Random Experiments’”](http://omega.albany.edu:8008/ETJ-PS/cc10i.ps):

> …Nevertheless, the existence of such a strong connection is clearly only an ideal limiting case unlikely to be realized in any real application. For this reason, the law of large numbers and limit theorems of probability theory can be grossly misleading to a scientist or engineer who naively supposes them to be experimental facts, and tries to interpret them literally in his problems. Here are two simple examples:
> 
> 1.  Suppose there is some random experiment in which you assign a probability _p_ for some particular outcome _A_. It is important to estimate accurately the fraction _f_ of times _A_ will be true in the next million trials. If you try to use the laws of large numbers, it will tell you various things about _f_; for example, that it is quite likely to differ from _p_ by less than a tenth of one percent, and enormously unlikely to differ from _p_ by more than one percent. But now, imagine that in the first hundred trials, the observed frequency of _A_ turned out to be entirely different from _p_. Would this lead you to suspect that something was wrong, and revise your probability assignment for the 101’st trial? If it would, then your state of knowledge is different from that required for the validity of the law of large numbers. You are not sure of the independence of different trials, and/or you are not sure of the correctness of the numerical value of _p_. Your prediction of _f_ for a million trials is probably no more reliable than for a hundred.
> 2.  The common sense of a good experimental scientist tells him the same thing without any probability theory. Suppose someone is measuring the velocity of light. After making allowances for the known systematic errors, he could calculate a probability distribution for the various other errors, based on the noise level in his electronics, vibration amplitudes, etc. At this point, a naive application of the law of large numbers might lead him to think that he can add three significant figures to his measurement merely by repeating it a million times and averaging the results. But, of course, what he would actually do is to repeat some unknown systematic error a million times. It is idle to repeat a physical measurement an enormous number of times in the hope that “good statistics” will average out your errors, because we cannot know the full systematic error. This is the old “Emperor of China” fallacy…
> 
> Indeed, unless we know that all sources of systematic error - recognized or unrecognized - contribute less than about one-third the total error, we cannot be sure that the average of a million measurements is any more reliable than the average of ten. Our time is much better spent in designing a new experiment which will give a lower probable error per trial. As Poincare put it, “The physicist is persuaded that one good measurement is worth many bad ones.”^[2](#fn2)^ In other words, the common sense of a scientist tells him that the probabilities he assigns to various errors do not have a strong connection with frequencies, and that methods of inference which presuppose such a connection could be disastrously misleading in his problems.

I excerpted & typed up these quotes for use in my [DNB FAQ appendix on systematic problems](http://www.gwern.net/DNB%20FAQ#flaws-in-mainstream-science-and-psychology); the applicability of Jaynes’s observations to things like publication bias is obvious. See also http://lesswrong.com/lw/g13/against_nhst/

* * *

1.  If I am understanding this right, Jaynes’s point here is that the random error shrinks towards zero as _N_ increases, but this error is added onto the “common systematic error” _S_, so the total error approaches _S_ no matter how many observations you make and this can force the total error up as well as down (variability, in this case, actually being helpful for once). So for example, ![](http://www.codecogs.com/png.latex?\frac{1}{3} + \frac{1}{\sqrt{10}} = 0.66 "\frac{1}{3} + \frac{1}{\sqrt{10}} = 0.66"); with _N_=100, it’s 0.43; with _N_=1,000,000 it’s 0.334; and with _N_=1,000,000 it equals 0.333365 etc, and never going below the original systematic error of ![](http://www.codecogs.com/png.latex?\frac{1}{3} "\frac{1}{3}"). This leads to the unfortunate consequence that the likely error of _N_=10 is 0.017<_x_<0.64956 while for _N_=1,000,000 it is the similar range 0.017<_x_<0.33433 - so it is possible that the estimate could be exactly as good (or bad) for the tiny sample as compared with the enormous sample, since neither can do better than 0.017![↩](#fnref1)
    
2.  Possibly this is what Lord Rutherford meant when he said, “If your experiment needs statistics you ought to have done a better experiment”.[↩](#fnref2)