From: cescript@mtu.edu (Charles Scripter)
Subject: Re: Oh No!  NSW homicides again
Organization: Michigan Technological University
Date: Mon, 21 Dec 1992 23:59:39 GMT

In article <LAMBERT.92Dec22033035@nankeen.spectrum.cs.unsw.oz.au>,
lambert@spectrum.cs.unsw.oz.au (Tim Lambert) writes:

> >>>>> On Sat, 19 Dec 1992 23:19:32 GMT, cescript@mtu.edu
>(Charles Scripter) said:
> > I did point it out.
> 
> > It is quite standard practice to use SQRT(N) for error, and
> > works surprisingly well for many systems (as many of them obey
> > Poisson statistics).
> 
> But the numbers above are homicide rates per 100k population, not the
> actual numbers of homicides,

Absolutely correct, Tim.  I'm really surprised that it took an "expert
in statistics" such as yourself **2 MONTHS** to figure this out.  The
SQRT(N) argument is valid for *unnormalized* data...  Yes, I knew it
was incorrect at the time I posted it; I posted it as an "exercise for
the student".  You didn't fare very well... 

> which might be expected to be Poisson distributed.

Quite often...

> > Here, let's do it a more conventional (and probably simplest) way.
> > Everybody get out your calculators, and run the following data through
> > your stats function (separately, please).  Obtain the two constants
> > "a" and "b", and estimated errors:
> 
> > [hopefully Tim won't argue that SQRT(SUM((X-Xi)**2))/N-1) is an
> > invalid error estimate].
> 
> Tim will so argue.  Check a stats book under "confidence interval for
> the mean".  The standard deviation of the mean is estimated by:
>   s/sqrt(n)

Yup, s/sqrt(n) gives the _error_of_the_mean_, and tells you nothing
about the reliability of your data, only the reliability of your MEAN.

Example:  Randomly generate data of a Gaussian form, centered at
some value X (define X and the width, and randomize the the events).
Take a large number of samples.  The mean will be "found" at ~ X, and
the probability that the true mean (=X) is within the measured mean
+/- SD(sample)/SQRT(N) is 68%.  As the number of events goes to infinity,
the measured mean will become *exactly* X, and SD(mean) --> 0, but the
SD(sample) will remain finite (the defined width of the Gaussian).

>   where s is the sample standard deviation (which is the SQRT formula you
>    give above)
> 
> >> 1910-1920 rates (before) 2.6 2.7 2.4 2.1 2.4 2.6 2.1 1.9 1.4 2.7 2.7
> >> 1921-1930 rates (after)  1.8 1.6 1.0 0.9 1.1 1.6 1.6 1.9 1.7 1.7
> 
> > 1910-1920 rates (before)  2.33 +/- 0.41  ; 1.92 < a < 2.74 
> > 1921-1930 rates (after)   1.49 +/- 0.35  ; 1.14 < b < 1.84
> 
> The correct numbers:
> 1910-1920 rates (before)  2.33 +/- 0.13  ; 2.20 < a < 2.46 
> 1921-1930 rates (after)   1.49 +/- 0.11  ; 1.38 < b < 1.60
> 
> (These are 68% confidence intervals.)

Note that Lambert is trying mislead everyone by posting the
_standard_deviation_of_the_mean_ in response to my
_standard_deviation_of_the_sample_.  The small SD(mean) does not
reduce the actual SD(sample), but only gives information on how good
your estimate for the _AVERAGE_ was (i.e. he has 68% confidence that
2.33 is within 0.13 of the *TRUE* mean [if one could know the "true
mean"]).  Must be Tim's been reading "How to Lie with Statistics"...

In reality, one could argue that a large SD(mean) (in comparison to
the SD(sample)) actually fuzzes out the edges of the SD(sample), with
the net effect of increasing the error bars. 

> Actually, both your estimates were wrong

False, Tim.  The SD(sample) *is* the _correct_measure_ of the error in
the sample.  The SD(mean) does not provide an estimate of the
probability that a measurement will fall within a certain range. 

For example, 9 of 11 years fall *OUTSIDE* your SD(mean) for the 1st set,
and 7 of 10 for the 2nd set.  What can you conclude from this?...

It's more useful to note that 9 of 11 fall *WITHIN* the SD(sample) for
the 1st set, and 6 of 10 for the 2nd set.  Here one can get a feel for
the distribution of the data (68.26% should fall within +/-SD(sample))

The fact that the distributions of the 2 samples overlap by so much
indicates that it is unwise to use 2 parameters to model this system.

>and it is mere coincidence that they came out the same.

Correct, it *was* coincidence that SQRT(N) ~= SD(sample)

> > Shucks, my error bars don't overlap any more.  Should we put any
> > statistical significance to the fact that the ranges fail to overlap
> > by 1/5 of an error bar?...  (what the old rule about 3 sigma?... ;-)
> 
> In fact, the means differ by over 5 sigma, which is significant in
> anyone's book.

Not true, *each* SD(mean) has 3 sigma, and all that tells you is that
each "true mean" has a 99.7% chance of actually being within 3
SD(mean) of the average calculated (or vice-versa).  In fact, 3 SD(mean)
applied to each mean, causes the 2 estimates of the *mean* to overlap. 

You can *claim* that the only real measure of a system is the SD(mean),
but when the distributions overlap by this much, your argument is specious. 

> > Now let's try something really interesting, and average them all together:
> 
> > 1910-1930 rates (all of them) 1.93 +/- 0.57 ; 1.36 < rate < 2.50
> 
> > Not exactly a large increase in the size of the error bars.  I have a
> > hard time believing the claim of 2 different constants for this system.
> 
> > Do *YOU* find it surprising that you can reduce the error bars
> > slightly by addition of an extra parameter?...  Why not fold in 21
> > parameters and reduce the error to zero?...  Do you *really* attribute
> > any statistical significance to your model?!!
> 
> This is not the way to measure goodness of fit.

It's a better way to "measure goodness of fit" than to extract imaginary
"effects" from noise.  2 parameters *will* give a smaller SD(sample)
than one parameter, but 21 parameters could reduce it to zero; Not
very significant, nor enlightening.  Your 2 parameter model has 1 too
many parameters in it; You will not convince me that your addition of
this parameter is valid, with the noisy data given.

-- 
Charles Scripter   *   cescript@phy.mtu.edu
Dept of Physics, Michigan Technological University, Houghton, MI 49931
-------------------------------------------------------------------
With your one remaining eye, please do NOT look back into the laser
Newsgroups: talk.politics.guns