From: lambert@spectrum.cs.unsw.oz.au (Tim Lambert)
Subject: Oh No!  NSW homicides again
Organization: CS&E, Uni of NSW, Australia
	<LAMBERT.92Dec18025613@nankeen.spectrum.cs.unsw.oz.au>
	<1992Dec19.231932.3652@mtu.edu>
Date: Mon, 21 Dec 1992 17:30:35 GMT

>>>>> On Sat, 19 Dec 1992 23:19:32 GMT, cescript@mtu.edu (Charles Scripter) said:
> I did point it out.

>> > I see in another post you give your model(s):
>> 
>> >>11*a = 25.6
>> >>10*b = 14.9
>> 
>> > But you do not give the errors on your experimental data.  It's quite
>> > possible that once I assign error bars to your data that I can draw a
>> > straight line through all of points.  (and deviations of 2-3 times the
>> > standard deviation are not unusual in experiments with low statistics).
>> 
>> > Here, let's pretend I'm a Nuclear Physicist... (so I can get error bars)
>> 
>> > 11*a = 25.6 +/- sqrt(25.6)  ;  a=2.33 +/- 0.46  ;  1.87 < a < 2.79
>> > 10*b = 14.9 +/- sqrt(14.9)  ;  b=1.49 +/- 0.39  ;  1.10 < b < 1.88
>> 
>> > Examining these two systems, we can trivially see that "a" and "b"
>> > are indistinguishable.  QED  ;-)
>> 
>> As a case in point, we can easily see how Charles has fudged the
>> statistics here to prove that `a' and `b' are not significantly
>> different.  He just made up some errors out of thin air.

> Ho ho ho!!  No real fudging here.  I simply applied the uncertainty I
> would use if I were studying a system which obeys Poisson statistics
> (as systems which are limited by Quantum noise do: radioactive
> halflives, etc.).  You can easily find that Poisson statistics show
> deviations of SQRT(N). 

> It is quite standard practice to use SQRT(N) for error, and
> works suprisingly well for many systems (as many of them obey
> Poisson statistics).

But the numbers above are homicide rates per 100k population, not the
actual numbers of homicides, which might be expected to be Poisson
distributed.

Suppose we decide to measure homicides per 10k population.  Then
repeating your calculations:
 11*a = 256 +/- sqrt(256)  ;  a=23.3 +/- 1.5  ;  21.8 < a < 24.8
 10*b = 149 +/- sqrt(149)  ;  b=14.9 +/- 1.2  ;  13.7 < b < 16.1

Now a and b seem to differ greatly.

> Here, let's do it a more conventional (and probably simplest) way.
> Everybody get out your calculators, and run the following data through
> your stats function (separately, please).  Obtain the two constants
> "a" and "b", and estimated errors:

> [hopefully Tim won't argue that SQRT(SUM((X-Xi)**2))/N-1) is an
> invalid error estimate].

Tim will so argue.  Check a stats book under "confidence interval for
the mean".  The standard deviation of the mean is estimated by:
  s/sqrt(n)
  where s is the sample standard deviation (which is the SQRT formula you
   give above)

>> 1910-1920 rates (before) 2.6 2.7 2.4 2.1 2.4 2.6 2.1 1.9 1.4 2.7 2.7
>> 1921-1930 rates (after)  1.8 1.6 1.0 0.9 1.1 1.6 1.6 1.9 1.7 1.7

> 1910-1920 rates (before)  2.33 +/- 0.41  ; 1.92 < a < 2.74 
> 1921-1930 rates (after)   1.49 +/- 0.35  ; 1.14 < b < 1.84

The correct numbers:
1910-1920 rates (before)  2.33 +/- 0.13  ; 2.20 < a < 2.46 
1921-1930 rates (after)   1.49 +/- 0.11  ; 1.38 < b < 1.60

(These are 68% confidence intervals.)

> OH NO!!  Look how far my *FUDGED* errors bar were OFF from ACTUAL!!!
> You are absolutely right, Tim, my previous estimate of error was
> completely bogus!...  =;->  [Could it be that the system actually
> obeys Poisson statistics, Tim?]

Actually, both your estimates were wrong and it is mere coincidence
that they came out the same.

> Shucks, my error bars don't overlap any more.  Should we put any
> statistical significance to the fact that the ranges fail to overlap
> by 1/5 of an error bar?...  (what the old rule about 3 sigma?... ;-)

In fact, the means differ by over 5 sigma, which is significant in
anyone's book.

> Now let's try somthing really interesting, and average them all together:

> 1910-1930 rates (all of them) 1.93 +/- 0.57 ; 1.36 < rate < 2.50

> Not exactly a large increase in the size of the error bars.  I have a
> hard time believing the claim of 2 different constants for this system.

> Do *YOU* find it surprising that you can reduce the error bars
> slightly by addition of an extra parameter?...  Why not fold in 21
> parameters and reduce the error to zero?...  Do you *really* attribute
> any statistical significance to your model?!!

This is not the way to measure goodness of fit.

The usual method is to compute sum ((0i-Ei)/SDi)^2, where Oi is the
ith observation, Ei is your model's prediction for the ith
observation, and SDi is the standard deviation of the ith observation.
For your simple model, the prediction is 1.93 (the mean) for all
observations.

You also need to know the standard deviations to do the calculation.
Here they are:
1910-20 0.34 0.34 0.33 0.33 0.32 0.32 0.32 0.32 0.32 0.31 0.31 
1921-30 0.30 0.30 0.30 0.29 0.29 0.29 0.28 0.28 0.28 0.28
(I got these by assuming homicides were Poisson distributed, and the
homicide rate did not change over the period.  They get smaller
because the population grew.)

I compute a chi-square value of 69.2 for your model.  This statistic
has 20 degrees of freedom (21 observations - one fitted parameter).
The probability of a value this large is 0.0000002.  I think we can
reject the simple model.

Let's repeat the calculation with my model: a rate of 2.33 for 10-20
and a rate of 1.49 for 21-30.

The chi-square statistic is 30.4, with 19 degrees of freedom (two
fitted parameters this time).  This has a probability of 0.05.  This
model looks pretty good to me.  If we want to raise that probability a
bit more, I could fit a paramter for the effects of enlistments during
WW1 on the homicide rate in the years 1915-18.

Tim
Newsgroups: talk.politics.guns