From: cescript@mtu.edu (Charles Scripter)
Subject: Re: Statistical Analysis
Organization: Michigan Technological University
Date: Sat, 19 Dec 1992 23:19:32 GMT

In article <LAMBERT.92Dec18025613@nankeen.spectrum.cs.unsw.oz.au>,
lambert@spectrum.cs.unsw.oz.au (Tim Lambert) writes:

> >>>>> On Wed, 16 Dec 1992 00:06:30 GMT, cescript@mtu.edu
>(Charles Scripter) said:
> 
> > Your refutatation of my last objection must not have made it to the U.S.
> 
> I've appended it below.

Excellent!

> > In your notation, is "degree of freedom" identical to "degree of
> > freedom of a chi-square statistic"?
> 
> No, could be degree of freedom in a t statistic.  However. I said
> "degree of freedom in a chi-square statistic", so there is no excuse
> for confusion.

False...  The posting I flamed (the only one I saw, in fact) referred
only to "degree of freedom", no qualifiers were added...  And I knew
one could least square fit an elephant with that many parameters. ;-)

> Try looking up "degree of freedom" in a statistics book, or an
> encyclopedia of science or mathematics.

OK, Let's try the "CRC Handbook of Chemistry and Physics" (70th, 1989-90):

degree of freedom: "The number of variables determining the state of a
                    system to which arbitrary values can be assigned".

Gee, that doesn't sound like what *you* used, Tim...  It sounds more
like the number of variables required to span the space (you must span
the space, if you wish to completely determine a system's *state*)

> >>> On Sun, 18 Oct 1992 22:03:09 GMT, cescript@mtu.edu (Charles Scripter) said:
> >> > lambert@spectrum.cs.unsw.oz.au (Tim Lambert) writes:
> >> Very amusing, but that's 20 or 30 FITTED PARAMETERS to fit the
> >> elephant.  My 19 df are the degrees of freedom in the DATA, which
> >                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Here was the first time you qualified your meaning to "degrees of
freedom in the DATA", previous to this you only referred to DoF.

> > Here's a good way to confuse everyone on the net.  In the real world,
> > data are numbers/events; They are fixed, so talking about 'degrees of
> > freedom in the DATA" is contradictory.  The minute you refer to 19
> > degrees of freedom, every scientist on the network will immediately
> > understand that you are fitting with 19 unresolved constants.  To them, you
> > have defined the number of degrees of freedom necessary to span the space.
> 
> I'm sorry to confuse you but there really is no way to refer to the
> degrees of freedom of a Chi square statistic without using the phrase
> "degrees of freedom".

You could try using the full statement, rather than an abbreviation.

> I suggest you consult an elementary book on
> statistics and look in the index under Chi square.  I hope that will
> clear up your misconception that "'degrees of freedom in the data' is
> contradictory.

I would argue that the _data_ has *NO* degrees of freedom (unless
you're given to adjusting the data to satisfy your model). 

> I should hope that any scientist worthy of the name would know some
> very basic statistics and know what degrees of freedom means in a
> statistical context.

See the CRC Handbook again.

> Even one who didn't should have noted the
> unfamiliar phrase "Chi square statistic" and tried to determine what
> it meant before making unfounded assertions about pachyderms.

No sense of humor whatsoever, huh?  However, my comment was hardly
unfounded. 

> > I will pose the opposite question:  Why should I trust your results?
> > This is the way *science* is done.  You propose a theory, and *prove*
> > the results.  This means that the author must give convincing evidence,
> > and eliminate cause for disbelief in your results (i.e. show that your
> > opponents questions of the validity of your data/results are without
> > basis).
> 
> This is Usenet, not a journal article.  In a journal article, one must
> answer all possible objections, since the reader has no easy way to
> raise objections with the author.  If you have some substantive
> objection or alternative model you only have to post or mail me.

I posted an alternate model.  You have failed to show that you model
is any more valid than the absolute simplest model.

You are the one who is claiming that one can model the system, and
show statistically/scientifically that firearm ownership is related to
crime.  Since you have asserted this, it is well within reason to
bring you to task for your claims. 

> >   Merely stating the chi-squared of your fit does not constitute
> > "convincing evidence".  A good start would be to show that *NO OTHER*
> > model describes the system as well as your model (this is where you
> > can apply your chi-squared arguments...).
> 
> This is an absurd requirement.  There are an infinite number of
> alternative models, including ones with 21 parameters that will fit
> the data perfectly and an elephant pretty well.

Yes, there are an infinite number of models, however some of them are
far simpler than others.  Complicated models, coupled with minimal data
sets are well known for producing "interesting" results.  Unfortunately,
these "interesting" results are generally wrong...

>  I *have* compared it
> with a "gradual decline starting before gun control" model and shown
> that it gives a better fit.

Ah, "I've compared it to *ONE* other model, and mine worked better,
so it *MUST* be _statistically_significant_".  You've yet to convince
me that there is anything even statistically interesting in the data
presented.

> > My suspicions are based on the fact that I have met very few
> > staticians whose results I would trust.  And you and I both know that
> > if someone wishes to make their data say something particular, that
> > the statistics can be "fudged", via the equation and fitting technique,
> > in such a way that it will say exactly what one wants.
> 
> Such fudging can easily be exposed by anyone who knows something about
> statistics.  If I have "fudged" the statistics I think someone would
> have pointed it out by now.

I did point it out.

> > I see in another post you give your model(s):
> 
> >>11*a = 25.6
> >>10*b = 14.9
> 
> > But you do not give the errors on your experimental data.  It's quite
> > possible that once I assign error bars to your data that I can draw a
> > straight line through all of points.  (and deviations of 2-3 times the
> > standard deviation are not unusual in experiments with low statistics).
> 
> > Here, let's pretend I'm a Nuclear Physicist... (so I can get error bars)
> 
> > 11*a = 25.6 +/- sqrt(25.6)  ;  a=2.33 +/- 0.46  ;  1.87 < a < 2.79
> > 10*b = 14.9 +/- sqrt(14.9)  ;  b=1.49 +/- 0.39  ;  1.10 < b < 1.88
> 
> > Examining these two systems, we can trivially see that "a" and "b"
> > are indistinguishable.  QED  ;-)
> 
> As a case in point, we can easily see how Charles has fudged the
> statistics here to prove that `a' and `b' are not significantly
> different.  He just made up some errors out of thin air.

Ho ho ho!!  No real fudging here.  I simply applied the uncertainty I
would use if I were studying a system which obeys Poisson statistics
(as systems which are limited by Quantum noise do: radioactive
halflives, etc.).  You can easily find that Poisson statistics show
deviations of SQRT(N). 

It is quite standard practice to use SQRT(N) for error, and
works suprisingly well for many systems (as many of them obey
Poisson statistics).

> Since he got
> to control how big his made up errors were, it is not suprising that
> he got the answer he did.

Here, let's do it a more conventional (and probably simplest) way.
Everybody get out your calculators, and run the following data through
your stats function (separately, please).  Obtain the two constants
"a" and "b", and estimated errors:

[hopefully Tim won't argue that SQRT(SUM((X-Xi)**2))/N-1) is an
invalid error estimate].

> 1910-1920 rates (before) 2.6 2.7 2.4 2.1 2.4 2.6 2.1 1.9 1.4 2.7 2.7
> 1921-1930 rates (after)  1.8 1.6 1.0 0.9 1.1 1.6 1.6 1.9 1.7 1.7

1910-1920 rates (before)  2.33 +/- 0.41  ; 1.92 < a < 2.74                                         
1921-1930 rates (after)   1.49 +/- 0.35  ; 1.14 < b < 1.84

OH NO!!  Look how far my *FUDGED* errors bar were OFF from ACTUAL!!!
You are absolutely right, Tim, my previous estimate of error was
completely bogus!...  =;->  [Could it be that the system actually
obeys Poisson statistics, Tim?]

Shucks, my error bars don't overlap any more.  Should we put any
statistical significance to the fact that the ranges fail to overlap
by 1/5 of an error bar?...  (what the old rule about 3 sigma?... ;-)

Now let's try somthing really interesting, and average them all together:

1910-1930 rates (all of them) 1.93 +/- 0.57 ; 1.36 < rate < 2.50

Not exactly a large increase in the size of the error bars.  I have a
hard time believing the claim of 2 different constants for this system.

Do *YOU* find it surprising that you can reduce the error bars
slightly by addition of an extra parameter?...  Why not fold in 21
parameters and reduce the error to zero?...  Do you *really* attribute
any statistical significance to your model?!!

-- 
Charles Scripter   *   cescript@phy.mtu.edu
Dept of Physics, Michigan Technological University, Houghton, MI 49931
-------------------------------------------------------------------
With your one remaining eye, please do NOT look back into the laser
Newsgroups: talk.politics.guns