From: hrubin@pop.stat.purdue.edu (Herman Rubin)
Subject: Re: Bashing, truth, etc.
Organization: Purdue University Statistics Department
Date: Thu, 31 Dec 1992 19:55:18 GMT

In article <17942@pitt.UUCP> geb@cs.pitt.edu (Gordon Banks) writes:
>In article <Bzo4A6.uE@mentor.cc.purdue.edu> hrubin@pop.stat.purdue.edu (Herman Rubin) writes:
>>>>I do not need any data whatever to know that saccharin has an effect on
>>>>cancer rates, or that vitamin C has an effect on longevity, or even that
>>>>the most disreputable quack treatments have an effect.

>>>is not exactly what the rest of us would consider consistent with
>>>scientific medicine.  If these statements are true, it should be
>>>possible to design studies to prove them true.  If the beneficial effect
>>>of some treatment is unmeasurable, then its utility is negligible.

>>Read the statements more carefully.  I said that I do not need any data
>>to show that it has an effect.  This means that the effect is not
>>EXACTLY zero.  This statement says nothing about the magnitude of
>>the effect.  

>No, you need data to "show that it has an effect".  It could very well
>be zero, and although you may suppose it isn't, without data you can't
>show that.  As long as it is close enough to zero that you can't
>discover which direction it lies, we are justified in the position
>that the effect is zero.  What's more, it could well be exactly
>zero.  Do you really believe that whether or not you get up from
>your desk and go get a drink of water in the next five minutes
>has a systematic effect on your life-span?  Sure you could encounter
>a murderer at the drinking fountain, but if you stayed at your
>terminal a meteor could crash through the roof killing you.

I cannot imagine a model of the universe so that the probability of
the effect being exactly zero is not zero.  There is a major difference
between saying that an effect should be ignored and saying that it is
not there.  You are thinking in terms of classical statistics rather
than looking at the problem from clear epistemological considerations.

If the effect is close enough to zero that we cannot discover in which
direction it lies, it may either be that we do not have enough data, in
which case we should very definitely NOT assume that it is zero, or that
we have enough data to recognize that it does not pay to take it into 
account.  Now if we have still more data, and can decide in which direction
it lies, we may still decide that it does not pay to take it into account,
or we may not, or some people should do one and some the other.

But whatever the model, it is not classical statistical procedures which
provide a sound basis for action.  Also, one does not get correct information
about an individual by looking at population data.  If we have two treatments,
and one works considerably better in 30% of the people, while the other works
better in 29%, and there is some distribution of comparable effectiveness in
the ramaining 41%, population studies by having some people take one treatment
and some the other will take a sample of around 10000 to have a good chance of
finding that one treatment is 1% better than the other.  But the important 
part of the information has been missed, if both treatments can be tried.
If only one, we need further research to find out for the 59% of the people
which treatment should be used.
-- 
Herman Rubin, Dept. of Statistics, Purdue Univ., West Lafayette IN47907-1399
Phone: (317)494-6054
hrubin@snap.stat.purdue.edu (Internet, bitnet)  
{purdue,pur-ee}!snap.stat!hrubin(UUCP)