
Gee!  Maybe I've misjudged you, Russell.  Anyone who agrees with something 
I say can't be all bad.  ;-)

Seriously, I'm not sure whether I misjudged you or not, in one respect.  
I still have a major problem, though, with your insistence that science 
is mainly about avoiding mistakes.  And I still disagree with your 
contention that nobody who doesn't use methods deemed "scientific" 
can possibly know what's true and what's not.  


Okay, so let's see if we agree on this: FIRST of all, there are degrees 
of certainty.  It might be appropriate, for instance, to demand carefully 
controlled trials before we accept as absolute scientific truth (to the 
extent that there is any such thing) the effectiveness of a certain 
treatment. On the other hand, highly favorable clinical experience, even 
if uncontrolled, can be adequate to justify a *preliminary* judgement that
a treatment is useful.  This is often the best evidence we can hope for
from investigators who do not have institutional or corporate support.
In this case, it makes sense to tentatively treat claims as credible
but to reserve final judgement until establishment scientists who are
qualified and have the necessary resources can do more careful testing.

SECONDLY, it makes sense to be more tolerant in our standards of 
evidence for a pronounced effect than for one that is marginal.  


I come to this dispute about what science is  not only as a
mathematician but as a veteran of many arguments in sci.psychology (and
occasionally in sci.med) about NLP (Neurolinguistic Programming).  Much
of the work done to date by NLPers can be better categorized as
informal exploration than as careful scientific research.  For years
now I have been trying to get scientific and clinical psychologists to
just take a look at it, to read a few of the books and watch some of
the videotapes (courtesy of your local university library).  Not for
the purpose of making a definitive judgement, but simply to look at the
NLP methodology (especially the approach to eliciting information from
subjects) and look for ideas and hypotheses which might be of
scientific interest.  And most especially to be aware of the
*questions* NLP suggests which might be worthy of scientific
investigation.

Over and over again the response I get in sci.pychology is  "If this
hasn't been thoroughly validated by the accepted form of empirical
research then it can't be of any interest to us."  

To me, the ultimate reducio ad absurdum of the extreme "There've got to
be controlled studies" position is an NLP technique called the Fast
Phobia/Trauma Cure.

Simple phobias (as opposed to agoraphobia) may not be the world's most 
important psychological disorder, but the nice thing about them is that 
it doesn't take a sophisticated instrument to diagnose them or tell 
when someone is cured of one.  The NLP phobia cure is a simple 
visualization which requires less than 15 minutes.  (NLPers claim that
it can also be used to neutralize a traumatic memory, and hence is
useful in treating Post-traumatic Stress Syndrome.)  It is essentially
a variation on the classic desensitization process used by behavioral
therapists.  A subject only needs to be taken through the technique once
(or, in the case of PTSD, once for each traumatic incident).  The
process doesn't need to be repeated and the subject doesn't need to
practice it over again at home.

Now to me, it seems pretty easy to test the effectiveness of this cure. 
(Especially if, as NLPers claim, the success rate is extremely high.)  
Take someone with a fear of heights (as I used to have).  Take them up 
to a balcony on the 20th floor and observe their response.  Spend 15 
minutes to have them do the simple visualization.  Send them back up to 
the balcony and see if things have changed.  Check back with them in a 
few weeks to see if the cure seems to be lasting.  (More long term 
follow-up is certainly desirable, but from a scientific point of view 
even a cure that lasts several weeks has significance.  In any case, 
there are many known cases where the cure has lasted years.  To the best 
of my knowledge, there is no known case where the cure has been reversed 
after holding for a few weeks.)  (My own cure, incidentally, was done
with a slightly different NLP technique, before I learned of the Fast 
Phobia/Trauma Cure.  Ten years later now, I enjoy living on the 17th
floor of my building and having a large balcony.)  

The folks over in sci.psychology have a hundred and one excuses not to
make this simple test.  They claim that only an elaborate outcome study
will be satisfactory --- a study of the sort that NLP practitioners, 
many of whom make a barely marginal living from their practice, can ill 
afford to do.  (Most of them are also just plain not interested, because 
the whole idea seems frivolous.  And since they're not part of the
scientific establishment, they have no tangible rewards to gain 
from scientific acceptance.) 

The Fast Phobia/Trauma Cure is over ten years old now and the clinical 
psychology establishment is still saying "We don't have any way of 
knowing that it's effective."  

These academics themselves have the resources to do a study as elaborate 
as anyone could want, of course, but they say  "Why should I prove your 
theory?"  and  "The burden of proof is on the one making the claim."  
One academic in sci.psychology said that it would be completely 
unscientific for him to test the phobia cure since it hasn't 
been described in a scientific journal.  (It's described in a number of 
books and I've posted articles in sci.psychology describing it in as much 
detail as I'm capable of.)  

Actually, at least one fairly careful academic study has been done (with 
favorable results), but it's apparently not acceptable because it's a
doctoral dissertation and not published in a refereed journal.

To me, this sort of attitude does not advance science but hinders it.  
This is the kind of thing I have in mind when I talk about "doctrinnaire" 
attitudes about science.  

Now maybe I have been unfair in imputing such attitudes to you, Russell.  
If so, I apologize. 
 
--
In the arguments between behaviorists and cognitivists, psychology seems 
less like a science than a collection of competing religious sects.   