In article <lsu7q7INNia5@saltillo.cs.utexas.edu> turpin@cs.utexas.edu (Russell Turpin) writes: >-*---- >I agree with everything that Lee Lady wrote in her previous post in >this thread.    Gee!  Maybe I've misjudged you, Russell.  Anyone who agrees with something  I say can't be all bad.  ;-)  Seriously, I'm not sure whether I misjudged you or not, in one respect.   I still have a major problem, though, with your insistence that science  is mainly about avoiding mistakes.  And I still disagree with your  contention that nobody who doesn't use methods deemed "scientific"  can possibly know what's true and what's not.    >  [Deleted material which I agree with.]   > >Back to Lee Lady: > >> These are not the rules according to many who post to sci.med and >> sci.psychology.  According to these posters  "If it's not supported by >> carefully designed controlled studies then it's not science." > >These posters are making the mistake that I have previously >criticized of adhering to a methodological recipe.  A "carefully ... >     ....   >Rules such as "support the hypothesis by a carefully designed and >controlled study" are too narrow to apply to *all* investigation. >I think that the requirements for particular reasoning to be >convincing depends greatly on the kinds of mistakes that have >occurred in past reasoning about the same kinds of things.  (To >reuse the previous example, we know that conclusions from >uncontrolled observations of the treatment of chronic medical >problems are notoriously problematic.)    Okay, so let's see if we agree on this: FIRST of all, there are degrees  of certainty.  It might be appropriate, for instance, to demand carefully  controlled trials before we accept as absolute scientific truth (to the  extent that there is any such thing) the effectiveness of a certain  treatment. On the other hand, highly favorable clinical experience, even  if uncontrolled, can be adequate to justify a *preliminary* judgement that a treatment is useful.  This is often the best evidence we can hope for from investigators who do not have institutional or corporate support. In this case, it makes sense to tentatively treat claims as credible but to reserve final judgement until establishment scientists who are qualified and have the necessary resources can do more careful testing.  SECONDLY, it makes sense to be more tolerant in our standards of  evidence for a pronounced effect than for one that is marginal.     I come to this dispute about what science is  not only as a mathematician but as a veteran of many arguments in sci.psychology (and occasionally in sci.med) about NLP (Neurolinguistic Programming).  Much of the work done to date by NLPers can be better categorized as informal exploration than as careful scientific research.  For years now I have been trying to get scientific and clinical psychologists to just take a look at it, to read a few of the books and watch some of the videotapes (courtesy of your local university library).  Not for the purpose of making a definitive judgement, but simply to look at the NLP methodology (especially the approach to eliciting information from subjects) and look for ideas and hypotheses which might be of scientific interest.  And most especially to be aware of the *questions* NLP suggests which might be worthy of scientific investigation.  Over and over again the response I get in sci.pychology is  "If this hasn't been thoroughly validated by the accepted form of empirical research then it can't be of any interest to us."    To me, the ultimate reducio ad absurdum of the extreme "There've got to be controlled studies" position is an NLP technique called the Fast Phobia/Trauma Cure.  Simple phobias (as opposed to agoraphobia) may not be the world's most  important psychological disorder, but the nice thing about them is that  it doesn't take a sophisticated instrument to diagnose them or tell  when someone is cured of one.  The NLP phobia cure is a simple  visualization which requires less than 15 minutes.  (NLPers claim that it can also be used to neutralize a traumatic memory, and hence is useful in treating Post-traumatic Stress Syndrome.)  It is essentially a variation on the classic desensitization process used by behavioral therapists.  A subject only needs to be taken through the technique once (or, in the case of PTSD, once for each traumatic incident).  The process doesn't need to be repeated and the subject doesn't need to practice it over again at home.  Now to me, it seems pretty easy to test the effectiveness of this cure.  (Especially if, as NLPers claim, the success rate is extremely high.)   Take someone with a fear of heights (as I used to have).  Take them up  to a balcony on the 20th floor and observe their response.  Spend 15  minutes to have them do the simple visualization.  Send them back up to  the balcony and see if things have changed.  Check back with them in a  few weeks to see if the cure seems to be lasting.  (More long term  follow-up is certainly desirable, but from a scientific point of view  even a cure that lasts several weeks has significance.  In any case,  there are many known cases where the cure has lasted years.  To the best  of my knowledge, there is no known case where the cure has been reversed  after holding for a few weeks.)  (My own cure, incidentally, was done with a slightly different NLP technique, before I learned of the Fast  Phobia/Trauma Cure.  Ten years later now, I enjoy living on the 17th floor of my building and having a large balcony.)    The folks over in sci.psychology have a hundred and one excuses not to make this simple test.  They claim that only an elaborate outcome study will be satisfactory --- a study of the sort that NLP practitioners,  many of whom make a barely marginal living from their practice, can ill  afford to do.  (Most of them are also just plain not interested, because  the whole idea seems frivolous.  And since they're not part of the scientific establishment, they have no tangible rewards to gain  from scientific acceptance.)   The Fast Phobia/Trauma Cure is over ten years old now and the clinical  psychology establishment is still saying "We don't have any way of  knowing that it's effective."    These academics themselves have the resources to do a study as elaborate  as anyone could want, of course, but they say  "Why should I prove your  theory?"  and  "The burden of proof is on the one making the claim."   One academic in sci.psychology said that it would be completely  unscientific for him to test the phobia cure since it hasn't  been described in a scientific journal.  (It's described in a number of  books and I've posted articles in sci.psychology describing it in as much  detail as I'm capable of.)    Actually, at least one fairly careful academic study has been done (with  favorable results), but it's apparently not acceptable because it's a doctoral dissertation and not published in a refereed journal.  To me, this sort of attitude does not advance science but hinders it.   This is the kind of thing I have in mind when I talk about "doctrinnaire"  attitudes about science.    Now maybe I have been unfair in imputing such attitudes to you, Russell.   If so, I apologize.    -- In the arguments between behaviorists and cognitivists, psychology seems  less like a science than a collection of competing religious sects.