Newsgroups: bit.listserv.stat-l
Date:         Wed, 16 Dec 1992 11:42:13 ECT
From:         Lary Jones <LJONES@BINGVMB.BITNET>
Subject: "By Inspection" Summary -- 176 lines

On December 8 I inquired:

>Yesterday, a client asked an interesting question. In his area (Biology)
>he sometimes has data which is so obviously different between groups/
>treatments that he has reported that the results are "significantly
>different by inspection." ...
>
>What is the history of this practice? Is it an intuitive sign test? How
>widely is it accepted? What references are available which accept or
>reject this approach?.

I want to thank everyone who contributed. If my editing has altered any
of the points made, please correct me.

Many responses included a reference to the IOT, Inter-Ocular Trauma,
test--which basically says that if the pattern in the data hits you
right between the eyes, then report it on that basis. Phil Gallagher's
contribution nicely covered his memory of the origin, and durability, of
this concept:

     --------------------------------------------------
Twenty-four years ago Gary Koch taught his class something that he
may now wish we had forgotten. He spoke of the Inter-Ocular Trauma
(IOT) test; "it hits you right between the eyes!"

If I remember correctly he has attributed that to Dana Quade, but it
may be that Dr. Quade disavows it, too. I use the concept when I
encounter data so grossly different that there is no analysis that I
would believe if it reported that the difference were not
statistically significant. Of course, the truth of the situation is
that I have done in my head what we all do by instinct: I have a quick
look at the mean difference and the variability, and I do a (totally
by seat-of-the-pants) calculation by inspection.

... I must admit that I usually restrict this kind of inference to
situations where the substantive info from clinic and lab is also
well-known and well-nigh incontrovertible. Bradford Hill had an
example in his book on Statistics in Clinical Trials and Preventive
Medicine (I'm sure that title is a little off) in which he said that
although he had never seen any studies that presented evidence to
support his thesis that dropping babies onto their heads from a
height of six feet was injurious to their health, he was unwilling
to undertake a study to obtain such data. I think that kind of
reasoning can be closely akin to my refusing to do formal tests of
obvious results.

                                                     Phil Gallagher
                                                     uphilg@unc
       -------------------------------------------------------

"UNHCES::URBAN"@UNHH.UNH.EDU (Will Urban--603/862-1582) and
John R. Vokey <vokey@hg.uleth.ca> also mentioned the IOT test .

       -------------------------------------------------------

"Dennis Roberts" <DMR@PSUVM> referred back to the discussion of "rules
of thumb":

While I don't have any specific reference, this sounds like some of
the discussion we were having about testing assumptions ... using
graphical methods ... or (pardon the expression) thumb rules ...
that I objected to since every person could have their own "rules".
       -------------------------------------------------------

Russ Lenth <rlenth@STAT.UIOWA.EDU> noted:

I have no historical reference, but I do know that if two sets of data
of sizes n>=4 each cover disjoint intervals (i.e., the min of one is
greater than the max of the other), then they test statistically
different at the .05 level based on a two-tailed Mann-Whitney test.  If
doing a one-tailed M-W test, you only need n>=3.
       -------------------------------------------------------

To which, Andy Taylor, ataylor@lala.zoo.Hawaii.Edu replied:

If people really are using this justification, then why not say so?
More importantly, the rank-sum test is pretty broadly applicable,
but it _does_ have its assumptions/sensitivity, primarily to unequal
variances, so shouldn't be used absolutely everywhere.

As I wrote to Lary Jones privately, my own guess (as a biologist) is
that there are areas in biology who trust their subjective judgement
(i.e. use the "InterOcular Trauma Test") without any accurate feel
for how poor subjective judgement can be. Just my somewhat
frustrated 2-cents worth.
      ---------------------------------------------------------

To which, Bill Halteman, "HALTEMAN@MAINE.MAINE.EDU" said:

Isn't it the case that the equal variances assumption is only necessary
if one is interested in only a shift in location?  I believe the MW tests
equality of distribution functions with possible alternatives stated
in terms of stochastically larger ( or smaller), (i.e. P(X<Y) < 1/2 or
P(X<Y) > 1/2).  If that is the nature of the alternative then Russ's
Inter-occular test should be fine.

This is what I conclude after reading Conover's _Practical Nonparametric
Statsitics_ and E.L. Lehman's book on tests based on ranks (the title of
which I cannot, at this moment, remember.)
     -------------------------------------------------------

Gary Carson <Eco_Gxc@SHSU> added:

There is a difference between statistical significance and operational
significance. Sometimes statistical significance is not operational and
sometimes otherwise. ...

2. I recently submitted a paper to an application journal (J. of
Security Management) which asserted a significant diff sithout doing
a hypothesis test. I didn't think I needed to. We were comparing 911
calls for service to a particular location before and after
implementation of a security program. For all catogories of call,
for every time period, the frequncies were less than 1/2 what they
had been before implemtation. I claim this is operationally
significant. I don't care about statistical significance. I haven't
heard from the editor yet. If he required it, then I'll put in a
hyupothesis test and still claim operational significance no matter
what the outcome of the statistical test.
       -------------------------------------------------------

"R.A.Reese" <R.A.Reese@seq.hull.ac.uk> contributed:

I have given similar advice in situations where the experimental
results are so clear cut that a formal statistical exposition would
seem to be fatuous. For example, a colleague here used to work on
the percentage germination of fungal spores in the context of
disease control. If he got results such as

Group A: 100 100 95 100 98
Group B:  20  15  0  10 20

would anyone insist on an anova of transformed data values?
     -------------------------------------------------------

From: "Jim Mitchell" <jim_mitchell@msmail.kgs.ukans.edu> said:

... Statistics can be a useful *guide* in making a decision on
whether the difference between two sets of results is *biologically*
significant.

    I believe a statement that something is "significant by inspection"
falls into the sibjective realm of opinion, without further
qualification. The qualification can be statistical or substantive.
Because you have a type I and type II error associated with every
statistical result, can you ever really be sure?
     -------------------------------------------------------

Jim Mitchell also forwared the following:

  From: Bob_Buddemeier@msmail.kgs.ukans.edu

In radiochemistry (which has the advantage of haviong reasonably
well-defined statistics attached to the data by definition) it is
customary to treat as different (without detailed statistical
analysis) activity measurements that differ by more than 2 standard
deviations of the counting statistics, which implicitly assumes that
variations in effeciency, background, etc., are small by comparison.
This works pretty well for the physicists and some chemists, but it
breaks down in applications where people don't have a good
understnaidng of their system or in cases of low-level counting. You
will also notice that this is what happens in hydrology -- we treat
a 10-foot multi-year well decline as significant (and few
hydrologists would challenge that in a review setting), and the
irrigators are looking at year-to-year fluctuations and casting
doubts.

__________________________________________________________________
Lary Jones...Computing Services...Binghamton Univ...(607) 777-2879
School of Management Research Support ...... LJones@BingVMB.Bitnet